JA EN
shiichan

Amazon Redshift system tables can now outlive the 7-day limit!

AWS What's New ★★★★ View Markdown
#aws#infrastructure#devops

Hello, it's me! Today I've got a slightly nerdy but genuinely useful Amazon Redshift update for you!

AWS What's New aws.amazon.com

What was announced?

AWS What's New announced that Amazon Redshift now supports long-term retention for system table data through native integration with Amazon S3 Tables. Once you enable it, AWS automatically writes your system table data to S3 Tables in Apache Iceberg format and manages partitioning, compaction, and retention for you.

The story so far

Redshift's system tables have long been used to monitor query performance, audit data warehouse activity, and meet compliance requirements — but retention was capped at 7 days. Extending it meant building and maintaining your own ETL pipelines to copy and store the data, which added development effort and ongoing operational overhead. If you ran multiple data warehouses, you also needed Redshift data sharing to consolidate each warehouse's system table data into one place for cross-warehouse analysis.

What changes

With this feature, system table data is replicated automatically to S3 Tables, so you no longer need to build custom ETL pipelines, and there's no resource contention with your production workloads. If you operate multiple data warehouses, you can consolidate their system table data into a single location for cross-warehouse observability and analysis.

Dive Deep

Because the data lands in open Apache Iceberg format, you can query it not just from Redshift but also from Amazon Athena or any other Iceberg-compatible engine. It's supported on Redshift Provisioned (RA3) and Redshift Serverless, across 30+ regions spanning the US, Canada, Europe, Asia Pacific, Israel, and Brazil.

Wrap-up

  • Redshift system table data can now be retained long-term via S3 Tables integration
  • Previously capped at 7 days, and extending it required a custom ETL pipeline
  • AWS automatically handles partitioning, compaction, and retention once enabled
  • Data lands in Apache Iceberg format, so it's queryable from Athena and other engines too
  • Great news for teams juggling compliance audits or cross-warehouse observability

If you've been wrestling with Redshift compliance retention or multi-cluster observability, this one's worth a look!

Related posts