JA EN
shiichan

Redshift Can Now Create Apache Iceberg Materialized Views, Readable by Athena, Spark, and Trino!

AWS What's New ★★★★ View Markdown
#aws#infrastructure

Hi, it's Shii-chan! Today Redshift and Iceberg get even friendlier!

AWS What's New aws.amazon.com

What was announced?

According to AWS What's New, Amazon Redshift now supports the creation and refresh of Apache Iceberg materialized views.

Expensive joins and aggregations are pre-computed once, and the results are stored in an Iceberg table in Amazon S3 or Amazon S3 table buckets, registered in the AWS Glue Data Catalog.

Why it matters

Data teams often build analytics in stages, stitching together different engines to clean and transform raw data before serving it.

That adds pipeline orchestration overhead and can introduce semantic differences between engines.

What changes

The original post describes two kinds of value.

  • Instead of hundreds of users and teams re-running the same expensive joins and aggregations and re-scanning source tables on every query, you compute the result once and everyone reads the precomputed table
  • You can run an end-to-end pipeline with a single engine like Apache Spark and now Amazon Redshift, and every downstream consumer shares the same open result

Dive Deep

You create them with familiar SQL.

CREATE MATERIALIZED VIEW ... USING ICEBERG
  • The results are instantly queryable by Amazon Athena, Apache Spark on Amazon EMR and AWS Glue, and third-party engines such as Trino or Snowflake
  • Redshift keeps them current with manual incremental refresh, recomputing only what has changed
  • Because they are Iceberg tables in the Glue Data Catalog, they are governed and discovered like any other catalog table
  • They are available where Redshift Serverless and provisioned Graviton instances are available

See the Amazon Redshift Database Developer Guide for details.

Wrap-up

  • Redshift supports creating and refreshing Iceberg materialized views
  • Results are Iceberg tables in S3 / S3 table buckets, registered in the Glue Data Catalog
  • Readable from Athena, Spark, Trino, Snowflake, and more
  • Manual incremental refresh recomputes only what changed

This is for anyone on a multi-engine data platform who keeps re-running the same aggregations!

Related posts