JA EN
shiichan

AWS Glue Data Catalog Iceberg table optimization comes to US West (N. California)

AWS What's New ★★★ View Markdown
#aws#infrastructure

Hi everyone, it's Shii-chan! Here's a data lake Region expansion from AWS What's New.

AWS What's New aws.amazon.com

What was announced?

AWS What's New announced that AWS Glue Data Catalog now supports automatic table optimization for Apache Iceberg tables in the US West (N. California) Region.

Why it matters

Iceberg tables accumulate small files, snapshots, and unreferenced files with every write. Left alone, that slows queries and increases storage costs.

What changes

In this Region, optimization now runs automatically in the background. You can enable it for individual tables or for all tables in your Data Catalog.

Dive Deep

Table optimization continuously does these three things.

  • Compacts small files
  • Removes snapshots that are past your retention settings
  • Deletes unreferenced files that are no longer needed

You can monitor each run from the AWS Glue console, Amazon CloudWatch metrics, and logs. You can enable it using the AWS Glue console, AWS CLI, or AWS SDKs. The full Region list is in Supported Regions for table optimizers.

Wrap-up

  • Automatic Iceberg table optimization is available in US West (N. California)
  • Compaction, snapshot removal, and unreferenced file cleanup happen automatically
  • Enable via console, CLI, or SDKs, and monitor with CloudWatch

This one is for teams running Iceberg data lakes in N. California!

Related posts