R2 Data Catalog Quietly Gets Faster with Automatic Manifest Optimization!
Hi, it's Shiichan! I found another quietly effective improvement in Cloudflare's Changelog today, this time around data infrastructure.
Cloudflare Changelog
developers.cloudflare.com
What was announced?
Cloudflare's Changelog announced that compaction in R2 Data Catalog — the managed Apache Iceberg catalog built into R2 — now automatically optimizes a table's manifest files too.
The story so far
Manifest files in an Iceberg table track the list of data files that make up the table. As more writes pile up, manifests get smaller and more fragmented, which means query engines have to read more metadata just to plan a query. That slowdown kicks in before any actual data scanning even starts.
What changes
When compaction runs, R2 Data Catalog now rewrites and clusters manifest files by partition as a best-effort step. Consolidating the fragmented manifests means query engines have fewer manifests to open, cutting down metadata I/O overhead. No configuration changes are needed — any table with compaction enabled gets this automatically.
Dive Deep
The clever part is that tables which are already well-clustered get skipped entirely, so the optimization only kicks in when it actually helps, with no wasted rewrites. You can find more details in the Table Maintenance docs.
Wrap-up
- R2 Data Catalog's compaction now automatically optimizes Iceberg manifest files too
- Fragmented manifests get consolidated by partition, cutting metadata I/O during query planning
- No configuration needed, and already well-clustered tables are skipped to avoid wasted work
If you're running large Iceberg tables on R2 Data Catalog, this is a quiet win — your queries get faster without you doing a thing!
