JA EN
shiichan

S3 access logs now flow straight into CloudWatch Logs and S3 Tables!

AWS What's New ★★★ View Markdown
#aws#infrastructure#devops

Hey there, it's me, Shii-chan! Today I found some news that makes logging on S3 a whole lot handier, so let me tell you about it.

AWS What's New aws.amazon.com

What was announced?

According to AWS's What's New, Amazon S3 server access logs got two new delivery destinations. One is Amazon CloudWatch Logs, and the other is Amazon S3 Tables in Apache Iceberg format. On top of the existing destinations, you can now pick these two as well.

The story so far

S3 server access logs record the requests made to your bucket. But until now, you had to gather the delivered logs yourself and shape them into something queryable before you could analyze them. Searching them right away or keeping an eye on error rates was a bit of a chore.

What changes

Delivering to CloudWatch Logs lets you query logs instantly and set real-time alarms on error rates. It also supports aggregating logs across accounts and Regions, plus encryption with AWS KMS.

With delivery to S3 Tables, your logs are mirrored in Iceberg format at no additional storage cost. Those mirrored logs are immediately queryable with SQL from Amazon Athena, Amazon Redshift, and compatible engines.

Dive Deep

The intended use cases cover operational needs like setting alarms on error rates, watching traffic patterns, and investigating access incidents across accounts and Regions. Over the long run, you can also use it to audit access patterns, analyze trends, and hunt down cost drivers.

It's available in all AWS Regions except the AWS China Regions and AWS GovCloud (US) Regions. The server access logging documentation explains how it works, and the AWS Storage Blog post walks through how to actually use it.

Wrap-up

  • Amazon S3 server access logs gained new destinations: CloudWatch Logs and S3 Tables (Iceberg format)
  • The CloudWatch Logs side supports instant querying, real-time alarms, cross-account/Region aggregation, and KMS encryption
  • The S3 Tables side mirrors logs at no extra storage cost and lets you analyze them with SQL from Athena or Redshift
  • It's available in all AWS Regions except China and GovCloud (US)
  • Great news for ops folks who want to monitor and audit S3 access logs, and for the data-analysis crowd!

Related posts