Bulk Writes from S3 to DynamoDB: Bulk Put Items Is Now Generally Available!
Hi, it's Shii-chan!
AWS What's New
aws.amazon.com
If you load S3 data into DynamoDB, this one is worth a look!
What was announced?
In AWS's What's New feed, bulk put items for Amazon DynamoDB is now generally available. It is a fully managed feature that writes data from Amazon S3 into your existing DynamoDB tables.
The story so far
Writing data that other systems produce and store in S3 into existing DynamoDB tables has typically meant building custom pipelines and managing table capacity.
What changes
You point bulk put items at the S3 location of your input files, and DynamoDB handles the rest.
- Writes don't impact existing workloads
- No custom ingestion pipeline to build
- Cost-efficient writes
Dive Deep
Condition expressions
Each item supports its own condition expression, so you can enforce insert-only or version-guarded writes.
Two tiers
- Standard: fits most workloads
- Accelerated: for more time-sensitive workloads
Availability
- Standard tier: all commercial AWS Regions and the AWS GovCloud (US) Regions
- Accelerated tier: limited Regions, so check the regional availability page
Common use cases
- Daily refreshes
- Large catalog enrichments
- Backfills
- Migrations
For more, see the DynamoDB blog post and the Amazon DynamoDB Developer Guide.
Wrap-up
- Fully managed bulk writes from S3 into existing DynamoDB tables (GA)
- No custom ingestion pipeline or capacity management
- Per-item condition expressions enable insert-only or version-guarded writes
- Standard and Accelerated tiers
This is for anyone doing regular data refreshes, backfills, or migrations on DynamoDB!
