# Amazon S3 Tables adds managed migration from Apache Hive to Apache Iceberg

Hi, it's Shiichan! Here's news about moving from Hive to Iceberg that data lake folks will care about.

## What was announced?

The AWS Blog announces that Amazon S3 Tables now offers managed migration of Apache Hive tables to fully managed Apache Iceberg tables.

The source must be Parquet data in Amazon S3 that is registered in the AWS Glue Data Catalog. You don't build pipelines or provision compute, and production workloads can keep running during the migration.

## The story so far

Migrating from Hive tables to Iceberg meant building validation, type conversion, recompression and rollback yourself, which got complicated.

Managed migration has S3 Tables take on that complexity.

## What changes

Here's how the migration works:

- Validation and planning: before copying, S3 Tables identifies every source object in scope and generates a compatibility report
- Automated processing: it handles data validation, compatibility checks, type conversions, file recompression and compaction
- Repeatable syncs: you can run sync as often as needed, and each one copies only the changes since the last one. Writes to the source can continue
- Cutover: writes to the source pause only for a few minutes during cutover
- State during migration: the destination table is read-only during migration and becomes read-write after completion
- Defaults: the destination Iceberg table is created with recommended defaults for query performance and storage efficiency

## Dive Deep

There are two ways to run it: the console and the CLI/SDK.

In the console:

- Select the destination table bucket and choose Create table from migration
- Choose a namespace and table name for the destination
- Start the migration, and S3 Tables validates the source, generates the compatibility report and begins copying
- Sync as needed, then complete the migration
- Query with Amazon Athena, Amazon SageMaker Unified Studio, or any Iceberg REST API-compatible tool

With the CLI/SDK, you prepare a JSON input file that specifies the IAM role, destination table bucket, namespace, table name, encryption, storage class, Iceberg properties, schema, partition spec, source Glue table, maintenance settings and report locations. Then run:

```
aws s3tables start-table-migration
aws s3tables sync-table-migration
```

Repeat sync as many times as needed, then complete the migration. There's also a `migrating-hive-tables-to-s3-tables` skill in the Agent Toolkit for AWS that guides an agent from analyzing IAM policies through the Start, Sync and Complete calls and monitoring progress and cutover.

Requirements and limits:

- The source must be a Hive table in Parquet format
- The source table must be registered in AWS Glue
- At launch, migration works only within the same AWS account and Region
- The post doesn't list numeric quotas or limits

Pricing: you pay only for the data copied, and there is no charge for the Hive-to-Iceberg conversion. Source objects in S3 Intelligent-Tiering stay in their current access tier.

Available Regions:

- US: N. Virginia, Ohio, Oregon
- Asia Pacific: Mumbai, Singapore, Sydney, Tokyo
- Europe: Frankfurt, Ireland, Stockholm

## Wrap-up

- Parquet-based Hive tables registered in Glue can be migrated to S3 Tables Iceberg tables as a managed process
- Incremental syncs are supported, and writes pause only for a few minutes at cutover
- Available through the console, CLI/SDK and an agent skill
- Same account and Region only, and you pay for the copied data

This is for data engineers who want to move a Hive-based data lake to Iceberg!