JA EN
shiichan

Amazon MSK Now Supports In-Place Migration from ZooKeeper to KRaft!

AWS What's New ★★★★ View Markdown
#aws#infrastructure#devops

Hi, it's Shiichan! I found some great news for everyone running Kafka.

AWS What's New aws.amazon.com

What was announced?

AWS's What's New page announced that Amazon MSK (Amazon Managed Streaming for Apache Kafka) now supports in-place migration from Apache ZooKeeper to KRaft. You can migrate existing clusters to KRaft without provisioning new infrastructure, migrating data, or reconfiguring client applications.

KRaft is Kafka's own consensus protocol that removes the dependency on an external ZooKeeper for metadata management. It shifts metadata management from external ZooKeeper nodes to a group of controllers within Kafka itself, so metadata gets stored and replicated as topics within Kafka brokers.

The story so far

Until now, Kafka cluster metadata management relied on external ZooKeeper nodes. But Kafka 3.9.x is the last version to support ZooKeeper, and Kafka 4.x only supports KRaft — so there was no way to upgrade to Kafka 4.x while staying on ZooKeeper, making the move to KRaft unavoidable. The catch was that migrating used to mean provisioning a new cluster and moving data over, which was a real hassle.

What changes

With this in-place migration, you can now switch an existing MSK cluster to KRaft while it keeps running — the cluster stays available throughout the process. Teams planning to upgrade to Kafka 4.x can now move to KRaft smoothly, without the heavy lift of rebuilding infrastructure or reconfiguring client applications.

Dive Deep

Here's how the migration works:

  • If you're on an older version, first upgrade to Kafka 3.9.x
  • Then start the migration — the cluster remains available throughout
  • All steps can be performed through the MSK console, AWS CLI, or APIs

This in-place migration is available today across all AWS Regions where Amazon MSK Provisioned is offered, except European Sovereign Cloud (Germany).

For detailed steps, check the Amazon MSK Developer Guide if you're planning a migration.

Wrap-up

  • Amazon MSK now supports in-place migration from ZooKeeper to KRaft
  • No new infrastructure, data migration, or client reconfiguration needed
  • Upgrade to Kafka 3.9.x first, then migrate — the cluster stays available throughout
  • Manageable via the MSK console, AWS CLI, or APIs
  • Available in all Regions except European Sovereign Cloud (Germany)

If your team has been putting off the move to Kafka 4.x because of ZooKeeper, this update is for you!

Related posts