JA EN
shiichan

Run A/B tests right inside AWS AppConfig!

AWS What's New ★★★ View Markdown
#aws#devops

Hi everyone, it's Shiichan! Today I've got a happy update that makes running experiments a whole lot easier.

AWS What's New aws.amazon.com

What was announced?

From the AWS What's New feed: AWS AppConfig now has managed experimentation tools for A/B testing and feature experiments, and they are generally available. You no longer need to stand up separate experimentation infrastructure - you can run experiments right inside AppConfig.

The story so far

Until now, running an A/B test usually meant building your own experimentation setup and taking care of traffic splitting and result aggregation yourself. Even when the value of experimenting was clear, all that groundwork tended to get in the way.

What changes

The flow becomes really simple. You define feature variations, pick your target audience with a rule builder, and set the traffic allocation. You can do this from the AWS Management Console, the CLI, the API, or the AWS CDK.

Results show up in Amazon CloudWatch or the analytics platform you already use, and the winning treatment can be promoted to production through AppConfig's standard safe rollout procedures. It runs in a lot of places too: Amazon EC2, AWS Lambda, Amazon ECS, Amazon EKS, and even on-premises servers via the AppConfig Agent.

Dive Deep

Here are the finer details you can pull from the experimentation documentation.

  • AI-assisted experiment design: it validates your setup against Amazon's experimentation best practices.
  • It supports exposure control and locked treatment allocations.
  • Beyond UI changes, you can experiment on recommendation algorithms, AI model selection, and prompts.
  • It is available in all AWS Regions, including the AWS GovCloud (US) Regions.
  • Pricing is pay-as-you-go based on experimentation hours.

Wrap-up

  • AWS AppConfig now has managed experimentation tools for A/B testing, generally available.
  • Define variations, target an audience, set traffic allocation, and review results in CloudWatch or your own analytics.
  • It works on Amazon EC2, AWS Lambda, Amazon ECS, Amazon EKS, and on-premises, across all Regions, with pay-as-you-go pricing by experimentation hours.

If you already use AppConfig for feature flags, or you're a dev team that doesn't want to build A/B testing plumbing from scratch, this one's for you.

Related posts