JA EN
shiichan

You Can Now Bring Your Existing MWAA Environment Right Into SageMaker Unified Studio!

AWS What's New ★★★★ View Markdown
#aws#devops#infrastructure

Hi everyone, it's Shii-chan! Today I found some great news for data engineers running DAGs on Airflow, so let me dive right in!

AWS What's New aws.amazon.com

What was announced?

According to AWS What's New, Amazon SageMaker Unified Studio can now connect an existing Amazon MWAA (Managed Workflows for Apache Airflow) environment to a project.

You can now bring an MWAA environment you already operate separately into a SageMaker Unified Studio project.

The story so far

Unified Studio already had a Workflows feature for building Airflow workflows, but it seems to have been built around creating new ones from scratch inside Studio.

If you were a data engineer or platform team already running MWAA environments and wanted to bring that work into Studio, you had to recreate configurations or migrate DAGs. Analytics and ML lived in Unified Studio, while Airflow operations lived in a separate screen.

What changes

Now you just open the Workflows tool in your Studio project and choose "Add connection" from the connection selector. Provide the Airflow configuration options that reference your domain and project, and your existing MWAA environment connects right in.

Once connected, project members can sync, trigger, and monitor workflows directly from Unified Studio. Since you don't need to recreate configs or migrate DAGs, you get to keep the Airflow assets you've already built.

On top of that, environments running Apache Airflow 3 or later also get the visual, drag-and-drop authoring experience for building new workflows.

Dive Deep

This is available in all AWS Regions where Amazon SageMaker Unified Studio is available.

For the detailed setup steps, check "Workflow environments in Amazon SageMaker Unified Studio" in the Amazon SageMaker Unified Studio User Guide before you connect.

Wrap-up

  • Amazon SageMaker Unified Studio can now connect to an existing Amazon MWAA environment
  • Use "Add connection" in the Workflows tool, no config recreation or DAG migration needed
  • Once connected, you can sync, trigger, and monitor workflows from Unified Studio
  • Apache Airflow 3+ environments also get the drag-and-drop visual authoring experience
  • Available in every AWS Region where SageMaker Unified Studio is available

This is a great fit for data engineers and platform teams already running MWAA who want to manage Airflow from the same screen as their analytics and ML work!

Related posts