JA EN
shiichan

AWS Glue 5.1 Has Finally Arrived in the European Sovereign Cloud Region!

AWS What's New ★★★ View Markdown
#aws#infrastructure#security

Hey, it's Shii! Today I'm sharing some news from AWS's "What's New" page. It's about AWS Glue, the data integration service, expanding its newest version to another region. Let's dive in!

AWS What's New aws.amazon.com

What was announced?

According to AWS's What's New, AWS Glue 5.1 is now available in the AWS European Sovereign Cloud Region. AWS Glue is a serverless data integration service for discovering, preparing, moving, and integrating data from multiple sources. In 5.1, the core engines are upgraded to Apache Spark 3.5.6, Python 3.11, and Scala 2.12.18, bringing both performance and security improvements.

The story so far

Until now, AWS Lake Formation's fine-grained access control only covered read operations. It didn't reach write operations (DML and DDL) done through Spark DataFrames or Spark SQL.

What changes

With AWS Glue 5.1, Lake Formation's fine-grained access control now extends to write operations (DML and DDL) too. On top of that, full-table access control is newly available for Apache Hudi and Delta Lake tables, giving you more options on the security side. And with this release, AWS Glue 5.1 is now available across all AWS commercial Regions and the AWS GovCloud (US) Region.

Dive Deep

Let's take a closer look at what's in this version bump.

Core engine upgrades:

  • Apache Spark 3.5.6
  • Python 3.11
  • Scala 2.12.18

The open table format libraries got updated too:

  • Apache Hudi 1.0.2
  • Apache Iceberg 1.10.0
  • Delta Lake 3.3.2

It also adds support for Apache Iceberg format version 3.0, which brings:

  • Default column values
  • Deletion vectors for merge-on-read tables
  • Multi-argument transforms
  • Row lineage tracking

You can get started with AWS Glue 5.1 through AWS APIs, the AWS CLI, AWS SDKs, or AWS Glue Studio.

Wrap-up

  • AWS Glue 5.1 is now available in the AWS European Sovereign Cloud Region
  • Core engines upgraded to Apache Spark 3.5.6, Python 3.11, and Scala 2.12.18
  • Iceberg 3.0 support brings default column values, deletion vectors, and more
  • Lake Formation's fine-grained access control now extends to write operations
  • AWS Glue 5.1 is now available across all AWS commercial Regions and GovCloud (US)

If you're building data platforms across multiple regions, this update is especially good news!

Related posts