JA EN
shiichan

VARIANT Types, Geospatial Data, Nanosecond Timestamps — AWS Glue 6.0 Is Packed with Features!

AWS Blog ★★★★ View Markdown
#aws#infrastructure#devops

Hey there, it's me, Shiichan — thanks for hanging in! Today I'm digging into an AWS Blog post about AWS Glue 6.0!

AWS Blog aws.amazon.com

What was announced?

AWS's Blog published a detailed write-up on the general availability of AWS Glue 6.0. It's the same update AWS What's New already announced, but the Blog version goes much deeper into the architecture and how to actually use it. The headline is still a 30% price reduction plus full support for Apache Iceberg v3.

The story so far

Earlier Glue versions didn't support Iceberg v3-generation features, and ran on an older generation of Spark, Python, and Scala. They simply couldn't express things like fast semi-structured reads, row-level updates, or spatial data handling the way Glue 6.0 now can.

What changes

Glue 6.0 runs on a fully modernized runtime — Apache Spark 4.1, Python 3.12, Scala 2.13 — and puts the full Iceberg v3 feature set to work:

  • VARIANT data type: with automatic shredding, so you can store and query JSON, log, and event data without flattening the schema — reads are faster than cramming everything into string columns
  • Geometry / geography data types: run spatial data pipelines directly on managed Spark for GIS analysis and location intelligence
  • Nanosecond-precision timestamps: built for IoT sensor data, scientific computing, and high-frequency financial workloads where precision matters
  • The UNKNOWN type: handles unexpected or evolving schemas, so pipelines hold up better against upstream schema changes

Dive Deep

The Spark 4.1 features are worth a close look too. Spark Declarative Pipelines let the engine decide execution order and optimization on its own, cutting out boilerplate orchestration code. Arrow-native Python UDFs/UDTFs remove serialization overhead to speed up complex transformations, and Real-Time Mode runs stateless streaming with single-digit-millisecond latency.

Nothing changes about how you invoke it — same APIs, CLI, SDK, and IDEs. In Glue Studio, just pick "Glue 6.0 – Supports Spark 4.1, Scala 2, Python 3" from the Job Details tab. From a notebook, use the %glue_version magic command, and existing jobs can auto-upgrade via the Spark Upgrade Agent.

On pricing, crawlers and ETL jobs stay on hourly billing (just 30% cheaper), and the Data Catalog remains free for the first million stored objects and the first million requests. It's available in every region where AWS Glue runs.

Wrap-up

  • AWS's Blog gives a deep technical walkthrough of the Glue 6.0 GA release
  • Iceberg v3 features — VARIANT, geometry/geography types, nanosecond timestamps, and the UNKNOWN type — are all available now
  • Spark Declarative Pipelines, Arrow-native Python UDFs, and Real-Time Mode all improve the developer experience
  • Existing jobs can move to 6.0 smoothly via the Spark Upgrade Agent
  • Worth a proper read if you run data infrastructure at scale!

Related posts