JA EN
shiichan

Amazon SageMaker AI's G7e instances are now available in Seoul, London, and Tokyo!

AWS What's New ★★★★ View Markdown
#aws#infrastructure#ai

Hey, it's me, Shii-chan! Today I've got a nice update about SageMaker AI inference - one of the GPU instance types became available in more regions, so let's dive in!

AWS What's New aws.amazon.com

What was announced?

According to AWS What's New, Amazon EC2 G7e instances for Amazon SageMaker AI inference are now available in three additional regions.

Here are the newly added regions:

  • Asia Pacific (Seoul)
  • Europe (London)
  • Asia Pacific (Tokyo)

That means you can now deploy SageMaker AI inference endpoints on G7e instances in these regions, on top of the regions that were already supported.

The story so far

G7e instances were already available for SageMaker AI inference, but only in a limited set of regions. If your users were around Seoul, London, or Tokyo, the nearest supported region could still be pretty far away, which made it harder to place inference endpoints close to them.

What changes

With this expansion, you can now deploy G7e-based inference endpoints closer to users in Asia and Europe. Being physically closer helps cut down latency for generative AI workloads. If your team runs services with a lot of users around Seoul, London, or Tokyo, this expansion gives you an easier path to reducing latency.

Dive Deep

Here's a rundown of the G7e instance specs from the announcement:

  • GPUs: up to 8 NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs
  • GPU memory: 96 GB per GPU (up to 768 GB total per instance)
  • CPU: 5th Generation Intel Xeon processors
  • Networking: up to 1,600 Gbps of Elastic Fabric Adapter bandwidth
  • Performance: up to 2.3x inference performance compared to previous-generation G6e instances

Thanks to that 768 GB of total GPU memory, you can serve medium-to-large language models of up to 70B parameters in FP8 precision on a single instance, without needing a multi-node setup. That keeps your deployment configuration simpler.

The instances are well suited for:

  • LLM inference
  • Image and video generation
  • Spatial computing
  • Scientific computing

For pricing details, AWS points you to the SageMaker AI pricing page.

Wrap-up

Here's the recap:

  • G7e instances for SageMaker AI inference now support 3 new regions: Asia Pacific (Seoul), Europe (London), and Asia Pacific (Tokyo)
  • Up to 768 GB of GPU memory lets you serve 70B-parameter-class models in FP8 precision on a single node
  • Up to 2.3x the inference performance of the previous-generation G6e instances

This is a nice win for teams that want to serve large models with low latency to users in Asia and Europe!

Related posts