SageMaker Notebook Instances now run on G6e — up to eight L40s GPUs!

Hey there, it's Shiichan! Today I've got good news for everyone who wants to spin up GPUs for machine learning.

AWS What's New aws.amazon.com

What was announced?

According to AWS's What's New, SageMaker Notebook Instances now support Amazon EC2 G6e instances. G6e can pack up to eight NVIDIA L40s Tensor Core GPUs, each with 48 GB of memory, alongside third-generation AMD EPYC processors — and now you can pick them straight from your notebook environment.

The story so far

Until now, if you wanted to work with larger models in a notebook, your choice of GPU instances was limited. With G6e in the mix, you get up to 2.5x the performance of the previous-generation G5, right from the notebook you already use.

What changes

Interactively testing model deployments, fine-tuning generative AI... those heavier workflows become easier to keep entirely inside your notebook. It targets large language models up to 13B parameters, plus diffusion models for generating images, video, and audio.

Dive Deep

Supported environments include JupyterLab, CodeEditor, SageMaker Studio, and SageMaker notebook instances.

Available regions are US East (N. Virginia and Ohio), US West (Oregon), Asia Pacific (Tokyo), Middle East (Dubai), and Europe (Frankfurt, Sweden, Spain). Tokyo being on the list is a nice touch for folks in Japan.

Wrap-up

• SageMaker Notebook Instances now support G6e instances
• G6e offers up to eight NVIDIA L40s (48 GB each) plus third-gen AMD EPYC, for up to 2.5x the performance of G5
• Great for fine-tuning and test-deploying LLMs up to 13B and diffusion models
• Available across several regions, including Tokyo

This is a handy update for ML engineers who want to run bigger models right in their notebooks!