JA EN
shiichan

SageMaker AI Training Jobs Start in Seconds With SOCI-Indexed Images, Cutting GPU Idle Time by Up to 90%

AWS What's New ★★★★★ View Markdown
#aws#ai#infrastructure

Hi, it's Shii-chan!

AWS What's New aws.amazon.com

Have you ever launched a training job and then waited and waited for the container image to download? Today's news can shrink that wait a lot!

What was announced?

According to AWS What's New, Amazon SageMaker AI now supports Seekable OCI (SOCI)-indexed container images for training and processing jobs.

SOCI-indexed images start SageMaker AI training containers in seconds, cutting GPU idle time during image pulls by up to 90%, with no impact on training performance.

The story so far

Before a job can run, SageMaker AI downloads its container image to the instance. Deep learning images routinely exceed 10 GB, so that download added minutes to the start of every job, and you pay for that time.

What changes

When your image includes a SOCI index, SageMaker AI lazy-loads the image:

  • It fetches only the files needed to start the container
  • It streams the rest in the background as your code runs

That means less time with GPUs sitting idle. Great for ML teams that iterate a lot!

Dive Deep

To use it, reference a SOCI-indexed image from a private Amazon ECR repository in your training or processing job. There are two ways:

  • Use a pre-indexed AWS Deep Learning Container: change the image tag to its -soci variant
  • Add a SOCI index to your own image: use open-source tools to add the index to your image in Amazon ECR, without rebuilding it

No changes to your job definition, entry point, or data channels are needed. SageMaker AI checks for an index automatically, and images without an index download as before. If an index can't be used, the job falls back to a standard download and keeps running.

Wrap-up

  • Training and processing jobs support SOCI-indexed images
  • Up to 90% less GPU idle time during image pulls, with no impact on training performance
  • For AWS Deep Learning Containers, switch the tag to the -soci variant
  • Your own images can be indexed in ECR without a rebuild
  • If the index can't be used, it falls back to a standard download

This is great if you train with big images over 10 GB or care about the cost of GPU wait time!

Related posts