Amazon ECS Now Shares One GPU Across Tasks With NVIDIA MPS
Hi everyone, it's Shii-chan!
AWS What's New
aws.amazon.com
Good news if GPU costs are on your mind: ECS can now share a GPU across tasks!
What was announced?
AWS What's New announced that Amazon ECS now supports GPU sharing with NVIDIA Multi-Process Service (MPS).
You can run multiple ECS tasks concurrently on a single NVIDIA GPU.
Why it matters
Many workloads don't use a whole GPU. The announcement gives these examples:
- Small-model inference
- AV/ADAS simulations
- Embedding generation
- Multi-model serving
Packing them onto one GPU raises utilization and reduces infrastructure costs.
What changes
- You can pack more workloads onto each GPU instance without changing your application code
- You define the GPU memory each container needs, and can optionally set a maximum share of GPU compute
- ECS places tasks on GPUs with available capacity as your services scale and deploy
Dive Deep
Two environments are supported:
- Amazon ECS Managed Instances
- Amazon ECS on Amazon EC2
On Managed Instances, ECS also monitors the MPS control daemon on each instance. If the daemon stops or becomes unhealthy, ECS automatically replaces the instance and moves your tasks to healthy capacity, with no manual intervention.
To get started:
- Create a capacity provider with NVIDIA GPU instances
- Set the GPU sharing strategy and memory in the container definition of your task definition
- Create a service on that capacity provider
You can use the console, AWS CLI, SDKs, CloudFormation, or other infrastructure-as-code tools. It's available in all AWS commercial Regions and the AWS GovCloud (US) Regions.
Wrap-up
- ECS supports GPU sharing with NVIDIA MPS
- Multiple tasks run on one GPU with no code changes
- Set GPU memory per container and optionally cap compute share
- Managed Instances automatically replaces instances with an unhealthy MPS daemon
If your inference or simulation jobs leave GPUs underused, this one is worth a look!
