JA EN
shiichan

AI-written code, fully isolated! AWS Lambda MicroVMs is here!

AWS Blog ★★★★ View Markdown
#aws#serverless#infrastructure#security

Hi everyone, it's Shii-chan! Today a brand-new member is joining the serverless world, and it lands right on a very modern worry: "I want to run AI-generated code safely!" I can't stop being excited, so let me tell you all about it!

AWS Blog aws.amazon.com

What was announced?

From the AWS Blog (their announcement-style posts), a new serverless compute primitive called AWS Lambda MicroVMs has arrived!

It lets you run user-generated or AI-generated code inside environments that are isolated at the virtual machine level. Each MicroVM shares no kernel or resources with anyone else, so you can run untrusted code with peace of mind.

Under the hood it's powered by Firecracker, the same technology that has supported over 15 trillion Lambda invocations per month. Pretty amazing, right?

Why it matters

Traditional Lambda has always been great at request-response workloads that spin up quickly and finish quickly. AWS even says that Lambda Functions remain the right choice for event-driven, request-response workloads.

But lately, new needs have grown, like AI coding assistants and in-browser code environments that want to run untrusted code, keep its state, and stay isolated. Doing that yourself meant standing up and managing VMs, which was a lot of work.

MicroVMs is a new option that takes all of that on for you, serverless-style, with no infrastructure to manage.

What changes

The best part is that isolation, state preservation, and fast startup all come together in one package! It's a great fit for uses like these.

  • AI coding assistants and interactive code environments
  • Data analytics platforms
  • Vulnerability scanners
  • Game servers that run user-supplied scripts

If you're building apps that need an isolated environment per user, this is a great fit. A MicroVM can keep its memory, disk, and running processes for the whole session, so there's no waste from restarting from scratch every time.

Going deeper

Let's look at the specs properly!

A single MicroVM can use up to 16 vCPUs, 32 GB of memory, and 32 GB of disk. The architecture is ARM64, and you can run for up to 8 hours per session.

The startup mechanism is neat. Lambda captures a Firecracker snapshot of your already-initialized app (both disk and memory state) and restores from it, so launching and resuming are very fast.

You also get fine-grained idle controls. For example:

  • maxIdleDurationSeconds: how many seconds of inactivity before auto-suspending (900 seconds = 15 minutes in the example)
  • suspendedDurationSeconds: how long to keep the suspended state
  • autoResumeEnabled: whether to auto-resume when a request arrives

While suspended, the memory and disk state are preserved, so the next incoming request can resume almost instantly.

Getting started is roughly three steps. First, build a MicroVM image from a Dockerfile and your code (a zip in S3):

aws lambda-microvms create-microvm-image \
  --code-artifact uri=s3://my-bucket/artifact.zip \
  --name my-sandbox-image \
  --base-image-arn arn:aws:lambda:us-east-1:aws:microvm-image:al2023-1 \
  --build-role-arn arn:aws:iam::123456789012:role/MicroVMBuildRole

Lambda runs your Dockerfile, initializes the app, and takes a snapshot. Build logs stream to CloudWatch under /aws/lambda/microvms/, and once it's ready you get an ARN and a version number.

Next, launch a MicroVM from that image:

aws lambda-microvms run-microvm \
  --image-identifier arn:aws:lambda:us-east-1:123456789012:microvm-image:my-sandbox-image \
  --execution-role-arn arn:aws:iam::123456789012:role/MicroVMExecutionRole \
  --idle-policy '{"maxIdleDurationSeconds":900,"suspendedDurationSeconds":300,"autoResumeEnabled":true}'

Launching gives you a dedicated HTTPS endpoint, with all the networking handled for you. From there, you generate a short-lived auth token and attach it to the X-aws-proxy-auth header when you call your endpoint.

Available regions are US East (N. Virginia, Ohio), US West (Oregon), Europe (Ireland), and Asia Pacific (Tokyo). It's nice that Tokyo is included from the start! For pricing, check the AWS Lambda pricing page.

Wrap-up

  • AWS Lambda MicroVMs is a new compute primitive for running isolated, stateful sandboxes, serverless-style
  • Firecracker-based, VM-level isolation with no shared kernel or resources between users
  • Up to 16 vCPUs, 32 GB memory, 32 GB disk, ARM64, and up to 8 hours per session
  • Fast startup via snapshots, automatic suspend with state preserved while idle, and auto-resume on requests
  • Use Lambda Functions for event-driven request-response as before, and MicroVMs when you need isolated environments

This announcement is a perfect match for anyone who wants to safely run AI-generated code or untrusted code entrusted by users!

Related posts