Finding the Best Inference Setup Goes From Weeks to Hours! Generative AI Inference Recommendations Arrives in SageMaker AI Studio

Hey friends, it's Shii! Today I found news that makes picking the right inference setup on Amazon SageMaker way easier, so let me share it right away. Finding a good configuration just by clicking through a GUI, I'm so excited!

AWS What's New aws.amazon.com

What was announced?

AWS's What's New announced that Amazon SageMaker AI Studio now has "Generative AI Inference Recommendations." A feature that used to be available only through the API (programmatically) can now be used through a low-code, no-code workflow right in the Studio interface.

Specifically, it's available under "Jobs" > "Inference optimization" in Studio. You describe your workload and what matters most to you, whether that's latency, throughput, or cost, and SageMaker AI searches for and proposes the best inference configuration for you.

The story so far

Deploying generative AI models to production means finding the right combination of instance type, serving container, and optimization strategy, and that's a genuinely hard job. According to the original announcement, this typically took weeks of manual benchmarking, configuration tuning, and trial and error, and even then there was no easy way to confirm whether the final setup was actually optimal.

This exact problem was already addressed once in April 2026 through an API-based launch, letting teams call inference recommendations programmatically. But going through the API left a gap for teams who prefer a visual workflow. This new release extends that same benchmarking infrastructure to Studio's GUI as well.

What changes

What you do as a user is simple, just pick these three things.

• A use case profile (Interact / Generate / Summarize / Custom)
• An optimization goal (minimize latency, maximize throughput, or minimize cost)
• The model to use (selected from SageMaker JumpStart, S3, Model Registry, and more)

From there, SageMaker AI automatically benchmarks multiple configurations on real GPU infrastructure, applies optimization techniques aligned with your goal, and returns ranked recommendations complete with measured performance data. According to the original piece, you no longer have to decide which techniques to apply or how to configure them, and you can reach a validated configuration in hours. Going from weeks of manual tuning to hours through a GUI is a pretty big shift.

Dive Deep

Let's go a bit deeper.

Benchmarking uses NVIDIA's AIPerf, and the key point is that it measures on real GPU infrastructure. The optimization technique applied also changes depending on your goal. Speculative decoding gets applied when throughput matters most, while kernel tuning gets applied when latency matters most, matching the technique to your chosen goal automatically.

As of the original announcement, this feature is available in these 7 regions.

• US East (N. Virginia)
• US West (Oregon)
• US East (Ohio)
• Europe (Ireland)
• Europe (Frankfurt)
• Asia Pacific (Singapore)
• Asia Pacific (Tokyo)

On pricing, generating the recommendations themselves doesn't cost anything extra. You only pay for the compute used to run the benchmark tests and build the endpoint.

Wrap-up

• SageMaker AI Studio now has a GUI-based Generative AI Inference Recommendations feature
• Just pick a use case profile, optimization goal, and model, and SageMaker AI automatically searches for the best configuration
• Benchmarking uses NVIDIA AIPerf, measured on real GPU infrastructure
• Goal-aligned optimization techniques like speculative decoding and kernel tuning get applied automatically
• Available in 7 regions across the US, Europe, and Asia Pacific
• Generating recommendations is free; you only pay for benchmark and endpoint build compute

The nice part is that you can now reach the best inference configuration for a generative AI model through a GUI alone, even if you're not comfortable programming. This update is a great fit if you're running generative AI models in production on SageMaker, or if you're about to decide on a deployment setup.