JA EN
shiichan

Prompt tuning goes automatic: Amazon Bedrock adds Advanced Prompt Optimization!

AWS Blog ★★★★ View Markdown
#aws#ai#devops

Hey everyone, it's Shiichan! AWS dropped a tool today that makes prompt-building a whole lot easier, so let me walk you through it.

AWS Blog aws.amazon.com

What was announced?

On the AWS Blog, Amazon Bedrock announced a new feature called Advanced Prompt Optimization. It optimizes your prompts for any model on Bedrock and lets you compare performance before and after the optimization. You can even line up as many as 5 models at once, which makes it a great fit when you are thinking about migrating to a different model.

It handles more than plain text too: multimodal inputs like PNG, JPG, and PDF are supported. You can also run regression checks on known use cases to make sure things did not get worse.

The story so far

Tuning prompts used to be pretty hands-on. You would tweak the wording, test it, eyeball the result, tweak again, and repeat. And when you switched models, you often had to rewrite the prompt to match that model's quirks, which was quietly a lot of work.

What changes

Advanced Prompt Optimization runs on a metric-driven feedback loop. Here is how it works:

  1. You provide a prompt template, example user inputs, ground truth answers, and evaluation metrics.
  2. The tool automatically sends the templates and data to inference models.
  3. It scores the responses using your chosen metric.
  4. It iteratively rewrites the prompt to improve the results.
  5. It outputs the original and final templates along with evaluation scores, cost estimates, and latency.

In other words, the tool runs the "test, measure, fix" loop for you. Instead of tuning on gut feeling, you get to improve based on numbers, which is the best part.

Dive Deep

You can pick from 3 scoring methods:

  • Lambda function: Write your own scoring logic in Python. Great for concrete metrics like accuracy, F1, execution accuracy, or JSON matching.
  • LLM-as-a-Judge: An LLM scores against a rubric. Good for open-ended tasks like summarization, and Claude Sonnet 4.6 is the default judge.
  • Steering criteria: Describe the qualities you want, such as brand voice, format, or safety, in natural language and have them evaluated holistically.

Both console and API paths are available. In the console, open the Advanced Prompt Optimization page, choose "Create prompt optimization", pick up to 5 models, upload a JSONL file with your templates and evaluation data, and specify an S3 output location. From the API, just call CreateAdvancedPromptOptimizationJob.

The main fields you put in the JSONL look like this:

version: bedrock-2026-05-14
templateId
promptTemplate
evaluationSamples

As for pricing, you pay for the Bedrock inference tokens used during optimization, at the same per-token rate as regular Bedrock inference. The post puts it this way:

Charged based on the Bedrock model-inference tokens consumed during optimization, at the same per-token rates as regular Bedrock inference.

The region coverage is wide: US (N. Virginia, Ohio, Oregon), Asia Pacific (Mumbai, Seoul, Singapore, Sydney, Tokyo), Canada (Central), Europe (Frankfurt, Ireland, London, Zurich), and South America (Sao Paulo). Tokyo is on the list, which is good news for folks in Japan.

Wrap-up

  • Amazon Bedrock gets a new tool, Advanced Prompt Optimization, that optimizes prompts for you automatically.
  • You can compare up to 5 models at once and use it for model migration or regression checks.
  • Scoring comes in 3 flavors: Lambda function, LLM-as-a-Judge (Claude Sonnet 4.6 by default), and steering criteria.
  • Pricing is the inference tokens used during optimization, at the same per-token rate as regular Bedrock inference.
  • It is available in many regions, including Tokyo.

If you have been sinking time into prompt tuning, or you are weighing a move to another model, this one is for you. Prep a JSONL next week and give it a spin to feel out the improvement!

Related posts