Bedrock Advanced Prompt Optimization Can Now Shorten Your Prompts
Hi, it's Shii-chan!
AWS What's New
aws.amazon.com
Good news if you're worried about prompts getting too long and token costs creeping up!
What was announced?
AWS What's New announced that Amazon Bedrock Advanced Prompt Optimization now supports prompt shortening, an optional setting that reduces the length of your prompt while maintaining or improving your evaluation scores.
Advanced Prompt Optimization optimizes prompts for any model on Amazon Bedrock against your own evaluation criteria.
The story so far
Until now, optimization focused on quality alone, and optimized prompts were often longer than the original. Longer prompts raise input token costs and can increase time to first token, which matters at scale and in latency-sensitive applications.
What changes
When you turn on prompt shortening for a prompt template in your input dataset, the optimizer reduces the prompt's length.
- The optimized prompt is returned only when it is shorter than your original and scores at least as high on your evaluation
- Otherwise your original prompt is returned, so you never end up with a longer or lower-scoring prompt
Dive Deep
Results include a compression ratio and the original and optimized token counts, in the output file and on the console results page. For example, a compression ratio of 0.7 means the optimized prompt uses 30% fewer tokens.
It works with any evaluation method, including steering criteria, LLM-as-a-judge rubrics, and custom Lambda evaluators.
It's available in all AWS Regions where Advanced Prompt Optimization is available.
Wrap-up
- Prompt shortening is an optional setting
- The optimized prompt is returned only if it is shorter and scores at least as high
- Compression ratio and token counts appear in the results
- Works with any evaluation method
This is for anyone trimming token costs and latency!
