AgentCore Gateway Now Supports InvokeModel and Automatic Model Fallback!

Hi, it's Shii-chan!

AWS What's New aws.amazon.com

Here's an update for anyone who wants their agents to keep running!

What was announced?

AWS What's New announced an update to Amazon Bedrock AgentCore Gateway. Its inference endpoint now has two new capabilities:

• Support for the Amazon Bedrock InvokeModel and InvokeModelWithResponseStream APIs
• Model fallback routing for inference traffic

The story so far

When you used several model providers, you had to rewrite client code for each one.

Retry logic for throttling or capacity problems also had to be implemented in every client.

What changes

You can point your existing Amazon Bedrock SDK integration at the gateway and reach any model provider through one governed endpoint.

When the primary destination is throttled, out of capacity, or unreachable, the gateway automatically routes the request to an alternative model or provider. No client needs its own retry logic, so your agents are less likely to go down.

Dive Deep

Here's how fallback is set up:

• In a gateway rule, you define an ordered chain of alternative destinations
• You also define the conditions that trigger a switch, such as throttling, server errors, or response timeouts

A built-in circuit breaker skips destinations that are currently failing. Requests reach a healthy destination faster, and you avoid added latency from failed attempts.

Fallback happens behind the scenes, so you send one request and receive one response, regardless of the internal trial and error.

It's available in all AWS Regions where AgentCore Gateway is available, except those in China.

Wrap-up

• The inference endpoint supports InvokeModel and InvokeModelWithResponseStream
• Point your existing Bedrock SDK integration at the gateway to reach multiple providers
• Fall back to alternative destinations in order on throttling, server errors, timeouts and more
• A circuit breaker skips destinations that are failing

This is for anyone running agents on multiple models who wants better availability!