Cloudflare's AI Platform brings every model's inference under one API!

Hey everyone, it's me, Shiichan! Today Cloudflare dropped a big foundation for building AI agents, so let me walk you through it.

Cloudflare Blog blog.cloudflare.com

What was announced?

On the Cloudflare Blog, they announced that AI Gateway is evolving into a single inference layer. The goal is easy to say and powerful in practice: people building AI agents can reach models from many providers through one API. You get access to 70+ models across a whole bunch of providers from the same entry point.

This time multimodal models -- image, video, and speech -- join the party too. Access expands to models from Alibaba Cloud, Google, OpenAI, Runway, MiniMax, and more, and you can pick an agent-optimized model like Kimi K2.5.

The story so far

Until now, when each model lived behind a different provider, both the calling style and the billing were scattered. Cloudflare notes that companies today call an average of 3.5 models across multiple providers, so no single provider could show you the whole picture of how much you spend on AI.

What changes

With the new AI.run() binding, you can switch providers with a one-line change. Swapping or mixing models becomes really lightweight.

Billing gets nicer too: AI Gateway gives you one central place to see all of your AI usage. Attach custom metadata and you can break costs down by free versus paid users, a specific customer, or a specific workflow.

Dive Deep

Agents chain work dozens of times, so the speed and stability of inference really matter. Here's how Cloudflare puts it:

An agent might chain ten calls together to complete a single task and suddenly, a single slow provider doesn't add 50ms, it adds 500ms.

So there are several reliability tricks. If one provider goes down, it fails over automatically, and streaming responses are buffered. And if your agent gets cut off mid-inference...

If your agent is interrupted mid-inference, it can reconnect to AI Gateway and retrieve the response without having to make a new inference call.

In other words, you pick the response back up without redoing the inference. Fewer wasted calls is kinder to your wallet too.

On speed, the plan is to shrink latency across Cloudflare's 330 data center locations. For Cloudflare-hosted models there's no extra hop over the public Internet, which helps that all-important time to first token for responsive agents.

There's also a bring-your-own-model path. Using Replicate's Cog containerization, you deploy with a cog.yaml config and a Python predict script (Replicate just joined Cloudflare). GPU snapshotting for faster cold starts is in the works.

For long-running agents, you can pair the checkpoint-friendly Agents SDK with Workers AI underneath.

Wrap-up

• AI Gateway grows into an inference layer where one API reaches any provider's models
• The AI.run() binding makes switching providers a one-liner, with spend managed in one place
• Automatic failover, streaming buffering, and mid-inference reconnection keep agents from stalling
• 330 locations and no extra hop speed up time to first token
• Cog lets you bring your own model, and it all connects to the Agents SDK / Workers AI

This one is perfect for developers stitching agents across many models, and for teams wrestling with AI cost and reliability.