JA EN
shiichan

Workers AI and AI Gateway merge into Cloudflare's new AI control plane!

Cloudflare Blog ★★★★ View Markdown
#cloudflare#ai#serverless#infrastructure

Hey friends, it's Shii! Today there's news from Cloudflare that should make working with AI a lot simpler for developers. Let me walk you through it!

Cloudflare Blog blog.cloudflare.com

What was announced?

According to Cloudflare's official blog (Cloudflare Blog), Cloudflare announced that it's merging two products, Workers AI and AI Gateway, into a single "AI control plane." This means you can now manage observability, billing, and routing for both models running on Cloudflare-managed GPUs and models from external providers like OpenAI and Anthropic through the same system.

The story so far

Until now, Workers AI and AI Gateway were offered as two separate products.

  • Workers AI: a product that hosts models on Cloudflare-managed GPU infrastructure, with its own dedicated API endpoint
  • AI Gateway: a product that proxies requests to external providers like OpenAI and Anthropic, providing logging, observability, and security

You had to pick which one to use, and bindings and billing were completely separate. Observability wasn't unified by default either, so teams using both Workers AI and external providers had to compare two different dashboards and manage logs and costs through two separate systems.

What changes

With this unification, the bindings for Workers AI and external providers have become one. By passing a gateway argument to the same env.AI.run() call, you now get gateway-based observability no matter which model you're calling.

  • Unified binding: call both Workers AI models and external provider models from the same code
  • Unified REST API: consolidated into a single endpoint under /ai/, replacing the old separate APIs
  • Automatic observability: a default gateway is created automatically on your first authenticated request, and you get full request/response payloads, per-model token counts, and cost breakdowns with no dashboard setup required
  • Unified billing: load credits into a wallet and spend them across supported providers including OpenAI, Anthropic, and Workers AI. AI Gateway credits now apply to Workers AI usage too
  • Elevated rate limits: using AI Gateway's unified billing gets you elevated rate limits on Workers AI models (check the developer docs for the current limits)

For teams debugging model behavior or auditing AI output, it sounds like going from flying blind to having full visibility all at once.

Dive Deep

On the code side, you just add a gateway argument to your existing Workers AI calls to turn on AI Gateway observability.

const response = await env.AI.run(
  '@cf/zai-org/glm-5.2',
  { messages: [{ role: 'user', content: 'Hello!' }] },
  { gateway: { id: 'default' } }
);

Over the REST API, you can call the same model just by hitting a single unified endpoint under /ai/run/.

https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/run/@cf/zai-org/glm-5.2

Still in piloting, there's also a feature called "model-first routing." You just specify the model name you want, and Cloudflare's gateway handles provider selection, failover, and load balancing among the providers hosting that model's weights. The article gives Kimi K2.7 Code as an example:

This way, you can request Kimi K2.7 Code and not care whether it comes from Workers AI, Moonshot's own API, or another provider that hosts the same weights.

If Workers AI has capacity, it's used first; if Workers AI is at capacity, requests are transparently load-balanced to another provider serving the same model. It's also designed with Zero Data Retention requirements in mind.

There's also a "smart routing" feature in internal piloting, where a classifier reads the content of your prompt to judge the task type and complexity, then automatically routes it to the best model from a curated pool.

The article also covers migration:

  • Existing Workers AI users: just add gateway: { id: 'default' } as the third argument to your calls
  • Existing AI Gateway users: add Workers AI models to your requests and consolidate billing through the unified wallet

Either way, existing users can migrate with a small change.

Wrap-up

  • Workers AI and AI Gateway have been unified into a single "AI control plane"
  • Bindings and the REST API are unified, so you can call Cloudflare-managed GPU models and external provider models from the same code
  • Observability turns on automatically from your first authenticated request, giving you logs, token counts, and costs with no dashboard setup
  • Wallet-based unified billing means AI Gateway credits now apply to Workers AI usage too
  • Model-first routing (in piloting) lets the gateway handle provider selection and failover, so you only need to specify a model name

This is a welcome update whether you're already using both Workers AI and AI Gateway, or you're an engineer about to start combining multiple AI providers!

Related posts