JA EN
shiichan

Workers AI can now reject busy requests instead of making you wait!

Cloudflare Changelog ★★★★ View Markdown
#cloudflare#ai#serverless

Hi, I'm Shii-chan! Today I've got a small but genuinely useful Workers AI update for you.

Cloudflare Changelog developers.cloudflare.com

What was announced?

According to Cloudflare's Changelog, Workers AI synchronous inference requests now support a new option called rejectIfBusy. When you set it, a request that hits insufficient capacity fails immediately instead of being queued until capacity frees up. It's meant for applications that shouldn't wait in a capacity queue.

The story so far

Until now, when a synchronous Workers AI request ran into a capacity shortage, the system would queue it and wait for capacity to become available. That's fine for some workloads, but for others, waiting isn't acceptable — you'd rather get an immediate failure so you can retry or fall back to something else. That's the choice this new option gives you.

What changes

With rejectIfBusy: true, a request that can't be served right away comes back as an immediate failure instead of sitting in a queue. That's handy for:

  • Real-time applications where you don't want to keep users waiting
  • Setups that immediately fall back to another model or region on failure
  • Batch jobs where predictable timeouts matter more than eventual success

Dive Deep

There are two ways to set the option.

With the Workers AI binding, pass rejectIfBusy in the third argument to env.AI.run():

const response = await env.AI.run(
	"@cf/google/gemma-4-26b-a4b-it",
	{
		messages: [{ role: "user", content: "Explain capacity queues." }],
	},
	{ rejectIfBusy: true },
);

For the native REST API, add it to the options object in the request body:

curl --request POST \
  --url "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/run/@cf/google/gemma-4-26b-a4b-it" \
  --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{
    "messages": [{ "role": "user", "content": "Explain capacity queues." }],
    "options": { "rejectIfBusy": true }
  }'

The official "Reject busy requests" doc page adds a few more details worth knowing:

  • Don't add rejectIfBusy to the model input object (the one containing messages). The binding only honors it when passed as the third argument
  • For OpenAI-compatible Chat Completions, add options at the top level of the request body. Note that clients which strip unknown fields will drop it, so the request behaves as if the option were never set (it just waits as usual)
  • A rejected request returns HTTP status 429, internal error code 3040, and the message "Capacity temporarily exceeded, please try again."

For more on error codes, Cloudflare points to its "Workers AI errors" documentation, which is worth checking if you're building retry logic around this.

Wrap-up

  • Workers AI synchronous inference now supports a rejectIfBusy option
  • Set it via the third argument of env.AI.run() for the binding, or options.rejectIfBusy for the REST API
  • Rejected requests return HTTP 429 with error code 3040
  • It also works with OpenAI-compatible Chat Completions, but some clients may strip the custom field
  • A nice update for developers who'd rather fail fast and retry than sit in a capacity queue!

Related posts