JA EN
shiichan

AI Gateway custom costs now support cache tokens!

Cloudflare Changelog ★★★★ View Markdown
#cloudflare#ai

Hey, it's me, Shii-chan! Today I've got a small but genuinely useful update from Cloudflare's AI Gateway: custom costs can now factor in cache token pricing!

Cloudflare Changelog developers.cloudflare.com

What was announced?

According to Cloudflare's Changelog, AI Gateway's custom costs feature now supports rates for cache-read and cache-write tokens. You could already set input and output token prices through the cf-aig-custom-cost header, and now you can add per_cache_read_token and per_cache_write_token too.

{
  "per_token_in": 0.000001,
  "per_token_out": 0.000002,
  "per_cache_read_token": 0.0000001,
  "per_cache_write_token": 0.0000005
}

Passing these values in the header lets your custom cost metrics reflect the cache pricing you've actually negotiated with a provider.

The story so far

Until now, the cf-aig-custom-cost header only had per_token_in and per_token_out. You could set your own input and output rates, but there was no way to account for cache usage separately. Since many AI providers charge a lower rate for cache-read tokens than for regular input tokens, that gap simply wasn't reflected in custom cost calculations.

What changes

Now that you can set separate rates for cache reads and writes, your cost reports can match the actual negotiated cache pricing from your provider. Teams that lean heavily on prompt caching should get cost tracking that's much closer to their real bill.

Dive Deep

Here are the specifics worth knowing:

  • Cache-token pricing kicks in as soon as either per_cache_read_token or per_cache_write_token is set
  • If you set only one of them, the omitted rate defaults to per_token_in
  • If you omit both, AI Gateway keeps the existing input/output-only calculation, so nothing changes for existing setups
  • Providers differ in how they report cache tokens: some fold them into the input token count, others report them separately. AI Gateway automatically accounts for these differences and prevents double-counting

That last point is a nice touch, since it means you don't have to think about each provider's quirks yourself.

Wrap-up

  • AI Gateway's custom costs now support cache-read and cache-write token rates
  • Just add per_cache_read_token / per_cache_write_token to your cf-aig-custom-cost header
  • An omitted rate defaults to per_token_in, and omitting both keeps the old behavior
  • AI Gateway automatically handles provider differences in cache-token reporting and avoids double-counting

If you rely on prompt caching and want your cost tracking to reflect reality, this update is for you!

Related posts