AI Search similarity cache: pick your own freshness, now 48 hours by default!

Hi everyone, it's me, Shii-chan! Today we've got a nice little control landing in Cloudflare's AI Search.

Cloudflare Changelog developers.cloudflare.com

What was announced?

Cloudflare's Changelog announced that you can now fine-tune the freshness of the similarity cache in AI Search. The similarity cache reuses responses for semantically similar queries, which cuts down latency and inference costs.

The story so far

Until now, the cache duration was fixed at 30 days. Handy, sure, but sometimes you'd rather not keep serving a slightly stale answer.

What changes

First, the default duration is now 48 hours (down from the fixed 30 days). On top of that, you can set cache_ttl yourself, anywhere from 10 minutes to 6 days. Whether you want fresher answers or more cache hits, you can tune it to your use case.

Dive Deep

cache_ttl is specified in seconds. For example, 6 days looks like this:

cache_ttl: 518400  # 6 days in seconds

There's also an on-demand purge for cached responses. You can trigger it from the API or the instance settings in the Cloudflare dashboard, and purging does not delete your indexed content or source files.

Wrap-up

• You can now control the freshness of AI Search's similarity cache
• Default is 48 hours; tune cache_ttl from 10 minutes to 6 days
• On-demand cache purge is supported (your index and source files stay intact)

If you've been wrestling with RAG cache freshness, this one's for you!