Cloudflare Bots Now Lets You Control Personal Agents One by One

Hi everyone, it's Shii-chan! If you have been wondering how to handle AI agents visiting your site, here is some good news!

Cloudflare Changelog developers.cloudflare.com

What was announced?

In the Cloudflare Changelog, the Bots settings were updated:

• The Configure AI bot policies setting is now named AI Crawler and Agent access policies
• You can set actions for the top Personal Agents individually

Personal agents act on a person's behalf, fetching information or browsing in response to a request.

Why it matters

Controlling all agent access in one go has downsides. For example, you may want visitors to use agents to find answers or compare products, while still limiting access you don't want.

What changes

With this update, you no longer need to apply one action to all agent traffic. You can configure the top Personal Agents one by one, which gives you control over agent access while still allowing visitors to use agents with your site's content.

Dive Deep

Here is where to find it and what to know:

• Location: Security > Settings > AI Crawler and Agent access policies
• Default: Personal Agents inherit your existing Agents policy
• Cloudflare's recommendation: Allow (do not block) for Personal Agents

The changelog entry does not say which agents are covered by the individual settings. For details, see Cloudflare's blog post and the documentation for AI Crawler and Agent access policies.

Wrap-up

• Configure AI bot policies is now AI Crawler and Agent access policies
• Actions can be set for the top Personal Agents individually
• The default inherits your existing Agents policy
• Cloudflare recommends Allow (do not block)

If you run a site and want finer control over how AI agents treat it, this update is for you!