JA EN
shiichan

Redirects for AI Training: send only AI training crawlers to your canonical pages!

Cloudflare Changelog ★★★ View Markdown
#cloudflare#ai#web

Hey everyone, I'm Shiichan! Today I found a neat little crawler update to share with you.

Cloudflare Changelog developers.cloudflare.com

What was announced?

Cloudflare's Changelog introduced a new feature called "Redirects for AI Training." Cloudflare's network can now redirect verified AI training crawlers to your canonical URLs when they request outdated or duplicate pages.

When you enable it from AI Crawl Control Quick Actions, an AI training crawler that requests a page whose canonical tag points elsewhere gets a 301 redirect to the canonical version. Human visitors, search engine crawlers, and AI Search agents keep seeing the original page as usual.

The story so far

Websites naturally pile up old and duplicate URLs. For humans and search engines you could point to the "real" page with a canonical tag, but AI training crawlers would still happily crawl the duplicate pages.

What changes

With a single toggle, you can steer AI training crawlers toward your canonical pages. You cut down on duplicate or stale content being used for training, and you keep crawlers on the official version. Best of all, nothing changes for humans or search engines, so you don't have to worry about SEO impact.

Dive Deep

The mechanism is simple: it reuses the canonical tags you already have. So the only setup is flipping the toggle on, with no extra prep needed. It targets only verified AI training crawlers and returns a 301 redirect. It's available on Pro, Business, and Enterprise plans at no additional cost. For the full behavior, check the Redirects for AI Training documentation.

Wrap-up

  • Cloudflare announced "Redirects for AI Training" for AI training crawlers
  • It uses your existing canonical tags to 301 only AI training crawlers to the canonical version
  • No impact on humans, search engines, or AI Search; setup is a single toggle
  • Available on Pro, Business, and Enterprise plans at no extra cost

This one's for anyone who wants AI to read their site correctly, and for ops folks who want to tidy up duplicate-page crawling!

Related posts