JA EN
shiichan

URL Scanner now checks your site's "AI agent readiness"!

Cloudflare Changelog ★★★★ View Markdown
#cloudflare#security#web#ai

Hello, it's Shii! Today I'm sharing some news I found in Cloudflare's Changelog. A new report feature for AI agents has been added to the dashboard's "URL Scanner" diagnostic tool. Let's take a look!

Cloudflare Changelog developers.cloudflare.com

What was announced?

According to Cloudflare's Changelog, a new "Agent Readiness" tab has been added to URL Scanner reports. URL Scanner is a tool you can use from the dashboard to scan any URL and diagnose the state of a site, and with this update the scan results now include an evaluation of how well a site supports emerging AI standards. It comes with six specialized scores designed to help you optimize your site for the next generation of AI agents and automated discovery.

Why it matters

Here's how the original post puts it.

The Internet is shifting from a human-read web to a machine-read web.

AI agents are no longer just browsing sites; they're increasingly interacting with them and even completing transactions on them.

If a site isn't "agent-ready," bots may end up consuming excessive bandwidth, failing to find critical information, or being unable to navigate services efficiently. Even if a site looks great to humans, a structure that's hard for machines to read or handle could become a real disadvantage going forward.

What changes

Site owners can now scan their own site with URL Scanner and see exactly where it falls short from an AI agent's perspective, across six categories. Instead of just assuming your site is probably fine, you get concrete scores that show what to fix.

Dive Deep

Here are the six specialized scores.

  • Basic Web Presence
  • Discoverability
  • Content Accessibility
  • Bot Access Control
  • Protocol Discovery
  • Commerce

Each one looks at how easy your site is for a machine to find, read, and access, from a different angle.

Here's how to check them.

  • From the dashboard: Go to Protect & Connect > Application Security > Investigate, run a scan on a URL, then select the Agent Readiness tab in the report to see the scores
  • Programmatically: You can also retrieve these scores automatically using the URL Scanner API

Cloudflare has also published a separate post explaining the methodology behind how each score is calculated, so check that out too if you're curious.

Wrap-up

Here's a recap of today's key points.

  • URL Scanner reports now include a new "Agent Readiness" tab
  • Scores are broken down into six categories: Basic Web Presence, Discoverability, Content Accessibility, Bot Access Control, Protocol Discovery, and Commerce
  • You can check them from the dashboard (Protect & Connect > Application Security > Investigate) or via the URL Scanner API
  • The goal is to help sites prepare for the shift from a human-read web to a machine-read web

With AI agent traffic only set to keep growing, this feature is a great fit for anyone who wants to check just how "agent-ready" their own site or service really is!

Related posts