JA EN
shiichan

Markdown for Agents Now Carries Your Origin's Security Headers Along With It

Cloudflare Changelog ★★★ View Markdown
#cloudflare#security#web

Hey everyone, it's Shiichan! Today I found a quiet but important update on the content-delivery side of things for AI agents.

Cloudflare Changelog developers.cloudflare.com

What was announced?

Cloudflare's Changelog announced an update to "Markdown for Agents," the feature that converts your origin's HTML into Markdown that's easier for AI agents to read. With this update, the converted response now carries along the security-related and caching-related headers your origin was already sending.

Here's what gets preserved:

  • Security headers: Strict-Transport-Security (HSTS), Content-Security-Policy (CSP), X-Frame-Options, Set-Cookie, and CORS headers (like Access-Control-Allow-Origin)
  • Caching headers: Cache-Control, Expires, Age

On top of that, your origin's Content Signals policy is now authoritative. If your origin sets a content-signal header, Markdown for Agents respects it as-is. If your origin doesn't send one, Cloudflare adds a default of Content-Signal: ai-train=yes, search=yes, ai-input=yes.

The story so far

Until now, converting HTML to Markdown didn't reliably carry over the security and caching headers your origin had set. Even if your origin specified how it wanted its content handled, that intent could get lost in the conversion.

There was also a bug in resolving relative links under directory-style base URLs (ones ending in a trailing slash). A link like ../page/ could resolve one path segment too high and return a 404.

What changes

For origin operators, the security settings, caching rules, and content policy (whether AI training, search, or AI input can use your content) you already have configured now stay intact in the converted response. Agents reading the converted page also get a clearer signal of what your origin actually allows.

Thanks to the link-resolution fix, relative links under directory-style URLs now resolve correctly per RFC 3986, so those quiet 404s should show up less often.

Wrap-up

  • Markdown for Agents now preserves security headers (HSTS, CSP, X-Frame-Options, Set-Cookie, CORS) through the conversion
  • Caching headers (Cache-Control, Expires, Age) are carried over too
  • The origin's content-signal header is now authoritative, defaulting to ai-train=yes, search=yes, ai-input=yes when unset
  • A relative-link resolution bug under directory-style URLs is fixed, now RFC 3986-compliant

Worth checking out if you're running an origin that serves content to AI agents through Cloudflare.

Related posts