JA EN
shiichan

AWS WAF can now charge AI bots! Content owners can finally collect money from crawlers

AWS Blog ★★★★ View Markdown
#aws#security#web

Hi everyone, it's Shii-chan! Today I found an announcement that feels like a little glimpse of the future for anyone who runs a website. You can now literally charge AI bots for access to your content!

AWS Blog aws.amazon.com

What was announced?

Over on the AWS Blog, AWS WAF got a new capability called AI traffic monetization. It's a new Bot Control feature that lets content providers and publishers set a price, meter usage, and collect payment from the AI bots and agents that access their content and APIs.

Even better, that pricing decision happens right at the network edge. So before a request ever reaches your origin server, AWS WAF can tell the bot, "this access costs money."

The story so far

Until now, dealing with AI crawlers mostly meant a binary choice: allow or block. You could ask nicely with robots.txt, or block them with Bot Control. But there was no middle path for "I'm happy to let you read this, but not for free."

According to the article, AI bot traffic now makes up more than half of all web traffic for many content providers, and AI-specific crawlers have grown more than 300% year over year. AI reads your content and turns it into a summary, but very few readers come back to the original site. Meanwhile you still pay to serve the content, and you lose ad impressions and subscription sign-ups. For publishers, that's a real problem.

What changes

Now there's a third option beyond "block or allow": "come on in, if you pay."

If you own content, you can set a price per page and collect fair compensation from AI bots. And if you run an AI agent, you can follow the rules, pay, and access the content out in the open. It becomes a fair deal for both sides. I think it's an especially welcome option for news media, database sites, and anyone offering an API!

Dive Deep

Let's look at how it works in a bit more detail.

Built on Bot Control

This feature sits on top of AWS WAF Bot Control. Bot Control can identify more than 650 AI bot and agent types, including GPTBot, Claude-Web, and Perplexity-Bot, and it assigns each one a verification tier:

  • Verified: bots whose identity is confirmed by a cryptographic signature or documented IP ranges
  • Unverified: bots identified by user-agent matching and behavioral fingerprinting

Payment via HTTP 402

When a monetization rule matches, AWS WAF returns the wonderfully old-school HTTP 402 Payment Required. The response body carries a machine-readable price manifest in JSON, using the open x402 protocol.

The manifest includes the price in USDC stablecoins, the accepted blockchain networks (Base and Solana), the destination wallet address, timeout limits, and the payment scheme. Any x402-compatible agent runtime can complete this flow on its own, so the whole money exchange can happen machine to machine without a human in the loop.

Configuring a "protection pack"

On the content side, you start in the AWS Management Console under WAF & Shield by creating a "protection pack." There you define which content paths are monetized, the price per verification tier, the accepted payment methods, and your license terms.

For pricing, you set a base price per page in USDC per blockchain network. Then, for each verification tier, you can pick from six actions:

  • Monetize
  • Allow
  • Block
  • Count
  • CAPTCHA
  • Challenge

Coinbase handles settlement

Payment settlement and verification are provided by Coinbase's x402 Facilitator. The important part: AWS doesn't process payments and doesn't take a fee on your content revenue. Disbursement is self-managed or handled by your wallet provider. Support for Stripe and the Machine Payments Protocol (MPP) is on the way.

Dashboard and revenue visibility

A dedicated dashboard splits traffic into four buckets: all bot requests, AI bot requests, verified AI bot traffic, and unverified AI bot traffic. For revenue, you can track total revenue, revenue from verified bots, revenue from unverified bots, and the average per request.

Start in test mode

Moving real money in production straight away is nerve-wracking, right? For that, you can switch "Currency mode" to Test and try the flow on testnets like Base Sepolia or Solana Devnet, using test funds from a faucet.

A few things to keep in mind

There are some limitations worth noting:

  • The Monetize action only works together with Amazon CloudFront distributions
  • Regional web ACLs can't use monetization actions
  • The dashboard reflects only real-currency-mode activity

To get started, you need Bot Control enabled at the Common or Targeted level on the associated web ACL.

Wrap-up

Here's a quick recap:

  • AWS WAF Bot Control adds "AI traffic monetization," letting you charge AI bots for content access
  • Pricing happens at the edge; on a match it returns HTTP 402 with an x402 price manifest in JSON
  • Prices are set in USDC, settlement runs through Coinbase's x402 Facilitator, and AWS takes no fee
  • You configure it from a "protection pack," choosing actions like Monetize/Allow/Block per verification tier
  • It's available now for Amazon CloudFront with no extra charge beyond standard AWS WAF pricing

For news media, publishers, and API providers who were tired of AI crawlers taking their content for free, this looks like a fresh way to earn from it. A world where machines negotiate payments at the edge? That's kind of exciting!

Related posts