An AI agent fleet auto-hunts vulnerabilities! How Cloudflare builds its own vulnerability harness
Hi everyone, it's Shii-chan! I found a really exciting write-up today! Cloudflare just laid out, end to end, how to build a 'vulnerability harness' where a fleet of AI agents automatically hunts for security bugs!
Cloudflare Blog
blog.cloudflare.com
What was announced?
On the Cloudflare Blog, a post called 'Build your own vulnerability harness' went live. This isn't a product announcement — it's an engineering-flavored deep dive that walks through the technical architecture of the vulnerability discovery system Cloudflare runs internally.
The system splits into two big parts:
- Vulnerability Discovery Harness (VDH): the discovery engine that keeps surfacing bugs
- Vulnerability Validation System (VVS): the triage side that re-checks each candidate with a different model
The key idea is treating 'models as interchangeable components,' so they can swap in whichever frontier model is currently best as models keep evolving.
Why it matters
If you just hand code to an AI and say 'find the bugs,' you get a mountain of plausible-but-empty findings (false positives). And if you make one agent do everything, the context window fills up fast and the model starts cannibalizing its own memory.
What makes this post great is that it shows concretely how they got past the walls you always hit at scale. And it comes with real mileage: 128 repositories across many languages like Rust, Go, C, Lua, TypeScript, and Python!
What changes
The best part is that this approach was published in a form anyone can try. Cloudflare open-sourced security-audit-skill, the foundation the whole thing is built on.
It's a skill for coding agents that runs multiple agents in parallel and audits security in stages: recon → hunt → validate, and so on. According to the post, a single run finds only about half the bugs you'd catch across multiple runs — so it's designed so that coverage climbs the more you re-run it.
Dive Deep
Let's look inside the machinery!
An 8-stage pipeline (VDH)
VDH runs eight stages: Recon → Hunt → Validate → Gapfill → Dedup → Trace → Feedback → Report.
- Recon: three parallel agents map the architecture and attack paths and produce
architecture.md - Hunt: per-attack-class agents actually run code fragments in sandboxes to attack the target
- Validate: first deterministic schema checks, then a separate agent tries to disprove the finding
- Gapfill / Trace / Feedback: spawn new tasks for thin coverage, walk dependency graphs into other repos, and rewrite the next prompts based on failures
- Report: a model-free script that renders human-readable output
The validation side, VVS, has three stages: Dedup → Judgment → Fixing. In Judgment it even consults MCP servers, Jira, and git configs to decide whether a path is truly reachable in production.
Push all state outside the model
What struck me as clever is treating the LLM as a stateless compute engine. Every stage's results are written to a single SQLite database keyed by (run_id, repo, stage). So if something crashes, the only thing lost is the one in-flight task, and any stage can resume.
They also keep each agent's scope very narrow so context usage stays below 25% of the total window. It started as a single ~450-line monolithic skill, but that hit its limit after about an hour, so they split it into per-phase agents.
How do they crush false positives?
Every finding must state an explicit attacker definition and threat model. On top of that, a working PoC (proof of concept) has to run successfully against the original, unmodified source. That prevents agents from quietly editing the code to force an exploit. The validator agent can't log findings of its own — its only job is to aggressively disprove the hunter's theory. And because VVS uses a different model, everything gets double-checked by an entirely different set of 'logical weights.'
Ask for what's missing via a wishlist
When an agent lacks a tool it needs, it can request external resources through a central wishlist. That was used 25,472 times across 128 repos. And when an agent spots interesting out-of-scope code, instead of wandering off it forks a 'sibling' agent to handle it. That branching accounts for 0–20% of tasks depending on the model, and roughly 9% fleet-wide.
One fun detail: they wired in the static analyzer Semgrep, but over a month it was invoked zero times — the agents preferred reading and running the code themselves!
The numbers
Across the whole fleet, of 20,799 raw candidates from VDH, 12,057 survived independent validation (about 58%). After improving Recon, the validation rejection rate dropped from 40% to 11%, and the share of high-integrity findings rose from 35% to 58%. Through VVS, from 13,841 items (including other sources), they filtered out 5,442 duplicates and 1,154 wrong-repo/low-risk items, leaving 7,245 actionable findings for engineering teams.
When Cloudflare ran it on a single ~30k-line repo, 100 raw candidates compressed to 80, discovery took 3–4 hours, the automated Fixer took about 5 minutes per bug, and the whole path from discovery to PR generation took roughly 14 hours. But it never merges a fix on its own — human review is required. That gate is the non-negotiable safeguard.
Wrap-up
- Cloudflare walked through the full architecture of its in-house vulnerability discovery systems (VDH and VVS)
- LLMs are treated as stateless components, state lives in SQLite, and context stays below 25%
- Different models for discovery and validation, plus mandatory PoCs on unmodified code, crush false positives
- Across the fleet, 20,799 candidates were narrowed down to 7,245 actionable findings
- The underlying
security-audit-skillis already on GitHub, so you can try it with your own agent - A great read for engineers into security automation and AI agent design!
