Awesome Security Agent Harnesses

AI agents for pentesting, code audit, fuzzing, vulnerability discovery, and reverse engineering — harnesses, sandboxes, security MCP servers, benchmarks, and evals.

Please read the contribution guidelines before opening a pull request.

Contents

• What Is a Security Agent Harness
• Code Audit Harnesses
• Pentesting Agents
• Fuzzing and Vulnerability Discovery
• DARPA AIxCC Cyber Reasoning Systems
• Agent Tooling and Integrations
• Agent Sandboxes
• Benchmarks and Evals
• Readings

What Is a Security Agent Harness

A security agent harness is everything wrapped around the model: the sandbox it runs in, the analysis tools it can call, the prompts and skills that encode a methodology, and the evals you use to check it. Most of the engineering lives here rather than in the model — or as Cloudflare put it after scaling one across its own fleet, the harness is the bit that lasts.

Agents are good at producing plausible findings and bad at telling which ones are real. An agent will edit the source so its own exploit works, then report the bug it just created. So the metric that matters is not recall, which nobody can measure without already knowing every bug in a codebase, but how few unconfirmed findings reach a human: Cloudflare's pipeline cut 20,799 raw candidates down to 12,057 that survived validation, then folded away another 5,442 as duplicates. A harness that can reproduce a crash, replay an input, or re-run a static analyzer is how you throw out the bad ones before a human ever sees them.

Entries tagged SKILL.md are skill packs rather than runnable harnesses: methodology encoded as Agent Skills that a general coding agent like Claude Code executes. They live in the section matching what they do.

Code Audit Harnesses

Harnesses that run coding agents against source code: discovery, triage, validation, and patching.

• Anthropic Defending Code Reference Harness - Reference implementation for autonomous vulnerability discovery and remediation with Claude, with skills for threat modeling, scanning, triage, and patching.
• audit - Eight-stage discovery agent built on the Claude Code SDK, combining many narrow agents, deliberate disagreement between them, and an explicit reachability gate.
• Capital One VulnHunter - Agentic security tool applying proactive, attacker-first analysis directly to source code.
• Clearwing - Autonomous source-code hunter that ranks files, fans out specialist agents, and treats sanitizer crashes as ground truth, with a separate network-pentest mode.
• Cloudflare Security Audit - Six-phase audit skill: parallel hunting agents attack the codebase from different angles, separate agents try to disprove each finding, and fresh agents verify the schema-validated findings.json against source. SKILL.md.
• OpenAI Codex Security - CLI and TypeScript SDK for finding, validating, and fixing security vulnerabilities with Codex.
• OpenHack - Multi-agent scanner running recon, specialist hunts, independent validation, and sandbox and browser verification, using only open-source models.
• RAPTOR - Autonomous research framework chaining static analysis, binary analysis, vulnerability validation, exploit generation, and patch writing over a codebase or binary. SKILL.md.
• Trail of Bits Skills - Skills for security research, vulnerability detection, and audit workflows, distilled from the firm's audit practice. SKILL.md.
• Vercel Labs Deepsec - Security harness for finding vulnerabilities in a codebase using coding agents.
• Visa Vulnerability Agentic Harness - Agentic SAST pipeline for autonomous vulnerability discovery, remediation, and validation, emitting Markdown and SARIF reports.

Pentesting Agents

Agents that attack running applications and infrastructure: reconnaissance, exploitation, and proof of impact.

• AIDA - Model-agnostic pentesting agent that reasons over a defined scope, executes in an isolated container, and keeps persistent assessment state across sessions.
• AWE - Research framework pairing a lightweight orchestration layer with memory-augmented, vulnerability-specific agent pipelines, evaluated on the XBOW benchmark.
• BlacksmithAI - Multi-agent pentesting framework that walks a target from reconnaissance through post-exploitation inside a Docker image preloaded with standard security tooling, driven from either a web UI or a CLI.
• CAI - Alias Robotics' framework for building cybersecurity agents, with tool and workflow primitives for offensive testing.
• CyberStrike - Offensive-security harness coordinating autonomous agents over signed attack skills, built-in tools, and MCP integrations, mapped to MITRE ATT&CK and OWASP WSTG.
• hackingBuddyGPT - TU Wien research framework for writing LLM pentesting agents in roughly 50 lines of code, released alongside reusable Linux privilege-escalation benchmarks and open-access evaluations.
• HPTSA - Research implementation behind Teams of LLM Agents can Exploit Zero-Day Vulnerabilities, using a supervisor agent to coordinate vulnerability-specialized subagents.
• Nebula - Penetration-testing assistant that automates recon, note-taking, and tool orchestration.
• NeuroSploit - Rust harness that turns a URL, repository, or host into an autonomous engagement, selecting only the specialist agents matching the surface and validating findings across models.
• PentAGI - Fully autonomous multi-agent pentesting system with Docker isolation, planning and supervision, persistent memory, and a web control plane.
• Pentest Copilot - Agent that drives a Kali attack box end to end, installing tooling as needed, operating Burp and a real browser, and spawning parallel subagents.
• Pentest Swarm AI - Go harness coordinating recon, classification, exploitation, and reporting agents through a shared pheromone blackboard, waking each agent by finding weight rather than a fixed pipeline order.
• PentestAgent - Black-box testing framework with autonomous and multi-agent modes, attack playbooks, Kali execution, and persistent sessions.
• PentestCode - Hard fork of OpenCode rebuilt for offense, coordinating a lead strategist and parallel specialist subagents over shared engagement state, with attack-path search across a relationship graph and persistent sessions.
• PentestGPT - Automated penetration-testing agentic framework powered by large language models.
• RedAmon - End-to-end platform chaining recon, exploitation, and post-exploitation over a Neo4j attack graph, then triaging findings, patching the code, and opening pull requests.
• reverse-skill - Security skill router that guides coding agents through repeatable reverse-engineering and penetration-testing workflows with tool bootstrapping and evidence tracking. A recent Tessl review scored its primary router 75/100 (92% quality), with no impact evaluation and a failed security scan. SKILL.md.
• Shannon - AI pentester for web applications and APIs that analyzes source code, identifies attack vectors, and executes real exploits to prove findings.
• Strix - Open-source AI penetration-testing agent that finds and helps fix application vulnerabilities.
• Transilience Community Tools - Twenty-six skills and three tool integrations covering the pentest lifecycle from recon to reporting, with a published 104/104 result on the maintainers' CTF benchmark. SKILL.md.

Fuzzing and Vulnerability Discovery

Harnesses that use models to find crashes and generate patches.

• ChatAFL - Protocol fuzzer built on AFLNet that prompts a model for a machine-readable grammar, for richer seed messages, and for new inputs whenever coverage plateaus, published at NDSS 2024 and packaged as a ProfuzzBench artifact.
• FirmAgent - Hybrid IoT firmware pipeline from Tsinghua's VUL337 group, pairing a device-aware API fuzzer with an LLM taint-analysis agent that turns sink-reaching findings into proof-of-concept exploits, published at NDSS 2026.
• Fuzz4All - Universal fuzzer using a model as its input generation and mutation engine, with an autoprompting step that tunes the prompt per target so one tool covers many input languages, from the ICSE 2024 paper.
• OSS-Fuzz-Gen - Google's framework for generating and benchmarking fuzz targets with LLMs across C/C++, Java, and Python.
• PromptFuzz - Fuzz-driver generator that mutates prompts in a coverage-guided loop to explore complex API interrelationships, reporting 40.12% branch coverage on its tested libraries, 1.61x OSS-Fuzz's, and 33 confirmed security bugs including three CVEs.

DARPA AIxCC Cyber Reasoning Systems

The seven finalist systems from the DARPA AI Cyber Challenge, each built to autonomously find and patch vulnerabilities in real open-source code, and each released as a competition snapshot.

• ARTIPHISHELL - Shellphish's cyber reasoning system, released with its components, pipelines, services, and full deployment stack.
• Atlantis - Team Atlanta's cyber reasoning system that won the final competition.
• BugBuster - Team 42-b3yond-6ug's cyber reasoning system, preserved with the core components and deployment configuration needed to run it.
• Buttercup - Trail of Bits' second-place cyber reasoning system, pairing an OSS-Fuzz fuzzing campaign with a multi-agent patcher.
• FuzzingBrain - Team all_you_need_is_a_fuzzing_brain's system, pairing coverage-guided fuzzing with agents that reason about suspicious points, build proofs of vulnerability, and dynamically verify every finding.
• Lacrosse - SIFT's multi-agent cyber reasoning system, combining fuzzing and symbolic reasoning to find and patch bugs in C and Java.
• Robo Duck - Theori's third-place cyber reasoning system, released as the complete finals submission.

Agent Tooling and Integrations

Security capabilities exposed to somebody else's agent: MCP servers, disassembler plugins, and broad skill libraries. These supply the tools and methodology a harness calls; they do not own the agent loop themselves.

• Anthropic-Cybersecurity-Skills - Community library of 800+ skills across 29 security domains, mapped to MITRE ATT&CK and NIST CSF; not affiliated with Anthropic despite the name. SKILL.md.
• Binary Ninja Headless MCP - Headless Binary Ninja MCP server exposing 180 analysis tools to agents.
• Burp Suite MCP - PortSwigger's own MCP server, connecting agents to Burp Suite's proxy, scanner, and repeater.
• Claude Code CyberSecurity Skills - Nineteen skills spanning offensive security, defensive operations, reverse engineering, threat hunting, and SOC automation. SKILL.md.
• DAILA - Decompiler-agnostic plugin for using AI assistance inside your decompiler.
• Gepetto - IDA plugin that queries language models to explain and rename decompiled functions.
• HexStrike AI - MCP server that gives agents a large toolkit of offensive security tools for recon, scanning, and exploitation.
• IDA Pro MCP - MCP server exposing IDA Pro decompilation, cross-references, and type inference to coding agents.
• LLM4Decompile - Open models and pipeline for decompiling binary code into readable C.
• ReVa - Ghidra extension providing an MCP server for agent-driven reverse engineering.

Agent Sandboxes

Isolation for running agent-generated code and untrusted targets.

• E2B - Sandboxed cloud environments purpose-built for running agent-generated code.
• Firecracker - AWS's minimal microVM monitor, a common hardware-isolation primitive for running untrusted code.
• gVisor - Google's application kernel that intercepts syscalls to sandbox untrusted workloads without a full VM.
• Microsandbox - Local-first microVM runtime for executing untrusted agent code.
• Sandbox Runtime - Anthropic's lightweight tool for enforcing filesystem and network restrictions on agents.

Benchmarks and Evals

Task sets and ground truth for measuring whether a harness actually works.

• ADR - Uber's agent detection and response system, including a benchmark of 300+ agent-attack tasks across 133 MCP servers.
• AIRTBench - Dreadnode's benchmark measuring autonomous AI red-teaming capability.
• ARVO - Atlas of reproducible open-source vulnerabilities, used as ground truth for patching agents.
• CVE-GENIE - Boston University multi-agent framework that reconstructs a vulnerable environment from a CVE entry and produces a verifiable exploit, reproducing 428 of 841 CVEs published in 2024-2025 at an average of $2.77 each.
• Cybench - Framework for evaluating language model agents on 40 professional CTF tasks.
• CyberGym - Berkeley's large-scale benchmark evaluating agents on real-world vulnerability reproduction.
• HackSynth - Autonomous pentesting agent released with two CTF benchmark sets of 200 challenges drawn from PicoCTF and OverTheWire.
• Linux Privilege-Escalation Benchmark - Reproducible local privilege-escalation scenarios from the hackingBuddyGPT team, used as the ground truth in their published agent evaluations.
• NYU CTF Bench - D-CIPHER and baseline agents for the NYU CTF benchmark.

Readings

Writing that has shaped how these harnesses get built. Same bar as the tools: results, not takes.

• AI Pentesting Agents in 2026 - Field survey of 39 open-source pentesting agents across six architecture patterns, aggregating eight benchmarks into one finding: agents solve 87% of one-day CVEs handed a description but 13% of realistic ones, and multi-agent designs beat monolithic ones 4.3x. Makes the case that stars are not a quality signal and peer-reviewed benchmarks are.
• Build Your Own Vulnerability Harness - How Cloudflare grew a single audit skill into a fleet scanner across 128 repos in six weeks: externalized state in SQLite for resumability, each agent held under 25% of its context window, one model hunting and a different model judging, and deduplication given its own agents. Reports that a single skill run finds only about half the bugs of repeated runs.
• Defend Against Frontier Cyber Models - Cloudflare's defensive architecture for a world where models find, chain, and prove exploits faster than teams can patch: ML attack scoring ahead of signature rules, positive-security API schema validation, and containment as the layer that matters once a bug lands. The blue-team counterpart to the harness posts.
• Patterns and Problems in Emerging Multiagent Systems - Anthropic research on coordination failures and swarm behavior among agents, including a vulnerability-discovery experiment where a coordinated swarm found 266 vulnerabilities across 15 open-source projects against 21 for independent parallel agents.
• Project Glasswing: What Mythos Showed Us - What Cloudflare saw pointing a frontier security model at more than fifty of its own repositories, and the four lessons that pushed it toward a harness: scope each task tightly, put a second agent in deliberate disagreement with the first, ask "is this buggy" and "can an attacker reach it" separately, and fan out then deduplicate. Notes that a single agent session covers roughly a tenth of a percent of a 100,000-line repository before compaction starts discarding findings.
• Towards Cybersecurity SuperIntelligence: What's the Best Harness for Cybersecurity? - Benchmarks five agent scaffolds on 33 cybersecurity challenges and finds no single harness wins; a blackboard architecture combining structurally diverse scaffolds covers 57.6% of problems against 45.5% for the best individual scaffold.
• Turning LLM Memory into Program Analysis - Reframes long-running vulnerability research as a Datalog problem so the model handles fuzzy fact extraction while a database does deterministic reasoning and auto-invalidates conclusions when premises change, cutting context 38x and beating full-context prompting on adversarial memory benchmarks.