AgentShield Benchmark

The first head-to-head benchmark of commercial agent protection providers.

AgentShield is an open, reproducible benchmark suite that evaluates how well commercial AI agent security products defend against real-world attacks — and how much they cost you in latency, false positives, and dollars.

Disclosure

This benchmark is maintained by the team behind [Agent Guard](https://agentguard.co/). To ensure credibility, Agent Guard's results were obtained using our [Commit-Reveal Integrity Protocol](src/protocol/README.md) — a commit-reveal scheme with Ed25519 signatures that allows proprietary solutions to participate without revealing their implementation, while cryptographically proving result integrity. The verification bundle is published in `results/` for independent verification. Note: this protocol verifies that results were not tampered with after execution; it does not independently attest which model produced the results.

The test corpus, scoring methodology, and all adapter code are fully open source and auditable. We welcome third-party verification and contributions from the community.

If you believe any aspect of the methodology unfairly advantages or disadvantages a particular provider, please [open an issue](../../issues).

Current Status

This benchmark currently includes tested results for **7 providers** across ML models, SaaS APIs, and pattern-based scanners with **537 test cases** across 8 categories. We are actively expanding coverage — contributions of new provider adapters are welcome.

Latest Results

Provider	Score	PI	Jailbreak	Data Exfil	Tool Abuse	Over-Refusal	Multi-Agent	Provenance	P50 (ms)
AgentGuard²	98.4	98.5%	97.8%	100.0%	100.0%	100.0%	100.0%	85.0%	1
*