Prompt Injection Benchmark

prompt-injection-benchmark

[](https://github.com/jkelly-dev1/prompt-injection-benchmark/actions/workflows/ci.yml)

A reproducible attack and defense measurement suite, built as a **personal learning project**, that measures which prompt-injection defenses actually work and publishes the ones that do not. Fifty two synthetic payloads across five delivery channels and nine attack classes are run against thirteen independently toggleable defense configurations, three times each, and every rate is reported with a bootstrap confidence interval next to the noise floor it has to clear.

The headline offline result is a defense that fails. Against the deterministic mock agent the entire prompt-level family, delimiter fencing and instruction hierarchy and provenance tagging, produces an improvement that sits **inside its own variance** and is reported as NOT SHOWN. The controls with a demonstrated effect there are the structural ones and the output-level guard, and all of them work the same way: by making it not matter that the agent was fooled rather than by preventing it.

**Two full sweeps against real models then reordered that table, and the disagreement is the most useful thing in this repository.** Against `gpt-5.6-terra` the prompt-level family produced the LARGEST measured reductions and cleared its noise floor, which is the opposite of the offline verdict. Against `claude-opus-5` nothing cleared the floor at all, because that model complied 27 times in 2,028 trials and there was almost nothing left to reduce. Same corpus, same defenses, same code, three different answers. See [Real model results](#real-model-results); the reason a mock cannot stand in for a model is limit 1.

The rule this repo follows: no claim without a test. The table below maps each claim in this README to the test that enforces it.

The problem it addresses

Injection write-ups report a blo