Red First Review Skill

red-first-review

**A test that has never been shown failing is a claim, not a check.**

This is a Claude Code skill plus a small standalone tool for reviewing someone else's pull request by measuring the one thing you cannot get by reading it: *which of the guards this PR adds would survive being deleted?*

You neutralise each guard in turn, run the project's own test suite, and record what nothing notices. Survivors are the finding. Everything else in the review — the suggested test, the argument, the ranking — hangs off that table.

Standard library only. Python 3.8+. Nothing to install.

python3 mutmatrix.py \
  --repo . \
  --test "pytest tests/unit -q -p no:randomly" \
  --file src/pkg/api.py \
  --call-identity redact_secrets

What that produces, from the hand-rolled run this repository was built out of ([google/adk-python#6957](https://github.com/google/adk-python/pull/6957), a PR redacting OAuth secrets at eight call sites):

mutant	verdict	killed by
/run	SURVIVED	nothing
/run_sse	killed	test_agent_run_sse_redacts_oauth2_client_secret
/run_live (websocket)	SURVIVED	nothing
_redacted_session_response	killed	test_get_session_redacts_oauth2_client_secret
_redacted_sessions_response (list)	SURVIVED	nothing
dev get_eval_result_legacy	SURVIVED	nothing
dev get_eval	killed	test_get_eval_redacts_oauth2_client_secret
dev get_eval_result	SURVIVED	nothing

The PR's title named three endpoints. One of them was actually guarded by a test. Reverting the `/run` line alone left the entire suite green.

---

Why it refuses to guess

Most of this tool is not the mutation loop — that part is twenty lines. Most of it is the checks that stop it printing a table it cannot stand behind.

gate	what it catches	exit
baseline	suite already red → nothing is attributable to the PR	3
canary	t