Agent Reliability Harness — Git skill for Claude Code
Evals, tracing and CI regression gates for tool-using agents: tool-selection, final-state, cost, latency and step metrics, pass^k reliability, fault injection, and a PR gate that fails when agent qual.
How to install Agent Reliability Harness
This entry records only its repository, not the path inside it, so there is no
exact command to give. Open djmoore-projects/agent-reliability-harness and copy the folder into
~/.claude/skills/, or the file into ~/.claude/agents/.
What Agent Reliability Harness does
Evals, tracing and CI regression gates for tool-using agents: tool-selection, final-state, cost, latency and step metrics, pass^k reliability, fault injection, and a PR gate that fails when agent quality drops.
Alternatives in Git
- Cc Usage — by ryoppippi - Handy CLI tool for managing and analyzing Claude Code usage, based on analyzing local Claude Co 11.7k ★
- Skill Copilot Provider — GitHub Copilot CLI as optional zero-cost provider via copilot -p programmatic mode 2.8k ★
- Droidconkotlin — by touchlab - Delivers comprehensive Gradle commands for cross-platform Kotlin Multiplatform development with 1.1k ★
README
agent-reliability-harness
[](https://github.com/djmoore-projects/agent-reliability-harness/actions/workflows/ci.yml) [](LICENSE) [](https://www.python.org/downloads/)
**Evals, tracing and CI regression gates for tool-using agents.** Point it at a task suite and it answers the question every agent project eventually gets asked — *"how do we know it works?"* — with numbers: did the agent pick the right tools, is the final state (and the real-world side effect) correct, what did each task cost, how long did it take, how many steps did it need, and does it still pass when an API falls over halfway through. Then it runs as a CI check that fails the PR when any of that gets worse.
It's the agent-shaped sequel to [fintech-rag-eval](https://github.com/djmoore-projects/fintech-rag-eval): that project grades RAG *answers*; this one grades agent *runs*. Same philosophy — typed contracts, structured judge outputs, scores computed in Python rather than reported by a model.
First real target: b2b-account-pipeline
The harness was built against [b2b-account-pipeline](https://github.com/djmoore-projects/b2b-account-pipeline), a LangGraph agent that researches a company with Claude + Tavily, writes a 3-step outreach sequence and syncs a CRM record. Its README says it "works" and shows a clean run. The 11-task suite ([`evals/suite.yaml`](https://github.com/djmoore-projects/b2b-account-pipeline/blob/agent-reliability-evals/evals/suite.yaml)) checks only the contract the pipeline states for itself, and the checks were frozen before the baseline ran.
**Baseline: 17% task success** (3/18 clean trials, 95% CI 6–39%). **All 13 clean happy-path trials failed:**
[Name]/[Your Name]placeholders in outreach that is meant to be sent as-is:
Related Skills
Senior Engineering Agent
Claude Code plugin: a Senior Software Engineer & IT Architect agent with deterministic safety hooks — asks bef
Sitemend
SEO regression tests for your repo. Fails the PR when a change breaks crawling or indexing (preview URLs in ca
Final Pass
look at the git branch we are on to see what we have been working on
Slice Review
Run a multi-agent code review over a local diff (no PR or remote required) and report findings by severity wit
Fix Faults
Analyze all fault reports, diagnose root causes using debug logs, research fixes, implement in parallel, and c
Dora
Measure, benchmark, instrument, and debug DORA metrics (Deployment Frequency, Lead Time for Changes, Change Fa
Related Agents
Evolve Bug Reproduction
Bug reproduction agent for the Evolve Loop (Evaluate archetype). The advisor INSERTS this phase on bugfix cycl
Dst
Deterministic Simulation Testing methodology, harnesses, fault injection, and shadow state
Qe Chaos Engineer
Resilience testing with controlled fault injection and blast radius management