ingridtoulotte

Claudebench — Testing skill for Claude Code

Testing community

Stop arguing about prompts.

How to install Claudebench

This entry records only its repository, not the path inside it, so there is no exact command to give. Open ingridtoulotte/claudebench and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Claudebench does

Stop arguing about prompts. Measure them. A reproducible, statistically-honest benchmark harness for Claude Code workflows (CLAUDE.md, prompts, memory). Wilson CIs, significance-tested verdicts, test-based grading, record/replay. Zero deps.

Alternatives in Testing

Test Skill — Test a skill with 3 prompts to validate quality 339 ★
Test Login And Upload — End-to-end test of the CLI on the dev environment 97 ★
Smoke Test — Give me prompts to test in the F5 window that opens in C:\GameDev\iemis repo 73 ★

README

⬡ ClaudeBench

Stop arguing about prompts. Measure them.

A reproducible benchmark harness for **Claude Code** workflows — prompts, `CLAUDE.md` files, memory strategies, MCP stacks. Run the same tasks under each setup, get a score with a **confidence interval**, and find out whether the difference is real or just noise.

[![CI](https://github.com/ingridtoulotte/claudebench/actions/workflows/ci.yml/badge.svg)](https://github.com/ingridtoulotte/claudebench/actions/workflows/ci.yml) [![node](https://img.shields.io/badge/node-%E2%89%A518-58a6ff)](package.json) [![runtime deps](https://img.shields.io/badge/runtime%20deps-0-3fb950)](package.json) [![no LLM judge](https://img.shields.io/badge/grading-test--based-3fb950)](docs/METHODOLOGY.md) [![license](https://img.shields.io/badge/license-MIT-blue)](LICENSE)

**[60-second demo](#60-second-demo--no-api-key-needed)** · **[Why it's not snake oil](#why-this-isnt-snake-oil)** · **[Write a suite](#write-your-own-suite)** · **[Methodology](docs/METHODOLOGY.md)**

*"This CLAUDE.md is better."* *"This prompt saves tokens."* *"This memory setup works best."*

Cool. **Show the ClaudeBench results.**

The Claude Code ecosystem runs on confident claims and zero evidence. ClaudeBench turns "I think it's better" into "it's **+24pp** better, **p<0.001**, here's the run you can replay."

⭐ **If you've ever guessed whether a `CLAUDE.md` change actually helped, star the repo** — it's the fastest way to push for more suites.

60-second demo — no API key needed

npx github:ingridtoulotte/claudebench demo

⬡ ClaudeBench  ·  suite claude-md  ·  model claude-opus-4-8  ·  trials 90×2

╭─ RESULTS ──────────────────────────────────────────────────────────────────────╮
│     CONFIG         SCORE  PASS RATE  (95% CI)         TOKENS  LATENCY     COST │
│ ●   terse-rules       88  ███████████[░] 88% 79%–93%    7.8k     7.7s  $0.2177 │
│ ○   verbose-rules     63  ███████[█░]░░░