jchilcher-godaddy

Claude Benchmark — Development skill for Claude Code

Development community

CLI tool for benchmarking CLAUDE.md configurations against standardized coding tasks.

How to install Claude Benchmark

This entry records only its repository, not the path inside it, so there is no exact command to give. Open jchilcher-godaddy/claude-benchmark and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Claude Benchmark does

CLI tool for benchmarking CLAUDE.md configurations against standardized coding tasks.

Alternatives in Development

OpenAI Codex CLI — (Rust implementation) 67.8k ★
Vibe Kanban — Get 10X more out of Claude Code, Gemini CLI, Codex, Amp and other coding agents 23.5k ★
Ralph Loop — Autonomous coding loop — works through tasks, commits, repeats 14k ★

README

claude-benchmark

A CLI tool for benchmarking CLAUDE.md configurations against standardized coding tasks. Test how well your CLAUDE.md instructions guide AI code generation across bug fixes, code generation, refactoring, and instruction-following scenarios.

Prerequisites

Python 3.11+
Claude Code CLI installed and configured
ANTHROPIC_API_KEY environment variable set

Cross-language scoring tools (optional)

Required for scoring Go, JavaScript, and C# tasks. Python tasks use pytest/ruff/radon which are installed automatically.

**Go:**

# Install Go: https://go.dev/doc/install
brew install golangci-lint                                       # linter
go install github.com/fzipp/gocyclo/cmd/gocyclo@latest          # complexity
export PATH="$HOME/go/bin:$PATH"                                 # add to ~/.zshrc

**JavaScript:**

npm install -g eslint jest

**C#:**

# Install .NET SDK: https://dotnet.microsoft.com/download
# dotnet test, dotnet format are included with the SDK

Installation

pip install .

For development:

pip install -e .

Quick Start

Run a benchmark against your CLAUDE.md file:

claude-benchmark run --claudemd path/to/your/CLAUDE.md

Generate a report from results:

claude-benchmark report results/

CLI Commands

Command	Description
`run`	Execute benchmark tasks against a CLAUDE.md configuration
`experiment`	Run a multi-variant experiment from a TOML configuration file
`report`	Generate an HTML report from one or more result sets
`compare`	Compare results across cataloged runs with statistical analysis
`rescore`	Re-run scoring on existing results (e.g., after judge recalibration)
`calibrate`	Calibrate and compare LLM judge models
`catalog`	View and filter the catalog of completed benchmark runs
`intake`	Import external