Claude Code Routing Benchmark — AI skill for Claude Code

benchmark-claude-code

Give the same coding task to differently configured copies of Claude Code, then compare what each one did.

Every profile gets a fresh git worktree at the same commit and the same prompt. The only thing that differs is which model does the work. The result is a single HTML page: timings, cost, tool usage, and the diff each agent produced.

**[Read the latest results](https://kolezka.github.io/claude-code-routing-benchmark/)**

Reading the results honestly!

This is one task, run once, on one machine. It shows how these setups behaved on a specific bug. It is not a model ranking.

Wall time includes your network and machine load. Fewer tool calls is not automatically better work: an agent that reads less code can still land on a worse fix. Read the diffs, not just the counters.

How a profile is configured

Claude Code is Anthropic's terminal coding agent. A profile in `config.yml` says how to start it:

profiles:
  - id: anthropic
    label: Anthropic only
    command: ["ccr", "CC - Anthropic Only", "cli", "--"]
    env:
      CLAUDE_CONFIG_DIR: ./.profile/claude-code-anthropic

`command` is the argv prefix. The harness appends its own flags and the prompt. `env` is added to the child process, and `./` paths resolve from the repo root.

That gives you two ways to vary the model:

• With a proxy. CCR (claude-code-router) runs locally, Claude Code points at it, and it forwards each request to whichever provider you configured. The profiles committed here use it. CCR is a third party project, not shipped with this repo.
• Without one. Use command: ["claude"] and give each profile its own CLAUDE_CONFIG_DIR, holding a settings.json that selects the model you want.

The first element of `command` is looked up on PATH. Override it with `CCR_BIN` or `CLAUDE_BIN`.

Usage

bun install
bun benchmark.ts setup       # clone the target repo at the pinned ref
bun b