10x Bench Kit — AI skill for Claude Code


10x-bench-kit

Template repo for an internal AI agent benchmark. This repo is the basis for a company's **benchmark instance**, created with [10xCLI](https://github.com/przeprogramowani/10x-cli) (`10x bench-kit init`) — a separate repo that holds the tasks, the evaluation pool, the configuration, and the results, and runs agent trials against the company's product repositories in full isolation.

**The benchmark is local-first.** Attempts and evaluations run on the operator's machine (or a VPS — same thing: a long-lived host with a container engine). GitHub Actions is plain CI/CD: readiness checks and leaderboard publication — never paid execution.

Initial harness: **OpenCode** (exclusively). Measured: quality (execution guards + LLM-as-judge-with-tools), cost, and execution time.

Prerequisites

• 10xCLI — no global install needed (npx @przeprogramowani/10x-cli bench-kit init ).
• A container engine: Docker Desktop or Podman.
• Node.js >= 20.
• Provider API keys in the shell environment (e.g. OPENROUTER_API_KEY).

Three zones

The repo structure is split into zones with different owners and different behavior during `10x bench-kit update`:

Zone	Owner	On update
.bench-kit/	kit (us)	replaced wholesale (atomic)
.agents/skills/	shared	proposed diff — the company decides
tasks/, evaluation-pool/, bench.config.yaml, results/, attempts/	company	untouchable

Details of each zone's contract live in that zone's README.

Execution and evaluation are two independent processes

The core design decision (see `.bench-kit/ATTEMPT_FORMAT.md` for the full contract):

1. bench attempt executes trials in throwaway containers and produces preserved attempts — attempts///trial-N/ with attempt.json, patch.diff, agent.log, metrics.json (cost and tokens read from the trial's local OpenCode storage