aaronjmars

Headless Harness Bench — Security skill for Claude Code

Security community

Benchmark of 6 coding-agent harnesses (omp/pi/fx/opencode/dsh/crush) as headless agent loops driven by a control plane.

How to install Headless Harness Bench

This entry records only its repository, not the path inside it, so there is no exact command to give. Open aaronjmars/headless-harness-bench and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Headless Harness Bench does

Benchmark of 6 coding-agent harnesses (omp/pi/fx/opencode/dsh/crush) as headless agent loops driven by a control plane. Two tiers: static source audit + live runs.

Alternatives in Security

OpenTag — Open-source, channel-native agent gateway for Slack 499 ★
Audit Harnesses — Audit AI Harness Consistency 150 ★
Code Audit — Adversarial code review tree 123 ★

README

headless-harness-bench

Benchmark comparing 8 coding-agent harnesses for the role of a **headless agent loop driven by a control plane**: another program drives it as a child and parses, scopes, and publishes its output. Scoring axis is *orchestratability by a control plane*, not human-at-a-keyboard UX.

Harnesses: **omp** (Oh My Pi), **pi**, **fx** (Vercel), **opencode** (SST), **dsh** (DeepSeek Harness), **crush** (Charmbracelet), **flue** (Astro), **eve** (Vercel).

**flue + eve are frameworks, not drop-in CLIs.** Unlike the first six (invoke a binary, pass flags), a control plane must first author + scaffold a TypeScript agent project, then drive `flue run` / `eve invoke`. They are scored on the same headless-driver axis, with the scaffold held constant across both.

TL;DR

Deploy-today rank (control-plane role): OMP > opencode > Pi.
Capability rank: OMP ~= dsh > opencode > Pi > fx > Crush.
flue + eve (frameworks) land mid-pack on Tier-1 (54.9 / 52.1 of 81): both bring best-in-class secret hygiene (env-scrub / sandbox isolation, no $HOME MCP leak) but lose on the driver contract - no per-turn JSONL on stdout, no per-run CLI overrides, no Claude-sub OAuth. Neither is a deploy-today swap; both are worth tracking.
All 8 completed the golden task once wired. The order is about auth fit, structured output, isolation, maturity, and cost - not raw ability.
Full reasoning + file:line citations in BENCHMARK.md; this README carries every finding as tables.

1. Identity + distribution

	omp	pi	fx	opencode	dsh	crush	flue	eve
Language / runtime	TS + Rust / Bun	TS / Node 22+	Zig (native 6 MiB)	TS / Bun	TS / Node 22+ (+Py wheel)	Go (native)	TS / Node 22+ (Vite)	TS / Node (Nitro)
License	MIT	MIT	Apache-2.0	MIT	MIT	FSL-1.1-MIT	Apache-2.0	Apache-2.0
OSI-open?	yes	yes	yes	yes	yes	no (MIT after 2y)	yes