DYAI2025

Bench Watch — AI skill for Claude Code

AI community

Launch or attach to a Plumbline benchmark slice, poll it to completion, and emit the canonical anti-Goodhart per-arm-model summary (catch-rate AND cry-wolf).

How to install Bench Watch

Installs to ~/.claude/commands/dyai2025-plumbline-bench-watch.md

Terminal

mkdir -p ~/.claude/commands && curl -fsSL https://raw.githubusercontent.com/DYAI2025/Plumbline/HEAD/.claude/commands/bench-watch.md -o ~/.claude/commands/dyai2025-plumbline-bench-watch.md

Restart Claude Code, or start a new session, for it to be picked up.

What Bench Watch does

description: Launch or attach to a Plumbline benchmark slice, poll it to completion, and emit the canonical anti-Goodhart per-arm-model summary (catch-rate AND cry-wolf). For the run-slice / wait-for-it / show-the-per-arm-model-summary loop. allowed-tools: Bash, Read, Glob, Grep, Task

Context

Plumbline's identity is *measure, don't assert*: every claim earns a deterministic bench. Bench runs are long, and the manual loop "wait for the slice to finish → show the per-arm-model summary"

Alternatives in AI

Ralph For Claude Code — by Frank Bria - An autonomous AI development framework that enables Claude Code to work iteratively on project 8k ★
Minuet Ai.nvim — Code completion as-you-type from popular LLMs including OpenAI, Gemini, Claude, Ollama 1.1k ★
Cwc Long Running Agents — Claude Code's built-in /goal command gives you a generator/evaluator loop out of the box: set a completion con 668 ★