Shellbench — AI skill for Claude Code
The agent benchmark that scores the full stack — harness, config, and model — not just the LLM.
How to install Shellbench
This entry records only its repository, not the path inside it, so there is no
exact command to give. Open steipete/shellbench and copy the folder into
~/.claude/skills/, or the file into ~/.claude/agents/.
What Shellbench does
The agent benchmark that scores the full stack — harness, config, and model — not just the LLM. Trace-based scoring, reliability metrics, configuration diagnostics.
Alternatives in AI
- Ccstatusline — by sirmalloc - A highly customizable status line formatter for Claude Code CLI that displays model info, git b 5.5k ★
- Codebase Digest — 🗜️ Codebase-digest is your AI-friendly codebase packer and analyzer 375 ★
- Lighthouse MCP — Allows AI assistants such as Amp/Cline/Cursor/Claude Code/Codex/GitHub Copilot to use Google's lighthouse tool 214 ★
README
title: ClawBench emoji: 🦞 colorFrom: red colorTo: yellow sdk: docker app_port: 7860 pinned: true license: mit
ClawBench
**Rigorous agent evaluation. Signal-curated tasks. Dynamical-systems diagnostics.**
[](https://www.python.org/downloads/) [](LICENSE) [](tasks-public/) [](#3-dynamical-systems-diagnostics-how-agents-fail-not-just-whether) [](https://huggingface.co/datasets/openclaw/clawbench-results)
What's new in Core v1 (2026-04-20)
A reproducibility-first public release of the benchmark, informed by a full 8-model, 1,080-run sweep audit and five new methodology layers that most agent benchmarks simply don't have:
| Innovation | What it means | Why it matters |
|---|---|---|
| Signal-curated task set | 19 tasks selected from 40-task dev pool by greedy SNR-preserving elimination | Drops tasks where seed noise exceeds capability signal (21 such tasks exist in the raw 40) |
| Variance decomposition | Measures and reports seed-noise vs capability-signal ratio per task | 47% of 40-task variance is seed noise — we quantify it; most benchmarks hide it |
| Dynamical-systems diagnostics | Per-run regime classification (trapped / limit-cycle / diffusive / mixed) | Reveals how agents fail, not just whether. Inspired by Markov-kernel / attractor-basin framework |
| Constraint Index C(q) | Principled task-weighting via participation ratio + entropy + Bayes prediction | Distinguishes "everyone converges" fr |
Related Skills
Agent Eval Hub
Multi-provider + cross-device reliability harness for AI agents. Runs the same suite across Claude, OpenAI, Ge
Full Stack Chat Application With Multi Platform Clients AI Agent Integration
A Claude Code-style AI agent harness in Python. Uses Ollama + native LLM tool-calling to read/write files and
ExoQuery Bench
Open benchmark for LLM assistants that turn plain-English astronomy questions into ADQL. Scores Claude, OpenAI
Benchboard
Local-first benchmark for AI coding agents: one command runs an agent (OpenCode, Claude Code or any CLI) on a
AI CLI Observability
Self-hosted OpenTelemetry observability stack for Codex, Claude Code, and Gemini CLI, with Grafana dashboards
Agent Observability Stack
Self-hostable observability for LLM agents + a Linux host (Intel iGPU/NPU + NVIDIA RTX 3090 eGPU aware): Prome
Related Agents
Harness Optimizer
Analyze and improve the local agent harness configuration for reliability, cost, and throughput.
Kavach Config
KAVACH infrastructure/config/ops-security specialist. Audits security headers, debug/verbose in prod, stack-tr
Eval Auditor
Audits an evaluation setup (benchmark, A/B test, or model comparison) for methodology errors that would invali
