prime-radiant-inc

Superpowers Evals — Testing skill for Claude Code

Testing community

Behavioral eval lab (Quorum) for the superpowers project that drives real coding-agent CLIs (Claude, Codex, Gemini, Kimi, and more) through a QA agent and grades them on workflow compliance against sc.

How to install Superpowers Evals

This entry records only its repository, not the path inside it, so there is no exact command to give. Open prime-radiant-inc/superpowers-evals and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Superpowers Evals does

Alternatives in Testing

Spec Kitty — Spec-Driven Development for serious software developers 1.6k ★
Next Evals OSS — Evals for Next.js up to 15.5.6 to test AI model competency at Next.js 310 ★
Design QA — Visual QA on an HTML/CSS implementation — checks responsive behavior at mobile/tablet/desktop, design token co 309 ★

README

Superpowers Evals

Behavioral eval lab for [superpowers](https://github.com/obra/superpowers). **Quorum** drives real coding-agent CLIs (Claude, Codex, Antigravity, Gemini, Hermes, Kimi, OpenCode, Pi, and Copilot) through a Gauntlet QA agent and grades them against scenario acceptance criteria plus deterministic post-checks.

Code, CLI, paths, and inline prose all use lowercase `quorum`; the capitalized form `Quorum` appears in headings and the actor table.

This is not a generic benchmark suite. It is an eval lab for workflow compliance: skill triggering, worktree behavior, subagent coordination, verification reflexes, review quality, and cost-shaping patterns.

Safety Model

quorum has two very different execution modes:

Static/unit checks are safe for public CI. They run biome, tsc, and bun test. They do not call model APIs and do not launch agent CLIs.
Live evals are trusted-maintainer operations. They launch Claude Code, Codex CLI, Antigravity CLI, Gemini CLI, Hermes CLI, Kimi Code, OpenCode CLI, Pi CLI, or Copilot CLI in permissive modes and collect raw transcripts, tool calls, filesystem state, and session logs.

Public CI must stay on the static/unit side of that line. Never add API keys, live `quorum run …` invocations, or dangerous-mode agent launches to public CI.

Live Eval Risk

Live evals run the Coding-Agent under test with broad execution power:

Claude uses --dangerously-skip-permissions.
Codex uses --dangerously-bypass-approvals-and-sandbox.
Antigravity uses --dangerously-skip-permissions and authenticates with the per-run seeded copy of the operator's agy OAuth token file.
Gemini uses --skip-trust --approval-mode=yolo; API-key auth is default, with opt-in OAuth auth for trusted local runs.
Hermes uses --yolo and API-key auth in the run-local .env.
Kimi uses --yolo.
OpenCode uses --dangerously-skip-permissions.
Pi uses explicit tool allowlists and API-key auth in a run-local con