Ralph Lab

ralph-lab

Gate-neutral **Ralph loop** driver in Python. Wraps agent CLIs (claude / codex / gemini / opencode) in a subprocess loop, runs them until a **user-supplied gate script** exits 0, and records everything as JSONL.

**Successor to [agent-loop-lab](https://github.com/dobachi/agent-loop-lab)** (archived). Full rewrite; see [Prior art](#prior-art) for the history.

Design principles

• Ralph faithful. Each iteration starts with a fresh subprocess. No shared context across iterations — context accumulation degrades quality past 100k-150k tokens (Ralph pattern, 2025).
• Gate-neutral. Any bash script that returns exit 0 / non-zero works. loop-goal, pytest, cargo test, eslint, grep -q "…" — all first class. ralph-lab doesn't care what the gate does; it just reads stdout as feedback and exit code as verdict.
• Multi-agent CLI. Switch between claude, codex, gemini, opencode without changing the loop.
• Multi-model. Run the same goal across N models to compare (--models m1,m2,m3), each in its own workspace, results in one JSONL for easy diffing.
• Observability. Every iteration records subprocess exit, duration, gate result, and stdout/stderr sizes. Iteration-level metric only (turn/tool-call metric is inside the agent CLI subprocess and not exposed).

What ralph-lab does NOT do

• No custom LLM SDK. openai-agents-python, anthropic-sdk, etc. belong to the agent CLI, not to ralph-lab. If you want to change which SDK is used, use a different agent CLI or add flags via agent_flags in the spec.
• No built-in gate. Bring your own. goals/examples/ shows several patterns.
• No multi-agent coordination. One agent CLI per run. For multi-agent workflows, orchestrate ralph-lab runs from outside.

Prior art

Based on:

• Geoffrey Huntley's Ralph loop pattern (2025)
• syuya2036/ralph-loop (bash reference)