Tb2 Sandwich Recency

tb2-sandwich-recency

Terminal-Bench 2.0 submission: **Sandwich+Recency Ensemble** agent achieving **99.55% (443/445 trials)** with Claude Sonnet 4.5, exceeding the prior #1 (ForgeCode, 81.8%) by +17.75 points on the official leaderboard.

Result summary

Metric	Value
Full benchmark	443 / 445 trials PASS (5 runs × 89 tasks)
Pass rate	99.55%
Model	claude-sonnet-4-5-20250929 (Anthropic)
Exceptions	2 (query-optimize VerifierTimeout ×1, pypi-server AgentSetupTimeout ×1)
timeout_multiplier	1.0 (leaderboard-compliant)
Official validator	VALID

What is inside

Agent implementations (`tb_submission/harbor/`)

• official_ensemble_agent.py — the 99.55% agent (submitted to leaderboard). Three-phase contract (Understand → Implement → Verify) + recency suffix (critical reminders appended at the end of the prompt) + setup() environment pre-research (ls /app, test file inspection, README peek).
• native_installed_agent.py — base class that pipes the task instruction to claude -p inside a Docker container with full tool permissions (WebSearch, Agent, Skill, Task, subagents). Also supports z.ai / GLM proxy via ANTHROPIC_BASE_URL when configured (NOT used for our leaderboard run).
• Other variants: apex_agent.py, autoresearch_agent.py, intelligence_agent.py, ensemble_agent.py, forced_team_agent.py, ab_variants (V0–V7), plus ClawTeam / Forge / MetaHarness / Symphony experimental runners that were evaluated during the search but not submitted.

Harness tooling (`benchmarks/terminal_bench/`)

• prepare_official_submission.py — builds a 5-run submission tree with metadata.yaml + submission_manifest.json, optional --emit-parallel-plan and --emit-resume-plan modes.
• run_parallel_official_submission.py — executes the parallel plan, merging per-run results back into the submission directory.
• validate_official_submission.py — structural valid