Agent Arena — AI skill for Claude Code
Open-source evaluation framework for LLM agents.
How to install Agent Arena
This entry records only its repository, not the path inside it, so there is no
exact command to give. Open RitikPatill/agent-arena and copy the folder into
~/.claude/skills/, or the file into ~/.claude/agents/.
What Agent Arena does
Open-source evaluation framework for LLM agents. Run head-to-head A/B tests, score with LLM-as-judge rubrics, and visualize results in a Streamlit dashboard. Model-agnostic, self-hosted, zero external infrastructure.
Alternatives in AI
- Agor — Orchestrate Claude Code, Codex, and Gemini sessions on a multiplayer canvas 1.2k ★
- Tokentap — Intercept LLM API traffic and visualize token usage in a real-time terminal dashboard 763 ★
- Claudish CLI — for external AI models) 679 ★
README
AgentArena
**Video walkthrough:** https://youtu.be/G1DcT4BMSz4 **60-second overview:** https://youtu.be/31n8B-MZrkM
Pit two agent configurations against each other on real tasks, score with rubric-based LLM-as-judge, and visualize win rates + traces in a live dashboard.
[](https://github.com/RitikPatill/agent-arena/actions/workflows/ci.yml)  

What it is
AgentArena is a self-hostable evaluation harness for LLM agents. You define an **agent config** (system prompt, model, tools, temperature), a **task suite** (input cases with optional reference answers stored as YAML), and a **rubric** (weighted criteria such as correctness, tool-use efficiency, safety, and format). AgentArena runs one or more configs against every task, invokes a separate judge model to score each output per criterion, and aggregates results into head-to-head win rates backed by bootstrap confidence intervals.
The workflow maps to a question agent engineers ask every day: is version B actually better than A, and on which tasks does it fall short? Everything runs locally against your own API key — no external services, no data leaves your machine, and all runs, spans, and judgements land in a local SQLite file you can query directly.
Quickstart
git clone https://github.com/RitikPatill/agent-arena.git
cd agent-arena
pip install -e ".[dev]"
export ANTHROPIC_API_KEY=sk-ant-...
# Verify the install — all LLM calls are mocked, no key needed
pytest
# Run the demo: 2 configs × 3 tasks, prints a win-rate matrix
python examples/quickstart.py
For the full visual experience:
# Termi
Related Skills
Harness Arena
Blind, LMSYS-style arena for comparing AI coding agent harnesses (Claude Code, Codex CLI, OpenClaw, Hermes, On
Harness Arena Fe
Blind, LMSYS-style arena for comparing AI coding agent harnesses (Claude Code, Codex CLI, OpenClaw, Hermes, On
RivalCode
Open source profiler, replay debugger and benchmark arena for AI coding agents. Compare Codex, Claude Code, Ge
Decisions Judge MCP
Typed decisions for AI agents as an MCP tool: yes/no probability (noul), choice, and score in one fast request
Persona Portability Benchmark
One persona, one frozen memory, N models: how much of an agent's character survives a model swap? Harness + bl
Agent Starter Kit
A minimal, working Claude-powered AI agent, structured to stay maintainable as tools, memory, APIs, and tests
Related Agents
Streamlit Survey UI Builder
Use this agent when you need to create or modify a Streamlit-based frontend interface for an LLM survey pipeli
Bm Eval Planner
Designs a structured evaluation strategy for an AI phase. Identifies critical failure modes, selects eval dime
Gsd Eval Planner
Designs a structured evaluation strategy for an AI phase. Identifies critical failure modes, selects eval dime