RitikPatill

Agent Arena — AI skill for Claude Code

AI community

Open-source evaluation framework for LLM agents.

How to install Agent Arena

This entry records only its repository, not the path inside it, so there is no exact command to give. Open RitikPatill/agent-arena and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Agent Arena does

Open-source evaluation framework for LLM agents. Run head-to-head A/B tests, score with LLM-as-judge rubrics, and visualize results in a Streamlit dashboard. Model-agnostic, self-hosted, zero external infrastructure.

Alternatives in AI

  • Agor — Orchestrate Claude Code, Codex, and Gemini sessions on a multiplayer canvas 1.2k ★
  • Tokentap — Intercept LLM API traffic and visualize token usage in a real-time terminal dashboard 763 ★
  • Claudish CLI — for external AI models) 679 ★

README

AgentArena

**Video walkthrough:** https://youtu.be/G1DcT4BMSz4 **60-second overview:** https://youtu.be/31n8B-MZrkM

Pit two agent configurations against each other on real tasks, score with rubric-based LLM-as-judge, and visualize win rates + traces in a live dashboard.

[![CI](https://github.com/RitikPatill/agent-arena/actions/workflows/ci.yml/badge.svg)](https://github.com/RitikPatill/agent-arena/actions/workflows/ci.yml) ![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue) ![License: MIT](https://img.shields.io/badge/license-MIT-green)

![demo](docs/demo.gif)

What it is

AgentArena is a self-hostable evaluation harness for LLM agents. You define an **agent config** (system prompt, model, tools, temperature), a **task suite** (input cases with optional reference answers stored as YAML), and a **rubric** (weighted criteria such as correctness, tool-use efficiency, safety, and format). AgentArena runs one or more configs against every task, invokes a separate judge model to score each output per criterion, and aggregates results into head-to-head win rates backed by bootstrap confidence intervals.

The workflow maps to a question agent engineers ask every day: is version B actually better than A, and on which tasks does it fall short? Everything runs locally against your own API key — no external services, no data leaves your machine, and all runs, spans, and judgements land in a local SQLite file you can query directly.

Quickstart

git clone https://github.com/RitikPatill/agent-arena.git
cd agent-arena
pip install -e ".[dev]"
export ANTHROPIC_API_KEY=sk-ant-...

# Verify the install — all LLM calls are mocked, no key needed
pytest

# Run the demo: 2 configs × 3 tasks, prints a win-rate matrix
python examples/quickstart.py

For the full visual experience:

# Termi