Phoenix Claude Code — Development skill for Claude Code

phoenix-claude-code

Monitor, evaluate, and improve your Claude Code usage with [Arize Phoenix](https://github.com/Arize-ai/phoenix).

TRACES --> MONITOR --> ANNOTATE --> JUDGE --> DATASET --> CI --> IMPROVE --> repeat

Most people use Claude Code without knowing what's actually happening: which models are called, how many tokens are spent, what errors occur, or whether their prompts are effective. This repo gives you full observability and an eval-driven improvement loop.

What You Get

• Full trace capture — Every LLM call from Claude Code routed through a LiteLLM proxy into Phoenix
• Usage analysis — Parse ~/.claude/history.jsonl for prompt patterns, project breakdown, monthly trends
• 4 automated judges — Evaluate prompt quality, secret hygiene, session discipline, and topic coherence
• Error analysis workflow — Jupyter notebook for manual session review (the highest-ROI activity in AI evals)
• Golden dataset builder — Export sessions as annotatable CSV, build your eval dataset from real failures

Quickstart

git clone https://github.com/rachittshah/phoenix-claude-code.git
cd phoenix-claude-code
cp .env.example .env
# Edit .env: add your ANTHROPIC_API_KEY

# Start Phoenix + LiteLLM proxy
docker compose up -d

# Install Python package
pip install -e .

# Install Phoenix CLI
npm install -g @arizeai/phoenix-cli

Configure Claude Code to route through the proxy:

# In your shell profile (~/.zshrc)
export ANTHROPIC_BASE_URL=http://localhost:4000
export PHOENIX_HOST=http://localhost:6006
export PHOENIX_PROJECT=claude-code

Restart Claude Code. All LLM calls now flow through LiteLLM → Phoenix.

Open Phoenix UI at **http://localhost:6006** to see traces.

Usage

Analyze Your Prompt History

python analysis/history_analyzer.py

Output: monthly breakdown, project usage, prompt length trends, command frequency.

Analyze Phoenix Traces

python analysis/trace_analyzer.py

Output: