anapsousa

Evals Engineer — Data & AI agent for Claude Code

Data & AI community

Use for anything under evals/ — authoring or curating golden-set tasks, implementing the outcome/trajectory/component scorers, writing and calibrating LLM-as-judge prompts against human labels, comput.

How to install Evals Engineer

Installs to ~/.claude/agents/anapsousa-agent-reliability-lab-evals-engineer.md

Terminal
mkdir -p ~/.claude/agents && curl -fsSL https://raw.githubusercontent.com/anapsousa/agent-reliability-lab/HEAD/.claude/agents/evals-engineer.md -o ~/.claude/agents/anapsousa-agent-reliability-lab-evals-engineer.md

Restart Claude Code, or start a new session, for it to be picked up.

What Evals Engineer does


name: evals-engineer description: Use for anything under evals/ — authoring or curating golden-set tasks, implementing the outcome/trajectory/component scorers, writing and calibrating LLM-as-judge prompts against human labels, computing pass rate / cost / p95 latency, and maintaining the CI regression gate. Trigger on "add a golden task", "write a scorer", "calibrate the judge", "why did pass rate drop", "judge/human agreement", "size the golden set", or any question about how the agent is

Alternatives in Data & AI

Full documentation available on GitHub

View Source Repository