jddavenportOpen

Judge Writer — Data & AI agent for Claude Code

Data & AI community

Writes a binary LLM-as-judge prompt for exactly one failure mode, from a definition and a handful of human-labeled examples, and sets up its validation against held-out human labels.

How to install Judge Writer

Installs to ~/.claude/agents/jddavenportopen-agent-evals-starter-judge-writer.md

Terminal
mkdir -p ~/.claude/agents && curl -fsSL https://raw.githubusercontent.com/jddavenportOpen/agent-evals-starter/HEAD/.claude/agents/judge-writer.md -o ~/.claude/agents/jddavenportopen-agent-evals-starter-judge-writer.md

Restart Claude Code, or start a new session, for it to be picked up.

What Judge Writer does


name: judge-writer description: Writes a binary LLM-as-judge prompt for exactly one failure mode, from a definition and a handful of human-labeled examples, and sets up its validation against held-out human labels. Use when a failure mode can't be checked with code. Refuses to label the validation data itself. tools: Read, Grep, Glob, Write, Bash

You write LLM judges that a team can trust because they were checked against people.

Inputs you need before writing anything: the failure mod

Alternatives in Data & AI

  • Gn Voice Judge — gn-voice 윤문 결과물의 문체 품질을 판정하는 심사 에이전트 93 ★
  • Leptos Designer — Use this agent when designing or implementing UI components for ccboard's web frontend (Leptos WASM) 92 ★
  • Analytics Analyst — Invoke when the user needs help with marketing measurement, KPI definition, dashboard design, attribution mode 64 ★

Full documentation available on GitHub

View Source Repository