jddavenportOpen

Eval Task Auditor — Project Management agent for Claude Code

Project Management community

Audits eval task files for the problems that most often break real benchmarks: ambiguous success criteria, wrong reference solutions, graders that are too strict or too loose, and shortcuts that pass.

How to install Eval Task Auditor

Installs to ~/.claude/agents/jddavenportopen-agent-evals-starter-eval-task-auditor.md

Terminal
mkdir -p ~/.claude/agents && curl -fsSL https://raw.githubusercontent.com/jddavenportOpen/agent-evals-starter/HEAD/.claude/agents/eval-task-auditor.md -o ~/.claude/agents/jddavenportopen-agent-evals-starter-eval-task-auditor.md

Restart Claude Code, or start a new session, for it to be picked up.

What Eval Task Auditor does


name: eval-task-auditor description: Audits eval task files for the problems that most often break real benchmarks: ambiguous success criteria, wrong reference solutions, graders that are too strict or too loose, and shortcuts that pass without doing the work. Use after drafting or changing tasks, and before trusting a new suite. tools: Read, Grep, Glob, Bash

You audit eval tasks. You do not fix them; you report, with evidence, so a human decides.

For each task file you are given (or e

Alternatives in Project Management

  • Qualityqueen — Quality Assurance & Code Standards Specialist who ensures code meets project standards across any technology s 38.1k ★
  • Requirements Analyst — Transform ambiguous project ideas into concrete specifications through systematic requirements discovery and s 1.5k ★
  • Orchestrator Agent — You are a senior software architect and project coordinator 1.3k ★

Full documentation available on GitHub

View Source Repository