avelikiy

AI Eval Engineer — Data & AI agent for Claude Code

Data & AI community

Builds and maintains the eval pipeline for ai-system / agent-product archetypes.

How to install AI Eval Engineer

Installs to ~/.claude/agents/avelikiy-great_cto-ai-eval-engineer.md

Terminal

mkdir -p ~/.claude/agents && curl -fsSL https://raw.githubusercontent.com/avelikiy/great_cto/HEAD/agents/ai-eval-engineer.md -o ~/.claude/agents/avelikiy-great_cto-ai-eval-engineer.md

Restart Claude Code, or start a new session, for it to be picked up.

What AI Eval Engineer does

name: ai-eval-engineer description: Builds and maintains the eval pipeline for ai-system / agent-product archetypes. Outputs tests/eval/EVAL-*.md files (golden citation, refuse-when-uncertain, output schema, prompt injection, cost-overrun, cross-user isolation). Runs regression on every prompt or model change. Detects drift. model: haiku tools: Read, Write, Edit, Bash, Glob, Grep, WebFetch, WebSearch, memory_20250929, advisor_20260301 maxTurns: 30 timeout: 600 effort: MEDIUM memory: project

Alternatives in Data & AI

LLM AI Hunter — LLM and Agentic AI vulnerability specialist 812 ★
LLM Expert — Use this agent for AI SDK integration, LLM provider configuration, prompt template management, error handling 434 ★
API Cost Scanner — Scans codebase for AI/API cost optimization opportunities including model usage, token estimates, caching stra 160 ★