Vox Agent

vox-agent

A customer-support AI agent with **inline response evaluation**, hallucination detection, and automatic retry/fallback. Built with Claude Sonnet + Claude Haiku (as judge), FastAPI, and Postgres.

---

What it does

vox-agent is a FastAPI chat service where every agent response is evaluated inline by a second model before reaching the user. Claude Sonnet generates the response; Claude Haiku scores it against a rubric covering relevance, groundedness, and hallucination risk. Responses that fail get one targeted retry with a failure-specific correction; unrecoverable failures return a safe fallback message with a lookup ID. Every turn and every evaluation is persisted to Postgres for historical analysis.

Why it exists

Most LLM demos skip the hardest part of running agents in production: **knowing when the agent is wrong**. Post-hoc eval runs on sampled traffic miss live failures; system-prompt engineering only goes so far. vox-agent treats evaluation as a first-class request-time concern, with the cost shape (Haiku as judge) that makes evaluating every turn affordable in real-world deployments.

Architecture

┌──────────────┐
│ POST /chat   │
└──────┬───────┘
       ▼
┌────────────────────────────────────────┐
│ agent.chat()                           │
│  1. Load history (memory.py)           │
│  2. Generate (Claude Sonnet)           │
│  3. Evaluate:                          │
│     ├─ Heuristics (sync, free)         │
│     └─ LLM judge (Claude Haiku, JSON)  │
│  4. If retry: targeted correction → re-gen + re-eval
│  5. If escalate: hardcoded fallback    │
│  6. Persist turn + evaluation          │
└──────┬─────────────────────────────────┘
       ▼
┌──────────────┐
│   Postgres   │  turns + evaluations (asyncpg)
└──────────────┘

**Two-layer evaluator.** Heuristics (length, refusal detection, hedge density, off-topic vocabulary) run first — deterministic, free, short-circuit the judge when a response is obviously bad. The LLM jud