RAG Support Agent — AI skill for Claude Code

Financial document AI: evidence-checked extraction, cited Q&A and a verified agent

[](https://github.com/Ron-Salama/rag-support-agent/actions/workflows/docker.yml)

**Stack:** Python · RAG (embeddings + Chroma vector search) · AI agent with tool calling and a verifier · structured outputs (Pydantic) · LLM evals + LLM-as-judge · FastAPI REST API · Gemini / Ollama · Docker image built and smoke-tested in GitHub Actions CI · built with Claude Code

Lending and finance workflows run on numbers buried in invoices, financial statements, loan agreements and appraisals, and a wrong number that looks right is worse than no number. This project reads 15 public financial documents and turns them into **validated structured data**: every value must come with a verbatim quote that code finds in the document and that contains the value, business rules check the numbers, and anything doubtful goes to a **human review queue** instead of downstream. On the same documents it **answers questions with page-level citations, or refuses**, and runs a tool-using **agent whose answer a second model verifies** before anyone sees it. Extraction is scored against hand labels made blind, retrieval against a small question set drafted by Claude Code; answer quality and the agent are not measured yet. It runs on a laptop for $0: local embeddings and Chroma, with the LLM on the Gemini free tier (cloud) or a local Ollama model, served by FastAPI and packaged for Docker.

Results at a glance

• Extraction: 128/131 fields (97.7%) against blind hand labels on 15 documents, 1 false fill out of 18 empty fields (baseline 126/131; one of the two points gained is a grader fix). The fixes were designed on these same 15 documents, and the 2026-10-04 run replayed 11 of them from the LLM disk cache (C26).
• Retrieval: the right page is in the top 5 for 20/24 questions (MRR@10 0.698), on a small questi