Myrag — Data skill for Claude Code

MyRAG: cited answers over messy government documents, with real evals

MyRAG is a retrieval-augmented generation (RAG) system over five public government documents on AI risk and cybersecurity: three NIST PDFs and two government Word templates. You ask a question in plain English. It finds the relevant passages, and a language model answers **only from those passages**, citing each claim with a link to the exact sentence, section, and page it came from. When the documents don't cover a question, it says so instead of guessing.

**It's free to run.** By default everything runs on your own computer: a small open-weight model (Qwen 2.5 3B via [Ollama](https://ollama.com)) writes the answers, and local models handle search. No account, API key, or payment is needed. If you have an Anthropic API key, one setting switches answering to **Claude** for higher-quality answers and guaranteed-verbatim citations.

The repo also includes the evaluation that shows how well it works: a labeled test set with exact source quotes, search metrics (recall@k, MRR, nDCG), an LLM-graded check of answer faithfulness and correctness, and a web dashboard for exploring the results.

Ask screen	Streams the answer with clickable citations; a source panel shows the cited text highlighted in context; "Why these sources?" shows how each retrieval method ranked every passage.
Evals dashboard	Plain-English explanation of the tests, headline scores, recall@k per retrieval method, a chunk-size sweep, scores by question type, and a drill-down for every test question.

---

Quick start

These are macOS commands, run from the project root; see [Setup](#setup) for details, Linux, and [Windows](#setup-on-windows). Everything is free: no account or API key needed.

**First time** (*roughly 30–45 minutes, an estimate*: mostly downloads and the one-time document analysis):

brew install node python@3.12 postgresql@18 pgvector poppler tesseract ollama
brew servic