Enterprise IT Support Agentic RAG Copilot With Evals And Safety — AI skill for Claude Code

Enterprise IT Support Agentic RAG Copilot

An end-to-end **Forward Deployed Engineer (FDE) project** that turns a notebook-style Agentic RAG workflow into a deployable internal product using **LangGraph, FastAPI, Pinecone, OpenAI, Tavily, HTML, CSS, and JavaScript** — with an offline evaluation harness (RAGAS + DeepEval, judged by Claude) and a metrics dashboard on top.

**Complete documentation:** see the [architecture, demo, safety, and evaluation guide](docs/PROJECT_GUIDE.md) for verified screenshots, Mermaid flow diagrams, hands-on demo scenarios, evaluation methodology, reference results, and operational guidance.

---

1. Business Problem

Customer

**NovaRetail**, a fictional 3,000-employee retail company.

Problem

The internal IT team maintains many documents: VPN instructions, password rules, MFA policy, software installation rules, laptop troubleshooting guides, and service-desk runbooks.

Employees still create repetitive support tickets because:

• They do not know where the correct document is.
• Traditional keyword search returns too many results.
• A normal chatbot may hallucinate an answer.
• Internal documents can be incomplete or outdated.
• Some questions require current vendor information from the public web.

Example

An employee asks:

**“How do I connect to the company VPN from home?”**

The answer exists in the private company KB, so the system should **not search the public internet**.

Another employee asks:

**“What is the latest Microsoft Teams outage guidance?”**

The internal KB may not contain current outage information. The system should recognize weak private evidence, use an external search, grade that evidence, and answer with an external-source warning.

Business Goal

Build a secure IT Support Copilot that:

1. Searches trusted private knowledge first.
2. Checks whether retrieved evidence is good enough.
3. Uses web search only for explicitly public, time-sensitive vendor questions.
4. Rewrites weak querie