PHIacademy

AI Engineering Claude C3 Agent Evaluation And Observability Project — AI skill for Claude Code

AI community

How to install AI Engineering Claude C3 Agent Evaluation And Observability Project

This entry records only its repository, not the path inside it, so there is no exact command to give. Open PHIacademy/ai-engineering-claude-c3-agent-evaluation-and-observability-project and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What AI Engineering Claude C3 Agent Evaluation And Observability Project does

Three Claude AI agent pipelines demonstrating evaluation and observability: validated retry with human-in-the-loop routing, schema-enforced two-pass extraction with consistency checks, and provenance-preserving multi-source synthesis. Includes reproducibility tests, static analysis, calibration reporting, and deliberate perturbation experiments.

Alternatives in AI

Geo SEO Claude — GEO-first SEO skill for Claude Code 3.6k ★
Retain — Retain is a native macOS application that aggregates AI conversations from multiple platforms into a unified 168 ★
Rails AI Context — 45 MCP tools that give AI coding agents ground truth about your Rails app: schema, models, routes, controllers 155 ★

README

AI Engineering with Claude — Evaluation & Observability Project

This repository contains the evidence pack, perturbation log, and reflection for the **Evaluation and Observability** project, covering three reference systems built across the course `cd15552 Claude AI Engineer Evaluation and Observability`.

Each system ships as a separate project in the course repo, one folder per system. Every run below was executed from the `solution/` directory of that project's **final** exercise (the `starter/` folders hold the fill-in exercises — the capstone runs the finished `solution/`). Install with `pip install -e ".[dev]"` from inside the solution dir, which puts the console command on your `PATH`.

Evidence folder	Course project → final-exercise `solution/`	Console command
`01-policy-pipeline/`	`Build a Validated, Routed Insurance Policy Extraction Pipeline/04-hitl-routing/solution/`	`policy-extractor`
`02-mortgage-extraction/`	`Build a Resilient Mortgage Document Extraction System/04-validate-mathematical-consistency/solution/`	`mortgage-extract`
`03-supply-chain/`	`Investigate Supply Chain Risk with Multi-Source Synthesis/03-resilient-coordinator/solution/`	`supply-chain-investigate`

No `ANTHROPIC_API_KEY` was available for this submission — every extraction/pipeline run uses recorded/replay clients, and any live-only test is explicitly SKIPPED (visible in each `tests.txt`). Where a rubric item required a live API call that couldn't be made, an offline-equivalent substitute is used and called out explicitly in that item's own file or in [`reflection-brief.md`](reflection-brief.md).

Top-level files

`reflection-brief.md` — the completed reflection brief, answering every prompt with a citation to a specific artifact below.
`environment.txt` — python3 --version + OS, and how the three per-project venvs