PHIacademy

AI Engineering Claude C3 Agent Evaluation And Observability Project — AI skill for Claude Code

AI community

How to install AI Engineering Claude C3 Agent Evaluation And Observability Project

This entry records only its repository, not the path inside it, so there is no exact command to give. Open PHIacademy/ai-engineering-claude-c3-agent-evaluation-and-observability-project and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What AI Engineering Claude C3 Agent Evaluation And Observability Project does

Three Claude AI agent pipelines demonstrating evaluation and observability: validated retry with human-in-the-loop routing, schema-enforced two-pass extraction with consistency checks, and provenance-preserving multi-source synthesis. Includes reproducibility tests, static analysis, calibration reporting, and deliberate perturbation experiments.

Alternatives in AI

Gensee Crate — Run AI harnesses like Claude Code, Codex, Copilot, Antigravity, and Omnigent in disposable workspace forks 123 ★
SwiftAgent — A type-safe, declarative AI agent framework for Swift 100 ★
Bmad Bmm Create Architecture — Create architecture solution design decisions for AI agent consistency 98 ★

README

AI Engineering with Claude — Evaluation & Observability Project

This repository contains the evidence pack, perturbation log, and reflection for the **Evaluation and Observability** project, covering three reference systems built across the course `cd15552 Claude AI Engineer Evaluation and Observability`.

Each system ships as a separate project in the course repo, one folder per system. Every run below was executed from the `solution/` directory of that project's **final** exercise (the `starter/` folders hold the fill-in exercises — the capstone runs the finished `solution/`). Install with `pip install -e ".[dev]"` from inside the solution dir, which puts the console command on your `PATH`.

Evidence folder	Course project → final-exercise `solution/`	Console command
`01-policy-pipeline/`	`Build a Validated, Routed Insurance Policy Extraction Pipeline/04-hitl-routing/solution/`	`policy-extractor`
`02-mortgage-extraction/`	`Build a Resilient Mortgage Document Extraction System/04-validate-mathematical-consistency/solution/`	`mortgage-extract`
`03-supply-chain/`	`Investigate Supply Chain Risk with Multi-Source Synthesis/03-resilient-coordinator/solution/`	`supply-chain-investigate`

No `ANTHROPIC_API_KEY` was available for this submission — every extraction/pipeline run uses recorded/replay clients, and any live-only test is explicitly SKIPPED (visible in each `tests.txt`). Where a rubric item required a live API call that couldn't be made, an offline-equivalent substitute is used and called out explicitly in that item's own file or in [`reflection-brief.md`](reflection-brief.md).

Top-level files

`reflection-brief.md` — the completed reflection brief, answering every prompt with a citation to a specific artifact below.
`environment.txt` — python3 --version + OS, and how the three per-project venvs