Agent Evals Playground — Development skill for Claude Code

Agent evals against real dependencies

A shopping agent, 66 labelled test cases, and a runner that scores it against either a JSON fixture or a live Kubernetes cluster.

There is no flag for picking which. The runner reads `INVENTORY_SERVICE_URL` and `ORDER_SERVICE_URL` from its environment. Unset, the agent falls back to a catalogue snapshot checked into the repo. Set, it calls the real services.

npm run eval                            # scores against the fixture
mirrord exec ... -- npm run eval        # scores against your cluster

The second command inherits its environment from a running deployment, so those URLs arrive already set and resolve inside the cluster. The agent itself still runs locally. Nothing is deployed and no environment is created.

What's here

apps/shop/chat-service	the shopping agent (a tool-calling loop) and its eval suite
apps/shop/inventory-service, order-service	the dependencies the agent reads
apps/shop/metal-mart-frontend	a storefront with a support chat, so you can watch the agent work
apps/visualization-shop	a live picture of the cluster, including mirrord sessions
overlays/	deploy it: base, plus local or gke
.mirrord/agent-evals.json	the mirrord config, 20 lines

[apps/shop/chat-service/eval/README.md](apps/shop/chat-service/eval/README.md) covers the dataset, the scoring modes and the judge in detail.

What you need

• a Kubernetes cluster you can reach. kind and minikube are fine
• the mirrord Operator installed on it, and the mirrord CLI locally
• an Anthropic API key
• Node 20+

All container images are published publicly, so on an **amd64** cluster nothing here needs building.

On **arm64** (kind or minikube on Apple Silicon) four of them will not pull: `chat-service`, `metal-mart-frontend` and the two visualization