observability-oss

Eval From Trace — AI skill for Claude Code

AI community

Build an LLM-as-a-Judge eval grounded in real traces from the Progress Observability Platform.

How to install Eval From Trace

Installs to ~/.claude/skills/observability-oss-progress-observability-plugin-eval-from-trace/SKILL.md

Terminal

mkdir -p ~/.claude/skills/observability-oss-progress-observability-plugin-eval-from-trace && curl -fsSL https://raw.githubusercontent.com/observability-oss/progress-observability-plugin/HEAD/commands/eval-from-trace.md -o ~/.claude/skills/observability-oss-progress-observability-plugin-eval-from-trace/SKILL.md

Restart Claude Code, or start a new session, for it to be picked up.

What Eval From Trace does

description: Build an LLM-as-a-Judge eval grounded in real traces from the Progress Observability Platform. argument-hint: [application/service name and optional symptom, e.g. "checkout-agent, wrong tool calls"]

Use the `generate-eval` skill, Workflow A (from traces).

Target: $ARGUMENTS

Steps:

Survey the system's recent traffic with the metadata-only observability tools (list_observations) over the last 24–72h.
Choose a single failure mode and config from what the traces actua

Alternatives in AI

Deepreasoning — A high-performance LLM inference API and Chat UI that integrates DeepSeek R1's CoT reasoning traces with Anthr 5.4k ★
Manifest — Real-time cost observability for OpenClaw agents — track tokens, costs, messages, and model usage 4.1k ★
AI Avatar System — 🎭 AI Avatar / digital human platform — upload a photo, clone a voice, talk to any face in real time with lip 422 ★