aws-solutions-library-samples

Live Eval And Cost — Development skill for Claude Code

Development community

Live Evaluation & Cost Analysis — GenAI IDP Accelerator.

How to install Live Eval And Cost

Installs to ~/.claude/skills/aws-solutions-library-samples-accelerated-intelligent-document-processing-on-aws-live-eval-and-cost/SKILL.md

Terminal

mkdir -p ~/.claude/skills/aws-solutions-library-samples-accelerated-intelligent-document-processing-on-aws-live-eval-and-cost && curl -fsSL https://raw.githubusercontent.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/HEAD/.claude/skills/live-eval-and-cost.md -o ~/.claude/skills/aws-solutions-library-samples-accelerated-intelligent-document-processing-on-aws-live-eval-and-cost/SKILL.md

Restart Claude Code, or start a new session, for it to be picked up.

What Live Eval And Cost does

Live Evaluation & Cost Analysis — GenAI IDP Accelerator

Use this when validating a change against **real deployed stacks**: benchmark A/B tests (model, prompt, config), v0.5→v0.6-style upgrade tests, and reading accuracy/cost/confidence from a run. This is the "prove it live" workflow, complementary to unit tests (`testing-qa.md`).

All AWS + `idp-cli` calls use `AWS_PROFILE=default`. Confirm the account first: `AWS_PROFILE=default aws sts get-caller-identity`. Never clobber the `Config

Alternatives in Development

Quick Eval — Quick job evaluation 473 ★
Eval Skill — /eval-skill — Six-Dimension Evaluation 216 ★
Link Outreach — Link Outreach: review the listicle authors who replied to your backlink outreach, and reply to them in-thread 195 ★