SkillEvaluator — AI skill for Claude Code
Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect a.
How to install SkillEvaluator
This entry records only its repository, not the path inside it, so there is no
exact command to give. Open NVIDIA/SkillEvaluator and copy the folder into
~/.claude/skills/, or the file into ~/.claude/agents/.
What SkillEvaluator does
Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior.
Alternatives in AI
- Welcome — AI Research Skills — You now have access to 86 production-ready skills covering the entire AI research lifecycle: literature survey 5.4k ★
- Generative Media Skills — Multi-modal Generative Media Skills for AI Agents (Claude Code, Cursor, Gemini CLI) 3k ★
- Develop — Development phase - Build solutions with multi-AI implementation and quality gates 2.8k ★
README
SkillEvaluator
[](LICENSE) [](https://www.python.org/) [](https://docs.nvidia.com/skills/skillevaluator/)
SkillEvaluator is an open-source, multi-tier framework for evaluating AI agent artifacts, starting with agent skills: deterministic quality gates, semantic overlap detection, synthetic eval dataset generation, and live agent evaluation.
Agent skills are folders of instructions and supporting files that extend AI agents, as defined by the [Agent Skills specification](https://agentskills.io/). SkillEvaluator is part of the [NVIDIA Verified Skills pipeline](https://github.com/NVIDIA/skills).
Three-tier overview
Tiers are independent entry points; nothing requires running earlier ones first.
| Tier | Purpose | Representative commands | Requires |
|---|---|---|---|
| Tier 1: Validation | Safe & well-formed? | validate, quality-check, security-scan, pii-scan, lint-scripts, rubric-eval |
No API key for deterministic checks; the security extra plus external Semgrep, SkillSpector, and Gitleaks for full scanner coverage; a provider key for LLM checks |
| Tier 2: Deduplication | Overlap with what exists? | context-optimization-check, similarity-check |
An embeddings provider; intra-skill analysis also needs a chat LLM — local OpenAI-compatible endpoints work |
| Tier 3: Live Evaluation | Does it help the agent? | create-eval-dataset, tier3 evaluate, compare |
No credential for keyless templates and report inspection; a provider key for LLM generation and grading; live evaluation a |
Related Skills
Jev Curate
High-throughput synthetic & pretraining dataset sifter powered by TypeSafe AI Jev (api.typesafe.ai). Stream, f
30k Corpus
28,721 GitHub repositories analyzed for AI instruction quality across Claude, Codex, Copilot, Cursor, and Gemi
Synthetic Market Research
Agent skill for fast, cheap market research using LLM synthetic surveys + Semantic Similarity Rating (SSR). No
Pbi Diff
Generate a human-readable summary of all uncommitted changes in the semantic model since the last git commit.
Vox Agent
LLM agent for customer support with inline evaluation, hallucination detection, retry/fallback logic, and RAG
Agent Eval Workbench
MLflow-based evaluation kit for AI agents: golden-task runner, calibrated LLM judge, list-price cost accountin
Related Agents
Codeweb Domain Mapper
Read-only domain-tagging and overlap-detection pass for the codeweb plugin. Given the merged node/edge graph,
Fe A11y Auditor
Specialized accessibility (a11y) audit — semantic HTML, ARIA, keyboard navigation, color contrast, focus manag
Dual Channel Drift Auditor
Cold pairwise audit of a declared @dual-pair anchor group. Given ONLY an anchor (never the PR diff or narrativ
