DDDFXYqiming

LLM As A Verifier — AI skill for Claude Code

AI community

LLM-as-a-Verifier is a general-purpose framework that provides fine-grained feedback for any agent without requiring additional training.

How to install LLM As A Verifier

This entry records only its repository, not the path inside it, so there is no exact command to give. Open DDDFXYqiming/llm-as-a-verifier and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What LLM As A Verifier does

LLM-as-a-Verifier is a general-purpose framework that provides fine-grained feedback for any agent without requiring additional training. It achieves SOTA performance across coding, robotics, and medical agentic benchmarks.

Alternatives in AI

Fine Tuning — ai-research-skills Axolotl, LLaMA-Factory, PEFT, Unsloth 5.4k ★
Deepreasoning — A high-performance LLM inference API and Chat UI that integrates DeepSeek R1's CoT reasoning traces with Anthr 5.4k ★
Vllm Mlx — High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon 1.5k ★

README

Any modality, Many Applications, One Unified Verification Framework

🔥 LLM-as-a-Verifier achieves SOTA performance across agentic benchmarks, including Terminal-Bench, SWE-Bench Verified, MedAgentBench, RoboRewardBench and more. We invite the community to contribute more use cases!

Installation

pip install llm-verifier

To install the latest from a clone:

pip install -e .

**What's new in 0.2.0** (full notes in [CHANGELOG.md](CHANGELOG.md)):

Prefix-cache optimization: ~3.4× fewer uncached input tokens on trajectory-heavy benchmarks
Terminal-Bench 2.1 self-verification benchmark
deepseek-v4-flash verifier backend
Token accounting (llm_verifier.token_usage())

About

LLM-as-a-Verifier is a general-purpose framework that provides **fine-grained feedback** for any agent. The key idea is simple: 1) use fine-grained scoring granularity, 2) take the expectation over the full logprob distribution of LLM score tokens, and 3) scale repeated evaluation and criteria decomposition. The resulting fine-grained feedback can be used for test-time scaling, progress tracking, and reinforcement learning.

LLM-as-a-Verifier overview