imtiazrayhan

Benchmark Rerankers — Development skill for Claude Code

Development community

Measure whether adding a reranker actually improves retrieval, by scoring reranked vs. un-reranked results on a labeled query set.

How to install Benchmark Rerankers

Installs to ~/.claude/skills/imtiazrayhan-agentscamp-library-benchmark-rerankers/SKILL.md

Terminal

mkdir -p ~/.claude/skills/imtiazrayhan-agentscamp-library-benchmark-rerankers && curl -fsSL https://raw.githubusercontent.com/imtiazrayhan/agentscamp-library/HEAD/commands/benchmark-rerankers.md -o ~/.claude/skills/imtiazrayhan-agentscamp-library-benchmark-rerankers/SKILL.md

Restart Claude Code, or start a new session, for it to be picked up.

What Benchmark Rerankers does

description: "Measure whether adding a reranker actually improves retrieval, by scoring reranked vs. un-reranked results on a labeled query set." argument-hint: "" allowed-tools: "Read, Grep, Glob, Bash" model: sonnet

Scope

Treat `$ARGUMENTS` as the retrieval setup to benchmark — a path to an eval set and retrieval results, or a description of the pipeline (retriever, candidate count, reranker). Restate what you

Alternatives in Development

Skill Creator — Create new skills, modify existing skills, and measure skill performance 94.1k ★
Fidelity — Measure how faithfully a clone reproduces a site — pixel-diff plus motion-fidelity into one 0-100 score, a let 3.6k ★
Evo — turns your codebase into an autoresearch loop — discovers what to measure, instruments the benchmark, then run 1.4k ★