imtiazrayhan

Benchmark Rerankers — Development skill for Claude Code

Development community

Measure whether adding a reranker actually improves retrieval, by scoring reranked vs. un-reranked results on a labeled query set.

How to install Benchmark Rerankers

Installs to ~/.claude/skills/imtiazrayhan-agentscamp-library-benchmark-rerankers/SKILL.md

Terminal

mkdir -p ~/.claude/skills/imtiazrayhan-agentscamp-library-benchmark-rerankers && curl -fsSL https://raw.githubusercontent.com/imtiazrayhan/agentscamp-library/HEAD/commands/benchmark-rerankers.md -o ~/.claude/skills/imtiazrayhan-agentscamp-library-benchmark-rerankers/SKILL.md

Restart Claude Code, or start a new session, for it to be picked up.

What Benchmark Rerankers does

description: "Measure whether adding a reranker actually improves retrieval, by scoring reranked vs. un-reranked results on a labeled query set." argument-hint: "" allowed-tools: "Read, Grep, Glob, Bash" model: sonnet

Scope

Treat `$ARGUMENTS` as the retrieval setup to benchmark — a path to an eval set and retrieval results, or a description of the pipeline (retriever, candidate count, reranker). Restate what you

Alternatives in Development

Evo — turns your codebase into an autoresearch loop — discovers what to measure, instruments the benchmark, then run 1.4k ★
AgentSkillOS — Build your agent from 200,000+ skills via skill RETRIEVAL & ORCHESTRATION 373 ★
Contributing To Smart Ralph — "I'm learnding 260 ★