R3 Skill — AI skill for Claude Code

Skill Is Not Document

A query-conditional benchmark (**R3-Skill**) and a two-stage retriever (**R3-Embedding + R3-Reranker**) for LLM agent skill routing.

• 📄 Paper: Skill Is Not Document: A Query-Conditional Benchmark and Two-Stage Retriever for LLM Agent Skill Routing
• 🧩 This repository contains the code, docs, R3-Skill data, and a toy example. Model weights are hosted on Hugging Face — R3-Embedding-0.6B / R3-Rerank-0.6B — and should be placed under models/ for reproduction.

---

1. Layout

├── infer.py              two-stage retrieval (embedding recall → reranker rerank)
├── reproduce.py          reproduce the paper's R3-Skill test numbers
├── example_skills.jsonl  toy skill library for a quick test
├── requirements.txt
├── README.md / CONTRIBUTING.md / CODE_OF_CONDUCT.md / LICENSE
├── models/               not tracked by Git; place downloaded model weights here
│   ├── r3-embedding/     bi-encoder    (fine-tuned from Qwen3-Embedding-0.6B)
│   └── r3-reranker/      cross-encoder (fine-tuned from Qwen3-Reranker-0.6B)
└── data/                 R3-Skill benchmark data
    ├── test.jsonl        5,696 test queries
    └── skills_test.jsonl  2,050 test-pool skills

Skill identifiers are released as opaque ids (`skill-00000`, …); each id maps consistently across the query and skill-pool files. Train and test skill pools are disjoint.

The skill pool and query set draw on the following public resources:

• https://huggingface.co/datasets/ThakiCloud/SKILLRET
• https://huggingface.co/datasets/pipizhao/SkillRouter-Eval-Core
• https://gitcode.com/zhoukc42/AiSkill
• https://github.com/jnMetaCode/agency-agents-zh

---

2. Architecture

                    query
                      │
                      ▼
        ┌─────────────────────────────┐
        │  Stage 1 · R3-Embedding      │   bi-encoder, cosine