Arize-ai

Instruction Budget — Development skill for Claude Code

Development community

A public repo of an evaluations research project.

How to install Instruction Budget

This entry records only its repository, not the path inside it, so there is no exact command to give. Open Arize-ai/instruction-budget and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Instruction Budget does

A public repo of an evaluations research project.

Alternatives in Development

GDScript VS C# — Skill Instruction Comparison — GDScript vs C# — Skill Instruction Comparison skill 2.8k ★
04 Cyber Risk Instruction — Cyber Risk Instruction 2.3k ★
Claude Code Plugins — jeremylongshore Instruction-template plugins and MCP plugin packs 1.7k ★

README

instruction-budget

IFScale benchmark replication on the May 2026 frontier models, instrumented with Arize AX.

The full plan is in [IFSCALE_REPLICATION.md](IFSCALE_REPLICATION.md). Read it before running anything.

Quickstart

python -m venv .venv && source .venv/bin/activate
pip install -e .

# 1. Generate the (model, density, seed) -> keywords sampling matrix
python -m scripts.generate_samples

# 2. Run the smoke grid against Claude Opus 4.7 (week-1 self-validation)
python -m src.runner --model claude-opus-4-7 --densities 10,100,250,500 --seeds 1

Required env vars

In `.env` (already gitignored):

ANTHROPIC_API_KEY=...
OPENAI_API_KEY=...
GEMINI_API_KEY=...
DEEPSEEK_API_KEY=...

# Arize AX — required. The runner will refuse to start without these.
ARIZE_SPACE_ID=...
ARIZE_API_KEY=...