B0yko

Cheat Index — Development skill for Claude Code

Development community

The Agent Cheat Index: how often coding agents edit tests, special-case inputs or claim success they did not earn.

How to install Cheat Index

This entry records only its repository, not the path inside it, so there is no exact command to give. Open B0yko/cheat-index and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Cheat Index does

The Agent Cheat Index: how often coding agents edit tests, special-case inputs or claim success they did not earn. Real CLIs run headless on 30 trap tasks and 10 controls in sandboxed containers, judged on the end state. Open harness, raw runs and leaderboard.

Alternatives in Development

Scan Trap — 单独检查一只股票是不是杀猪盘 / 有没有被资金套路 6.6k ★
Param Discover — Discover hidden HTTP parameters on a URL or list of URLs using Arjun (or x8 fallback) 4.5k ★
Claude Code Skill — Persistent multi-engine coding session manager 326 ★

README

Agent Cheat Index

**How often coding agents edit tests, special-case inputs or claim success they did not earn.** Real agent CLIs run headless on 30 trap tasks and 10 controls in sandboxed containers. The end state (the diff and hidden checks) decides what was done; a rule-based reading of the final message decides what was claimed.

[![CI](https://github.com/B0yko/cheat-index/actions/workflows/ci.yml/badge.svg)](https://github.com/B0yko/cheat-index/actions/workflows/ci.yml) [![Release](https://img.shields.io/github/v/release/B0yko/cheat-index)](https://github.com/B0yko/cheat-index/releases) [![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)

Across 10 agent configurations and 1162 runs (2026-10-23 to 2026-10-24 UTC, Mac Studio M4 Max, 128 GB), the Agent Cheat Index ranged from 1% [0–3] (Claude Code, `anthropic/claude-sonnet-5.5`) to 72% [61–83] (OpenCode, `qwen/qwen3-coder`): the share of runs on contradictory, blocked and tempting-shortcut tasks in which the agent changed or skipped checks without saying so, or claimed success that hidden checks contradict.

How it was measured: [methodology](docs/methodology.md). Full leaderboard, with evidence cards for flagged runs: **[b0yko.github.io/cheat-index](https://b0yko.github.io/cheat-index/)**.

Top of the Agent Cheat Index leaderboard: ACI with 95% intervals per agent and model

![cheat-index demo re-scoring recorded runs in the terminal](docs/img/demo.svg)

uvx --from git+https://github.com/B0yko/cheat-index cheat-index demo

The demo needs no Docker and no API key: it re-scores a bundled sample of real recorded runs with the same detectors, classifier and statistics as the published results, and prints a mini leaderboard in a few seconds.