wangrunji0408

SinkingStarBench — AI skill for Claude Code

AI community

An LLM coding agent benchmark based on Order of the Sinking Star.

How to install SinkingStarBench

This entry records only its repository, not the path inside it, so there is no exact command to give. Open wangrunji0408/SinkingStarBench and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What SinkingStarBench does

An LLM coding agent benchmark based on *Order of the Sinking Star*.

Alternatives in AI

Loushang — AI-native agent harness for coding workflows by python: multi-model LLM orchestration, stateful sessions, tool 1.3k ★
Style Distill — style-distill skill — 三阶段 13 模板 LLM 蒸馏（style-distill-order） 669 ★
CoreCoder — Minimal AI coding agent (~1,400 LoC Python) inspired by Claude Code 616 ★

README

Sinking Star Bench

[中文版](README-CN.md) | English

An LLM coding agent benchmark based on *Order of the Sinking Star*, a sokoban-style puzzle game. The task: **"Beat this game. Reverse-engineering the binary is forbidden."** — infer the rules from the game's own CLI and solve all 12 levels.

Results

#	Model	Effort	Provider	Solved	Time	Tokens	Context	Tool Calls	Cost
🏅	GPT-6 Astra	High	Codex	12/12	3 min	0.46M	31K	18	$1.02
🥈	DeepSeek v4.1 Flash	High	DSH	12/12	8 min	5.8M	117K	86	$0.07
🥉	Claude Fable 5.1	High	CC	12/12	11 min	1.49M	70K	30	$2.94
4	Claude Opus 5	High	CC	12/12	16 min	3.7M	79K	63	$3.85
5	DeepSeek v4 Pro	High	DSH	12/12	27 min	3.8M	98K	81	$0.08
6	GPT-5.6 Sol	High	Codex	12/12	40 min	5.5M	147K	34	$17.19
7	DeepSeek v4 Flash	Max	CC	12/12	117 min	49.3M	509K	195	$0.30
8	Kimi K3	High	CC	12/12	163 min	10.2M	274K	74	$7.28
9	DeepSeek v4 Pro Preview	Max	CC	9/12	60 min	26.6M	234K	176	$0.41
10	Claude Fable 5	N/A	CC	N/A	—	—	—	—	Refused

Harness: CC = Claude Code, DSH = DeepSeek Harness.

Key Observations

GPT-6 Astra — the fastest 12/12 and the most token-efficient: pure black-box, zero web searches, no binary introspection. Probed the public CLI, inferred the Warrior/Thief/Wizard mechanics from gameplay alone, wrote a solver, and verified every level through the official run interface (Won yes).
DeepSeek v4.1 Flash — pure black-box on DeepSeek Harness with the standard agent preset (write/edit/read available): zero web searches, no binary introspection. Rebuilt the game as a simulator from CLI probing, validated it by differential fuzzing (~3,900