Superforecasting Skill

Rainman Superforecast

English | [中文](README.zh-CN.md)

Most of the judgments we worry about — "should I", "will it work out", "how likely" — never get tested. We say something vague, the world unfolds, we re-narrate the past to fit. Nothing about our judgment actually improves, because there was never a contract with reality in the first place.

This skill makes judgment **testable**. It walks you through turning each concern into a resolvable forecast (specific event, deadline, settlement criterion), anchors a probability to a reference class instead of a personal story, updates it as evidence arrives, settles it when the deadline hits, and Brier-scores you over time. The ledger is global — every forecast across every project lands in `~/.superforecast/`, so your calibration trains on real outcomes, not on memory.

**Superforecasting = probabilization + testability.** Inspired by Philip Tetlock's *Superforecasting* and the Good Judgment Project. The book gives the methodology — reference classes, base rates, Bayesian updating, Brier scoring; this project makes the methodology executable. The LLM agent walks you through eight gates (resolvability, Fermi-ization, three-layer reference class, base rate, internal-view adjustments, forecast-vs-decision separation, ...). A deterministic Python engine enforces a state machine that blocks the two most common forecast-killers: predictions without deadlines, and after-the-fact reinterpretation. The goal is not "AI predicts the future" — it is "AI engineers your forecasting process; the human keeps judgment ownership and bears decision responsibility."

---

How It Works

User input ("should I leave Beijing?", "will the deal close?", anxiety)
  │
  ▼
[Mode classification]  new / update / settle / review / coach
  │
  ▼ (Mode = new)
[Gate 1] Forecast vs decision vs emotion
[Gate 2] Resolvable? — rewrite if not
[Gate 3] Cloud-like? — Fermi-ize into 3-7 sub-questions
[Gate 4] Type: binary (default) / multi_outcome /