sshh12

Agent PR Replay — Git skill for Claude Code

Git community

Agent PR Replay takes merged PRs from any repository, reverse-engineers the task prompt, runs Claude Code against it, and compares what the agent did versus what humans actually shipped.

How to install Agent PR Replay

This entry records only its repository, not the path inside it, so there is no exact command to give. Open sshh12/agent-pr-replay and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Agent PR Replay does

Agent PR Replay takes merged PRs from any repository, reverse-engineers the task prompt, runs Claude Code against it, and compares what the agent did versus what humans actually shipped. The result is targeted, empirical guidance.

Alternatives in Git

Analysis Claude Code — GitHub Repo stars Complete reverse engineering research and analysis of Claude Code v1.0.33 Reverse engineerin 35k ★
Merger — 增量 Merge Prompt 24k ★
Bulletproof Quick Start Guide — Target Audience: Complete beginners Never used Python/git before 11.1k ★

README

Agent PR Replay

**Discover how AI coding agents actually navigate your codebase, and where they diverge from human developers.**

The best way to improve an agent's ability to work in a codebase is to observe its default behavior, measure the gap against real human solutions, and steer it based on evidence.

**[See example reports →](examples/)**

How It Works

flowchart TB
    subgraph ground["① Ground Truth: Human PRs"]
        H1["PR #1234 merged"] --> H2["Human diff:
auth.py +12 -3"]
    end

    subgraph replay["② Replay"]
        R1["Checkout base commit"] --> R2["Reverse-engineer prompt:
'Fix login validation for empty passwords'"]
    end

    subgraph agent["③ Run Claude Code"]
        A1["Claude explores codebase"] --> A2["Claude diff:
auth.py +28 -3, test_auth.py +45"]
    end

    subgraph compare["④ Compare & Synthesize"]
        C1["Human: minimal fix
Claude: added tests, refactored"] --> C2["Gap: over-engineering"]
        C2 --> C3["CLAUDE.md / AGENTS.md:
'Match scope of request,
don't add tests unless asked'"]
    end

    ground --> replay
    replay --> agent
    ground --> compare
    agent --> compare

Ground Truth: Start with real merged PRs as human-validated solutions
Replay Setup: Checkout the repo at the PR's base commit, reverse-engineer a task prompt from the diff
Run Agent: Execute Claude Code with the prompt (same starting point, same task, different solver)
Compare: Diff what Claude changed vs what humans shipped; identify systematic gaps
Synthesize: Generate targeted guidance (CLAUDE.md, AGENTS.md, skills) based on observed behavioral patterns