Dead Mans Switch

dead-mans-switch

A freshness checker for scheduled agent jobs. It does not watch for errors. It watches for the absence of success, and says so before you notice on your own.

The problem

A scheduled job that stops firing does not announce itself. Nothing throws, nothing pages, no log fills up with stack traces. The job simply stops running, and the report, update, or sync it was supposed to produce is quietly missing from where you would expect it.

This happens more often than error-driven monitoring catches. A cron entry survives a server migration but the script it calls does not. A credential the job depends on expires and the job exits cleanly before doing anything, because nothing forced it to check. An agent's own recurring task gets dropped without a trace when its scheduler config changes. One lane behind this tool went dark for five weeks, unnoticed. No errors. No alerts. Just an increasingly stale set of outputs nobody was checking day to day.

Catching more failure modes does not fix this. Checking for success directly does.

How it works

Each job you track writes a plain success string (a sentinel) into its own log file when it finishes. `deadmans.py` scans your log directory for each task's most recent log, inside a staleness window you configure, and checks for that sentinel. A task is a finding if:

• no log exists yet, and the task is not flagged manual
• the most recent log is older than its configured window
• the most recent log has no success sentinel
• the most recent log carries a failure sentinel instead

The exit code inverts what you would expect from a linter. 0 means every tracked task is fresh. 1 means at least one needs attention. That makes `deadmans.py check` a one-line addition to a cron job or a CI pipeline.

Quickstart

python deadmans.py init          # writes a starter deadmans.json

Edit `deadmans.json` for your own tasks, then make sure each job writes its sentinel string into a log file that matches