Lecture To Notes

lecture-to-notes

[](https://github.com/drpwchen/lecture-to-notes/actions/workflows/secret-scan.yml)

**English** · [繁體中文](README.zh-TW.md)


Turn a lecture or conference recording into structured, slide-illustrated notes — and into a **synced HTML viewer** where the video, the timestamped transcript, and the curated summary sit on one page: the video highlights the matching note as it plays, and clicking any note timestamp seeks the video.

Every expensive stage runs **locally**: Whisper ASR on your GPU, frame extraction, OCR, and a local vision model for slide semantics. An LLM is used only at the end, to write prose from evidence the pipeline already assembled.

The design goal is not "summarize a video". It is **traceability**: every claim in the finished note should be attachable to a moment in the transcript and to the slide that was on screen at that moment. Most of the machinery here exists to make that link trustworthy rather than plausible.

Why this exists

Rehabilitation-medicine courses are notoriously hard to take notes on. Manual therapy, ultrasound scanning — the knowledge is in the *motion*, so at every course you see a forest of tripods: everyone records everything, planning to rewatch over dinner. Nobody actually rewatches hours of video.

With AI the first instinct was to teach it precise screenshotting, to squeeze the video back into a traditional text-plus-figures note. The turn was realizing a note doesn't have to be that shape at all: build a webpage that ties the video to the transcript, and you can jump straight to the moment you care about. What you actually want to know is *how the maneuver is performed* — and that only lives in motion. So the summary is for studying, the transcript is for verifying, and the video is o