Claude Sound Ear — AI skill for Claude Code

sound-ear

**Give Claude eyes and ears on audio.** An installable [Claude](https://claude.com/product/claude-code) skill that turns any audio or video soundtrack into things an AI can actually reason over — a readable spectrogram, objective loudness/timing facts, a semantic map of every sound effect, offline speech transcription, and sound-driven cymatic visuals (including seamless long-form loops for sleep/ambient channels).

Claude can't natively *hear* a `.wav`, but it can read images and structured text. sound-ear converts sound into exactly those, then Claude reasons over the result.

What it does

Command	Purpose	Needs
hear.sh see	Spectrogram + waveform + loudness/silence/scene-cut report	ffmpeg
hear.sh map	Semantic sound map: every SFX with timestamp, label, description, mix flags	Vertex or CLAP
hear.sh transcribe	Offline speech-to-text	faster-whisper
hear.sh cymatics	Sound-driven visuals (any aspect ratio)	numpy + PIL + ffmpeg
hear.sh loop --minutes N	Seamless long-form loop (sleep/ambient) rendered from a short seed	numpy + PIL + ffmpeg
hear.sh doctor	Device-compatibility report + engine routing	—

Timing always comes from ffmpeg / spectral-flux onset detection, never from the language model (audio LLMs describe sound well but localize it poorly), so timestamps stay accurate.

Device-aware by design

Run `hear.sh doctor` and the skill reports what your machine supports and picks the best available engine for `map`:

1. vertex — Gemini on your own Google Cloud (best quality). Needs google-genai + ADC.
2. clap — offline zero-shot labeling. Needs PyTorch + transformers.

If neither is present it falls back to `see` (visual) and `transcribe` (speech), which run anywhere with ffmpeg.

Install

git clone https://github.com/tysitv/claude-sound-ear.git ~/.claude/skills/sound-ear
cd ~/.claude/skills/sound-ear
./se