grandamenium

Multimodal RAG — AI skill for Claude Code

AI community

Fully local multimodal RAG knowledge base for AI agents.

How to install Multimodal RAG

This entry records only its repository, not the path inside it, so there is no exact command to give. Open grandamenium/multimodal-rag and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Multimodal RAG does

Fully local multimodal RAG knowledge base for AI agents. Ingest videos, images, audio, docs into ChromaDB using Gemini Embedding 2.

Alternatives in AI

Skill Seekers Roadmap — Transform Skill Seekers into the easiest way to create Claude AI skills from any knowledge source - documentat 11.1k ★
RAG Skill — A Skill dedicated to local knowledge base retrieval 712 ★
Karpathy LLM Wiki — Agent Skills-compatible LLM wiki for Claude Code, Cursor, and Codex 618 ★

README

Multimodal RAG Knowledge Base

A fully local multimodal knowledge base for AI agents. Ingest videos, images, audio, documents (PDF, DOCX, PPTX, XLSX), and code into a local vector database using Google's Gemini Embedding 2 model. Query it from Claude Code, OpenClaw, or any script.

Quick Start

# 1. Clone into your skills directory
git clone https://github.com/grandamenium/multimodal-rag.git ~/.claude/skills/multimodal-rag

# 2. Run setup (installs deps, prompts for free Gemini API key)
bash ~/.claude/skills/multimodal-rag/scripts/setup.sh

# 3. Ingest and query
python3 ~/.claude/skills/multimodal-rag/scripts/mmrag.py ingest ~/Documents/my-business/
python3 ~/.claude/skills/multimodal-rag/scripts/mmrag.py query "What's our pricing strategy?" --json

What It Does

Most RAG systems only handle text. This one handles everything - videos, images, audio, PDFs, Office docs, and code - all in one local database.

The key insight: you can't just embed a video and expect useful answers. This system runs every non-text file through Gemini Flash first to generate a detailed text description (transcript, visual description, key topics), then embeds that description alongside the raw media. So when you query "how does the heartbeat work?", you get back an actual text answer with the source file path - not just a video clip.

**For videos**: FFmpeg splits into 60-second chunks with overlap. Large files (1GB+) automatically extract the audio track for processing. Each chunk gets a full transcript + topic summary.

**For images**: Gemini Flash describes everything visible - text, diagrams, layout, concepts. The description + raw image are embedded together.

**For Office docs**: .docx, .pptx, .xlsx are extracted locally using python-docx/pptx/openpyxl. No API calls needed.

How It Works

Ingest: File -> type detection -> Gemini Flash describes (media) or chunk (text)
     -> Gemini Embedding 2 (768-dim vectors) -> ChromaDB (local)

Query: Question -> embe