tristan-mcinnis

Multimodal Voice Assistant — AI skill for Claude Code

AI community

This project is a multi-modal AI voice assistant that uses LM Studio, OpenAI API or Claude Code, audio processing with WhisperModel, speech recognition, clipboard extraction, and image processing to r.

How to install Multimodal Voice Assistant

This entry records only its repository, not the path inside it, so there is no exact command to give. Open tristan-mcinnis/Multimodal-voice-assistant and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Multimodal Voice Assistant does

Alternatives in AI

Generative Media Skills — Multi-modal Generative Media Skills for AI Agents (Claude Code, Cursor, Gemini CLI) 3k ★
Qwen Audio Agent — A realtime voice runtime that keeps Agents talking, working, and present 2.3k ★
Vllm Mlx — High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon 1.5k ★

README

Multi-Modal AI Voice Assistant

A multi-modal AI voice assistant supporting **DeepSeek (default)**, OpenAI, Anthropic Claude, and local LM Studio LLMs with configurable text-to-speech (OpenAI streaming or Kokoro). Combines voice transcription, tool calling, clipboard extraction, screenshot analysis, and web search to respond with rich context.

Features

Multi-provider LLM support: DeepSeek (default, fast & cheap), OpenAI (GPT-5), local LM Studio, Anthropic Claude
Tool calling: Screenshot capture, webcam capture, clipboard extraction, DuckDuckGo search
Flexible TTS: OpenAI streaming voices or offline Kokoro synthesis
Model Context Protocol (MCP): Pluggable context providers for external integrations
Wake word activation: Say "nova" followed by your prompt
Graceful fallbacks: Models and TTS providers fall back automatically on failure
.env support: All credentials/config can live in a single gitignored .env file

Installation

# Clone the repository
git clone https://github.com/tristan-mcinnis/Multimodal-voice-assistant
cd Multimodal-voice-assistant

# Create virtual environment (recommended)
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Or install as a package
pip install -e .

Quick Start

Default mode (DeepSeek + Kokoro)

# 1. Copy the example .env and add your DeepSeek key
cp .env.example .env
# Edit .env, set DEEPSEEK_API_KEY=sk-...

# 2. (Optional) Download Kokoro TTS models for offline speech (~335MB)
mkdir -p models
curl -L -o models/kokoro-v1.0.onnx https://github.com/nazdridoy/kokoro-tts/releases/download/v1.0.0/kokoro-v1.0.onnx
curl -L -o models/voices-v1.0.bin https://github.com/nazdridoy/kokoro-tts/releases/download/v1.0.0/voices-v1.0.bin

# 3. Run the assistant
python run.py

Get a DeepSeek API key at . The default model is `deepseek-v4-flash` (