tristan-mcinnis

Multimodal Voice Assistant

Name: Multimodal Voice Assistant
Author: tristan-mcinnis

AI community

SOURCE https://github.com/tristan-mcinnis/Multimodal-voice-assistant

Description

This project is a multi-modal AI voice assistant that uses LM Studio, OpenAI API or Claude Code, audio processing with WhisperModel, speech recognition, clipboard extraction, and image processing to respond to user prompts.

Installation

This entry records only its repository, not the path inside it, so there is no exact command to give. Open the source below and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

README

Multi-Modal AI Voice Assistant

A multi-modal AI voice assistant supporting **DeepSeek (default)**, OpenAI, Anthropic Claude, and local LM Studio LLMs with configurable text-to-speech (OpenAI streaming or Kokoro). Combines voice transcription, tool calling, clipboard extraction, screenshot analysis, and web search to respond with rich context.

Features

Multi-provider LLM support: DeepSeek (default, fast & cheap), OpenAI (GPT-5), local LM Studio, Anthropic Claude
Tool calling: Screenshot capture, webcam capture, clipboard extraction, DuckDuckGo search
Flexible TTS: OpenAI streaming voices or offline Kokoro synthesis
Model Context Protocol (MCP): Pluggable context providers for external integrations
Wake word activation: Say "nova" followed by your prompt
Graceful fallbacks: Models and TTS providers fall back automatically on failure
.env support: All credentials/config can live in a single gitignored .env file

Installation

# Clone the repository
git clone https://github.com/tristan-mcinnis/Multimodal-voice-assistant
cd Multimodal-voice-assistant

# Create virtual environment (recommended)
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Or install as a package
pip install -e .

Quick Start

Default mode (DeepSeek + Kokoro)

# 1. Copy the example .env and add your DeepSeek key
cp .env.example .env
# Edit .env, set DEEPSEEK_API_KEY=sk-...

# 2. (Optional) Download Kokoro TTS models for offline speech (~335MB)
mkdir -p models
curl -L -o models/kokoro-v1.0.onnx https://github.com/nazdridoy/kokoro-tts/releases/download/v1.0.0/kokoro-v1.0.onnx
curl -L -o models/voices-v1.0.bin https://github.com/nazdridoy/kokoro-tts/releases/download/v1.0.0/voices-v1.0.bin

# 3. Run the assistant
python run.py

Get a DeepSeek API key at . The default model is `deepseek-v4-flash` (