Gemini Model Router

gemini-model-router

A local-first agentic terminal that routes each prompt across **local Gemma 4 (vLLM)**, **Gemini CLI**, and **Claude Code** — picking the right one to optimize cost, latency, and quality. Logs every decision in a format the [gemini-dreams](https://github.com/jswortz/gemini-dreams) nightly loop can mine to propose its own improvements.

[](https://asciinema.org/a/P3pkHoXam0N1MJKM)

Install

Recommended — install as a uv tool so the scripts land on `PATH` and work from any directory in any new shell:

cd gemini-model-router
uv tool install -e .
uv tool update-shell           # one-time PATH fix; open a new terminal after

Alternative — editable install into a project venv (you must activate the venv or add `.venv/bin` to `PATH` to use `router`):

uv pip install -e ".[dev]"
source .venv/bin/activate

This wires four console scripts: `router`, `router-repl`, `router-eval`, `router-config`. See [`docs/cli.md`](docs/cli.md#install) for extras and upgrade flow.

Prerequisites

• Local vLLM serving Gemma 4 on localhost:8000:

  vllm serve google/gemma-4-E4B-it --port 8000

• gemini CLI on PATH, with GEMINI_API_KEY (or Google Sign-in) configured.
• claude CLI on PATH, with ANTHROPIC_API_KEY configured.
• (Optional) HF_TOKEN for HuggingFace — silences the unauthenticated-Hub warning when the MiniLM anchor cache is rebuilt.

Override any of these in `config/router.yaml`. Or drop them in a `.env` (see `.env.example`); the router auto-loads it on startup, never overriding already-exported shell vars.

Usage

**One-shot mode:**

router "what is the capital of France"
router "refactor src/auth/login.py to use async/await" --why
router --force gemini "search the web for the latest k8s release notes"

**Chat mode (TUI)** — `router` with no prompt at a TTY drops into a chat REPL with slash-command completion, a live status t