Flow Llm

Flow LLM

**Local LLM gateway for Apple Silicon**

Run GGUF and MLX models locally. Proxy OpenAI & Anthropic API requests. Real-time monitoring. Built for coding agents.

[](LICENSE) [](https://www.python.org/downloads/) [](https://support.apple.com/en-us/HT211814) [](https://buymeacoffee.com/aitamedia)

[Install](#quick-install) · [Features](#features) · [Quick Start](#quick-start) · [API](#api-endpoints) · [Architecture](docs/architecture.md)

Flow LLM is a local LLM gateway for macOS. It manages GGUF and MLX models on Apple Silicon, proxies OpenAI- and Anthropic-compatible API requests, and exposes real-time monitoring — so tools like **OpenClaw**, **[Hermes](https://github.com/nousresearch/hermes-agent)**, **Claude Code**, and **Codex** (via [AIRun](https://github.com/andisearch/airun)) can talk to local models without Ollama or LM Studio.


Features

• JIT Model Loading — Models auto-load on first request and auto-unload after idle cooldown. Togglable in Settings (defaults ON, 5 min cooldown). Circuit breaker prevents memory exhaustion
• Real-time Monitor — Per-request lifecycle tracking (queued → prefilling → generating → completed), odometer-style token counter, WebSocket push, idle waveform
• OpenAI & Anthropic APIs — Drop-in proxy for /v1/chat/completions and `/v1