YuhuanStudio

Yunshu — AI skill for Claude Code

AI community

Fast local LLM / VLM inference engine for Apple Silicon (MLX).

How to install Yunshu

This entry records only its repository, not the path inside it, so there is no exact command to give. Open YuhuanStudio/Yunshu and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Yunshu does

Fast local LLM / VLM inference engine for Apple Silicon (MLX). OpenAI- and Anthropic-compatible, lossless speculative decoding (DFlash2 / MTP), hybrid-model prefix cache with RAM/SSD tiers, native Claude Code / Codex / opencode support. Qwen3.8-27B tuned first.

Alternatives in AI

Vllm Mlx — High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon 1.5k ★
Mac Code — mac code — Claude Code, but it runs on your Mac for free 905 ★
Wax — Shared Single-file memory layer for all your agents, sub mili-second RAG over text, photo and video on Apple S 791 ★

README

Yunshu

**A fast local LLM / VLM inference engine for Apple Silicon.**

One process, OpenAI- and Anthropic-compatible, running on-device via MLX. Built for low latency on a single Mac: fast first token, fast lossless decode, and prefix reuse that skips work you already paid for. The first fully tuned model is **Qwen3.8-27B**.

[![Python 3.13+](https://img.shields.io/badge/python-3.13+-blue.svg)](https://www.python.org/downloads/) [![License: Apache 2.0](https://img.shields.io/badge/license-Apache%202.0-green.svg)](LICENSE) [![CI](https://github.com/YuhuanStudio/Yunshu/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/YuhuanStudio/Yunshu/actions/workflows/ci.yml)

**English** · [简体中文](README.zh-CN.md) · [繁體中文](README.zh-TW.md)

Yunshu console overview: live request phases, tok/s, prefix-cache hits

Highlights

Lossless speculative decoding — DFlash2 or MTP drafting with batch-invariant kernels: greedy output with speculation on is token-identical to speculation off.
Prefix cache for hybrid models — exact checkpoints for attention + GatedDeltaNet models, in RAM and on SSD, surviving restarts; optional extra storage tiers.
The full API on the fast path — tools, JSON schema, stop, logprobs, reasoning, cancellation, across OpenAI Chat / Responses, Anthropic Messages and Ollama.
Typed decisions — /v1/decisions answers yes/no, choice and score questions with calibrated probabilities in one forward pass of a decision checkpoint, without generating text.
Native coding-agent support — Claude Code, Codex and opencode work through their own APIs, including server-side web search / fetch and MCP.
Lossless by default — anything that can change output is an explicit setting.
**Local and private