Bonsai AI Deploy — DevOps skill for Claude Code
Deploy Bonsai-2-27B (ternary 27B, 5.95GB) on a single 8GB consumer GPU via an AI agent.
How to install Bonsai AI Deploy
This entry records only its repository, not the path inside it, so there is no
exact command to give. Open Zephyr236/bonsai-ai-deploy and copy the folder into
~/.claude/skills/, or the file into ~/.claude/agents/.
What Bonsai AI Deploy does
Deploy Bonsai-2-27B (ternary 27B, 5.95GB) on a single 8GB consumer GPU via an AI agent. 用 AI 部署 Bonsai-2-27B:一条提示词让 AI 装驱动、编 CUDA、下模型并接入 Claude Code。实测 51 tok/s @4K / 39 tok/s @100K,含按机器调参指南与完整测评报告。
Alternatives in DevOps
- Cloudflare Deploy — Deploy apps to Cloudflare using Workers, Pages, and platform services 14.6k ★
- Vinext — Vite plugin that reimplements the Next.js API surface — deploy anywhere 8.7k ★
- Claudable — Claudable is an open-source web builder that leverages local CLI agents, such as Claude Code, Codex, Gemini CL 4k ★
README
bonsai-ai-deploy
[English](README.en.md) | **中文**
**用 AI 部署 Bonsai-2-27B** —— 把一根提示词交给 AI,它自己装驱动、编 CUDA、下模型。
**在单张显卡上部署 [Bonsai-2-27B](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf)(三元量化 27B),并接入 Claude Code。**
模型是 Qwen3.8-27B 的权重全部三元化(−1 / 0 / +1)后的产物:**只有 5.95 GB**, 保留原模型 **98.2%** 的能力 —— 一张 8GB 显卡就能跑,实测生成速度 **51 t/s**。
一眼看全(At a glance)
- 模型:Ternary Bonsai 2 27B —— Qwen3.8-27B 的每个权重三元量化(−1 / 0 / +1)
- 体积:5.95 GB —— 一张 8 GB 显卡就能装下
- 速度:4K 上下文 51 tok/s,100K 上下文 39 tok/s(RTX 3070 实测)
- 质量:保留 FP16 原模型的 98.2%
- 部署方式:给 AI 助手一句话(Claude Code / Cursor / Codex 均可)
- 上下文上限:全 GPU 100K;用
-nkvo可达 262K - 技术栈:PrismML llama.cpp fork + CUDA 12.8 + Anthropic 桥接
怎么用(一句话)
**把 `AI-DEPLOY.md` 发给你的 AI 编码助手**(Claude Code / Cursor / Codex …), 让它照着在这台机器上部署。文档里有分支判断与故障处理,比固定脚本更能应对真实环境。
请照着这个文档帮我部署:AI-DEPLOY.md
或者直接说:
按 https://github.com/Zephyr236/bonsai-ai-deploy 里的 AI-DEPLOY.md 部署 Bonsai-2-27B
部署时 AI 会读 [`docs/PARAM-TUNING.md`](docs/PARAM-TUNING.md), **根据你机器的显存/内存/CPU 和用途**调整参数,而不是套用固定默认值。
仓库内容
| 路径 | 作用 |
|---|---|
AI-DEPLOY.md |
⭐ 核心:给 AI agent 的部署指令(环境探查 → 驱动 → CUDA → 编译 → 下载 → 验证)。含决策分支与故障速查 |
scripts/start-bonsai.sh |
一键启动 / 停止 / 状态。相对路径,整个目录可任意移动 |
scripts/anthropic-bridge.py |
Anthropic API 桥接代理(接入 Claude Code 必需,见下) |
scripts/claude-settings-bonsai.json |
Claude Code 配置 |
docs/PARAM-TUNING.md |
⭐ 参数调优:按显存/内存/CPU/用途调整 llama.cpp 参数,含 KV 显存公式、实测上限逼近法、速度-上下文曲线 |
docs/TROUBLESHOOTING.md |
故障速查 |
**本仓库不含**模型权重(5.95GB)与 llama.cpp 编译产物 —— 二者超出 GitHub 体积限制, 且编译产物与 GPU 架构绑定。部署时由 `AI-DEPLOY.md` 引导下载/编译。
硬件要求
| GPU | NVIDIA RTX 20/30/40/50 系,显存 ≥ 8 GB |
| 驱动 | 580 或更新 |
| 内存 | ≥ 16 GB |
| 磁盘 | ≥ 20 GB 空闲 |
| 系统 | Ubuntu 22.04 / 24.04 |
内网共享(让其他机器也能用)
服务默认监听 `0.0.0.0`,同网段的机器可以直接调用。
# ── 服务端(装了显卡的那台)──
API_KEY=$(openssl rand -hex 16) ./scripts/star
Related Skills
GPU Server Setup
Agent skill that prepares a Linux server with NVIDIA GPUs for LLM workloads - driver and CUDA, Docker with NVI
Gputw Deploy
用 GPUtw API 部署 GPU 執行個體(選型號、選機器、範本或自訂映像、輪詢到 RUNNING、關機)
Neynar Deploy
Deploy websites with a single API call. Built-in versioning with instant rollback. Agent skill for Claude Code
Multi Target Publish
When a single change spans multiple repos / packages / deploy targets with different deploy mechanisms (auto-d
Agent Local Cloud Sync
Push-to-deploy for agent builders and vibe coders. Auto-pull your repo on a single cloud host when you git pus
Openclaw Agents
Deploy multiple AI agents simultaneously in chat groups with a single command using OpenClaw's easy multi-agen
Related Agents
Bonsai
Runs a task on the local Bonsai 2 27B model (PrismML ternary 27B, MLX, on this Mac) and returns its answer. Us
Qe
Performs functional and acceptance testing against the task spec — Julia Pkg.test (CPU always, CUDA/GPU when r
Nvidia Cuda Engineer
You are the system GPU Engineer — a senior NVIDIA platform specialist embedded in the system Distributed AI Ho