Laguna S Vllm — Development skill for Claude Code

Laguna-S 2.1 NVFP4 on one RTX PRO 6000

This repository is the reproducible setup for serving [`poolside/Laguna-S-2.1-NVFP4`](https://huggingface.co/poolside/Laguna-S-2.1-NVFP4) with its [`poolside/Laguna-S-2.1-DFlash-NVFP4`](https://huggingface.co/poolside/Laguna-S-2.1-DFlash-NVFP4) draft on a single 96 GB RTX PRO 6000 Blackwell GPU. It pins vLLM 0.25.1, uses the native NVFP4 path, and includes the single-request FlashInfer patch that removed the main host synchronization bubble in our profiles.

The default is deliberately narrow: one request at a time, a 262,144-token context, DFlash K=3, thinking enabled, and Poolside's recommended temperature 0.7 / top-p 0.95 sampling defaults.

See [BENCHMARK.md](BENCHMARK.md) for the no-speculation versus DFlash benchmark.

Requirements

• Linux on an NVIDIA Blackwell GPU with enough memory for roughly 71 GB of target weights plus the draft, CUDA graphs, and KV cache. This setup was tested on an RTX PRO 6000 Blackwell Workstation Edition with 96 GB.
• A CUDA 13-capable NVIDIA driver and CUDA JIT toolchain.
• Python 3.12 headers, uv, patch, and nvidia-smi.
• Claude Code only if you want to use the optional claude-laguna launcher.

On Ubuntu, install the non-Python prerequisites first:

sudo apt-get update
sudo apt-get install -y python3.12-dev patch
curl -LsSf https://astral.sh/uv/install.sh | sh

Install and serve

From a fresh clone:

./scripts/setup.sh
./scripts/serve.sh

`setup.sh` creates `.venv`, installs vLLM 0.25.1 against the CUDA 13.0 PyTorch wheels, installs the matching FlashInfer nightly trio, and applies [`patches/vllm-0.25.1-single-request-gpu-metadata.patch`](patches/vllm-0.25.1-single-request-gpu-metadata.patch) to the installed vLLM package. It is safe to rerun.

Validate an existing installation without changing it:

./scripts/setup.sh --check

The first serve downloads the model weights and builds any missing JIT kernels. Keep `MAX_JOBS=4` on a cold Fl