Kaggle Tpu Lab — Development skill for Claude Code

kaggle-tpu-lab

**Serve Qwen3.8-27B — a frontier-class 27B hybrid-attention model — on Kaggle's free TPU v5e-8, with a public OpenAI-compatible endpoint you can plug into Claude Code, Codex CLI, opencode, or anything else that speaks the OpenAI API.**

No paid GPU, no cloud account, no quantization. Full bf16 weights, up to the model's native **262,144-token context**, and real speed:

What	Measured (TPU v5e-8, bf16, TP=8)
Decode, single stream	~130 tok/s with MTP speculative decoding (78 without, measured before the token-bucket change)
Decode, 8 concurrent streams	~540 tok/s aggregate (~107 tok/s each, MTP on)
Decode, 16 concurrent streams	~900 tok/s aggregate (with --mtp 0 — see tuning note below)
Prefill	10,300 tok/s — a 105k-token prompt in ~10 s
Native 262k context	works — 225k-token prompt prefilled in ~28 s
Time to live endpoint	~22 min with the env dataset attached (~12 with --text-only, ~6 with --fast-start; ~40 without the dataset)
Output correctness with MTP	verified lossless — 12/12 greedy prompts exactly match non-speculative

Tuning note: speculative decoding pays off up to ~8 concurrent streams and fades beyond that (verification competes with batch compute). Serving many users? Launch with `--max-model-len 131072 --max-num-seqs 16 --mtp 0` for max aggregate throughput.

Why this works (the one-paragraph version)

Qwen3.8-27B is a hybrid: 48 of its 64 layers are **gated-DeltaNet linear attention**, only 16 are classic full attention. That makes its KV cache tiny (~64 KB/token), which is why a 27B can serve 131k+ contexts on 8×16 GB TPU chips with room to spare. Until recently no TPU stack could run the DeltaNet layers — [vllm-tpu](https://github.com/vllm-project/tpu-inference) 0.28.0 shipped native Pallas kernels for them (Aug 2026), and this repo is the recipe that puts it all together on Kaggle's free tier: a pre-built Python runtime with pinned versions, p