Turboquant Skill — AI skill for Claude Code

TurboQuant Skill — LLM KV Cache Compression for AI Coding Agents

[](LICENSE) [](https://github.com/Ryuketsukami/turboquant-skill/actions/workflows/tests.yml) [](https://github.com/anthropics/skills) [](https://python.org) [](https://arxiv.org/abs/2504.19874)

**An AI agent skill implementing Google's TurboQuant compression algorithm** — the data-oblivious vector quantization framework that reduces LLM KV cache memory by **6x** and delivers up to **8x speedup** with **zero accuracy loss**. Compatible with Claude Code, Codex CLI, and all agents supporting the [Agent Skills specification](https://github.com/anthropics/skills).

**Looking for the standalone Python library?** See [turboquant-compression](https://github.com/Ryuketsukami/turboquant-compression) — pip-installable with 27 tests and full documentation.

---

What is TurboQuant?

**TurboQuant** is a compression algorithm introduced by Google Research (Zandieh et al.) at **ICLR 2026** that solves the primary memory bottleneck in large language model inference: the key-value (KV) cache.

During text generation, transformers store computed key and value vectors for every token in the context window. For long-context applications (100K+ tokens), this KV cache can consume **tens of gigabytes of GPU memory** — often more than the model weights themselves.

TurboQuant compresses each KV cache vector from 32-bit floats down to **3-4 bits per coordinate** — without retraining, without calibration data, and without measurable accuracy loss.

Key Results (from the paper)

Metric	Result
Memory reduction	6x