jhammant

AIonDemandCluster — AI skill for Claude Code

AI community

Spin up any open LLM on rented GPUs (vast.ai/RunPod), serve it with vLLM or llama.cpp, and drive it from Claude Code via Claude Code Router.

How to install AIonDemandCluster

This entry records only its repository, not the path inside it, so there is no exact command to give. Open jhammant/AIonDemandCluster and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What AIonDemandCluster does

Spin up any open LLM on rented GPUs (vast.ai/RunPod), serve it with vLLM or llama.cpp, and drive it from Claude Code via Claude Code Router. Pay by the hour.

Alternatives in AI

Repomix — 📦 Repomix is a powerful tool that packs your entire repository into a single, AI-friendly file 22.7k ★
Inference Serving — ai-research-skills vLLM, SGLang, TensorRT-LLM, llama.cpp 5.4k ★
Vllm Mlx — High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon 1.5k ★

README

AI on Demand Cluster (`aiod`)

Spin up any HuggingFace model on a **vast.ai** GPU, serve it with **vLLM**, and drive it from **Claude Code** through **Claude Code Router (CCR)** — all from one command.

Give it a HuggingFace link; `aiod` figures out how much VRAM the model needs, finds the cheapest matching machine on vast.ai, shows you the live **$/hr** before you commit, rents it, waits for the model to load, and writes your CCR config so you can run `ccr code` against your own hosted model.

  HuggingFace link
        │  (params, dtype, KV shape, context)
        ▼
  ┌─────────────┐   estimate VRAM per quant      ┌──────────────────────┐
  │   sizing    │ ─────────────────────────────▶ │  live vast.ai offers  │  ← real $/hr
  └─────────────┘                                 └──────────────────────┘
        │  rent cheapest fit
        ▼
  vast.ai GPU box ──▶ vLLM (OpenAI-compatible /v1) ──▶ model weights
        ▲
        │  http://:/v1/chat/completions  (+ bearer token)
        │
  Claude Code Router (local)  ◀── translates Anthropic ⇄ OpenAI
        ▲
        │
  Claude Code  (ccr code)

Why CCR sits in the middle: Claude Code speaks the **Anthropic** Messages API, while vLLM serves the **OpenAI** API. CCR runs locally and translates between them, so the remote box can stay a plain OpenAI-compatible server.

Prerequisites

**Python 3.10+**
A **vast.ai** account ([sign up](https://cloud.vast.ai/?ref_id=25480)) + an API key from [Account → API Keys](https://cloud.vast.ai/manage-keys/) (required)
A **HuggingFace token** — only for gated/private models like Llama/Gemma (optional)
**Claude Code Router (classic CLI)** installed locally:
```
npm install -g @musistudio/claude-code-router
```
Note: the project has a Desktop app (v3) that stores config under `~/Library/Application Support/...`. `aiod` targets the **classic CLI**, whose config lives at `~/.claude-code-router/config.json` and uses `ccr