Blackman99

Toolsmoke — AI skill for Claude Code

AI community

Tell you in one command whether your LLM endpoint actually works for agents (tool calling, streaming, structured output) and how fast it is.

How to install Toolsmoke

This entry records only its repository, not the path inside it, so there is no exact command to give. Open Blackman99/toolsmoke and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Toolsmoke does

Tell you in one command whether your LLM endpoint actually works for agents (tool calling, streaming, structured output) and how fast it is.

Alternatives in AI

Vllm Mlx — High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon 1.5k ★
Cersei — The Rust SDK for building coding agents 274 ★
Svelte Markdown — 📝 Markdown and HTML renderer for Svelte 5 — built for streaming AI agent output from Claude Code, ChatGPT, an 131 ★

README

toolsmoke

**Tell you in one command whether your LLM endpoint actually works for agents (tool calling, streaming, structured output) and how fast it is.**

toolsmoke --demo broken finds 13 failing probes; --demo good passes and prints AGENT-READY

Every "OpenAI-compatible" server answers a plain chat request. Agents need more: structured `tool_calls` instead of `` text in `content`, `tool_choice` that is enforced, streamed tool-call deltas that assemble into valid JSON, JSON-schema output, reasoning kept out of the answer, tool results that survive the chat template. When one of these is broken (see [ollama#11621](https://github.com/ollama/ollama/issues/11621), [vllm#22403](https://github.com/vllm-project/vllm/issues/22403), [llama.cpp#15012](https://github.com/ggml-org/llama.cpp/issues/15012)), your agent fails somewhere far from the cause. toolsmoke runs 31 small probes against your endpoint (OpenAI chat completions, optionally Anthropic Messages) and gives you a pass/warn/fail verdict, time-to-first-token, tokens/sec and an exit code.

Install

uvx --from git+https://github.com/Blackman99/toolsmoke toolsmoke --help     # run without installing
pipx install git+https://github.com/Blackman99/toolsmoke                   # or install the command

Zero dependencies, Python 3.9+. Wheels are on the [releases page](https://github.com/Blackman99/toolsmoke/releases). Try it without a model: `toolsmoke --demo broken`.

Usage

Real output against llama.cpp (`llama-server --jinja`, b11490) serving Qwen2.5-0.5B-Instruct on 8 CPU cores:

$ toolsmoke --base-url http://localhost:8080/v1 --model qwen --anthropic
  PROBE                   RESULT    TIME  DETAIL
  models.list             PASS       1ms  1 model(s) listed, 'qwen' present
  chat.basic              PASS      81ms  fi