tikoci

Bench Routeros Tools — Development skill for Claude Code

Development community

Benchmark RouterOS agent-support strategies across MCPs, skills, and retrieval.

How to install Bench Routeros Tools

This entry records only its repository, not the path inside it, so there is no exact command to give. Open tikoci/bench-routeros-tools and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Bench Routeros Tools does

Benchmark RouterOS agent-support strategies across MCPs, skills, and retrieval.

Alternatives in Development

Monetization Strategy — Brainstorm 3-5 monetization strategies with validation experiments 7.8k ★
Mcporter — Call MCPs via TypeScript, masquerading as simple TypeScript API 4.1k ★
AgentSkillOS — Build your agent from 200,000+ skills via skill RETRIEVAL & ORCHESTRATION 373 ★

README

RouterOS Agent-Support Benchmark

A repeatable harness that measures **how best to help an AI agent work with MikroTik RouterOS**. RouterOS training data is sparse and its CLI is idiosyncratic, so several augmentation strategies exist. This benchmark grounds the choice between them in numbers instead of intuition.

**This is a _structural_ benchmark, not an "effectiveness winner" benchmark.** Without a model API wired in, it does not auto-run end-to-end agent success. It measures the structural properties that determine effectiveness (context cost, tool-selection burden, retrieval coverage, command validity, capability, safety) and ships a pluggable harness so live-agent success can be added later. A small **live pilot** (`claude -p`) is now wired in on top of that. See [`REPORT.md`](REPORT.md) for the analysis and the tradeoff-framed recommendation.

TL;DR — results at a glance (2026-05-31)

A 10,000-foot view of what the data says. Full analysis + caveats in [`REPORT.md`](REPORT.md); every claim traces to a CSV under `data/`.

No single winner — it's a cost/capability tradeoff. Compare within a capability tier (knowledge vs execution), not across.
For read-only/planning, rosetta is the efficient frontier: ~6.3K always-on tokens buys retrieval that surfaces the right command for 89% of tasks (structural) and reconstructs the gold path for 100%. Read-only, version-aware, safe.
For live mutation, `mikrotik-mcp` is uniquely capable but heavy: its 166 SSH tools cost ~28K always-on tokens (4.5× rosetta, 25× skills), an avg 36/166 tools lexically match each task, the right tool is a clear top-3 pick only 62% of the time, and 27 tools are destructive with no dry-run. Token cost is the visible symptom; tool-selection ambiguity and destructive-tool proximity are the deeper risk.
**Don't stack everyt