Computer Use — Development skill for Claude Code
Computer use for Grok Build: a plugin and MCP server that let the agent see the screen, click and type in desktop apps and browsers on macOS, Windows and Linux.
How to install Computer Use
This entry records only its repository, not the path inside it, so there is no
exact command to give. Open bo-516/computer-use and copy the folder into
~/.claude/skills/, or the file into ~/.claude/agents/.
What Computer Use does
Computer use for Grok Build: a plugin and MCP server that let the agent see the screen, click and type in desktop apps and browsers on macOS, Windows and Linux. Accessibility-tree first, with safety hooks. Built for UI testing in the dev loop.
Alternatives in Development
- Let Fate Decide — Trail of Bits security skill for let fate decide 3.7k ★
- 3D Portfolio — Let's build a cool 3D website together 1.3k ★
- Skill.Color Expert — Agent skill for color science expertise 426 ★
README
Grok Computer
**English** | [简体中文](README.zh-CN.md)
Computer use for Grok Build: a Grok Build plugin (`computer-use`) plus a purpose-built MCP server (the `grok-computer-mcp` facade) that let Grok Build see the screen, click and type in desktop apps and browsers on macOS, Windows and Linux. The main use is checking UI inside the development loop (change code → launch the app → look at the UI → change again), and operating desktop and web tools that have no API.
Design document: [docs/goal.md](docs/goal.md). Installation and configuration: [docs/install.md](docs/install.md). The documents under `docs/` are written in Chinese.
How it works
main agent ──delegates──▶ computer subagent (own context; only reports go back)
│
├─▶ computer: grok-computer-mcp facade ──▶ Cua Driver ──▶ desktop (host / sandbox)
└─▶ browser: Playwright MCP (--isolated)
hooks: guard (risk grading and confirmation) · audit · verify_gate (report and final check) · selfcheck
- Screenshots stay out of the main session: all GUI work goes to the
computer-use:computersubagent; the main agent only receives a structured report of at most 2 KB (STATUS / SUMMARY / EVIDENCE / BLOCKERS / NEXT). - Accessibility tree first, pixels last: an observation is a compact list of elements by default (one interactive element per line, with a stable ref). When the tree is not enough it falls back to a numbered Set-of-Mark screenshot, and only then to a grounding model or raw coordinates.
- One coordinate space: every coordinate the model sees or sends is a pixel of the current observation image (long edge 1280 px). HiDPI scaling, window offsets and multiple displays are all handled inside the facade, and an action based on an observation of a screen that has since changed is refused (
STALE_OBSERVATION). - Two safety layers: the hooks ask you before high-risk actions (send, delete, pay, ...), and t
Related Skills
Device Control Agent
You are an autonomous macOS agent with full control of this device via two CLI tools: - steer — GUI automation
Quick Computer Use
QCU (Quick Computer Use) — a faster computer-use skill that prefers the accessibility tree over screenshots. W
Protonctl
Local, read-only MCP server and CLI for Proton Mail, Proton Drive and Proton Calendar, in Rust, for macOS and
Dev Routes
List all registered _dev/ routes / screens for a product (Flask blueprint, RN screen tree, or desktop dev menu
Glance
The window, as text, as it changes: reads a window through its own accessibility tree, or through pixels and t
Turn Anything Cu
CCU (Claude Computer Use): let Claude Code use your Mac via ChatGPT's Computer Use, set up in one sentence. 让
Related Agents
Computer Use Operator
Desktop automation specialist. Controls native macOS apps via Computer Use MCP. Handles Finder, System Setting
Byok Token Security Expert
BYOK (Bring-Your-Own-Key) and OAuth token security expert. Deep on cross-platform OS keychain integration (mac
Slim Wrapper
tfx-route.sh 경유 전용 래퍼 에이전트. 코드를 직접 읽거나 수정하지 않고 Bash(tfx-route.sh)를 통해 Codex/Antigravity에 위임하는 thin wrapper. OS