bo-516

Computer Use — Development skill for Claude Code

Development community

Computer use for Grok Build: a plugin and MCP server that let the agent see the screen, click and type in desktop apps and browsers on macOS, Windows and Linux.

How to install Computer Use

This entry records only its repository, not the path inside it, so there is no exact command to give. Open bo-516/computer-use and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Computer Use does

Computer use for Grok Build: a plugin and MCP server that let the agent see the screen, click and type in desktop apps and browsers on macOS, Windows and Linux. Accessibility-tree first, with safety hooks. Built for UI testing in the dev loop.

Alternatives in Development

README

Grok Computer

**English** | [简体中文](README.zh-CN.md)

Computer use for Grok Build: a Grok Build plugin (`computer-use`) plus a purpose-built MCP server (the `grok-computer-mcp` facade) that let Grok Build see the screen, click and type in desktop apps and browsers on macOS, Windows and Linux. The main use is checking UI inside the development loop (change code → launch the app → look at the UI → change again), and operating desktop and web tools that have no API.

Design document: [docs/goal.md](docs/goal.md). Installation and configuration: [docs/install.md](docs/install.md). The documents under `docs/` are written in Chinese.

How it works

main agent ──delegates──▶ computer subagent (own context; only reports go back)
                            │
                            ├─▶ computer: grok-computer-mcp facade ──▶ Cua Driver ──▶ desktop (host / sandbox)
                            └─▶ browser: Playwright MCP (--isolated)
hooks: guard (risk grading and confirmation) · audit · verify_gate (report and final check) · selfcheck
  • Screenshots stay out of the main session: all GUI work goes to the computer-use:computer subagent; the main agent only receives a structured report of at most 2 KB (STATUS / SUMMARY / EVIDENCE / BLOCKERS / NEXT).
  • Accessibility tree first, pixels last: an observation is a compact list of elements by default (one interactive element per line, with a stable ref). When the tree is not enough it falls back to a numbered Set-of-Mark screenshot, and only then to a grounding model or raw coordinates.
  • One coordinate space: every coordinate the model sees or sends is a pixel of the current observation image (long edge 1280 px). HiDPI scaling, window offsets and multiple displays are all handled inside the facade, and an action based on an observation of a screen that has since changed is refused (STALE_OBSERVATION).
  • Two safety layers: the hooks ask you before high-risk actions (send, delete, pay, ...), and t