nilltadios

Gnome Computer Use — Development skill for Claude Code

Development community

Claude Code skill for GNOME Wayland: mouse/keyboard without root (Mutter RemoteDesktop EIS), windows and sharp screenshots via a small shell extension, dialog-free recording (Mutter ScreenCast), contr.

How to install Gnome Computer Use

This entry records only its repository, not the path inside it, so there is no exact command to give. Open nilltadios/gnome-computer-use and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Gnome Computer Use does

Alternatives in Development

Memory Recording — 写作记忆记录 skill 669 ★
Package Assistant — Package OfficeCLI Skill as Wayland Assistant 589 ★
Contributing To Plane MCP Server — Thank you for showing an interest in contributing to Plane MCP Server 192 ★

README

gnome-computer-use

A [Claude Code](https://claude.com/claude-code) skill for controlling a **GNOME Wayland** desktop: mouse and keyboard, window control, screenshots, screen recording, and finding UI controls by name (accessibility) or by text (OCR). No root, no `ydotool`/uinput, no pip packages and no permission dialogs. It is the GNOME counterpart of [kde-computer-use](https://github.com/nilltadios/kde-computer-use), with the same commands.

Script	Does	How
`scripts/gnome-input.py`	move, click (left/right/middle, double), drag, scroll, key combos, typing, Shift/Ctrl-click	Mutter RemoteDesktop D-Bus API hands out a libei socket (`ConnectToEIS`); libei via ctypes
`scripts/gnome-window.py`	list, activate, minimize, maximize, fullscreen, keep-above, move, resize, close, wait-stable	small GNOME Shell extension (`extension/`) answering over D-Bus
`scripts/gnome-capture.py`	screenshots of the desktop, a monitor, an area or a window; recordings of the same	`Shell.Screenshot` in the extension; Mutter ScreenCast → PipeWire → GStreamer
`scripts/gnome-a11y.py`	list controls, find them by name, press them without the mouse	AT-SPI via PyGObject (`gi.repository.Atspi`)
`scripts/gnome-ocr.py`	find text on screen and where it is	tesseract
`scripts/gnome-learn.py`	add, list and check lessons in `LEARNINGS.md`	plain Python
`scripts/setup.sh`, `scripts/load-extension-live.py`	install the extension; load it without logging out	gsettings; Looking Glass driven by `gnome-input.py`

`SKILL.md` is what Claude reads: when to use which tool, safety rules, and every command. `LEARNINGS.md` holds verified lessons agents record while using the skill; `LEARNINGS.local.md` (`gnome-learn.py add --local`) is git-ignored, for this machine only.

Why an extension

GNOME Shell refuses `org.gnome.Shell.Introspect.GetWindows`, `org.gnome.Shell.Screenshot` and `Eval` to outside processes, and there is no KWin-style script