ReScienceLab

LightGUIAgent — Design skill for Claude Code

Design community

Lightweight GUI Automation Agent with Grid-Based Visual Grounding.

How to install LightGUIAgent

This entry records only its repository, not the path inside it, so there is no exact command to give. Open ReScienceLab/LightGUIAgent and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What LightGUIAgent does

Lightweight GUI Automation Agent with Grid-Based Visual Grounding.

Alternatives in Design

Brand Guidelines — Apply Anthropic's official brand colors and typography to artifacts 94.1k ★
Weather Orchestrator Command — Fetch the current temperature for Dubai, UAE and create a visual SVG weather card 20.2k ★
NotebookLM Automation — Complete programmatic access to Google NotebookLM—including capabilities not exposed in the web UI 6.8k ★

README

LightGUIAgent

Lightweight GUI Automation Agent with Grid-Based Visual Grounding

**A lightweight GUI automation agent based on grid coordinate system and Claude Opus 4.5**

[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/) [![uv](https://img.shields.io/badge/uv-managed-green.svg)](https://github.com/astral-sh/uv) [![Discord](https://img.shields.io/badge/Discord-Join%20Us-5865F2?logo=discord&logoColor=white)](https://discord.gg/3BmDnDPcM6) [![WeChat](https://img.shields.io/badge/WeChat-Community-07C160?logo=wechat&logoColor=white)](#wechat-community) [![简体中文](https://img.shields.io/badge/文档-简体中文-red.svg)](README_ZH.md)

**Core Innovation**: Divides the screen into a 10×20 grid (similar to chess notation), allowing the model to output "E5" instead of complex pixel coordinates.

Key Features

Grid Coordinate System: 10×20 grid (A-J × 1-20) instead of pixel coordinates
Claude Opus 4.5: State-of-the-art vision model for UI understanding
Visual Context Memory: Includes previous step's marked screenshot for better decision-making
Multilingual Support: Supports Chinese, English, and other languages with perfect input handling
Fast: 5-8s per step (vs 24-30s with local models)
Lightweight: 200MB RAM (vs 10GB for local models)
Easy Deploy: No GPU needed, just Python + API key

Prerequisites

Python 3.12+
uv package manager
Android device with USB debugging enabled
ADB (Android Debug Bridge)
Claude API key from Anthropic

Quick Start

1. Install Dependencies

# Clone the repository
git clone https://github.com/ReScienceLab/LightGUIAgent.git
cd LightGUIAgent

# Install dependencies (creates .venv automatically)
uv sync

# Or use the convenience command
make dev