Vnc Use

vnc-use

**LangGraph agent for Computer Use via VNC with Multi-Model LLM Support**

A Python package that enables autonomous desktop interaction through VNC using state-of-the-art Computer Use models (Gemini 2.5 or Anthropic Claude Haiku 4.5). The agent runs a tight *observe → propose → act* loop, grounding actions directly from screenshots without OCR or template matching.

Features

• Multi-model LLM support - Choose between Gemini 2.5 Computer Use or Anthropic Claude Haiku 4.5
• Direct screenshot grounding - No OCR, no template matching (per Computer Use design)
• Generic desktop automation - Works with any VNC-accessible desktop
• Secure credential management - Multi-tenant support with OS keyring, .netrc, or environment variables
• Safety-first HITL - Optional human-in-the-loop gates for risky actions
• Coordinate denormalization - Handles any screen resolution
• LangGraph orchestration - Stateless vision architecture for efficient token usage
• Structured logging - Full run artifacts with screenshots at each step
• Markdown execution reports - Human-readable reports with embedded screenshots showing what the agent saw, thought, and did
• MCP server - FastMCP 2.0 streaming server for AI agent integration

Architecture

The agent implements a continuous observe-propose-act loop:

Task
  ↓
Observe (VNC Screenshot)
  ↓
Propose (LLM: Gemini or Claude)
  ↓
HITL Gate (if confirmation needed)
  ↓
Act (Execute on VNC)
  ↓
Loop until done or timeout

Model Selection

vnc-use supports multiple LLM providers through a pluggable architecture:

Provider	Model	Best For	Speed	Cost
Gemini	gemini-2.5-computer-use-preview-10-2025	Complex tasks, multi-step workflows	Medium	Medium
Anthropic	claude-haiku-4-5-20251015	Fast responses, cost optimization	Fast	Low

**Select via environment variable:**

# Use Anthropic Claude Hai