AI SDK Computer Use Demo

An open-source AI chatbot demonstrating computer use capabilities with Anthropic Claude Sonnet 4.5, Vercel Sandboxes, and the AI SDK by Vercel.

Features · How It Works · Deploy Your Own · Running Locally

Features

• Streaming text responses powered by the AI SDK.
• Anthropic Claude Sonnet 4.5 with computer use and bash tool capabilities.
• Remote desktop environment running in a Vercel Sandbox with Chrome, a window manager, and VNC streaming.
• shadcn/ui components for a modern, responsive UI powered by Tailwind CSS.
• Built with the latest Next.js App Router.

How It Works

The app spins up a Vercel Sandbox from a pre-built snapshot that includes:

• Xvnc — a virtual X11 display server
• openbox — a lightweight window manager
• noVNC + websockify — streams the desktop to the browser via WebSocket
• Google Chrome — auto-launched so the AI agent has a browser ready
• xdotool + ImageMagick — for mouse/keyboard control and screenshots

When a user sends a message, Claude uses the `computer` tool (screenshot, click, type, scroll) and the `bash` tool (run shell commands) to interact with the sandbox desktop. The noVNC stream is displayed in a resizable iframe alongside the chat.

Architecture

User ↔ Next.js Chat UI ↔ AI SDK ↔ Claude Sonnet 4.5
                                        ↓
                                  Vercel Sandbox
                              ┌─────────────────────┐