Omniverse Agent Studio banner
JinwangMok JinwangMok

Omniverse Agent Studio

AI community

Description

Web-based Agentic AI editor for NVIDIA Isaac Sim — OpenUSD + Blender + Claude tool-use via MCP

Installation

This entry records only its repository, not the path inside it, so there is no exact command to give. Open the source below and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

README

Omniverse Agent Studio

A **web-based Agentic AI editor** for NVIDIA Isaac Sim. Chat with an LLM in your browser and watch it edit an OpenUSD scene live — while Isaac Sim streams the viewport via WebRTC, Blender round-trips assets via USD, and the agent drives both through MCP tools.

 ┌────────────────────────────────────────────────────────────────────────┐
 │                          Your web browser                              │
 │   ┌──────────────┐   ┌────────────────────┐   ┌─────────────────────┐  │
 │   │ Scene graph  │   │   Isaac Sim stream │   │   Chat with agent   │  │
 │   │  /World      │   │  ┌──────────────┐  │   │ > add a red cube    │  │
 │   │   ├ Ground   │   │  │   (WebRTC)   │  │   │ > export to USD     │  │
 │   │   └ Robot    │   │  └──────────────┘  │   │                     │  │
 │   └──────┬───────┘   └─────────┬──────────┘   └──────────┬──────────┘  │
 └──────────┼─────────────────────┼─────────────────────────┼─────────────┘
            │                     │                         │
            ▼ HTTP                ▼ WebRTC 49100/47998      ▼ SSE
       ┌─────────────────────────────────────────────────────────────┐
       │                   Agent (FastAPI + Claude)                  │
       │             invokes MCP tools via streamable-http           │
       └──┬───────────────────────────┬──────────────────────┬───────┘
          │                           │                      │
    ┌─────▼──────────┐        ┌───────▼────────┐     ┌───────▼────────┐
    │  usd-mcp :8001 │        │ blender-mcp    │     │   Isaac Sim    │
    │  (13 tools on  │        │ :8002 (11      │     │   runheadless  │
    │   usd-core)    │        │  tools via     │     │   --livestream │
    │                │        │  socket bridge │     │                │
    └───────┬────────┘        └────┬───────────┘     └────────┬───────┘
            │                      │                          │
            └──────────────────────┼─────────────────