End2end Sample

AgentOS 2 Live - Real-time Voice Assistant

🚀 Overview

**AgentOS 2 Live** is a high-performance, low-latency real-time voice assistant platform built with **OpenAI Realtime API**. It provides an end-to-end solution for building stateful AI assistants with voice-to-voice capabilities, integrated Voice Activity Detection (VAD), and a modular tool-calling system.

This project is structured as a **Monorepo** using npm workspaces, containing both the frontend client and the backend orchestration server.

Core Tech Stack

Layer	Technology
Architecture	Monorepo (npm workspaces)
Frontend	React, TypeScript, Tailwind CSS, Web Audio API, VAD (Voice Activity Detection), Opus Codec
Backend	Node.js, TypeScript, Express, WebSocket
AI Model	OpenAI Realtime API (GPT-4o Realtime)
Communication	WebSocket with a unified protocol for type safety

📂 Project Structure

Directory	Description
client/	Frontend: React application featuring the AgentSDK, VAD integration, and real-time UI components like the Robot Face.
server/	Backend: Node.js service managing WebSocket connections, OpenAI Realtime API orchestration, and static file serving.
shared/	Shared: Common TypeScript types and the communication protocol used by both client and server.
e2e_android/	Android: A WebView bridge project for OrionStar robots, exposing hardware capabilities (navigation, sensors) to the web frontend.

Key Features

• Low Latency: Direct voice-to-voice interaction using OpenAI's Realtime protocol.
• Robot Face UI: An interactive, animated robot face that reacts to user and assistant speech.
• Built-in Scenes:
  • Face Register: A specialized scene for face enrollment and identity recognition.
  • Advice 3C: An IT & Electronics shopping assistant scenario.
• Robot Hardware Integration: Includes an Android WebView bridge (`e2e_andr