tavlant

Realtime Agent Backend — Data skill for Claude Code

Data community

Production-style backend for real-time AI chat agents: FastAPI, WebSocket token streaming, Redis + PostgreSQL memory, per-conversation locking, tool calling, pgvector RAG, Claude & OpenAI adapters.

How to install Realtime Agent Backend

This entry records only its repository, not the path inside it, so there is no exact command to give. Open tavlant/realtime-agent-backend and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Realtime Agent Backend does

Production-style backend for real-time AI chat agents: FastAPI, WebSocket token streaming, Redis + PostgreSQL memory, per-conversation locking, tool calling, pgvector RAG, Claude & OpenAI adapters. Runs offline with a fake LLM.

Alternatives in Data

Private GPT — Complete API layer for private AI applications on local models: RAG, skills, tools, MCP, text-to-sql, and more 57.5k ★
LLM Redteam — LLM red-team corpus runner — fires categorized prompt-injection / jailbreak / system-prompt-leak / data-exfil 4.5k ★
Azure Event Hub .net — High-throughput event streaming 1.8k ★

README

realtime-agent-backend

[![CI](https://github.com/tavlant/realtime-agent-backend/actions/workflows/ci.yml/badge.svg)](https://github.com/tavlant/realtime-agent-backend/actions/workflows/ci.yml) ![Python](https://img.shields.io/badge/python-3.10%2B-blue) ![License](https://img.shields.io/badge/license-MIT-green)

A production-style **FastAPI backend for LLM chat agents**: assistant replies stream token by token over WebSockets, the agent can call tools, conversations have durable memory, and a small RAG pipeline answers questions from your own documents.

It is built the way I'd build this for real traffic: async end to end, safe under concurrency, resilient to slow or flaky model providers, observable, and **fully runnable offline**. No API key is needed to run it, test it or load-test it.

**Highlights**

Streaming over WebSockets. /ws/conversations/{id} streams tokens, tool calls and tool results as typed JSON events.
Two-tier memory. Recent context lives in Redis (with a TTL); Postgres keeps the full history and is the source of truth. Older turns are folded into a rolling summary once the context goes over a token budget.
No interleaved turns. A per-conversation distributed Redis lock (random owner token, TTL plus watchdog renewal) keeps turns in order across tabs and replicas.
Tool calling. A typed tool registry (Pydantic-validated arguments, timeouts, error isolation) with a safe AST calculator, a clock and knowledge-base search. The agent loop runs tool calls concurrently and feeds the results back to the model.
Mini RAG. Ingest Markdown or text, then chunk (heading-aware), embed, store in pgvector (HNSW index) and search semantically. An in-memory store is used for dev and tests.
Provider abstraction. Anthropic and OpenAI adapters (streaming plus tool calls) sit behind one interface, plus a deterministic FakeLLM for tests and offline use.
Reliability. First-token and idle timeouts, retries