Surreal Commands

Surreal Commands

A distributed task queue system similar to Celery, built with Python, SurrealDB, and LangChain. This system allows you to define, submit, and execute asynchronous commands/tasks with real-time processing capabilities.

Features

• Real-time Processing: Uses SurrealDB LIVE queries for instant command pickup
• Concurrent Execution: Configurable concurrent task execution with semaphore controls
• Type Safety: Pydantic models for input/output validation
• LangChain Integration: Commands are LangChain Runnables for maximum flexibility
• Dynamic CLI: Auto-generates CLI from registered commands
• Status Tracking: Track command status through lifecycle (new � running � completed/failed)
• Persistent Queue: Commands persist in SurrealDB across worker restarts
• Comprehensive Logging: Built-in logging with loguru

Architecture Overview

graph TD
    CLI[CLI Interface] --> SurrealDB[(SurrealDB Queue)]
    SurrealDB --> Worker[Worker Process]
    Worker --> Registry[Command Registry]
    Registry --> Commands[Registered Commands]
    Worker --> |Execute| Commands
    Commands --> |Results| SurrealDB

Installation

Install the package using pip:

pip install surreal-commands

Set up environment variables in `.env`:

# SurrealDB Configuration
SURREAL_URL=ws://localhost:8000/rpc
SURREAL_USER=root
SURREAL_PASSWORD=root
SURREAL_NAMESPACE=test
SURREAL_DATABASE=test

1. Ensure SurrealDB is running:

# Using Docker
docker run --rm -p 8000:8000 surrealdb/surrealdb:latest start --user root --pass root

# Or install locally
# See: https://surrealdb.com/install

Quick Start

1. Define Commands

Create your commands using the `@command` decorator:

# my_app/tasks.py
from surreal_commands import command, submit_command
from pydantic import BaseModel

class ProcessInput(BaseModel):
    message: str
    uppercase: bool = False

class ProcessOutput(BaseModel):
    res