Part 20: Agent Observability & Infrastructure Services

Part 20: Agent Observability & Infrastructure Services

*See what your agents are actually doing. Connect to 400+ apps. Make search 40% better.*

---

The Observability Problem

Running multiple agents without observability is flying blind. You don't know:

• How much each agent costs per day
• Which agents fail silently
• Where latency bottlenecks are
• Whether your search is returning relevant results

LangFuse — Agent Tracing

[LangFuse](https://langfuse.com) is open-source LLM observability. Self-host it in Docker for free.

Setup

# docker-compose.yml (simplified)
services:
  langfuse-web:
    image: langfuse/langfuse:3
    ports:
      - "3100:3000"
    depends_on:
      - postgres
  postgres:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: postgres

docker compose up -d
# Dashboard at http://localhost:3100

What You Get

• Traces — every LLM call logged with input, output, tokens, latency, cost
• Cost tracking — per agent, per model, per provider, per day
• Failure detection — find errors, timeouts, and silent failures
• Performance metrics — average latency, token patterns, model comparison

LangFuse runs at localhost:3100 — self-hosted, all data stays on your machine. Once you start tracing agent calls, the dashboard shows costs, latency, errors, and token usage across all your agents.

OpenClaw Integration

Add tracing via the LangFuse Python SDK in your scripts, or use the REST API:

from langfuse import Langfuse
langfuse = Langfuse(host="http://localhost:3100")
trace = langfuse.trace(name="vault-query", user_id="ops-agent")

LightRAG has built-in LangFuse integration — enable it in your `.env` and all graph RAG operations are automatically traced.

---

Reranker — Better Search Quality

A reranker re-scores retrieval results after initial vector search. It catches cases where keyword match matters more than semantic similarity.

Setup

Install a small cross-encoder model alongside your embedding server:

# reranker-server.py (FastAPI)
from sentence_transformers import CrossEncoder

model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")
# ~50MB VRAM, runs on any GPU with headroom

# POST /v1/rerank
# Input: {"query": "...", "documents": ["...", "..."], "top_n": 5}
# Output: reranked documents with scores

Impact

• 20-40% better search relevance on retrieval benchmarks
• ~50MB VRAM (tiny — runs alongside any embedding model)
• <400ms latency for reranking 10 documents

Integration

Add reranking between initial retrieval and final results:

Query → Embedding → Top-50 → Reranker → Top-5 (much better quality)

LightRAG supports rerankers natively — configure in `.env`.

---

n8n — Workflow Automation

[n8n](https://n8n.io) is a self-hosted workflow automation platform with 400+ app integrations.

Setup

services:
  n8n:
    image: n8nio/n8n
    ports:
      - "5678:5678"
    volumes