AI Analytics Platform

AI-Powered Real-Time Analytics Pipeline

A demonstration project showcasing a complete real-time analytics pipeline with AI-powered querying capabilities. This project simulates a healthcare analytics system that processes patient events in real-time and provides natural language query interfaces.

**⚠️ SECURITY NOTICE:** This project uses default credentials for local development. **Never use these in production!** See [SECURITY.md](SECURITY.md) for deployment guidelines.

Architecture Overview

┌─────────────────┐
│  Producer.py    │  Generates fake healthcare events
│  (Faker)        │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Kafka Topic    │  patient_events
│  (Streaming)    │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Consumer.py    │  Consumes & transforms events
│                 │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  PostgreSQL     │  Raw event storage
│  (Staging)      │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  DBT Models     │  Data transformation & aggregation
│                 │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Analytics      │  Business metrics & KPIs
│  Tables         │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  LangChain      │  Natural language queries
│  AI Agent       │
└─────────────────┘

Components

1. Infrastructure (Docker Compose)

• Apache Kafka + Zookeeper: Message streaming platform
• PostgreSQL: Data warehouse
• Kafka UI: Visual interface for Kafka debugging

2. Event Producer (`producer.py`)

• Generates realistic healthcare analytics events using Faker
• Event types: patient enrollment, prior authorization requests, therapy starts, claim submissions
• Configurable event generation rate
• Publishes to Kafka topic "patient_events"

3. Kafka Consumer (`consumer.py`)

• Consumes events from "patient_events" topic
• Implements consumer group pattern for