Transit Pulse Nyc

Transit Pulse NYC

*MTA Streaming Intelligence — real-time NYC transit intelligence on Confluent.*

Real-time NYC subway intelligence on the **Confluent Data Streaming Platform**.

Live MTA GTFS-Realtime feeds flow into Kafka; **Flink SQL** computes live arrival estimates and detects train **bunching** and **service gaps**; and a **Flink Streaming Agent** (LLM running *inside* Flink via `AI_RUN_AGENT`) turns each alert into a plain-English **dispatcher action** and a **rider-facing message**.

The dashboard is a Swiftly-style live map: real subway route lines (from static GTFS geometry) with subway trains and **~2,700 city buses** — buses carry real GPS lat/lon + heading — moving on top, plus mode/route filters and on-map alert markers. Below the map, three **interactive Claude agents** (rider advisor, operator prediction, and a new-bus-route designer) answer questions grounded in the live fleet state.

Built for **Confluent AI Day NYC**, modeled on the architecture of [`confluentinc/demo-confluent-intelligence-f1`](https://github.com/confluentinc/demo-confluent-intelligence-f1) — we keep its shape (topics-as-Flink-tables, `ML_DETECT_ANOMALIES`, `CREATE AGENT` + `AI_RUN_AGENT`, FastAPI/websocket dashboard) and swap F1 tire telemetry for the NYC subway.

Why this wins

Prize criterion	How we hit it
Business impact	Bunching and gaps are the #1 driver of unreliable transit. Dispatchers make hold/gap-fill calls in real time; riders need accurate ETAs.
Connectors / ingest	Two ingest paths: a producer streams the real MTA GTFS-RT feeds (protobuf, ~5-30s), and a Confluent fully-managed HTTP Source Connector pulls a real JSON service-alerts feed into mta.service_alerts.
Stream processing (Flink)	Deduplication, ETA transforms, MATCH_RECOGNIZE for consecutive-arrival headway, and anomaly detection on the headway series.
Flink-driven AI	The dispatcher LLM is a Flink operator (AI_RUN_AGENT), not a si