Data Pipeline Commentary Agent — Data skill for Claude Code

Data Pipeline Commentary Agent

An AI automation built with **n8n** and the **Claude API**: it watches a PostgreSQL table *and* a folder for new data, and whenever either one gets new rows or files, Claude writes a structured plain-language commentary — an overview, notable patterns, and anomalies to watch — and emails it to you. The same pipeline works regardless of what the data actually is, because nothing downstream of the initial read assumes any particular columns.


What it does

1. Two independent triggers run on a schedule: one polls Postgres for rows added since the last check, the other polls a folder for files it hasn't seen yet.
2. Whichever one fires, its output gets normalized into the same generic {columns, rows} shape.
3. A plain-JavaScript step computes descriptive stats (count, min, max, mean, stdev) for whatever columns turn out to be numeric, and flags 3-sigma outliers — no column names hardcoded anywhere.
4. Claude (claude-sonnet-5) turns those stats, the outliers, and a sample of rows into a structured commentary: an overview, notable patterns, and anomalies to watch.
5. The result is emailed via the Resend API.

The story behind it

This one had more debugging than the last project, mostly because "make it work with any data" is a much easier goal to state than to actually build.

The environment fights back first

Before a single node ran, the Docker setup itself pushed back:

• Adding a second container (Postgres) alongside the existing n8n one meant discovering they were on Docker's default bridge network, which doesn't support name-based resolution — fixed by creating a dedicated user-defined network and attaching both containers to it.
• Mounting a Windows folder into the container with a backslash path (C:\Users\...) hit a documented Docker Desktop bug where it mangles the path and fails with a cryptic mkdir ... access denied. Forward slashes (C:/Users/...) fixed it.
• n8n's