dshakers

Scraping Eagle — DevOps skill for Claude Code

DevOps community

🦅 Scraping Eagle: web scraping skill & CLI for Claude Code and AI agents.

How to install Scraping Eagle

This entry records only its repository, not the path inside it, so there is no exact command to give. Open dshakers/scraping-eagle and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Scraping Eagle does

🦅 Scraping Eagle: web scraping skill & CLI for Claude Code and AI agents. One command scrapes any website — Cloudflare/anti-bot bypass, JavaScript rendering, structured extraction to CSV/JSON, YouTube transcripts, Reddit, X, LinkedIn, Amazon, Shopify and full-site crawls to Markdown for RAG.

Alternatives in DevOps

Appwrite — Appwrite® - complete cloud infrastructure for your web, mobile and AI apps 57.5k ★
Claudable — Claudable is an open-source web builder that leverages local CLI agents, such as Claude Code, Codex, Gemini CL 4k ★
WP CLI And Ops — WordPress CLI and operations management 902 ★

README

🦅 Scraping Eagle

**The web scraping skill and CLI for Claude Code and AI agents.**

One command, `scrape URL`, turns any website into clean Markdown, JSON or CSV.
Past Cloudflare and JavaScript, on your own machine, with no API keys.

[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE) ![Tested on macOS](https://img.shields.io/badge/tested%20on-macOS-lightgrey) ![Python 3.10–3.13](https://img.shields.io/badge/python-3.10–3.13-3776AB) ![Claude Code skill](https://img.shields.io/badge/Claude%20Code-skill-D97757) ![Live checks: 23](https://img.shields.io/badge/live%20checks-23%20passing-brightgreen)

What is Scraping Eagle?

Scraping Eagle is an open-source web scraping CLI and Claude Code skill created by [dshakers](#author). It gives AI coding agents and developers a single command, `scrape`, that fetches any web page, escalates on its own from a fast HTTP request to a stealth browser when a site blocks bots, and returns clean Markdown, JSON or CSV. It runs locally, is free, and needs no API keys.

scrape https://example.com/article                        # any page → clean Markdown
scrape "https://www.youtube.com/watch?v=aircAruvnKk"      # video → metadata + transcript
scrape URL --each ".card" --field "name=h3::text" --field "price=.price::text" --csv
scrape search "best open source vector database"          # web search, no key
scrape crawl https://docs.example.com --max-pages 100     # whole site → Markdown for RAG

_Version 1.0.0 · every route last verified live on 2 October 2026._

Quick start

git clone https://github.com/dshakers/scraping-eagle ~/.claude/skills/scraping-eagle
bash ~/.claude/skills/scraping-eagle/scripts/setup.sh
scrape doctor --online

`setup.sh` builds an isolated environment in `~/.scraping-eagle` (about 400 MB with the browser, 1–2 minutes) and links the `scrape` command into `~/.local/bin`. It never touches your system Python.

**Requirements:**