Claudecrawl — Development skill for Claude Code
ClaudeCrawl is clone of FireCrawl's web scraping feature.
How to install Claudecrawl
This entry records only its repository, not the path inside it, so there is no
exact command to give. Open haandol/claudecrawl and copy the folder into
~/.claude/skills/, or the file into ~/.claude/agents/.
What Claudecrawl does
ClaudeCrawl is clone of FireCrawl's web scraping feature.
Alternatives in Development
- Claude Skills Cookbook — A comprehensive guide to using Claude's Skills feature by Anthropic 36.3k ★
- Plan Feature: Comprehensive Archon Implementation Planning — Plan Feature: Comprehensive Archon Implementation Planning skill 17.6k ★
- Feature Dev — Comprehensive feature development with specialized agents 14k ★
README
ClaudeCrawl
A web content scraper utilizing [Playwright](https://playwright.dev) and [Claude LLM](https://claude.ai) through [AWS Bedrock](https://aws.amazon.com/bedrock/) to intelligently extract and structure data from web pages.

Features
- Intelligent web content extraction using Claude LLM
- Structured data output in JSONL format
- Configurable schema-based parsing
- AWS Bedrock integration
- Logging support
Prerequisites
- Docker
- Python 3.13+
- AWS Account with Bedrock access
- AWS CLI configured
Installation
- Clone the repository:
git clone https://github.com/haandol/claude-web-scraper.git
cd claude-web-scraper
- Install
uv:
pip install uv
- Install dependencies
uv sync
- Install Playwright dependencies:
uv run playwright install
- Configure environment variables:
Create a
.envfile in the project root with:
MODEL_ID=us.anthropic.claude-3-5-haiku-20241022-v1:0
AWS_PROFILE_NAME=your_profile_name
AWS_REGION=your_aws_region
Usage
open `app.py` and modify `url`, `OutputSchema` and `instruction`
Run the scraper:
uv run -- python app.py
The script will:
- Crawl the specified data webpage
- Extract article information using Claude LLM
- Save the results in
output/output.jsonl, unless you specify a different path.
License
This project is licensed under the Apache 2.0 License - see the [LICENSE](/LICENSE) file for details.
Related Skills
Site Implement
Clone a section (or whole page) from a live website into a Shopify section using Firecrawl evidence
Websearch Setup
Guided setup for web search providers — pick from Exa, Tavily, Perplexity, Firecrawl, Valyu; get keys, install
Web Scraper
Intelligent web scraping Claude Code skill with automatic strategy selection and TypeScript-first Apify Actor
Find Contacts
Enrich contacts.xlsx: replace generic fallback emails (contact@, info@, hello@, team@, jobs@) with named decis
Amber
Clean, Simple Claude-Code Web-based Clone: Amber (amberagent.dev)
Feature Parity
Identify web/mobile feature gaps, generate parity plan, spawn parallel agents for both targets
Related Agents
Firecrawl Expert
Expert in Firecrawl API for web scraping, crawling, and structured data extraction. Handles dynamic content, a
Firecrawl Scraper
Web scraping and research subagent. Dispatched to scrape URLs, search the web, crawl sites, and extract struct
Data Growth Operator
Claude Code operator for scraping, ads intelligence, lead discovery, enrichment, Apify/Scrapling/DataForSEO/Fi