mukul975

MCP Web Scrape — AI skill for Claude Code

AI community

🚀 mcp-web-scrape — Clean, cache-aware web content fetcher for AI agents.

How to install MCP Web Scrape

This entry records only its repository, not the path inside it, so there is no exact command to give. Open mukul975/mcp-web-scrape and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What MCP Web Scrape does

🚀 mcp-web-scrape — Clean, cache-aware web content fetcher for AI agents. Fetch any URL → extract readable content → return Markdown/JSON with citations. ⚡ Fast caching, 🤝 robots.txt compliant, 📝 Markdown-ready output, �� works with ChatGPT/Claude Desktop.

Alternatives in AI

Riteway — Simple, readable, helpful unit tests 1.2k ★
Qiaomu Markdown Proxy — Fetch any URL as clean Markdown via proxy services (r.jina.ai / defuddle.md) or built-in scripts 426 ★
PromptEnhance — Rewrite the following prompt to be clearer, more specific, and more effective for an AI assistant 250 ★

README

🕷️ MCP Web Scrape

Clean, cached web content for agents—Markdown + citations, robots-aware, ETag/304 caching.

[![npm version](https://img.shields.io/npm/v/mcp-web-scrape.svg)](https://www.npmjs.com/package/mcp-web-scrape) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![GitHub stars](https://img.shields.io/github/stars/mukul975/mcp-web-scrape.svg?style=social&label=Star)](https://github.com/mukul975/mcp-web-scrape)

📦 Version

**Current Version:** `1.0.7`

🚀 Quick Start Demo

# Extract content from any webpage
npx mcp-web-scrape@1.0.7

# Example: Extract from a news article
> extract_content https://news.ycombinator.com
✅ Extracted 1,247 words with 5 citations
📄 Clean Markdown ready for your AI agent

🎯 Tool Examples

# Extract all forms from a webpage
> extract_forms https://example.com/contact
✅ Found 3 forms with 12 input fields

# Parse tables into structured data
> extract_tables https://example.com/data --format json
✅ Extracted 5 tables with 247 rows

# Find social media profiles
> extract_social_media https://company.com
✅ Found Twitter, LinkedIn, Facebook profiles

# Analyze sentiment of content
> sentiment_analysis https://blog.example.com/article
✅ Sentiment: Positive (0.85), Emotional tone: Optimistic

# Extract named entities
> extract_entities https://news.example.com/article
✅ Found 12 people, 8 organizations, 5 locations

# Check for security vulnerabilities
> scan_vulnerabilities https://mysite.com
✅ No XSS vulnerabilities found, 2 header improvements suggested

# Analyze competitor SEO
> analyze_competitors ["https://competitor1.com", "https://competitor2.com"]
✅ Competitor analysis complete: keyword gaps identified

# Monitor uptime and performance
> monitor_uptime https://mysite.com --interval 300
✅ Uptime: 99.9%, Average response: 245ms

# Generate comprehensive report
> generate_reports https://website.com --metrics ["seo", "performance", "sec