Jair33190Jair

Bob Crawl Site — Development skill for Claude Code

Development community

Crawl a homepage, extract all same-domain subpage links, fetch each with /bob-web2md, and write one .md per page to an output folder.

How to install Bob Crawl Site

Installs to ~/.claude/skills/jair33190jair-ai-python-human_project_os-bob-crawl-site/SKILL.md

Terminal

mkdir -p ~/.claude/skills/jair33190jair-ai-python-human_project_os-bob-crawl-site && curl -fsSL https://raw.githubusercontent.com/Jair33190Jair/AI-Python-Human_Project_OS/HEAD/commands/bob-crawl-site.md -o ~/.claude/skills/jair33190jair-ai-python-human_project_os-bob-crawl-site/SKILL.md

Restart Claude Code, or start a new session, for it to be picked up.

What Bob Crawl Site does

owner: bob description: Crawl a homepage, extract all same-domain subpage links, fetch each with /bob-web2md, and write one .md per page to an output folder. argument-hint: status: draft

/bob-crawl-site

Crawl a site one level deep: discover internal links from the homepage, fetch each page, and write one `.md` file per page.

**Arguments:** ` [--overwrite] [--allow-paid]`

url — homepage to crawl (http/https)
output_dir — folder to write `.

Alternatives in Development

Reddit Fetch — Fetches Reddit content via Gemini CLI when WebFetch is blocked or returns 403 errors 6.5k ★
Om Tidy — Self-maintenance pass — acts on every hygiene flag: archives completed work, groups loose clusters, splits ove 4.6k ★
/intel — Fetch actionable intelligence for a target 1.8k ★