mcpField

crw-crawl

usData & databases62 installs

About crw-crawl

Crawl an entire website or section and extract content from every page. Use when you need content from many pages under a common URL prefix: "crawl the whole site", "get all docs pages", "scrape every blog post", "download the full docs for RAG", "extract all pages under /api". Async BFS — starts a job and polls for results. Step 4 of the crw workflow ladder.

Install

Copy the command and run it in a terminal.

$npx skills add https://us/crw --skill crw-crawl

After installing, type/crw-crawlin Claude Code to confirm it loaded.

Comments

Markdown supported