Web Scraper & Crawler for RAG (Markdown)
RemoteAbout Web Scraper & Crawler for RAG (Markdown)
Web Scraper & Crawler for RAG is a hosted MCP server for AI agents that turns individual web pages or whole sites into clean Markdown or JSON, including described fields and site maps.
Key features
- 01Page and site crawling
- 02Clean Markdown or JSON output
- 03Site map generation
- 04Fields described in plain language
- 05Streamable HTTP connection
Use cases
- 01Preparing website content for RAG
- 02Extracting selected fields across a site
MCP support
Exposes hosted scraping and crawling tools to MCP clients over Streamable HTTP. Connection uses an Apify API token.
Configuration
Remote servers need no local install. Pick your client, then copy the config or install in one click.
https://mcp.apify.com/?tools=byteblisk/sitereaderclaude mcp add --transport http byteblisk-sitereader https://mcp.apify.com/?tools=byteblisk/sitereader{"mcpServers":{"byteblisk-sitereader":{"type":"http","url":"https://mcp.apify.com/?tools=byteblisk/sitereader"}}}Open in Cursor{"servers":{"byteblisk-sitereader":{"type":"http","url":"https://mcp.apify.com/?tools=byteblisk/sitereader"}}}Open in VS Code{"mcpServers":{"byteblisk-sitereader":{"type":"http","url":"https://mcp.apify.com/?tools=byteblisk/sitereader"}}}{"mcpServers":{"byteblisk-sitereader":{"serverUrl":"https://mcp.apify.com/?tools=byteblisk/sitereader"}}}Run commands in a terminal; paste JSON into the client’s MCP config and restart. Servers that need sign-in will prompt on first connect.
https://mcp.apify.com/?tools=byteblisk/sitereaderAuth requiredTools
Details
From the project READMEbyteblisk MCP servers
Three hosted MCP servers for AI agents, running on Apify. Nothing to install: connect over Streamable HTTP with your Apify API token, and pay per use. Every tool states its cost, and failed calls are free.
| Server | What your agent gets | Store page | |
|---|---|---|---|
 | Web Scraper & Crawler for RAG | Any page or whole site as clean Markdown or JSON, site maps, fields you describe in plain words | apify.com/byteblisk/sitereader |
 | App Store & Google Play Reviews | Reviews from both stores in one format, complaint themes with real quotes, competitor comparison | apify.com/byteblisk/app-reviews-insights |
 | Google Play Scraper & SERP Rank Checker | Google Play apps, reviews and search, top-100 rank on Google, Bing and Yandex | apify.com/byteblisk/google-play-search-toolkit |
Connect
Endpoints (Streamable HTTP, header Authorization: Bearer <YOUR_APIFY_TOKEN>):
https://byteblisk--sitereader.apify.actor/mcp
https://byteblisk--app-reviews-insights.apify.actor/mcp
https://byteblisk--google-play-search-toolkit.apify.actor/mcp
Claude Desktop, Cursor or any MCP client (mcp.json):
{
"mcpServers": {
"byteblisk-sitereader": {
"url": "https://byteblisk--sitereader.apify.actor/mcp",
"headers": { "Authorization": "Bearer YOUR_APIFY_TOKEN" }
},
"byteblisk-app-reviews": {
"url": "https://byteblisk--app-reviews-insights.apify.actor/mcp",
"headers": { "Authorization": "Bearer YOUR_APIFY_TOKEN" }
},
"byteblisk-google-play-search": {
"url": "https://byteblisk--google-play-search-toolkit.apify.actor/mcp",
"headers": { "Authorization": "Bearer YOUR_APIFY_TOKEN" }
}
}
}
Get a token at console.apify.com/settings/integrations. Prices are on each Store page; Apify's free plan includes monthly credit to try them.
Support
Open an issue on the actor's Store page; we answer every one.
Comments
New to MCP?
From what the protocol is to installing your first server, these answers cover most first questions.
An open protocol that defines how AI applications connect to external tools and data in a uniform way. Clients talk to the model; MCP servers provide tools, resources and prompt templates. They exchange standard messages, so one server can be reused by many clients.