Scrapingdog

Scrapingdog MCP Server

The Model Context Protocol server for the Scrapingdog APIs, hosted at https://mcp.scrapingdog.com/mcp.

It gives an AI assistant 35 tools for live web data — Google SERPs, Maps, News, Shopping, Lens, Amazon, LinkedIn, YouTube, X, Walmart, eBay, screenshots and a general-purpose scraper — each backed by the corresponding Scrapingdog endpoint.

There is nothing to install. Point your client at the URL and send your own API key.

Connect

Get an API key from the Scrapingdog dashboard, then:

Claude Code

claude mcp add --transport http scrapingdog https://mcp.scrapingdog.com/mcp \
  --header "Authorization: Bearer YOUR_API_KEY"

Cursor / Windsurf / any client with an mcp.json

{
  "mcpServers": {
    "scrapingdog": {
      "url": "https://mcp.scrapingdog.com/mcp",
      "headers": { "Authorization": "Bearer YOUR_API_KEY" }
    }
  }
}

claude.ai — Settings → Connectors → Add custom connector, URL https://mcp.scrapingdog.com/mcp, authentication "API key", paste your key.

Check it by hand

curl -s https://mcp.scrapingdog.com/mcp \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

Calls are billed to the key you send, at the same credit cost as the REST API. A tool's description states what it costs.

How it works

MCP client ──Authorization: Bearer <key>──> mcp.scrapingdog.com ──api_key=<key>──> api.scrapingdog.com

The server is stateless: every request builds its own MCP server and transport from the key on that request and tears them down with the response. One process serves many customers without a session map, so a key never outlives the request it came on and cannot leak into another caller's session. npm run smoke asserts exactly that, among other things.

Keys are verified against Scrapingdog's own /account endpoint (which resolves user keys and team member keys, and costs no credits), cached briefly. If that check is unreachable the request is let through, because the API itself still enforces the key on every tool call — an outage in the verifier must not take the whole MCP surface down. No database credentials are needed anywhere in this service.

Layout

Path	What it is
src/endpoints.ts	The 35 endpoints, declared. Adding an API means appending one object here — no handler code.
src/server.ts	Builds the MCP server and derives each tool's schema from that registry
src/client.ts	The HTTP client for api.scrapingdog.com
src/auth.ts	API-key verification and caching
src/http.ts	The hosted entry point (Streamable HTTP)
src/index.ts	A stdio entry point, for local development only
deploy/nginx/	The mcp.scrapingdog.com vhost
ecosystem.config.cjs	pm2 app definition
scripts/smoke.mjs	Pre-deploy checks against a stubbed API

Local development

npm ci
npm run build
npm run smoke          # stubs the API: no key needed, no credits spent

npm run dev            # the HTTP server on http://127.0.0.1:8731/mcp
SCRAPINGDOG_API_KEY=... npm run dev:stdio   # stdio, for poking at one tool

Inspect it interactively:

npx @modelcontextprotocol/inspector
# transport: Streamable HTTP, url: http://127.0.0.1:8731/mcp,
# header: Authorization: Bearer <your key>

Settings live in .env.example.

Deploying

The server runs on its own box, separate from the API servers, so a problem here cannot affect api.scrapingdog.com.

First time, on a bare Ubuntu box:

git clone https://github.com/Darshan972/Scrapingdog-mcp.git ~/Scrapingdog-mcp
cd ~/Scrapingdog-mcp && bash deploy/setup-server.sh

That installs Node (from .nvmrc), pm2, nginx and the vhost, builds and starts the app, opens the firewall, and issues the TLS certificate — skipping the certificate with a clear message if DNS doesn't point here yet. It is safe to re-run.

After that, every push to main typechecks, builds, runs the smoke suite and only then rolls out with pm2 reload (.github/workflows/deploy.yml). Needs three repository secrets: MCP_SERVER_HOST, MCP_SERVER_USER, MCP_SSH_PRIVATE_KEY.

The deploy health check requires /healthz to answer 200 and an unauthenticated POST /mcp to answer 401, so a deploy that accidentally opened the gateway fails instead of going green.

The app binds 127.0.0.1 so nginx is the only route in. Don't change that on a public host: the key check lives in this process, so an exposed port is an open proxy to the API.

License

MIT