Scrape Everything

Scrape Everything MCP server

An MCP server for Scrape Everything. Public social data as clean JSON. It lets Claude, Cursor, VS Code, Windsurf and any other MCP client fetch public data from Instagram, TikTok, X, YouTube, Facebook, Telegram, Spotify, Pinterest and Twitch.

• One tool per API endpoint. At startup the server reads the live catalog from GET /v1/scrapers and registers a tool for each scraper (instagram_profile, tiktok_posts, youtube_transcript, ...). New endpoints appear without a package update.
• Works offline. If the catalog can't be fetched, the server starts from a bundled snapshot.
• Shows the cost first. Every tool description lists its credit cost and the fields it returns. Failed calls are refunded.
• Gives readable errors. API errors come back as tool errors that keep the error code and request_id. On RATE_LIMITED the error includes the Retry-After wait. The server never retries on its own.
• Keeps results bounded. Results are JSON text, truncated at 100k characters by default, with a note when that happens.
• Sends no telemetry. The only network calls go to the Scrape Everything API.

Get an API key

Create a key at https://scrapeeverything.com/dashboard/keys. Keys look like ssk_live_....

Install

Claude Code

claude mcp add scrapeeverything -e SCRAPEEVERYTHING_API_KEY=ssk_live_... -- npx -y @scrapeeverything/mcp

Claude Desktop

Edit claude_desktop_config.json (Settings, then Developer, then Edit Config):

{
  "mcpServers": {
    "scrapeeverything": {
      "command": "npx",
      "args": ["-y", "@scrapeeverything/mcp"],
      "env": { "SCRAPEEVERYTHING_API_KEY": "ssk_live_..." }
    }
  }
}

On Windows, if npx is not found, use "command": "cmd" and "args": ["/c", "npx", "-y", "@scrapeeverything/mcp"].

Cursor

Add this to ~/.cursor/mcp.json (global) or .cursor/mcp.json (project):

{
  "mcpServers": {
    "scrapeeverything": {
      "command": "npx",
      "args": ["-y", "@scrapeeverything/mcp"],
      "env": { "SCRAPEEVERYTHING_API_KEY": "ssk_live_..." }
    }
  }
}

VS Code (GitHub Copilot agent mode)

Add this to .vscode/mcp.json. VS Code prompts for the key and stores it securely:

{
  "inputs": [
    { "type": "promptString", "id": "scrapeeverything-key", "description": "Scrape Everything API key", "password": true }
  ],
  "servers": {
    "scrapeeverything": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "@scrapeeverything/mcp"],
      "env": { "SCRAPEEVERYTHING_API_KEY": "${input:scrapeeverything-key}" }
    }
  }
}

Or from a terminal:

code --add-mcp '{"name":"scrapeeverything","command":"npx","args":["-y","@scrapeeverything/mcp"],"env":{"SCRAPEEVERYTHING_API_KEY":"ssk_live_..."}}'

Windsurf

Add this to ~/.codeium/windsurf/mcp_config.json:

{
  "mcpServers": {
    "scrapeeverything": {
      "command": "npx",
      "args": ["-y", "@scrapeeverything/mcp"],
      "env": { "SCRAPEEVERYTHING_API_KEY": "ssk_live_..." }
    }
  }
}

Self-hosted streamable HTTP

SCRAPEEVERYTHING_API_KEY=ssk_live_... npx -y @scrapeeverything/mcp --http --port 3333
# MCP endpoint: http://127.0.0.1:3333/mcp   health: /health

HTTP mode is stateless and accepts POST only. By default it binds to 127.0.0.1 and only accepts loopback Host and Origin headers (127.0.0.1, localhost, [::1]), which blocks DNS rebinding. On loopback, each request can bring its own key with Authorization: Bearer ssk_live_... or x-api-key. If a request has no key, the server uses SCRAPEEVERYTHING_API_KEY.

To listen on any other address, you must set an inbound auth token. Without one the server refuses to start, so it can never act as an open proxy that spends your key:

SCRAPEEVERYTHING_MCP_AUTH_TOKEN="$(openssl rand -hex 32)" \
  npx -y @scrapeeverything/mcp --http --host 0.0.0.0 --port 3333 --allowed-host mcp.example.com

• Clients send Authorization: Bearer <token>. The token is compared in constant time and must be at least 16 characters.
• With a token set, each client sends its own Scrape Everything key in x-api-key. The server's SCRAPEEVERYTHING_API_KEY is not used as a fallback unless you pass --allow-shared-key.
• --allowed-host (repeatable, or SCRAPEEVERYTHING_MCP_ALLOWED_HOSTS, comma-separated) restricts the Host header. Requests that send an Origin header must match an allowed host or an --allowed-origin (SCRAPEEVERYTHING_MCP_ALLOWED_ORIGINS).
• Terminate TLS in front of the server (a reverse proxy) so the token and keys are never sent in clear text.

Configuration

Variable	Default	Purpose
SCRAPEEVERYTHING_API_KEY	(none)	API key. Required for scraper tools and get_account.
SCRAPEEVERYTHING_API_URL	https://api.scrapeeverything.com	API base URL. Must be https unless the host is loopback (for local dev, http://127.0.0.1:4001).
SCRAPEEVERYTHING_MCP_AUTH_TOKEN	(none)	HTTP mode: inbound bearer token. Required to bind to a non-loopback host.
SCRAPEEVERYTHING_MCP_ALLOWED_HOSTS	(none)	HTTP mode: comma-separated extra Host values to accept.
SCRAPEEVERYTHING_MCP_ALLOWED_ORIGINS	(none)	HTTP mode: comma-separated extra Origin values to accept.
SCRAPEEVERYTHING_MCP_MAX_CHARS	100000	Max characters per tool result before truncation.
SCRAPEEVERYTHING_MCP_SCHEMA_ANYOF	off	Set it to 1 to emit one_of groups as a top-level anyOf of required sets.

About one_of: some endpoints accept one of several inputs, for example url or shortcode. Several clients reject a top-level anyOf in tool schemas, Anthropic's API among them. So by default the rule goes in the schema/tool description, and the server checks it before calling the API, which spends no credits. Set SCRAPEEVERYTHING_MCP_SCHEMA_ANYOF=1 if your client supports anyOf.

Tools

Tool	Credits	Required input
get_account	0	(none). Returns email, plan, credits and rate/concurrency limits.
list_scrapers	0	optional platform
instagram_profile	1	username
instagram_photos	2	username
instagram_post	1	one of url or shortcode
instagram_reel	1	one of url or shortcode
instagram_comments	1	one of url or shortcode
instagram_reels	2	username
instagram_reels_search	3	query
x_profile	1	username
x_posts	2	username
x_post	1	one of url or id
tiktok_profile	1	username
tiktok_posts	1	username
tiktok_video	1	one of url or video or id
tiktok_comments	1	one of url or id
tiktok_transcript	1	one of url or id
tiktok_music	2	one of url or id
tiktok_hashtag	1	hashtag
youtube_profile	1	one of username or channel
youtube_videos	1	one of username or channel
youtube_video	1	one of url or id or video
youtube_search	1	query
youtube_comments	1	one of url or id
youtube_transcript	1	one of url or id
youtube_playlist	1	one of url or id
facebook_profile	1	username
telegram_profile	1	username
telegram_posts	2	username
telegram_post	1	one of url or post or username + id
spotify_track	1	one of url or id or track
spotify_artist	1	one of url or id or artist
spotify_album	1	one of url or id or album
spotify_playlist	1	one of url or id or playlist
pinterest_profile	1	username
pinterest_pins	2	username
pinterest_pin	1	one of url or id
pinterest_search	2	query
pinterest_board	2	one of url or username + board
twitch_channel	1	username
twitch_videos	1	username
twitch_clips	1	username

The live catalog is the source of truth. Ask your assistant to run list_scrapers to see the current list.

Development

npm install
npm run build
npm test
npm run update-catalog   # refresh catalog/scrapers.json (honours SCRAPEEVERYTHING_API_URL)
node dist/index.js --help

Inspect the server interactively with npx @modelcontextprotocol/inspector node dist/index.js.

Org and scope placeholders

org.config.json is the one place that records the GitHub org (scrapeeverything) and the npm scope (@scrapeeverything). To change them, run:

npm run apply-org -- --github-org <org> --npm-scope @<scope>

That rewrites package.json (name, mcpName, repository), server.json, smithery.yaml and this README. The MCP Registry name io.github.<org>/mcp must match the GitHub org that publishes it. glama.json lists the GitHub usernames of the Glama maintainers, so put the founder's GitHub username there.

Releasing

1. Run npm version patch (or minor/major). This also syncs server.json.
2. Run git push --follow-tags.
3. The release.yml workflow publishes to npm with provenance (NPM_TOKEN secret), then publishes server.json to the official MCP Registry through GitHub OIDC.

About Scrape Everything

Scrape Everything is a REST API for public social media data. Send a username, link or ID and get clean, typed JSON from Instagram, TikTok, X, YouTube, Facebook, Telegram, Spotify, Pinterest and Twitch. Pay per successful request with prepaid credits that never expire. New accounts start with 100 free credits. https://scrapeeverything.com

License

MIT © 2026 Scrape Everything