About Mcp Server Scraper
A TypeScript script for LiteLLM that scrapes configurations from the official MCP server directory. It prioritizes Docker configurations, with npx and uvx as alternatives.
Key features
- 01Scrapes MCP server configurations
- 02Prioritizes Docker configurations
- 03Supports npx and uvx configurations
- 04Builds and runs as a TypeScript script
Use cases
- 01Collecting server configurations for LiteLLM
- 02Extracting configurations from directory pages
MCP support
The script scrapes configurations from the MCP server directory; it does not expose tools or resources to MCP clients.
Configuration
Local server config is the same for every client. Copy it into your client’s MCP config file.
We haven’t collected a package or remote endpoint for this server. Follow the instructions in the project README.
Tools
Local servers expose tools only once running on your machine, and the maintainer has not provided a tool list.
Details
From the project READMEPurpose
This script was created to scrape configurations for MCP servers found on 'https://github.com/modelcontextprotocol/servers/tree/main/src'. Meant for LiteLLM.
Implementation and Reasoning
The main self executing function navigates to the github page and puts the object together.
Most of the scraping logic is in 'scrapeServerConfiguration'. MCP server configurations come in JSON that are exposed in some pages (server directories), and a user can copy them to the browser's clipboard. On other pages the configuration is hidden under a dropdown, i.e., "Using docker". The scraper primarily looks for docker configurations, followed by npx configurations, and uvx configurations if the preference doesn't exist. For whatever reason, playwright's selector keeps favoring the npx configuration (admittedly, this might be some fault in the code logic), so another 'for loop' was created to specifically get docker configurations if they exist (contact author for further explanation).
Console logs were added for debugging. Delete console logs or comments as preferred.
Usage
- Compile typescript: npm run build
- Run script: npm start
Comments
New to MCP?
From what the protocol is to installing your first server, these answers cover most first questions.
An open protocol that defines how AI applications connect to external tools and data in a uniform way. Clients talk to the model; MCP servers provide tools, resources and prompt templates. They exchange standard messages, so one server can be reused by many clients.