mcpField
M
Mcp_web_scrapper

Mcp_web_scrapper

Local
github.com/Neche-StephenBrowser Automation0Updated 79w ago
Repository

About Mcp_web_scrapper

Mcp_web_scrapper is a Node.js scraper that collects MCP server links and details from glama.ai. It stores the results as JSON files for later use.

Key features

  1. 01Collects MCP server links
  2. 02Processes linked server pages
  3. 03Writes links to server-links.json
  4. 04Stores scraped data in servers-json

Use cases

  1. 01Build a local collection of MCP server links
  2. 02Extract details from linked server pages
  3. 03Review scraped server data as JSON files

MCP support

Despite its name, this project scrapes MCP server listings from glama.ai; it does not describe exposing an MCP server to clients.

Configuration

Local server config is the same for every client. Copy it into your client’s MCP config file.

No install method listed yet

We haven’t collected a package or remote endpoint for this server. Follow the instructions in the project README.

Repository

Tools

No tool list yet

Local servers expose tools only once running on your machine, and the maintainer has not provided a tool list.

Details

From the project README

Getting Started with MCP Server Scraper

This project allows you to scrape and extract MCP web servers from glama.ai. Follow the steps below to set up and run the scraper efficiently.

📦 Prerequisites
  • Node.js
  • npm
🔧 Installation

  1. Clone the repository:

    git clone https://github.com/Neche-Stephen/mcp_web_scrapper
cd mcp_web_scrapper

  2. Install dependencies:

    npm install
🗃 Check for Existing Data

Before running a fresh scrape:

  1. Check if the servers-json folder exists.

  2. If it does and contains files, it means data has been previously scraped.

  3. To run a fresh batch, delete the servers-json folder:

    rm -rf servers-json
Step 1: Scrape MCP Server Links

Run the scraper to collect all MCP server links from the website:

node scraper.js
  • The scraper will launch a headless browser and start collecting data.
  • Be patient — this might take a few minutes.
  • Once you see the message "Closing browser", the scraping is complete.
  • You can then safely terminate the process using CTRL + C.

Output: A file named server-links.json will be generated in a folder servers-json containing all scraped links.

Step 2: Process and Extract Server Data

Now, fetch and process detailed information from each server link:

node process-links.js
  • The script reads from server-links.json and visits each link to scrape the necessary data.
  • A progress report will be displayed in the terminal as the script runs.
  • When you see the success message indicating completion, press CTRL + C to terminate the process.

All scraped data will be stored in the servers-json directory in different files named after the mcp servers

Comments

Markdown supported

New to MCP?

From what the protocol is to installing your first server, these answers cover most first questions.

An open protocol that defines how AI applications connect to external tools and data in a uniform way. Clients talk to the model; MCP servers provide tools, resources and prompt templates. They exchange standard messages, so one server can be reused by many clients.