mcp-server-requests

中文

---

mcp-server-requests

An MCP server that provides HTTP request capabilities, enabling LLMs to fetch and process web content.

Features

• fetch: Web content fetching tool
  • Supports converting web content to Markdown format
  • Supports filtering web content (non-visual elements like script, style, meta)
  • Feeds into LLM context, reducing token consumption through Markdown conversion or filtering non-visual elements
• fetch_to_file: Web content saving tool
  • Web content is not fed into LLM context, avoiding excessive token consumption
• http_request: Generic HTTP request tool
  • Full HTTP method support (GET, POST, PUT, PATCH, DELETE)
  • Supports custom request headers
  • Supports text and JSON request bodies
  • Returns complete HTTP response information (status, headers, body) for LLMs
• Supports custom User-Agent
• Supports random User-Agent generation
• Supports using MCP Root capability, controlled by --use-root, usable with MCP Clients that have root capability

Installation

git clone https://github.com/coucya/mcp-server-requests.git
cd mcp-server-requests
pip install .

Usage

MCP Server Configuration

{
    "mcpServers": {
        "mcp-server-requests": {
            "command": "python",
            "args": [
                "-m",
                "mcp_server_requests"
            ]
        }
    }
}

Command Line

0. Start MCP Server

Start the MCP server directly:

python -m mcp_server_requests

Options

• --user-agent TEXT: Specify custom User-Agent string
• --random-user-agent [browser=xxx;os=xxx]: Use randomly generated User-Agent
• --force-user-agent: Force using command line specified User-Agent, ignoring LLM provided UA
• --list-os-and-browser: List available browsers and OS for random User-Agent generation
• --use-root: Enable workspace root support for file operations (requires MCP Client with root capability support)
• --allow-external-file-access: Allow file operations outside workspace (only with --use-root)

Option Details

• --user-agent and --random-user-agent are mutually exclusive and cannot be used together

• User-Agent setup methods:

  • Custom string: --user-agent "Mozilla/5.0 (...)"
  • Fully random: --random-user-agent
  • Conditional random generation:
    • Specify browser type: --random-user-agent browser=chrome
    • Specify OS: --random-user-agent os=windows
    • Both browser and OS: --random-user-agent browser=chrome;os=windows
    • Note: Browser and OS parameters are case insensitive

• Use --list-os-and-browser to view available browsers and OS for --random-user-agent.

• --force-user-agent option controls User-Agent priority:

  • When --force-user-agent is enabled: Force using command-line specified User-Agent (via --user-agent or --random-user-agent)
  • When --force-user-agent is disabled:
    • If LLM provides User-Agent in headers, use the LLM-provided one
    • Otherwise use command-line specified User-Agent
  • If none specified, use default User-Agent: Mozilla/5.0 (compatible; mcp-server-requests/{version})

• --use-root enables file operations relative to workspace root:

  • Prerequisite: Requires MCP Client with root capability support
  • When enabled: fetch_to_file allows absolute paths or relative paths, relative paths are relative to workspace root
  • When disabled: fetch_to_file must use absolute paths, files are saved to specified absolute path locations

• --allow-external-file-access: Controls whether file operations outside workspace are allowed (only usable with --use-root)

  • Prerequisite: Requires --use-root to be enabled, this option has no effect if --use-root is not enabled
  • When enabled: fetch_to_file can operate on files outside the workspace directory
  • When disabled: fetch_to_file can only operate on files within the workspace directory, even if absolute paths are provided, files outside the workspace cannot be operated on

---

1. fetch - Fetch Web Content

The fetch subcommand is equivalent to the MCP fetch tool functionality, used to demonstrate fetch capabilities.

python -m mcp_server_requests fetch <URL> [--return-content {raw,basic_clean,strict_clean,markdown}]

Options:

• --return-content: Return content type (default: markdown)
  • raw: Return raw unprocessed HTML content (includes response headers)
  • basic_clean: Basic cleanup, removing non-display tags (script, style, meta, etc.) while preserving structure
  • strict_clean: Strict cleanup, removing non-display tags and most HTML attributes, keeping only essential structure
  • markdown: Convert HTML to clean Markdown format

---

Functionality

MCP Tool Details

1. fetch - Fetch Web Content

Fetch content from the specified URL and return it in the specified format.

Parameters:

• url (required, string): Target URL, supports HTTP/HTTPS protocols
• return_content (optional, enum): Content processing format, defaults to "markdown"
  • "raw": Return raw HTML content with complete HTTP response headers
  • "basic_clean": Basic cleanup, removing non-display tags (script, style, meta, etc.) while preserving HTML structure
  • "strict_clean": Strict cleanup, removing non-display tags and most HTML attributes, keeping only essential structure
  • "markdown": Convert HTML content to clean Markdown format

Returns: Content processed according to the specified format

---

2. fetch_to_file - Fetch Web Content and Save to File

Fetch content from the specified URL and save it to a file.

Parameters:

• url (required, string): Target URL
• file_path (required, string): File save path
  • When --use-root is enabled: Must be a relative path (relative to workspace root)
  • When --use-root is disabled: Must be an absolute path
• return_content (optional, enum): Content processing format, defaults to "markdown", same options as fetch tool

Features:

• Automatically creates parent directories for nested paths
• All files saved with UTF-8 encoding
• When --use-root is enabled, can work with --allow-external-file-access to control file access scope

Returns: Operation result or error message

---

3. http_request - Generic HTTP Request Tool

Send HTTP requests using any method and get complete responses.

Parameters:

• url (required, string): Target URL
• method (optional, enum): HTTP method, defaults to "GET"
  • "GET", "POST", "PUT", "PATCH", "DELETE"
• query (optional, object): URL query parameters as key-value pairs, automatically URL-encoded
• headers (optional, object): Custom HTTP request headers
• data (optional, string): Text format request body data, mutually exclusive with json parameter
• json (optional, any type): JSON format request body data, mutually exclusive with data parameter

Parameter Constraints:

• data and json parameters cannot be used together
• When using json parameter, Content-Type: application/json is automatically set
• When using data parameter, it's recommended to manually set appropriate Content-Type

Returns: Complete HTTP response (status line, response headers, and response body)

---

Note: Individual HTTP method tools (http_get, http_post, http_put, http_patch, http_delete) have been removed and replaced by the unified http_request tool.

License

MIT