mcpField
M
mcp-llm-gateway

mcp-llm-gateway

Local
github.com/daedalusAI & Agents0Updated 27w ago
Repository

About mcp-llm-gateway

mcp-llm-gateway is an MCP server that routes completion requests to OpenAI-compatible downstream providers. It offers model discovery alongside configurable completion requests.

Key features

  1. 01List downstream models
  2. 02Submit completion requests
  3. 03Expose model-list resource
  4. 04Expose gateway configuration

Use cases

  1. 01Check models available from a provider
  2. 02Send prompts to a downstream LLM
  3. 03Inspect gateway settings

MCP support

The MCP server provides `list_models` and `complete` tools, plus `models://list` and `config://info` resources.

Configuration

Local server config is the same for every client. Copy it into your client’s MCP config file.

mcp.json · Universal config
{
  "mcpServers": {
    "daedalus-mcp-llm-gateway": {
      "command": "uvx",
      "args": [
        "mcp-llm-gateway"
      ]
    }
  }
}
Run commanduvx mcp-llm-gateway

Local servers need a runtime on your machine (e.g. Node.js 18+). Config file locations differ by client; see your client’s docs.

Packages
pypimcp-llm-gateway0.1.0

Tools

2

Saved tool list · updated 3w ago

ParameterTypeRequiredDescription
provideranyNoOptional provider ID to filter models.

Details

From the project README

MCP LLM Gateway

MCP-compatible LLM gateway that proxies completion requests to downstream OpenAI-compatible providers.

PyPI Python Ruff

mcp-name: io.github.daedalus/mcp-llm-gateway

Install

pip install mcp-llm-gateway

Usage

Configuration

Set the following environment variables:

  • DOWNSTREAM_URL: Base URL for the OpenAI-compatible downstream API (required)
  • DEFAULT_MODEL: Default model to use for completions (required)
  • MODEL_LIST_URL: URL to fetch available models from (optional, defaults to models.dev)
  • API_KEY: Optional API key for downstream (passthrough)
  • TIMEOUT: Request timeout in seconds (optional, default: 60)
MCP Server

Run the MCP server with stdio transport:

mcp-llm-gateway
MCP Tools

The server exposes the following tools:

  • list_models(): List all available models from the remote endpoint
  • complete(prompt, model, max_tokens, temperature): Send a completion request to the downstream LLM provider
MCP Resources
  • models://list: Returns the list of available models
  • config://info: Returns current gateway configuration

Development

git clone https://github.com/daedalus/mcp-llm-gateway.git
cd mcp-llm-gateway
pip install -e ".[test]"

# run tests
pytest

# format
ruff format src/ tests/

# lint
ruff check src/ tests/

# type check
mypy src/

API

core.models
  • Model: Dataclass representing an available LLM model
  • CompletionRequest: Dataclass for completion request payloads
  • GatewayConfig: Dataclass for gateway configuration
adapters.http
  • HTTPAdapter: HTTP client for downstream API communication
  • ModelListAdapter: Adapter for fetching model list from remote endpoints
services.gateway
  • ModelService: Service for managing model discovery and caching
  • CompletionService: Service for handling completion requests
  • ConfigService: Service for managing gateway configuration

Comments

Markdown supported

New to MCP?

From what the protocol is to installing your first server, these answers cover most first questions.

An open protocol that defines how AI applications connect to external tools and data in a uniform way. Clients talk to the model; MCP servers provide tools, resources and prompt templates. They exchange standard messages, so one server can be reused by many clients.