mcpField
C
codabench-mcp

codabench-mcp

Local
github.com/harshitAgrScience & Education0Updated 27w ago
Repository

About codabench-mcp

codabench-mcp is an MCP server for AI agents participating in Codabench ML benchmarks. It covers the participant workflow, from competition discovery and dataset download to submissions and leaderboard checks.

Key features

  1. 01Find competitions and inspect rules
  2. 02Browse phases and tasks
  3. 03Download datasets with SHA-256 checks
  4. 04Submit files and track submission status
  5. 05View leaderboards and profile
  6. 06Access the REST API with a GET-first fallback

Use cases

  1. 01Find a suitable ML benchmark and review its rules
  2. 02Download competition data for local analysis
  3. 03Submit benchmark results and check processing status
  4. 04Review leaderboard standings and submission logs

MCP support

Exposes Codabench REST API operations as tools for MCP clients. Authentication uses a Codabench API token.

Configuration

Local server config is the same for every client. Copy it into your client’s MCP config file.

mcp.json · Universal config
{
  "mcpServers": {
    "harshitagr-codabench": {
      "command": "uvx",
      "args": [
        "codabench-mcp"
      ],
      "env": {
        "CODABENCH_API_TOKEN": "<CODABENCH_API_TOKEN>"
      }
    }
  }
}
Run commanduvx codabench-mcp
Environment variables to set
CODABENCH_API_TOKENSecretDRF token for Codabench. Obtain via https://www.codabench.org/api/docs/ > api-token-auth > Try it out.

Local servers need a runtime on your machine (e.g. Node.js 18+). Config file locations differ by client; see your client’s docs.

Packages
pypicodabench-mcp0.1.1

Tools

No tool list yet

Local servers expose tools only once running on your machine, and the maintainer has not provided a tool list.

Details

From the project README

codabench-mcp

MCP server for the Codabench REST API. Lets an AI agent drive a full participant ML-benchmark workflow: discover competitions, read rules, download data, submit, poll, read leaderboards.

Features

Curated tools for the participant path:

  • search_competitions, get_competition, list_competition_phases, get_phase, get_competition_rules, list_competition_tasks, get_task
  • download_dataset (streaming + SHA-256)
  • list_my_submissions, get_submission, get_submission_logs
  • submit_to_phase (handles Codabench's 3-step upload flow)
  • poll_submission (backoff + timeout + non-error "still_running")
  • get_leaderboard, get_my_profile
  • codabench_request — generic REST escape hatch (GET-only by default)

Authentication

You need a Codabench API token. Two ways to get one:

1. Swagger UI (recommended)
  1. Visit https://www.codabench.org/api/docs/
  2. Find the api-token-auth endpoint → Try it out
  3. POST your username/password
  4. Copy the token value from the response
2. curl
curl -X POST https://www.codabench.org/api/api-token-auth/ \
  -H "Content-Type: application/json" \
  -d '{"username":"YOUR_USERNAME","password":"YOUR_PASSWORD"}'

The response is {"token": "..."}.

Install

uvx codabench-mcp

To run the bleeding-edge main branch instead of the last release:

uvx --from git+https://github.com/harshitAgr/codabench-mcp codabench-mcp

Or for development:

git clone https://github.com/harshitAgr/codabench-mcp.git
cd codabench-mcp
uv sync
uv run codabench-mcp  # requires CODABENCH_API_TOKEN

Configure an MCP client

Claude Desktop (claude_desktop_config.json)
{
  "mcpServers": {
    "codabench": {
      "command": "uvx",
      "args": ["codabench-mcp"],
      "env": {
        "CODABENCH_API_TOKEN": "paste-your-token-here"
      }
    }
  }
}
Claude Code (plugin marketplace)
/plugin marketplace add harshitAgr/codabench-mcp
/plugin install codabench-mcp@codabench-mcp

Then export your token in the shell where you launch Claude Code:

export CODABENCH_API_TOKEN=paste-your-token-here
Claude Code (manual)

If you'd rather skip the plugin layer:

claude mcp add codabench \
  --env CODABENCH_API_TOKEN=paste-your-token-here \
  -- uvx codabench-mcp

Environment variables

VariableRequiredDefaultPurpose
CODABENCH_API_TOKENyes—DRF token, sent as Authorization: Token <token>
CODABENCH_BASE_URLnohttps://www.codabench.orgOverride for tests
CODABENCH_ALLOW_WRITE_RAWno0Set to 1 to allow non-GET methods through codabench_request
CODABENCH_MAX_DOWNLOAD_BYTESno5368709120 (5 GB)Cap for download_dataset

Development

uv sync
uv run pytest                              # unit tests, no network
uv run ruff check .                        # lint
uv run ruff format --check .               # format check

License

MIT — see LICENSE.

mcp-name: io.github.harshitAgr/codabench-mcp

Comments

Markdown supported

New to MCP?

From what the protocol is to installing your first server, these answers cover most first questions.

An open protocol that defines how AI applications connect to external tools and data in a uniform way. Clients talk to the model; MCP servers provide tools, resources and prompt templates. They exchange standard messages, so one server can be reused by many clients.