fiftyone-mcp-server

FiftyOne MCP Server

What is the FiftyOne MCP Server?

Enable Agents to explore datasets, execute operators, and control the FiftyOne App through natural language. This server exposes 45+ MCP tools across data operations, App UI control, and the full operator/plugin ecosystem.

"List all my datasets"
"Load quickstart dataset and show summary"
"Find similar images in my dataset"

The server starts with 50 built-in operators. Install plugins to expand functionality - the AI can discover and install plugins automatically when needed (brain, zoo, annotation, evaluation, and more).

Available Tools

Tool modes

45+ tools organized by runtime mode:

• SDK: Data operations that work everywhere (datasets, aggregations, schema, samples, operators, plugins). No App connection needed.
• APP: Controls the FiftyOne App UI in real time (set_view, open_panel, notify, select_samples, reload, and 25+ more). Requires a connected browser via ctx.ops.
• SESSION: Bootstrap tools for starting a local App server (launch_app). Used from terminal environments.

Choosing your tools

Which tools are available depends on how you integrate the server:

Tool risk levels

Every tool is tagged with a risk level that your agent can use for auto-approval decisions:

• LOW Safe to auto-execute without prompting (read-only queries, UI state changes)
• OPERATOR Wraps a FiftyOne operator whose own severity should be checked before executing

Quick Start

Step 1: Install the MCP Server

pip install fiftyone-mcp-server

⚠️ Important: Make sure to use the same Python environment where you installed the MCP server when configuring your AI tool. If you installed it in a virtual environment or conda environment, you must activate that environment or specify the full path to the executable.

Step 2: Configure Your AI Tool

claude mcp add fiftyone -- fiftyone-mcp

{
  "mcpServers": {
    "fiftyone": {
      "command": "fiftyone-mcp"
    }
  }
}

{
  "mcpServers": {
    "fiftyone": {
      "command": "fiftyone-mcp"
    }
  }
}

{
  "servers": {
    "fiftyone": {
      "command": "fiftyone-mcp"
    }
  }
}

{
  "mcpServers": {
    "fiftyone": {
      "command": "fiftyone-mcp"
    }
  }
}

{
  "mcpServers": {
    "fiftyone": {
      "command": "uvx",
      "args": ["fiftyone-mcp-server"]
    }
  }
}

Step 3: Use It

"List all my datasets"
"Load quickstart dataset and show summary"
"Open the map panel and show me the embeddings"
"Select samples with confidence above 0.9"
"What plugins are available? Install the brain plugin"
"Find near-duplicate images in my dataset"

Claude will automatically discover operators and execute the appropriate tools.

Contributing

We welcome contributions! Here's how to set up a local development environment:

1. Clone the repository

   git clone https://github.com/voxel51/fiftyone-mcp-server.git
   cd fiftyone-mcp-server
   

2. Install dependencies

   poetry install
   

3. Run the server locally

   poetry run fiftyone-mcp
   

4. Test your changes

   poetry run pytest
   poetry run black -l 79 src/
   npx @modelcontextprotocol/inspector poetry run fiftyone-mcp
   

5. Submit a Pull Request

Resources

Community

Join the FiftyOne community to get help, share your ideas, and connect with other users:

• Discord: FiftyOne Community
• GitHub Issues: Report bugs or request features