mcpField
M
MCP YOLOE: Zero-Shot Object Detection & Segmentation

MCP YOLOE: Zero-Shot Object Detection & Segmentation

Local
github.com/rjn32sMedia & Design1Updated 35w ago
Repository

About MCP YOLOE: Zero-Shot Object Detection & Segmentation

MCP-YOLO is an MCP server for developers and AI agents that detects and segments objects in images using text prompts, without retraining. It accepts local paths, URLs, and Base64 image data.

Key features

  1. 01Text-prompt object detection
  2. 02Instance segmentation masks
  3. 03Local, URL, and Base64 images
  4. 04Custom detection classes

Use cases

  1. 01Find objects described in natural language
  2. 02Generate masks for objects in images
  3. 03Analyze images from paths or URLs

MCP support

Exposes the detect_objects and segment_objects tools to MCP clients for text-guided image detection and segmentation.

#vision#ai#object#detection#segmentation#zero-shot#yolo

Configuration

Local server config is the same for every client. Copy it into your client’s MCP config file.

mcp.json · Universal config
{
  "mcpServers": {
    "yolo": {
      "command": "uvx",
      "args": [
        "mcp-yolo"
      ]
    }
  }
}
Run commanduvx mcp-yolo

Local servers need a runtime on your machine (e.g. Node.js 18+). Config file locations differ by client; see your client’s docs.

Packages
pypimcp-yolo0.1.2

Tools

No tool list yet

Local servers expose tools only once running on your machine, and the maintainer has not provided a tool list.

Details

From the project README

MCP-YOLO

PyPI Downloads mcp-name: io.github.rjn32s/mcp-yolo

MCP-YOLO is an agent-first development platform that provides Zero-Shot Object Detection and Segmentation as a Model Context Protocol (MCP) server. Powered by Ultralytics YOLOE, it enables developers and AI agents to detect and segment objects using arbitrary text prompts without retraining.

Key Features

  • Zero-Shot Detection: Detect any object using natural language (e.g., "the blue coffee cup next to the spoon").
  • Instance Segmentation: Precise polygon masks for discovered objects.
  • Flexible Image Inputs: Supports local file paths, remote URLs, and Base64 encoded strings.
  • Agent Optimized: Includes custom "Skills" for autonomous deployment and benchmarking.

YOLOE Performance Reference

YOLOE builds upon the latest YOLO architectures (like YOLO11 and YOLO26) to provide state-of-the-art open-vocabulary performance.

ModelBased OnmAP (COCO)Speed (T4/ms)Params (M)
YOLOE26-NYOLO26-N40.91.7~3.0
YOLOE26-SYOLO26-S48.62.5~10.0
YOLOE26-LYOLO26-L55.06.2~40.0
YOLOE-LYOLO11-L~52.0~5.0~26.0

Note: Performance varies depending on the hardware and input resolution. mcp-yolo uses yoloe-26l-seg.pt by default for high precision.

Quick Start

Installation
uv pip install mcp-yolo
Running the Server
uv run mcp-yolo

MCP Tools

detect_objects

Performs zero-shot detection.

  • Arguments:
    • image_source (str): Path, URL, or Base64.
    • classes (list[str], optional): Custom text prompts to detect.
segment_objects

Performs zero-shot instance segmentation.

  • Arguments:
    • image_source (str): Path, URL, or Base64.
    • classes (list[str], optional): Custom text prompts to segment.

Publishing

This project is configured for automated PyPI publishing. See the pypi_setup_guide.md for details.

Comments

Markdown supported

New to MCP?

From what the protocol is to installing your first server, these answers cover most first questions.

An open protocol that defines how AI applications connect to external tools and data in a uniform way. Clients talk to the model; MCP servers provide tools, resources and prompt templates. They exchange standard messages, so one server can be reused by many clients.