mcpField
O
omni-combos-routing

omni-combos-routing

github.com/cheaper-inferenceCloud & DevOps82.3k1.3k installsUpdated today
Repository

About omni-combos-routing

omni-combos-routing is a skill for managing model-routing combos through the OmniRoute API. It is for configuring multi-provider routing and supports varied strategies, fallback chains, route simulation, and combo metrics.

Key features

  1. 01Create, update, and delete routing combos
  2. 02Choose from 19 routing strategies
  3. 03Configure model fallback chains
  4. 04Simulate route choices and inspect reasons
  5. 05Retrieve latency, success, and cost metrics

Use cases

  1. 01Distribute model requests by priority or weight
  2. 02Set fallback order for model failures
  3. 03Simulate routing before making a real call
  4. 04Review combo and target performance metrics

MCP support

When OmniRoute is used as an MCP server, the README lists MCP tools to list and switch combos, change routing strategy, simulate routes, and recommend a combo by task type.

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/cheaper-inference/omniroute --skill omni-combos-routing

After installing, type/omni-combos-routingin Claude Code to confirm it loaded.

Details

From the project README

Overview

Create and manage routing combos with 19 strategies (priority, weighted, round-robin, Auto-combo, and more). Configure fallback chains, test routing outcomes, and retrieve combo metrics.

Authentication

All requests require a valid Bearer token or session cookie. Obtain a token via POST /api/auth/login or configure REQUIRE_API_KEY=false for local development.

Endpoints

GET /api/combos

List routing combos

curl http://localhost:20128/api/combos \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN"
POST /api/combos

Create routing combo

curl -X POST http://localhost:20128/api/combos \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{}'
GET /api/combos/{id}

Get combo by ID

curl http://localhost:20128/api/combos/{id} \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN"
PUT /api/combos/{id}

Update combo

Partial update: the body is merged onto the stored combo, so a field left out keeps its current value. An array that IS sent replaces the stored one outright.

curl -X PUT http://localhost:20128/api/combos/{id} \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{}'
PATCH /api/combos/{id}

Update combo

Partial update: the body is merged onto the stored combo, so a field left out keeps its current value. An array that IS sent replaces the stored one outright.

curl -X PATCH http://localhost:20128/api/combos/{id} \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{}'
DELETE /api/combos/{id}

Delete combo

curl -X DELETE http://localhost:20128/api/combos/{id} \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN"
GET /api/combos/metrics

Get combo metrics

curl http://localhost:20128/api/combos/metrics \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN"
POST /api/combos/test

Test a combo configuration

curl -X POST http://localhost:20128/api/combos/test \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{}'
GET /api/fallback/chains

List fallback chains

Returns all registered fallback chains for model routing.

curl http://localhost:20128/api/fallback/chains \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN"
POST /api/fallback/chains

Create fallback chain

Registers a fallback routing chain for a model.

curl -X POST http://localhost:20128/api/fallback/chains \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{}'
DELETE /api/fallback/chains

Delete fallback chain

curl -X DELETE http://localhost:20128/api/fallback/chains \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN"
GET /api/combos/auto

GET combos › auto

curl http://localhost:20128/api/combos/auto \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN"
GET /api/combos/builder/options

GET combos › builder › options

curl http://localhost:20128/api/combos/builder/options \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN"
POST /api/combos/duplicate

POST combos › duplicate

curl -X POST http://localhost:20128/api/combos/duplicate \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{}'
POST /api/combos/reorder

POST combos › reorder

curl -X POST http://localhost:20128/api/combos/reorder \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{}'

Payloads

See the full OpenAPI specification at GET /api/openapi/spec or docs/openapi.yaml for detailed request/response schemas.

OmniRoute — Routing & Combos

Requires OMNIROUTE_URL and OMNIROUTE_KEY. See entry-point SKILL for setup.

What is a combo?

A combo is a named group of providers/models with a routing strategy. All requests through a combo are automatically distributed, failed-over, and load-balanced — the caller uses a single model ID like my-combo.

List existing combos

curl $OMNIROUTE_URL/api/combos \
  -H "Authorization: Bearer $OMNIROUTE_KEY"

Response includes id, name, strategy, enabled, and per-target stats.

Create a combo

curl -X POST $OMNIROUTE_URL/api/combos \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-combo",
    "strategy": "priority",
    "targets": [
      { "provider": "anthropic", "model": "claude-opus-4-7", "weight": 1 },
      { "provider": "openai",    "model": "gpt-4o",           "weight": 1 }
    ]
  }'

19 public routing strategies

StrategyDescription
priorityAlways use target[0]; fall back on error
weightedDistribute by weight percentage
round-robinRotate targets in order
context-relayChain models for very long contexts
fill-firstFill quota of target[0] before spilling
p2cPower-of-two choices: sample two targets, pick the better candidate
randomUniform random selection
least-usedRoute to the target with the lowest observed usage
cost-optimizedPick the cheapest eligible target for the token estimate
reset-awarePrefer targets based on quota-reset state
reset-windowOrder targets by their configured reset window
headroomPrefer targets with more remaining quota headroom
strict-randomRandom without repeating until all targets have been used
autoAuto-Combo scoring across 16 factors
lkgpLast-known-good-provider sticky routing
context-optimizedPick the best model for the request's context size
cache-optimizedPrefer targets with stronger cache affinity
fusionRun a panel in parallel and synthesize one judged response
pipelineRun a configured sequence of targets as a pipeline

Auto-combo (recommended for production)

Auto-combo scores each candidate on 16 factors every request:

curl -X POST $OMNIROUTE_URL/api/combos \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "prod-auto",
    "strategy": "auto",
    "targets": [
      { "provider": "anthropic", "model": "claude-sonnet-4-6" },
      { "provider": "openai",    "model": "gpt-4o-mini" },
      { "provider": "google",    "model": "gemini-2.0-flash" }
    ]
  }'

Then call it with:

curl -X POST $OMNIROUTE_URL/v1/chat/completions \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "model": "prod-auto", "messages": [{ "role": "user", "content": "Hello" }] }'

Activate / deactivate a combo

# Activate
curl -X PUT $OMNIROUTE_URL/api/combos/{id}/toggle \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -d '{ "enabled": true }'

Get combo metrics

curl $OMNIROUTE_URL/api/combos/{id}/metrics \
  -H "Authorization: Bearer $OMNIROUTE_KEY"

Returns p50/p95/p99 latency, success rate, cost, and per-target breakdown.

Simulate routing (dry run)

curl -X POST $OMNIROUTE_URL/api/routing/simulate \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "comboId": "{id}", "messages": [{ "role": "user", "content": "test" }] }'

Returns which provider would be selected and why — no actual API call is made.

Via MCP (if OmniRoute is your MCP server)

omniroute_list_combos     → list all combos
omniroute_switch_combo    → enable/disable a combo
omniroute_set_routing_strategy → change strategy at runtime
omniroute_simulate_route  → dry-run routing decision
omniroute_best_combo_for_task → get recommendation by task type

Errors

  • 404 combo not found → check id from /api/combos
  • 400 invalid strategy → use one of the 19 strategies above
  • 409 name conflict → combo name already exists

Comments

Markdown supported