mcpField
O
omni-inference

omni-inference

github.com/cheaper-inferenceDevelopment82.3k1.1k installsUpdated today
Repository

About omni-inference

omni-inference describes OmniRoute's OpenAI-compatible inference API for AI agents and other integrations. It covers chat and Responses requests alongside image, audio, embedding, moderation, reranking, and search capabilities, with model discovery and provider combos.

Key features

  1. 01OpenAI-compatible chat and Responses APIs
  2. 02Image generation and editing endpoints
  3. 03Text-to-speech and speech transcription
  4. 04Embedding, moderation, and reranking
  5. 05Model discovery across capability types
  6. 06RTK compression for tool results

Use cases

  1. 01Send chat or code-generation requests
  2. 02Generate or edit images through an API
  3. 03Transcribe audio or synthesize speech
  4. 04Create embeddings for text inputs

MCP support

OmniRoute also includes an MCP server, as stated in the README. This skill describes inference API endpoints and does not specify the MCP tools they expose.

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/cheaper-inference/omniroute --skill omni-inference

After installing, type/omni-inferencein Claude Code to confirm it loaded.

Bundled filesreferences/endpoints.md

Details

From the project README

Overview

The core OpenAI-compatible inference endpoints: chat completions, embeddings, images, audio (TTS/STT), moderations, rerank, and the Responses API. The primary integration surface for AI agents.

Authentication

All requests require a valid Bearer token or session cookie. Obtain a token via POST /api/auth/login or configure REQUIRE_API_KEY=false for local development.

Endpoints

Payloads

See the full OpenAPI specification at GET /api/openapi/spec or docs/openapi.yaml for detailed request/response schemas.

Chat completions

Requires OMNIROUTE_URL and OMNIROUTE_KEY. See entry-point SKILL for setup.

Endpoints
  • POST $OMNIROUTE_URL/v1/chat/completions — OpenAI format
  • POST $OMNIROUTE_URL/v1/messages — Anthropic Messages format
  • POST $OMNIROUTE_URL/v1/responses — OpenAI Responses API
Discover
curl $OMNIROUTE_URL/v1/models | jq '.data[].id'

Combos (e.g. auto, cost-optimized, subscription) auto-fallback through multiple providers.

OpenAI format example
curl -X POST $OMNIROUTE_URL/v1/chat/completions \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-opus-4-7",
    "messages": [{"role": "user", "content": "Refactor this function"}],
    "stream": true
  }'
Anthropic format example
curl -X POST $OMNIROUTE_URL/v1/messages \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-opus-4-7",
    "max_tokens": 4096,
    "messages": [{"role": "user", "content": "Hi"}]
  }'
Tool use

Supports OpenAI tools array and Anthropic tools block. Tool results auto-compressed via RTK (47 filters: git-diff, grep, test-jest, terraform-plan, docker-logs, etc.) — 20-40% token savings. Disable per-request with X-Omniroute-Rtk: off header.

Reasoning / thinking

Anthropic extended thinking and OpenAI Responses reasoning blocks are forwarded verbatim. Cached automatically via reasoning cache.

Errors
  • 401 → invalid API key
  • 400 invalid_model → model not in registry; check /v1/models
  • 503 circuit_open → provider circuit breaker tripped; retry later or use combo
  • 429 rate_limited → honor Retry-After; consider using a combo for auto-fallback

Image generation

Requires OMNIROUTE_URL and OMNIROUTE_KEY. See entry-point SKILL for setup.

Endpoints
  • POST $OMNIROUTE_URL/v1/images/generations — Text-to-image
  • POST $OMNIROUTE_URL/v1/images/edits — Image edit (mask)
  • POST $OMNIROUTE_URL/v1/images/variations — Variations
Discover
curl $OMNIROUTE_URL/v1/models/image | jq '.data[]'

Returns { id, owned_by, sizes:[...], capabilities:[...] } per model.

Generate example
curl -X POST $OMNIROUTE_URL/v1/images/generations \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "dall-e-3",
    "prompt": "a red bicycle on a wet street, photoreal",
    "n": 1,
    "size": "1024x1024",
    "response_format": "b64_json"
  }'

Response: { created, data: [{ url? or b64_json, revised_prompt }] }

Errors
  • 400 invalid_size → not supported by this model; check /v1/models/image
  • 400 content_policy_violation → blocked by provider safety
  • 503 → provider unavailable; try another model in /v1/models/image

Text-to-speech

Requires OMNIROUTE_URL and OMNIROUTE_KEY. See entry-point SKILL for setup.

Endpoint
  • POST $OMNIROUTE_URL/v1/audio/speech — returns binary audio (mp3/opus/wav/flac)
Discover
curl $OMNIROUTE_URL/v1/models/tts | jq '.data[]'

Each entry includes voices:[...] for the available voice names per provider.

Example
curl -X POST $OMNIROUTE_URL/v1/audio/speech \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "tts-1",
    "input": "Hello from OmniRoute.",
    "voice": "alloy",
    "response_format": "mp3"
  }' --output speech.mp3
Voices

Voice names vary by provider. Check /v1/models/tts — each entry has voices:[...]. Common OpenAI voices: alloy, echo, fable, onyx, nova, shimmer.

Errors
  • 400 invalid_voice → voice not supported by this model
  • 400 input_too_long → input exceeds model character limit
  • 503 → provider unavailable; try another model in /v1/models/tts

Speech-to-text

Requires OMNIROUTE_URL and OMNIROUTE_KEY. See entry-point SKILL for setup.

Endpoints
  • POST $OMNIROUTE_URL/v1/audio/transcriptions — multipart upload, returns text
  • POST $OMNIROUTE_URL/v1/audio/translations — transcribe + translate to English
Discover
curl $OMNIROUTE_URL/v1/models/stt | jq '.data[]'
Example
curl -X POST $OMNIROUTE_URL/v1/audio/transcriptions \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -F "[email protected]" \
  -F "model=whisper-1" \
  -F "response_format=verbose_json"

Response: { text, language, duration, segments?:[{ start, end, text }] }

Supported formats

Audio: mp3, mp4, mpeg, mpga, m4a, wav, webm. Response formats: json, text, srt, verbose_json, vtt.

Errors
  • 400 invalid_file_format → unsupported audio format
  • 400 file_too_large → exceeds provider limit (usually 25MB)
  • 503 → provider unavailable; try another model in /v1/models/stt

Embeddings

Requires OMNIROUTE_URL and OMNIROUTE_KEY. See entry-point SKILL for setup.

Endpoint
  • POST $OMNIROUTE_URL/v1/embeddings
Discover
curl $OMNIROUTE_URL/v1/models/embedding | jq '.data[]'

Each entry: { id, owned_by, dimensions, max_input_tokens }.

Example
curl -X POST $OMNIROUTE_URL/v1/embeddings \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "text-embedding-3-large",
    "input": ["first text", "second text"],
    "encoding_format": "float"
  }'

Response: { data:[{ embedding:[...], index }], usage:{ prompt_tokens, total_tokens } }

Batch input

input accepts a string or array of strings (up to provider batch limit, typically 2048 items).

Errors
  • 400 input_too_long → input exceeds max_input_tokens for this model
  • 400 invalid_encoding_format → use float or base64
  • 503 → provider unavailable; try another model in /v1/models/embedding

Web search

Requires OMNIROUTE_URL and OMNIROUTE_KEY. See entry-point SKILL for setup.

Endpoint
  • POST $OMNIROUTE_URL/v1/web/search — unified search format
Discover
curl $OMNIROUTE_URL/v1/models/web | jq '.data[] | select(.kind == "webSearch")'
Example
curl -X POST $OMNIROUTE_URL/v1/web/search \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "tavily/search",
    "query": "OmniRoute github latest release",
    "max_results": 5,
    "include_answer": true
  }'

Response: { answer?, results:[{ url, title, content, score }] }

Parameters
FieldTypeDescription
modelstringProvider model from /v1/models/web
querystringSearch query
max_resultsnumberMax results (default: 5)
include_answerbooleanInclude AI-synthesized answer
search_depthstringbasic or advanced (Tavily)
Errors
  • 400 query_too_long → shorten the search query
  • 503 → provider unavailable; try another model in /v1/models/web

Web fetch

Requires OMNIROUTE_URL and OMNIROUTE_KEY. See entry-point SKILL for setup.

Endpoint
  • POST $OMNIROUTE_URL/v1/web/fetch
Discover
curl $OMNIROUTE_URL/v1/models/web | jq '.data[] | select(.kind == "webFetch")'
Example
curl -X POST $OMNIROUTE_URL/v1/web/fetch \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "jina/reader",
    "url": "https://anthropic.com",
    "format": "markdown"
  }'

Response: { url, title, markdown, links?:[...], images?:[...] }

Parameters
FieldTypeDescription
modelstringProvider from /v1/models/web (e.g. jina/reader, firecrawl/scrape)
urlstringURL to fetch
formatstringmarkdown (default), html, text
Errors
  • 400 invalid_url → URL must be http/https
  • 403 blocked → provider blocked by target site; try a different model
  • 503 → provider unavailable; try another model in /v1/models/web

Comments

Markdown supported