model-shunt

Model-Shunt 🔀

A decoupled, zero-dependency, universal implementation of the Shunt model-routing pattern (originally conceived by Spotify Engineering).

Model-Shunt allows AI coding agents (Antigravity, Cursor, Windsurf, Claude Code, Aider, OpenHands, etc.) to delegate token-heavy I/O (bulk file reading/code analysis) and repetitive boilerplate generation (tests, mocks, stubs, configs) to fast, economical, or local worker models (Gemini 2.5 Flash, Groq/Llama, Ollama, DeepSeek, GPT-4o-mini). This cuts primary agent token consumption by up to 90% while keeping the main context window clean.

---

⚡ Key Highlights

• Zero External Dependencies: Built with pure Python 3 standard library (urllib, json, re, argparse). No pip install, no virtual environment, and no npm required.
• Agent-Agnostic: Works transparently across any AI coding agent via standard MCP (Model Context Protocol), standalone CLI scripts, or PreToolUse lifecycle hooks.
• Dynamic Model Discovery & Auto-Routing: Queries the worker endpoint in real time to discover available models and automatically routes to the best model for the task:
  • Reader Mode (Bulk I/O): Prioritizes massive context windows and ultra-low cost (e.g., gemini-2.5-flash, llama-3.3-70b-versatile, gpt-4o-mini).
  • Writer Mode (Code Generation): Prioritizes specialized coding models (e.g., qwen2.5-coder:latest, gemini-2.5-flash, deepseek-chat).
• Bypasses Linux ARG_MAX Limits: Unlike naive implementations that pass file contents as CLI arguments (capped at ~128 KB on Linux), Model-Shunt streams corpus data over stdin, allowing analysis of hundreds of thousands of lines without buffer overflows.
• Deterministic Line Numbering (N|): Automatically prefixes every line in file blocks with its 1-based index, forcing worker models to cite verifiable, exact line numbers instead of hallucinating locations.
• Binary File Protection: Inspects byte headers to reject binary files (PDFs, images, compiled objects) before sending them to the LLM.
• Network Resilience: Automatic exponential backoff retries for rate limits (HTTP 429) and transient server errors (HTTP 503/502), with configurable timeouts and token limits.
• Map-Reduce for Oversized Corpora: When a bulk_read payload exceeds the direct limit (SHUNT_MAX_DIRECT_TOKENS, default ~200k tokens), Model-Shunt automatically splits the corpus into chunks, maps the question over each chunk (preserving absolute N| line numbers), and reduces the extracts into one cited answer. Giant single-line files (minified JSON/JS) are sliced by characters with explicit position markers. Rate-limit pacing waits out provider quota windows instead of failing.

---

📁 Repository Structure

model-shunt/
├── src/model_shunt/
│   ├── worker.py              # Universal LLM worker engine with model discovery (zero-deps)
│   └── server.py              # Stdio MCP server exposing routing tools
├── bin/model-shunt.js         # npm/npx launcher shim (requires local Python 3)
├── plugin/
│   ├── .claude-plugin/        # Plugin manifest for hook-compatible agents
│   ├── hooks/                 # PreToolUse interceptor hooks (check-file-size, check-bash-read)
│   ├── scripts/               # Executable streaming CLIs (bulk-read, code-write)
│   └── skills/                # Agent skill manifests (/bulk-reader, /code-writer)
├── pyproject.toml             # PyPI packaging (uvx / pip install)
├── package.json               # npm packaging (npx)
├── config.example.json        # Configuration template
├── test_shunt.py              # Automated test suite
└── .gitignore                 # Credential and cache protection

---

⚙️ Configuration

Configure your worker model via environment variables or a config.json file (placed in ~/.config/model-shunt/config.json or in the project root):

Using config.json

{
  "provider": "gemini",
  "model": "auto",
  "timeout": 90,
  "max_tokens": 8192
}

Tip: Setting "model": "auto" (or passing --auto-model in the CLI) will automatically inspect the provider's active models and pick the optimal one for reading vs writing.

Security: Do not put your API key in config.json — use environment variables instead (e.g. GEMINI_API_KEY, GROQ_API_KEY, or SHUNT_API_KEY). An api_key field exists as a last-resort fallback, but keeping secrets out of files is strongly recommended.

Using Environment Variables

# Google Gemini (Recommended: 1M token context, high speed, ultra-low cost)
export SHUNT_PROVIDER="gemini"
export GEMINI_API_KEY="your-api-key"

# Groq (Ultra-low latency inference)
export SHUNT_PROVIDER="groq"
export GROQ_API_KEY="your-api-key"

# Ollama (100% private, local, and free)
export SHUNT_PROVIDER="ollama"
export SHUNT_BASE_URL="http://localhost:11434/v1"

# OpenAI / DeepSeek / OpenRouter / Anthropic
export SHUNT_PROVIDER="deepseek"
export DEEPSEEK_API_KEY="your-api-key"

---

🛠️ Usage Modes

Mode 1: Universal MCP Server (Recommended)

Model-Shunt provides a standard stdio MCP server exposing three tools:

1. get_available_models(provider?): Queries the provider endpoint and recommends reader and writer models. If discovery fails or is unsupported, returns built-in recommendations whose current availability is not verified.
2. bulk_read(question, file_paths, model?, provider?): Reads large or multiple files and outputs concise, structured bullets with exact line citations. A citation name (N|k) is dropped when source line k does not contain that name.
3. code_write(spec, reference_path, target_path?, model?, provider?): Generates code using a specification and reference file. Returns the code when target_path is omitted; otherwise creates missing parent directories and writes the code, overwriting an existing target file.

bulk_read sends the selected files and question to the configured worker provider; code_write sends the specification and reference file. The provider can be remote or local (such as Ollama). Token savings depend on the input and response.

The MCP server runs locally over stdio. Set provider credentials through environment variables. File access follows resolved paths, so symlinks pointing outside the allowed workspace roots are rejected. Invalid requests return structured errors and leave the server available for subsequent calls.

Installation

MCP Registry name: mcp-name: io.github.yasmanycastillo/model-shunt

Universal one-liner (detects uv / pip / pipx / npm, installs the model-shunt command, and registers it with Claude Code if present):

curl -fsSL https://yasmanycastillo.github.io/model-shunt/install.sh | bash

Manual alternatives:

claude mcp add model-shunt -- uvx model-shunt      # if you have uv
claude mcp add model-shunt -- npx -y model-shunt   # if you have Node + Python

Any MCP client (Cursor, Windsurf, Antigravity, Claude Desktop, etc.) — add to its MCP settings. No clone, no absolute paths:

{
  "mcpServers": {
    "model-shunt": {
      "command": "uvx",
      "args": ["model-shunt"],
      "env": {
        "SHUNT_PROVIDER": "gemini",
        "SHUNT_MODEL": "auto",
        "GEMINI_API_KEY": "your-api-key"
      }
    }
  }
}

Fallback (offline / no uv / no npx): run straight from a clone with Python 3.9+ — replace "command"/"args" with "command": "python3", "args": ["/absolute/path/to/model-shunt/src/model_shunt/server.py"].

Security: by default bulk_read/code_write only operate on files inside the server's working directory (the agent workspace). Set SHUNT_ALLOWED_ROOTS (PATH-style list) to expand the sandbox.

Map-Reduce Tuning (optional)

Variable	Default	Purpose
SHUNT_MAX_DIRECT_TOKENS	200000	Payloads above this estimated size switch to map-reduce
SHUNT_CHUNK_CHARS	600000	Chunk size in characters (~150k tokens)
SHUNT_CHUNK_RETRIES	3	Retries per chunk on rate limits
SHUNT_CHUNK_RETRY_DELAY	60	Seconds to wait out a provider quota window (free-tier TPM)

---

Mode 2: PreToolUse Interceptor Hooks

For agents supporting pre-execution hooks (e.g., Claude Code, custom agent loops):

1. File Read Interceptor (check-file-size):
   • If the agent attempts a whole-file read on a file exceeding the threshold (default: 350 lines, configurable via SHUNT_MIN_LINES), the hook blocks the call and instructs the agent to delegate to bulk-read.
   • Targeted reads with offset and limit are allowed, preserving surgical context for code editing.
2. Terminal Guard (check-bash-read):
   • Prevents agents from bypassing the read hook by executing commands like cat, less, or more on large files directly in the terminal context.

---

Mode 3: Standalone CLI & Scripts

You can also use Model-Shunt directly from the command line or from agent bash sessions:

Discover Available Models & Recommendations

python3 src/model_shunt/worker.py --list-models --provider gemini

Run Bulk Reading Analysis

./plugin/scripts/bulk-read \
  --question "How does the token refresh cycle work?" \
  --paths src/auth.py src/tokens.py \
  --auto-model

Generate Boilerplate Directly to Disk

./plugin/scripts/code-write \
  --spec "Create unit tests for the BillingService covering charge and refund" \
  --reference tests/test_user.py \
  --target tests/test_billing.py \
  --auto-model

---

🧪 Verification

Run the built-in test suite to verify your environment:

python3 test_shunt.py

The test suite validates:

• Configuration resolution, fallback cascades, and model selection.
• Binary file detection and rejection.
• Hook decisions (surgical reads allowed, large file reads blocked, bash flag parsing).
• MCP stdio protocol compliance and tool execution.
• CLI discovery flags.

---

📄 License

MIT. Inspired by Spotify Engineering's Shunt architecture.

🔒 Privacy

See PRIVACY.md for filesystem scope, provider data handling, and credential practices.