mcpField
A
add-model

add-model

github.com/berriaiCloud & DevOps103161 installsUpdated 25w ago
Repository

About add-model

add-model is a skill for adding an LLM to a live LiteLLM proxy, aimed at proxy administrators. It gathers the model name, provider, and credentials, adds the model through the management API, and checks request routing.

Key features

  1. 01Gather the model name and provider
  2. 02Set model parameters by provider
  3. 03Add a model through the management API
  4. 04Verify routing with a chat request
  5. 05Report the model_id and test result

Use cases

  1. 01Connect a new model to an existing LiteLLM proxy
  2. 02Check whether the proxy routes to a chosen model

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/berriai/litellm-skills --skill add-model

After installing, type/add-modelin Claude Code to confirm it loaded.

Details

From the project README

Add Model

Add a new LLM to a live LiteLLM proxy.

Setup

Ask for these if not already known:

LITELLM_BASE_URL  — e.g. https://my-proxy.example.com
LITELLM_API_KEY   — proxy admin key

API reference: https://litellm.vercel.app/docs/proxy/model_management

Ask the user

  1. Public model name — what callers will send in "model": "..." (e.g. gpt-4o, my-claude, llama3)
  2. Provider — pick from the table below
  3. Credentials — whatever that provider needs

Provider table

Providerlitellm_params.modelExtra params
OpenAIopenai/gpt-4oapi_key
Azure OpenAIazure/<deployment-name>api_key, api_base, api_version
Anthropicanthropic/claude-3-5-sonnet-20241022api_key
AWS Bedrockbedrock/anthropic.claude-3-5-sonnet-20241022-v2:0AWS creds via env
Google Vertexvertex_ai/gemini-1.5-provertex_project, vertex_location
Ollamaollama/llama3api_base (e.g. http://localhost:11434)
Groqgroq/llama-3.3-70b-versatileapi_key
Together AItogether_ai/meta-llama/Llama-3-70bapi_key
Mistralmistral/mistral-large-latestapi_key

Full list: https://docs.litellm.ai/docs/providers

Run

curl -s -X POST "$BASE/model/new" \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "<public-name>",
    "litellm_params": {
      "model": "<provider/deployment>",
      "api_key": "<key>",
      "api_base": "<base_if_needed>",
      "api_version": "<version_if_azure>"
    }
  }'

Test it

After adding, verify it routes:

curl -s -X POST "$BASE/chat/completions" \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<public-name>",
    "messages": [{"role": "user", "content": "say hi"}],
    "max_tokens": 10
  }'

Output

Show model_id from the response — needed to update or delete the model later. Report pass/fail from the test call.

Comments

Markdown supported