mcpField
T
together-chat-completions

together-chat-completions

github.com/togethercomputerDevelopment39107 installsUpdated 2w ago
Repository

About together-chat-completions

together-chat-completions is a guide and script collection for developers using Together AI’s serverless Chat Completions API for interactive text generation. It covers streaming, multi-turn chat, tool calling, structured output, and reasoning models.

Key features

  1. 01Basic and streaming text generation
  2. 02Multi-turn conversation state
  3. 03Tool and function calling
  4. 04Structured outputs
  5. 05Reasoning model configuration
  6. 06Request debugging and parallel calls

Use cases

  1. 01Build a chatbot on Together AI
  2. 02Add streaming to a real-time interface
  3. 03Implement a tool-calling loop
  4. 04Generate responses that follow a JSON schema

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/togethercomputer/skills --skill together-chat-completions

After installing, type/together-chat-completionsin Claude Code to confirm it loaded.

Bundled files

References and scripts installed along with SKILL.md.

agents/
openai.yaml
references/
api-parameters.md
function-calling-patterns.md
models.md
reasoning-models.md
structured-outputs.md
scripts/
async_parallel.py
chat_basic.py
chat_basic.ts
debug_headers.py
debug_headers.ts
reasoning_models.py
reasoning_models.ts
structured_outputs.py
structured_outputs.ts
tool_call_loop.py
tool_call_loop.ts

Details

From the project README

Together Chat Completions

Overview

Use Together AI's serverless chat/completions API for interactive inference workloads:

  • basic text generation
  • streaming responses
  • multi-turn chat state
  • tool and function calling
  • structured outputs
  • reasoning-capable models

Treat this skill as the default entry point for Together AI text generation unless the task is clearly offline batch processing, vector retrieval, model training, or infrastructure management.

When This Skill Wins

  • Build a chatbot, assistant, or text-generation endpoint on Together AI
  • Add streaming output to a real-time user experience
  • Implement tool calling or function-calling loops
  • Constrain model output to JSON or a regex-defined shape
  • Choose between standard chat models and reasoning models
  • Debug request parameters, model behavior, or response shapes

Hand Off To Another Skill

  • Use together-batch-inference for large offline runs, backfills, or lower-cost asynchronous jobs
  • Use together-embeddings for vector search, semantic retrieval, or reranking
  • Use together-fine-tuning when the user wants to train or adapt a model
  • Use together-dedicated-model-inference when the user needs always-on single-tenant hosting
  • Use together-dedicated-containers or together-gpu-clusters for custom infrastructure
  • For production stock-model workloads that need a defined SLA (committed throughput and reliability) without managing hardware, point users to Together's provisioned throughput tier (reserved PTU capacity, one-month minimum term, contact sales; uses the same chat/completions API surface)

Quick Routing

Workflow

  1. Confirm that the workload is interactive serverless inference rather than batch, retrieval, or training.
  2. Pick the smallest model that satisfies latency, quality, and context requirements.
  3. Decide whether the job needs plain text, tools, structured output, or reasoning.
  4. Start from the matching script instead of re-deriving request shapes from scratch.
  5. Pull deeper details from the relevant reference file only when needed.

High-Signal Rules

  • Python scripts require the Together v2 SDK (together>=2.0.0). If the user is on an older version, they must upgrade first: uv pip install --upgrade "together>=2.0.0".
  • Use client.chat.completions.create() for Python and client.chat.completions.create() for TypeScript.
  • Preserve full messages history for multi-turn conversations; do not rebuild context from final text only.
  • For tools, implement the full loop: model tool call -> execute tool -> append tool result -> second model call.
  • For tool definitions, prefer enum over free-form strings, set "additionalProperties": false, and add "strict": true on the function definition when you need argument generation to conform to the schema.
  • Tool names must not contain spaces, periods, or dashes. Branch on finish_reason ("tool_calls" vs "stop") instead of assuming a tool was called, and parse function.arguments as JSON inside a try/except.
  • Prefer json_schema over looser JSON modes when the user needs stable machine-readable output.
  • Use reasoning models only when the task benefits from deeper deliberation; otherwise prefer cheaper standard models.
  • Preserved thinking uses the reasoning key on both output and input (the field is symmetric). When you pass a prior assistant turn back to the API, include "reasoning": ... on the assistant message; reasoning_content is still accepted on input for backward compatibility but prefer reasoning in new code.
  • Reasoning models nest extra token counts OpenAI-style (usage.completion_tokens_details.reasoning_tokens, usage.prompt_tokens_details.cached_tokens), but some non-reasoning models return cached_tokens flat at the top of usage. Read both locations defensively — clients that only check one shape will silently return 0. See references/reasoning-models.md for the defensive-read pattern.
  • To combine tool calling with structured output, use a two-phase approach: Phase 1 sends tools (no response_format), Phase 2 sends response_format (no tools) after tool results are appended.
  • Streaming works with response_format; accumulate chunks and parse the final concatenated string as JSON.
  • If the user needs many independent requests, combine this skill with async_parallel.py or hand off to batch inference.

Resource Map

Scripts

Official Docs

Comments

Markdown supported