mcpField
T
together-gpu-clusters

together-gpu-clusters

github.com/togethercomputerCloud & DevOps3984 installsUpdated 2w ago
Repository

About together-gpu-clusters

together-gpu-clusters is a skill for planning and operating Together AI GPU clusters when workloads require control over cluster infrastructure. It covers capacity, orchestration, storage, credentials, scaling, and node health.

Key features

  1. 01Provision and manage GPU clusters
  2. 02Choose on-demand or reserved capacity
  3. 03Select Kubernetes or Slurm
  4. 04Manage shared storage and credentials
  5. 05Scale clusters and check node health

Use cases

  1. 01Run distributed training workloads
  2. 02Operate multi-node inference
  3. 03Run HPC or Slurm workloads
  4. 04Deploy custom Kubernetes jobs

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/togethercomputer/skills --skill together-gpu-clusters

After installing, type/together-gpu-clustersin Claude Code to confirm it loaded.

Bundled files

References and scripts installed along with SKILL.md.

agents/
openai.yaml
references/
api-reference.md
cli.md
cluster-management.md
pricing-and-discovery.md
scripts/
manage_cluster.py
manage_cluster.ts
manage_storage.py

Details

From the project README

Together GPU Clusters

Overview

Use Together AI GPU clusters when the user needs infrastructure control instead of a managed inference product.

Typical fits:

  • distributed training
  • multi-node inference
  • HPC or Slurm workloads
  • custom Kubernetes jobs
  • attached shared storage and cluster lifecycle management

When This Skill Wins

  • Provision a cluster and manage it over time
  • Choose between on-demand and reserved capacity
  • Choose Kubernetes or Slurm as the orchestration layer
  • Manage shared volumes and credentials
  • Scale up, scale down, or troubleshoot node health

Hand Off To Another Skill

  • Use together-dedicated-model-inference for managed single-model hosting
  • Use together-dedicated-containers for containerized inference without owning the full cluster
  • Use together-sandboxes for short-lived remote Python execution
  • Use together-fine-tuning for managed training jobs instead of raw cluster operations

Quick Routing

Workflow

  1. Decide whether the workload really needs cluster-level control.
  2. Choose on-demand vs reserved billing based on run duration and baseline utilization.
  3. Choose Kubernetes vs Slurm based on orchestration requirements and team tooling.
  4. Select region, GPU type, driver version, and shared storage plan.
  5. Provision first, then layer in access credentials, workload deployment, scaling, and health checks.

High-Signal Rules

  • Python scripts require the Together v2 SDK (together>=2.0.0). If the user is on an older version, they must upgrade first: uv pip install --upgrade "together>=2.0.0".
  • Prefer managed products unless the user explicitly needs raw infrastructure control.
  • Treat storage lifecycle separately from cluster lifecycle; volumes can outlive clusters.
  • When creating a cluster with new shared storage, prefer inline shared_volume over creating a volume separately and attaching via volume_id. Separately created volumes may land in a different datacenter partition than the cluster, causing a "does not exist in the datacenter" error even when the volume shows as available.
  • GPU stock-outs (409 "Out of stock") are common. Always call list_regions() first and be prepared to try multiple regions.
  • The regions response reports supported configurations, not prices or guaranteed stock. Never infer the cheapest GPU from response order or hardware generation; open the GPU Cluster pricing table and compare its numeric on-demand rates.
  • GPU clusters have an 8-GPU minimum. For an hourly estimate, multiply the displayed per-GPU-hour rate by at least 8.
  • ON_DEMAND has no one-hour duration flag. For a one-hour plan, omit --duration-days, create the cluster only when authorized, then delete it after the intended runtime. If the user requests a command without provisioning, print it but do not run it.
  • The API requires cuda_version and nvidia_driver_version as separate fields in addition to the combined driver_version string. Pass them via extra_body in the Python SDK.
  • Credentials retrieval is part of provisioning. Do not stop at cluster creation if the user needs to run workloads immediately.
  • Slurm and Kubernetes operational patterns differ materially; read the cluster-management reference before improvising.
  • For repeated cluster operations, start from the scripts instead of rebuilding request shapes.
  • Slurm startup scripts (worker/login init, worker/controller prolog and epilog, extra slurm.conf) are Slinky v1.0 only. A non-zero exit from a worker prolog or epilog drains the node, and calling Slurm commands (squeue, scontrol, sacctmgr) inside any prolog/epilog can deadlock the scheduler.

Resource Map

Official Docs

Comments

Markdown supported