mcpField
W
wandb-autoresearch

wandb-autoresearch

github.com/wandbData & databases71283 installsUpdated 2w ago
Repository

About wandb-autoresearch

A skill for running training experiments through W&B Launch, inspecting results, and selecting the next bounded trial. It organizes experiment stages and records while requiring queue, budget, and compute safety checks.

Key features

  1. 01Launch-based training trials
  2. 02Readiness checks for code and queue
  3. 03Run and Artifact experiment records
  4. 04Local experiment state tracking
  5. 05Trial budget and compute safeguards

Use cases

  1. 01Running bounded training searches
  2. 02Inspecting a completed trial before choosing the next
  3. 03Resuming experiment decisions across sessions

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/wandb/skills --skill wandb-autoresearch

After installing, type/wandb-autoresearchin Claude Code to confirm it loaded.

Bundled files

References and scripts installed along with SKILL.md.

references/
search.md
setup.md
understand.md
scripts/
autoresearch_state_impl.py
autoresearch_state.py

Details

From the project README

W&B autoresearch

Use W&B Launch to execute real training experiments, read their results, and choose the next bounded trial. Do not create a local wandb.init() run as a substitute for compute that was supposed to run through Launch.

The Launch implementation lives in skills/wandb-primary:

  • skills/wandb-primary/scripts/launch_helpers.py
  • the Launch section in skills/wandb-primary/SKILL.md

Install missing packages explicitly with uv; do not assume a fixed working directory or preinstalled environment.

Flow

Read only the reference for the current stage:

StageEntry conditionRead
UnderstandAlways first; inspect launchable code, queue capacity, data location, and prior statereferences/understand.md
Set upLaunchable code or usable queue capacity is missingreferences/setup.md
SearchLaunchable code and queue capacity both existreferences/search.md

The readiness gate is launchable code + queue with usable capacity. A dataset may be a versioned Artifact or may be supplied by the code/image; its storage form alone does not determine readiness.

Experiment records

Use each W&B object for one job:

  • Runs hold trial config, metrics, system metrics, status, and code/job lineage.
  • Job or code Artifacts make code relaunchable.
  • Dataset Artifacts provide versioned inputs when the workload needs them.
  • Launch queues and agents provide compute.
  • A small local markdown file holds the hypothesis, trial tally, decisions, queue, and next step.

Keep local state at a user-visible path in the current project or a path the user supplied. Sync it to a W&B Artifact only when cross-session resume is useful and the user authorizes the write:

S=skills/wandb-autoresearch/scripts/autoresearch_state.py
uv run --with wandb python "$S" load ENTITY PROJECT --dest .
uv run --with wandb python "$S" save ENTITY PROJECT --path AUTORESEARCH_STATE.md

Compute safety

  • Confirm the queue, metric direction, search axis, and trial/compute budget before the first non-smoke launch. Do not exceed the agreed budget.
  • Smoke-test the real job at a deliberately small setting before full trials.
  • Change only config fields the program actually reads. Use a code edit and a new job Artifact for architecture or pipeline changes.
  • Run trials serially unless the user explicitly authorizes parallel compute. Inspect each completed result before selecting the next trial.
  • Report failures, hardware utilization, and uncertainty. Re-run a likely winner with another seed when the apparent gap could be noise.
  • Never fabricate a run, metric, queue result, or completion state.

Comments

Markdown supported