mcpField
W
wandb-eval-tables

wandb-eval-tables

github.com/wandbData & databases71287 installsUpdated 2w ago
Repository

About wandb-eval-tables

A skill for previewing existing W&B Table data as EvalTables without changing source runs or tables. It guides users through choosing data, classifying columns, verifying results, and safely handling limits and removal.

Key features

  1. 01Read-only source scanning
  2. 02Preview one table key at a time
  3. 03Column role guidance
  4. 04Created-run verification
  5. 05Conversion limits and safe removal

Use cases

  1. 01Inspecting W&B tables before evaluation
  2. 02Creating EvalTable previews from selected tables
  3. 03Checking that a preview was logged correctly

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/wandb/skills --skill wandb-eval-tables

After installing, type/wandb-eval-tablesin Claude Code to confirm it loaded.

Bundled files

References and scripts installed along with SKILL.md.

references/
EVAL_TABLES.md
scripts/
table_artifact_to_eval_table_impl.py
table_artifact_to_eval_table.py

Details

From the project README

W&B EvalTable previews

Use the bundled helper to preview existing wandb.Table data as wandb.EvalTable data. A preview creates new runs and keys; it never overwrites the source runs or tables.

Read references/EVAL_TABLES.md before converting. It is the canonical source for EvalTable semantics, source selection, limits, column roles, verification, and removal.

Environment

Run the helper from an environment that provides a W&B SDK with wandb.EvalTable and Weave. Use uv to supply missing dependencies:

uv run --with 'wandb[workspaces]>=0.28.1' --with weave \
  python skills/wandb-eval-tables/scripts/table_artifact_to_eval_table.py --help

Credentials and default scope may come from WANDB_API_KEY, WANDB_ENTITY, and WANDB_PROJECT; otherwise pass explicit entity/project arguments. Downloads use the operating system's temporary directory, not a fixed current working directory.

Required workflow

  1. Choose the narrowest source that matches the request: exact artifact, one run, one sweep, or a project/workspace. Do not broaden an explicitly named source.
  2. Run scan. This is read-only preflight and is not proof of conversion.
  3. Review eligible_summary_by_table_key, table shapes, truncations, warnings, and existing-preview metadata. Select one table key unless the user asks for a broader batch.
  4. Classify columns only when confident. Input tuples must uniquely identify rows; score columns must be numeric or boolean; free-text labels and rationales are outputs. Leave ambiguous tables untyped.
  5. Run preview for exactly one --table-key per invocation. Keep the target in the source project unless the user requests another writable project.
  6. If runs were created, run verify-preview on the final created run and the exact logged key. Report success only when it returns verified: true.
  7. Report created and skipped sources plus every truncation or failed check.
T=skills/wandb-eval-tables/scripts/table_artifact_to_eval_table.py

uv run python "$T" scan \
  --source-workspace ENTITY/PROJECT --max-runs 4

uv run python "$T" preview \
  --source-workspace ENTITY/PROJECT \
  --target-project ENTITY/PROJECT \
  --max-runs 4 \
  --table-key predictions

uv run python "$T" verify-preview \
  --run ENTITY/PROJECT/PREVIEW_RUN_ID \
  --table-key predictions_preview

Safety

  • scan and preview --dry-run are read-only. preview creates W&B runs, EvalTables, and Weave evaluations; run it only when the user requested a conversion.
  • The helper caps tables at 10,000 rows and 100 columns, warns above 1,000 rows, and refuses more than 50 eligible table artifacts for one key. Surface every cap; never imply a truncated preview is complete.
  • Do not bypass helper failures with ad-hoc logging. Narrow or correct helper arguments, then retry only when the change is safe.
  • Removal deletes both Weave evaluations and preview runs. Run delete-preview --dry-run first, verify every target is helper-created, and obtain confirmation before the irreversible invocation. Never delete source runs.

Comments

Markdown supported