mcpField
W
weave-instrument

weave-instrument

github.com/wandbDevelopment7197 installsUpdated 2w ago
Repository

About weave-instrument

A skill for adding and checking Weave tracing in Python or Node applications. It guides developers through authentication checks, instrumentation choices, incremental changes, and trace verification.

Key features

  1. 01Check W&B login before changes
  2. 02Choose spans, auto-instrumentation, or OTel
  3. 03Instrument model, tool, and agent paths
  4. 04Verify traces against execution
  5. 05Run tests and conformance checks

Use cases

  1. 01Add tracing to an agent application
  2. 02Extend an existing OTel pipeline
  3. 03Check trace shape and backend delivery

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/wandb/skills --skill weave-instrument

After installing, type/weave-instrumentin Claude Code to confirm it loaded.

Bundled files

References and scripts installed along with SKILL.md.

references/
otel_auto.md
otel_endpoint.md
read_back.md
session_sdk_python.md
session_sdk_typescript.md
trace_fidelity.md

Details

From the project README

Weave instrumentation

Complete the login check before editing or initializing. If no valid login exists, send only that section's reply and stop.

After a valid login, inspect the application's dependencies, agent boundaries, and existing OTel provider. Confirm entity/project and runtime credentials before sending data; never read, print, or commit API keys. Read trace fidelity before changing any instrumentation path. Initialize once at startup with weave.init("entity/project") in Python or await weave.init(...) in Node.

Check login first

Before editing code or initializing tracing, verify the existing W&B login for the target deployment with a noninteractive, read-only identity request through the configured SDK or connector. Use credentials in place; never print, copy, or inspect secret values. Do not call login or tracing init as a login probe.

If no valid login exists, stop. Do not edit, initialize, or continue this skill. Do not run wandb login in this session. This session's shell is not a normal terminal, so wandb login --verify exits with No API key configured instead of prompting. Do not ask for a key in this chat or create credentials.

Send only this, and nothing else. No decisions, notes, plans, or documentation links:

There's no W&B login yet, so this stops here. Open a new terminal tab and run wandb login --verify. Create an API key at https://wandb.ai/authorize?ref=models. If the browser opens a different page, open that link again. Paste the key in that terminal, not here, and type continue.

On a self-hosted or dedicated deployment, use wandb login --verify --host <base-url> and the authorize page that command prints. Still send only that.

  • If the check is unavailable or fails because of connectivity or permissions, report login/access as unverified and stop; do not assume credentials are invalid.
  • After the user continues, confirm the application runtime can use its own configured credentials and access entity/project. A browser login does not establish SDK authentication. Node/Forge runtimes without a .netrc fallback need credentials configured by the user through their runtime secret mechanism.

Choose the mechanism

  • Explicit agent spans: use the Session SDK for turns, model calls, tools, and delegation when auto-instrumentation cannot produce the required tree. Read the Python or TypeScript reference.
  • Auto-instrumentation: use only when the installed SDK captures the library and desired span shape. Check Python's INTEGRATION_MODULE_MAPPING or Node's integrations/hooks.ts, then verify emitted spans. Read auto-instrumentation caveats; registry membership alone does not prove capture is active. Avoid duplicate manual wrapping.
  • Existing OTel pipeline: add Weave export to that provider using endpoint configuration.

A turn spans one user exchange, including its tool loop. An LLM span covers one model call through stream completion; a Tool covers execution; a SubAgent covers delegation. Record actual output/usage and preserve provider tool-call IDs. Keep return values, exceptions, retries, and cancellation unchanged. Close spans on failure and isolate concurrent conversations.

Implement incrementally

Add one traced path at a time: a model call, its tool loop, then delegation or concurrency where applicable. After each change, run the path and follow read-back verification. Compare emitted and stored spans with the actual execution before extending the instrumentation.

Fix mismatches and rerun the same check before continuing. Empty results, failed queries, and unavailable checks are unverified, not passes. Stop at that boundary and report what the user must do to unblock verification.

Verify

Run relevant application tests and the conformance check for the paths you instrumented. If the full suite requires CI, browsers, or unavailable services, run focused local checks covering success and failure, and report the skipped coverage. Check parentage and gen_ai.operation.name: invoke_agent for turns/subagents, chat for LLMs, execute_tool for tools. Agent-shaped traces belong in Agents; flat calls in Calls. Confirm backend arrival when credentials are available; an init banner or successful flush is not delivery proof. Otherwise report the unverified surface and give the application's exact smoke command.

Adapted from Weave. Use the installed SDK's exports when they differ from these references.

Comments

Markdown supported