mcpField
D
debug-my-harness

debug-my-harness

github.com/zernieAgent workflows151.5k installsUpdated 2d ago
Repository

About debug-my-harness

A skill for diagnosing harness behavior from the `.vigiles/runs.jsonl` flight recorder. It analyzes recorded skill activations, hook decisions, subagent tool violations, and metric changes, then recommends evidence-based fixes.

Key features

  1. 01Read and analyze JSONL run records
  2. 02Track skill activations and selection collisions
  3. 03Inspect hook decisions and observe mode
  4. 04Find subagent tool-contract violations
  5. 05Compare evaluation metrics and suggest fixes

Use cases

  1. 01Investigate why a skill stopped firing
  2. 02Find why a hook did not block an action
  3. 03Check whether a subagent used an unauthorized tool
  4. 04Compare run metrics to identify drift

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/zernie/vigiles --skill debug-my-harness

After installing, type/debug-my-harnessin Claude Code to confirm it loaded.

Details

From the project README

Diagnose harness misbehavior from the flight recorder — the local, append-only ledger at .vigiles/runs.jsonl that vigiles writes as your harness runs. It records what actually happened, so you debug from evidence instead of guessing.

What's in the ledger

One JSON record per line, each with a kind:

  • hook — a compiled-hook gate decision: {event, decision: allow|deny|ask, mode: enforce|observe, rule, cmd, reason}.
  • agent — a subagent tool-contract decision: {name, tool, allowed, reason} (a false = the agent went outside its lane).
  • skill — a skill activation: {name, fired}.
  • eval — a measured metric: {name, metric, value} (e.g. trigger-rate recall/precision).
  • capability-diff — a blast-radius change: {pr, added, removed, widened}.

Instructions

Step 1: Read the ledger

Read .vigiles/runs.jsonl (JSONL — one record per line; tolerate a torn last line). If it's absent or empty, say so — there's nothing recorded yet; suggest running the harness (or vigiles audit) first. Do NOT fabricate records.

Step 2: Answer the specific question, evidence-first

Match the user's question to the ledger:

  • "Why did skill X stop firing / why does the wrong one run?" — count skill fires by name over time. If X's fire-rate dropped, look for a sibling that fired on the same kinds of prompts (a selection collision) and check their descriptions for overlap. Recommend differentiating or merging the descriptions.
  • "Why didn't my hook block that?" — find hook records for the event. A decision: allow on something that should be denied, or mode: observe (shadow, never blocks), or the absence of any record, tells you which. Recommend flipping observe→enforce or fixing the gate logic.
  • "Did a subagent misbehave?" — list agent records with allowed: false: the agent reached for a tool outside its declared contract. Point at the contract to tighten or widen.
  • "Is it getting worse?" — compare eval metric values (recall/precision) across runs; a downward trend is drift (often after a harness/model upgrade).
Step 3: Recommend a fix, tied to the evidence

Prefer promoting an ignored-but-decidable rule from prose to a deterministic gate: a repeated agent violation or a rule the agent keeps breaking → a compiled hook or a tighter tool-contract (the strengthen skill can help). A description collision → differentiate the skill descriptions. Always cite the specific records you based the diagnosis on.

Step 4: Offer the next step

If the fix is a spec change, hand off to edit-spec. If it's promoting guidance to a linter rule, hand off to strengthen. If a behavioral claim needs measuring (does the skill fire now?), hand off to test-harness (measureTriggerRate).

Comments

Markdown supported