mcpField
V
validate-bug

validate-bug

github.com/trailofbitsTesting & quality7.9k170 installsUpdated today
Repository

About validate-bug

validate-bug is a skill for independently checking a bug reproduction package after reproduce-bug. It reruns the unchanged PoC or compiler command and records whether the claim is confirmed, not confirmed, or inconclusive.

Key features

  1. 01Review finding and raw receipts
  2. 02Rerun unchanged reproduction steps
  3. 03Assess claim against stated invariant
  4. 04Package validation receipts and decision

Use cases

  1. 01Independently verify a reproduced bug
  2. 02Resolve uncertain reproduction evidence

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/trailofbits/skills --skill validate-bug

After installing, type/validate-bugin Claude Code to confirm it loaded.

Details

From the project README

Validate Bug

When to Use

Use after reproduce-bug has supplied a finding and PoC package. Read the record contract. Before target commands, follow its project guidance.

When NOT to Use

Do not use to repair a PoC, develop a patch, or infer reproduction from the reproducer's narrative.

Procedure

  1. Read finding.json, the reproduction's proof type, commands, any PoC, and raw receipts. Do not read the reproducer's transcript.

  2. Rerun the same baseline compiler command for compile failures, or rebuild and rerun the unchanged runtime PoC and planned scoped inputs. Save stdout, stderr, exit status and timing in validate/receipts/. If an observation is unreliable, return inconclusive instead of inferring either presence or absence.

  3. Decide whether the target diagnostic or runtime assertion demonstrates the exact claim and violates the stated invariant. Write validate.json as confirmed, not_confirmed, or inconclusive with a short reason and agent identity.

    Validate record schema:

{
  "schema_version": 1,
  "outcome": "confirmed",
  "agent": "validator-1",
  "reason": "The unchanged PoC reached the assertion and demonstrated the claim."
}
  1. Manifest the complete validate/ directory with uv run --no-project <script> manifest <validate-step-dir>, then run the record checker. Return the package without editing the PoC or finding.

Rationalizations to Reject

ShortcutRequired action
Fix the PoC to make the claim work.Mark it inconclusive and explain why.
Trust the reproducer's conclusion.Rerun the unchanged package independently.

Comments

Markdown supported