verify-gate

Verify Gate

Machine verification gate between implementation and quality review. Runs the project's compile, test, and lint commands. If any fail, enters a fix loop. If all pass, unblocks simplify-and-harden.

This is the inner loop's verify step. Without it, the agent hands off code with zero machine signal about whether it actually works.

When to Use

• After any implementation work completes, before signaling "done"
• Before running simplify-and-harden
• After fixing findings from an independent simplify, harden, spec, or code review
• Any time you want a machine-verified green signal

Pipeline Position

[implementation] → verify-gate → simplify-and-harden → self-improvement
                   ↻ fix loop — on failure, hands diagnostics to self-healing
                   ↳ self-healing (diagnose → patch → verify → file HEAL); verify-gate re-checks

Step 1: Discover Project Commands

Read the project's configuration to find verification commands. Check these sources in order:

1. Project instruction files (CLAUDE.md, AGENTS.md, .github/copilot-instructions.md) — look for a ## Verification or ## Test Commands section
2. package.json — scripts.test, scripts.lint, scripts.typecheck, scripts.build. Also check for a bun.lock / bun.lockb alongside it → prefer bun run <script> over npm run <script> when present. Check for pnpm-lock.yaml → prefer pnpm run. Check for yarn.lock → prefer yarn.
3. Makefile / Justfile — test, lint, check, build targets
4. Cargo.toml — cargo build, cargo test, cargo clippy
5. pyproject.toml / setup.cfg — pytest, mypy, ruff
6. go.mod — go build ./..., go test ./..., go vet ./...
7. deno.json / deno.jsonc — deno task <name> for any defined tasks

If no commands are discoverable, ask the user once and suggest they add a ## Verification section to their project instruction files (CLAUDE.md, AGENTS.md, or equivalent) for future sessions:

## Verification

- Build: `npm run build`
- Test: `npm test`
- Lint: `npm run lint`
- Type check: `npx tsc --noEmit`

Define acceptance evidence

Before running commands, translate each user acceptance criterion into observable evidence. For host-integrated or user-facing work, record the full identity chain:

• Host/surface — the requested app, browser, desktop host, embedded surface, or target conversation
• Workspace — the repository, branch, environment, or document actually exercised
• Loaded artifact — the bundle, revision, or build the host loaded, not merely the source that built
• Outcome — the requested behavior observed through the real interaction and ordinary user path

Do not substitute intermediate proof for the requested outcome: an iframe src is not a rendered app, a successful build is not proof that the host loaded it, an expanded-only view does not prove the ordinary view, an instant synthetic click does not prove a human-duration press, and a queued or delivered acknowledgement is not a visible turn in the target conversation.

If the environment cannot observe a required outcome, record the exact acceptance gap. The gate cannot report PASSED for that criterion.

Step 2: Run Verification

Run discovered commands in this order. Stop at the first failure category.

Phase 1: Compile / Type Check

Run the build or type-check command. These catch structural errors before wasting time on tests.

Exit 0 → proceed to Phase 2
Exit non-zero → enter fix loop with compiler output

Phase 2: Tests

Run the test command. Scope to changed files if the test runner supports it.

If the selected project tests use agent-qa, read agent-qa verification evidence. Validation and dry-run selection do not satisfy the test phase. Bind completed run evidence to the acceptance identity from Step 1, and retain any outcome gap for Phase 5.

If the selected tests or outcome checks use Playwright, read Playwright verification evidence. Keep Playwright Test results separate from browser-only observations, and retain any outcome gap for Phase 5.

Exit 0 → proceed to Phase 3
Exit non-zero → enter fix loop with test output

Phase 3: Lint (optional, skippable with --skip-lint)

Run the lint command. Lint failures are lower severity but still worth catching.

Exit 0 → proceed to Phase 4
Exit non-zero → enter fix loop with lint output

Phase 4: Project-specific checks

Run configured custom verification tools after the standard phases. Each custom command must prove its stated invariant rather than merely exit successfully for unrelated input.

Phase 5: Outcome acceptance

Exercise every acceptance check defined in Step 1 through the requested host and ordinary user path. Capture enough evidence to identify the host, workspace, loaded artifact, interaction, and observed result. A proxy or intermediate state fails this phase even when compile, tests, and lint are green.

Step 3: Fix Loop

When a phase fails:

1. Read the output. Parse the error output for actionable diagnostics — file paths, line numbers, error messages.
2. Scope the fix. Only fix what the verification caught. Do not refactor, improve, or touch unrelated code.
3. Apply the fix. Make the minimal change to resolve the failure.
4. Re-run the failed phase. Not all phases — just the one that failed.
5. If it passes, continue to the next phase.
6. If it fails again, increment the attempt counter.

Fix Loop Limits

• Default max attempts: 3 per phase (configurable via --fix-limit N)
• Counter increments on every attempt, even if the error changes. Fixing Error A and uncovering Error B counts as attempt 2, not attempt 1. The counter tracks fix attempts, not unique errors.
• If limit reached: Stop. Report what failed, what was tried, and the remaining error output. Do not guess further — signal to the user that manual intervention is needed.
• Total budget: The fix loop should not exceed 20% of the original implementation effort. If fixes are snowballing, stop and report.

Step 4: Gate Signal

When all phases pass:

## Verify Gate: PASSED

- Build: passed
- Tests: passed (N tests, M suites)
- Lint: passed (or skipped)
- Custom checks: passed (or not applicable)
- Outcome acceptance: passed (or not applicable)

Ready for simplify-and-harden.

When the fix loop is exhausted:

## Verify Gate: BLOCKED

- Build: passed
- Tests: FAILED (attempt 3/3)
  - [file:line] error description
  - [file:line] error description
- Lint: not reached
- Outcome acceptance: not reached

Fix loop exhausted. Manual intervention needed before quality review.

Integration with Other Skills

skill-pipeline

verify-gate should run at every pipeline depth except Trivial:

Task size	Pipeline
Trivial	None
Small	verify-gate → simplify-and-harden
Medium	intent-framed-agent + verify-gate → simplify-and-harden
Large	Full pipeline with verify-gate before quality pass

Independent audit agents

After fixes prompted by simplify, harden, spec, or code-review agents, rerun verify-gate. An auditor's report is review evidence, not proof that the modified code still compiles or passes tests.

self-healing

On any failure during the verify run, hand the diagnostics to self-healing (don't just retry the same command). Self-healing runs the diagnose → patch → verify loop, files a HEAL- entry to .learnings/HEALS.md, and returns control. Verify-gate then re-runs the checks. Up to 3 heal attempts per phase before abandoning.

self-improvement

If the heal loop surfaces a recurring pattern (Recurrence-Count >= 3 in HEALS.md), the heal's Handoff block flags it for promotion via self-improvement to CLAUDE.md / AGENTS.md / a new skill. For non-heal learnings (corrections, knowledge gaps, feature requests), log to .learnings/LEARNINGS.md, ERRORS.md, or FEATURE_REQUESTS.md per the self-improvement skill.

What This Skill Does NOT Do

• Does not review code quality (that's simplify-and-harden)
• Does not check security (that's simplify-and-harden and independent review)
• Does not verify spec compliance (that's independent review against the plan/spec)
• Does not modify test files or add new tests
• Does not run tests for code it didn't change (unless the test runner doesn't support scoping)

Configuration

If the project has a .verify-gate.yml or a verify-gate section in its project instruction files (CLAUDE.md, AGENTS.md, or equivalent):

verify-gate:
  build: npm run build
  test: npm test
  lint: npm run lint
  type_check: npx tsc --noEmit
  fix_limit: 3
  skip_lint: false
  test_scope: changed  # changed | all

If no configuration exists, discover commands automatically (Step 1) and suggest persisting them.

Custom Verification Tools (mcp-scripts)

Projects with custom invariants can define inline verification tools using gh-aw's mcp-scripts. These run as additional phases after the standard compile/test/lint checks.

Example — a project that needs API schema validation and legacy import checks:

# In .github/workflows/verify-gate-ci.md or plugin config
mcp-scripts:
  verify-api-schema:
    lang: shell
    description: "Validate API schema matches implementation"
    run: |
      python scripts/validate_schema.py --strict

  check-no-legacy-imports:
    lang: shell
    description: "Ensure no imports from deprecated legacy/ directory"
    run: |
      ! grep -r "from legacy" src/ --include="*.py"

  verify-rate-limits:
    lang: javascript
    description: "All API routes must have rate limiting middleware"
    run: |
      const routes = require('./src/routes');
      const missing = routes.filter(r => !r.middleware.includes('rateLimit'));
      if (missing.length) { console.error('Missing rate limit:', missing); process.exit(1); }

When mcp-scripts are defined, verify-gate runs them as Phase 4 after lint. Each script's exit code determines pass/fail. Failed scripts enter the same fix loop as standard phases. Host and user-observable acceptance remains Phase 5 and cannot be replaced by a custom command that proves only an intermediate state.

This moves project-specific invariants from "knowledge in your head" to "knowledge in the harness" — exactly where the agent can reach it.