mcpField
V
verify

verify

github.com/niekcandaele测试与质量648 次安装2 天前更新
代码仓库

关于 verify

verify 是面向编码 Agent 的软件变更验证技能。它识别变更范围,协调审查与测试、运行端到端检查,并汇总结果以供分流或修复。

主要功能

  1. 01自动识别变更范围
  2. 02轻量与深度审查模式
  3. 03静态分析与测试执行
  4. 04端到端应用检查
  5. 05报告、分流与修复流程

适用场景

  1. 01完成前检查暂存或分支变更
  2. 02对高风险差异执行全面审查
  3. 03生成报告但不应用修复

安装

复制下面的命令,在终端中运行。

$npx skills add https://github.com/niekcandaele/skills --skill verify

安装后在 Claude Code 中输入/verify确认已加载。

详细介绍

验证更改 — 优先分诊的流水线

目标

在将更改视为完成之前,执行全面验证。此技能会检测更改内容,将文件分诊到相关技能,运行静态分析,并行调用审查技能,端到端运行应用,并生成包含可执行问题的统一报告。

阶段 0:解析参数和模式

解析 $ARGUMENTS,获取:

模式(--mode=):

  • interactive(默认):完整流水线 → 报告 → 交互式分诊 → 计划 → 修复
  • report-only:完整流水线 → 报告 → 停止
  • auto-fix:完整流水线 → 报告 → 自动接受严重性 >= 阈值 → 计划 → 修复 → 停止

深度(--depth=):

  • light(默认):一次独立判断审查,加上实证
  • deep:完整的判断审查分流

组合 — 流水线运行的内容,按深度划分:

light(默认)deep
分诊(阶段 3)仅 Job AJob A + Job B
判断审查一个:codex-reviewer;若 Codex 不可用,则使用 reviewerreviewer、codex-reviewer、comment-review、qa、ux-reviewer
确定性检查static-analysis、testerstatic-analysis、tester
实证检查exerciser,自带视觉视角exerciser;若分诊判定需要,则加上 visual-verify
诊断失败时运行 debugger失败时运行 debugger
计划完整性计划确定后检查计划确定后检查

此表是组合的唯一权威来源;调用方应引用此处,而不是重复说明。

两种深度代表不同的取舍,而不是同一种取舍的不同程度。 Deep 会针对同一个差异引入更多视角:五位审查者各有所长,也都可以提出其他审查者可能遗漏的调查方向。Light 则侧重模型多样性和实证:如果环境中有真正不同的模型,就由它进行一次审查;此外还会启动应用并实际使用该功能。两种深度都会保留那些不会偏离范围的检查:linter 和测试运行器只报告结果,不会自行扩大职责范围。

Light 是默认选项,因为调用方通常是已经审查并测试过自身工作的模型。在此基础上再叠加五位判断审查者,并不会让结果收敛:每一轮都会修复上一轮发现的问题,而审查者面对一个刚扩大的差异时,又会提出新的调查方向,重新检查前几轮已经通过的代码。每次修复都会带来新的待审查内容。

Deep 适用于值得付出这笔成本的更改——例如整个 epic 的集成结果、风险异常高的差异,或是人类明确要求的情况。请明确指定;不会有任何机制替你选择它。

范围控制:

  • --scope=staged:仅验证已暂存的更改
  • --scope=unstaged:仅验证尚未暂存的已修改文件
  • --scope=branch:验证当前分支相对于基准的所有更改
  • --scope=all:验证整个代码库(全面审计——跳过分诊,对所有内容运行全部 Agent)
  • --files="file1,file2":仅验证指定文件
  • --module=path:验证指定模块/目录
  • 默认(未提供范围参数):根据 git 状态自动检测
  • --base=<ref>:指定 --scope=branch 的精确基准 ref,例如 origin/release。其他任何范围都不接受此参数。知道 PR 目标分支的调用方必须传入该参数;不要用对默认分支的猜测替代。 其他选项:
  • --skip-ux:对纯后端更改跳过 UX 审查。**仅在 --depth=deep 时有效。**在 light 模式下,没有可跳过的 ux-reviewer,因此接受此标志会让调用方误以为它抑制了某项本来就不会运行的内容。应将其作为参数错误拒绝,而不是当作无操作处理。
  • --skip-visual:对人工已检查过的更改(例如仅修改文案)跳过视觉审查。在两种深度下均有效:在 deep 模式下跳过 visual-verify,在 light 模式下则抑制 exerciser 的视觉检查。若可用技能中没有 visual-verify,也会自动跳过视觉审查(出于防御性考虑——它通常是全局安装的,项目也可能用自己的副本覆盖它,以自定义捕获惯例)。
  • --auto-fix-threshold=N:自动修复模式的最低严重级别(默认值:3)
  • --plan-file=<path>:用于完整性检查的计划文件显式路径。如果未提供,则从上下文中发现计划——检查对话历史中是否有可见计划(例如,由读取过计划的 player-coach 调用,或在本次会话中较早创建或讨论过计划)。如果从任一来源找到计划,则解析其内容,用于 Phase 6 中的计划完整性检查。

在多次验证运行之间沿用结果:

对同一项更改重复运行验证的调用方——显而易见的例子是 player-coach——可以提供先前运行已确定的内容。若不提供,每次运行都是首次运行:新的审查者会重新审计不断扩大的差异,每一轮都会对前几轮已确认无误的代码开启新的调查方向。审查结果永远无法收敛,只会不断偏移。

  • --ledger=<path>:要读取的运行记录(参见 player-coach 的 references/run-ledger.md)。只读——verify 从不写入该文件。
  • --since=<sha>:上次验证运行批准的提交头。需要 --ledger。 用于定义增量。
  • --no-carry-forward:提供运行记录以供报告,但按首次运行的方式进行审查。这是在批准前请求确认性完整审计时使用的选项。

不提供 --ledger 时,verify 的行为与以往完全相同,因此仍可独立使用。

Epic 上下文:

  • --epic-context=<path>:描述此更改所属 Epic 的文件——已完成的内容、此问题的内容,以及后续待完成的内容。只读,并直接传递给判断审查者。它的作用是让审查者区分被遗忘的工作和已安排的工作;只查看 Epic 中一个问题的审查者无法自行判断二者的区别。

输出格式:

  • --format=:markdown(默认)或 json。使用 json 时,报告会序列化为符合 adversary CLI 架构的 JSON 对象(参见 Phase 8c)
  • --output=<path>:写入 JSON 报告的文件路径。使用 --format=json 时为必填项。使用 Write 工具写入文件。

阶段 1:范围检测

确定要验证哪些文件或更改。

1. 解析用户指定的范围(如有):

  • 检查 $ARGUMENTS 中是否包含 --scope=、--files= 或 --module= 标志
  • 如果已指定,则使用该确切范围
  • 跳过自动检测

对于显式分支范围,请先解析比较基准,再列出文件:

HEAD_SHA=$(git rev-parse HEAD)
BASE_REF="$exact_base_ref" # the parsed --base value, or the resolved default
# Without --base, resolve origin/<provider default branch>; only then fall back to main/master.
BASE_REMOTE=${BASE_REF%%/*}
if git remote get-url "$BASE_REMOTE" >/dev/null 2>&1; then
  BASE_BRANCH=${BASE_REF#*/}
  git fetch "$BASE_REMOTE" \
    "+refs/heads/$BASE_BRANCH:refs/remotes/$BASE_REMOTE/$BASE_BRANCH"
fi
git rev-parse --verify "$BASE_REF^{commit}"
MERGE_BASE=$(git merge-base "$BASE_REF" "$HEAD_SHA")
git diff --name-status "$MERGE_BASE" "$HEAD_SHA"
git diff -U0 "$MERGE_BASE" "$HEAD_SHA" -- "$scoped_file"

运行代码片段前,设置特定任务的 exact_base_ref 和 scoped_file 值。

如果存在对应的 origin/ 引用,切勿将分支范围与过时的本地分支进行比较。在报告中原样保留 BASE_REF,同时单独记录解析后的 MERGE_BASE 和 HEAD_SHA。未知的基准引用或合并基准计算失败都属于 验证错误;静默改用其他基准会导致验证错误的更改。

2. 自动检测范围(默认行为):

优先级顺序:

  1. 存在暂存的更改? → 仅将暂存文件纳入范围
  2. 存在未暂存的更改? → 仅将已修改文件纳入范围
  3. 分支领先基准且包含提交? → 将分支更改纳入范围
  4. 未检测到更改? → 报告“无内容需要验证”并停止

用于检测范围的 Git 命令:

# Check for staged changes
git diff --cached --name-only

# Check for unstaged changes
git diff --name-only

# Check for branch changes
if git remote get-url origin >/dev/null 2>&1; then
  git fetch origin
fi
BASE_REF=$(git symbolic-ref --quiet --short refs/remotes/origin/HEAD 2>/dev/null)
if [ -z "$BASE_REF" ]; then
  DEFAULT_BRANCH=$(git remote show origin 2>/dev/null | sed -n 's/^[[:space:]]*HEAD branch: //p')
  if [ -n "$DEFAULT_BRANCH" ] &&
     git rev-parse --verify "origin/$DEFAULT_BRANCH^{commit}" >/dev/null 2>&1; then
    BASE_REF="origin/$DEFAULT_BRANCH"
  elif git rev-parse --verify 'origin/main^{commit}' >/dev/null 2>&1; then
    BASE_REF=origin/main
  elif git rev-parse --verify 'origin/master^{commit}' >/dev/null 2>&1; then
    BASE_REF=origin/master
  elif git rev-parse --verify 'main^{commit}' >/dev/null 2>&1; then
    BASE_REF=main
  elif git rev-parse --verify 'master^{commit}' >/dev/null 2>&1; then
    BASE_REF=master
  else
    echo 'No provider default, origin/main, origin/master, main, or master base exists' >&2
    exit 1
  fi
fi
MERGE_BASE=$(git merge-base "$BASE_REF" HEAD)
git diff --name-only "$MERGE_BASE" HEAD

# Get line ranges for changed files
git diff --cached -U0 -- "$scoped_file"  # staged
git diff -U0 -- "$scoped_file"           # unstaged
git diff -U0 "$MERGE_BASE" HEAD -- "$scoped_file" # branch

3. 构建范围上下文:

创建范围内文件及其状态的列表:

  • file.ts(已修改,第 45-67、89-102 行)
  • new-file.ts(新增,整个文件)
  • old-file.ts(已删除)

将其存储为 SCOPE_CONTEXT,以便传递给 Agent。

还要构建可供机器读取的 SCOPE_METADATA 块,供需要精确重建差异的 Agent 使用:

  • scope_mode:staged、unstaged、branch、files、module、all,或解析出的自动检测模式
  • base_ref:用于生成范围差异的确切基准引用或提交
  • compare_ref:确切的比较目标(HEAD、INDEX、WORKTREE 或显式引用)
  • path_filter:确切的范围路径,或 ALL_SCOPED_FILES
  • diff_command:用于定义范围的确切 git diff 命令
  • merge_base:分支范围对应的确切合并基准哈希,否则为空
  • head_sha:分支范围对应的完整 git rev-parse HEAD SHA,否则为验证开始时 观察到的完整 HEAD SHA
  • included_files:纳入范围的文件/状态记录,按顺序排列
  • excluded_files:明确排除的路径/原因记录,按顺序排列;如果未枚举路径,则使用 ALL_OTHER_FILES (outside selected scope)

使用 --since=<sha> 时,添加另外三个字段,用于描述自上次验证的 HEAD 以来发生的更改。这些字段是附加的:上述每个字段都保留原有含义,审核者仍会收到完整的范围差异。

  • delta_base_sha:--since 的值
  • delta_files:git diff --name-status $DELTA_BASE $HEAD
  • delta_hunks:相同的比较,但使用 -U0

对于任何需要重建所选差异的技能,SCOPE_METADATA 都是唯一事实来源。

4. 为 Agent 整理范围信息:

VERIFICATION SCOPE:
Files in scope:
- src/auth/login.ts (modified, lines 45-67, 89-102)
- src/auth/middleware.ts (modified, lines 12-34)
- tests/auth/login.test.ts (added, entire file)

CRITICAL SCOPE CONSTRAINTS:
- ONLY flag issues in code that was ADDED or MODIFIED in these files/lines
- DO NOT flag issues in surrounding context or old code unless it blocks the new changes
- DO NOT flag issues in other files not listed above
- Focus exclusively on the quality of the NEW or CHANGED code

Exception: You MAY flag issues in old code IF:
1. The new changes directly interact with or depend on that old code
2. The old code issue is causing the new code to be incorrect
3. The old code issue creates a blocker for the new functionality

Git commands to see your scoped changes:
git diff HEAD -- <scoped-files>
git diff --cached -- <scoped-files>

机器可读元数据示例:

SCOPE_METADATA:
- scope_mode: unstaged
- base_ref: INDEX
- compare_ref: WORKTREE
- path_filter: src/auth/login.ts,src/auth/middleware.ts
- diff_command: git diff -- src/auth/login.ts src/auth/middleware.ts
- merge_base:
- head_sha: 9d8f...full SHA
- included_files: src/auth/login.ts,src/auth/middleware.ts
- excluded_files: ALL_OTHER_FILES (outside selected scope)

阶段 2:加载工程师技能

许多代码库都带有工程师技能——一个 <repo>-engineer 技能,用于说明如何构建、测试和运行特定代码库。按以下顺序通过两种方式查找:

  1. 在本次会话可用的技能中,查找名称以 -engineer 结尾的技能。这是确认技能是否存在的可靠方法。
  2. 在磁盘上找到其目录,因为后续阶段必须向子 Agent 提供真实路径——子 Agent 无法按名称解析技能,而且大多数运行框架只会公开名称,而不会公开路径:
ls -d .*/skills/*-engineer/ */skills/*-engineer/ 2>/dev/null | grep -v '^\.\./'

如果有可用的技能:

  • 阅读其 SKILL.md
  • 阅读它所指向的关键参考文件(TESTING.md、架构文档等)
  • 提取:测试命令、构建命令、代码检查命令、架构说明
  • 存储为 ENGINEER_CONTEXT——这是来自 /setup-engineer 的预验证知识
  • 将步骤 2 中的目录存储为 ENGINEER_SKILL_DIR。如果已加载技能,但 glob 未找到任何结果(技能位于代码库之外),请在报告中说明,并将 ENGINEER_SKILL_DIR 留空——这样子 Agent 就会改为使用 ENGINEER_CONTEXT 中的摘要,而不是自行读取文件。
  • 检查同一技能目录中是否有 VERIFICATION.md——如果有,请阅读并存储为 CUSTOM_GATES
    • 提取 Exerciser Gates → 将在阶段 7c 传递给 exerciser
    • 提取 Review Gates → 将在阶段 5/6 传递给审查 Agent
  • 发现阶段(阶段 3)将验证这些命令,并且只查找缺失项

如果没有工程师技能:

  • ENGINEER_CONTEXT 为空
  • CUSTOM_GATES 为空
  • ENGINEER_SKILL_DIR 为空
  • 发现阶段将从头开始进行完整发现

阶段 3:发现与分类

启动一个快速、只读的探索子 Agent,在一条提示词中执行两项任务。

在 --depth=light 下,仅发送任务 A。 任务 B 会产生两项内容,而轻量模式两项都用不上:按技能分配文件,但唯一的判断审查者无论如何都需要查看完整的限定差异;以及 ux-reviewer / visual-verify 的门控决策,但轻量模式不包含这两项技能,且视觉审查会根据 exerciser 实际执行的操作自行决定是否进行。为了生成没人会读取的路由表而阅读每个已更改文件,正是轻量模式旨在避免的成本。无论如何都要执行任务 A——static-analysis 和 tester 需要 TOOLCHAIN。 (在 Claude Code 中,使用 Explore Agent 类型;任何只读子 Agent 均可。)

任务 A:发现项目工具链

如果存在工程师技能: 验证提供的命令仍然有效,并发现任何缺失项:

The engineer skill provides these commands:
- Test: {test_command}
- Build: {build_command}
- Lint: {lint_commands}

Verify each command exists (which/type check). For any that fail, discover alternatives.
Discover any additional linters/type-checkers not covered by the engineer skill.

如果不存在工程师技能: 完整发现:

Discover the project's toolchain:
- Test command (npm test, pytest, cargo test, go test, etc.)
- Build command (npm run build, cargo build, go build, etc.)
- Linter commands (eslint, pylint, clippy, golangci-lint, etc.)
- Type-checker commands (tsc --noEmit, mypy, etc.)

Check package.json scripts, Makefile targets, CI config, pyproject.toml, Cargo.toml, go.mod.
Output concrete commands that can be executed.
任务 B:分类已更改文件

仅限深度模式——参见任务 A 上方的说明。

分类工作包括两项:为各项技能分配重点关注的文件,并判断两项面向用户的技能是否适用。

Read each changed file (not just the extension — look at actual content).

ALWAYS-ON SKILLS — these run on every verification, no exceptions.
Assign each the full scoped file list.

- reviewer          Business logic, architecture, patterns, security, robustness,
                    over-engineering. Needs breadth across the whole diff.
- codex-reviewer    General second-opinion pass, not a specialist router target.
- comment-review    Comment rot, verbose rationale, and comment density can appear in any changed file.
- qa                Test coverage assessment for everything that changed.
- tester            Any code change could affect tests.

GATED SKILLS — these run only if the scope contains surfaces they can review.
Decide applies: true/false for each, and state your reason either way.

- ux-reviewer   APPLIES if the scope touches ANY user-facing surface:
                UI components, CLI output or help text, user-facing strings,
                error messages, API response messages, user-visible log output.
                Assign the specific user-facing files.

                SKIP only when ALL of those are absent — pure backend,
                infrastructure, or internal refactoring with no user-visible
                surface whatsoever.

                WHEN IN DOUBT, APPLY. A wasted ux-reviewer run costs tokens;
                a missed one ships a confusing error message to users.

- visual-verify APPLIES if the scope contains UI-rendering files where the change
                affects what a user sees on the rendered page — components
                (.astro, .tsx, .vue, .svelte), pages, layouts, templates, CSS,
                design tokens, public assets the page depends on for rendering.
                Assign those files.

                SKIP if none are present. NOT JSON config, NOT server-side
                handlers without UI side effects, NOT pure copy strings
                (those are ux-reviewer's).

A file can be assigned to multiple skills.

Output a JSON-like mapping:
{
  "skill_assignments": {
    "reviewer": ["all scoped files"],
    "codex-reviewer": ["all scoped files"],
    "comment-review": ["all scoped files"],
    "qa": ["all scoped files"],
    "tester": ["all"],
    "ux-reviewer": ["UI/CLI/user-facing files"],
    "visual-verify": ["UI-rendering files"]
  },
  "gating": {
    "ux-reviewer": { "applies": true,  "reason": "CLI help text modified in src/cli/help.ts" },
    "visual-verify": { "applies": false, "reason": "no UI-rendering files in scope" }
  },
  "toolchain": {
    "test": "npm test",
    "build": "npm run build",
    "lint": ["npx eslint", "npx tsc --noEmit"]
  }
}

明确的用户覆盖选项无论分类结果如何,都会强制跳过:

  • --skip-ux → 跳过 ux-reviewer。
  • --skip-visual → 跳过 visual-verify。如果你已在浏览器中查看过渲染后的更改,并希望避免重复审查,请使用此选项。视觉审查是设计质量门槛,而不只是检查错误——它会询问“设计师会发布这个吗?”

如果 visual-verify 不在可用技能中,也会跳过该技能(防御性处理——它通常会全局安装且始终可用;项目可能会用自己的副本覆盖它,以自定义捕获规范)。

输出: 将技能分配和门控决策存储为 TRIAGE_RESULT,并将工具链命令存储为 TOOLCHAIN。

在轻量模式下,任务 B 未运行。将 TRIAGE_RESULT.skill_assignments 设置为轻量模式组合中每项技能对应的完整限定文件列表,并将其门控决策留空。之后所有对 TRIAGE_RESULT.skill_assignments 的引用都可以直接解析,无需根据深度进行分支判断。

阶段 4:静态分析

调用 static-analysis(使用速度快、成本低的模型即可——它会运行命令并报告输出),并提供:

  • 阶段 1 中限定范围的文件列表
  • 阶段 3 中发现的 linter/类型检查器命令
SCOPED FILES:
{scope file list}

COMMANDS TO RUN:
{each linter/type-checker command from TOOLCHAIN}

Run each command on the scoped files. Parse output into structured findings.
Report only findings in scoped files.

等待结果。保存为 STATIC_SUMMARY。

阶段 5:构建上下文包

为审查技能整理一个精简的上下文包(约 50–100 行):

CONTEXT_BUNDLE:

VERIFICATION SCOPE:
{SCOPE_CONTEXT from Phase 1}

SCOPE METADATA:
{SCOPE_METADATA from Phase 1}

{If --ledger was supplied:}
KNOWN FINDINGS (decided by previous verification runs on this change):

| ID | Location | Title | Root cause | Sev | Disposition | Reason |
|----|----------|-------|------------|-----|-------------|--------|
{one row per ledger finding that is not `fixed`}

DELTA SINCE LAST VERIFIED HEAD: {delta_base_sha}
{delta_files, one per line, with changed line ranges}

{If --epic-context was supplied:}
EPIC CONTEXT:
{the file's contents — completed issues, this issue, remaining issues}

Work whose natural home is one of the remaining issues is deferred, not reported: name it on
a DEFERRED_TO_EPIC line instead of filing a finding. Anything at severity 9-10, anything in
the security class, and any defect this change itself introduces is reported normally
regardless of what the backlog says.

OUTPUT ADDITION:
DEFERRED_TO_EPIC: <issue> — <one sentence>    (or "none")

ENGINEER SKILL SUMMARY:
{Brief summary from ENGINEER_CONTEXT, or "No engineer skill found — toolchain discovered via exploration"}
{If engineer skill exists: "Reference files available at {ENGINEER_SKILL_DIR} — read TESTING.md, ARCHITECTURE.md etc. for your domain"}

STATIC ANALYSIS SUMMARY:
{STATIC_SUMMARY from Phase 4 — just the findings table, not raw output}

TOOLCHAIN:
- Test: {command}
- Build: {command}
- Lint: {commands}

CUSTOM REVIEW GATES:
{Review Gates from VERIFICATION.md, or "None defined"}
These are repo-maintainer-defined requirements. If any rule falls in your review domain,
report PASS/FAIL for it. Failed gates should be reported as severity 9 findings.

DIFF STAT:
{output of: git diff --stat [scope args]}

保持精简。 技能会自行读取文件——上下文包只需告知它们要查看的位置,以及已知信息。

阶段 6:启动审查技能(并行)

并行调用适用的审查技能——在 Claude Code 中,使用一条消息发起多个 Skill 工具调用。

如果你的运行环境无法并发运行技能,则按下方列出的顺序逐个运行,并在阶段 7 前合并它们的报告。无论采用哪种方式,流程都正确;只是耗时更长。不要为了节省时间而省略技能——报告的价值来自综合结果。

如果此阶段在支持子 Agent 的运行环境中一次只运行一个审查器,应先怀疑并发上限,而不是怀疑此技能。 在 Codex CLI 上测得,相同的扇出在默认上限下从启动到启动耗时 359 s,提高上限后耗时 43 s,而这些说明并未改变。 此技能末尾的运行环境绑定部分会记录该上限所在位置。

在 --depth=deep 下

始终运行,每次都要运行: reviewer、codex-reviewer、comment-review、qa、tester。分诊会按文件分配任务,但绝不会跳过这些技能——对于面向正确性的审查,漏掉回归的代价高于多运行一个技能。

仅在分诊判断适用时运行: ux-reviewer 和 visual-verify(参见阶段 3 的任务 B 条件判定)。--skip-ux / --skip-visual 会覆盖分诊结果并强制跳过。

使用 --depth=light 时(默认)

每次都必须运行:tester,以及且仅运行一名判断型评审,按以下方式选择:

which codex

成功时,调用 codex-reviewer。否则调用 reviewer。记录实际运行的是哪一个——报告必须说明这一点,否则读者无法判断此次变更接受了哪种评审。

**绝不能在 codex-reviewer 内部处理这一回退逻辑。**该技能的全部价值在于模型多样性;如果用同一模型执行替代评审,却隐藏在它的名称背后,那么对于一次完全没有模型多样性的评审,报告仍会显示 codex-reviewer: COMPLETED。这个选择应在此处进行,确保其在组合方式和报告中都清晰可见。

reviewer 的成本高于两者中的另一个——它是套件中覆盖面最广的评审,使用的是可用模型中能力最强的成本档位。在没有 Codex 的机器上,light 会运行一个高成本 Agent,而不是一个低成本 Agent。它依然只是一个 Agent,也依然是合适的替代方案:另一种选择是让变更完全无人独立评审。

light 下绝不运行:comment-review、qa、ux-reviewer、visual-verify。这不是失败,也不是有原因的跳过——这些技能不属于当前组合。Phase 7 会相应记录它们。

在 Phase 7c 中,exerciser 在两种深度下都会运行;在 light 下,它自带视觉视角。

**模型路由由 harness 决定。**没有技能会固定指定模型——合适的模型取决于变更有多棘手,只有编排器知道这一点。以下是各技能所需模型的大致成本参考:

  • 能力最强的模型——reviewer。全面评审设计、架构、一致性、加固、安全性和过度设计问题。这是最能从原始能力中获益的技能。
  • 通用模型——codex-reviewer、comment-review、qa、ux-reviewer、exerciser、visual-verify。需要判断,但判断范围有限。
  • 快速、低成本模型——tester、static-analysis。这些技能运行命令并报告结果;不会要求它们对结果进行推理。

每个技能提示词都包含:

  1. Phase 5 中的 CONTEXT_BUNDLE
  2. TRIAGE_RESULT.skill_assignments 中该技能对应的文件列表
  3. 与其领域相关的静态分析发现
  4. 指示其在有可用的领域工程师技能参考文件时阅读这些文件
  5. 技能专属说明(见下方模板)

对于 codex-reviewer,SCOPE_METADATA 具有权威性。如果有精确的元数据,就不得根据文件名或说明文字推断范围模式。

延续规则(仅适用于提供了 --ledger 且未提供 --no-carry-forward 的情况)

将以下内容附加到判断型评审技能的提示词中——reviewer、codex-reviewer、qa、comment-review、ux-reviewer、visual-verify。绝不要附加到 tester、static-analysis 或 exerciser:这些技能会运行命令并报告结果,而机器不会像刚接触内容的读者那样产生偏差。

CARRY-FORWARD RULES

Tier A — regression lens, full scoped diff, always. You have the whole branch diff and you
review all of it for anything the DELTA broke. A defect the delta introduced or made
reachable is reportable at full severity even at a location listed in KNOWN FINDINGS: an
accepted finding covers the exact state its root cause describes, never the file, the
function, or the subsystem.

Tier B — new audit territory. Do not open a new line of inquiry into code that is unchanged
since DELTA BASE and that KNOWN FINDINGS does not mention. Previous rounds already reviewed
that code against this threshold and dispositioned what they found. Widening the audit a
little further each round is how a loop that was one round from done ends up reporting on a
different subsystem every time.

Matching a known finding:
- Matches a KNOWN finding AND the code there is unchanged since DELTA BASE
  -> do not report it as a finding. List it under REAFFIRMED with the known ID and one sentence.
- Matches a KNOWN finding BUT the code there changed in the delta
  -> report it normally, at full severity. It is a new finding about new code.
- Has new evidence of concrete harm — a failing test, a reproduction, an exploit path
  -> report it normally AND list it under REOPENED with that evidence.

Never suppressible, whatever KNOWN FINDINGS says: anything at severity 9 or 10, and
anything in the security class.

OUTPUT ADDITIONS:
REAFFIRMED: <known id> — <one sentence>    (or "none")
REOPENED:   <known id> — <harm evidence>   (or "none")

**编排器绝不进行过滤。**不要因为某条报告的问题与台账中的条目匹配,就将其丢弃。是否抑制是评审者在查看完整差异后作出的判断,且每次抑制都会以 REAFFIRMED 行显示出来。如果流水线中存在静默的字符串匹配过滤器,那么它恰好会在出错时无法被发现。

技能提示词模板

每个技能的提示词都遵循以下结构:

{CONTEXT_BUNDLE}

YOUR ASSIGNED FILES:
{files from TRIAGE_RESULT.skill_assignments for this skill}

RELEVANT STATIC FINDINGS:
{filtered findings from STATIC_SUMMARY relevant to this skill's domain}

{If engineer skill exists:}
ENGINEER SKILL REFERENCE:
Reference files are available at {ENGINEER_SKILL_DIR}
Read files relevant to your domain (e.g., TESTING.md for tester, architecture docs for reviewer).

{Skill-specific instructions...}

OUTPUT FORMAT: For each issue found, provide:
- Title (short description)
- Severity (1-10, where 1=trivial, 10=critical)
- Location (file:line)
- Description (what the issue is and why it matters)

技能专属说明块:

reviewer:

Comprehensive review across all five dimensions:
1. Design & Code Quality: design adherence, over-engineering, AI slop, test integrity, structural completeness
2. Architecture: module boundaries, dependency direction, god objects, abstraction opportunities, coupling
3. Coherence: reinvented wheels, pattern violations, convention mismatches, documentation drift, dead code
4. Hardening: invalid inputs, error paths, inconsistent validation, orphaned references, state transitions
5. Security: injection, auth/authz, multi-tenant isolation, data exposure, crypto

Research the project's structure, patterns, and security approach BEFORE evaluating changes.
Focus on: 'Is this change well-designed, structurally sound, pattern-consistent, robust, and secure?'

Work the over-engineering lens in Dimension 1 deliberately — it is easy to review only
for what is missing and never for what should be cut. Tag those findings
(delete / stdlib / native / yagni / shrink), verify each proposed replacement actually
exists before reporting it, and close the report with the net-lines metric.

codex-reviewer:

Run the local Codex CLI as an independent second-opinion reviewer.
Shell out to the local Codex CLI — the point is a second engine's opinion, not your own analysis again.
Follow the codex-reviewer skill's own execution procedure exactly; it owns the invocation, the completion detection, and the timeout. Do not substitute your own — this prompt is scope and context, not a runbook.
Use `SCOPE_METADATA` as the source of truth for scope reconstruction.
Adapt the verify scope into a temporary diff-only workspace under /tmp so Codex reviews only the intended changes.
Do NOT infer staged vs unstaged vs branch vs path-filtered scope from assigned files or prose if `SCOPE_METADATA` says otherwise.
If exact reconstruction from `SCOPE_METADATA` is not possible, report PATCH_CONSTRUCTION_FAILED instead of reviewing an approximate diff.
If Codex is unavailable (missing CLI, auth missing, network blocked, sandbox blocked), report BLOCKED status with a short factual reason.
If scope is `--scope=all`, report SKIPPED_UNSUPPORTED_SCOPE rather than attempting a whole-codebase audit.
Normalize Codex output into: title, severity, location, description.

comment-review:

Review ONLY comments and docstrings added or modified in the scoped diff
(plus pre-existing comments the changes make stale).
Flag: ephemeral review-ID references (VI-N, CI-N, "per review feedback"),
historical change-narration ("previously", "now we", "replaced X with Y"),
stale comments contradicting the code, reviewer-appeasement, redundant restatement,
verbose rationale that a 1–2 line comment would carry, and one diff-level density
finding per the skill's step 5 (default 10% of added code lines; pass the project's
threshold when its agent instructions set one).
Comments must describe the current code and its intent — git owns history.
Code correctness, design, and prose docs are out of scope — other skills own those.
Normalize findings into: title, severity (floor 5, cap 6 — the floor is deliberate),
location, category (tag), description including a concrete rewrite (or "delete").
If comments are clean, report COMPLETED with zero findings.

tester:

Run the full test suite using: {test command from TOOLCHAIN}
Report exact pass/fail counts.
If tests cannot run, report what prevented execution.
For EACH failure: title, severity, location, error message, and whether it's IN-SCOPE or OUT-OF-SCOPE.

ux-reviewer:

ONLY test user-facing changes in the scoped files.
Do not audit the entire UI/CLI for issues.
Focus on the UX of what changed in this scope.
Test any UI, CLI output, error messages, or API responses that were modified.

qa:

Evaluate whether the scoped changes are adequately tested.
Assess test quality, mock usage, and test type appropriateness.
Adapt expectations to codebase testing maturity.
Focus on: 'Are these changes well-tested with good tests?'

visual-verify:(仅当可用技能列表中包含此技能,且未传入 --skip-visual 时才调用)

Read the engineer skill for screenshot mechanics (wrapper command, dev URLs, viewport conventions) — typically VISUAL.md or the engineer SKILL.md.
For each UI-rendering file in YOUR ASSIGNED FILES, identify at least one route that renders it (grep imports for components; page files render directly).
Take screenshots at desktop (1440-wide, full-page) AND mobile (390-wide, full-page). If responsive sizing changed, sample boundary widths too.
Open every PNG with `Read` and apply the holistic articulation step described in the skill: write 2-3 sentences describing the composition in designer's terms (balance, weight, hierarchy, rhythm, alignment, density, color, typography). Anything in that paragraph that reads as a complaint becomes a finding.
Focus on: 'Would a designer ship this?' — not 'is anything imperfect?'
If no UI-rendering files in scope or no running app discoverable, return STATUS: SKIPPED with a one-line factual reason — same convention as codex-reviewer.
Output structured findings (Title / Severity / Location / Category / Description) per the skill's reviewer-mode protocol.
计划完整性检查(并行、有条件)

此检查与上述审查技能并行运行。它不依赖这些技能的输出,只需要计划和分支差异。

如果已解析计划(通过 --plan-file 标志,或从对话上下文中发现):

使用以下内容调用通用 Agent:

PLAN FILE:
{plan file contents}

BRANCH DIFF:
{output of the diff command from SCOPE_METADATA, or `git diff {base}...HEAD`}

SCOPE CONTEXT:
{SCOPE_CONTEXT from Phase 1}

You are checking whether the implementation is complete relative to the plan.

Instructions:
- Identify all phases, sections, and steps described in the plan
- For each, determine whether the branch diff contains changes that implement it
- A phase is "addressed" if the diff contains changes that clearly correspond to its requirements
- If significant portions of the plan are unimplemented, emit a single finding:
  - Title: "Plan incomplete — only phase N of M implemented" (or similar descriptive summary)
  - Severity: 8
  - Description: List which phases/sections are implemented and which are missing, with brief reasoning
  - Sources: ["plan-completeness"]
  - Location: null (omit)
- If the plan appears fully implemented, emit NO finding (do not emit a success finding)

OUTPUT FORMAT: For each issue found, provide:
- Title (short description)
- Severity (1-10, where 1=trivial, 10=critical)
- Location (file:line or null)
- Description (what the issue is and why it matters)

如果未解析计划(没有标志,上下文中也没有):完全跳过此检查,不生成任何发现。

阶段 7:收集结果 + 有条件的 Agent

7a. 等待所有阶段 6 技能完成

这是对整组技能进行一次等待,而不是对每项技能分别等待。

未产生结果的技能状态为 NOT_RUN,绝不能记为 COMPLETED。 如果技能返回空结果、遇到服务商配额或速率限制、超时或启动遭拒,应根据观察到的原因将其记录为 NOT_RUN。审查从未运行与审查未发现问题是截然相反的结果,只有后者才是好消息——将前者记录为后者,会把流程中的漏洞伪装成一切正常。

不属于当前深度组合的技能状态为 SKIPPED,绝不能记为 NOT_RUN。 应将其记录为 SKIPPED,原因写为 not in light composition。NOT_RUN 必须始终表示“本该运行但未运行”——上述规则的核心就在于此;若把它也用于“从未属于计划的一部分”,就会让真正的漏洞藏在预期情况后面。并发运行审查器会增加遇到配额拒绝的可能性,而不是降低,因此这一区分很重要。

收集每项技能的结构化发现。仅提取:

  • 标题
  • 严重性(1-10)
  • 位置(文件:行号)
  • 类别(技能专属)
  • 描述

丢弃调查过程叙述。 保持编排器上下文精简。

对于 codex-reviewer,如果没有产生任何发现,还要收集技能状态:

  • COMPLETED
  • BLOCKED
  • SKIPPED_UNSUPPORTED_SCOPE

对于 visual-verify,如果没有产生任何发现,还要收集技能状态:

  • COMPLETED
  • SKIPPED,并注明原因(NO_UI_FILES_IN_SCOPE、NO_RUNNING_APP_DISCOVERABLE、NO_SCREENSHOT_MECHANISM)。SKIPPED 属于预期行为,不是警告——它仅表示范围内没有需要进行视觉验证的内容。

处理 Codex BLOCKED 状态: 如果 codex-reviewer 报告 BLOCKED,这是重大事件——独立的第二模型审查未能运行。应在报告中醒目地标出:

  • 在 report-only 模式下:在 Agent 结果摘要中醒目地注明 BLOCKED 状态,并在摘要表格后添加醒目的警告。
  • 在 interactive 模式下:使用 AskUserQuestion 询问:“Codex 审查被阻止({reason})。是否在没有 Codex 审查的情况下继续,还是停止以解决问题?”

SKIPPED_UNSUPPORTED_SCOPE 在 --scope=all 的情况下属于预期行为,不应作为警告标出。

7b. 有条件执行:debugger

如果 tester 或 exerciser——或者在 deep 深度下的 ux-reviewer——报告了失败(严重性 7+):

两个深度都要运行 debugger。它是在观察到失败时触发的诊断工具,而非主动寻找问题的审查器,因此不会像判断类技能那样扩大审计范围。调用方依赖它:player-coach 只能根据 debugger 的判定创建隔离区条目,因此在 light 深度下省略它会悄然禁用隔离机制。

使用以下内容调用 debugger:

VERIFICATION SCOPE CONTEXT:
{SCOPE_CONTEXT}

FAILURES TO INVESTIGATE:
{list of failures from tester/ux/exerciser}

Analyze the root cause of these failures.
Focus on failures caused by the scoped changes.
If failures are unrelated to scope, note that explicitly.

For each failure, determine whether the scoped changes caused it or whether it was already
there. Check out {scope.mergeBase} in a scratch worktree and run the same command against it.
Report, per failure:
  PRE_EXISTING: yes | no | undetermined
  COMMAND: <the exact command you ran>
  EVIDENCE: <output excerpt showing the same failure signature at the merge base>
  TOUCHED_BY_DIFF: <paths in the failure's stack that appear in the scoped diff, or none>

每个独立的失败特征只需额外运行一次测试套件,代价仅为一次执行。它带来的收益远超成本:没有它时,与此次变更无关的环境故障会在每次出现时都从头重新诊断,每次都要付出完整的测试套件运行成本,而且会以足以盖过该轮实际议题的严重性再次报告。

7c. 始终执行:带问题验证的 exerciser

在 Phase 6 的评审完成后运行,不与评审并行运行——它需要将收集的问题列表作为输入。

调用 exerciser,并传入:

{CONTEXT_BUNDLE}

Exercise the changes end-to-end:
1. Read the engineer skill at {ENGINEER_SKILL_DIR} if one was found — follow its instructions for starting the environment, authenticating, and interacting with services
2. Start the full local environment (app + all backing services)
3. Determine exercise strategy based on change type:
   - Frontend/UI changes → use Playwright to navigate and interact
   - API/backend changes → make actual API calls via curl, verify responses and data state
   - Data/search/indexing changes → trigger operations, query services via CLI tools to verify data was written/indexed
   - Job/worker changes → trigger jobs, verify side effects via database/service queries
   - Mixed → exercise through all affected interfaces
4. Verify data flows end-to-end — don't stop at "endpoint returns 200", follow data through the system
5. Report whether the specific changes actually work with real data

If you hit a barrier (can't start, need credentials, unclear what to test, no engineer skill for complex backend):
- Return BLOCKED status with specific reason
- If you cannot determine HOW to exercise the change, that is severity 9-10

{At --depth=light, and unless --skip-visual was passed:}
VISUAL LENS: ACTIVE
If your exercise path went through rendered UI in a browser, also look at what you rendered
and report visual defects, following your skill's visual lens section. If you exercised the
feature through an API, a CLI, a job, or the database, there is nothing rendered to look at —
say so and move on.

{At --depth=deep:}
VISUAL LENS: INACTIVE — visual-verify owns visual review at this depth. Do not duplicate it.

ISSUES FOUND BY REVIEW SKILLS:
{List of all issues found in Phase 6 with VI-IDs, severity, title, location}

While exercising, attempt to trigger each reported issue and report verification status
(CONFIRMED / NOT REPRODUCED / NOT APPLICABLE / BLOCKED).

{If CUSTOM_GATES has exerciser gates:}
CUSTOM EXERCISER GATES:
{List of exerciser gates from VERIFICATION.md}

These are mandatory repo-maintainer-defined checks. After exercising the feature, you MUST
check each gate and report PASS/FAIL with evidence. Any failing gate means your overall
status cannot be PASSED — use FAILED instead.

处理 exerciser 阻塞情况:

  • 如果状态为 BLOCKED,且原因为 LOGIN_REQUIRED、UNCLEAR_FEATURE、NO_EXERCISE_STRATEGY、NO_ENGINEER_SKILL 或 SERVICE_UNAVAILABLE(仅限交互式/自动修复模式):
    1. 使用 AskUserQuestion 向用户寻求帮助
    2. 根据用户的回复重新调用 exerciser
    3. 如果仍然受阻,将最终状态记录为 BLOCKED
  • 在 report-only 模式下:在报告中记录 BLOCKED 状态,不向用户提问

Phase 8:生成统一报告

8a. 问题去重
  1. 以结构化格式收集每项技能的所有发现(包括 Phase 6 中任何关于计划完整性的发现)
  2. 识别重复项:文件/位置相同、根本原因相同,或从不同角度描述了相同症状
  3. 合并为单个问题:列出所有来源 Agent,合并描述,并采用最高严重性
  4. 按顺序分配 VI-{n} ID
  5. 为每个问题分配一个 class,因为技能只会将其作为提示输出,而调用方会据此进行判断: correctness、security、coverage、comment、ux、visual、test-failure、 environment、style、plan-completeness。需要明确说明的一条界限是 coverage 与 correctness 的区别:coverage 表示代码被认为是正确的,但未经测试;错误的测试——没有断言任何内容、模拟被测对象,或在删除实现后仍能通过——属于 correctness,因为缺陷在测试本身。调用方会限制重复的 coverage 发现,但不会限制 correctness;因此,如果不确定应归为哪一类,就归为 correctness。
  6. 按严重性降序排序

去重时绝不能下调 comment-review 发现的评级。 其 5-6 的严重性是有意设定的下限,而非评估结果——它用于纠正评论发现被评为低级、随后又因级别低而遭跳过这一可复现偏差。将其合并到严重性较低的问题中,或把它放在真实正确性缺陷旁边进行“语境化”,都会再次引入该下限旨在消除的偏差。

8b. 报告格式
# Verification Report

## Scope

**Depth:** [light / deep]
**Mode:** [staged / unstaged / branch / all / files / module]
**Base ref:** [exact `--base` value or resolved base; `null` outside branch scope]
**Merge base:** [full SHA or `null` outside branch scope]
**HEAD:** [full SHA observed when verification began]

**Files Verified:**
- src/auth/login.ts (modified, lines 45-67, 89-102)
- src/auth/middleware.ts (modified, lines 12-34)
- tests/auth/login.test.ts (added, entire file)

**Files Excluded:** [ordered paths with reasons, or "All other files (outside selected scope)"]

---

## Triage Summary

**Skills run:** reviewer, codex-reviewer, comment-review, tester, qa, ux-reviewer, visual-verify, exerciser
**Skills skipped:** [none, or each skipped skill with its reason — `visual-verify (gated: no UI-rendering files in scope)`, `ux-reviewer (--skip-ux)`, `visual-verify (not present in available skills)`, `qa (not in light composition)`]
**Judgement reviewer:** [at light: `codex-reviewer` or `reviewer (Codex unavailable)`; at deep: `all five`]
**Static analysis:** ESLint (3 findings), tsc (1 finding)

A gated skip must always name its reason. A mis-gate is only correctable if it is visible in the report.

At light the reviewer line is not decoration. `codex-reviewer` and `reviewer` are different
models looking for different things, and a reader deciding how much to trust this report needs
to know which one they got.

---

## Skill Results Summary

| Skill | Status | Notes |
|-------|--------|-------|
| static-analysis | Completed | 4 findings (3 warnings, 1 error) |
| tester | X passed, Y failed | [brief note] |
| reviewer | Completed | Found N items (design, arch, coherence, hardening, security; net -N lines possible) |
| codex-reviewer | Completed / **BLOCKED** / Skipped | Found N items / [reason] |
| comment-review | Completed | Found N items / Comments clean |
| qa | Completed | Found N items |
| ux-reviewer | Completed / Skipped | Found N items / [gating or override reason] |
| visual-verify | Completed / Skipped / Not Available | Found N items / [gating or override reason] |
| exerciser | PASSED / FAILED / BLOCKED | [reason if blocked] |
| plan-completeness | Complete / Incomplete / Skipped | [summary if incomplete, reason if skipped] |
| debugger | Ran / N/A | [if applicable] |

---

## Issues Found

[Deduplicated issues from all agents, sorted by severity descending]

| ID | Sev | Title | Sources | Location | Description |
|----|-----|-------|---------|----------|-------------|
| VI-1 | 9 | [Short title] | tester, reviewer | file:line | [Combined description] |
| VI-2 | 7 | [Short title] | security | file:line | [Description] |

*Severity: 9-10 Critical | 7-8 High | 5-6 Moderate | 3-4 Low | 1-2 Trivial*
*Sources column shows which agents flagged the issue. Multiple sources = higher confidence.*

**Total: N issues from M skill findings (deduplicated)**

---

## Deferred to Later Epic Issues

{Omit this section entirely when no --epic-context was supplied or nothing was deferred}

| Issue | Deferred item | Reported by |
|---|---|---|
| #47 | export endpoint has no pagination | reviewer |

These are not findings and never block. They are recorded so a deferral is visible and
correctable — a suppression nobody can see is only ever discovered by being wrong.

---

## Exerciser Verification

| Issue ID | Title | Exerciser Status | Notes |
|----------|-------|-----------------|-------|
| VI-1 | [title] | CONFIRMED | [observation] |
| VI-2 | [title] | NOT REPRODUCED | [what was tried] |
| VI-3 | [title] | NOT APPLICABLE | [reason] |

---

## Custom Verification Gates

{If no VERIFICATION.md exists or no custom gates defined: omit this section entirely}

### Exerciser Gates
| # | Rule | Status | Evidence |
|---|------|--------|----------|
| 1 | [rule from VERIFICATION.md] | PASS / FAIL / BLOCKED | [from exerciser report] |

### Review Gates
| # | Rule | Status | Checked By | Evidence |
|---|------|--------|------------|----------|
| 1 | [rule from VERIFICATION.md] | PASS / FAIL / NOT CHECKED | [skill name] | [from skill findings] |

**Custom Gates: X/Y passed, Z blocked**
8c. JSON 输出(--format=json --output=<path>)

设置这两个标志时,序列化与 Markdown 报告相同的去重数据。JSON v1 具有完整的追踪用结构,同时保留对抗性契约: schemaVersion、status 和 findings 保留现有名称与含义。

步骤 1——确定 status:

条件(按顺序检查)status
Verify 崩溃或无法执行(技能调用失败、基准无效、内部错误)"error"
Verify 无法生成有意义的分析(例如,没有工具链,且没有评审技能可以运行)"blocked"
流程已完成,包括发现代码/测试失败的情况"ok"

构建/类型检查错误和测试失败属于严重性为 9–10 的发现,而非终止性 schema 状态。blocked 表示验证本身无法提供有意义的分析。

步骤 2——仅规范化一次位置,然后投射到两种契约:

将文本位置解析为以下结构化问题位置:

{
  "path": "path/to/file.js",
  "line": 27,
  "endLine": 36
}

按最后一个后接行号或行范围的冒号拆分文本位置。单行时省略 endLine。从完整结构化对象投射 issues[].location。 将稳定的对抗性 findings[].location 投射为精确的 {"path": ..., "line": ...}; 此处绝不添加 endLine。没有适用位置时省略 findings[].location,并将 issues[].location 设为 null。绝不使用空对象或文字描述的位置替代。

步骤 3——严格按以下结构组装:

{
  "schemaVersion": 1,
  "status": "ok",
  "error": null,
  "findings": [
    {
      "title": "Issue title",
      "severity": 8,
      "description": "Why this matters",
      "sources": ["reviewer", "tester"],
      "location": {"path": "src/example.ts", "line": 27}
    }
  ],
  "overall": {"result": "issues-found", "mode": "branch"},
  "scope": {
    "depth": "light",
    "mode": "branch",
    "baseRef": "origin/release",
    "mergeBase": "0123456789abcdef0123456789abcdef01234567",
    "headSha": "89abcdef0123456789abcdef0123456789abcdef",
    "includedFiles": [
      {"path": "src/example.ts", "status": "modified", "lines": ["27-36"]}
    ],
    "excludedFiles": [
      {"path": "ALL_OTHER_FILES", "reason": "outside selected scope"}
    ],
    "files": [
      {"path": "src/example.ts", "status": "modified", "lines": ["27-36"]}
    ],
    "excluded": [
      {"path": "ALL_OTHER_FILES", "reason": "outside selected scope"}
    ],
    "deltaBaseSha": null,
    "deltaFiles": []
  },
  "knownFindings": null,
  "deferredToEpic": [],
  "triage": {
    "skillsRun": [
      {"skill": "reviewer", "reason": "code files require correctness review"},
      {"skill": "tester", "reason": "test command discovered"},
      {"skill": "exerciser", "reason": "user-visible flow changed"}
    ],
    "skillsSkipped": [
      {"skill": "visual-verify", "reason": "no UI-rendering files in scope"}
    ],
    "skills_run": ["reviewer", "tester", "exerciser"],
    "skills_skipped": ["visual-verify"]
  },
  "skillResults": [
    {"skill": "reviewer", "status": "COMPLETED", "notes": "Judgement reviewer at light depth (Codex unavailable). Found 1 item"},
    {"skill": "qa", "status": "SKIPPED", "notes": "not in light composition"},
    {"skill": "exerciser", "status": "PASSED", "notes": "Flow completed; visual lens active, 0 visual findings"}
  ],
  "issues": [
    {
      "id": "VI-1",
      "severity": 8,
      "title": "Issue title",
      "sources": ["reviewer", "tester"],
      "location": {"path": "src/example.ts", "line": 27, "endLine": 36},
      "description": "Why this matters"
    }
  ],
  "exerciserVerification": [
    {"issue": "VI-1", "status": "CONFIRMED", "evidence": "Observed during save"}
  ],
  "customGates": {
    "exerciser": [
      {"rule": "Saved records can be reloaded", "status": "PASS", "evidence": "Reloaded record 42", "checker": "exerciser"}
    ],
    "review": [
      {"rule": "Tenant queries remain scoped", "status": "PASS", "evidence": "Reviewed query predicate", "checker": "reviewer"}
    ]
  }
}

字段规则:

  • 当 status 为 error 时,overall.result 为 error;当 status 为 blocked 时,为 blocked;当 status 为 ok 且存在一个或多个问题时,为 issues-found;当 status 为 ok 且没有问题时,为 pass。这是它仅有的取值。
  • 对于 ok/blocked,error 为 null;对于 status: error,它是一个包含稳定的 code 和客观 message 的对象。
  • scope.depth 为 light 或 deep,且始终存在。报告若未说明检查力度,之后便无法解读,而且 light 和 deep 报告会相互比较。
  • scope.mode 是解析后的范围模式。baseRef 仅在分支范围下为字符串,其他情况下为 null。成功解析分支范围时,mergeBase 为解析出的完整 SHA;非分支范围下为 null;如果无效的 base 导致无法解析,也为 null。 headSha 始终是开始时观察到的完整 SHA。 includedFiles 和 excludedFiles 有顺序且不得省略。保留旧版别名 files 和 excluded,其中包含相同的有序记录。
  • triage.skillsRun 按调用顺序列出每个已调用的技能,使用 skill/reason 对象。triage.skillsSkipped 以相同结构记录每个已跳过或不可用的技能。每个条目都必须包含客观的路由原因。 保留 skills_run 和 skills_skipped 作为仅含名称的有序别名,供早期部分报告的使用者使用。
  • skillResults 是一个数组,包含静态分析、计划完整性检查、条件性调试器,以及每个已运行或已跳过的审查技能。每项必须且只能包含 skill、status 和 notes;notes 可以是空字符串。将 status 规范化为 COMPLETED、PASSED、FAILED、BLOCKED、SKIPPED 或 NOT_RUN;将计数和特定提供方的详细信息放入 notes。Exerciser 使用 PASSED/FAILED/BLOCKED,Codex 使用 COMPLETED/BLOCKED/SKIPPED,以便调用方能按精确值应用门禁。
  • issues 保留分配的 VI ID 和结构化位置。findings 镜像每个问题,但不包含其 ID,以便现有的 adversary 使用者继续正常工作。
  • exerciserVerification 为 Exerciser 评估过的每个问题包含一条记录。issue 是 VI ID,status 为 CONFIRMED、NOT_REPRODUCED、NOT_APPLICABLE 或 BLOCKED,且始终包含 evidence。
  • 即使为空,customGates.exerciser 和 customGates.review 也始终为数组。 每个门禁都记录其规则、状态、证据,以及作为检查器的技能或命令。
  • 未提供 --epic-context 时,deferredToEpic 为 [];否则,对每项延期内容都记录一条 {"issue": …, "item": …, "source": …}。它绝不会合并到 findings 或 issues 中:延期意味着不存在发现,按严重性设置门禁的使用者不应看到它。
  • 如果提供了 --since,scope.deltaBaseSha 和 scope.deltaFiles 描述该差异;否则 deltaBaseSha 为 null,deltaFiles 为 []。
  • 未提供账本时,knownFindings 为 null。否则,它会记录结转规则产生的内容:
"knownFindings": {
  "supplied": 14,
  "reaffirmed": [{"id": "F-3", "skill": "reviewer", "reason": "unchanged since delta base", "newHarmEvidence": false}],
  "reopened": [{"id": "F-7", "skill": "tester", "reason": "now reproduces", "evidence": "…"}]
}

这些都是新增键。忽略未知字段的使用者不会受影响,因此无需提升 schemaVersion。出于同样的原因,scope.depth 也是新增字段。

JSON-v1 的兼容边界是有意设定的:按照 adversary 使用者的要求,schemaVersion、status 和 findings 保留原有含义和结构。此前仅为占位内容的 skillResults 和 issues.location 现在分别定义为数组和结构化位置契约;不得为了模仿未记录的占位对象或文本位置而弱化这些新契约。没有冲突的 scope/triage 别名则继续保留,以便迁移。

将组装后的对象验证为 JSON,使用两个空格缩进美化输出,并将其写入准确的 --output 路径。绝不复用或覆盖之前验证运行所用的输出文件;调用方必须为每次调用提供唯一的路径。 当无效/缺失的基准或合并基准失败导致 status: error 时,仍须输出完整的 v1 结构:保留请求的 scope.baseRef,将 scope.mergeBase 设为 null;如果尚未开始发现文件,则使用空的包含/排除数组;将未运行的技能标记为 NOT_RUN 并说明原因,同时填入 error。这样可以让失败结果可解析,而不会假装存在合并基准。

阶段 9:按模式执行报告后操作

模式:report-only

输出报告并将控制权交还调用方。不要进行分类、规划或修复。

  • 不提示用户——不要使用 AskUserQuestion
  • 完全跳过分类/修复阶段
  • 输出报告后,verify 执行即告完成——由调用方继续其流程

如果设置了 --format=json 和 --output=<path>:

  • 阶段 8c 已写入 JSON 文件——该文件就是报告
  • 仅输出一行确认信息:JSON report written to <path>(不输出 Markdown 报告)
  • 将控制权交还调用方

否则(默认为 Markdown):

  • Markdown 报告是唯一输出
模式:interactive(默认)

呈现报告后,进行交互式分类:

如果没有发现问题: 输出报告并返回。无需分类。

交互式分类流程:

  1. 使用 AskUserQuestion,每批最多呈现 4 个问题
  2. 按严重程度降序排列问题(最严重的优先)
  3. 对于每个问题:读取源文件,生成 2-3 个具体修复方案,并提供 Explain 和 Skip
  4. 记录用户的决定

AskUserQuestion 格式(每批最多 4 个):

AskUserQuestion:
  questions:
    - header: "VI-1"
      question: "{Title} — {Description}. Found at {file:line} by: {sources} (severity {N})"
      multiSelect: false
      options:
        - label: "{Fix option 1}"
          description: "{Specific action with file:line reference}"
        - label: "{Fix option 2}"
          description: "{Alternative action with file:line reference}"
        - label: "Explain"
          description: "Get the full picture before deciding"
        - label: "Skip"
          description: "Accept this issue — will not fix in this change set"

处理“Explain”: 读取周边代码,提供更丰富的上下文并重新呈现(单独呈现,不分批)。保留所有修复选项。如果再次选择 Explain,则深入调查。

关键:呈现每一个问题。 切勿跳过问题。只有用户明确说出“stop”、“done”或“skip the rest”时,才可提前停止。

  1. 对所有问题完成分类后,显示分类决策摘要表
  2. 如果有待应用的修复:
    • 调用 EnterPlanMode
    • 读取所有受影响的文件,分析修复之间的依赖关系
    • 将实施计划写入计划文件
    • 调用 ExitPlanMode 请求批准
  3. 按计划顺序执行获批的修复
  4. 显示完成摘要
  5. 将控制权交还调用方——不要自动重新运行 verify 或提交更改
模式:auto-fix

呈现报告后:

  1. 自动接受所有严重程度 >= --auto-fix-threshold(默认值:3)的问题
  2. 跳过低于阈值的问题(在摘要中注明,但不采取操作)
  3. 不提示用户——不要使用 AskUserQuestion
  4. 进入计划模式,分析修复并编写计划
  5. 执行修复
  6. 显示完成摘要
  7. 交还控制权(不要停止——由调用方继续)
  8. 不要提交更改

关键:不得隐藏问题

报告必须绝对诚实:

  • 测试应始终通过——100% 的通过率是唯一可接受的结果
  • 测试应始终能够运行——任何设置/环境问题都是缺陷
  • 保持透明——醒目地呈现所有问题
  • 报告事实——由人来决定采取哪些行动
严重程度等级(1-10)
范围影响示例
9-10严重数据丢失、安全漏洞、无法运行
7-8高主要功能失效、存在重大问题
5-6中等存在明显问题,但有变通方法
3-4低小问题、轻微不便
1-2轻微润色、外观问题、可选改进

严重程度反映的是“这个问题有多严重?”——而不是“必须修复吗?”由人来决定采取哪些行动。

上下文窗口使用规范

这很关键,因为 verify 会在主上下文窗口中运行。

  • 精简编排器:解析参数、运行 git 命令、应用分诊、调用技能、收集结构化结果、整理报告。除检测范围外,不读取文件。
  • 繁重工作交由技能完成:每个辅助技能都有自己的上下文窗口。它们负责读取文件、搜索和调查。
  • 仅提取结构化信息:收集技能结果时,仅提取:标题、严重程度、位置、类别、描述。舍弃调查过程叙述。
  • 编排器不得完整读取文件:除修复执行阶段(交互模式/自动修复模式)外,绝不读取源文件。
  • 精简上下文包:范围 + 静态摘要 + 差异统计。最多约 50–100 行。

Harness 绑定

上述流水线与 harness 无关。这些是它对应的实现机制。对于第三种 harness,只需添加一列,其他内容保持不变。

之所以有这张表,是因为第二列长期缺失,而且缺失情况并不明显:流水线仍能生成正确的报告,只是速度慢了 8 倍。未声明绑定的能力不会明确报错——它会静悄悄地采用技能提供的任一回退方案。

能力Claude CodeCodex CLI
启动子 Agent 且不阻塞每个技能调用一次 Skill 或 Agent,全部放在同一条消息中每轮调用一次 spawn_agent;不到一秒即可返回句柄
等待所有未完成任务隐式等待——harness 会在它们完成后返回使用较长的 timeout_ms 调用一次 wait_agent;仅在其返回后再次调用
同时运行的子 Agent 数量上限实际中未遇到features.multi_agent_v2.max_concurrent_threads_per_session,默认值为 4;计算所有活跃祖先,包括此 Agent。启动被拒时会报告 agent thread limit reached
只读探索子 Agent(阶段 3)Explore Agent 类型使用只读沙箱调用 spawn_agent
为子 Agent 指定特定技能按名称调用 Skill 工具在启动的 Agent 消息中指定技能的 SKILL.md 绝对路径;会话的技能清单中列有该路径
为每个子 Agent 选择模型或推理力度在调用中指定 model,或使用技能的 frontmatter在 spawn_agent 中指定 model / reasoning_effort
向用户提问(阶段 7c、阶段 9)AskUserQuestion在 final channel 中提问;若没有用户参与,则改用 --mode=report-only 运行
在修复前进入并退出经审查的计划EnterPlanMode / ExitPlanMode在 final channel 中提交计划并等待回复
将 JSON 报告写入指定路径Write 工具apply_patch 或 shell 重定向

上限低于适用技能数量时,这是配置问题,而非流水线问题。 阶段 6 遇到这种情况时会正确降级,但修复应在配置该 harness 的位置进行——并且由于上限会计算祖先,真正重要的是整个树的深度,也就是当前 verify 所处的深度,而不是 verify 单独期望的深度。

适用技能的数量本身也取决于深度:light 会扩展到少数几个技能,因此不太可能触及上限;而 deep 才是较低上限会让并行阶段变成串行阶段的情况。在 light 下不会触发限制的上限,对 deep 来说仍可能严重不合适。

评论

支持 Markdown