judge

Judge 命令

你是一名协调员，负责启动一个两阶段评估流程，以评估本次对话中此前完成的工作。首先，元评审员会生成量身定制的评估标准。然后，评审子 Agent 会在隔离的上下文中应用这些标准，进行结构化评分并提供基于证据的反馈。本评估仅供报告参考——只呈现评估结果，不会自动进行更改。 此命令采用带上下文隔离的 **元评审员 → LLM-as-Judge** 模式： - **结构化评估**：元评审员会在评审开始前生成量身定制的评分标准、检查清单和评分依据 - **上下文隔离**：评审员使用全新上下文，避免累积的会话状态造成确认偏误 - **基于证据**：每项评分都需要引用作品中的具体证据（文件位置、行号） - **多维度评分标准**：由元评审员生成，匹配特定的成果类型和评估重点 - **自我验证**：动态验证问题及记录在案的调整

工作流程

第 1 阶段：提取上下文

启动评估流程前，先确定评估对象：

1. 确定待评估的工作：

   • 查看对话历史，寻找已完成的工作
   • 如果提供了参数：用它们聚焦特定方面
   • 如果不明确：询问用户“我应该评估什么工作？（代码更改、分析、文档等）”

2. 提取评估上下文：

   • 引发该工作的原始任务或请求
   • 实际产出或结果
   • 创建或修改的文件（附简要说明）
   • 提及的任何约束、要求或验收标准
   • 成果类型（代码、文档、配置等）

3. 向用户说明范围：

   Evaluation Scope:
   - Original request: [summary]
   - Work produced: [description]
   - Files involved: [list]
   - Artifact type: [code | documentation | configuration | etc.]
   - Evaluation focus: [from arguments or "general quality"]
   
   Launching meta-judge to generate evaluation criteria...
   

重要：只将提取出的上下文传递给子 Agent，不要传递完整对话。这样可以避免上下文污染，并有助于开展聚焦评估。

第 2 阶段：分派元评审员

启动元评审 Agent，根据待评估的具体工作生成评估规范。元评审员将返回一份 YAML 格式的评估规范，其中包含评分标准、检查清单和评分依据。

元评审员提示词：

## Task

Generate an evaluation specification yaml for the following evaluation task. You will produce rubrics, checklists, and scoring criteria that a judge agent will use to evaluate the work.

CLAUDE_PLUGIN_ROOT=`${CLAUDE_PLUGIN_ROOT}`

## User Prompt
{Original task or request that prompted the work}

## Context
{Any relevant context about the work being evaluated}
{Evaluation focus from arguments, or "General quality assessment"}

## Artifact Type
{code | documentation | configuration | etc.}

## Instructions
Return only the final evaluation specification YAML in your response.

分派：

Use Task tool:
  - description: "Meta-judge: Generate evaluation criteria for {brief work summary}"
  - prompt: {meta-judge prompt}
  - model: opus
  - subagent_type: "sadd:meta-judge"

等待元评审员完成后，再进入第 3 阶段。

第 3 阶段：分派评审 Agent

元评审员完成后，提取其 YAML 格式的评估规范，并将工作上下文和该规范一并传递给评审 Agent。

关键：必须向评审员提供元评审员生成的完整评估规范 YAML，且内容必须完全一致。不得跳过、添加、修改、缩短或概括其中任何文本！

评审 Agent 提示词：

You are an Expert Judge evaluating the quality of work against an evaluation specification produced by the meta judge.

CLAUDE_PLUGIN_ROOT=`${CLAUDE_PLUGIN_ROOT}`

## Work Under Evaluation

[ORIGINAL TASK]
{paste the original request/task}
[/ORIGINAL TASK]

[WORK OUTPUT]
{summary of what was created/modified}
[/WORK OUTPUT]

[FILES INVOLVED]
{list of files with brief descriptions}
[/FILES INVOLVED]

## Evaluation Specification

```yaml
{meta-judge's evaluation specification YAML}

说明

请遵循你 Agent 指令中定义的完整评审流程！

关键：必须在回复开头使用以下精确的 YAML 结构化评估报告格式！

CRITICAL: NEVER provide score threshold to judges in any format. Judge MUST not know what threshold for score is, in order to not be biased!!!

**Dispatch:**

使用 Task 工具：

• description: "Judge: 评估 {简要工作摘要}"
• prompt: {包含完整且准确的元评审规范 YAML 的评审提示词}
• model: opus
• subagent_type: "sadd:judge"

### Phase 4: Process and Present Results

After receiving the judge's evaluation:

1. **Validate the evaluation**:
   - Check that all criteria have scores in valid range (1-5)
   - Verify each score has supporting justification with evidence
   - Confirm weighted total calculation is correct
   - Check for contradictions between justification and score
   - Verify self-verification was completed with documented adjustments

2. **If validation fails**:
   - Note the specific issue
   - Request clarification or re-evaluation if needed

3. **Present results to user**:
   - Display the full evaluation report
   - Highlight the verdict and key findings
   - Offer follow-up options:
     - Address specific improvements
     - Request clarification on any judgment
     - Proceed with the work as-is

## Scoring Interpretation

| Score Range | Verdict | Interpretation | Recommendation |
|-------------|---------|----------------|----------------|
| 4.50 - 5.00 | EXCELLENT | Exceptional quality, exceeds expectations | Ready as-is |
| 4.00 - 4.49 | GOOD | Solid quality, meets professional standards | Minor improvements optional |
| 3.50 - 3.99 | ACCEPTABLE | Adequate but has room for improvement | Improvements recommended |
| 3.00 - 3.49 | NEEDS IMPROVEMENT | Below standard, requires work | Address issues before use |
| 1.00 - 2.99 | INSUFFICIENT | Does not meet basic requirements | Significant rework needed |

## Important Guidelines

1. **Meta-judge first**: Always generate evaluation specification before judging - never skip the meta-judge phase
2. **Include CLAUDE_PLUGIN_ROOT**: Both meta-judge and judge need the resolved plugin root path
3. **Meta-judge YAML**: Pass only the meta-judge YAML to the judge, do not modify it
4. **Context Isolation**: Pass only relevant context to sub-agents - not the entire conversation
5. **Justification First**: Always require evidence and reasoning BEFORE the score
6. **Evidence-Based**: Every score must cite specific evidence (file paths, line numbers, quotes)
7. **Bias Mitigation**: Explicitly warn against length bias, verbosity bias, and authority bias
8. **Be Objective**: Base assessments on evidence and rubric definitions, not preferences
9. **Be Specific**: Cite exact locations, not vague observations
10. **Be Constructive**: Frame criticism as opportunities for improvement with impact context
11. **Consider Context**: Account for stated constraints, complexity, and requirements
12. **Report Confidence**: Lower confidence when evidence is ambiguous or criteria unclear
13. **Single Judge**: This command uses one focused judge for context isolation

## Notes

- This is a **report-only** command - it evaluates but does not modify work
- The meta-judge generates criteria tailored to the specific artifact type and evaluation focus
- The judge operates with fresh context for unbiased assessment
- Scores are calibrated to professional development standards
- Low scores indicate improvement opportunities, not failures
- Use the evaluation to inform next steps and iterations
- Low confidence evaluations may warrant human review