qe-test-execution

QE 测试执行

目的

指导如何使用 v3 的测试执行能力，包括并行编排、智能测试选择、处理不稳定测试，以及在多个环境中分布式执行。

激活条件

• 运行测试套件时
• 优化测试执行时间时
• 处理不稳定测试时
• 设置 CI/CD 测试流水线时
• 在多个环境中执行测试时

快速入门

# Run all tests with parallelization
aqe test run --parallel --workers 4

# Run affected tests only
aqe test run --affected --since HEAD~1

# Run with retry for flaky tests
aqe test run --retry 3 --retry-delay 1000

# Run specific test types
aqe test run --type unit,integration --exclude e2e

Agent 工作流

// Orchestrate test execution
Task("Execute test suite", `
  Run the full test suite with:
  - 4 parallel workers
  - Retry flaky tests up to 3 times
  - Generate JUnit report
  - Fail fast on critical tests
  Report results and any failures.
`, "qe-test-executor")

// Smart test selection
Task("Run affected tests", `
  Analyze changes in PR #123 and:
  - Identify affected test files
  - Run only relevant tests
  - Include integration tests for changed modules
  - Report coverage delta
`, "qe-test-selector")

执行策略

1. 并行执行

await testExecutor.runParallel({
  suites: ['unit', 'integration'],
  workers: 4,
  distribution: 'by-file',  // or 'by-test', 'by-duration'
  isolation: 'process',
  sharding: {
    enabled: true,
    total: 4,
    index: process.env.SHARD_INDEX
  }
});

2. 智能测试选择

await testExecutor.runAffected({
  changes: gitChanges,
  selection: {
    direct: true,      // Tests for changed files
    transitive: true,  // Tests for dependents
    integration: true  // Integration tests touching changed code
  },
  fallback: 'full-suite'  // If analysis fails
});

3. 处理不稳定测试

await testExecutor.handleFlaky({
  detection: {
    enabled: true,
    threshold: 0.1,  // 10% flake rate
    window: 100      // Last 100 runs
  },
  strategy: {
    retry: 3,
    quarantine: true,
    notify: ['#flaky-tests']
  }
});

执行配置

execution:
  parallel:
    workers: auto  # CPU cores - 1
    timeout: 30000
    bail: false

  retry:
    count: 2
    delay: 1000
    only_failed: true

  reporting:
    formats: [junit, json, html]
    include_timing: true
    include_logs: true

  environments:
    - name: node-18
      image: node:18-alpine
    - name: node-20
      image: node:20-alpine

CI/CD 集成

# GitHub Actions example
test:
  runs-on: ubuntu-latest
  strategy:
    matrix:
      shard: [1, 2, 3, 4]
  steps:
    - uses: actions/checkout@v4
    - name: Run tests
      run: |
        aqe test run \
          --shard ${{ matrix.shard }}/4 \
          --parallel \
          --report junit
    - name: Upload results
      uses: actions/upload-artifact@v4
      with:
        name: test-results-${{ matrix.shard }}
        path: reports/

结果汇总

interface ExecutionResults {
  summary: {
    total: number;
    passed: number;
    failed: number;
    skipped: number;
    flaky: number;
    duration: number;
  };
  shards: ShardResult[];
  failures: TestFailure[];
  flakyTests: FlakyTest[];
  coverage: CoverageReport;
  timing: TimingAnalysis;
}

注意事项

• 在容器中运行完整测试套件可能导致 OOM——尽管 CLAUDE.md 中的规则写着“不要运行完整测试套件”，但这一规则仍被违反了 20 次。解决方法：让测试套件更轻量，而不是只添加更多规则。
• 少量目标明确的 Agent（3-4 个）效果优于许多目标模糊的 Agent（6-8 个）——务必在每个 Agent 提示词中加入验证命令。
• 新模型发布可能导致 Agent 在冲刺中途改变行为——模型更新后，昨天还会遵守的规则，今天可能就被忽略。
• 并行运行所有测试可能掩盖不稳定测试——初步诊断时使用 --workers=1。
• 会话崩溃会导致所有上下文丢失——将中间结果保存到磁盘，而不只是保存在内存中。

协调

主要 Agent：qe-test-executor、qe-test-selector、qe-flaky-detector 协调器：qe-test-execution-coordinator 相关技能：qe-test-generation、qe-coverage-analysis