qe-quality-assessment

QE Quality Assessment

Purpose

Guide the use of v3's quality assessment capabilities including automated quality gates, metrics aggregation, trend analysis, and deployment readiness evaluation.

Activation

• When evaluating code quality
• When setting up quality gates
• When assessing deployment readiness
• When tracking quality metrics
• When generating quality reports

Quick Start

# Run quality assessment
aqe quality assess --scope src/ --gates all

# Check deployment readiness
aqe quality deploy-ready --environment production

# Generate quality report
aqe quality report --format dashboard --period 30d

# Compare quality between releases
aqe quality compare --from v1.0 --to v2.0

Agent Workflow

// Comprehensive quality assessment
Task("Assess code quality", `
  Evaluate quality for src/:
  - Code complexity (cyclomatic, cognitive)
  - Test coverage and mutation score
  - Security vulnerabilities
  - Code smells and technical debt
  - Documentation coverage
  Generate quality score and recommendations.
`, "qe-quality-analyzer")

// Deployment readiness check
Task("Check deployment readiness", `
  Evaluate if release v2.1.0 is ready for production:
  - All tests passing
  - Coverage thresholds met
  - No critical vulnerabilities
  - Performance benchmarks passed
  - Documentation updated
  Provide go/no-go recommendation.
`, "qe-deployment-advisor")

Quality Dimensions

1. Code Quality Metrics

await qualityAnalyzer.assessCode({
  scope: 'src/**/*.ts',
  metrics: {
    complexity: {
      cyclomatic: { max: 15, warn: 10 },
      cognitive: { max: 20, warn: 15 }
    },
    maintainability: {
      index: { min: 65 },
      duplication: { max: 3 }  // percent
    },
    documentation: {
      publicAPIs: { min: 80 },
      complexity: { min: 70 }
    }
  }
});

2. Quality Gates

await qualityGate.evaluate({
  gates: {
    coverage: { min: 80, blocking: true },
    // Fault detection — coverage is necessary but NOT sufficient (ADR-113).
    // A suite can hit 90% coverage and catch no bugs; mutation score does not lie.
    mutationScore: { min: 0.6, blocking: false },   // warn-by-default; opt-in blocking
    regenerability: { min: 0.5, blocking: false },  // Deletion Test: durable oracle backing per module
    complexity: { max: 15, blocking: false },
    vulnerabilities: { critical: 0, high: 0, blocking: true },
    duplications: { max: 3, blocking: false },
    techDebt: { maxRatio: 5, blocking: false }
  },
  action: {
    onPass: 'proceed',
    onFail: 'block-merge',
    onWarn: 'notify'
  }
});

2a. Regenerability gate (ADR-113)

Coverage measures lines executed; mutation score measures whether the tests would notice a bug, and regenerability answers the Deletion Test — "if this module were deleted and regenerated, would a wrong rebuild be caught?" Regenerability = mutation score discounted by the durability tier backing the module (durable > live > ephemeral

none); ephemeral-only tests score low because they don't survive a reimplementation.

import { evaluateRegenerabilityGate } from '../../../src/feedback/regenerability-gate.js';

const verdict = evaluateRegenerabilityGate(moduleProfiles, {
  mutationScoreMin: 0.6,
  regenerabilityMin: 0.5,
  mode: 'warn',   // 'block' to fail CI; warn-by-default so adoption never breaks pipelines
});
// verdict.passed (thresholds met) · verdict.blocking (should fail CI) · verdict.failures[]

Surface it next to coverage in reports: Coverage 88% ✅ / Mutation 41% ⚠️ / Regenerability: ephemeral-only ⚠️.

3. Deployment Readiness

await deploymentAdvisor.assess({
  release: 'v2.1.0',
  criteria: {
    testing: {
      unitTests: 'all-pass',
      integrationTests: 'all-pass',
      e2eTests: 'critical-pass',
      performanceTests: 'baseline-met'
    },
    quality: {
      coverage: 80,
      noNewVulnerabilities: true,
      noRegressions: true
    },
    documentation: {
      changelog: true,
      apiDocs: true,
      releaseNotes: true
    }
  }
});

Quality Score Calculation

quality_score:
  components:
    test_coverage:
      weight: 0.25
      metrics: [statement, branch, function]

    code_quality:
      weight: 0.20
      metrics: [complexity, maintainability, duplication]

    security:
      weight: 0.25
      metrics: [vulnerabilities, dependencies]

    reliability:
      weight: 0.20
      metrics: [bug_density, flaky_tests, error_rate]

    documentation:
      weight: 0.10
      metrics: [api_coverage, readme, changelog]

  scoring:
    A: 90-100
    B: 80-89
    C: 70-79
    D: 60-69
    F: 0-59

Quality Dashboard

interface QualityDashboard {
  overallScore: number;  // 0-100
  grade: 'A' | 'B' | 'C' | 'D' | 'F';
  dimensions: {
    name: string;
    score: number;
    trend: 'improving' | 'stable' | 'declining';
    issues: Issue[];
  }[];
  gates: {
    name: string;
    status: 'pass' | 'fail' | 'warn';
    value: number;
    threshold: number;
  }[];
  trends: {
    period: string;
    scores: number[];
    alerts: Alert[];
  };
  recommendations: Recommendation[];
}

CI/CD Integration

# Quality gate in pipeline
quality_check:
  stage: verify
  script:
    - aqe quality assess --gates all --output report.json
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
  artifacts:
    reports:
      quality: report.json
  allow_failure:
    exit_codes:
      - 1  # Warnings only

Run History

After each quality assessment, append results to run-history.json in this skill directory:

node -e "
const fs = require('fs');
const h = JSON.parse(fs.readFileSync('.claude/skills/qe-quality-assessment/run-history.json'));
h.runs.push({date: new Date().toISOString().split('T')[0], gate_result: 'PASS_OR_FAIL', failed_checks: []});
fs.writeFileSync('.claude/skills/qe-quality-assessment/run-history.json', JSON.stringify(h, null, 2));
"

Read run-history.json before each run — alert if quality gate failed 3 of last 5 runs.

Skill Composition

• Before assessment → Run /qe-coverage-analysis and /mutation-testing first
• If issues found → Use /test-failure-investigator to diagnose failures
• For PR review → Combine with /code-review-quality for comprehensive review

Gotchas

• NEVER trust agent-reported pass/fail status — 12 test failures were caught that agents claimed were passing (Nagual pattern, reward 0.92)
• Completion theater: agent hardcoded version '3.0.0' instead of reading from package.json — verify actual values in output
• Fix issues in priority waves (P0 → P1 → P2) with verification between each wave — don't fix everything in parallel
• quality-assessment domain has 53.7% success rate — expect failures and have fallback
• If HybridMemoryBackend initialization fails, run aqe health to diagnose, or aqe init to re-initialize

Coordination

Primary Agents: qe-quality-analyzer, qe-deployment-advisor, qe-metrics-collector Coordinator: qe-quality-coordinator Related Skills: qe-coverage-analysis, security-testing