test-gap-analysis

测试缺口分析

通过系统分析测试的完整性、有效性和风险覆盖情况，找出应测试的需求／功能与实际测试覆盖之间的差异，从而提升软件质量和可靠性。

何时使用

在以下情况下使用此技能：

• 不确定系统哪些部分已得到充分测试
• 添加了新功能，但没有相应的测试覆盖
• 测试资源有限，需要优化分配
• 准备发布或部署，并希望确认测试覆盖情况
• 审核测试有效性，以满足合规或质量标准
• 识别测试不足的高风险领域
• 规划测试自动化或测试改进计划
• 帮助新团队成员了解测试覆盖情况
• 评估测试基础设施方面的技术债务
• 验证关键功能是否得到适当的测试保护

我能做什么

1. 需求可测试性分析

• 分析需求的可测试性：识别每项需求中可测试的方面
• 从需求文档中提取测试条件和场景
• 按测试类型对需求分类：单元、集成、系统、验收、性能、安全
• 评估需求描述对测试是否清晰：区分模糊需求与可测试需求
• 识别应进行测试但未记录的隐含需求

2. 测试覆盖分析

• 分析现有测试套件：单元测试、集成测试、端到端测试、手动测试
• 将测试映射到需求：确定哪些需求已获得测试覆盖
• 衡量覆盖率指标：行覆盖率、分支覆盖率、需求覆盖率、风险覆盖率
• 识别测试缺口：没有测试的需求、覆盖不完整的需求、过时的测试
• 评估测试有效性：测试质量、不稳定性、维护负担

3. 缺口识别与分类

• 测试覆盖缺失：没有对应测试的需求
• 测试覆盖不完整：仅有部分测试覆盖的需求
• 过时的测试：与当前需求不匹配的测试
• 无效的测试：无法充分验证需求的测试
• 风险覆盖缺口：测试不足的高风险领域
• 测试类型缺口：缺少某些测试类型（安全、性能、无障碍）

4. 基于风险的优先级排序

• 评估未测试需求的业务影响
• 评估测试不足带来的技术风险
• 根据影响和可能性计算测试缺口的严重程度
• 为修复计划确定测试缺口的优先级
• 估算弥补每个测试缺口所需的工作量
• 根据风险状况提出测试策略建议

5. 测试改进建议

• 生成具体测试用例以弥补缺口
• 针对每个缺口推荐测试类型和方法
• 提出测试自动化机会
• 建议改进测试基础设施
• 为尚未覆盖的场景设计测试数据策略
• 制定测试维护计划，防止未来出现缺口

测试缺口类型

1. 覆盖缺口

测试覆盖缺失：完全没有测试的需求
测试覆盖不完整：测试覆盖不完整的需求
示例：对身份验证需求只测试了成功场景，没有测试失败场景

2. 类型缺口

缺少测试类型：未包含关键测试类型（安全、性能等）
测试类型不当：为需求选择了错误的测试类型（单元测试与集成测试）
示例：对性能要求严格的功能只有单元测试，没有性能测试

3. 有效性缺口

无效的测试：无法充分验证需求的测试
不稳定的测试：不可靠、会带来虚假信心的测试
示例：测试通过了，却没有实际验证需求

4. 维护缺口

测试过时：与当前需求不符的测试
测试未维护：频繁失败但未修复的测试
示例：测试套件中仍包含针对已弃用功能的测试

5. 风险覆盖缺口

高风险未测试：缺少充分测试的关键功能
低风险过度测试：接受过多测试的非关键功能
示例：支付处理只有少量测试，而 UI 有大量测试

分析技术

需求测试提取

def extract_testable_requirements(requirements_doc):
    """
    Extract testable conditions from requirements.
    """
    testable_items = []
    
    for req in requirements_doc:
        test_conditions = analyze_requirement_for_tests(req)
        
        if test_conditions:
            testable_items.append({
                'requirement_id': req['id'],
                'description': req['description'],
                'test_conditions': test_conditions,
                'test_types': determine_appropriate_test_types(req),
                'priority': calculate_test_priority(req),
                'risk_level': assess_requirement_risk(req)
            })
    
    return testable_items

def analyze_requirement_for_tests(requirement):
    """Extract testable conditions from a requirement."""
    test_conditions = []
    
    # Look for explicit testable statements
    if 'shall' in requirement['text'].lower():
        # Extract what the system shall do
        import re
        shall_pattern = r'shall\s+([^.!?]+)'
        matches = re.findall(shall_pattern, requirement['text'], re.IGNORECASE)
        test_conditions.extend(matches)
    
    # Look for acceptance criteria
    if 'acceptance_criteria' in requirement:
        for criterion in requirement['acceptance_criteria']:
            test_conditions.append(criterion)
    
    # Look for constraints
    if 'constraints' in requirement:
        for constraint in requirement['constraints']:
            test_conditions.append(f"Constraint: {constraint}")
    
    return test_conditions

测试覆盖映射

class TestCoverageAnalyzer:
    def __init__(self, requirements, tests):
        self.requirements = requirements
        self.tests = tests
        
    def analyze_coverage(self):
        """Analyze test coverage of requirements."""
        coverage_report = {
            'requirements_count': len(self.requirements),
            'tests_count': len(self.tests),
            'covered_requirements': [],
            'uncovered_requirements': [],
            'partial_coverage': [],
            'coverage_percentage': 0
        }
        
        for req in self.requirements:
            matching_tests = self.find_tests_for_requirement(req)
            
            if not matching_tests:
                coverage_report['uncovered_requirements'].append({
                    'requirement': req,
                    'gap_type': 'missing_coverage'
                })
            elif len(matching_tests) < self.get_expected_test_count(req):
                coverage_report['partial_coverage'].append({
                    'requirement': req,
                    'matching_tests': matching_tests,
                    'expected_count': self.get_expected_test_count(req),
                    'actual_count': len(matching_tests),
                    'gap_type': 'partial_coverage'
                })
            else:
                coverage_report['covered_requirements'].append({
                    'requirement': req,
                    'matching_tests': matching_tests
                })
        
        coverage_report['coverage_percentage'] = (
            len(coverage_report['covered_requirements']) / 
            len(self.requirements) * 100 if self.requirements else 0
        )
        
        return coverage_report
    
    def find_tests_for_requirement(self, requirement):
        """Find tests that cover a requirement."""
        matching_tests = []
        
        for test in self.tests:
            if self.test_covers_requirement(test, requirement):
                matching_tests.append(test)
        
        return matching_tests
    
    def test_covers_requirement(self, test, requirement):
        """Determine if a test covers a requirement."""
        # Check test name for requirement reference
        if requirement['id'] in test.get('name', ''):
            return True
        
        # Check test description for requirement reference
        if requirement['id'] in test.get('description', ''):
            return True
        
        # Check test tags for requirement reference
        if 'tags' in test and requirement['id'] in test['tags']:
            return True
        
        # Semantic analysis (simplified)
        if self.semantic_match(test.get('purpose', ''), requirement['description']):
            return True
        
        return False

基于风险的缺口优先级排序

def prioritize_test_gaps(test_gaps, risk_assessment):
    """
    Prioritize test gaps based on risk and impact.
    """
    prioritized_gaps = []
    
    for gap in test_gaps:
        # Calculate priority score
        priority_score = calculate_priority_score(gap, risk_assessment)
        
        prioritized_gaps.append({
            **gap,
            'priority_score': priority_score,
            'priority_level': determine_priority_level(priority_score),
            'remediation_effort': estimate_remediation_effort(gap),
            'risk_exposure': calculate_risk_exposure(gap, risk_assessment)
        })
    
    # Sort by priority score (descending)
    prioritized_gaps.sort(key=lambda x: x['priority_score'], reverse=True)
    
    return prioritized_gaps

def calculate_priority_score(gap, risk_assessment):
    """Calculate priority score for a test gap."""
    weights = {
        'business_impact': 0.4,
        'technical_risk': 0.3,
        'user_impact': 0.2,
        'regulatory_requirement': 0.1
    }
    
    scores = {}
    
    # Business impact score
    if gap['requirement']['business_criticality'] == 'high':
        scores['business_impact'] = 1.0
    elif gap['requirement']['business_criticality'] == 'medium':
        scores['business_impact'] = 0.5
    else:
        scores['business_impact'] = 0.2
    
    # Technical risk score
    if gap['requirement']['technical_complexity'] == 'high':
        scores['technical_risk'] = 1.0
    elif gap['requirement']['technical_complexity'] == 'medium':
        scores['technical_risk'] = 0.5
    else:
        scores['technical_risk'] = 0.2
    
    # User impact score
    if gap['requirement']['user_exposure'] == 'high':
        scores['user_impact'] = 1.0
    elif gap['requirement']['user_exposure'] == 'medium':
        scores['user_impact'] = 0.5
    else:
        scores['user_impact'] = 0.2
    
    # Regulatory requirement score
    if gap['requirement'].get('regulatory_requirement', False):
        scores['regulatory_requirement'] = 1.0
    else:
        scores['regulatory_requirement'] = 0.0
    
    # Calculate weighted score
    weighted_score = sum(
        scores[factor] * weights[factor] 
        for factor in weights
    )
    
    return weighted_score

示例

# Analyze test coverage gaps
npm run test-gap-analysis:analyze -- --requirements specs/ --tests tests/ --output gaps.json

# Map tests to requirements
npm run test-gap-analysis:map -- --requirements requirements.md --test-suite tests/unit/ --output coverage-map.yaml

# Identify high-risk untested requirements
npm run test-gap-analysis:risk -- --requirements specs/ --tests tests/ --risk-factors risk-assessment.yaml --output risk-gaps.md

# Generate test recommendations
npm run test-gap-analysis:recommend -- --gaps gaps.json --output recommendations.md

# Calculate test coverage metrics
npm run test-gap-analysis:metrics -- --requirements specs/ --tests tests/ --output metrics.json

# Continuous test gap monitoring
npm run test-gap-analysis:monitor -- --watch --requirements specs/ --tests tests/ --alert-on-gap

输出格式

测试缺口分析报告：

Test Gap Analysis Report
────────────────────────
System: Payment Processing Service
Analysis Date: 2026-02-26
Requirements Analyzed: 42
Tests Analyzed: 127

Test Coverage Summary:
✅ Fully Tested: 28 requirements (66.7%)
⚠️ Partially Tested: 8 requirements (19.0%)
❌ Untested: 6 requirements (14.3%)
📊 Overall Coverage: 76.2%

Critical Test Gaps:

1. ❌ Payment Fraud Detection (High Risk)
   • Requirement: RQ-042: "System shall detect suspicious payment patterns"
   • Risk: Financial loss, regulatory compliance
   • Test Gap: No fraud detection tests
   • Priority: CRITICAL
   • Recommendation: Add fraud detection test suite with edge cases

2. ⚠️ Payment Refund Processing (Medium Risk)
   • Requirement: RQ-038: "System shall process refunds within 24 hours"
   • Risk: Customer dissatisfaction, financial reconciliation
   • Test Gap: Partial coverage (success cases only)
   • Priority: HIGH
   • Recommendation: Add failure scenarios, timeout tests, concurrency tests

3. ⚠️ Multi-Currency Support (Medium Risk)
   • Requirement: RQ-035: "System shall support 15+ currencies"
   • Risk: International expansion blocked
   • Test Gap: Only 5 currencies tested
   • Priority: MEDIUM
   • Recommendation: Add remaining currency tests, exchange rate tests

4. ℹ️ Payment Receipt Generation (Low Risk)
   • Requirement: RQ-041: "System shall generate PDF receipts"
   • Risk: Minor user inconvenience
   • Test Gap: No PDF validation tests
   • Priority: LOW
   • Recommendation: Add PDF generation and validation tests

Risk Analysis:
┌────────────────────┬──────────┬────────────┬──────────────┐
│ Risk Category      │ Coverage│ Risk Level │ Action Needed│
├────────────────────┼──────────┼────────────┼──────────────┤
│ Security          │ 40%      │ HIGH       │ ⚠️ Immediate │
│ Financial         │ 75%      │ MEDIUM     │ 📅 Soon      │
│ Compliance        │ 90%      │ LOW        │ ℹ️ Optional   │
│ User Experience   │ 85%      │ LOW        │ ℹ️ Optional   │
└────────────────────┴──────────┴────────────┴──────────────┘

Test Type Distribution:
• Unit Tests: 68 (53.5%)
• Integration Tests: 42 (33.1%)
• End-to-End Tests: 12 (9.4%)
• Performance Tests: 3 (2.4%)
• Security Tests: 2 (1.6%)

Test Effectiveness:
• Flaky Tests: 8 (6.3%)
• Slow Tests (>1s): 15 (11.8%)
• Unmaintained Tests: 5 (3.9%)
• High-Value Tests: 42 (33.1%)

Remediation Plan:
1. Week 1: Implement fraud detection tests (critical)
2. Week 2: Complete refund processing tests (high)
3. Week 3: Add missing currency tests (medium)
4. Week 4: Fix flaky tests, add PDF tests (low)
5. Ongoing: Test gap monitoring, prevention

Estimated Effort: 2-3 weeks
Target Coverage: 90% by 2026-03-19

测试缺口 JSON 输出：

{
  "analysis": {
    "system": "payment-processing",
    "timestamp": "2026-02-26T19:00:00Z",
    "requirements_analyzed": 42,
    "tests_analyzed": 127,
    "coverage_percentage": 76.2
  },
  "coverage_summary": {
    "fully_tested": 28,
    "partially_tested": 8,
    "untested": 6,
    "coverage_by_type": {
      "unit": 85.7,
      "integration": 71.4,
      "e2e": 42.9,
      "performance": 14.3,
      "security": 28.6
    }
  },
  "test_gaps": [
    {
      "id": "gap-test-001",
      "requirement_id": "RQ-042",
      "requirement_description": "System shall detect suspicious payment patterns",
      "gap_type": "missing_coverage",
      "risk_level": "critical",
      "business_impact": "high",
      "technical_complexity": "high",
      "user_exposure": "medium",
      "test_types_needed": ["unit", "integration", "security"],
      "recommended_tests": [
        "Test fraud pattern detection",
        "Test threshold-based alerts",
        "Test false positive handling",
        "Test integration with fraud service"
      ],
      "priority_score": 92,
      "priority_level": "critical",
      "estimated_effort_hours": 16,
      "owner": "security-qa-team"
    },
    {
      "id": "gap-test-002",
      "requirement_id": "RQ-038",
      "requirement_description": "System shall process refunds within 24 hours",
      "gap_type": "partial_coverage",
      "existing_tests": 3,
      "needed_tests": 8,
      "missing_test_scenarios": [
        "Refund timeout handling",
        "Concurrent refund processing",
        "Partial refund scenarios",
        "Refund failure recovery"
      ],
      "risk_level": "high",
      "priority_score": 78,
      "priority_level": "high",
      "estimated_effort_hours": 8,
      "owner": "payment-qa-team"
    }
  ],
  "risk_analysis": {
    "high_risk_untested": 2,
    "medium_risk_partial": 3,
    "low_risk_gaps": 4,
    "risk_coverage_score": 65.8
  },
  "test_effectiveness": {
    "flaky_tests": 8,
    "slow_tests": 15,
    "unmaintained_tests": 5,
    "high_value_tests": 42,
    "effectiveness_score": 72.4
  },
  "recommendations": {
    "immediate": [
      "Implement fraud detection test suite",
      "Review and fix flaky security tests"
    ],
    "short_term": [
      "Complete refund processing test coverage",
      "Add missing currency tests"
    ],
    "long_term": [
      "Implement test gap monitoring",
      "Improve test maintenance process"
    ]
  }
}

测试覆盖仪表板：

Test Coverage Dashboard
───────────────────────
Status: ACTIVE
Last Analysis: 2026-02-26 19:00:00
Next Analysis: 2026-02-27 07:00:00

Coverage Trends:
┌──────────────────────────────────────┐
│ Coverage Trend (Last 30 Days)       │
│                                      │
│ 100 ┤                               │
│     │                               │
│  90 ┤                               │
│     │                █              │
│  80 ┤               █ █             │
│     │              █   █            │
│  70 ┤            ██     █           │
│     │          ██        █          │
│  60 ┼───────██───────────█─────────│
│       1   5   10   15   20   25   30 │
│                 Days                 │
└──────────────────────────────────────┘

Current Coverage by Risk Category:
• Critical Security: 40% ⚠️
• High Business Impact: 75% ⚠️
• Medium Complexity: 88% ✅
• Low Risk: 95% ✅

Test Gap Distribution:
• Missing Coverage: 6 gaps
• Partial Coverage: 8 gaps
• Outdated Tests: 5 gaps
• Ineffective Tests: 8 gaps

Test Health Metrics:
• Flaky Test Rate: 6.3% (⚠️ Above threshold)
• Slow Test Rate: 11.8% (✅ Within limits)
• Test Maintenance Score: 72/100 (⚠️ Needs improvement)
• Test Value Score: 65/100 (⚠️ Needs improvement)

Alert Status:
⚠️  2 critical test gaps aging > 7 days
⚠️  Flaky test rate above 5% threshold
✅  Overall coverage above 75% target
✅  High-risk coverage improving

Recommended Actions:
1. Address 2 critical test gaps
2. Reduce flaky test rate below 5%
3. Improve test maintenance score to 80+
4. Increase high-value test percentage

注意事项

• 测试缺口分析不是为了追责——应着眼于提升质量，而不是追究责任
• 100% 的测试覆盖率很少是目标——应着重提高基于风险的测试有效性
• 测试缺口表明存在质量风险——并不代表测试失败
• 定期分析可防止缺口累积——小而频繁的分析优于大规模审计
• 让利益相关者参与——开发人员、QA 和产品经理应协作解决测试缺口
• 根据风险确定优先级——并非所有测试缺口都同等重要
• 跟踪缺口闭合情况——衡量一段时间内测试缺口的减少进展
• 主动开展测试缺口分析——不仅用于发现问题，也用于提升质量
• 融入开发工作流——测试缺口分析应为测试策略提供依据
• 记录分析流程——以便重复执行并加以改进
• 肯定测试缺口的减少——认可测试覆盖率和质量方面的改进