test-failure-investigator

Test Failure Investigator

Runbook-style skill for systematic test failure investigation. Given a failing test, determines root cause and recommends action.

Activation

/test-failure-investigator [test-name-or-file]

Investigation Flow

Step 1: Classify the Failure

Run the test 3 times and classify:

Result Pattern	Classification	Action
Fails consistently	Regression or Environment	Continue to Step 2
Fails intermittently	Flaky	Skip to Step 4
Passes now	Transient	Check CI logs, environment diff

# Run test 3 times
for i in 1 2 3; do npx jest {{test_file}} 2>&1 | tail -5; echo "--- Run $i ---"; done

Step 2: Narrow the Scope

# When did it start failing?
git log --oneline -20 -- {{related_source_files}}

# What changed recently?
git diff HEAD~5 -- {{related_source_files}}

# Does it fail in isolation?
npx jest {{test_file}} --testNamePattern="{{test_name}}"

# Does it fail with other tests?
npx jest --runInBand  # sequential execution

Step 3: Root Cause Analysis

Symptom	Likely Cause	Investigation
Timeout	Network/DB dependency	Check external service availability
Assertion mismatch	Logic change	Compare expected vs actual, check git blame
Import error	Dependency change	Check package.json changes, run npm ci
Permission denied	Environment	Check file permissions, Docker volumes
Out of memory	Resource leak	Profile with --detectOpenHandles

Step 4: Flaky Test Investigation

# Run 10 times to confirm flakiness
for i in $(seq 1 10); do npx jest {{test_file}} --forceExit 2>&1 | grep -E 'PASS|FAIL'; done

# Common flaky causes:
# - Shared state between tests (missing cleanup)
# - Time-dependent assertions (use fake timers)
# - Race conditions (missing await)
# - Port conflicts (use random ports)
# - Order dependency (run with --randomize)

Step 5: Report

## Test Failure Report
- **Test**: {{test_name}}
- **File**: {{test_file}}
- **Classification**: Regression / Flaky / Environment / Transient
- **Root Cause**: {{description}}
- **First Failed**: {{commit_hash}} ({{date}})
- **Fix**: {{recommended_action}}
- **Verified**: [ ] Fix applied and test passes 3x consecutively

Composition

After investigation, compose with:

• /bug-reporting-excellence — if regression found, file a bug report
• /regression-testing — if regression, add to regression suite
• /qe-test-execution — for re-running tests after fix

Gotchas

• Agent may guess at root cause without running the test — always reproduce first
• "Works on my machine" is not a diagnosis — compare environments (node version, OS, deps)
• Flaky tests that pass 9/10 times will still be reported as "passing" by CI — run 10+ times
• Test isolation failures are the #1 cause of flaky tests — check for shared state in beforeAll/afterAll