About training-run-forensics
training-run-forensics is a skill for diagnosing model-training problems from run curves, configuration, sample outputs, and code or data changes. It separates symptoms from possible causes and prioritizes checks and fixes before another sweep.
Key features
- 01Classify training symptoms
- 02Generate likely causes
- 03Identify discriminating checks
- 04Prioritize fixes by probability and cost
Use cases
- 01Investigate divergence or overfitting
- 02Diagnose noisy validation or training collapse
- 03Choose checks before a new hyperparameter sweep
Install
Copy the command and run it in a terminal.
$
npx skills add https:/After installing, type/training-run-forensicsin Claude Code to confirm it loaded.
Details
From the project READMETraining Run Forensics
Treat the run as evidence.
Intake
Ask for:
- loss/metric curves;
- optimizer, learning rate schedule, batch size, precision, regularization;
- train/validation samples or predictions;
- recent code/data changes;
- hardware/distributed setup if relevant.
Workflow
- Gather curves, configs, sample outputs, and recent code changes.
- Classify the symptom: divergence, overfitting, underfitting, collapse, dead training, noisy validation.
- Generate a shortlist of likely causes.
- Identify the smallest discriminating checks.
- Prioritize fixes by probability and cost.
- State which checks should run before any new sweep.
Rules
- Do not shotgun hyperparameters blindly.
- Separate symptom from cause.
- A curve shape is not a diagnosis until matched against config and data evidence.
- If the symptom points to leakage or split mismatch, route to
data-pipeline-auditor.
Output
Return:
- symptom classification;
- likely causes;
- discriminating checks;
- fix order;
- what evidence would falsify the diagnosis.
Comments
Markdown supported