mcpField
R
reproducibility-audit

reproducibility-audit

github.com/jurgendnTesting & quality1163 installsUpdated 6w ago
Repository

About reproducibility-audit

A skill for checking whether a research result can be reproduced from its available artifacts and documented steps. It records blockers and assigns a status without claiming success unless the work was actually run and checked.

Key features

  1. 01Check code, data, configuration, and versions
  2. 02Map paper claims to executable steps
  3. 03Record ambiguities and blockers
  4. 04Log commands and outputs
  5. 05Classify reproduction status and fixes

Use cases

  1. 01Assess a paper's reproducibility before relying on its results
  2. 02Identify missing artifacts in a replication attempt
  3. 03Document evidence and blockers during reproduction

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/jurgendn/agent-skills --skill reproducibility-audit

After installing, type/reproducibility-auditin Claude Code to confirm it loaded.

Bundled filesreferences/good-enough-baseline.md

Details

From the project README

Reproducibility Audit

Assume the result is not reproducible until the missing pieces are enumerated.

Workflow

  1. Identify the target result to reproduce.

  2. Check for the required ingredients:

    • code
    • commit/version pinning
    • data access
    • config files
    • preprocessing steps
    • hardware assumptions
    • seed handling

    references/good-enough-baseline.md expands this into a concrete presence/absence checklist (data provenance, pinned dependencies, smoke test, version identity, determinism) grounded in Good Enough Practices in Scientific Computing (Wilson et al., 2017) — a missing item there is a finding, not a minor annoyance.

  3. Attempt to map paper claims to executable steps.

  4. Record ambiguities and blockers explicitly.

  5. If running code, log exact commands and outputs.

  6. Classify status:

    • reproduced
    • partially reproduced
    • blocked
    • inconsistent with paper

  7. Recommend the smallest fixes needed for reproducibility.

Rules

  • Do not say reproduced unless you ran and checked it.
  • Missing config details are findings, not minor annoyances.
  • Separate code availability from result reproducibility.

Output shape

Return:

  • Target result
  • Available artifacts
  • Missing artifacts
  • Reproduction status
  • Commands / evidence
  • Recommended fixes

Comments

Markdown supported