mcpField
P
programmatic-eda

programmatic-eda

github.com/nimrodfisherData & databases504259 installsUpdated 5w ago
Repository

About programmatic-eda

programmatic-eda is a data-analysis skill for profiling new or questionable datasets before analysis. It combines scripted checks with a checklist and report templates to document data quality and findings.

Key features

  1. 01Dataset shape and type overview
  2. 02Null and outlier checks
  3. 03Distribution and correlation analysis
  4. 04EDA checklist and report templates
  5. 05Pandas and Polars pattern recipes

Use cases

  1. 01Assess a new dataset before analysis
  2. 02Investigate surprising analysis results
  3. 03Check data quality before modeling
  4. 04Summarize dataset issues for stakeholders

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/nimrodfisher/data-analytics-skills --skill programmatic-eda

After installing, type/programmatic-edain Claude Code to confirm it loaded.

Bundled files

References and scripts installed along with SKILL.md.

assets/
eda_report_template.md
findings_summary.md
references/
eda_checklist.md
pandas_polars_recipes.md
quality_thresholds.md
scripts/
correlation_explorer.py
data_overview.py
distribution_summary.py
null_profiler.py
outlier_detector.py

Details

From the project README

When to use

  • You receive a new dataset and need to understand its shape and quality before analysis
  • An analysis produces surprising numbers and you want to verify the underlying data first
  • A stakeholder asks "is this data reliable?" or "what's in this table?"
  • You're about to run a model or statistical test and need data-quality assurance

Process

  1. Load and overview — run scripts/data_overview.py to get row count, dtypes, memory usage, and a sample. Confirm grain (what one row represents).
  2. Null profile — run scripts/null_profiler.py; compare output against thresholds in references/quality_thresholds.md and flag columns above limits.
  3. Outlier detection — run scripts/outlier_detector.py (IQR + z-score) on numeric columns; document flagged values and decide: real signal or data error?
  4. Distribution summary — run scripts/distribution_summary.py for descriptive stats and univariate histograms on each numeric column.
  5. Correlation exploration — run scripts/correlation_explorer.py; flag pairs with |r| > 0.8 as potential multicollinearity or redundancy.
  6. EDA checklist sign-off — work through references/eda_checklist.md and confirm each item before declaring the dataset profiled.
  7. Write findings — fill assets/eda_report_template.md with full profiling output; distil top issues into assets/findings_summary.md.

For pattern recipes (e.g. polars vs pandas equivalents, chunked reads for large files), see references/pandas_polars_recipes.md.

Inputs the skill needs

  • Required: dataset path (CSV / Parquet / Excel) or a DataFrame already in scope
  • Required: business context — what does one row represent?
  • Optional: quality threshold overrides (defaults in references/quality_thresholds.md)
  • Optional: columns to skip (PII, binary blobs, high-cardinality IDs)

Output

  • assets/eda_report_template.md (filled) — full profiling report with per-column stats
  • assets/findings_summary.md (filled) — top 3–5 quality issues and recommended next steps
  • Console output / plots from scripts for interactive inspection

Comments

Markdown supported