csv-analyzer

CSV Analyzer

Overview

Comprehensive CSV data analysis and visualization engine. Run the script, then use this guide to interpret results and provide insights to users.

Quick Start

cd ~/.claude/skills/csv-analyzer/scripts
export $(grep -v '^#' /path/to/project/.env | xargs 2>/dev/null)
python3 analyze_csv.py /path/to/data.csv

Chart Selection Decision Tree

IMPORTANT: Choose charts based on what the user needs to understand:

What is the user trying to understand?
│
├── "What does my data look like?" (Overview)
│   └── Run with defaults → overview_dashboard.png
│
├── "Is my data clean?" (Quality)
│   └── Check: quality_score, missing_values, duplicates
│   └── Show: missing_values.png if problems exist
│
├── "What's the distribution?" (Single Variable)
│   ├── Numeric → numeric_distributions.png (histogram + KDE)
│   ├── Categorical → categorical_distributions.png (bar chart)
│   └── Time-based → time_series.png
│
├── "Are there outliers?" (Anomalies)
│   └── box_plots.png → points beyond whiskers are outliers
│
├── "How are variables related?" (Relationships)
│   ├── 2 numeric vars → correlation_heatmap.png
│   ├── 2-6 numeric vars → pairplot.png (scatter matrix)
│   ├── Numeric vs Categorical → violin_plot.png
│   └── All numeric → correlation_heatmap.png
│
└── "Can I predict X from Y?" (Predictive)
    └── correlation_heatmap.png → |r| > 0.5 suggests predictive power

How to Interpret Results (For Claude)

Quality Score Interpretation

Score	Grade	What to Tell User
90-100	A	"Your data is excellent quality - ready for analysis"
80-89	B	"Good quality data with minor issues worth noting"
70-79	C	"Moderate quality - address missing values before critical analysis"
60-69	D	"Significant quality issues - recommend data cleaning first"
<60	F	"Critical issues - data needs substantial cleaning"

Correlation Interpretation

|r| Value	Strength	What to Say
0.9 - 1.0	Very Strong	"X and Y are very strongly related - almost deterministic"
0.7 - 0.9	Strong	"X and Y have a strong relationship - X could help predict Y"
0.5 - 0.7	Moderate	"X and Y are moderately correlated - some predictive value"
0.3 - 0.5	Weak	"X and Y have a weak relationship - limited predictive power"
0.0 - 0.3	Negligible	"X and Y appear unrelated"

Sign matters:

• Positive: "As X increases, Y tends to increase"
• Negative: "As X increases, Y tends to decrease"

Skewness Interpretation

Skewness	Distribution Shape	Recommendation
< -1	Heavy left tail	"Most values are high, with some very low outliers"
-1 to -0.5	Mild left skew	"Slightly more low outliers than high"
-0.5 to 0.5	Symmetric	"Nicely balanced distribution - good for most analyses"
0.5 to 1	Mild right skew	"Slightly more high outliers than low"
> 1	Heavy right tail	"Most values are low, with some very high outliers. Consider log transform for modeling."

Outlier Assessment

When reporting outliers:

• Few outliers (<1%): "A few extreme values that may warrant investigation"
• Moderate outliers (1-5%): "Notable outliers - check if they're errors or genuine extremes"
• Many outliers (>5%): "High outlier rate suggests either data issues or a non-normal distribution"

Insight Generation Framework

After running analysis, provide insights in this order:

1. Data Overview (Always)

"Your dataset has [rows] records and [cols] columns:
- [n] numeric columns: [list top 3]
- [n] categorical columns: [list top 3]
- Data quality score: [score]/100 ([grade])"

2. Key Findings (Pick most relevant)

If quality issues exist:

"I noticed some data quality concerns:
- [X]% missing values in [column] - [recommend: drop/impute/investigate]
- [N] duplicate rows detected - [recommend: keep first/remove all/investigate]"

If strong correlations found:

"Interesting relationships I found:
- [col1] and [col2] are strongly correlated (r=[value]) - [interpretation]
- This suggests [actionable insight]"

If outliers detected:

"I detected outliers in [columns]:
- [column]: [n] values beyond normal range ([min outlier] to [max outlier])
- These could be [data errors / genuine extremes / worth investigating]"

If skewed distributions:

"[Column] has a [right/left]-skewed distribution:
- Most values cluster around [median]
- But there are extreme values up to [max]
- For modeling, consider [log transform / robust methods]"

3. Recommendations (Based on findings)

Finding	Recommendation
Missing >20% in column	"Consider dropping this column or investigating why it's missing"
Missing <5% scattered	"Safe to impute with median (numeric) or mode (categorical)"
High correlation (>0.9)	"These columns may be redundant - consider keeping only one"
Many outliers	"Use robust statistics (median instead of mean) or investigate data collection"
Highly skewed	"Apply log transform before linear modeling"
Low quality score	"Prioritize data cleaning before analysis"

Multi-Chart Dashboard Requests

When user asks for a "dashboard" or "comprehensive view":

# Generate all visualizations
python3 analyze_csv.py data.csv --format html --max-charts 10

Then present charts in this order:

1. overview_dashboard.png - "Here's your data at a glance"
2. correlation_heatmap.png - "Key relationships between variables"
3. numeric_distributions.png - "How your numeric data is distributed"
4. box_plots.png - "Outlier analysis"
5. categorical_distributions.png - "Category breakdowns" (if applicable)

Command Reference

Basic Analysis

python3 analyze_csv.py data.csv

Full Report with All Charts

python3 analyze_csv.py data.csv --format markdown --max-charts 10

Quick Analysis (No Charts)

python3 analyze_csv.py data.csv --no-charts

Large Files (>100MB)

python3 analyze_csv.py huge.csv --sample 50000

Specific Date Columns

python3 analyze_csv.py data.csv --date-columns created_at updated_at

JSON for Programmatic Use

python3 analyze_csv.py data.csv --format json --no-charts

Custom Output Location

python3 analyze_csv.py data.csv --output-dir /path/to/project/.tmp/analysis

Chart Descriptions (For Explaining to Users)

Chart	When to Show	How to Describe
overview_dashboard.png	Always for first look	"Here's a bird's eye view of your data"
missing_values.png	If missing data exists	"This shows where your data has gaps"
numeric_distributions.png	When exploring distributions	"This shows how your numeric values are spread out"
box_plots.png	When checking for outliers	"The dots outside the boxes are potential outliers"
correlation_heatmap.png	When exploring relationships	"Darker colors = stronger relationships"
categorical_distributions.png	For category analysis	"This shows the breakdown of your categories"
time_series.png	For temporal data	"Here's how your data changes over time"
pairplot.png	For multivariate exploration	"Each cell shows how two variables relate"
violin_plot.png	Comparing groups	"This shows how distributions differ across groups"

Common User Questions → Actions

User Says	Action
"Analyze this CSV"	Run full analysis, show overview + key insights
"Is my data clean?"	Focus on quality_score, missing values, duplicates
"Find patterns"	Show correlation_heatmap, highlight strong correlations
"Are there outliers?"	Show box_plots, list outlier counts per column
"Compare X across Y"	Generate violin_plot for numeric X vs categorical Y
"Show me trends"	Generate time_series if datetime column exists
"Create a dashboard"	Generate all charts, present organized summary
"What should I clean?"	List columns with missing >5%, duplicates, outliers

Output Locations

Charts are saved to:

• Default: ~/.claude/skills/csv-analyzer/scripts/.tmp/csv_analysis/
• Custom: Use --output-dir /path/to/project/.tmp/analysis

Always copy charts to user's project .tmp for visibility:

cp ~/.claude/skills/csv-analyzer/scripts/.tmp/csv_analysis/*.png /path/to/project/.tmp/csv_analysis/

Cost

Free - runs entirely locally using pandas, matplotlib, seaborn, scipy.

Dependencies

pip install pandas matplotlib seaborn scipy numpy