mcpField
M
model-comparison-tool

model-comparison-tool

github.com/dkyazzentwatwaData & databases12454 installsUpdated 29w ago
Repository

About model-comparison-tool

A model-comparison skill for data scientists and practitioners evaluating machine-learning models. It combines cross-validation, metrics, statistical tests, visualizations, and criteria-based model recommendations.

Key features

  1. 01Compare multiple algorithms
  2. 02K-fold, stratified, and time-series validation
  3. 03Classification and regression metrics
  4. 04Paired t-test significance checks
  5. 05Performance and learning visualizations
  6. 06Criteria-based model recommendations

Use cases

  1. 01Benchmark algorithms for a task
  2. 02Compare hyperparameter configurations
  3. 03Evaluate feature engineering choices
  4. 04Select a production model

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/dkyazzentwatwa/chatgpt-skills --skill model-comparison-tool

After installing, type/model-comparison-toolin Claude Code to confirm it loaded.

Bundled filesscripts/model_comparison_tool.pyscripts/requirements.txt

Details

From the project README

Model Comparison Tool

Compare multiple machine learning models systematically with cross-validation, metric evaluation, and automated model selection.

Purpose

Model comparison for:

  • Algorithm selection and benchmarking
  • Hyperparameter tuning comparison
  • Model performance validation
  • Feature engineering evaluation
  • Production model selection

Features

  • Multi-Model Comparison: Test 5+ algorithms simultaneously
  • Cross-Validation: K-fold, stratified, time-series splits
  • Comprehensive Metrics: Accuracy, F1, ROC-AUC, RMSE, MAE, R²
  • Statistical Testing: Paired t-tests for significance
  • Visualization: Performance charts, ROC curves, learning curves
  • Auto-Selection: Recommend best model based on criteria

Quick Start

from model_comparison_tool import ModelComparisonTool

# Compare classifiers
comparator = ModelComparisonTool()
comparator.load_data(X_train, y_train, task='classification')

results = comparator.compare_models(
    models=['rf', 'gb', 'lr', 'svm'],
    cv_folds=5
)

best_model = comparator.get_best_model(metric='f1')

CLI Usage

# Compare models on CSV data
python model_comparison_tool.py --data data.csv --target target --task classification

# Custom model comparison
python model_comparison_tool.py --data data.csv --target price --task regression --models rf,gb,lr --cv 10

# Export results
python model_comparison_tool.py --data data.csv --target y --output comparison_report.html

Limitations

  • Requires sufficient data for meaningful cross-validation
  • Large datasets may have long comparison times
  • Deep learning models not included (use dedicated frameworks)
  • Feature engineering must be done beforehand

Comments

Markdown supported