scikit-learn-machine-learning

scikit-learn

概述

scikit-learn 是用于经典机器学习的标准 Python 库。它提供一致的 API，支持监督学习（分类、回归）、无监督学习（聚类、降维）、模型评估和预处理，并可无缝集成到 NumPy/pandas 工作流中。

适用场景

• 使用带标签的数据构建分类模型（垃圾邮件检测、疾病诊断、物种识别）
• 使用回归预测连续结果（价格预测、剂量-反应建模）
• 将无标签数据聚类成不同组别（患者分层、基因表达聚类）
• 降低数据维度以便可视化或进行特征工程（对多组学数据使用 PCA、t-SNE）
• 使用交叉验证评估和比较模型性能
• 系统地调整超参数（网格搜索、随机搜索）
• 通过预处理和建模步骤构建可复现的 ML 管道
• 对于深度学习任务（图像、NLP），请改用 pytorch 或 transformers
• 对于大规模梯度提升，请改用 xgboost 或 lightgbm

前置条件

• Python 包：scikit-learn、numpy、pandas
• 可选：使用 matplotlib、seaborn 进行可视化
• 数据：以 NumPy 数组或 pandas DataFrame 表示的表格数据

pip install scikit-learn numpy pandas matplotlib seaborn

快速入门

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report
from sklearn.datasets import load_breast_cancer

# Load dataset, split, train, evaluate in 10 lines
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

clf = RandomForestClassifier(n_estimators=100, random_state=42)
clf.fit(X_train, y_train)
y_pred = clf.predict(X_test)

print(f"Accuracy: {accuracy_score(y_test, y_pred):.3f}")
print(classification_report(y_test, y_pred, target_names=["malignant", "benign"]))

核心 API

模块 1：数据预处理

缩放、编码、插补和特征工程。

from sklearn.preprocessing import StandardScaler, MinMaxScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
import numpy as np

# Scaling: zero mean, unit variance
X = np.array([[1, 2], [3, 4], [5, 6]])
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
print(f"Mean: {X_scaled.mean(axis=0)}, Std: {X_scaled.std(axis=0)}")
# Mean: [0. 0.], Std: [1. 1.]

# Imputation: fill missing values
X_missing = np.array([[1, np.nan], [3, 4], [np.nan, 6]])
imputer = SimpleImputer(strategy="median")
X_filled = imputer.fit_transform(X_missing)
print(f"Filled:\n{X_filled}")

from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder, LabelEncoder

# One-hot encoding for nominal categories
enc = OneHotEncoder(sparse_output=False, handle_unknown="ignore")
X_cat = np.array([["red"], ["blue"], ["green"], ["red"]])
X_encoded = enc.fit_transform(X_cat)
print(f"Categories: {enc.categories_}")
print(f"Encoded shape: {X_encoded.shape}")  # (4, 3)

模块 2：监督学习 — 分类

用于预测离散目标的分类器。

from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)

# Compare classifiers
classifiers = {
    "LogisticRegression": LogisticRegression(max_iter=200),
    "RandomForest": RandomForestClassifier(n_estimators=100, random_state=42),
    "SVM": SVC(kernel="rbf", C=1.0),
    "GradientBoosting": GradientBoostingClassifier(n_estimators=100, random_state=42),
}
for name, clf in classifiers.items():
    clf.fit(X_train, y_train)
    print(f"{name}: accuracy = {clf.score(X_test, y_test):.3f}")

模块 3：监督学习 — 回归

用于预测连续目标的回归器。

from sklearn.linear_model import LinearRegression, Ridge, Lasso, ElasticNet
from sklearn.ensemble import RandomForestRegressor
from sklearn.datasets import make_regression
from sklearn.metrics import mean_squared_error, r2_score

X, y = make_regression(n_samples=200, n_features=10, noise=10, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

models = {
    "Linear": LinearRegression(),
    "Ridge": Ridge(alpha=1.0),
    "Lasso": Lasso(alpha=0.1),
    "RandomForest": RandomForestRegressor(n_estimators=100, random_state=42),
}
for name, model in models.items():
    model.fit(X_train, y_train)
    y_pred = model.predict(X_test)
    print(f"{name}: RMSE={mean_squared_error(y_test, y_pred, squared=False):.2f}, R²={r2_score(y_test, y_pred):.3f}")

模块 4：无监督学习 — 聚类

用于无标签数据的聚类算法。

from sklearn.cluster import KMeans, DBSCAN, AgglomerativeClustering
from sklearn.metrics import silhouette_score
from sklearn.datasets import make_blobs

X, y_true = make_blobs(n_samples=300, centers=4, random_state=42)

# K-Means with elbow method
for k in [2, 3, 4, 5, 6]:
    km = KMeans(n_clusters=k, random_state=42, n_init=10)
    labels = km.fit_predict(X)
    sil = silhouette_score(X, labels)
    print(f"k={k}: silhouette={sil:.3f}, inertia={km.inertia_:.1f}")

# DBSCAN — no need to specify k
from sklearn.cluster import DBSCAN

db = DBSCAN(eps=0.5, min_samples=5)
labels = db.fit_predict(X)
n_clusters = len(set(labels)) - (1 if -1 in labels else 0)
n_noise = (labels == -1).sum()
print(f"DBSCAN: {n_clusters} clusters, {n_noise} noise points")

模块 5：降维

用于可视化和特征降维的 PCA、t-SNE 及其他方法。

from sklearn.decomposition import PCA
from sklearn.manifold import TSNE
from sklearn.datasets import load_digits

X, y = load_digits(return_X_y=True)
print(f"Original shape: {X.shape}")  # (1797, 64)

# PCA — preserve 95% variance
pca = PCA(n_components=0.95)
X_pca = pca.fit_transform(X)
print(f"PCA: {X_pca.shape[1]} components, explained variance: {pca.explained_variance_ratio_.sum():.3f}")

# t-SNE — 2D visualization
tsne = TSNE(n_components=2, perplexity=30, random_state=42)
X_tsne = tsne.fit_transform(X)
print(f"t-SNE shape: {X_tsne.shape}")  # (1797, 2)

模块 6：模型评估与选择

交叉验证、指标、超参数调优。

from sklearn.model_selection import cross_val_score, GridSearchCV, StratifiedKFold
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report, confusion_matrix
from sklearn.datasets import load_iris

X, y = load_iris(return_X_y=True)

# Cross-validation
clf = RandomForestClassifier(n_estimators=100, random_state=42)
scores = cross_val_score(clf, X, y, cv=StratifiedKFold(5), scoring="accuracy")
print(f"CV accuracy: {scores.mean():.3f} ± {scores.std():.3f}")

# Hyperparameter tuning with GridSearchCV
param_grid = {
    "n_estimators": [50, 100, 200],
    "max_depth": [5, 10, None],
    "min_samples_split": [2, 5]
}
grid = GridSearchCV(
    RandomForestClassifier(random_state=42),
    param_grid, cv=5, scoring="accuracy", n_jobs=-1
)
grid.fit(X, y)
print(f"Best params: {grid.best_params_}")
print(f"Best score: {grid.best_score_:.3f}")

模块 7：管道

串联预处理和模型；防止数据泄漏。

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.ensemble import GradientBoostingClassifier

# Mixed-type preprocessing
numeric_features = ["age", "income"]
categorical_features = ["gender", "occupation"]

preprocessor = ColumnTransformer([
    ("num", Pipeline([
        ("imputer", SimpleImputer(strategy="median")),
        ("scaler", StandardScaler())
    ]), numeric_features),
    ("cat", Pipeline([
        ("imputer", SimpleImputer(strategy="most_frequent")),
        ("onehot", OneHotEncoder(handle_unknown="ignore"))
    ]), categorical_features),
])

pipe = Pipeline([
    ("preprocessor", preprocessor),
    ("classifier", GradientBoostingClassifier(random_state=42))
])
# pipe.fit(X_train, y_train); pipe.predict(X_test)
print("Pipeline steps:", [name for name, _ in pipe.steps])

常见工作流

工作流 1：端到端分类

目标：完成从数据加载到评估的完整分类工作流。

import pandas as pd
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report
from sklearn.datasets import load_breast_cancer

# Load data
X, y = load_breast_cancer(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)

# Build pipeline
pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("clf", RandomForestClassifier(n_estimators=200, random_state=42))
])

# Cross-validate
cv_scores = cross_val_score(pipe, X_train, y_train, cv=5, scoring="f1")
print(f"CV F1: {cv_scores.mean():.3f} ± {cv_scores.std():.3f}")

# Final evaluation
pipe.fit(X_train, y_train)
y_pred = pipe.predict(X_test)
print(classification_report(y_test, y_pred))

工作流 2：可视化聚类

目标：对数据进行聚类，并通过降维进行可视化。

from sklearn.datasets import make_blobs
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
from sklearn.decomposition import PCA
from sklearn.metrics import silhouette_score
import matplotlib.pyplot as plt

# Generate and scale data
X, _ = make_blobs(n_samples=500, centers=4, random_state=42)
X_scaled = StandardScaler().fit_transform(X)

# Cluster
km = KMeans(n_clusters=4, random_state=42, n_init=10)
labels = km.fit_predict(X_scaled)
print(f"Silhouette: {silhouette_score(X_scaled, labels):.3f}")

# Visualize
X_2d = PCA(n_components=2).fit_transform(X_scaled)
plt.scatter(X_2d[:, 0], X_2d[:, 1], c=labels, cmap="viridis", s=20, alpha=0.7)
plt.title("K-Means Clustering (PCA projection)")
plt.savefig("clustering_result.png", dpi=150, bbox_inches="tight")
print("Saved clustering_result.png")

工作流 3：特征选择 + 模型管道

目标：选择最佳特征并构建经过调优的模型。

from sklearn.datasets import make_classification
from sklearn.feature_selection import SelectKBest, f_classif
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV

X, y = make_classification(n_samples=500, n_features=50, n_informative=10, random_state=42)

pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("selector", SelectKBest(f_classif)),
    ("svm", SVC(kernel="rbf"))
])

param_grid = {
    "selector__k": [5, 10, 20],
    "svm__C": [0.1, 1, 10],
    "svm__gamma": ["scale", "auto"]
}

grid = GridSearchCV(pipe, param_grid, cv=5, scoring="accuracy", n_jobs=-1)
grid.fit(X, y)
print(f"Best params: {grid.best_params_}")
print(f"Best accuracy: {grid.best_score_:.3f}")

关键参数

参数	模块	默认值	范围 / 选项	作用
n_estimators	RandomForest, GradientBoosting	100	50-1000	树的数量；数量越多，效果越好但速度越慢
max_depth	基于树的模型	None	1-50、None	树的深度；None = 无限制（可能过拟合）
C	SVM, LogisticRegression	1.0	0.001-1000	正则化强度（倒数）；值越低，正则化越强
alpha	Ridge, Lasso	1.0	0.001-100	正则化强度；值越高，正则化越强
n_clusters	KMeans	必填	2-N	要形成的簇数
eps	DBSCAN	0.5	0.01-10	邻域半径；值越小，簇越多
n_components	PCA	必填	1-N 或 0.0-1.0	保留的成分数；浮点数表示方差比例
perplexity	t-SNE	30	5-50	平衡局部与全局结构
cv	GridSearchCV	5	2-10	交叉验证折数
scoring	GridSearchCV, cross_val_score	不固定	accuracy、f1、roc_auc 等	评估指标

常见方案

方案：特征重要性分析

适用场景：了解哪些特征影响模型预测。

import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris

X, y = load_iris(return_X_y=True)
clf = RandomForestClassifier(n_estimators=200, random_state=42).fit(X, y)

importances = clf.feature_importances_
indices = np.argsort(importances)[::-1]
feature_names = load_iris().feature_names
for i in range(X.shape[1]):
    print(f"{feature_names[indices[i]]}: {importances[indices[i]]:.4f}")

方案：学习曲线诊断

适用场景：诊断过拟合与欠拟合。

from sklearn.model_selection import learning_curve
import matplotlib.pyplot as plt
import numpy as np

train_sizes, train_scores, val_scores = learning_curve(
    clf, X, y, cv=5, train_sizes=np.linspace(0.1, 1.0, 10), scoring="accuracy"
)
plt.plot(train_sizes, train_scores.mean(axis=1), label="Train")
plt.plot(train_sizes, val_scores.mean(axis=1), label="Validation")
plt.xlabel("Training size"); plt.ylabel("Accuracy"); plt.legend()
plt.savefig("learning_curve.png", dpi=150, bbox_inches="tight")
print("Saved learning_curve.png")

方案：保存和加载模型

适用场景：持久化训练好的模型，以便日后使用。

import joblib

# Save
joblib.dump(pipe, "model_pipeline.joblib")
print("Model saved to model_pipeline.joblib")

# Load
loaded_pipe = joblib.load("model_pipeline.joblib")
y_pred = loaded_pipe.predict(X_test)
print(f"Loaded model predictions: {y_pred[:5]}")

故障排除

问题	原因	解决方案
ConvergenceWarning	模型未收敛	增大 max_iter（例如，设为 1000），或使用 StandardScaler 缩放特征
训练准确率高，测试准确率低	过拟合	添加正则化、减小 max_depth，并使用交叉验证
ValueError: unknown categories	测试数据中出现新类别	使用 OneHotEncoder(handle_unknown='ignore')
大数据集出现 MemoryError	整个数据集都载入了内存	使用 SGDClassifier/MiniBatchKMeans 进行增量学习
聚类效果不佳	特征未缩放或 k 值不合适	先缩放特征；使用轮廓系数寻找最佳 k 值
NotFittedError	拟合前就进行预测	先调用 model.fit(X_train, y_train)
每次运行的结果都不同	缺少 random_state	在模型和 train_test_split 中设置 random_state=42
GridSearchCV 速度较慢	参数网格过大	使用 RandomizedSearchCV 或 HalvingGridSearchCV；添加 n_jobs=-1

参考资料

• scikit-learn 用户指南 — 官方文档
• scikit-learn API 参考 — 完整 API
• scikit-learn 示例集 — 教程
• Pedregosa 等（2011）。Scikit-learn：Python 中的机器学习。JMLR 12:2825-2830。