matchms-spectral-matching

Matchms——质谱匹配与代谢物鉴定

概述

Matchms 是一个用于质谱数据处理的 Python 库，专注于谱图相似度计算和化合物鉴定。它支持多种格式的输入/输出，提供 50 多种用于元数据协调和峰处理的谱图过滤器、8 种相似度评分函数，以及一个用于构建可复现分析工作流的流水线框架。

适用场景

• 通过将未知代谢物的 MS/MS 谱图与参考库匹配来进行鉴定
• 计算谱图相似度分数（余弦、修正余弦、基于指纹）
• 处理和标准化多种格式（mzML、MGF、MSP、JSON）的质谱数据
• 构建可复现的谱图处理流水线，用于质量控制
• 协调不同谱图数据库中的元数据（化合物名称、SMILES、InChI、加合物）
• 大规模谱图库比较和重复项检测
• 对于完整的 LC-MS/MS 蛋白质组学工作流（特征检测、蛋白质 ID），请改用 pyopenms
• 对于不依赖质谱的化学结构相似度计算，请使用 rdkit 指纹比较

前置条件

uv pip install matchms numpy pandas
# For chemical structure processing (SMILES, InChI, fingerprints):
uv pip install matchms[chemistry]

• Python 3.8+；峰数组操作需要 NumPy
• 输入：MGF、MSP、mzML、mzXML、JSON 或 pickle 格式的谱图数据
• 用于匹配的参考库，格式不限，但必须受支持

事前访谈

编写任何分析代码前，请先与用户确认以下事项。

decisions:
  - id: D1
    param: referenceLibrary
    kind: required
    source: user
    ask: "Which spectral library should queries be matched against?"
    default: null

  - id: D2
    param: similarityMeasure
    kind: required
    source: user
    ask: "Should matching require the same precursor mass, or allow a mass shift so modified analogues still match?"
    default: "cosine on matched peaks, precursor mass fixed"

  - id: D3
    param: peakMatchTolerance
    kind: required
    source: data
    ask: "How close must two peaks be in m/z to count as the same fragment?"
    default: "0.1 Da - tighten toward 0.005 for high-resolution data"

  - id: D4
    param: spectrumPreprocessing
    kind: required
    source: user
    ask: "Which peaks should be discarded before scoring - low-intensity noise, peaks near the precursor, or spectra with too few peaks to be informative?"
    default: "drop peaks under 1% relative intensity, require at least 10 peaks"

  - id: D5
    param: scoreCutoff
    kind: required
    source: user
    ask: "What similarity score, and how many matched peaks, should a hit need before it is reported as an identification?"
    default: null

  - id: D6
    param: peakWeighting
    kind: optional
    source: user
    ask: "Should the score weight fragment m/z as well as intensity, favouring informative high-mass fragments?"
    default: "intensity only"

  - id: D7
    param: fingerprintSettings
    kind: optional_conditional
    source: user
    ask: "Which molecular fingerprint should structural similarity use, when comparing hits to known structures?"
    default: "not computed"

D3 和 D5 没有安全的通用默认值，因为二者直接相互制约：较宽松的容差搭配较低的分数阈值，会为每个查询谱图都返回一个鉴定结果，而这些结果看起来都似乎合理。请根据仪器的实际精度设置容差，并根据参考库的支持情况设置阈值。

快速入门

from matchms.importing import load_from_mgf
from matchms.filtering import default_filters, normalize_intensities
from matchms.filtering import select_by_relative_intensity, require_minimum_number_of_peaks
from matchms import calculate_scores
from matchms.similarity import CosineGreedy

# Load and process query spectra
queries = list(load_from_mgf("queries.mgf"))
queries = [default_filters(s) for s in queries]
queries = [normalize_intensities(s) for s in queries if s is not None]
queries = [require_minimum_number_of_peaks(s, n_required=5) for s in queries if s is not None]

# Load reference library
refs = list(load_from_mgf("library.mgf"))
refs = [default_filters(s) for s in refs]
refs = [normalize_intensities(s) for s in refs if s is not None]

# Calculate similarity scores
scores = calculate_scores(references=refs, queries=queries,
                          similarity_function=CosineGreedy(tolerance=0.1))
# Get best matches for first query
best = scores.scores_by_query(queries[0], sort=True)[:5]
for match, score_tuple in best:
    print(f"Score: {score_tuple['score']:.3f}, Matches: {score_tuple['matches']}")

核心 API

模块 1：谱图输入/输出

从多种文件格式导入谱图，并导出处理后的数据。

from matchms.importing import (load_from_mgf, load_from_mzml, load_from_msp,
                                load_from_json, load_from_mzxml, load_from_pickle,
                                load_from_usi)
from matchms.exporting import save_as_mgf, save_as_msp, save_as_json, save_as_pickle

# Import from various formats (returns generators)
spectra_mgf = list(load_from_mgf("library.mgf"))
spectra_mzml = list(load_from_mzml("data.mzML"))
spectra_msp = list(load_from_msp("nist_library.msp"))
spectra_json = list(load_from_json("gnps_spectra.json"))
print(f"Loaded: MGF={len(spectra_mgf)}, mzML={len(spectra_mzml)}")

# Export processed spectra
save_as_mgf(spectra_mgf, "processed.mgf")
save_as_json(spectra_mgf, "processed.json")
save_as_pickle(spectra_mgf, "spectra.pickle")  # Fast for intermediate results

# Pickle for large datasets (fastest I/O)
from matchms.importing import load_from_pickle
spectra = list(load_from_pickle("spectra.pickle"))

from matchms import Spectrum
import numpy as np

# Create spectrum manually
mz = np.array([100.0, 150.0, 200.0, 250.0, 300.0])
intensities = np.array([0.1, 0.5, 0.9, 0.3, 0.7])
metadata = {
    "precursor_mz": 325.5,
    "ionmode": "positive",
    "compound_name": "Caffeine",
    "smiles": "CN1C=NC2=C1C(=O)N(C(=O)N2C)C"
}
spectrum = Spectrum(mz=mz, intensities=intensities, metadata=metadata)

# Access spectrum data
print(f"Peaks: {spectrum.peaks.mz}")
print(f"Precursor: {spectrum.get('precursor_mz')}")
print(f"Name: {spectrum.get('compound_name')}")

# Visualize
spectrum.plot()

模块 2：谱图过滤与处理

应用元数据协调和峰处理过滤器。Matchms 提供 50 多种过滤器。

from matchms.filtering import (
    default_filters, normalize_intensities,
    select_by_relative_intensity, select_by_mz,
    require_minimum_number_of_peaks, reduce_to_number_of_peaks,
    remove_peaks_around_precursor_mz, add_losses
)

# default_filters applies: metadata cleanup, charge correction, adduct parsing
spectrum = default_filters(spectrum)

# Peak normalization (max intensity → 1.0)
spectrum = normalize_intensities(spectrum)

# Filter peaks by relative intensity (remove noise below 1%)
spectrum = select_by_relative_intensity(spectrum, intensity_from=0.01, intensity_to=1.0)

# Filter peaks by m/z range
spectrum = select_by_mz(spectrum, mz_from=50.0, mz_to=500.0)

# Keep top N peaks only
spectrum = reduce_to_number_of_peaks(spectrum, n_max=50)

# Remove peaks near precursor (common contaminants)
spectrum = remove_peaks_around_precursor_mz(spectrum, mz_tolerance=17.0)

# Require minimum peaks for matching
spectrum = require_minimum_number_of_peaks(spectrum, n_required=5)

# Add neutral losses (useful for NeutralLossesCosine)
spectrum = add_losses(spectrum)

if spectrum is not None:
    print(f"After filtering: {len(spectrum.peaks.mz)} peaks")

# Chemical annotation filters (require matchms[chemistry])
from matchms.filtering import (
    derive_inchi_from_smiles, derive_inchikey_from_inchi,
    derive_smiles_from_inchi, add_fingerprint,
    repair_inchi_inchikey_smiles, require_valid_annotation
)

# Derive chemical identifiers from SMILES
spectrum = derive_inchi_from_smiles(spectrum)
spectrum = derive_inchikey_from_inchi(spectrum)

# Add molecular fingerprint for structural similarity
spectrum = add_fingerprint(spectrum, fingerprint_type="morgan", nbits=2048)

# Validate annotations
spectrum = require_valid_annotation(spectrum)
if spectrum is not None:
    print(f"InChIKey: {spectrum.get('inchikey')}")

模块 3：相似度评分

使用多种相似度指标比较谱图。

from matchms import calculate_scores
from matchms.similarity import (
    CosineGreedy, CosineHungarian, ModifiedCosine,
    NeutralLossesCosine, FingerprintSimilarity,
    MetadataMatch, PrecursorMzMatch
)

# CosineGreedy — fast peak matching (greedy algorithm)
scores = calculate_scores(references=library, queries=unknowns,
                          similarity_function=CosineGreedy(tolerance=0.1))

# ModifiedCosine — accounts for precursor mass differences (best for analog search)
scores = calculate_scores(references=library, queries=unknowns,
                          similarity_function=ModifiedCosine(tolerance=0.1))

# CosineHungarian — optimal peak matching (slower but more accurate)
scores = calculate_scores(references=library, queries=unknowns,
                          similarity_function=CosineHungarian(tolerance=0.1))

# NeutralLossesCosine — similarity based on neutral loss patterns
scores = calculate_scores(references=library, queries=unknowns,
                          similarity_function=NeutralLossesCosine(tolerance=0.1))

# Access results
for i, query in enumerate(unknowns[:3]):
    best_matches = scores.scores_by_query(query, sort=True)[:3]
    print(f"\nQuery {i}: precursor_mz={query.get('precursor_mz')}")
    for ref, score_tuple in best_matches:
        print(f"  {ref.get('compound_name', 'Unknown')}: "
              f"score={score_tuple['score']:.3f}, matches={score_tuple['matches']}")

# FingerprintSimilarity — structural similarity (requires fingerprints)
from matchms.similarity import FingerprintSimilarity
scores = calculate_scores(references=library, queries=unknowns,
                          similarity_function=FingerprintSimilarity(
                              similarity_measure="jaccard"))

# PrecursorMzMatch — fast mass-based pre-filtering
from matchms.similarity import PrecursorMzMatch
scores = calculate_scores(references=library, queries=unknowns,
                          similarity_function=PrecursorMzMatch(tolerance=0.1))

# Multi-metric scoring: combine peak + structural similarity
cosine_scores = calculate_scores(references=library, queries=unknowns,
                                  similarity_function=CosineGreedy(tolerance=0.1))
fp_scores = calculate_scores(references=library, queries=unknowns,
                              similarity_function=FingerprintSimilarity())

模块 4：处理流水线

构建可复用、可复现的多步骤处理工作流。

from matchms import SpectrumProcessor
from matchms.filtering import (
    default_filters, normalize_intensities,
    select_by_relative_intensity, require_minimum_number_of_peaks,
    remove_peaks_around_precursor_mz, add_losses
)

# Define reusable pipeline
pipeline = SpectrumProcessor([
    default_filters,
    normalize_intensities,
    lambda s: select_by_relative_intensity(s, intensity_from=0.01),
    lambda s: remove_peaks_around_precursor_mz(s, mz_tolerance=17.0),
    lambda s: require_minimum_number_of_peaks(s, n_required=5),
    add_losses
])

# Apply to all spectra (filters returning None remove the spectrum)
processed = [pipeline(s) for s in raw_spectra]
processed = [s for s in processed if s is not None]
print(f"Processed: {len(processed)}/{len(raw_spectra)} spectra retained")

核心概念

相似度函数比较

函数	速度	准确度	适用场景
CosineGreedy	快	良好	通用谱图库匹配
CosineHungarian	慢	最佳	小规模比较、验证
ModifiedCosine	快	良好	类似物搜索（前体离子不同）
NeutralLossesCosine	中等	良好	结构类别鉴定
FingerprintSimilarity	快	中等	基于结构的预筛选
PrecursorMzMatch	最快	不适用	基于质量的预筛选

过滤器类别

类别	示例	用途
元数据清理	default_filters、clean_compound_name、clean_adduct	标准化元数据字段
化学信息推导	derive_inchi_from_smiles、add_fingerprint	计算化学标识符
质量/电荷	add_precursor_mz、correct_charge、add_parent_mass	修正并验证质量信息
峰归一化	normalize_intensities、select_by_relative_intensity	缩放和筛选峰
峰缩减	reduce_to_number_of_peaks、remove_peaks_around_precursor_mz	移除噪声/伪影
质量控制	require_minimum_number_of_peaks、require_precursor_mz	确保达到最低质量要求
中性丢失	add_losses	计算前体离子与碎片离子之间的质量差

分数元组结构

所有相似度函数均返回 (score, matches)：

• score：浮点数 0.0–1.0（余弦相似度值）
• matches：整数（查询谱图与参考谱图之间匹配的峰数）

分数越高且匹配峰越多，表示匹配效果越好。常见阈值：分数 > 0.7 且匹配峰数 > 6，可视为鉴定结果可靠。

常见工作流

工作流 1：代谢物鉴定的谱库匹配

from matchms.importing import load_from_mgf
from matchms.filtering import default_filters, normalize_intensities
from matchms.filtering import select_by_relative_intensity, require_minimum_number_of_peaks
from matchms import calculate_scores
from matchms.similarity import ModifiedCosine
import pandas as pd

# Load and process both queries and library identically
def process_spectra(spectra):
    processed = []
    for s in spectra:
        s = default_filters(s)
        if s is None: continue
        s = normalize_intensities(s)
        s = select_by_relative_intensity(s, intensity_from=0.01)
        s = require_minimum_number_of_peaks(s, n_required=5)
        if s is not None:
            processed.append(s)
    return processed

queries = process_spectra(load_from_mgf("unknowns.mgf"))
library = process_spectra(load_from_mgf("reference_library.mgf"))
print(f"Queries: {len(queries)}, Library: {len(library)}")

# Score all query-reference pairs
scores = calculate_scores(references=library, queries=queries,
                          similarity_function=ModifiedCosine(tolerance=0.1))

# Extract best matches
results = []
for query in queries:
    best = scores.scores_by_query(query, sort=True)[:1]
    if best:
        ref, score_tuple = best[0]
        results.append({
            "query_precursor_mz": query.get("precursor_mz"),
            "match_name": ref.get("compound_name", "Unknown"),
            "match_smiles": ref.get("smiles", ""),
            "score": score_tuple["score"],
            "matched_peaks": score_tuple["matches"]
        })

df = pd.DataFrame(results)
confident = df[df["score"] > 0.7]
print(f"Confident matches (score>0.7): {len(confident)}/{len(df)}")
df.to_csv("identification_results.csv", index=False)

工作流 2：质量控制与数据清理

from matchms.importing import load_from_msp
from matchms.exporting import save_as_mgf
from matchms import SpectrumProcessor
from matchms.filtering import (
    default_filters, normalize_intensities,
    select_by_relative_intensity, require_minimum_number_of_peaks,
    require_precursor_mz, add_parent_mass
)

# Define QC pipeline
qc_pipeline = SpectrumProcessor([
    default_filters,
    require_precursor_mz,
    add_parent_mass,
    normalize_intensities,
    lambda s: select_by_relative_intensity(s, intensity_from=0.001),
    lambda s: require_minimum_number_of_peaks(s, n_required=3)
])

# Process and filter
raw = list(load_from_msp("raw_library.msp"))
cleaned = [qc_pipeline(s) for s in raw]
cleaned = [s for s in cleaned if s is not None]

print(f"Input: {len(raw)}, Output: {len(cleaned)} ({len(cleaned)/len(raw)*100:.1f}% retained)")
save_as_mgf(cleaned, "cleaned_library.mgf")

工作流 3：格式转换

1. 从源格式加载谱图（Core API Module 1 — load_from_mzml）
2. 应用 default_filters 以统一元数据（Core API Module 2）
3. 导出为目标格式（Core API Module 1 — save_as_mgf）

关键参数

参数	函数/模块	默认值	范围/选项	作用
tolerance	CosineGreedy/ModifiedCosine	0.1	0.005–0.5 Da	峰匹配的 m/z 容差
mz_power	CosineGreedy	0.0	0.0–2.0	评分中 m/z 的权重（0=忽略）
intensity_power	CosineGreedy	1.0	0.0–2.0	评分中强度的权重
intensity_from	select_by_relative_intensity	0.0	0.0–1.0	保留峰的最低相对强度
n_required	require_minimum_number_of_peaks	10	1–100	谱图保留的最少峰数
n_max	reduce_to_number_of_peaks	100	10–500	谱图保留的最多峰数
mz_tolerance	remove_peaks_around_precursor_mz	17.0	0.5–50 Da	前体离子周围要移除的窗口范围
fingerprint_type	add_fingerprint	"daylight"	"daylight"/"morgan"/"maccs"	分子指纹类型
nbits	add_fingerprint	2048	256–4096	指纹位向量长度

最佳实践

1. 始终以相同方式处理查询谱图和参考谱图 — 对两组数据应用相同的筛选流程，避免相似度评分产生系统性偏差
2. 将中间结果保存为 pickle 格式 — pickle 格式重新加载速度最快；与其他工具共享时使用 MGF/MSP
3. 按前体质量预筛选 — 对于大型谱库，先使用 PrecursorMzMatch 缩小比较范围，再用 CosineGreedy 评分
4. 结合多种指标 — 同时使用 CosineGreedy（峰匹配）和 FingerprintSimilarity（结构），以获得更可靠的鉴定结果
5. 筛选后检查 None — 谱图未通过质量要求时，筛选器会返回 None。务必进行筛选：[s for s in processed if s is not None]
6. 使用 ModifiedCosine 搜索类似物 — 查询的谱库可能不包含完全相同的化合物时，ModifiedCosine 可处理前体质量差异

常用方法

方法 1：前体离子筛选的谱库搜索（高效）

from matchms import calculate_scores
from matchms.similarity import PrecursorMzMatch, CosineGreedy

# Step 1: Fast mass filter
mass_scores = calculate_scores(references=library, queries=unknowns,
                                similarity_function=PrecursorMzMatch(tolerance=0.5))

# Step 2: Detailed scoring only for mass-matched pairs
cosine = CosineGreedy(tolerance=0.1)
for query in unknowns:
    candidates = mass_scores.scores_by_query(query, sort=True)
    mass_matched = [ref for ref, score in candidates if score["score"] > 0]
    if mass_matched:
        detailed = calculate_scores(references=mass_matched, queries=[query],
                                     similarity_function=cosine)
        best = detailed.scores_by_query(query, sort=True)[:3]
        for ref, s in best:
            print(f"{ref.get('compound_name')}: {s['score']:.3f}")

方法 2：按离子模式分别处理

from matchms.importing import load_from_mgf
from matchms.filtering import default_filters, normalize_intensities

spectra = list(load_from_mgf("mixed_library.mgf"))
spectra = [default_filters(s) for s in spectra]
spectra = [s for s in spectra if s is not None]

# Separate by ion mode
positive = [s for s in spectra if s.get("ionmode") == "positive"]
negative = [s for s in spectra if s.get("ionmode") == "negative"]
print(f"Positive: {len(positive)}, Negative: {len(negative)}")

# Process each mode with mode-specific filtering
positive = [normalize_intensities(s) for s in positive]
negative = [normalize_intensities(s) for s in negative]

方法 3：元数据补全报告

import pandas as pd
from matchms.importing import load_from_mgf
from matchms.filtering import default_filters

spectra = [default_filters(s) for s in load_from_mgf("library.mgf")]
spectra = [s for s in spectra if s is not None]

# Extract metadata summary
rows = []
for s in spectra:
    rows.append({
        "compound_name": s.get("compound_name", ""),
        "precursor_mz": s.get("precursor_mz"),
        "ionmode": s.get("ionmode", ""),
        "smiles": s.get("smiles", ""),
        "inchikey": s.get("inchikey", ""),
        "num_peaks": len(s.peaks.mz)
    })

df = pd.DataFrame(rows)
print(f"Library: {len(df)} spectra")
print(f"Named: {(df.compound_name != '').sum()}")
print(f"With SMILES: {(df.smiles != '').sum()}")
print(f"Ion modes: {df.ionmode.value_counts().to_dict()}")

故障排除

问题	原因	解决方案
所有评分均为 0.0	容差范围内没有匹配峰	增大 tolerance（尝试 0.2–0.5 Da）；确认两张谱图都包含峰
同一化合物的评分仍然较低	碎裂条件不同	使用 ModifiedCosine 替代 CosineGreedy；检查离子模式是否一致
大量谱图被筛选为 None	质量筛选条件过严	降低 require_minimum_number_of_peaks 中的 n_required；放宽强度阈值
元数据字段触发 KeyError	字段名称未统一	先应用 default_filters 以统一元数据键
大型谱库导致内存错误	进行了全对全比较	详细评分前，按前体质量预筛选（PrecursorMzMatch）
add_fingerprint 失败	未安装 RDKit	安装化学扩展：pip install matchms[chemistry]
导入结果为空列表	文件格式或路径错误	确认格式与加载器匹配（.mgf 使用 MGF，.msp 使用 MSP）；检查文件是否为空
多次运行的评分不一致	处理流程不同	使用 SpectrumProcessor 确保查询谱图和参考谱图的处理方式一致

随附资源

• references/filtering_catalog.md — 按类别整理的 50 多个 matchms 过滤函数完整目录（元数据处理、化学结构、质量/电荷、峰处理、质量控制）。

  • 涵盖：所有过滤函数的签名、参数和简要说明，以及常见的过滤组合
  • 已移至正文：核心过滤器（default_filters、normalize_intensities、select_by_relative_intensity、reduce_to_number_of_peaks、remove_peaks_around_precursor_mz、add_fingerprint）位于核心 API 模块 2；过滤类别摘要位于关键概念
  • 已省略：与核心 API 模式重复的单个过滤器示例

• references/workflows_similarity.md — 汇总了两个原始参考文档中的扩展工作流和详细的相似度函数文档。

  • 涵盖：全部 8 个相似度函数及其参数表、大规模比较策略、多指标评分模式、7 个扩展工作流（库匹配、QC、多指标、格式转换、元数据丰富、大规模比较、自动识别报告）、性能注意事项
  • 已移至正文：CosineGreedy/ModifiedCosine/CosineHungarian/FingerprintSimilarity 的用法位于核心 API 模块 3；相似度比较表位于关键概念；库匹配和 QC 工作流位于常见工作流
  • 已省略：基于网络的光谱聚类工作流——需要使用外部工具（spec2vec）；光谱可视化教程——核心 API 模块 1 中的 spectrum.plot() 已涵盖

原始参考文件的处理方式：

• importing_exporting.md（417 行）→ 已完整整合到核心 API 模块 1（I/O 函数、格式列表、Spectrum 创建、pickle 用法）。保留：全部 7 个导入函数、4 个导出函数、Spectrum 类的创建、格式选择指南。省略：USI 详细示例——小众用例
• filtering.md（289 行）→ 已迁移至 references/filtering_catalog.md，核心过滤器已移至核心 API 模块 2
• similarity.md（381 行）→ 已整合到 references/workflows_similarity.md，核心函数位于核心 API 模块 3
• workflows.md（648 行）→ 已整合到 references/workflows_similarity.md，主要工作流位于常见工作流

相关技能

• pyopenms-mass-spectrometry — 完整的 LC-MS/MS 蛋白质组学和代谢组学工作流（特征检测、蛋白质 ID）
• rdkit-cheminformatics — 分子指纹生成和化学结构相似度计算

参考资料

• matchms 文档：https://matchms.readthedocs.io
• Huber 等人（2020），matchms——质谱数据的处理与相似度评分。Journal of Open Source Software，DOI：10.21105/joss.02411
• GitHub：https://github.com/matchms/matchms