etetoolkit

ETE Toolkit：系统发育树分析与可视化

概述

ETE Toolkit（ETE3）是一个用于系统发育树探索、操作和出版级可视化的 Python 框架。它支持读取和写入 Newick、NHX、PhyloXML 和 NeXML 格式，提供丰富的节点注释、程序化树遍历、NCBI 分类学集成，以及灵活的渲染引擎，可自定义树图。ETE3 广泛应用于比较基因组学、系统发育基因组学和进化生物学工作流。

适用场景

• 从 Newick、NHX、PhyloXML 或 NeXML 文件解析系统发育树，并以编程方式遍历或修改拓扑结构
• 为树节点添加元数据（自举值、基因名称、分类学等级、表达数据），用于可视化或下游分析
• 使用 TreeStyle 和 NodeStyle，通过自定义节点形状、颜色、分支宽度和面部装饰，渲染出版级树图
• 将 NCBI 分类学 ID 映射到谱系信息、验证物种名称，或构建分类学感知树
• 计算进化统计信息：分支长度、树距离（Robinson-Foulds）、LCA 查询、单系群测试
• 构建 PhyloTree 对象用于比较基因组学——注释基因复制／物种形成事件，推断直系同源／旁系同源关系
• 在将树传递给下游工具（BEAST、IQ-TREE 等）之前，以编程方式对树进行修剪、重新定根或超度量化
• 若需在构建树之前进行序列比对，请改用 biopython-molecular-biology

前置条件

• Python 包：ete3、numpy、PyQt5（用于交互式渲染）、lxml（用于 PhyloXML）
• 数据要求：Newick 字符串或树文件；NCBI 分类学数据库（首次使用 NCBI 模块时下载）
• 环境：Python 3.6+；TreeStyle 渲染和交互式 GUI 需要 PyQt5；无头渲染需要 xvfb

安装前请检查：当前环境中可能已经提供了该工具（例如，在 pixi / conda 环境中）。先运行 command -v python；如果返回路径，请跳过下方的安装命令。在 pixi 项目中运行时，请使用 pixi run python 调用该工具，而不要直接使用 python。

pip install ete3 numpy lxml PyQt5
# For headless rendering on Linux servers:
# apt-get install xvfb python3-pyqt5

快速开始

from ete3 import Tree

# Load a Newick tree and inspect basic properties
t = Tree("((A:0.1,B:0.2)AB:0.3,(C:0.4,D:0.1)CD:0.2)root;")
print(f"Number of leaves: {len(t.get_leaves())}")
print(f"Leaf names: {t.get_leaf_names()}")
print(f"Tree depth: {t.get_farthest_leaf()[1]:.3f}")
# Number of leaves: 4
# Leaf names: ['A', 'B', 'C', 'D']
# Tree depth: 0.700
t.show()  # Opens interactive viewer (requires PyQt5)

核心 API

模块 1：树的 I/O（树解析与序列化）

从字符串或文件加载树；以多种格式写入。

from ete3 import Tree, PhyloTree

# Parse Newick string (format 1 = standard Newick with support values)
t = Tree("((A:0.1,B:0.2)90:0.3,(C:0.4,D:0.1)85:0.2)root;", format=1)
print(f"Root children: {[n.name for n in t.children]}")

# Load from file
t_file = Tree("my_tree.nwk")

# Write Newick with internal names and supports
nwk_str = t.write(format=1)
print(f"Newick: {nwk_str}")

# Write to file
t.write(outfile="output_tree.nwk", format=0)
print("Saved output_tree.nwk")

from ete3 import PhyloTree

# Load PhyloXML tree (retains sequence annotations)
# pt = PhyloTree("my_phylo.xml", parser="phyloxml")

# Load NHX format (extended Newick with key=value annotations)
nhx = Tree("((A[&&NHX:S=human:D=Y],B[&&NHX:S=mouse:D=N]))")
for leaf in nhx.get_leaves():
    print(f"{leaf.name}: species={leaf.S}, duplication={leaf.D}")
# A: species=human, duplication=Y
# B: species=mouse, duplication=N

模块 2：树遍历与搜索

使用前序、后序或广度优先遍历浏览节点；按名称或属性搜索。

from ete3 import Tree

t = Tree("((Homo_sapiens:0.1,Pan_troglodytes:0.05)Hominidae:0.2,(Mus_musculus:0.3,Rattus_norvegicus:0.25)Muridae:0.4)Euarchontoglires;")

# Iterate all nodes (preorder by default)
for node in t.traverse("preorder"):
    depth = node.get_distance(t)
    print(f"{'leaf' if node.is_leaf() else 'internal'}: {node.name or 'unnamed'} depth={depth:.3f}")

# Search by name
human = t.search_nodes(name="Homo_sapiens")[0]
print(f"Human branch length: {human.dist:.3f}")
print(f"Ancestors: {[a.name for a in human.get_ancestors()]}")

from ete3 import Tree

t = Tree("((Homo_sapiens:0.1,Pan_troglodytes:0.05)Hominidae:0.2,(Mus_musculus:0.3,Rattus_norvegicus:0.25)Muridae:0.4)Euarchontoglires;")

# Lowest common ancestor (LCA) query
human = t & "Homo_sapiens"   # shorthand for search_nodes(name=...)[0]
mouse = t & "Mus_musculus"
lca = t.get_common_ancestor(human, mouse)
print(f"LCA of human and mouse: {lca.name}")
# LCA of human and mouse: Euarchontoglires

# Check monophyly
is_mono, mono_type, broken = t.check_monophyly(
    values=["Homo_sapiens", "Pan_troglodytes"], target_attr="name"
)
print(f"Hominids monophyletic: {is_mono}, type: {mono_type}")
# Hominids monophyletic: True, type: monophyletic

模块 3：节点注释

为节点添加自定义属性，以支持由元数据驱动的可视化和分析。

from ete3 import Tree

t = Tree("((Homo_sapiens,Pan_troglodytes)Hominidae,(Mus_musculus,Rattus_norvegicus)Muridae)Euarchontoglires;")

# Annotate leaves with arbitrary metadata
metadata = {
    "Homo_sapiens":      {"genome_size_gb": 3.2, "ploidy": 2, "color": "blue"},
    "Pan_troglodytes":   {"genome_size_gb": 3.1, "ploidy": 2, "color": "green"},
    "Mus_musculus":      {"genome_size_gb": 2.7, "ploidy": 2, "color": "orange"},
    "Rattus_norvegicus": {"genome_size_gb": 2.9, "ploidy": 2, "color": "red"},
}
for leaf in t.get_leaves():
    for attr, val in metadata[leaf.name].items():
        setattr(leaf, attr, val)

# Access annotations
for leaf in t.get_leaves():
    print(f"{leaf.name}: {leaf.genome_size_gb} Gb, {leaf.color}")

from ete3 import Tree
import pandas as pd

t = Tree("((A,B)AB,(C,D)CD)root;")

# Load annotations from a DataFrame and apply to tree
df = pd.DataFrame({
    "name":  ["A", "B", "C", "D"],
    "value": [1.2, 3.4, 0.8, 2.1],
    "group": ["x", "x", "y", "y"],
})
name_to_row = df.set_index("name").to_dict(orient="index")

for leaf in t.get_leaves():
    if leaf.name in name_to_row:
        leaf.add_features(**name_to_row[leaf.name])
        print(f"{leaf.name}: value={leaf.value}, group={leaf.group}")

模块 4：树操作

修剪、重新定根、超度量化并计算距离。

from ete3 import Tree

t = Tree("((A:0.1,B:0.2)AB:0.3,(C:0.4,D:0.1,(E:0.2,F:0.3)EF:0.1)CD:0.2)root;")
print(f"Original leaves: {t.get_leaf_names()}")

# Prune to a subset of taxa
t.prune(["A", "C", "E"], preserve_branch_length=True)
print(f"Pruned leaves: {t.get_leaf_names()}")

# Reroot on midpoint
t2 = Tree("((A:0.5,B:0.1):0.2,(C:0.3,D:0.4):0.1);")
midpoint_node, midpoint_dist = t2.get_midpoint_outgroup()
t2.set_outgroup(midpoint_node)
print(f"Rerooted at midpoint; root children: {[n.name for n in t2.children]}")

# Robinson-Foulds distance between two topologies
t_ref = Tree("((A,B),(C,D));")
t_alt = Tree("((A,C),(B,D));")
rf, rf_max, common_attrs, discard_t1, discard_t2, parts1, parts2 = t_ref.robinson_foulds(t_alt)
print(f"RF distance: {rf}, normalized: {rf/rf_max:.3f}")

模块 5：树可视化（TreeStyle / NodeStyle）

使用自定义样式渲染出版级树图。

from ete3 import Tree, TreeStyle, NodeStyle, faces, AttrFace, CircleFace

t = Tree("((Homo_sapiens,Pan_troglodytes)Hominidae,(Mus_musculus,Rattus_norvegicus)Muridae)Euarchontoglires;")

# Define node styles
for node in t.traverse():
    nstyle = NodeStyle()
    if node.is_leaf():
        nstyle["shape"] = "circle"
        nstyle["size"] = 8
        nstyle["fgcolor"] = "darkblue"
    else:
        nstyle["shape"] = "sphere"
        nstyle["size"] = 6
        nstyle["fgcolor"] = "gray"
    node.set_style(nstyle)

# Add text face to leaves
for leaf in t.get_leaves():
    name_face = AttrFace("name", fsize=12, fgcolor="black")
    leaf.add_face(name_face, column=0, position="branch-right")

# Configure TreeStyle
ts = TreeStyle()
ts.mode = "r"          # rectangular (use "c" for circular)
ts.show_leaf_name = False
ts.branch_vertical_margin = 15
ts.title.add_face(faces.TextFace("Phylogenetic Tree", fsize=16), column=0)

# Render to file (no display needed)
t.render("tree_figure.png", tree_style=ts, w=800, units="px")
print("Saved tree_figure.png")

from ete3 import Tree, TreeStyle, NodeStyle, faces, RectFace

t = Tree("((A,B)AB,(C,D)CD)root;")

# Circular cladogram with colored rectangles
metadata = {"A": "red", "B": "red", "C": "blue", "D": "blue"}
for leaf in t.get_leaves():
    leaf.color = metadata[leaf.name]
    leaf.add_face(RectFace(width=20, height=20, fgcolor=leaf.color, bgcolor=leaf.color),
                  column=0, position="aligned")

ts = TreeStyle()
ts.mode = "c"          # circular layout
ts.arc_start = -180
ts.arc_span = 359
ts.show_leaf_name = True

t.render("circular_tree.png", tree_style=ts, w=600, units="px")
print("Saved circular_tree.png")

模块 6：NCBI 分类学集成

将物种映射到 NCBI 分类学，检索谱系并构建分类学树。

from ete3 import NCBITaxa

# Initialize (downloads ~50 MB taxonomy DB on first call)
ncbi = NCBITaxa()
# ncbi.update_taxonomy_database()  # Refresh to latest NCBI taxonomy

# Name → taxid
taxid_map = ncbi.get_name_translator(["Homo sapiens", "Mus musculus", "Danio rerio"])
print(f"Taxid map: {taxid_map}")
# Taxid map: {'Homo sapiens': [9606], 'Mus musculus': [10090], 'Danio rerio': [7955]}

# Taxid → lineage
lineage = ncbi.get_lineage(9606)
names = ncbi.get_taxid_translator(lineage)
ranks = ncbi.get_rank(lineage)
for taxid in lineage[-6:]:
    print(f"  {ranks[taxid]:15s}: {names[taxid]}")

from ete3 import NCBITaxa, Tree, TreeStyle

ncbi = NCBITaxa()

# Build a taxonomy tree for a set of taxids
taxids = [9606, 10090, 7955, 6239, 7227]   # human, mouse, zebrafish, C. elegans, fruit fly
tree = ncbi.get_topology(taxids, intermediate_nodes=True)

# Annotate with common names
translator = ncbi.get_taxid_translator([int(n.name) for n in tree.get_leaves()])
for leaf in tree.get_leaves():
    leaf.sci_name = translator.get(int(leaf.name), leaf.name)
    print(f"Taxid {leaf.name}: {leaf.sci_name}")

# Render taxonomy tree
ts = TreeStyle()
ts.show_leaf_name = True
tree.render("taxonomy_tree.png", tree_style=ts, w=600, units="px")
print("Saved taxonomy_tree.png")

模块 7：用于比较基因组学的 PhyloTree

为基因树注释基因复制／物种形成事件，并查询直系同源关系。

from ete3 import PhyloTree

# Build a gene tree with species mapping
# Format: leaf names must follow "gene_SPECIES" or use sp_naming_function
nwk = "((Hsap_BRCA1:0.1,Ptro_BRCA1:0.05)0.99:0.2,(Mmus_Brca1:0.3,Rnor_Brca1:0.25)0.95:0.1);"
t = PhyloTree(nwk, sp_naming_function=lambda name: name.split("_")[0])

# Annotate events (duplication vs speciation)
t.get_descendant_evol_events()

# Report events per node
for node in t.traverse():
    if not node.is_leaf() and hasattr(node, "evoltype"):
        print(f"Node evoltype: {node.evoltype} "
              f"(D=duplication, S=speciation)")

# Get orthologs for a given leaf
leaf = t & "Hsap_BRCA1"
orthologs = leaf.get_sisters()
print(f"Orthologs of Hsap_BRCA1: {[n.name for n in t.get_leaves() if n != leaf]}")

核心概念

Newick 格式变体

ETE3 的 format 参数用于控制要解析的 Newick 格式类型：

格式	内部节点标签	分支长度	典型用途
0	灵活	是	IQ-TREE、RAxML 输出
1	名称 + 支持值	是	标准注释树
5	仅内部节点名称	否	仅含拓扑结构的树
9	仅叶节点名称	是	简单标记树
100	无名称	否	纯拓扑结构

from ete3 import Tree
# RAxML output (bootstrap values as internal labels)
t = Tree("((A:0.1,B:0.2)90:0.3,(C:0.4,D:0.1)85:0.2);", format=1)
for node in t.traverse():
    if not node.is_leaf():
        print(f"Support: {node.support}, dist: {node.dist}")

常见工作流

工作流 1：根据比对结果生成注释树图

目标：加载 IQ-TREE 结果，为叶节点添加元数据注释，并渲染出版用图。

from ete3 import Tree, TreeStyle, NodeStyle, AttrFace, faces
import pandas as pd

# Load IQ-TREE Newick output (format=1 handles support values)
t = Tree("iqtree_output.treefile", format=1)

# Midpoint rooting
outgroup, _ = t.get_midpoint_outgroup()
t.set_outgroup(outgroup)

# Load metadata table
meta = pd.read_csv("sample_metadata.csv")   # columns: name, clade, color
meta_dict = meta.set_index("name").to_dict(orient="index")

# Annotate leaves
for leaf in t.get_leaves():
    info = meta_dict.get(leaf.name, {})
    leaf.clade = info.get("clade", "unknown")
    leaf.face_color = info.get("color", "gray")

# Apply per-node styles
for node in t.traverse():
    ns = NodeStyle()
    if node.is_leaf():
        ns["size"] = 8
        ns["fgcolor"] = node.face_color if hasattr(node, "face_color") else "black"
    else:
        ns["size"] = 0   # hide internal nodes
    node.set_style(ns)

# Add name labels
for leaf in t.get_leaves():
    leaf.add_face(AttrFace("name", fsize=10), column=0, position="branch-right")
    leaf.add_face(AttrFace("clade", fsize=9, fgcolor="gray"), column=1, position="branch-right")

ts = TreeStyle()
ts.show_leaf_name = False
ts.branch_vertical_margin = 12
ts.scale = 200   # pixels per branch length unit

t.render("annotated_tree.pdf", tree_style=ts)
print("Saved annotated_tree.pdf")

工作流 2：基于分类体系构建物种树

目标：根据一组 NCBI taxid 构建拓扑正确的物种树，并导出为 Newick 格式。

from ete3 import NCBITaxa, TreeStyle

ncbi = NCBITaxa()

# Input: list of NCBI taxids
taxids = [9606, 10090, 7955, 6239, 7227, 3702]  # human, mouse, zebrafish, worm, fly, Arabidopsis

# Build topology tree
species_tree = ncbi.get_topology(taxids, intermediate_nodes=False)

# Translate taxid leaf names to scientific names
translator = ncbi.get_taxid_translator([int(n.name) for n in species_tree.get_leaves()])
for leaf in species_tree.get_leaves():
    leaf.name = translator.get(int(leaf.name), leaf.name)

# Export Newick
nwk = species_tree.write(format=9)
print(f"Species tree Newick:\n{nwk}")
with open("species_tree.nwk", "w") as f:
    f.write(nwk)

# Render figure
ts = TreeStyle()
ts.show_leaf_name = True
ts.mode = "r"
species_tree.render("species_tree.png", tree_style=ts, w=800, units="px")
print("Saved species_tree.nwk and species_tree.png")

工作流 3：批量进行 Robinson-Foulds 比较

目标：将一组 bootstrap 重复树与参考拓扑进行比较。

from ete3 import Tree
import pandas as pd

# Reference tree
ref_tree = Tree("reference.nwk", format=1)
ref_leaves = set(ref_tree.get_leaf_names())

results = []
for i, line in enumerate(open("bootstrap_trees.nwk")):
    bt = Tree(line.strip(), format=1)
    # Prune to common taxa
    common = ref_leaves & set(bt.get_leaf_names())
    ref_pruned = ref_tree.copy()
    bt_pruned = bt.copy()
    ref_pruned.prune(list(common))
    bt_pruned.prune(list(common))

    rf, rf_max, *_ = ref_pruned.robinson_foulds(bt_pruned, unrooted_trees=True)
    norm_rf = rf / rf_max if rf_max > 0 else 0
    results.append({"replicate": i + 1, "rf": rf, "rf_max": rf_max, "norm_rf": norm_rf})

df = pd.DataFrame(results)
print(df.describe())
df.to_csv("rf_distances.csv", index=False)
print(f"Mean normalized RF: {df['norm_rf'].mean():.4f}")

关键参数

参数	模块	默认值	范围 / 选项	作用
format	Tree I/O	0	0–9、100	Newick 格式类型；控制对支持值与内部名称的解析
quoted_node_names	Tree I/O	False	True、False	允许节点名称中包含空格和特殊字符
preserve_branch_length	Pruning	False	True、False	修剪子树时保留 patristic 距离
unrooted_trees	Robinson-Foulds	False	True、False	比较无根拓扑（大多数基因树都需要此设置）
ts.mode	TreeStyle	"r"	"r"、"c"	矩形或圆形布局
ts.scale	TreeStyle	None	正整数	每个分支长度单位对应的像素数；控制水平展开程度
ts.branch_vertical_margin	TreeStyle	5	整数（px）	叶分支之间的垂直间距
intermediate_nodes	NCBITaxa	False	True、False	在拓扑树中包含祖先 NCBI taxid

常用方案

方案：提取所有与某个分支名称匹配的子树

适用场景：识别根节点名称与某个模式匹配的所有子树。

from ete3 import Tree

t = Tree("((Homo_sapiens,Pan_troglodytes)Hominidae,(Mus_musculus,Rattus_norvegicus)Muridae)Euarchontoglires;")

target_clade = "Hominidae"
nodes = t.search_nodes(name=target_clade)
for node in nodes:
    print(f"Clade {node.name}: {node.get_leaf_names()}")
    subtree_nwk = node.write(format=1)
    print(f"  Newick: {subtree_nwk}")

方案：将非超度量树转换为超度量树

适用场景：为需要超度量输入的时间校准分析准备树。

from ete3 import Tree

t = Tree("((A:0.3,B:0.1):0.4,(C:0.7,D:0.2):0.1);")
print(f"Before: max leaf distance = {max(t.get_distance(l) for l in t.get_leaves()):.4f}")

# Convert to ultrametric using ETE's method
t.convert_to_ultrametric()
dists = [t.get_distance(l) for l in t.get_leaves()]
print(f"After: all leaf distances = {set(round(d, 6) for d in dists)}")

方案：按组为叶着色并导出 SVG

适用场景：快速生成用于演示的彩色树，无需手动设置样式。

from ete3 import Tree, TreeStyle, NodeStyle

t = Tree("((A,B,C)GroupX,(D,E)GroupY,(F,G,H)GroupZ);")
group_colors = {"GroupX": "steelblue", "GroupY": "tomato", "GroupZ": "seagreen"}

for node in t.traverse():
    ns = NodeStyle()
    if node.is_leaf():
        parent_name = node.up.name if node.up else ""
        ns["fgcolor"] = group_colors.get(parent_name, "black")
        ns["size"] = 10
    node.set_style(ns)

ts = TreeStyle()
ts.show_leaf_name = True
ts.branch_vertical_margin = 14
t.render("colored_tree.svg", tree_style=ts, w=600, units="px")
print("Saved colored_tree.svg")

预期输出

• 树对象：Tree / PhyloTree 实例——可导航的节点层级结构，包含 .children、.up、.name、.dist、.support
• Newick 字符串：由 t.write(format=N) 生成，供下游工具使用的标准文本
• 图像文件：通过 t.render() 生成 .png、.pdf、.svg 文件；分辨率由 w 参数和 dpi 控制
• 分类树：NCBITaxa.get_topology() 返回节点名称为 taxid 的 Tree
• RF 距离：robinson_foulds() 返回整数原始 RF 值和整数最大 RF 值；可用 rf / rf_max 进行归一化

故障排查

问题	原因	解决方案
NewickError: Unexepected char	Newick 格式不匹配（支持值字段与名称字段不一致）	尝试 format=0、1 或 5；检查内部节点是否包含支持值或名称
RuntimeError: cannot connect to display	PyQt5 渲染时需要显示器	使用 t.render("out.png", ...) 代替 t.show()；在无头服务器上通过 xvfb-run 运行
AttributeError: 'Tree' object has no attribute	节点特征尚未标注	检查 dir(node)，或在访问前使用 hasattr(node, "attr")
未找到 NCBI taxonomy DB	首次使用；尚未下载数据库	调用一次 NCBITaxa().update_taxonomy_database()
树图中的所有叶节点都挤在一起	所有分支长度均为零，且 scale 设置过低	设置 ts.scale = None 以自动缩放，或增大 ts.scale 的值
robinson_foulds 出现 ValueError	两棵树没有共同叶节点	比较前将两棵树修剪为共享的分类单元集合
大型树（500 个以上叶节点）渲染缓慢	逐节点执行 Python 渲染循环	设置 ts.show_leaf_name = False 并尽量减少 faces；也可考虑导出 Newick 并使用 FigTree 渲染

参考资料

• ETE3 文档 — 涵盖所有模块的官方教程
• ETE3 API 参考 — 完整的类和方法参考
• ETE3 GitHub 仓库 — 源代码、问题跟踪器和示例笔记本
• Huerta-Cepas J、Serra F、Bork P（2016）ETE 3：系统发育基因组数据的重建、分析与可视化。Mol Biol Evol 33:1635-1638 — ETE3 的主要引用文献