optimizing-python-performance

Python 性能优化

快速开始分析性能

# PyInstrument (statistical, readable output)
python -m pyinstrument script.py

# cProfile (detailed, built-in)
python -m cProfile -s cumulative script.py

# Memory profiling
uv add --dev memray
memray run script.py
memray flamegraph memray-*.bin

PyInstrument 用法

from pyinstrument import Profiler

profiler = Profiler()
profiler.start()
result = my_function()
profiler.stop()
print(profiler.output_text(unicode=True, color=True))

内存分析

import tracemalloc

tracemalloc.start()
# ... code ...
snapshot = tracemalloc.take_snapshot()
for stat in snapshot.statistics('lineno')[:10]:
    print(stat)

基准测试（pytest-benchmark）

def test_encode_benchmark(benchmark):
    result = benchmark(encode, 37.7749, -122.4194)
    assert len(result) == 12

pytest tests/ --benchmark-only
pytest tests/ --benchmark-compare

常见优化

# Use set for membership (O(1) vs O(n))
valid = set(items)
if item in valid: ...

# Use deque for queue operations
from collections import deque
queue = deque()
queue.popleft()  # O(1) vs list.pop(0) O(n)

# Use generators for large data
def process(items):
    for item in items:
        yield transform(item)

# Cache expensive computations
from functools import lru_cache

@lru_cache(maxsize=1000)
def expensive(x):
    return compute(x)

# String building
result = "".join(str(x) for x in items)  # Not += in loop

算法复杂度

操作	list	set	dict
查找	O(n)	O(1)	O(1)
插入	O(1)	O(1)	O(1)
删除	O(n)	O(1)	O(1)

如需了解详细策略，请参阅：

• PROFILING.md - 高级性能分析技术
• BENCHMARKS.md - CI 基准测试回归测试

优化检查清单

Before Optimizing:
- [ ] Confirm there's a real problem
- [ ] Profile to find actual bottleneck
- [ ] Establish baseline measurements

Process:
- [ ] Algorithm improvements first
- [ ] Then data structures
- [ ] Then implementation details
- [ ] Measure after each change

After:
- [ ] Add benchmarks to prevent regression
- [ ] Verify correctness unchanged
- [ ] Document why optimization needed

了解更多

本技能基于 Will McGinnis 编写的《高质量 Python 库开发指南》中的 性能部分。以下文章提供了更深入的介绍：

• 性能基准测试
• 使用 PyInstrument 进行性能分析
• 使用 Memray 进行内存分析