named-entity-extractor

命名实体提取器

从文本中提取命名实体，包括人物、组织、地点、日期等。

功能

• 实体类型：人物、组织、地点、日期、金额、百分比
• 多种模型：使用 spaCy 确保准确性，使用正则表达式提升速度
• 批量处理：处理多个文档
• 实体链接：对文本中的相同实体进行分组
• 导出：支持 JSON、CSV 格式
• 可视化：高亮显示实体

快速开始

from entity_extractor import EntityExtractor

extractor = EntityExtractor()

text = "Apple Inc. was founded by Steve Jobs in Cupertino, California in 1976."

entities = extractor.extract(text)
for entity in entities:
    print(f"{entity['text']}: {entity['type']}")

# Output:
# Apple Inc.: ORG
# Steve Jobs: PERSON
# Cupertino: GPE
# California: GPE
# 1976: DATE

CLI 用法

# Extract from text
python entity_extractor.py --text "Steve Jobs founded Apple in California."

# Extract from file
python entity_extractor.py --input document.txt

# Batch process folder
python entity_extractor.py --input ./documents/ --output entities.csv

# Filter by entity type
python entity_extractor.py --input document.txt --types PERSON,ORG

# Use regex mode (faster, less accurate)
python entity_extractor.py --input document.txt --mode regex

# JSON output
python entity_extractor.py --input document.txt --json

API 参考

EntityExtractor 类

class EntityExtractor:
    def __init__(self, mode: str = "spacy", model: str = "en_core_web_sm")

    # Extraction
    def extract(self, text: str) -> list
    def extract_file(self, filepath: str) -> list
    def extract_batch(self, folder: str) -> dict

    # Filtering
    def filter_entities(self, entities: list, types: list) -> list
    def get_unique_entities(self, entities: list) -> list
    def group_by_type(self, entities: list) -> dict

    # Analysis
    def entity_frequency(self, text: str) -> dict
    def find_relationships(self, text: str) -> list

    # Export
    def to_csv(self, entities: list, output: str) -> str
    def to_json(self, entities: list, output: str) -> str
    def highlight_text(self, text: str) -> str

实体类型

标准实体类型（spaCy）

类型	描述	示例
PERSON	人物，包括虚构人物	"Steve Jobs"
ORG	公司、机构、组织	"Apple Inc."
GPE	国家、城市、州	"California"
LOC	非 GPE 地点、山脉、水域	"Pacific Ocean"
DATE	日期、时期	"January 2024"
TIME	时间	"3:30 PM"
MONEY	货币金额	"$1.5 million"
PERCENT	百分比	"20%"
PRODUCT	产品	"iPhone"
EVENT	事件	"World Cup"
WORK_OF_ART	书籍、歌曲等	"The Great Gatsby"
LAW	法律、法规	"GDPR"
LANGUAGE	语言	"English"
NORP	国籍、群体	"American"

正则表达式模式实体

使用正则表达式模式进行更快速的提取：

类型	描述
EMAIL	电子邮件地址
PHONE	电话号码
URL	网页 URL
DATE	常见日期格式
MONEY	货币金额
PERCENTAGE	百分比

输出格式

实体结果

{
    "text": "Steve Jobs",
    "type": "PERSON",
    "start": 10,
    "end": 20,
    "confidence": 0.95
}

完整提取结果

{
    "text": "Original text...",
    "entities": [
        {"text": "Steve Jobs", "type": "PERSON", "start": 10, "end": 20},
        {"text": "Apple Inc.", "type": "ORG", "start": 30, "end": 40}
    ],
    "summary": {
        "total_entities": 5,
        "unique_entities": 4,
        "by_type": {
            "PERSON": 2,
            "ORG": 1,
            "GPE": 2
        }
    }
}

筛选和分组

按类型筛选

entities = extractor.extract(text)

# Get only people and organizations
filtered = extractor.filter_entities(entities, ["PERSON", "ORG"])

获取唯一实体

# Remove duplicates, keep first occurrence
unique = extractor.get_unique_entities(entities)

按类型分组

grouped = extractor.group_by_type(entities)

# Returns:
{
    "PERSON": ["Steve Jobs", "Tim Cook"],
    "ORG": ["Apple Inc."],
    "GPE": ["California", "Cupertino"]
}

实体频率

frequency = extractor.entity_frequency(text)

# Returns:
{
    "Steve Jobs": {"count": 5, "type": "PERSON"},
    "Apple": {"count": 8, "type": "ORG"},
    "California": {"count": 2, "type": "GPE"}
}

批量处理

处理文件夹

results = extractor.extract_batch("./documents/")

# Returns:
{
    "doc1.txt": {
        "entities": [...],
        "summary": {...}
    },
    "doc2.txt": {
        "entities": [...],
        "summary": {...}
    }
}

导出为 CSV

extractor.to_csv(results, "entities.csv")

# Creates CSV with columns:
# filename, entity_text, entity_type, start, end

文本高亮

生成带有高亮实体的 HTML：

html = extractor.highlight_text(text)

# Returns HTML with colored spans for each entity type

示例工作流

文档分析

extractor = EntityExtractor()

# Analyze a document
text = open("article.txt").read()
result = extractor.extract(text)

# Get key people mentioned
people = extractor.filter_entities(result, ["PERSON"])
print(f"People mentioned: {len(people)}")

# Get frequency
freq = extractor.entity_frequency(text)
top_entities = sorted(freq.items(), key=lambda x: x[1]["count"], reverse=True)[:10]

联系信息提取

extractor = EntityExtractor(mode="regex")

text = """
Contact John Smith at [email protected]
or call (555) 123-4567.
"""

entities = extractor.extract(text)
# Finds: EMAIL, PHONE entities

内容标记

extractor = EntityExtractor()

articles = ["article1.txt", "article2.txt", "article3.txt"]
tags = {}

for article in articles:
    entities = extractor.extract_file(article)
    tags[article] = extractor.get_unique_entities(entities)

依赖项

• spacy>=3.7.0
• pandas>=2.0.0
• en_core_web_sm（spaCy 模型）

注意：运行 python -m spacy download en_core_web_sm 安装该模型。