neo4j-document-import-skill

Neo4j 文档导入技能

适用场景

• 将 PDF、HTML、纯文本、Markdown 导入 Neo4j，构建知识图谱
• 对文档进行分块，并存储带有嵌入向量的 :Chunk 节点
• 使用大模型从文本中提取实体和关系
• 以编程方式使用 SimpleKGPipeline（neo4j-graphrag）
• 使用 Neo4j LLM Graph Builder（无代码 Web UI）
• 通过 apoc.load.json 加载半结构化 JSON
• 将 LangChain 或 LlamaIndex 文档加载器连接到 Neo4j

不适用场景

• 结构化 CSV / 关系型数据 → neo4j-import-skill
• 导入后进行 GraphRAG 检索 → neo4j-graphrag-skill
• 创建向量索引 → neo4j-vector-search-skill
• 编写 Cypher 查询 → neo4j-cypher-skill

---

方案决策表

情况	方案
不需要编写代码；希望使用拖放式 UX	LLM Graph Builder Web UI
以编程方式构建管道；处理 PDF/文本	SimpleKGPipeline（neo4j-graphrag）
JSON / REST API 响应	apoc.load.json 或 Python + UNWIND
技术栈中已使用 LangChain	Neo4jGraph + 文档加载器
技术栈中已使用 LlamaIndex	Neo4jQueryEngine / Neo4jVectorStore
仅需分块（不提取实体）	手动分块 + MERGE 模式

---

安装

pip install neo4j-graphrag                   # includes SimpleKGPipeline
pip install neo4j-graphrag[openai]           # + OpenAI LLM/embedder
pip install neo4j-graphrag[anthropic]        # + Anthropic Claude
pip install neo4j-graphrag[google]           # + Vertex AI / Gemini
pip install neo4j-graphrag[bedrock]          # + Amazon Bedrock (boto3) — added v1.15.0
pip install neo4j-graphrag[ollama]           # + Ollama (local)
pip install neo4j-graphrag[mistralai]        # + MistralAI
pip install neo4j-graphrag[fuzzy-matching]   # + FuzzyMatchResolver (rapidfuzz)
# spaCy entity resolver (Python <= 3.13 only — unsupported on 3.14+):
pip install neo4j-graphrag[nlp]

要求：neo4j>=5.17.0（支持 6.x 驱动）、Python>=3.10、Neo4j>=5.18.1（Aura>=5.18.0）。

---

第 1 步 — 定义图架构

架构决定大模型提取的内容。请在构建管道前定义。

# Option A — Simple string lists (LLM infers descriptions)
entities = ["Person", "Organization", "Location", "Product", "Event"]
relations = ["WORKS_AT", "LOCATED_IN", "KNOWS", "MENTIONS", "PART_OF"]
patterns = [
    ("Person", "WORKS_AT", "Organization"),
    ("Organization", "LOCATED_IN", "Location"),
    ("Person", "KNOWS", "Person"),
    ("Article", "MENTIONS", "Organization"),
]

# Option B — Rich GraphSchema (production; best extraction quality)
from neo4j_graphrag.components.schema import (
    GraphSchema, NodeType, RelationshipType, PropertyType, ConstraintType
)
schema = GraphSchema(
    node_types=[
        NodeType(
            label="Person",
            description="A human individual",
            properties=[
                PropertyType(name="name", type="STRING"),
                PropertyType(name="role", type="STRING"),
            ],
        ),
        NodeType(
            label="Organization",
            description="A company or institution",
            properties=[
                PropertyType(name="name", type="STRING"),
                PropertyType(name="industry", type="STRING"),
            ],
        ),
    ],
    relationship_types=[
        RelationshipType(label="WORKS_AT", description="Employment relationship"),
    ],
    patterns=[("Person", "WORKS_AT", "Organization")],
    # Optional: constraints emitted to ParquetWriter metadata (v1.15.0+)
    constraints=[
        ConstraintType(label="Person", property_name="name", type="UNIQUENESS"),
        ConstraintType(label="Organization", property_name="name", type="KEY"),
    ],
)

# Option C — Auto-extract schema from text (no constraints)
schema = "EXTRACTED"   # LLM infers types; noisier output
schema = "FREE"        # No schema guidance; most noise

生产环境使用选项 B；原型开发使用选项 A；仅在探索时使用 "EXTRACTED"。

---

第 2 步 — 设置 SimpleKGPipeline

import asyncio
from neo4j import GraphDatabase
from neo4j_graphrag.experimental.pipeline.kg_builder import SimpleKGPipeline
from neo4j_graphrag.llm import OpenAILLM
from neo4j_graphrag.embeddings import OpenAIEmbeddings

driver = GraphDatabase.driver(
    "neo4j+s://xxxx.databases.neo4j.io",
    auth=("neo4j", "password")
)

llm = OpenAILLM(
    model_name="gpt-4.1",
    model_params={"temperature": 0},
    # Note: SimpleKGPipeline auto-enables structured output for OpenAI/VertexAI LLMs (v1.14.0+)
    # Do NOT set response_format manually — it is managed by the pipeline
)
embedder = OpenAIEmbeddings()   # OPENAI_API_KEY from env

pipeline = SimpleKGPipeline(
    llm=llm,
    driver=driver,
    embedder=embedder,
    schema=schema,              # GraphSchema, dict, "FREE", or "EXTRACTED"
    from_file=True,             # False → pass text= instead of file_path=
    on_error="IGNORE",          # RAISE to surface extraction failures
    perform_entity_resolution=True,
    neo4j_database="neo4j",     # omit to use default
)

大模型替代方案（接口相同）：

• AnthropicLLM(model_name="claude-3-5-sonnet-20241022")
• VertexAILLM(model_name="gemini-2.0-flash")
• OllamaLLM(model_name="llama3") — 本地运行；无需 API 密钥
• BedrockLLM(model_id="anthropic.claude-3-5-sonnet-20241022-v2:0") — Amazon Bedrock（v1.15.0+）

---

第 3 步 — 运行管道

# From PDF file:
result = asyncio.run(pipeline.run_async(
    file_path="report.pdf",        # auto-dispatches to PdfLoader
    document_metadata={"source": "Q4 report", "year": 2025},
))

# From Markdown file (v1.15.0+):
result = asyncio.run(pipeline.run_async(
    file_path="notes.md",          # auto-dispatches to MarkdownLoader
    document_metadata={"source": "meeting notes"},
))

# Note: old `from_pdf=True` parameter is DEPRECATED since v1.15.0; use `from_file=True` instead
# pipeline = SimpleKGPipeline(..., from_file=True)   ← correct
# pipeline = SimpleKGPipeline(..., from_pdf=True)    ← deprecated

# From raw text:
result = asyncio.run(pipeline.run_async(
    text=document_text,
))

# Batch — process multiple files:
async def ingest_all(paths):
    for p in paths:
        await pipeline.run_async(file_path=str(p))

asyncio.run(ingest_all(list(pdf_dir.glob("*.pdf"))))

document_metadata 字典会作为属性存储在 :Document 节点上。

---

第 4 步 — 分块配置

默认分块器：FixedSizeSplitter(chunk_size=300, chunk_overlap=50)。

from neo4j_graphrag.components.text_splitters.fixed_size_splitter import FixedSizeSplitter

splitter = FixedSizeSplitter(
    chunk_size=512,       # tokens; 300–512 typical for GPT-4o
    chunk_overlap=50,     # ~10% of chunk_size; preserves boundary context
    approximate=True,     # respect sentence/word boundaries when possible
)

pipeline = SimpleKGPipeline(
    ...,
    text_splitter=splitter,
)

分块建议：

文档类型	chunk_size	chunk_overlap
高密度技术文本	256–512	50–80
叙事类 / 新闻文章	512–1024	80–128
法律 / 财务文档	256–384	40–64

规则：分块必须适合大模型的提取上下文窗口，也必须符合嵌入模型的限制。GPT-4o：128k 上下文；text-embedding-3-small：8191 个 token。chunk_size 绝不可设置为大于 2048。

---

第 5 步 — 实体消歧

管道运行后合并重复提取的实体。

from neo4j_graphrag.components.resolver import (
    SinglePropertyExactMatchResolver,   # identical name → merge
    FuzzyMatchResolver,                  # Levenshtein similarity; needs rapidfuzz
    SpaCySemanticMatchResolver,          # cosine similarity; needs neo4j-graphrag[nlp]
)

# Exact match (fastest; good baseline)
resolver = SinglePropertyExactMatchResolver(driver)
asyncio.run(resolver.run())

# Fuzzy match (handles typos / alternate spellings)
from neo4j_graphrag.components.resolver import FuzzyMatchResolver
resolver = FuzzyMatchResolver(driver, threshold=0.9)
asyncio.run(resolver.run())

# Scope resolution to specific labels only:
resolver = SinglePropertyExactMatchResolver(
    driver,
    filter_query="WHERE n:Organization OR n:Person",
)
asyncio.run(resolver.run())

请在导入后运行消歧器，而不要将其内联运行 — 批量合并速度更快。

---

生成的图结构

管道始终会生成以下词汇图层：

(:Document {id, fileName, status, ...metadata})
    -[:HAS_CHUNK]->
(:Chunk {id, text, index, embedding, ...})
    -[:NEXT_CHUNK]->          ← linked list for ordered traversal
(:Chunk {...})

(:Chunk)-[:FROM_DOCUMENT]->(:Document)   ← back-pointer

实体提取还会添加：

(:Chunk)-[:MENTIONS]->(:Person {name, ...})
(:Chunk)-[:MENTIONS]->(:Organization {name, ...})
(:Person)-[:WORKS_AT]->(:Organization)

导入后进行验证：

CYPHER 25
MATCH (d:Document)-[:HAS_CHUNK]->(c:Chunk)
RETURN d.fileName, count(c) AS chunks LIMIT 10;

MATCH (c:Chunk)-[:MENTIONS]->(e)
RETURN labels(e)[0] AS type, count(*) AS cnt ORDER BY cnt DESC LIMIT 20;

---

LLM Graph Builder（无代码 UI）

适用场景：非开发人员需要导入文档；快速原型开发；没有 Python 环境。

托管版：https://llm-graph-builder.neo4jlabs.com/

本地版（Docker）：

git clone https://github.com/neo4j-labs/llm-graph-builder
cd llm-graph-builder
# Set OPENAI_API_KEY (or other provider keys) in .env
docker-compose up
# Opens at http://localhost:8080

支持的数据源：PDF、纯文本、Markdown、图片、网页、YouTube 转录文本、S3/GCS 存储桶上传。

大模型提供方：OpenAI、Gemini、Claude、Llama3、Diffbot、Qwen。

限制：最适合长篇英文文本；处理表格数据效果较差（CSV/Excel 请使用 neo4j-import-skill）；不会提取可视化图表。

---

APOC JSON 导入（半结构化）

适用于从 REST API、S3 或文件导出获取 JSON 数据的场景。

CYPHER 25
CALL apoc.load.json("https://example.com/articles.json") YIELD value
UNWIND value.articles AS article
CALL (article) {
  MERGE (d:Document {id: article.id})
  SET d.title = article.title, d.url = article.url, d.publishedAt = article.publishedAt
  FOREACH (tag IN article.tags |
    MERGE (t:Tag {name: tag})
    MERGE (d)-[:HAS_TAG]->(t)
  )
} IN TRANSACTIONS OF 1000 ROWS

本地文件：apoc.load.json("file:///import/data.json")。文件必须位于 $NEO4J_HOME/import/ 下，或配置 APOC allowlist。

检查 APOC 是否可用：RETURN apoc.version()。所有 Aura 套餐均包含 APOC。

---

LangChain 集成模式

from langchain_community.graphs import Neo4jGraph
from langchain_community.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from neo4j import GraphDatabase

graph = Neo4jGraph(
    url="neo4j+s://xxxx.databases.neo4j.io",
    username="neo4j",
    password="password",
)

loader = PyPDFLoader("report.pdf")
docs = loader.load()
splitter = RecursiveCharacterTextSplitter(chunk_size=512, chunk_overlap=64)
chunks = splitter.split_documents(docs)

embedder = OpenAIEmbeddings()
driver = GraphDatabase.driver(url, auth=("neo4j", "password"))

for i, chunk in enumerate(chunks):
    emb = embedder.embed_query(chunk.page_content)
    driver.execute_query(
        """
        MERGE (doc:Document {id: $doc_id})
        SET doc.source = $source
        CREATE (c:Chunk {id: $chunk_id, text: $text, embedding: $emb, index: $idx})
        CREATE (doc)-[:HAS_CHUNK]->(c)
        """,
        doc_id=chunk.metadata.get("source", "unknown"),
        source=chunk.metadata.get("source"),
        chunk_id=f"chunk-{i}",
        text=chunk.page_content,
        emb=emb,
        idx=i,
    )

如需使用 LangChain 提取实体：请使用 LLMGraphTransformer（来自 langchain_experimental.graph_transformers）。生成的结构与 :Document/:Chunk/实体模式相同。

---

约束和索引（摄取前运行）

CYPHER 25
// Prevent duplicate documents
CREATE CONSTRAINT doc_id_unique IF NOT EXISTS
  FOR (d:Document) REQUIRE d.id IS UNIQUE;

// Prevent duplicate chunks
CREATE CONSTRAINT chunk_id_unique IF NOT EXISTS
  FOR (c:Chunk) REQUIRE c.id IS UNIQUE;

// Entity deduplication
CREATE CONSTRAINT person_name_unique IF NOT EXISTS
  FOR (p:Person) REQUIRE p.name IS UNIQUE;
CREATE CONSTRAINT org_name_unique IF NOT EXISTS
  FOR (o:Organization) REQUIRE o.name IS UNIQUE;

// Vector index for chunk embeddings (adjust dims for your model)
CREATE VECTOR INDEX chunk_embeddings IF NOT EXISTS
  FOR (c:Chunk) ON c.embedding
  OPTIONS {indexConfig: {`vector.dimensions`: 1536, `vector.similarity_function`: 'cosine'}};

// Poll until index ONLINE:
// SHOW INDEXES YIELD name, state WHERE state <> 'ONLINE'

在所有索引都处于 ONLINE 状态之前，不要开始摄取：

SHOW INDEXES YIELD name, state WHERE state <> 'ONLINE';

如果返回了行：等待，然后重新运行。ONLINE = 可以安全摄取。

---

常见错误

错误	原因	修复方法
大模型提取了架构中未定义的节点类型	架构过于宽松，或处于 "EXTRACTED" 模式	定义明确的 entities + patterns；使用 Option B 架构
MissingEmbedderError	未指定 embedder=	始终传入 embedder=，即使不进行向量搜索也一样——管道会将嵌入存储在 Chunk 节点上
未提取到任何实体	大模型上下文溢出	减小 chunk_size；改用上下文更大的模型
摄取后出现重复的实体节点	未运行实体消解	批量摄取后运行 SinglePropertyExactMatchResolver
apoc.load.json 权限被拒绝	未配置 APOC 允许列表	将 URL 添加到 apoc.import.file.enabled=true 和 dbms.security.allow_csv_import_from_file_urls=true
分块时句子中途被截断	approximate=False（默认值）会按精确的令牌数截断	在 FixedSizeSplitter 中设置 approximate=True
chunk_size 过大 → 大模型超时	提取提示词加上分块内容超出上下文限制	GPT-4o 提取时将 chunk_size 控制在 ≤ 512；绝对最大值为 ≤ 2048
SpaCySemanticMatchResolver 在 Python 3.14 上失败	spaCy 不支持 3.14 及更高版本	使用 FuzzyMatchResolver，或将 Python 降级到 3.13
找不到 neo4j-driver 包	该包名自 6.0 起已弃用	使用 neo4j 包：pip install neo4j>=5.17.0
NodeType 没有属性时出现 ValidationError	自 v1.13.0 起，NodeType 至少需要 1 个属性	至少添加 PropertyType(name="name", type="STRING")；字符串列表标签会自动添加该属性
from_pdf 弃用警告	from_pdf=True 已在 v1.15.0 中移除	改用 from_file=True
model_params 中的 response_format 被忽略	SimpleKGPipeline 会为 OpenAI/VertexAI 自动启用结构化输出（v1.14.0+）	从 model_params 中移除 response_format；由管道进行管理

---

验证清单

• 开始摄取前已创建约束，且状态为 ONLINE
• 存储嵌入前已创建向量索引
• chunk_size 未超过嵌入模型限制（≤2048；提取时 ≤512）
• chunk_overlap 设置为 chunk_size 的 10–15%
• 使用 Document→HAS_CHUNK→Chunk 模式（可在检索中进行图遍历）
• document_metadata 已填入来源标识符
• 批量摄取后已运行实体消解器
• 使用 apoc.load.json 时已确认 apoc.version()
• .env 中包含 API 密钥；.env 已列入 .gitignore
• 验证结构：MATCH (d:Document)-[:HAS_CHUNK]->(c:Chunk) RETURN count(c)
• 验证实体：MATCH (c:Chunk)-[:MENTIONS]->(e) RETURN labels(e)[0], count(*)

---

GraphSchema — 当前 API（≥1.8.0）

entities/relations/potential_schema 已弃用。请使用 schema=GraphSchema(...)。

from neo4j_graphrag.components.schema import (
    GraphSchema, NodeType, RelationshipType, PropertyType,
    ConstraintType, GraphConstraintType,
)

schema = GraphSchema(
    node_types=[
        NodeType(label="Person", properties=[PropertyType(name="name", type="STRING")]),
        NodeType(label="Organization", properties=[PropertyType(name="name", type="STRING")]),
    ],
    relationship_types=[RelationshipType(label="WORKS_AT")],
    patterns=[("Person", "WORKS_AT", "Organization")],
    # Constraints (v1.15.0+) — emitted to ParquetWriter metadata and enforce schema
    constraints=[
        # UNIQUENESS — property must be unique across nodes of that label
        ConstraintType(label="Person", property_name="name", type=GraphConstraintType.UNIQUENESS),
        # KEY — uniqueness + existence (null not allowed)
        ConstraintType(label="Organization", property_name="name", type=GraphConstraintType.KEY),
        # EXISTENCE — property must be non-null (replaces deprecated PropertyType.required)
        ConstraintType(label="Event", property_name="date", type=GraphConstraintType.EXISTENCE),
        # Composite KEY (v1.15.0+)
        ConstraintType(
            label="Person",
            property_names=("first_name", "last_name"),
            type=GraphConstraintType.KEY,
        ),
    ],
)
pipeline = SimpleKGPipeline(llm=llm, driver=driver, embedder=embedder, schema=schema)

schema="FREE"（无引导）或 schema="EXTRACTED"（大模型推断类型）——仅用于探索，输出噪声较多。

从文本自动提取架构（v1.15.0+）

向 SimpleKGPipeline 传入 schema 时，会自动运行 SchemaFromTextExtractor。 显式运行方式：

from neo4j_graphrag.components.graph_schema_extraction import (
    SchemaFromTextExtractor,
    SchemaFromExistingGraphExtractor,
)

# Infer schema from sample text
extractor = SchemaFromTextExtractor(llm=llm, use_structured_output=True)
schema = asyncio.run(extractor.run(text=sample_text))

# Derive schema from an existing Neo4j graph
extractor = SchemaFromExistingGraphExtractor(driver=driver)
schema = asyncio.run(extractor.run())

Parquet 导出（实验性，v1.14.0+）

from neo4j_graphrag.components.parquet_output import ParquetWriter

# Use ParquetWriter instead of KGWriter inside a Pipeline to export to Parquet files
writer = ParquetWriter(output_dir="/data/kg_export/")
# Produces one Parquet file per node label and per relationship type
# Metadata includes UNIQUENESS, EXISTENCE, and KEY constraints (v1.15.0/1.16.0)

---

LexicalGraphConfig — 自定义标签

覆盖默认的词汇层标签（除非要与现有图集成，否则请保留默认值）：

from neo4j_graphrag.components.types import LexicalGraphConfig
# All fields have sensible defaults — only override what differs from your graph's conventions
config = LexicalGraphConfig(
    document_node_label="Article",             # default: "Document"
    chunk_node_label="Passage",                # default: "Chunk"
    node_to_chunk_relationship_type="HAS_ENTITY",  # default: "MENTIONS"
    chunk_text_property="content",             # default: "text"
)
pipeline = SimpleKGPipeline(..., lexical_graph_config=config)

---

自定义文档加载器

默认的 file_loader 会根据扩展名自动分派（.pdf→PdfLoader，.md→MarkdownLoader）。 支持 fsspec URI（s3://、gcs://）。为 HTML/网页/自定义格式创建 DataLoader 子类：

from neo4j_graphrag.components.data_loader import DataLoader
from neo4j_graphrag.components.types import DocumentInfo, LoadedDocument

class WebPageLoader(DataLoader):
    async def run(self, filepath, metadata=None):
        import httpx
        text = httpx.get(filepath).text   # strip HTML in real impl
        return LoadedDocument(text=text,
            document_info=DocumentInfo(path=filepath, metadata=metadata))

pipeline = SimpleKGPipeline(..., file_loader=WebPageLoader(), from_file=True)

有关按用例选择的分块策略和完整的消解器配置，请参阅：references/kg-construction.md。

---

参考资料

按需加载：

• neo4j-graphrag KG Builder 指南
• neo4j-graphrag 库概览
• LLM Graph Builder（托管版）
• LLM Graph Builder GitHub
• APOC 加载过程
• GraphAcademy：使用 LLM 构建知识图谱
• LangChain Neo4j 集成
• LlamaIndex Neo4jQueryEngine
• 扩展版 KG 构建参考