llm-data-automation

面向建筑行业的 LLM 数据自动化

概述

本技能基于 DDC 方法论（第 2.3 章），可使用大语言模型（LLM）自动处理建筑数据。无需手动编写数据转换代码，只需用自然语言描述需求，LLM 就会生成所需的 Python/Pandas 代码。

书籍参考：《Pandas DataFrame и LLM ChatGPT》／《Pandas DataFrame and LLM ChatGPT》

“LLM-модели, такие как ChatGPT и LLaMA, позволяют специалистам без глубоких знаний программирования внести свой вклад в автоматизацию и улучшение бизнес-процессов компании.” — DDC Book，第 2.3 章

快速入门

选项 1：在线使用 ChatGPT/Claude

只需用自然语言描述数据处理任务：

Prompt: "Write Python code to read an Excel file with construction materials,
filter rows where quantity > 100, and save to CSV."

选项 2：运行本地 LLM（Ollama）

# Install Ollama from ollama.com
ollama pull mistral

# Run a query
ollama run mistral "Write Pandas code to calculate total cost from quantity * unit_price"

选项 3：使用 LM Studio（图形界面）

1. 从 lmstudio.ai 下载
2. 安装并选择一个模型（例如 Mistral、LLaMA）
3. 开始与本地 AI 聊天

核心概念

将 DataFrame 作为通用格式

import pandas as pd

# Construction project as DataFrame
# Rows = elements, Columns = attributes
df = pd.DataFrame({
    'element_id': ['W001', 'W002', 'C001'],
    'category': ['Wall', 'Wall', 'Column'],
    'material': ['Concrete', 'Brick', 'Steel'],
    'volume_m3': [45.5, 32.0, 8.2],
    'cost_per_m3': [150, 80, 450]
})

# Calculate total cost
df['total_cost'] = df['volume_m3'] * df['cost_per_m3']
print(df)

面向建筑任务的 LLM 提示词

数据导入：

"Write code to import Excel file with construction schedule,
parse dates, and create a Pandas DataFrame"

数据筛选：

"Filter construction elements where category is 'Structural'
and cost exceeds budget limit of 50000"

数据聚合：

"Group construction data by floor level,
calculate total volume and cost for each floor"

报告生成：

"Create summary report with material quantities grouped by category,
export to Excel with formatting"

常见用例

1. 从 PDF 文档中提取数据

# Prompt to ChatGPT:
# "Write code to extract tables from PDF and convert to DataFrame"

import pdfplumber
import pandas as pd

def pdf_to_dataframe(pdf_path):
    """Extract tables from PDF file"""
    all_tables = []
    with pdfplumber.open(pdf_path) as pdf:
        for page in pdf.pages:
            tables = page.extract_tables()
            for table in tables:
                if table:
                    df = pd.DataFrame(table[1:], columns=table[0])
                    all_tables.append(df)

    if all_tables:
        return pd.concat(all_tables, ignore_index=True)
    return pd.DataFrame()

# Usage
df = pdf_to_dataframe("construction_spec.pdf")
df.to_excel("extracted_data.xlsx", index=False)

2. 处理 BIM 构件数据

# Prompt: "Analyze BIM elements, group by category, calculate volumes"

import pandas as pd

def analyze_bim_elements(csv_path):
    """Analyze BIM element data from CSV export"""
    df = pd.read_csv(csv_path)

    # Group by category
    summary = df.groupby('Category').agg({
        'Volume': 'sum',
        'Area': 'sum',
        'ElementId': 'count'
    }).rename(columns={'ElementId': 'Count'})

    return summary

# Usage
summary = analyze_bim_elements("revit_export.csv")
print(summary)

3. 成本估算工作流

# Prompt: "Create cost estimation from quantities and unit prices"

import pandas as pd

def calculate_cost_estimate(quantities_df, prices_df):
    """
    Calculate project cost estimate

    Args:
        quantities_df: DataFrame with columns [item_code, quantity]
        prices_df: DataFrame with columns [item_code, unit_price, unit]

    Returns:
        DataFrame with cost calculations
    """
    # Merge quantities with prices
    result = quantities_df.merge(prices_df, on='item_code', how='left')

    # Calculate costs
    result['total_cost'] = result['quantity'] * result['unit_price']

    # Add summary
    result['cost_percentage'] = (result['total_cost'] /
                                  result['total_cost'].sum() * 100).round(2)

    return result

# Usage
quantities = pd.DataFrame({
    'item_code': ['C001', 'S001', 'W001'],
    'quantity': [150, 2000, 500]
})

prices = pd.DataFrame({
    'item_code': ['C001', 'S001', 'W001'],
    'unit_price': [120, 45, 85],
    'unit': ['m3', 'kg', 'm2']
})

estimate = calculate_cost_estimate(quantities, prices)
print(estimate)

4. 进度数据处理

# Prompt: "Parse construction schedule, calculate durations, identify delays"

import pandas as pd
from datetime import datetime

def analyze_schedule(schedule_path):
    """Analyze construction schedule for delays"""
    df = pd.read_excel(schedule_path)

    # Parse dates
    df['start_date'] = pd.to_datetime(df['start_date'])
    df['end_date'] = pd.to_datetime(df['end_date'])
    df['actual_end'] = pd.to_datetime(df['actual_end'])

    # Calculate durations
    df['planned_duration'] = (df['end_date'] - df['start_date']).dt.days
    df['actual_duration'] = (df['actual_end'] - df['start_date']).dt.days

    # Identify delays
    df['delay_days'] = df['actual_duration'] - df['planned_duration']
    df['is_delayed'] = df['delay_days'] > 0

    return df

# Usage
schedule = analyze_schedule("project_schedule.xlsx")
delayed_tasks = schedule[schedule['is_delayed']]
print(f"Delayed tasks: {len(delayed_tasks)}")

本地 LLM 设置（无需互联网）

使用 Ollama

# Install
curl -fsSL https://ollama.com/install.sh | sh

# Download models
ollama pull mistral      # General purpose, 7B params
ollama pull codellama    # Code-focused
ollama pull deepseek-coder  # Best for coding tasks

# Run
ollama run mistral "Write Pandas code to merge two DataFrames on project_id"

使用 LlamaIndex 处理公司文档

# Load company documents into local LLM
from llama_index import SimpleDirectoryReader, VectorStoreIndex

# Read all PDFs from folder
reader = SimpleDirectoryReader("company_documents/")
documents = reader.load_data()

# Create searchable index
index = VectorStoreIndex.from_documents(documents)

# Query your documents
query_engine = index.as_query_engine()
response = query_engine.query(
    "What are the standard concrete mix specifications?"
)
print(response)

IDE 推荐

IDE	适用场景	功能
Jupyter Notebook	学习、实验	交互式单元格、可视化
Google Colab	免费 GPU、快速入门	基于云端、预装库
VS Code	专业开发	扩展、GitHub Copilot
PyCharm	大型项目	高级调试、重构

使用 Jupyter 快速设置

pip install jupyter pandas openpyxl pdfplumber
jupyter notebook

最佳实践

1. 从简单开始：先使用清晰、具体的提示词
2. 反复迭代：根据结果改进提示词
3. 验证：运行前务必检查生成的代码
4. 记录：保存有效的提示词以便重复使用
5. 确保安全：处理敏感公司数据时使用本地 LLM

常用提示词库

数据导入

• “读取 Excel 文件并显示前 10 行”
• “使用自定义分隔符和编码导入 CSV”
• “将多个 Excel 工作表加载到 DataFrame 字典中”

数据清理

• “根据 element_id 删除重复行”
• “使用列均值填充缺失值”
• “将列转换为数值类型并处理错误”

数据分析

• “计算数值列的描述性统计”
• “查找成本与工期之间的相关性”
• “使用 IQR 方法识别异常值”

数据导出

• “导出到包含多个工作表的 Excel 文件”
• “以指定编码保存为 CSV”
• “生成格式化的 PDF 报告”

资源

• 书籍：Artem Boiko 著《Data-Driven Construction》，第 2.3 章
• 网站：https://datadrivenconstruction.io
• Pandas 文档：https://pandas.pydata.org/docs/
• Ollama：https://ollama.com
• LM Studio：https://lmstudio.ai
• Google Colab：https://colab.research.google.com

后续步骤

• 查看 pandas-construction-analysis，了解高级 Pandas 操作
• 查看 pdf-to-structured，了解文档处理
• 查看 etl-pipeline，了解自动化数据管道
• 查看 rag-construction，了解如何使用建筑文档实现 RAG