LLMling

LLMling

一个用于声明式大模型应用开发的框架，专注于资源管理、提示词模板和工具执行。

此软件包为两类使用方提供后端：一个 MCP 服务和一个基于 pydantic-AI 的 Agent

核心概念

LLMLing 为大模型应用提供基于 YAML 的配置系统。 它可以设置自定义 MCP 服务，通过 YAML 文件定义并提供内容。

• 静态声明：在 YAML 中定义大模型的环境，无需编写代码
• MCP 协议：基于 Machine Chat Protocol（MCP），实现标准化的大模型交互
• 组件类型：
  • 资源：内容提供方（文件、文本、CLI 输出等）
  • 提示词：带参数的消息模板
  • 工具：可由大模型调用的 Python 函数

YAML 配置会创建一个完整环境，为大模型提供：

• 通过资源访问内容

• 用于一致交互的结构化提示词

• 用于扩展能力的工具

• 使用现代 Python 从头编写（最低要求为 3.12）

• 100% 类型化

• 基于 pydantic(-ai)

整个系统概览：

graph TB
    subgraph LLMling[LLMling Core Package]
        RT[RuntimeConfig]

        subgraph Core_Components[Core Components]
            Resources[Resource Management<br/>- Load files/URLs<br/>- Process content<br/>- Watch changes]
            Tools[Tool System<br/>- Execute functions<br/>- Register new tools<br/>- OpenAPI integration]
            Prompts[Prompt System<br/>- Static/Dynamic prompts<br/>- Template rendering<br/>- Completion support]
        end

        CLI[Core CLI<br/>- config add/remove/list<br/>- resource list/load<br/>- tool list/execute<br/>- prompt list/render]

        Core_Components -->|YAML configuration| RT
        RT -->|All components| CLI
        CLI -->|modify| Core_Components
    end

    subgraph Direct_Access[mcp-server-llmling<br/>Direct Component Access]
        MCP[HTTP/SSE Server<br/>- Start/Stop server]
        MCP_CLI[Server CLI<br/>- Start/Stop server]
        Injection[Injection Server<br/>- Inject components<br/>during runtime]
    end

    subgraph Function_Access[llmling-agent<br/>Access via Function Calling]
        LLM[LLM Integration<br/>- Function calling<br/>- Resource access<br/>- Tool execution<br/>- Structured output]
        Agent_CLI[Agent CLI<br/>- One-shot execution<br/>- Batch processing]
        Agent_Web[Agent Web UI<br/>- Interactive chat]
    end

    RT -->|All components| MCP
    RT -->|Resources & Tools<br/>via function calling| LLM
    MCP_CLI --> CLI
    Agent_CLI --> CLI

    classDef core fill:#e1f5fe,stroke:#01579b
    classDef comp fill:#e3f2fd,stroke:#1565c0
    classDef cli fill:#fff3e0,stroke:#e65100
    classDef mcp fill:#f3e5f5,stroke:#4a148c
    classDef agent fill:#e8f5e9,stroke:#1b5e20
    classDef access fill:#e8eaf6,stroke:#666
    classDef serverBox fill:#7986cb,stroke:#3949ab
    classDef agentBox fill:#81c784,stroke:#2e7d32

    class RT core
    class Resources,Tools,Prompts comp
    class CLI,MCP_CLI,Agent_CLI cli
    class MCP,Injection mcp
    class LLM,Agent_Web agent
    class Direct_Access serverBox
    class Function_Access agentBox

使用方法

1. 命令行用法

创建一个基本配置文件：

# Create a new config file with basic settings
llmling config init my_config.yml

# Add it to your stored configs
llmling config add myconfig my_config.yml
llmling config set myconfig  # Make it active

基本命令行命令：

# List available resources
llmling resource list

# Load a resource
llmling resource load python_files

# Execute a tool
llmling tool call open_url url=https://github.com

# Show a prompt
llmling prompt show greet

# Many more commands. The CLI will get extended when installing
# llmling-agent and mcp-server-llmling

2. Agent 用法（由 pydantic-AI 提供支持）

创建配置文件（config.yml）：

tools:
  open_url:
    import_path: "webbrowser.open"

resources:
  bookmarks:
    type: text
    description: "Common Python URLs"
    content: |
      Python Website: https://python.org

使用此配置运行 Agent：

from llmling import RuntimeConfig
from llmling_agent import LLMlingAgent
from pydantic import BaseModel

class WebResult(BaseModel):
    opened_url: str
    success: bool

async with RuntimeConfig.open("config.yml") as runtime:
    agent = LLMlingAgent[WebResult](runtime)
    result = await agent.run(
        "Load the bookmarks resource and open the Python website URL"
    )
    print(f"Opened: {result.data.opened_url}")

Agent 将会：

1. 加载书签资源
2. 提取 Python 网站 URL
3. 使用 open_url 工具打开该网页
4. 返回结构化结果

3. 服务用法

使用 Zed Editor

在 settings.json 中将 LLMLing 添加为上下文服务：

{
  "context_servers": {
    "llmling": {
      "command": {
        "env": {},
        "label": "llmling",
        "path": "uvx",
        "args": [
          "mcp-server-llmling@latest",
          "start",
          "path/to/your/config.yml",
          "--zed-mode"
        ]
      },
      "settings": {}
    }
  }
}

使用 Claude Desktop

在 claude_desktop_config.json 中配置 LLMLing：

{
  "mcpServers": {
    "llmling": {
      "command": "uvx",
      "args": [
        "mcp-server-llmling@latest",
        "start",
        "path/to/your/config.yml"
      ],
      "env": {}
    }
  }
}

手动启动服务

直接从命令行启动服务：

# Latest version
uvx mcp-server-llmling@latest start path/to/your/config.yml

资源

资源是内容提供方，可从各种来源加载和预处理数据。

基本资源类型

global_config:  # declare dependencies if used for tools or function prompts
  requirements: ["myapp"]
  scripts:
    - "https://gist.githubusercontent.com/.../get_readme.py"


resources:
  # Load and watch a file or directory
  python_files:
    type: path
    path: "./src/**/*.py"  # Glob patterns supported
    watch:  # Optional file watching
      enabled: true
      patterns:
        - "*.py"
        - "!**/__pycache__/**"  # Exclude patterns with !
    processors:  # Optional processing steps
      - name: format_python
      - name: add_header
        required: false  # Optional step

  # Static text content
  system_prompt:
    type: text
    content: |
      You are a code reviewer specialized in Python.
      Focus on these aspects:
      - Code style (PEP8)
      - Best practices
      - Performance
      - Security

  # Execute CLI commands
  git_changes:
    type: cli
    command: "git diff HEAD~1"  # String or list of args
    shell: true  # Use shell for command
    cwd: "./src"  # Optional working directory
    timeout: 5.0  # Optional timeout in seconds

  # Load Python source code
  utils_module:
    type: source
    import_path: myapp.utils
    recursive: true  # Include submodules
    include_tests: false  # Exclude test files

  # Execute Python callables
  system_info:
    type: callable
    import_path: platform.uname
    keyword_args:  # Optional arguments
      aliased: true

资源组

将相关资源分组，以便轻松访问：

resource_groups:
  code_review:
    - python_files
    - git_changes
    - system_prompt

  documentation:
    - architecture
    - utils_module

文件监视

支持文件监视的资源（path、image）可以配置为检测更改：

resources:
  config_files:
    type: path
    path: "./config"
    watch:
      enabled: true
      patterns:  # .gitignore style patterns
        - "*.yml"
        - "*.yaml"
        - "!.private/**"  # Exclude private files
      ignore_file: ".gitignore"  # Use existing ignore file

资源处理

资源可以通过处理器管道进行处理：

# First define processors
context_processors:
  uppercase:
    type: function
    import_path: myapp.processors.to_upper
    async_execution: false  # Sync function

# Then use them in resources
resources:
  processed_file:
    type: path
    path: "./input.txt"
    processors:
      - name: uppercase

提示词

提示词是可使用参数格式化的消息模板。LLMLing 同时支持声明式 YAML 提示词和基于函数的提示词。

基于 YAML 的提示词

prompts:
  code_review:
    description: "Review Python code changes"
    messages:
      - role: system
        content: |
          You are a Python code reviewer. Focus on:
          - Code style (PEP8)
          - Best practices
          - Performance
          - Security

          Always structure your review as:
          1. Summary
          2. Issues Found
          3. Suggestions

      - role: user
        content: |
          Review the following code changes:

          {code}

          Focus areas: {focus_areas}

    arguments:
      - name: code
        description: "Code to review"
        required: true
      - name: focus_areas
        description: "Specific areas to focus on (one of: style, security, performance)"
        required: false
        default: "style"

基于函数的提示词

基于函数的提示词能提供更多控制，并支持自动补全：

prompts:
  analyze_code:
    # Import path to the prompt function
    import_path: myapp.prompts.code_analysis
    # Optional overrides
    name: "Code Analysis"
    description: "Analyze Python code structure and complexity"
    # Optional message template override
    template: |
      Analyze this code: {code}
      Focus on: {focus}
    # Auto-completion functions for arguments
    completions:
      focus: myapp.prompts.get_analysis_focus_options

# myapp/prompts/code_analysis.py
from typing import Literal

FocusArea = Literal["complexity", "dependencies", "typing"]

def code_analysis(
    code: str,
    focus: FocusArea = "complexity",
    include_metrics: bool = True
) -> list[dict[str, str]]:
    """Analyze Python code structure and complexity.

    Args:
        code: Python source code to analyze
        focus: Analysis focus area (one of: complexity, dependencies, typing)
        include_metrics: Whether to include numeric metrics
    """
    # Function will be converted to a prompt automatically
    ...

def get_analysis_focus_options(current: str) -> list[str]:
    """Provide auto-completion for focus argument."""
    options = ["complexity", "dependencies", "typing"]
    return [opt for opt in options if opt.startswith(current)]

消息内容类型

提示词支持不同的内容类型：

prompts:
  document_review:
    messages:
      # Text content
      - role: system
        content: "You are a document reviewer..."

      # Resource reference
      - role: user
        content:
          type: resource
          content: "document://main.pdf"
          alt_text: "Main document content"

      # Image content
      - role: user
        content:
          type: image_url
          content: "https://example.com/diagram.png"
          alt_text: "System architecture diagram"

参数验证

提示词会在格式化前验证参数：

prompts:
  analyze:
    messages:
      - role: user
        content: "Analyze with level {level}"

    arguments:
      - name: level
        description: "Analysis depth (one of: basic, detailed, full)"
        required: true
        # Will be used for validation and auto-completion
        type_hint: Literal["basic", "detailed", "full"]

工具

工具是可由大模型调用的 Python 函数或类。它们提供了一种安全的方式，可通过自定义功能扩展大模型的能力。

基本工具配置

tools:
  # Function-based tool
  analyze_code:
    import_path: myapp.tools.code.analyze
    description: "Analyze Python code structure and metrics"

  # Class-based tool
  browser:
    import_path: llmling.tools.browser.BrowserTool
    description: "Control web browser for research"

  # Override tool name
  code_metrics:
    import_path: myapp.tools.analyze_complexity
    name: "Analyze Code Complexity"
    description: "Calculate code complexity metrics"

# Include pre-built tool collections
toolsets:
  - llmling.code  # Code analysis tools
  - llmling.web   # Web/browser tools

工具集

顾名思义，工具集是一组工具。目前 LLMling 支持：

• 扩展点系统
• OpenAPI 端点
• 基于类的工具集

基于函数的工具

可以根据任意 Python 函数创建工具：

# myapp/tools/code.py
from typing import Any
import ast

async def analyze(
    code: str,
    include_metrics: bool = True
) -> dict[str, Any]:
    """Analyze Python code structure and complexity.

    Args:
        code: Python source code to analyze
        include_metrics: Whether to include numeric metrics

    Returns:
        Dictionary with analysis results
    """
    tree = ast.parse(code)
    return {
        "classes": len([n for n in ast.walk(tree) if isinstance(n, ast.ClassDef)]),
        "functions": len([n for n in ast.walk(tree) if isinstance(n, ast.FunctionDef)]),
        "complexity": _calculate_complexity(tree) if include_metrics else None
    }

基于类的工具

复杂工具可以通过类来实现：

# myapp/tools/browser.py
from typing import Literal
from playwright.async_api import Page
from llmling.tools.base import BaseTool

class BrowserTool(BaseTool):
    """Tool for web browser automation."""

    name = "browser"
    description = "Control web browser to navigate and interact with web pages"

    ...

    def get_tools(self):
        return [self.open_url, self.click_button]

工具集合（工具集）

将相关工具分组到可重复使用的集合中：

# myapp/toolsets.py
from typing import Callable, Any

def get_mcp_tools() -> list[Callable[..., Any]]:
    """Entry point exposing tools to LLMling."""
    from myapp.tools import (
        analyze_code,
        check_style,
        count_tokens
    )
    return [
        analyze_code,
        check_style,
        count_tokens
    ]

在 pyproject.toml 中：

[project.entry-points.llmling]
tools = "llmling.testing:get_mcp_tools"

工具进度报告

工具可以向客户端报告进度：

from llmling.tools.base import BaseTool

class AnalysisTool(BaseTool):
    name = "analyze"
    description = "Analyze large codebase"

    async def execute(
        self,
        path: str,
        _meta: dict[str, Any] | None = None,  # Progress tracking
    ) -> dict[str, Any]:
        files = list(Path(path).glob("**/*.py"))
        results = []

        for i, file in enumerate(files):
            # Report progress if meta information provided
            if _meta and "progressToken" in _meta:
                self.notify_progress(
                    token=_meta["progressToken"],
                    progress=i,
                    total=len(files),
                    description=f"Analyzing {file.name}"
                )

            results.append(await self._analyze_file(file))

        return {"results": results}

完整工具示例

以下是一个结合多种工具功能的完整示例：

# Configuration
tools:
  # Basic function tool
  analyze:
    import_path: myapp.tools.code.analyze

  # Class-based tool with lifecycle
  browser:
    import_path: myapp.tools.browser.BrowserTool

  # Tool with progress reporting
  batch_analysis:
    import_path: myapp.tools.AnalysisTool

toolsets:
  - llmling.code
  - myapp.tools

# Tool implementation
from typing import Any
from pathlib import Path
from llmling.tools.base import BaseTool

class AnalysisTool(BaseTool):
    """Tool for batch code analysis with progress reporting."""

    name = "batch_analysis"
    description = "Analyze multiple Python files"

    async def startup(self) -> None:
        """Initialize analysis engine."""
        self.analyzer = await self._create_analyzer()

    async def execute(
        self,
        path: str,
        recursive: bool = True,
        _meta: dict[str, Any] | None = None
    ) -> dict[str, Any]:
        """Execute batch analysis.

        Args:
            path: Directory to analyze
            recursive: Whether to analyze subdirectories
            _meta: Optional progress tracking metadata
        """
        files = list(Path(path).glob("**/*.py" if recursive else "*.py"))
        results = []

        for i, file in enumerate(files, 1):
            # Report progress
            if _meta and "progressToken" in _meta:
                self.notify_progress(
                    token=_meta["progressToken"],
                    progress=i,
                    total=len(files),
                    description=f"Analyzing {file.name}"
                )

            # Analyze file
            try:
                result = await self.analyzer.analyze_file(file)
                results.append({
                    "file": str(file),
                    "metrics": result
                })
            except Exception as e:
                results.append({
                    "file": str(file),
                    "error": str(e)
                })

        return {
            "total_files": len(files),
            "successful": len([r for r in results if "metrics" in r]),
            "failed": len([r for r in results if "error" in r]),
            "results": results
        }

    async def shutdown(self) -> None:
        """Clean up analysis engine."""
        await self.analyzer.close()

更多内容：（注意：以下内容大多由 AI 生成且已过时，不保证正确） 介绍 快速示例 用法 yaml_config CLI

资源 提示词 工具 服务 扩展