model-supply-chain-security

模型供应链安全

保护模型和推理组件，防止篡改、依赖项遭到入侵，以及不可信制品被提升。

何时使用此技能

在以下情况下使用此技能：

• 从公共注册表（Hugging Face、TensorFlow Hub）拉取预训练模型
• 为生产部署构建模型服务容器
• 为跨环境的 ML 制品提升建立信任策略
• 响应影响 ML 依赖项的供应链事件
• 满足 AI 系统的 SLSA 或 SOC2 合规要求

前提条件

• 安装 cosign v2+，用于签名和验证
• 使用 syft 为模型服务镜像生成 SBOM
• 使用 crane 或 skopeo 检查 OCI 镜像
• 使用支持签名的容器注册表（GHCR、ECR、ACR、Artifact Registry）
• 具备生成来源证明能力的 CI/CD 流水线

威胁

• 被投毒的预训练权重或适配器
• 恶意模型转换工具或加载器
• 构建流水线和注册表遭到入侵
• 存在严重 CVE 的不安全运行时镜像
• 模型注册表上的拼写仿冒
• 通过 pickle 或自定义加载器发起的反序列化攻击

控制目标

• 端到端验证制品完整性
• 为每个提升的模型证明其来源
• 在部署前检测存在漏洞的依赖项
• 限制执行范围，仅允许可信的已签名制品

使用 Cosign 对模型签名

对模型制品签名

# Generate a keypair (store private key securely)
cosign generate-key-pair

# Sign an OCI-packaged model image
cosign sign --key cosign.key ghcr.io/acme/ml-models/sentiment:v2.1.0

# Keyless signing with Sigstore (uses OIDC identity)
cosign sign ghcr.io/acme/ml-models/sentiment:v2.1.0

# Verify the signature
cosign verify --key cosign.pub ghcr.io/acme/ml-models/sentiment:v2.1.0

# Keyless verification (requires certificate identity)
cosign verify \
  [email protected] \
  --certificate-oidc-issuer=https://accounts.google.com \
  ghcr.io/acme/ml-models/sentiment:v2.1.0

直接对模型权重文件签名

# For model files stored as blobs (not OCI images)
# Compute digest and sign
sha256sum model-weights.safetensors > model-weights.sha256
cosign sign-blob --key cosign.key model-weights.safetensors \
  --output-signature model-weights.sig \
  --output-certificate model-weights.crt

# Verify blob signature
cosign verify-blob --key cosign.pub \
  --signature model-weights.sig \
  model-weights.safetensors

ML 流水线的 SLSA

模型构建的 SLSA 级别要求

# slsa-requirements.yaml
slsa_levels:
  level_1:
    - Build process is scripted (not manual)
    - Provenance document generated automatically
  level_2:
    - Build runs on hosted CI service
    - Provenance is authenticated (signed)
    - Source is version controlled
  level_3:
    - Build environment is ephemeral and isolated
    - Provenance is non-falsifiable (hardened builder)
    - Source integrity verified (two-person review)

为模型训练生成 SLSA 来源证明

# .github/workflows/model-build-slsa.yml
name: Model Build with SLSA Provenance
on:
  push:
    tags: ['model-v*']

jobs:
  train-and-package:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
      packages: write
    steps:
      - uses: actions/checkout@v4

      - name: Train model
        run: python train.py --config configs/production.yaml

      - name: Package model as OCI artifact
        run: |
          oras push ghcr.io/acme/ml-models/sentiment:${{ github.ref_name }} \
            model-weights.safetensors:application/vnd.acme.model.safetensors \
            model-config.json:application/json

      - name: Generate SBOM for training environment
        run: |
          syft dir:. -o cyclonedx-json > training-sbom.json

      - name: Sign and attest
        run: |
          cosign sign ghcr.io/acme/ml-models/sentiment:${{ github.ref_name }}
          cosign attest --predicate training-sbom.json \
            --type cyclonedx \
            ghcr.io/acme/ml-models/sentiment:${{ github.ref_name }}

      - name: Generate provenance
        uses: slsa-framework/slsa-github-generator/.github/workflows/[email protected]
        with:
          image: ghcr.io/acme/ml-models/sentiment
          digest: ${{ steps.push.outputs.digest }}

用于来源证明的模型卡

# model-card.yaml
model_details:
  name: "sentiment-classifier-v2.1.0"
  version: "2.1.0"
  type: "text-classification"
  framework: "pytorch"
  license: "Apache-2.0"

provenance:
  training_data:
    source: "s3://acme-datasets/sentiment-v3/"
    hash: "sha256:abc123..."
    data_card_ref: "https://internal.acme.com/data-cards/sentiment-v3"
  training_config:
    source: "git://github.com/acme/ml-models@abc123"
    hyperparameters:
      learning_rate: 0.00005
      epochs: 10
      batch_size: 32
  build_environment:
    builder: "github-actions"
    runner: "ubuntu-22.04"
    python: "3.11.7"
    torch: "2.1.2"
    cuda: "12.1"
  build_id: "gh-actions-12345"
  commit_sha: "abc123def456"
  build_timestamp: "2025-01-15T10:30:00Z"
  signed_by: "[email protected]"

performance:
  accuracy: 0.94
  f1_score: 0.93
  evaluation_dataset: "s3://acme-datasets/sentiment-eval-v3/"
  evaluation_hash: "sha256:def456..."

security:
  vulnerability_scan: "clean"
  sbom_ref: "ghcr.io/acme/ml-models/sentiment:v2.1.0.sbom"
  last_security_review: "2025-01-10"
  known_limitations:
    - "May produce biased outputs for underrepresented languages"
    - "Not evaluated for adversarial robustness"

注册表扫描

# Scan model-serving image for CVEs
trivy image ghcr.io/acme/ml-models/sentiment-serving:v2.1.0

# Generate SBOM for the serving container
syft ghcr.io/acme/ml-models/sentiment-serving:v2.1.0 -o spdx-json > serving-sbom.json

# Scan SBOM for vulnerabilities
grype sbom:serving-sbom.json --fail-on critical

# Check for known-malicious model files (pickle scanning)
pip install fickling
fickling --check model.pkl

自动化注册表扫描流水线

# .github/workflows/registry-scan.yml
name: Nightly Registry Scan
on:
  schedule:
    - cron: '0 2 * * *'

jobs:
  scan:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        image:
          - ghcr.io/acme/ml-models/sentiment-serving:latest
          - ghcr.io/acme/ml-models/embedding-serving:latest
          - ghcr.io/acme/ml-models/rag-api:latest
    steps:
      - name: Scan image
        run: |
          trivy image --severity CRITICAL,HIGH \
            --exit-code 1 \
            --format json \
            --output scan-$(echo ${{ matrix.image }} | tr '/:' '-').json \
            ${{ matrix.image }}

      - name: Verify signatures are still valid
        run: |
          cosign verify \
            [email protected] \
            --certificate-oidc-issuer=https://accounts.google.com \
            ${{ matrix.image }}

强制执行提升策略

#!/usr/bin/env python3
"""model_promotion_gate.py - Verify model meets all promotion criteria."""

import subprocess
import json
import sys

def check_signature(image: str) -> bool:
    result = subprocess.run(
        ["cosign", "verify", "[email protected]",
         "--certificate-oidc-issuer=https://accounts.google.com", image],
        capture_output=True, text=True,
    )
    return result.returncode == 0

def check_vulnerabilities(image: str) -> bool:
    result = subprocess.run(
        ["trivy", "image", "--severity", "CRITICAL", "--exit-code", "1",
         "--quiet", image],
        capture_output=True, text=True,
    )
    return result.returncode == 0

def check_sbom_exists(image: str) -> bool:
    result = subprocess.run(
        ["cosign", "verify-attestation", "--type", "cyclonedx",
         "[email protected]",
         "--certificate-oidc-issuer=https://accounts.google.com", image],
        capture_output=True, text=True,
    )
    return result.returncode == 0

def check_model_card(image: str) -> bool:
    result = subprocess.run(
        ["cosign", "verify-attestation", "--type", "custom",
         "[email protected]",
         "--certificate-oidc-issuer=https://accounts.google.com", image],
        capture_output=True, text=True,
    )
    return result.returncode == 0

def main():
    image = sys.argv[1]
    checks = {
        "signature_valid": check_signature(image),
        "no_critical_cves": check_vulnerabilities(image),
        "sbom_attached": check_sbom_exists(image),
        "model_card_present": check_model_card(image),
    }
    all_passed = all(checks.values())
    for name, passed in checks.items():
        status = "PASS" if passed else "FAIL"
        print(f"  [{status}] {name}")
    if not all_passed:
        print("Promotion BLOCKED: not all checks passed.")
        sys.exit(1)
    print("Promotion APPROVED: all checks passed.")

if __name__ == "__main__":
    main()

运行时强化

• 以非 root 用户身份运行推理容器。
• 限制出站流量，防止未经授权的下载。
• 尽可能以只读方式挂载模型卷。
• 对拉取未签名制品的尝试发出警报。
• 使用 safetensors 格式代替 pickle，以防止反序列化攻击。

# kubernetes deployment hardening
apiVersion: apps/v1
kind: Deployment
metadata:
  name: model-serving
spec:
  template:
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        fsGroup: 1000
      containers:
        - name: inference
          image: ghcr.io/acme/ml-models/sentiment-serving:v2.1.0
          securityContext:
            readOnlyRootFilesystem: true
            allowPrivilegeEscalation: false
            capabilities:
              drop: ["ALL"]
          volumeMounts:
            - name: model-weights
              mountPath: /models
              readOnly: true
          resources:
            limits:
              memory: "4Gi"
              nvidia.com/gpu: "1"
      volumes:
        - name: model-weights
          persistentVolumeClaim:
            claimName: model-weights-pvc
            readOnly: true

用于准入控制的 Kyverno 策略

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-signed-model-images
spec:
  validationFailureAction: Enforce
  rules:
    - name: verify-model-image-signature
      match:
        any:
          - resources:
              kinds: ["Pod"]
              namespaces: ["ml-serving"]
      verifyImages:
        - imageReferences: ["ghcr.io/acme/ml-models/*"]
          attestors:
            - entries:
                - keyless:
                    subject: "[email protected]"
                    issuer: "https://accounts.google.com"

故障排查

问题	原因	解决方案
cosign verify 因 "no matching signatures" 失败	镜像推送时未签名	重新运行签名步骤；检查 CI 流水线日志
缺少来源证明声明	未配置 SLSA 生成器	将 slsa-github-generator 添加到构建工作流
Trivy 报告基础镜像中存在 CVE	基础镜像过旧	更新 Dockerfile 中的 FROM 镜像；重新构建并签名
出现 Pickle 反序列化警告	模型以不安全格式保存	转换为 safetensors：model.save_pretrained(".", safe_serialization=True)
无密钥验证失败	OIDC 颁发者或身份不正确	检查 --certificate-identity 和 --certificate-oidc-issuer 标志
找不到制品对应的模型卡	声明未附加到摘要	使用 cosign attest --predicate model-card.yaml --type custom IMAGE 附加

相关技能

• sbom-supply-chain - 生成 SBOM 和来源证明
• container-hardening - 强化运行时容器安全态势
• model-registry-governance - 受控生命周期与审批