monorepo-ci-optimizer

Monorepo CI 优化器

在 Monorepo 中仅运行受影响的构建和测试，加快 CI 速度。

受影响范围检测策略

使用 Turborepo

# .github/workflows/ci.yml
name: CI

on:
  pull_request:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0 # Need full history for affected detection

      - uses: actions/setup-node@v4
        with:
          node-version: "20"
          cache: "pnpm"

      - run: pnpm install

      - name: Build affected
        run: |
          pnpm turbo run build --filter='...[origin/main]'

      - name: Test affected
        run: |
          pnpm turbo run test --filter='...[origin/main]'

      - name: Lint affected
        run: |
          pnpm turbo run lint --filter='...[origin/main]'

使用 Nx

build:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0

    - uses: nrwl/nx-set-shas@v4

    - uses: actions/setup-node@v4
      with:
        node-version: "20"
        cache: "npm"

    - run: npm ci

    - name: Build affected projects
      run: npx nx affected:build --base=$NX_BASE --head=$NX_HEAD

    - name: Test affected projects
      run: npx nx affected:test --base=$NX_BASE --head=$NX_HEAD --parallel=3

    - name: Lint affected projects
      run: npx nx affected:lint --base=$NX_BASE --head=$NX_HEAD

远程缓存（Turborepo）

- name: Setup Turborepo cache
  uses: actions/cache@v3
  with:
    path: .turbo
    key: ${{ runner.os }}-turbo-${{ github.sha }}
    restore-keys: |
      ${{ runner.os }}-turbo-

- name: Build with remote cache
  run: pnpm turbo run build
  env:
    TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
    TURBO_TEAM: ${{ secrets.TURBO_TEAM }}

Nx Cloud 集成

- name: Setup Nx Cloud
  run: |
    npx nx-cloud start-ci-run --distribute-on="3 linux-medium-js"
  env:
    NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}

- name: Run affected
  run: |
    npx nx affected --target=build --parallel=3
    npx nx affected --target=test --parallel=3
    npx nx affected --target=lint --parallel=3

手动检测受影响范围

// scripts/get-affected.ts
import * as path from "path";
import { execSync } from "child_process";
import * as fs from "fs";

interface Package {
  name: string;
  path: string;
  dependencies: string[];
}

function getChangedFiles(base: string = "origin/main"): string[] {
  const output = execSync(`git diff --name-only ${base}...HEAD`, {
    encoding: "utf-8",
  });
  return output.split("\n").filter(Boolean);
}

function getPackages(): Package[] {
  const packagesDir = path.join(process.cwd(), "packages");
  return fs.readdirSync(packagesDir).map((name) => {
    const packageJson = JSON.parse(
      fs.readFileSync(path.join(packagesDir, name, "package.json"), "utf-8")
    );
    return {
      name: packageJson.name,
      path: `packages/${name}`,
      dependencies: [
        ...Object.keys(packageJson.dependencies || {}),
        ...Object.keys(packageJson.devDependencies || {}),
      ],
    };
  });
}

function getAffectedPackages(): string[] {
  const changedFiles = getChangedFiles();
  const packages = getPackages();
  const affected = new Set<string>();

  // Direct changes
  changedFiles.forEach((file) => {
    packages.forEach((pkg) => {
      if (file.startsWith(pkg.path)) {
        affected.add(pkg.name);
      }
    });
  });

  // Dependent packages
  let changed = true;
  while (changed) {
    changed = false;
    packages.forEach((pkg) => {
      if (!affected.has(pkg.name)) {
        const hasAffectedDep = pkg.dependencies.some((dep) =>
          affected.has(dep)
        );
        if (hasAffectedDep) {
          affected.add(pkg.name);
          changed = true;
        }
      }
    });
  }

  return Array.from(affected);
}

// Output for GitHub Actions
const affected = getAffectedPackages();
console.log(`::set-output name=packages::${affected.join(",")}`);
console.log(`Affected packages: ${affected.join(", ")}`);

受影响软件包的矩阵策略

detect-affected:
  runs-on: ubuntu-latest
  outputs:
    packages: ${{ steps.affected.outputs.packages }}
  steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0

    - id: affected
      run: |
        PACKAGES=$(node scripts/get-affected.ts)
        echo "packages=$PACKAGES" >> $GITHUB_OUTPUT

test-affected:
  needs: detect-affected
  if: needs.detect-affected.outputs.packages != ''
  runs-on: ubuntu-latest
  strategy:
    matrix:
      package: ${{ fromJSON(needs.detect-affected.outputs.packages) }}
  steps:
    - uses: actions/checkout@v4

    - uses: actions/setup-node@v4
      with:
        node-version: "20"
        cache: "pnpm"

    - run: pnpm install

    - name: Test ${{ matrix.package }}
      run: pnpm --filter ${{ matrix.package }} test

支持工作区的缓存

- name: Cache workspace builds
  uses: actions/cache@v3
  with:
    path: |
      packages/*/dist
      packages/*/node_modules/.cache
    key: ${{ runner.os }}-workspace-${{ hashFiles('**/pnpm-lock.yaml') }}-${{ hashFiles('packages/**/*.ts') }}
    restore-keys: |
      ${{ runner.os }}-workspace-${{ hashFiles('**/pnpm-lock.yaml') }}-
      ${{ runner.os }}-workspace-

并行执行

- name: Build all packages in parallel
  run: pnpm turbo run build --parallel

- name: Test with controlled parallelism
  run: pnpm turbo run test --concurrency=3

优化指标

## Before Optimization

- Full build: 25 minutes
- All tests: 15 minutes
- Total: 40 minutes
- Runs on every PR

## After Affected Detection

- Affected build: 5 minutes (80% reduction)
- Affected tests: 3 minutes (80% reduction)
- Total: 8 minutes (80% reduction)
- Only runs necessary checks

## With Remote Caching

- Cache hit build: 30 seconds (98% reduction)
- Cache hit tests: 1 minute (93% reduction)
- Total: 1.5 minutes (96% reduction)

最佳实践

1. 获取完整历史记录：使用 fetch-depth: 0 确保差异准确
2. 按拓扑顺序执行：先构建依赖项
3. 远程缓存：在不同 CI 运行之间共享缓存
4. 并行执行：并发运行互不依赖的任务
5. 增量构建：仅重新构建发生变更的内容
6. 依赖关系图：跟踪软件包之间的关系
7. 强制完整构建：在合并到主分支时执行

输出检查清单

• 已实现受影响范围检测
• 已配置 Turborepo 或 Nx
• 已启用远程缓存
• 已优化并行执行
• 已为软件包设置矩阵策略
• 已启用支持工作区的缓存
• 已记录依赖关系图
• 已跟踪优化前后的指标