frappe-testing-cicd

CI/CD Pipelines

Quick Reference

Task	Tool / File
Install pre-commit hooks	pre-commit install --hook-type pre-commit --hook-type commit-msg
Run all pre-commit checks	pre-commit run --all-files
Run linter	ruff check .
Run formatter	ruff format .
Run ESLint	npx eslint "**/*.js" --quiet
Run tests in CI	bench --site test_site run-tests --app myapp
Run parallel tests	bench --site test_site run-parallel-tests --total-builds 2 --build-number 0
Generate coverage	coverage run -m pytest && coverage xml
Generate JUnit XML	bench --site test_site run-tests --junit-xml-output report.xml

Decision Tree: CI/CD Setup

Setting up CI for a Frappe app?
├─ Start with GitHub Actions workflow
│   ├─ ALWAYS include MariaDB + Redis services
│   ├─ ALWAYS use test matrix for Python versions
│   └─ Optionally add PostgreSQL for dual-DB support
├─ Add pre-commit hooks
│   ├─ ALWAYS include ruff (Python linting + formatting)
│   ├─ ALWAYS include eslint + prettier (JS/Vue)
│   └─ Add commitlint for conventional commits
├─ Add security scanning?
│   └─ YES → Add semgrep with Frappe-specific rules
└─ Need release automation?
    └─ YES → Add tag-based release workflow

GitHub Actions Workflow for Frappe Apps

Standard Server Test Workflow

name: Server Tests
on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main, develop]

concurrency:
  group: server-tests-${{ github.ref }}
  cancel-in-progress: true

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 60

    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.11", "3.12"]
        db: ["mariadb"]

    services:
      mariadb:
        image: mariadb:11.4
        ports:
          - 3306:3306
        env:
          MARIADB_ROOT_PASSWORD: db_root
        options: >-
          --health-cmd="healthcheck.sh --connect --innodb_initialized"
          --health-interval=5s
          --health-timeout=5s
          --health-retries=10

      redis-cache:
        image: redis:alpine
        ports:
          - 13000:6379
      redis-queue:
        image: redis:alpine
        ports:
          - 11000:6379

    steps:
      - name: Checkout frappe
        uses: actions/checkout@v4
        with:
          repository: frappe/frappe
          path: frappe-bench/apps/frappe

      - name: Checkout app
        uses: actions/checkout@v4
        with:
          path: frappe-bench/apps/myapp

      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: 20

      - name: Install bench
        run: pip install frappe-bench

      - name: Init bench
        working-directory: frappe-bench
        run: |
          bench init --skip-assets --skip-redis-config-generation .
          bench set-config -g db_root_password db_root
          bench set-config -g redis_cache redis://localhost:13000
          bench set-config -g redis_queue redis://localhost:11000

      - name: Install app
        working-directory: frappe-bench
        run: |
          bench get-app --skip-assets myapp ./apps/myapp
          bench setup requirements --dev
          bench new-site test_site \
            --db-root-password db_root \
            --admin-password admin \
            --no-mariadb-socket
          bench --site test_site install-app myapp
          bench build --apps myapp

      - name: Run tests
        working-directory: frappe-bench
        run: bench --site test_site run-tests --app myapp --failfast

      - name: Upload coverage
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.db }}
          path: frappe-bench/sites/coverage.xml

PostgreSQL Support (Additional Matrix Entry)

    strategy:
      matrix:
        include:
          - python-version: "3.12"
            db: "postgres"

    services:
      postgres:
        image: postgres:16
        ports:
          - 5432:5432
        env:
          POSTGRES_PASSWORD: db_root
        options: >-
          --health-cmd pg_isready
          --health-interval=10s
          --health-timeout=5s
          --health-retries=5

When using PostgreSQL, change the bench new-site command:

bench new-site test_site \
  --db-type postgres \
  --db-root-password db_root \
  --admin-password admin

Pre-Commit Configuration

Minimal .pre-commit-config.yaml for Frappe Apps

exclude: "node_modules|.git"
default_stages: [pre-commit]
fail_fast: false

repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.6.0
    hooks:
      - id: trailing-whitespace
        files: "myapp.*"
        exclude: ".*json$|.*txt$|.*csv$|.*md$|.*svg$"
      - id: check-merge-conflict
      - id: check-ast
      - id: check-json
      - id: check-toml
      - id: check-yaml
      - id: debug-statements

  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.8.0
    hooks:
      - id: ruff
        args: [--select=I, --fix]
        name: ruff (import sorter)
      - id: ruff
        name: ruff (linter)
      - id: ruff-format
        name: ruff (formatter)

  - repo: https://github.com/pre-commit/mirrors-prettier
    rev: v2.7.1
    hooks:
      - id: prettier
        types_or: [javascript, vue, scss]
        exclude: ".*dist.*|node_modules"

  - repo: https://github.com/pre-commit/mirrors-eslint
    rev: v8.44.0
    hooks:
      - id: eslint
        types: [javascript]
        args: [--quiet]
        exclude: ".*dist.*|node_modules"

  - repo: https://github.com/alessandrojcm/commitlint-pre-commit-hook
    rev: v9.16.0
    hooks:
      - id: commitlint
        stages: [commit-msg]
        additional_dependencies:
          - conventional-changelog-conventionalcommits

ALWAYS run pre-commit install --hook-type pre-commit --hook-type commit-msg after cloning.

Ruff Configuration (pyproject.toml)

[tool.ruff]
line-length = 110
target-version = "py311"

[tool.ruff.lint]
select = [
    "F",   # Pyflakes
    "E",   # pycodestyle errors
    "W",   # pycodestyle warnings
    "I",   # isort
    "UP",  # pyupgrade
    "B",   # flake8-bugbear
    "RUF", # Ruff-specific rules
]
ignore = [
    "E501",  # line too long (handled by formatter)
    "F401",  # unused import (common in __init__.py)
    "F403",  # wildcard import (Frappe convention)
    "F405",  # undefined from wildcard (Frappe convention)
    "E402",  # module-level import order (Frappe convention)
]

[tool.ruff.format]
quote-style = "double"
indent-style = "tab"
docstring-code-format = true

[tool.ruff.lint.per-file-ignores]
# Ignore in auto-generated boilerplate
"**/doctype/**/boilerplate/**" = ["ALL"]

Rules:

• ALWAYS use indent-style = "tab" for Frappe projects — this is the framework convention
• ALWAYS ignore F401/F403/F405 — Frappe uses wildcard imports by convention
• NEVER set line-length below 110 — Frappe standard is 110 characters

ESLint Configuration

{
    "env": {
        "browser": true,
        "node": true,
        "es2021": true
    },
    "extends": "eslint:recommended",
    "globals": {
        "frappe": "readonly",
        "cur_frm": "readonly",
        "__": "readonly",
        "cur_dialog": "readonly",
        "cur_page": "readonly",
        "cur_list": "readonly"
    },
    "rules": {
        "no-unused-vars": ["warn", { "argsIgnorePattern": "^_" }],
        "no-console": "warn"
    }
}

ALWAYS declare Frappe globals (frappe, cur_frm, __, etc.) — otherwise ESLint reports false positives.

Semgrep Security Rules

# .semgrep/frappe-security.yml
rules:
  - id: frappe-sql-injection
    pattern: frappe.db.sql($X.format(...))
    message: "NEVER use .format() in SQL — use parameterized queries"
    severity: ERROR
    languages: [python]

  - id: frappe-raw-sql-concat
    pattern: frappe.db.sql($X + ...)
    message: "NEVER concatenate strings in SQL — use parameterized queries"
    severity: ERROR
    languages: [python]

  - id: frappe-eval-usage
    pattern: eval(...)
    message: "NEVER use eval() — use frappe.safe_eval() instead"
    severity: ERROR
    languages: [python]

Add to CI:

      - name: Run Semgrep
        uses: returntocorp/semgrep-action@v1
        with:
          config: .semgrep/

Test Coverage

Setup coverage.py

# .coveragerc
[run]
source = myapp
omit =
    */test_*.py
    */tests/*
    */setup.py

[report]
exclude_lines =
    pragma: no cover
    if frappe.flags.in_test:
    if TYPE_CHECKING:

CI Coverage Step

      - name: Run tests with coverage
        working-directory: frappe-bench
        run: |
          cd apps/myapp
          coverage run -m pytest
          coverage xml -o ../../sites/coverage.xml

      - name: Upload to Codecov
        uses: codecov/codecov-action@v4
        with:
          file: frappe-bench/sites/coverage.xml
          token: ${{ secrets.CODECOV_TOKEN }}

Release Workflow

name: Release
on:
  push:
    tags:
      - "v*"

jobs:
  release:
    runs-on: ubuntu-latest
    permissions:
      contents: write
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Generate changelog
        id: changelog
        uses: mikepenz/release-changelog-builder-action@v4
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v2
        with:
          body: ${{ steps.changelog.outputs.changelog }}
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Branch Protection Rules

ALWAYS configure these branch protection rules for main and develop:

1. Require status checks: Server Tests must pass before merging
2. Require pull request reviews: At least 1 approval
3. Require up-to-date branches: Force rebase before merge
4. Require linear history: Enforce squash or rebase merging

Configure via GitHub CLI:

gh api repos/{owner}/{repo}/branches/main/protection -X PUT \
  -f "required_status_checks[strict]=true" \
  -f "required_status_checks[contexts][]=Server Tests" \
  -f "required_pull_request_reviews[required_approving_review_count]=1"

Common CI Failures and Fixes

Failure	Cause	Fix
MariaDB not ready	Health check too short	Increase --health-retries to 10+
Redis connection refused	Wrong port mapping	Verify port mapping matches bench set-config
ModuleNotFoundError	Missing app install	Ensure bench get-app and bench install-app both run
Site not found	Missing bench new-site	ALWAYS create site before running tests
Permission denied on bench	pip install location	Use pip install frappe-bench without sudo
Assets build failed	Node version mismatch	Use Node 18 or 20 (NEVER Node 16)
frappe.exceptions.DoesNotExistError	Missing test fixtures	Ensure test_records.json exists for dependent DocTypes
Timeout in parallel tests	Too many parallel builds	Reduce --total-builds or increase timeout-minutes

See Also

• references/examples.md — Complete workflow examples
• references/anti-patterns.md — CI/CD mistakes to avoid
• references/github-actions.md — Full GitHub Actions reference
• references/linting.md — Linting and formatting deep dive
• frappe-testing-unit — Unit and integration testing