mcpField
A
airflow-dag-patterns

airflow-dag-patterns

github.com/sickn33Data & databases48.8k257 installsUpdated today
Repository

About airflow-dag-patterns

A skill for teams using Apache Airflow to design, test, deploy, and troubleshoot production data pipelines. It covers DAG structure alongside operators, sensors, and operational practices.

Key features

  1. 01DAG design and dependencies
  2. 02Custom operators and sensors
  3. 03Local DAG testing
  4. 04Deployment and troubleshooting guidance
  5. 05Idempotency, retries, and observability

Use cases

  1. 01Designing an Airflow data pipeline
  2. 02Building custom operators or sensors
  3. 03Testing DAGs before deployment
  4. 04Investigating failed DAG runs

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/sickn33/agentic-awesome-skills --skill airflow-dag-patterns

After installing, type/airflow-dag-patternsin Claude Code to confirm it loaded.

Bundled filesresources/implementation-playbook.md

Details

From the project README

Apache Airflow DAG Patterns

Production-ready patterns for Apache Airflow including DAG design, operators, sensors, testing, and deployment strategies.

Use this skill when

  • Creating data pipeline orchestration with Airflow
  • Designing DAG structures and dependencies
  • Implementing custom operators and sensors
  • Testing Airflow DAGs locally
  • Setting up Airflow in production
  • Debugging failed DAG runs

Do not use this skill when

  • You only need a simple cron job or shell script
  • Airflow is not part of the tooling stack
  • The task is unrelated to workflow orchestration

Instructions

  1. Identify data sources, schedules, and dependencies.
  2. Design idempotent tasks with clear ownership and retries.
  3. Implement DAGs with observability and alerting hooks.
  4. Validate in staging and document operational runbooks.

Refer to resources/implementation-playbook.md for detailed patterns, checklists, and templates.

Safety

  • Avoid changing production DAG schedules without approval.
  • Test backfills and retries carefully to prevent data duplication.

Resources

  • resources/implementation-playbook.md for detailed patterns, checklists, and templates.

Limitations

  • Use this skill only when the task clearly matches the scope described above.
  • Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
  • Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.

Comments

Markdown supported