- Introduction
-
Welcome! 0 hr 1 min
- DAG Versioning
-
Why DAG Versioning? 0 hr 4 min
-
What is DAG Versioning? 0 hr 5 min
- DAG Bundles
-
What is a DAG Bundle? 0 hr 4 min
-
Configure DAG Bundles 0 hr 3 min
- Finishing up...
-
Quiz!
-
Summary
-
How was it?
Airflow: DAG Versioning and Bundles
Learn to version your DAGs for tracking changes and never get lost again!
Welcome! We're so glad you're here 😍
With DAG versioning and DAG bundles, Airflow 3.0 revolutionizes how you manage code changes and track DAG evolution over time.
DAG versioning automatically tracks structural changes to your DAGs, creating new versions when modifications occur, while DAG bundles provide flexible ways to organize and deploy your DAGs from multiple sources like Git repositories and local file systems.
This feature eliminates the historical pain points of lost execution context and unpredictable behavior during code updates, making Airflow production-ready with proper version control and historical accuracy.
Therefore, you can now confidently manage DAG changes, rerun historical executions with their original code, and organize your data pipelines across different repositories while maintaining complete traceability.
🎯Objectives
At the end of this course, you'll be able to:
- Understand the problems DAG versioning solves in Airflow 3.0
- Differentiate between versioned and unversioned DAG bundles
- Configure Git DAG bundles for production environments
- Implement multiple DAG bundles to pull from different sources
- Leverage DAG versioning for reliable historical DAG execution and debugging
👥 Audience
Who should take this course:
- Data Engineers managing production Airflow deployments
- DevOps Engineers responsible for Airflow infrastructure
- Data Platform Engineers building robust data pipelines
- Software Engineers working with Airflow in production environments
Set aside 25 minutes to complete the course.
💻 Setup Requirements
You need to have the following:
- Docker and Docker compose on your computer (cf: get Docker)
- The Astro CLI
- Access to a web browser
- A GitHub account (for Git DAG bundle configuration)
- Basic familiarity with Airflow DAGs and Git version control