Brain Food: Airflow

🧠 Brain Food is a series of expert workshops organized by Dataroots. Here you can find some extra info about the Brain food concept.

🗣 Our very own Nicolas Gelders will be hosting this Airflow workshop.

✔️ What do you need to attend this workshop?

• Basic knowledge of data pipelines, data lake, data warehouse and the data landscape in general
• Intermediate level of Python
• A laptop. You need to install Docker (desktop) and VSCode on your laptop.

🎯 We will show you how to get started with Airflow at your company.

1. Airflow basics

• What isa workflow manager
• What is Airflow? What is Airflow not?
• What is a DAG?
• Airflow Architecture
• Operators, sensors and XCOM
• DAG Scheduling
• How to write a DAG?
• How to interact with Airflow?

2. Airflow advanced topics (hands-on)

• Writing a simple DAG
• Creating an operator and a sensor
• Creating a timetable
• Creating a deferrable sensor
• Dynamic Task Mapping
• Trigger DAG via API and CLI

3. Wrapping up with some best practices, tips and tricks

• Airflow and testing
• RBAC
• Dynamic DAGs (YAMLs)
• Example architectures + different executors
• End2End CICD flow

🍕 The workshop will last 4 hours. We will end the day with pizza!