In this series, a full Extract, Load, Transform pipeline is built from the ground up, using Airflow 2.0’s newest and straightforward syntax, and leveraging the flexibility of the Raspberry Pi.

Content

  1. Airflow 2.0 installation
  2. A simple DAG
  3. Getting started with Task Groups
  4. Extract and Load data
  5. A full ETL pipeline
  6. BONUS: Transforming data in Apache Spark