Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Zero To Mastery

Fundamentals of Apache Airflow

via Zero To Mastery Path

Go to class Write review

Overview

Automate your data pipelines with Apache Airflow. This hands-on course starts with the basics and builds up to real-world orchestration, from retries to Spark and external data ingestion.
  • How to build and schedule workflows with Apache Airflow
  • Core concepts of DAGs, tasks, operators, and scheduling
  • Error handling, retries, and making workflows fault-tolerant
  • Ensuring idempotency and robustness in data pipelines
  • Using sensors to wait for external systems or events
  • Orchestrating Apache Spark jobs within Airflow
  • Connecting Airflow with external data sources
  • Ingesting data into a data lake with automation

Syllabus

  •   Introduction
    • Introduction
    • What Is Apache Airflow?
    • Airflow’s Architecture
    • Exercise: Meet Your Classmates and Instructor
    • Course Resources
  •   Setup and Useful Resources
    • [Optional] UNIX CLI Commands
    • [Optional] Using Windows
    • Installing Software for the Course
    • [Optional] What Is a Virtualenv?
    • [Optional] What Is Docker?
  •   [Optional] Apache Spark Introduction
    • Installing Spark
    • Installing Spark on Linux
  •   Implementing Data Pipelines with Apache Airflow
    • Installing Airflow
    • Defining an Airflow DAG
    • Errors Handling
    • Idempotent Tasks
    • Creating a DAG - Part 1
    • Creating a DAG - Part 2
    • Handling Failed Tasks
    • [Exercise] Data Validation
    • [Exercise] Data Validation - Solution
    • Spark with Airflow
    • Using Spark with Airflow - Part 1
    • Using Spark with Airflow - Part 2
    • Sensors In Airflow
    • Using File Sensors
    • Data Ingestion
    • Reading Data From Postgres - Part 1
    • Reading Data from Postgres - Part 2
    • [Exercise] Average Customer Review
    • [Exercise] Average Customer Review - Solution
    • Advanced DAGs
  •   Where To Go From Here?
    • Let's Keep Learning Together!
    • Review This Byte!

Taught by

Ivan Mushketyk

Reviews

Start your review of Fundamentals of Apache Airflow

Go to class

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free
Someone learning on their laptop while sitting on the floor.