What you'll learn:
- You will learn how to build a real world data project using Azure Databricks and Spark Core. This course has been taught using real world data.
- You will acquire professional level data engineering skills in Azure Databricks, Delta Lake, Spark Core, Azure Data Lake Gen2 and Azure Data Factory (ADF)
- You will learn how to create notebooks, dashboards, clusters, cluster pools and jobs in Azure Databricks
- You will learn how to ingest and transform data using PySpark in Azure Databricks
- You will learn how to transform and analyse data using Spark SQL in Azure Databricks
- You will learn about Data Lake architecture and Lakehouse Architecture. Also, you will learn how to implement a Lakehouse architecture using Delta Lake.
- You will learn how to build and orchestrate data pipelines using Lakeflow Jobs in Databricks.
- You will learn how to implement incremental data processing using Delta Lake.
- You will gain a comprehensive understanding of Unity Catalog and how it is used to organise and manage data in Databricks.
- You will gain practical experience working with modern Databricks features and best practices used in real-world data engineering projects.
- You will build practical skills that support certifications such as Databricks Certified Data Engineer Associate and Databricks Certified Associate Developer fo
- You will strengthen your understanding of key concepts commonly tested in Databricks and Azure Databricks certification exams.
Course Fully Refreshed for 2026
This course has been completely rebuilt for 2026 using the latest Azure Databricks features and best practices.
Instead of relying on legacy approaches such as Hive Metastore and external orchestration tools, this course focuses on modern Databricks capabilities like Unity Catalog, Lakeflow Jobs, Databricks SQL Dashboards, and Genie.
Welcome!
In this course, you will build a complete end-to-end data engineering project using Azure Databricks and Apache Spark based on Formula 1 Motor Racing data.
You won’t just learn individual concepts. You will design and implement a cloud data platform from scratch, following the same approach used in real-world data engineering and data platform projects.
What You Will Build
Throughout the course, you will:
Design a modern Data Lakehouse architecture using Azure Databricks
Implement the Medallion Architecture (Bronze, Silver, Gold) for scalable data pipelines
Ingest, transform, and model data using Apache Spark (PySpark and Spark SQL)
Store and manage data using Delta Lake in Databricks
Organise and govern data using Unity Catalog in Azure Databricks
Build and orchestrate pipelines using Lakeflow Jobs in Databricks
Create analytical views and dashboards using Databricks SQL and Dashboards
Enhance the pipeline with incremental data processing using Delta Lake
By the end of the course, you will have built a production-ready data engineering pipeline on Azure Databricks.
Technologies You Will Use
As part of building the project, you will learn:
Azure Databricks
Apache Spark using PySpark and Spark SQL
Delta Lake and modern Lakehouse architecture
Unity Catalog for data governance and organisation in Databricks
Databricks SQL and Dashboards for analytics and reporting
How You Will Learn
This is a hands-on, project-based Azure Databricks course.
You will build the solution step by step
Concepts are explained in the context of a real-world project
Each section builds on the previous one
This approach ensures that you not only understand the concepts, but also know how to apply them in real-world data engineering scenarios.
I value your time as much as I do mine. So, I’ve designed this course to be focused, practical, and to the point. The lessons are explained in simple English, without unnecessary jargon, and we start from the basics. By the end of the course, you will be confident building real-world data engineering solutions.
How This Course Supports Certification Preparation
This course can help you build many of the core skills required for the following certifications:
Databricks Certified Data Engineer Associate
Databricks Certified Associate Developer for Apache Spark
Microsoft Exam DP-750: Implementing Data Engineering Solutions Using Azure Databricks
Databricks Certified Data Engineer Professional
The hands-on project will strengthen your practical understanding of key Databricks and Spark concepts tested in these exams.
However, this course is not designed as a certification preparation course and does not cover all exam topics.
What’s Included (and What’s Not)
This course focuses on core Spark and Databricks concepts
It does not cover Spark Streaming, Spark ML, and Lakeflow Declarative Pipelines
Spark is taught using PySpark and Spark SQL (not Scala or Java)
Final Outcome
By the end of this course, you will have built a complete, production-ready data engineering solution using Azure Databricks and Spark, and gained the confidence to apply these skills in real-world projects.
