Overview

Updated in May 2025. This course now features Coursera Coach! A smarter way to learn with interactive, real-time conversations that help you test your knowledge, challenge assumptions, and deepen your understanding as you progress through the course. This advanced course on Azure Databricks will empower you with the skills to manage complex data workflows efficiently. With a focus on advanced features like Unity Catalog, Delta Tables, and Databricks Ingestion Tools, you will gain hands-on experience in managing large-scale data pipelines, ensuring data consistency, and implementing data governance across the Databricks platform. By the end of the course, you'll have a comprehensive understanding of Databricks' capabilities in data management, equipping you to handle enterprise-level data solutions. The course begins by introducing Unity Catalog, showing how it can be set up and used for managing user access and securing objects in your Databricks environment. You’ll learn how to configure the Unity Catalog and work with various securable objects, ensuring a secure and organized data landscape. As you progress, you will dive deeper into Delta Lake and Delta Tables, starting with an introduction to Delta Lake's features, followed by a thorough exploration of how to create and manage Delta Tables, including reading and optimizing them for performance. In the later modules, you’ll explore Databricks' incremental ingestion tools. You will be introduced to the architecture and use cases of incremental data ingestion, including how to leverage tools like Copy Into and Databricks Autoloader with schema evolution. You’ll also work with streaming data ingestion to ensure real-time data processing with minimal effort. The course concludes with an introduction to Delta Live Tables (DLT), where you’ll learn to create DLT pipelines and workloads using SQL and Python, solidifying your knowledge in streamlining real-time analytics. This course is ideal for experienced data engineers, data architects, and data scientists who want to specialize in Azure Databricks. Prior experience with cloud-based data platforms, SQL, and Python is recommended. With a focus on practical application, this course is designed to take your expertise in data management to the next level.

Syllabus

Working with Unity Catalog

In this module, we will guide you through the foundational concepts and practical steps to work with Unity Catalog in Databricks. You will learn how to set up Unity Catalog, provision users, and manage secure objects within it to ensure seamless data governance and collaboration.

Working with Delta Lake and Delta Tables

In this module, we will explore Delta Lake and its powerful features, including Delta Tables, schema management, and time travel. You’ll also learn how to convert files to Delta format and optimize your tables for better performance and data consistency.

Working with Databricks Incremental Ingestion Tools

In this module, we will delve into Databricks' incremental ingestion tools, covering both manual and automatic schema evolution. You'll also learn how to efficiently ingest streaming data and utilize the Databricks Autoloader for automatic schema management.

Working with Databricks Delta Live Tables (DLT)

In this module, we will explore Databricks Delta Live Tables (DLT), focusing on their practical applications, from setting up datasets to creating DLT workloads in both SQL and Python. You’ll also learn how to build DLT pipelines to manage and process data efficiently.