Data Pipelines Courses and Certifications

Databricks

From Idea to Model: Productionizing Data Pipelines with Apache Airflow

Learn to productionize data pipelines using Apache Airflow, bridging data science and engineering. Explore collaborative workflows, custom operators, and scalable solutions for efficient model deployment and iteration.

• YouTube
• 22 minutes
• Self-Paced
• Free Video

Databricks

Modern ETL Pipelines with Change Data Capture - Building Resilient Data Streams

Discover how GetYourGuide built a resilient ETL pipeline using Debezium, Kafka, Spark, and Airflow. Learn to handle schema changes automatically, stream data from SQL databases, and refresh data lakes multiple times daily for fresher insights.

• YouTube
• 43 minutes
• Self-Paced
• Free Video

Databricks

Best Practices for Building and Deploying Data Pipelines in Apache Spark

Learn best practices for building and deploying efficient, reproducible data pipelines in Apache Spark. Discover a toolkit that simplifies pipeline creation for non-engineers and explore deployment strategies.

• YouTube
• 41 minutes
• Self-Paced
• Free Video

Databricks

Simplify and Scale Data Engineering Pipelines with Delta Lake

Learn to build scalable data engineering pipelines using Delta Lake's multi-hop architecture. Discover how to progress from raw data ingestion to machine learning-ready datasets efficiently and effectively.

• YouTube
• 38 minutes
• Self-Paced
• Free Video

GOTO Conferences

Data Pipelines à La Mode

Explore pure data pipelines: reproducible, version-tracked, and efficient. Learn techniques from distributed build systems to improve algorithm updates and avoid redundant work.

• YouTube
• 48 minutes
• Self-Paced
• Conference Talk

Databricks

Delta Live Tables in Depth - Best Practices for Intelligent Data Pipelines

Explore best practices, new features, and future developments for Delta Live Tables with Databricks experts. Learn pipeline development, data quality, and serverless compute for intelligent data pipelines.

• YouTube
• 1 hour 8 minutes
• Self-Paced
• Free Video

EuroPython Conference

Modern Data Science with Vaex - A New Approach to DataFrames and Pipelines

Explore efficient data processing for large datasets using Vaex, combining computational graphs and lazy evaluations with memory-mapped storage and out-of-core algorithms.

• YouTube
• 51 minutes
• Self-Paced
• Conference Talk

ACCU Conference

Powerful Data Processing Pipelines with C++20 Coroutines

Explore building efficient data processing pipelines using C++20 coroutines. Learn composable interfaces, concurrent execution, and hands-on examples for simplified yet powerful data processing systems.

• YouTube
• 1 hour 27 minutes
• Self-Paced
• Conference Talk

Rust

Serverless Data Pipelines in Rust - Rust Vienna Meetup

Explore building serverless data pipelines in Rust using DataFusion and object store, with deployment on AWS Lambda. Learn scalable techniques for complex pipelines in production environments.

• YouTube
• 41 minutes
• Self-Paced
• Free Video

Shaw Talebi

Automating Data Pipelines with Python and GitHub Actions - Code Walkthrough

Automate data pipelines using Python and GitHub Actions. Learn to create ETL scripts, set up repositories, configure workflows, and implement ML applications for efficient data processing.

• YouTube
• 31 minutes
• Self-Paced
• Free Video

Shaw Talebi

How to Build Data Pipelines for ML Projects with Python Code

Learn to build data pipelines for machine learning projects using Python. Explore ETL vs ELT, extraction, transformation, loading, and orchestration. Includes practical example of processing YouTube video transcripts.

• YouTube
• 23 minutes
• Self-Paced
• Free Video

SQLBits

Declarative ETL Pipelines with Delta Live Tables - Modern Software Engineering for Data Analysts and Engineers

Modernize ETL pipelines using Delta Live Tables for efficient data management. Learn development, production, and best practices for reliable, testable, and scalable data processing.

• YouTube
• 51 minutes
• Self-Paced
• Free Video

Confluent

Data Pipelines Evolution from Batch to Streaming

Explore the evolution from batch to streaming data pipelines using Apache Flink and Kafka. Learn strategies for seamless transition, including query-based connectors, change data capture, and handling late events.

• YouTube
• 41 minutes
• Self-Paced
• Free Video

Google Cloud Tech

Build Continuous Data and AI Pipelines with BigQuery Continuous Queries

Revolutionize data engineering with BigQuery continuous queries for real-time AI insights. Explore event-driven pipelines using Vertex AI, Pub/Sub, and Bigtable through Cloud SQL.

• YouTube
• 41 minutes
• Self-Paced
• Free Video

InfoQ

Streaming from Apache Iceberg - Building Low-Latency and Cost-Effective Data Pipelines

Discover how to build efficient data pipelines using Apache Iceberg streaming, comparing Kafka and Iceberg sources while exploring stream processing implementations for low-latency, cost-effective solutions.

• YouTube
• 49 minutes
• Self-Paced
• Free Video