Databricks Courses

Demystifying Delta Lake - Data Reliability for Data Lakes

Explore Delta Lake's reliability features for data lakes with Apache Spark creator Michael Armbrust. Learn about ACID transactions, metadata handling, and unified data processing.

• YouTube
• 26 minutes
• Self-Paced
• Free Video

Implementing a GAN to Remove People from Images - Databricks Lightning Talk

Learn to implement a pix2pix GAN on Databricks for removing people from images, exploring the basics, idea, demo, and analysis of this innovative image editing technique.

• YouTube
• 30 minutes
• Self-Paced
• Free Video

Taking Machine Learning to Production with MLflow - New Features and Best Practices

Explore MLflow's latest features for productionizing machine learning, including model management, CI/CD, data schemas, and integration with PyTorch for seamless deployment and operation of ML applications.

• YouTube
• 41 minutes
• Self-Paced
• Free Video

SQL Analytics and the Lakehouse Architecture - Data + AI Summit EU 2020 Keynote

Explore lakehouse architecture, Delta Lake, and SQL Analytics for unified data management. Learn how this approach combines data warehouse capabilities with data lake flexibility, enabling faster analytics and decision-making.

• YouTube
• 54 minutes
• Self-Paced
• Free Video

How Apache Spark 3.0 and Delta Lake Enhance Data Lake Reliability

Explore Apache Spark 3.0 and Delta Lake's enhancements for data lake reliability, including ACID transactions, schema enforcement, and time travel. Learn about performance improvements and new features in Databricks Runtime 7.0 Beta.

• YouTube
• 58 minutes
• Self-Paced
• Free Video

Taking Deep Learning to Production with MLflow and RedisAI

Explore MLflow and RedisAI integration for efficient deep learning model deployment, featuring multi-framework support, auto-batching, and DAGing capabilities in a reliable runtime environment.

• YouTube
• 30 minutes
• Self-Paced
• Free Video

User Behavior Hashing for Audience Expansion - Deep Learning Approach

Explore deep learning-based hashing for sequential behavior data, utilizing Spark for large-scale preprocessing and distributed inference on Databricks with Pandas UDF.

• YouTube
• 27 minutes
• Self-Paced
• Free Video

Fugue: Unifying Big Data Analytics Ecosystems for ETL and Machine Learning

Unifying big data ecosystems with Fugue: SQL-like framework for ETL and ML pipelines, compatible with Spark, TensorFlow, and more. Simplifies development, improves performance, and enhances maintainability.

• YouTube
• 22 minutes
• Self-Paced
• Free Video

Understanding and Improving Code Generation in Spark

Explore Spark's code generation techniques, challenges with large queries, and Workday's improvements for handling whole-stage codegen, enhancing performance in production workloads.

• YouTube
• 24 minutes
• Self-Paced
• Free Video

Presto on Apache Spark - A Tale of Two Computation Engines

Explore the integration of Presto and Spark, combining low-latency evaluation with robust execution for enhanced SQL experience in both interactive and batch use cases at Facebook scale.

• YouTube
• 25 minutes
• Self-Paced
• Free Video

From Idea to Model: Productionizing Data Pipelines with Apache Airflow

Learn to productionize data pipelines using Apache Airflow, bridging data science and engineering. Explore collaborative workflows, custom operators, and scalable solutions for efficient model deployment and iteration.

• YouTube
• 22 minutes
• Self-Paced
• Free Video

DataSource V2 and Cassandra - A New Era of Speed and Flexibility

Explore DataSource V2 in Spark Cassandra Connector: enhanced speed, flexibility, and usability. Learn about Spark's understanding of Cassandra clustering and direct catalogue manipulation for improved data integration and analysis.

• YouTube
• 39 minutes
• Self-Paced
• Free Video

Designing the Next Generation of Data Pipelines with Apache Spark - Zillow's Approach

Zillow's data engineering team shares their journey in redesigning data pipelines using Apache Spark, focusing on scalability, maintainability, and robustness while reducing code complexity.

• YouTube
• 27 minutes
• Self-Paced
• Free Video

Apache Pulsar - Next Generation Messaging and Queuing System

Overview of Apache Pulsar's architecture, features, and benefits as a scalable messaging system with unified storage, processing, and serverless capabilities for building end-to-end streaming applications.

• YouTube
• 38 minutes
• Self-Paced
• Free Video

Simplify and Boost Spark 3 Deployments with Hypervisor-Native Kubernetes

Explore integrating Spark 3 with VMware Pacific for scalable, reliable, and high-performance big data analytics deployments using hypervisor-native Kubernetes on vSphere 7.

• YouTube
• 33 minutes
• Self-Paced
• Free Video