Databricks Courses

Using Apache Spark for Predicting Degrading and Failing Parts in Aviation

Leveraging Apache Spark for predictive maintenance in aviation, enabling efficient data processing, anomaly detection, and component health scoring to improve fleet readiness and prevent failures.

• YouTube
• 28 minutes
• Self-Paced
• Free Video

Care and Feeding of Catalyst Optimizer - Practical Troubleshooting for Spark SQL

Explore Spark's Catalyst Optimizer challenges, including UDF issues, codegen limitations, and JVM tuning. Learn to diagnose and solve complex query problems for improved Spark SQL performance.

• YouTube
• 42 minutes
• Self-Paced
• Free Video

Faster Data Integration Pipeline Execution Using Spark-Jobserver

Optimize data integration pipelines using Spark-Jobserver for faster execution, context reuse, and efficient resource management. Learn configuration, security, and API integration for improved performance.

• YouTube
• 32 minutes
• Self-Paced
• Free Video

Encryption and Masking for Sensitive Apache Spark Analytics Addressing CCPA and Governance

Practical methods for balancing data encryption and analytics in Apache Spark, addressing CCPA and governance requirements while maintaining analytical value.

• YouTube
• 27 minutes
• Self-Paced
• Free Video

Data Mesh in Practice - From Data Lake to Distributed Architecture at Zalando

Explore Zalando's journey from centralized Data Lake to distributed Data Mesh architecture, focusing on data ownership, quality, and accessibility in large-scale e-commerce environments.

• YouTube
• 30 minutes
• Self-Paced
• Free Video

Running Apache Spark Jobs Using Kubernetes

Learn to run Apache Spark jobs on Kubernetes, unifying analytics and data science on a cloud-native architecture. Discover how this approach simplifies infrastructure, enables on-demand deployment, and eliminates big data cluster overhead.

• YouTube
• 52 minutes
• Self-Paced
• Free Video

Tackling Scaling Challenges of Apache Spark at LinkedIn - Infrastructure Optimization and User Productivity

Explore LinkedIn's journey scaling Apache Spark, addressing infrastructure challenges, optimizing resource management, and enhancing user productivity through automated analysis and efficiency improvements.

• YouTube
• 26 minutes
• Self-Paced
• Free Video

Introducing MLflow for End-to-End Machine Learning on Databricks

Explore end-to-end machine learning with MLflow on Databricks: from data engineering to model deployment, using health data to predict life expectancy through Spark, hyperopt, and interactive dashboards.

• YouTube
• 40 minutes
• Self-Paced
• Free Video

Scaling Security Threat Detection with Apache Spark and Databricks

Explore advanced security threat detection using Apache Spark and Databricks. Learn about innovative solutions for notebook-based testing, self-tuning alerts, automated investigations, and custom SDK development for enhanced cybersecurity.

• YouTube
• 24 minutes
• Self-Paced
• Free Video

Pandas UDFs and Python Type Hints in Apache Spark 3.0

Explore redesigned pandas UDFs in Apache Spark 3.0, leveraging Python type hints for clearer, more Pythonic data science workflows and improved static analysis capabilities.

• YouTube
• 22 minutes
• Self-Paced
• Free Video

Scalable AutoML for Time Series Forecasting Using Ray

Explore scalable AutoML for time series forecasting using Ray, covering feature generation, model selection, and hyper-parameter tuning. Learn from real-world applications and experiences in various industries.

• YouTube
• 22 minutes
• Self-Paced
• Free Video

Accelerating the ML Lifecycle with Enterprise-Grade Feature Stores

Explore enterprise-grade feature store for ML lifecycle acceleration, addressing data engineering challenges, and empowering data scientists to streamline ML application delivery with centralized data management and instant feature serving.

• YouTube
• 29 minutes
• Self-Paced
• Free Video

How Adobe Processes 2 Million Records Per Second Using Apache Spark

Explore Adobe's high-performance data processing using Apache Spark, covering optimization techniques, join strategies, structured streaming, skew handling, and integration with Redis for enhanced efficiency.

• YouTube
• 41 minutes
• Self-Paced
• Free Video

Accelerating Data Processing in Spark SQL with Pandas UDFs - Optimization Techniques

Learn to optimize Spark SQL data processing using Pandas UDFs. Discover techniques to speed up queries, profile jobs efficiently, and leverage custom UDFs for significant performance gains in large-scale data pipelines.

• YouTube
• 27 minutes
• Self-Paced
• Free Video

Productionizing Machine Learning Pipelines with Databricks and Azure ML

Jumpstart ML projects with Databricks and Azure ML. Learn to build, track, and deploy production-ready pipelines using tools like MLflow, TensorFlow, and PyTorch for efficient, reproducible data science workflows.

• YouTube
• 49 minutes
• Self-Paced
• Free Video