Azure Spark Databricks Essential Training

Introduction

• Optimize data pipelines
• What you should know
• About using cloud services

1. Big Data on Azure Databricks

• Meet Databricks Apache Spark clusters
• Business scenarios for Spark
• Understand Spark key components
• Azure Databricks concepts
• Quick start: Use a notebook
• Set up Databricks AI Playground
• Use Databricks AI Playground

2. Core Azure Databricks Workloads

• Review Databricks Azure cluster setup
• Use a Python notebook with dashboards
• Use an R notebook
• Use a Scala notebook for visualization
• Use a notebook with scikit-learn
• Use a Spark Streaming notebook
• Use an external Scala library: variant-spark

3. Scaling Azure Databricks Workloads

• Understand data engineering workload steps
• Understand cluster configurations
• Understand Spark job execution overhead
• Explore optimization control planes
• Optimize a cluster and job
• Run a production-size job

4. Data Pipelines with Azure Databricks

• Use Databricks jobs and role-based control
• Use Databricks Runtime ML
• Understand ML Pipelines API
• Use ML Pipelines API
• Use distributed ML training
• Understand Databricks Delta
• Use Databricks Delta
• Use Azure Blob storage
• Understand MLflow

5. Machine Learning Architectures

• Azure Databricks pipeline considerations
• Azure Databricks for data warehousing
• Azure Databricks and machine learning
• Azure Databricks for churn analysis
• Azure Databricks for intrusion detection

Conclusion

• Next steps