PySpark Courses and Certifications

Coursera

Machine Learning with PySpark: Recommender System

Master building scalable recommendation systems using PySpark and OpenAI embeddings to create personalized product suggestions through K-means clustering and data processing pipelines for e-commerce applications.

• Coursera
• 1 hour 57 minutes
• On-Demand
• Paid Course

Edureka

Machine Learning with PySpark

Master distributed machine learning using PySpark MLlib to build scalable models, process large datasets, and implement advanced algorithms for classification, regression, and clustering in a big data environment.

• Coursera
• 13 hours 27 minutes
• On-Demand
• Paid Course

EDUCBA

PySpark: Apply & Evaluate Predictive ML Models

Master PySpark's MLlib to build, evaluate, and deploy scalable regression, classification, and clustering models using distributed computing for real-world data analytics.

• Coursera
• 5 hours 15 minutes
• On-Demand
• Paid Course

Edureka

Data Streaming and NLP with PySpark

Master real-time data processing and natural language analysis using PySpark's distributed computing capabilities for streaming applications and NLP tasks with hands-on implementation experience.

• Coursera
• 17 hours 41 minutes
• On-Demand
• Paid Course

PySpark Tutorial

1 review

Learn PySpark, an interface for Apache Spark in Python. PySpark is often used for large-scale data processing and machine learning.

• freeCodeCamp
• 1-2 hours
• On-Demand
• Free Video

Databricks

Architecting for Data Quality in the Lakehouse with Delta Lake and PySpark

Learn techniques for ensuring data quality in lakehouses using Delta Lake and PySpark. Explore data observability, reliability lifecycle, and practical implementation of quality checks across metadata, storage, and query layers.

• YouTube
• 1 hour 1 minute
• On-Demand
• Free Video

Databricks

From Python to PySpark and Back Again - Unifying Single-Host and Distributed Deep Learning with Maggy

Unifying single-host and distributed deep learning with Maggy framework, enabling seamless transition from laptop to cluster for efficient model development and training using TensorFlow and PySpark.

• YouTube
• 28 minutes
• On-Demand
• Free Video

Databricks

PySpark in Apache Spark 3.3 and Beyond

Explore PySpark's evolution in Apache Spark 3.3, including improved Pandas API, enhanced productivity features, and future developments. Learn about new functionalities, usability improvements, and performance enhancements.

• YouTube
• 35 minutes
• On-Demand
• Free Video

Open Data Science

PySpark Best Practices

Learn best practices for PySpark, including dependency management, code structuring, testing, and performance optimization. Gain insights on writing efficient and scalable Python applications using Spark's distributed computing model.

• YouTube
• 46 minutes
• On-Demand
• Free Video

NashKnolX

Introduction to PySpark

Learn the fundamentals of PySpark for efficient big data processing and analytics using Python in Apache Spark's distributed computing environment.

• YouTube
• 43 minutes
• On-Demand
• Free Video

Databricks

Rapid PySpark Custom Processing on Time Series Big Data in Databricks

Efficient PySpark processing for time series big data, enabling rapid custom calculations on rolling windows with constant time complexity, demonstrated using Sleep Number Smartbed sensor data.

• YouTube
• 28 minutes
• On-Demand
• Free Video

Databricks

Exploring UDTFs and Polymorphism in PySpark

Learn to create and enhance User-Defined Table Functions in PySpark, including polymorphic UDTFs for flexible data processing. Gain practical insights through examples and expert guidance.

• YouTube
• 20 minutes
• On-Demand
• Free Video

Databricks

Best Practices for Unit Testing PySpark

Learn effective techniques for unit testing PySpark code, including local and CI testing, code structuring for testability, and integration testing with clusters for enhanced reliability and easier refactoring.

• YouTube
• 19 minutes
• On-Demand
• Free Video

Databricks

What's New in PySpark - TVFs, Subqueries, Plots, and Profilers

Explore PySpark's latest features including table-valued functions, subqueries, lateral joins, plotting tools, and profiling capabilities for enhanced data workflows.

• YouTube
• 39 minutes
• On-Demand
• Free Video

Databricks

Creating a Custom PySpark Stream Reader with PySpark 4.0

Discover how to build custom PySpark stream readers in 4.0 to directly consume data from unsupported sources like ActiveMQ, eliminating middleware complexity for streamlined pipelines.

• YouTube
• 28 minutes
• On-Demand
• Free Video