Master the blend of SQL with PySpark to run complex queries and joins. Utilize User Defined Functions to enhance functionality, empowering you to extract meaningful insights from your data analysis workflow with ease and precision.
Overview
Syllabus
- Unit 1: Getting Started with PySpark SQL
- Query Data with SQL and PySpark
- Modify Temporary View and Query
- Fix PySpark SQL Query Execution
- Fetch Data with PySpark SQL
- Unit 2: Understanding Operations and Execution Plans
- Fill in the Missing Method
- Switching Data Formats in PySpark
- Switching Queries for New Insights
- Completing the PySpark SQL Script
- Unit 3: Exploring SQL Joins Through PySpark
- Complete Your SQL Join Query
- Fix Errors in SQL Join Code
- Exploring Left Joins in PySpark
- Write the SQL Join with PySpark
- Unit 4: Utilizing User Defined Functions in SQL with PySpark
- Apply UDF Integration in SQL
- Fix the SQL UDF Bug
- Implementing Dynamic UDF in PySpark
- Master UDFs in PySpark