Google, IBM & Meta Certificates — All 10,000+ Courses at 40% Off
One annual plan covers every course and certificate on Coursera. 40% off for a limited time.
Get Full Access
Discover best practices for unit testing PySpark code in this 19-minute talk by Matthew Powers, Staff Developer Advocate at Databricks. Learn how to create PySpark unit tests that run locally and in CI via GitHub actions, structure PySpark code for easy unit testing, and implement integration tests with a cluster for staging datasets. Gain insights on reducing production bugs, simplifying code refactoring, and enhancing overall code safety. Explore additional resources like the Big Book of Data Engineering and The Data Team's Guide to the Databricks Lakehouse Platform to further expand your knowledge in this field.
