Free courses from frontend to fullstack and AI
Python, Prompt Engineering, Data Science — Build the Skills Employers Want Now
Overview
Google, IBM & Meta Certificates — All 10,000+ Courses at 40% Off
One annual plan covers every course and certificate on Coursera. 40% off for a limited time.
Get Full Access
Embark on a comprehensive data engineering journey with this full course that combines Apache Airflow, Docker, Spark Clusters, Scala, Python, and Java. Create an end-to-end project, developing basic jobs in multiple programming languages, submitting them to a Spark cluster for processing, and observing live results. Learn to set up a Spark Cluster and Airflow on Docker, create Spark jobs using Python, Scala, and Java, build and compile Scala and Java jobs, and analyze cluster computation results. Gain hands-on experience in workflow automation, big data analytics, and data processing techniques essential for aspiring data engineers.
Syllabus
Introduction
Creating The Spark Cluster and Airflow on Docker
Creating Spark Job with Python
Creating Spark Job with Scala
Building and Compiling Scala Jobs
Creating Spark Job with Java
Building and Compiling Java Jobs
Cluster computation results
Taught by
CodeWithYu