Build Streaming Data Pipelines on Google Cloud

Overview

Design and operate batch data pipelines on Google Cloud using Dataflow, Serverless Spark, Cloud Composer, and data validation techniques.

Explore streaming data pipeline architectures on Google Cloud. This course covers Pub/Sub, Managed Service for Apache Kafka, Dataflow, and BigQuery for real-time data processing. You'll learn architectural considerations and apply your knowledge through an esports streaming use case.

Syllabus

Course introduction

This module introduces the fundamentals of building streaming data pipelines on Google Cloud, providing a foundation for the entire course. It begins by outlining the course's overall learning objectives and introducing a practical, hands-on scenario that will be used throughout the content and labs to make the concepts tangible.

Streaming use cases and reference architectures

This module provides an introduction to streaming data use cases and architectures. You will learn about the applications and common architectural patterns for real-time data processing across four key scenarios: Streaming ETL, Streaming AI/ML, Streaming Application, and Reverse ETL.

Product deep dives

This module provides a comprehensive overview of building streaming data pipelines on Google Cloud, covering the core services for messaging, processing, and analysis. It's designed to give you a hands-on understanding of how these components work together in a cohesive, real-time architecture.

Key takeaways

This module provides a comprehensive wrap-up of the course, summarizing the key concepts you've learned for building resilient and robust streaming data pipelines on Google Cloud.