Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Google

Building Realtime Pipelines in Cloud Data Fusion

Google via Google Skills

Go to class Write review
CodeSignal Ad

Introduction to Programming with Python

Learn More → Boot.dev Ad

The Most Addictive Python and SQL Courses

Learn More →

Overview

Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
In addition to batch pipelines, Data Fusion also allows you to create realtime pipelines, that can process events as they are generated. Currently, realtime pipelines execute using Apache Spark Streaming on Cloud Dataproc clusters. In this lab you you will learn how to build a streaming pipeline using Data Fusion.

Syllabus

  • GSP808
  • Overview
  • Setup and requirements
  • Task 1. Project permissions
  • Task 2. Ensure that the Dataflow API is successfully enabled
  • Task 3. Load the data
  • Task 4. Setting up Pub/Sub Topic
  • Task 5. Add a Pub/Sub subscription
  • Task 6. Add necessary permissions for your Cloud Data Fusion instance
  • Task 7. Navigate the Cloud Data Fusion UI
  • Task 8. Build a realtime pipeline
  • Task 9. Send messages into Cloud Pub/Sub
  • Task 10. Viewing your pipeline metrics
  • Congratulations!

Tags

Reviews

Start your review of Building Realtime Pipelines in Cloud Data Fusion

Go to class

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free
Someone learning on their laptop while sitting on the floor.