Orchestrate ML Workflows with Vertex AI Pipelines

Overview

Google, IBM & Meta Certificates — All 10,000+ Courses at 40% Off

One annual plan covers every course and certificate on Coursera. 40% off for a limited time.

Discover how to orchestrate ML workflows on Google Cloud. Explore the business drivers for orchestration and the technical architecture of Vertex AI Pipelines. Learn to create MLOps pipelines using a flexible, hybrid approach: utilize the no-code Template Gallery or construct custom workflows with the Kubeflow Pipelines (KFP) SDK and Google's pre-built components. Finally, accelerate your workflows using the Data Science Agent—an AI-powered collaborator that automates pipeline code generation.

Syllabus

Introduction

This lesson guides learners through the course structure, which is built upon the transition from ad-hoc experimentation to robust, production-grade systems using Vertex AI Pipelines . It outlines the strategies for ML orchestration—ranging from no-code to hybrid pipelines—and introduces learners to the Data Science Agent for accelerating the automation of complex, deployable workflows .

The Case for Orchestration

Examine the operational bottlenecks complex ML processes and determine the need for automated reproducible workflows orchestration.

Introduction to Vertex AI Pipelines

Explore Vertex AI and the core mechanics of ML pipelines, including compilers, DAGs, runners, artifact passing, and metadata lineage.

Authoring Pipelines

Optimize ML workflows using the "Hybrid" pipeline strategy. Evaluate specific workflow requirements to determine the balance between using Google’s validated Pre-built Component and creating custom Lightweight Python Components for proprietary logic.

The Data Science Agent

Leverage the Data Science Agent to automate code generation and troubleshoot architectural errors using the Context-Task-Constraint (CTC) prompt engineering framework.

Course Summary and Next Steps

This lesson summarizes the course by addressing the transition from ad-hoc notebooks to robust, production-grade systems using Vertex AI Pipelines . It reviews the core concepts of ML Orchestration and Hybrid Pipelines, highlights tools like Google's pre-built components and the Kubeflow SDK, and recaps technologies such as the Data Science Agent for automating complex challenges like media sales forecasting and customer churn prediction .