Scaling Machine Learning Pipelines in Cloud

This talk shows how Kubeflow can support repeatable, portable machine-learning deployments and scale infrastructure from a laptop to a production cluster. It demonstrates creating and running a pipeline, including GPU workers and Kubernetes autoscaling.

Syllabus

Intro
About Salman
Giveaway
Agenda
Who is this talk for
Training Models
Docker
Where can we run Kubernetes
Benefits of Kubernetes
Scaling Pipelines with Kubernetes
Introducing Kubernetes
Data Science Machine Learning Engineers
Kubernetes tools
Operators
Fashion mnist
Neural Network
Hyper Parameters
Auto ML
Serving
kubeflow
Why Kubernetes
Data Science and Machine Learning
QFlow Pipeline
MiniCube
Python Pipeline SDK
GPU Pipeline
GPU Worker
Experimenting
Running the pipeline
Kale
Notebook
Kubernetes Extension
Adding Labels
Creating a Pipeline
Scaling Pipelines
Kubernetes Autoscaling
Considerations
Platform Management
OpenAI
Summary
References
Thanks
Outro