Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

From Pipeline Hell to Platform Paradise - Metadata-Driven Data Systems Architecture

Conf42 via YouTube

Overview

Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Learn how to transform chaotic data pipelines into streamlined, metadata-driven platforms in this 23-minute conference talk from Conf42 Platform Engineering 2025. Discover the hidden costs plaguing traditional data systems and explore how metadata-driven architectures can revolutionize your platform engineering approach. Examine key patterns for implementing metadata-driven systems, including practical strategies for moving from pipeline complexity to platform simplicity. Understand the real-world benefits and measurable impact of adopting metadata-driven approaches through concrete examples and case studies. Explore the future landscape of metadata-driven architecture and learn how to position your organization for success in the evolving platform engineering ecosystem. Gain actionable insights for reducing operational overhead, improving system reliability, and creating more maintainable data infrastructure that scales with your business needs.

Syllabus

00:00 Introduction and Problem Statement
01:17 The Hidden Costs of Traditional Data Systems
05:06 The Power of Metadata-Driven Platforms
07:42 Key Patterns for Metadata-Driven Systems
10:17 Benefits and Real-World Impact
18:53 Future of Metadata-Driven Architecture
22:08 Conclusion and Final Thoughts

Taught by

Conf42

Reviews

Start your review of From Pipeline Hell to Platform Paradise - Metadata-Driven Data Systems Architecture

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.