The 12 Factor App for Data - Principles for Designing Data-Centric Applications

Explore a comprehensive presentation on applying the 12-factor app principles to data-centric applications. Delve into four key categories: Architecture & Design, Quality & Validation, Audit & Explainability, and Consumption. Learn how to implement best practices for designing scalable, testable, and production-ready data platforms. Discover practical examples and counter-examples from real-world projects, covering topics such as data structures as code, append-only data structures, optimization for access and retrieval, separation of data and logic, strong typing, regression testability, metadata management, defined APIs, SLAs, and treating data as a product. Gain insights to create high-quality data-centric applications that scale effectively and deploy quickly in today's increasingly data-driven world.

Syllabus

Intro
What & why?
Architecture & design
No. 1: Data structures as code
No. 2: Append-only data structures
No. 3: Optimize for access & retrieval
No. 4: Separate data from logic
No. 5: Strongly type your data fields
Quality & validation
No. 6: Architect for regression testability
No. 7: Track changes in your test data
Audit & explainability
No. 8: Mind your metadata: Data cataloguing
No. 9: Mind your metadata: Code traceability
Consumption
No. 10: Defined APIs for accessing data
No. 11: Defined SLAs & SLOs for data
No. 12: Treat data as a product
Summary
Outro