Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Explore the complexities of schema management and translation in modern data lake architectures through this 24-minute lightning talk from Databricks. Learn how Unity Catalog centralizes diverse schemas and discover Coinbase's innovative "Schemaster" approach to automating schema inference, translation, and evolution across popular systems including Delta, Iceberg, Snowflake, Kafka, MongoDB, DynamoDB, and Postgres. Examine the intricacies and challenges of schema differences among various data systems, understand a proposed field-level metadata model, and compare two key translation patterns: point-to-point versus hub-and-spoke architectures. Discover how data profiling can be enhanced to improve schema understanding and translation processes, and see how these concepts integrate with ingestion and reverse-ETL workflows within a Databricks-oriented ecosystem. Gain insights into standardizing schema lineage and translation to improve productivity and automation in data platform management, presented by Eric Sun, Head of Data Platform at Coinbase.
