Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Explore how to build high-performance libraries for reading and writing Delta tables through this 34-minute conference talk from Databricks. Learn about Delta Kernel's design principles and implementation strategies that enable engines and connectors like DuckDB, Clickhouse, Spice AI, and delta-dotnet to integrate seamlessly without managing Delta protocol complexities. Discover the lessons learned in creating APIs that allow flexible engine integration while maintaining robust performance. Examine the streamlined API design, its evolutionary changes, and the underlying motivations driving these decisions. Dive into new highlight features including write support capabilities and Change Data Feed (CDF) scanning functionality. Gain insights into the future roadmap for the Delta Kernel project and understand what developments to expect in the coming year. The session covers practical approaches to building connector libraries that balance ease of use with performance requirements, making it valuable for developers working with data lake technologies and Delta table implementations.
