Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Learn to build custom data pipelines for niche and hard-to-access data sources using the Fivetran Connector SDK integrated with Databricks in this 36-minute conference talk. Discover how to overcome the significant time and resource challenges of incorporating hundreds of diverse data sources into your lakehouse architecture. Explore the powerful capabilities of the Fivetran Connector SDK for creating scalable connectors that seamlessly ingest data from custom APIs, niche systems, and sources requiring specific data filtering into Databricks environments. Master advanced features including flexible control over historical and incremental syncs, delete capture mechanisms, state management, multithreading data extraction techniques, and custom schema implementation. Gain practical insights through real-world examples, detailed code snippets, and architectural considerations designed to solve complex data integration challenges and maximize your Databricks environment's potential. Presented by CL Abeel, Senior Solution Architect at Fivetran, and Kelly Kohlleffel, Senior Global Director of Partner Sales Engineering at Fivetran, this session provides actionable strategies for building production-ready data pipelines that enable your team to focus on higher-value projects while ensuring comprehensive data accessibility across your organization.
