Data Engineering in the AI Era: Building a Robust Foundation for AI Applications

In this comprehensive tutorial from DSC EUROPE 24, Artemiy Kozyr delves into essential strategies for data engineering in the AI era. Learn how to construct robust data pipelines and implement advanced data modeling techniques that form the backbone of successful AI applications. Discover practical methods for enabling data streaming with Debezium and Snowplow while reducing dependency on third-party vendors through open-source alternatives—a strategy that can save organizations over $20,000 annually. Explore cost optimization techniques using Snowflake, which delivers a 30% reduction in compute and storage expenses compared to Amazon Redshift. Master approaches to maintaining complete data ownership and security through role-based access control, data masking policies, and proper PII handling. Gain valuable insights into effective data flow management, transformation processes, and security protocols specifically designed for AI projects. This Belgrade conference session provides actionable knowledge for data professionals looking to build the solid foundations necessary for advanced AI implementations.