The ASF Courses

Application and Practice of Data Integration Technology at Xiaomi

Explore Xiaomi's data integration technology, including service evolution, self-developed engine capabilities, and platformization practices. Gain insights into real-world solutions for data synchronization and heterogeneous system integration.

• YouTube
• 27 minutes
• Self-Paced
• Free Video

Interconnection and Integration Between Apache DolphinScheduler and Cloud Object Storage

Explore interconnection between Apache DolphinScheduler and cloud object storage for file management and remote task log storage in data lake scenarios.

• YouTube
• 31 minutes
• Self-Paced
• Free Video

How to Use Apache SeaTunnel to Quickly Access New Data Sources

Develop new data sources for Apache SeaTunnel to interconnect with over 100 sources. Learn architecture, best practices, and contribution guidelines.

• YouTube
• 37 minutes
• Self-Paced
• Free Video

Apache DolphinScheduler Indicator System for Stability and Monitoring

Explore DolphinScheduler's indicator system and core service burial points to ensure stability and fast task tracking on Alibaba Cloud's implementation.

• YouTube
• 34 minutes
• Self-Paced
• Free Video

Next Generation Data Integration Platform Apache SeaTunnel

Explore Apache SeaTunnel's innovative approach to batch data synchronization, addressing integration challenges and ensuring data consistency across all scenarios.

• YouTube
• 31 minutes
• Self-Paced
• Free Video

How to Build a Large-Scale API Hub

Explore building a large-scale API hub, covering API management, automated delivery, and lifecycle. Learn to address challenges in distributed organizations and add business value.

• YouTube
• 23 minutes
• Self-Paced
• Free Video

Practical Implementation of Apache Shenyu Microservice Gateway for Billions of Traffic

Implement Apache Shenyu microservice gateway for high-traffic scenarios. Learn gateway concepts, customization, and real-world use cases from an experienced engineer.

• YouTube
• 25 minutes
• Self-Paced
• Free Video

OpenLineage: An Open Standard for Data Lineage

Explore OpenLineage, an open framework for data lineage metadata collection. Learn its purpose, data model, and integration with tools like Apache Airflow and Spark to enhance pipeline understanding and management.

• YouTube
• 40 minutes
• Self-Paced
• Free Video

Logging Modernization and Cybersecurity at Scale with MiNiFi, Kafka and Flink

Modernize logging and enhance cybersecurity using MiNiFi, Kafka, and Flink. Learn efficient log collection, processing, and streaming analytics for global network monitoring and alert generation.

• YouTube
• 39 minutes
• Self-Paced
• Free Video

Integrated Audits: Streamlined Data Observability with Apache Iceberg

Explore streamlined data observability using Apache Iceberg's Integrated Audits pattern for massive-scale data validation and quality assurance.

• YouTube
• 39 minutes
• Self-Paced
• Free Video

Morel: A Data-Parallel Programming Language - Lecture

Explore Morel, a functional programming language with relational extensions, designed for efficient data-parallel processing on various scales. Learn its evolution and capabilities in graph and recursive queries.

• YouTube
• 49 minutes
• Self-Paced
• Free Video

Iceberg's Best Secret: Exploring Metadata Tables

Explore Iceberg's metadata tables to gain insights, optimize queries, and enhance data management. Learn advanced techniques for auditing, quality control, and performance improvement in Apache Iceberg.

• YouTube
• 38 minutes
• Self-Paced
• Free Video

Git for Data Lakes: How LakeFS Scales Data Versioning to Billions of Objects

Learn how lakeFS enables Git-like versioning for data lakes, solving challenges of object storage and scaling to billions of objects without compromising performance. Explore branching, committing, and merging for petabyte-scale data management.

• YouTube
• 43 minutes
• Self-Paced
• Free Video

Elastic Heterogeneous Cluster and Heterogeneity-Aware Job Configuration

Explore Apache Daffodil's internal organization, DFDL schema compiler, and multiple runtime environments. Learn about new features like the VSCode-based data debugger.

• YouTube
• 42 minutes
• Self-Paced
• Free Video

Dive into Avro: Everything a Data Engineer Needs to Know

Explore Apache Avro's key features, best practices, and schema evolution for optimized data serialization in big data systems and event-driven architectures.

• YouTube
• 40 minutes
• Self-Paced
• Free Video