Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Explore a conference presentation that introduces equality saturation as a solution to optimization challenges in machine learning compilers. Learn how traditional ML compilers rely on manually-tuned heuristics for graph-level transformations, which often become obsolete with new hardware and model architectures, and discover how local transformations can create unpredictable effects on downstream optimizations involving data layout, parallelization, and memory management. Understand the proposed equality saturation approach that replaces hand-written local heuristics with robust global performance models capable of accounting for downstream transformations. Examine the practical implementation challenges addressed when applying equality saturation to real-world ML compute graphs and state-of-the-art hardware, including scalability issues from considering wide ranges of algebraic optimizations and different cost modeling approaches for fusion and layout optimization. Study the design and implementation of an equality saturation pass for the XLA compiler using C++ and Rust, and analyze the performance results showing an average speedup of 3.45% over XLA's optimization flow across various CPU and GPU platforms, with maximum improvements reaching 56.26% for specific models like NasRNN on CPU. Gain insights into e-graphs, compiler optimization techniques, and the future of automated, adaptable ML compiler design from researchers at leading institutions including University of Chicago, University of Oxford, OpenAI, and Google.
