Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Learn block-based parallel programming fundamentals through this comprehensive conference talk that demystifies parallel programming by introducing a newcomer-friendly paradigm. Explore how block-based programming models divide inputs into local arrays (tiles) processed concurrently by thread groups (blocks), allowing developers to write sequential array-centric code while frameworks handle parallelization, synchronization, and data movement automatically. Discover the growing popularity of block-based models in GPU programming languages including Triton, JAX/Pallas, and Warp, designed to make parallelism more accessible and increase portability. Examine NVIDIA's new cuTile block-based programming model for Python, C++, and other languages, along with the Tile IR compiler stack, with exclusive new technology details revealed for the first time. Compare and contrast block-based models with traditional parallel programming approaches through practical examples including a tile-based LLAMA3 large language model demo, stencil code, and FFT solver. Master best practices for writing block-based parallel applications for both CPUs and GPUs, gain insights into performance characteristics and execution mechanisms, learn debugging and reasoning techniques for block-based applications, and understand when to choose block-based versus traditional parallel programming paradigms for developing intuitive, portable, and efficient high-performance data-parallel applications.
