Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
This 28-minute video from Google I/O 2025 demonstrates how to build a transformer model from scratch using JAX. Learn about the origins and structure of transformer architecture that has revolutionized the AI landscape. Follow along as speaker Yufeng Guo walks through constructing a small transformer using JAX, implementing Flax NNX for model architecture, Optax for loss functions and optimizers, and training on accelerated hardware with Orbax and XLA. Gain practical experience with JAX development and prepare for building your own AI models. The presentation includes links to companion Colab and Kaggle notebooks, the original "Attention is All You Need" research paper, the Open Web Text dataset, and additional resources on transformer mathematics and JAX documentation.
