Google, IBM & Meta Certificates — All 10,000+ Courses at 40% Off
One annual plan covers every course and certificate on Coursera. 40% off for a limited time.
Get Full Access
Explore theoretical and practical insights into Linear Transformers in this 34-minute lecture by Xiang Cheng from the Massachusetts Institute of Technology. Delve into recent research highlighting Linear Transformers as proxies for understanding full-fledged Transformer models. Examine theoretical proofs demonstrating how Linear Transformers learn linear regression tasks in-context through gradient-based optimization during forward passes. Gain insights into the mechanisms behind Transformers' in-context learning capabilities. Discover intriguing empirical observations suggesting that the optimization landscape of Linear Transformers may serve as a valuable approximation for understanding the optimization of real Transformers. Enhance your knowledge of optimization and algorithm design in the context of transformer models.
