Understanding the Expressive Power of Transformers Through Formal Language Theory

Explore recent developments at the intersection of formal language theory and machine learning in this 37-minute lecture by Anthony Lin from TU Kaiserslautern. Delve into the expressiveness of transformers and learn about their capabilities through the lens of formal language theory. Examine results connected to Lin's recent work, including findings from ICLR'24, and gain insights into the synthesis of models and systems. Discover what can be learned with transformers and how this understanding contributes to the broader field of machine learning and language processing.