Clustering Dynamics in Mean-Field Models of Transformers

Explore clustering dynamics in mean-field models of transformers through this mathematical seminar presented by Dr. Andrea Agazzi from Universität Bern. Delve into the theoretical foundations of transformer architectures from a statistical mechanics perspective, examining how clustering behaviors emerge in mean-field approximations of these neural network models. Gain insights into the mathematical principles underlying transformer dynamics and their implications for understanding large-scale neural network behavior. Learn about the intersection of statistical physics and machine learning through rigorous mathematical analysis of transformer models. This presentation is part of the "Representing, calibrating & leveraging prediction uncertainty from statistics to machine learning" research programme at the Isaac Newton Institute for Mathematical Sciences.