Spectral Transformers for Long-Range Sequence Modeling and Prediction

Join this AI seminar exploring a groundbreaking technique for sequence modeling that addresses long-range dependencies and fast inference challenges through spectral state space models (SSMs). Dive into the innovative approach of using linear dynamical systems with spectral filtering algorithms, understanding their robust performance characteristics independent of spectrum dynamics and problem dimensionality. Learn how these models utilize fixed convolutional filters without requiring learning while delivering superior performance compared to traditional SSMs. Examine practical applications through synthetic dynamical systems and long-range prediction tasks across various modalities, discovering how spectral filtering benefits tasks demanding extended memory capabilities. Explore recent developments in fast generation methods and proven length generalization properties of these transformative models.