Large Scale Private Learning on Data Streams and the Buffered Linear Toeplitz Operators

Explore a 30-minute lecture on large-scale private learning for data streams and Buffered Linear Toeplitz operators (BLTs). Delve into the advancements in Differentially Private Follow the Regularized Leader (DP-FTRL) algorithms for training large models, particularly in applications like Gboard next-word prediction. Examine the challenges of correlated noise addition in these algorithms and discover how BLTs offer a solution for generating noise in a streaming fashion with reduced storage requirements. Learn about the construction of BLTs using rational function approximations and constant recurrence sequences, and understand their utility compared to Lower Triangular Toeplitz factorizations. Gain insights into the practical applications of this research through simulation experiments presented by Abhradeep Guha Thakurta from Google DeepMind.