Kolmogorov-Arnold Networks - Alternatives to Multi-Layer Perceptrons

Explore Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs) in this comprehensive 1 hour 35 minute talk by Ziming Liu from Valence Labs. Learn how KANs, inspired by the Kolmogorov-Arnold representation theorem, utilize learnable activation functions on edges instead of fixed activation functions on nodes. Discover why smaller KANs can achieve comparable or better accuracy than larger MLPs in data fitting and PDE solving, and how they possess faster neural scaling laws. Examine the interpretability advantages of KANs, including intuitive visualization and easy interaction with human users. Through examples in mathematics and physics, see how KANs can assist scientists in (re)discovering mathematical and physical laws. The talk covers background information, comparisons between MLPs and KANs, accuracy and scaling, interpretability in scientific applications, strengths and weaknesses, philosophical aspects, behind-the-scenes anecdotes, and concludes with a Q&A session.