Learn about Neural Architecture Growth and efficient network scaling in this 23-minute conference talk from the Thematic Programme on "Infinite-dimensional Geometry: Theory and Applications" at the Erwin Schrödinger International Institute. Explore methods for dynamically expanding small neural networks during training by leveraging backpropagation information to create Directed Acyclic Graph (DAG) architectures. Discover strategies for reducing computational bottlenecks, optimizing parameter efficiency, and minimizing the environmental and financial costs associated with training large neural networks. Gain insights into innovative approaches for balancing model performance with resource constraints through intelligent architecture growth techniques.