Yannic Kilcher Courses

Dynamic Inference with Neural Interpreters - Interview with the Authors

Explore Neural Interpreters: a modular deep learning architecture for systematic generalization. Learn about its structure, function routing, and applications in image classification and abstract reasoning.

• YouTube
• 1 hour 23 minutes
• Self-Paced
• Free Video

Noether Networks - Meta-Learning Useful Conserved Quantities

Explore Noether Networks: a novel approach to meta-learning conserved quantities in sequential prediction problems, inspired by Noether's theorem and aimed at discovering useful symmetries and inductive biases.

• YouTube
• 1 hour 9 minutes
• Self-Paced
• Free Video

Resolution-Robust Large Mask Inpainting with Fourier Convolutions - With Author Interview

Explore LaMa, an advanced image inpainting system using Fourier convolutions for large mask removal. Learn about its architecture, loss function, and impressive results in reconstructing complex structures.

• YouTube
• 54 minutes
• Self-Paced
• Free Video

NÜWA - Visual Synthesis Pre-training for Neural Visual World Creation

Explore NÜWA, a unified multimodal model for visual synthesis tasks. Learn about its 3D transformer framework, 3D Nearby Attention mechanism, and applications in text-to-image/video generation and manipulation.

• YouTube
• 53 minutes
• Self-Paced
• Free Video

Sparse Is Enough in Scaling Transformers - ML Research Paper Explained

Detailed explanation of Scaling Transformers and Terraformer architecture, focusing on leveraging sparsity to improve efficiency and speed in large language models while maintaining accuracy.

• YouTube
• 57 minutes
• Self-Paced
• Free Video

Towards Extreme Multi-Task Scaling for Transfer Learning

Explores ExT5, an advanced NLP model pre-trained on 107 diverse tasks. Analyzes multi-task scaling benefits, task co-training effects, and demonstrates ExT5's superior performance across various NLP benchmarks.

• YouTube
• 41 minutes
• Self-Paced
• Free Video

Parameter Prediction for Unseen Deep Architectures - With First Author Boris Knyazev

Explore parameter prediction for neural networks using graph hypernetworks. Learn about DeepNets-1M dataset, training techniques, and experimental results for efficient network training paradigms.

• YouTube
• 48 minutes
• Self-Paced
• Free Video

Learning Rate Grafting: Transferability of Optimizer Tuning - Machine Learning Research Paper Review

Explore grafting technique for transferring learning rate schedules between optimizers, improving deep learning model performance and reducing computational costs in hyperparameter tuning.

• YouTube
• 39 minutes
• Self-Paced
• Free Video

Gradients Are Not All You Need - Machine Learning Research Paper Explained

Explores limitations of differentiable programming in machine learning, focusing on chaos-based failures in various systems. Discusses alternatives to backpropagation for gradient estimation in complex, stochastic environments.

• YouTube
• 49 minutes
• Self-Paced
• Free Video

Autoregressive Diffusion Models - Machine Learning Research Paper Explained

Explore Autoregressive Diffusion Models, a novel approach combining autoregressive and diffusion models for efficient, order-agnostic generation and compression of text and image data.

• YouTube
• 34 minutes
• Self-Paced
• Free Video

EfficientZero - Mastering Atari Games with Limited Data

Explore EfficientZero, a groundbreaking reinforcement learning algorithm achieving human-level performance on Atari games with minimal data, outperforming previous methods in sample efficiency and performance.

• YouTube
• 29 minutes
• Self-Paced
• Free Video

Symbolic Knowledge Distillation- From General Language Models to Commonsense Models Explained

Explore how to distill commonsense knowledge from large language models to create high-quality knowledge graphs and train smaller, specialized models for improved performance in commonsense reasoning tasks.

• YouTube
• 45 minutes
• Self-Paced
• Free Video

Topographic VAEs Learn Equivariant Capsules - Machine Learning Research Paper Explained

Explore Topographic VAEs: a novel approach to deep generative models with organized latent variables, bridging topographic organization and equivariance in neural networks for improved feature learning and transformation handling.

• YouTube
• 32 minutes
• Self-Paced
• Free Video

Infinite Memory Transformer - Research Paper Explained

Explores innovative Transformer model with unbounded memory, enabling processing of arbitrarily long sequences. Discusses continuous attention mechanisms, sticky memories, and potential applications in language modeling.

• YouTube
• 37 minutes
• Self-Paced
• Free Video

Train Short, Test Long - Attention With Linear Biases Enables Input Length Extrapolation

ALiBi: A novel attention mechanism enabling transformers to process longer sequences than trained on. Uses linear biases instead of position encodings, improving efficiency and extrapolation capabilities.

• YouTube
• 31 minutes
• Self-Paced
• Free Video