Andrej Karpathy Courses

Practical Guide to Using Large Language Models

2 ratings

Explore practical applications of Large Language Models through Andrej Karpathy's comprehensive walkthrough of tools, features, and real-world examples—from basic interactions to advanced capabilities like code generation, multimodal inputs, and custom a…

• YouTube
• 2 hours 11 minutes
• Self-Paced
• Free Video

Introduction to Large Language Models - Future and Security Challenges

1 rating

Comprehensive introduction to Large Language Models, covering their functionality, training, future potential, and security challenges. Explores applications, scaling, tool use, and emerging paradigms in AI.

• YouTube
• 1 hour
• Self-Paced
• Free Video

Deep Dive into Large Language Models: From ChatGPT to Training and Applications

Dive into a comprehensive exploration of Large Language Models, from fundamental concepts to advanced topics like RLHF, with insights from an industry expert on their development and practical applications.

• YouTube
• 3 hours 31 minutes
• Self-Paced
• Free Video

Let's Build GPT - From Scratch, in Code, Spelled Out

Build a GPT from scratch in PyTorch, progressing from a bigram language model to token embeddings, self-attention, and a decoder-only Transformer.

• YouTube
• 1 hour 56 minutes
• Self-Paced
• Free Video

Building Makemore - Building a WaveNet

Build a character-level language model with a hierarchical, WaveNet-like convolutional architecture while exploring PyTorch modules and deep-learning experimentation.

• YouTube
• 56 minutes
• Self-Paced
• Free Video

Building Makemore - Becoming a Backprop Ninja

Manually backpropagate through a batch-normalized MLP to understand tensor-level gradient flow, optimize neural networks, and debug their implementations.

• YouTube
• 1 hour 55 minutes
• Self-Paced
• Free Video

Building Makemore - Activations & Gradients, BatchNorm

Explores how activation and gradient statistics reveal training problems in multilayer perceptrons, then uses initialization and BatchNorm to stabilize deep network training.

• YouTube
• 1 hour 56 minutes
• Self-Paced
• Free Video

Building Makemore - MLP

Build a multilayer perceptron character-level language model from scratch, implementing embeddings, network layers, loss, training, evaluation, and sampling.

• YouTube
• 1 hour 16 minutes
• Self-Paced
• Free Video

The Spelled-Out Intro to Language Modeling - Building Makemore

Build a character-level bigram language model from scratch, comparing count-based probabilities with a one-layer neural network in PyTorch.

• YouTube
• 1 hour 58 minutes
• Self-Paced
• Free Video

Intro to Neural Networks and Backpropagation - Building Micrograd

Builds a minimal neural-network library from scratch to reveal backpropagation, gradient descent, and training under the hood.

• YouTube
• 2 hours 26 minutes
• Self-Paced
• Free Video

Building the GPT Tokenizer - From Strings to Tokens and Back

Comprehensive guide to building a GPT tokenizer from scratch, covering Unicode, Byte Pair Encoding, implementation details, and practical considerations for large language models.

• YouTube
• 2 hours 14 minutes
• Self-Paced
• Free Video

Reproducing GPT-2 (124M) from Scratch - Implementation and Optimization

Comprehensive tutorial on reproducing GPT-2 (124M) from scratch, covering implementation, optimization, training setup, and results analysis. Includes practical coding and performance tuning tips.

• YouTube
• 4 hours 1 minute
• Self-Paced
• Free Video