Information Theory Courses and Certifications

Information Theory of Deep Learning - Naftali Tishby

Explores the connection between deep learning and information theory, offering new insights into neural network design, generalization, and optimization through an information bottleneck framework.

• YouTube
• 1 hour 25 minutes
• On-Demand
• Free Video

Introduction to Complexity

149 reviews

In this course you'll learn about the tools used by scientists to understand complex systems. The topics you'll learn about include dynamics, chaos, fractals, information theory, self-organization, agent-based modeling, and networks.

• Complexity Explorer
• 10 weeks, 3-6 hours a week
• On-Demand
• Free Online Course

• Best of All Time

Center for Science of Information NSF STC

Information Theory and Network Coding

Master fundamental limits in communication systems through comprehensive information theory principles, from entropy and data compression to channel capacity and rate-distortion theory.

• YouTube
• 15 hours 21 minutes
• On-Demand
• Free Video

Complexity Explorer

Introduction to Information Theory Tutorial

Discover the fundamentals of information theory, from measuring bits and entropy to Shannon's coding theorem and communication capacity in this comprehensive tutorial.

• YouTube
• 2 hours 6 minutes
• On-Demand
• Free Video

Information Theory: Defining Entropy and Information - Lecture 1

2 reviews

Delve into the mathematical foundations of entropy and information measurement, exploring how random variables contribute to quantifying information in probability theory.

• YouTube
• 54 minutes
• On-Demand
• Free Video

Connecting Molecular Details to Macroscopic Behaviors with Thermodynamics and Information Theory

Explore how molecular details connect to macroscopic behaviors using thermodynamics and information theory, focusing on E. coli chemotaxis and the energetic costs of biological information processing.

• YouTube
• 1 hour 2 minutes
• On-Demand
• Free Video

Information Theory: Optimal Codes and Block Coding - Lecture 5

Delve into optimal coding principles and block coding techniques, exploring theoretical optimums, finite sequence encoding, and the construction of asymptotically optimal codes using typical sequences.

• YouTube
• 50 minutes
• On-Demand
• Free Video

Information Theory: Typical Sequences and the Kraft-McMillan Inequality - Lecture 4

Delve into the Kraft-McMillan inequality and Asymptotic Equipartition Property, exploring how the law of large numbers influences sequences of random observations in information theory.

• YouTube
• 53 minutes
• On-Demand
• Free Video

Introducing Codes in Information Theory - Lecture 3

Delve into compression codes and efficient information storage, exploring the Kraft-McMillan theorem and its implications for encoding character sequences in information theory.

• YouTube
• 48 minutes
• On-Demand
• Free Video

Basic Properties of Information Theory - Lecture 2

Delve into entropy, Shannon information, and their algebraic relationships, exploring how conditional variables affect entropy and connect random outcomes in information theory.

• YouTube
• 51 minutes
• On-Demand
• Free Video

Information Theory: Huffman and Arithmetic Codes - Lecture 7

Explore Huffman's optimal code construction and the practical advantages of Arithmetic codes in this information theory lecture from Oxford's 3rd year mathematics course.

• YouTube
• 51 minutes
• On-Demand
• Free Video

Information Theory: Shannon's Code - Lecture 6

Explore Shannon's code construction in information theory, including optimal prefix codes related to rooted trees and recursive code construction techniques.

• YouTube
• 52 minutes
• On-Demand
• Free Video

International Centre for Theoretical Sciences

From Information Theory to Learning via Statistical Physics - Introduction

Explore connections between information theory, statistical physics, and learning. Delve into classical statistics, high-dimensional problems, and Bayesian approaches, with applications in signal processing and regression.

• YouTube
• 1 hour 33 minutes
• On-Demand
• Free Video

Information Theory: Tunstall's Code - Lecture 8

Explore Tunstall's algorithm for optimal variable-input, fixed-output encoding in information theory, building on arithmetic codes to create efficient compression systems.

• YouTube
• 53 minutes
• On-Demand
• Free Video

The Leverage of Language - How Information Theory Impacts Security - Lecture 2.2.2

Explore information theory and its impact on language, communication, and security in this thought-provoking talk by Conrad Constantine.

• YouTube
• 54 minutes
• On-Demand
• Free Video