Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Beijing Institute of Technology

Information ThInformation Theoryeory

Beijing Institute of Technology via XuetangX

Overview

Google, IBM & Meta Certificates — All 10,000+ Courses at 40% Off
One annual plan covers every course and certificate on Coursera. 40% off for a limited time.
Get Full Access

The Massive Open Online Course (MOOC) "Information Theory," led by Beijing Institute of Technology, features renowned expert Professor Chien-Hung Yeh, who developed and recorded this course. We believe this course will help university students master information theory, along with its application methods and techniques in related fields, laying a solid foundation for future research and engineering applications.


This MOOC course has three outstanding characteristics:


First, the tight integration of theory and practice. Based on Shannon's information theory, the course reveals the nature and connotation of information, focusing on sources and channels to explore how to achieve effective and reliable information transmission. This combination of theory and practice not only helps learners understand the fundamental principles of information theory but also enables them to see the application of these theories in practical systems, thereby deepening their appreciation of the importance of information theory.


Second, interdisciplinary knowledge integration. The content covers fundamental principles of information theory, information-theoretic measures, channel capacity, coding techniques, and more. These topics not only pertain to information theory itself but also intersect with multiple disciplines including probability theory, stochastic processes, signal processing, and cryptography. Through this course, learners can acquire interdisciplinary knowledge, which holds significant value for students in solving complex problems in their future academic or professional careers.


Third, real-world application case studies. Through practical case analyses, learners can intuitively see the role of information theory in solving real-world problems. The course aims to enhance learners' ability to apply theoretical knowledge to practice, laying a solid foundation for their future application of information theory in their respective fields.

Syllabus

  • 01 An Overview of Information Theory
    • 1.1 Introduction to Information Theroy
    • 1.2 Main Research Content of Information Theory
    • 1.3 Development and Application of Information Theory
    • 1.4 Exercises
  • 02 Source Model
    • 2.1 Source Model
    • 2.2 Self-Information
    • 2.3 Definition of Shannon Entropy
    • 2.4 Properties of Entropy
    • 2.5 Joint/Conditional Self-Information
    • 2.6 Mutual Information and Properties
    • 2.7 Extended Discrete Memoryless Sources
    • 2.8 Entropy of Discrete Memory Sources
    • 2.9 Entropy Rate of Markov Chains
    • 2.10 Entropy Rate of Discrete Sources and Information Efficiency
    • 2.11 Differential Entropy of Continuous Random Variables and Mutual Information
    • 2.12 Exercises
  • 03 Channels and Channel Capacity
    • 3.1 Channel Model and Classification
    • 3.2 Mathematical Model of DMC
    • 3.3 Probability Calculation of DMC
    • 3.4 Doubtful Measure, Dispersion Measure, and Average Mutual Information
    • 3.5 Channel Capacity
    • 3.6 Extended Channel and Its Capacity
    • 3.7 Channel Combinations
    • 3.8 Source-Channel Matching
    • 3.9 Continuous Channel and Its Channel Capacity
    • 3.10 Waveform Channel and Its Channel Capacity
    • 3.11 Exercises
  • 04 Information Theory and Coding
    • 4.1 Introduction to Source Coding
    • 4.2 Uniquely Decodable Code
    • 4.3 Fixed-Length Coding Theorem and Methods
    • 4.4 Variable-Length Source Coding Theorem
    • 4.5 Variable-Length Codes
    • 4.6 Exercises
  • Final examination

    Taught by

    Chien-Hung Yeh and Wenbin Shi

    Tags

    Reviews

    Start your review of Information ThInformation Theoryeory

    Never Stop Learning.

    Get personalized course recommendations, track subjects and courses with reminders, and more.

    Someone learning on their laptop while sitting on the floor.