Explore

Information Theory

Approx. 33 hours to complete

Save Course

Go to Course

Course Summary

This course covers the fundamentals of Information Theory, including entropy, data compression, channel coding, and rate-distortion theory.

Key Learning Points

Gain a deep understanding of the principles and concepts of Information Theory
Apply Information Theory to practical problems in data compression and communication systems
Learn about the latest developments in Information Theory research

Learning Outcomes

Develop a deep understanding of the principles and concepts of Information Theory
Apply Information Theory to solve practical problems in data compression and communication systems
Stay up-to-date with the latest developments in Information Theory research

Prerequisites or good to have knowledge before taking this course

Basic knowledge of probability and statistics
Familiarity with linear algebra and calculus

Course Difficulty Level

Advanced

Course Format

Self-paced
Video lectures
Assignments and quizzes

Similar Courses

Introduction to Communication Theory
Digital Signal Processing
Probability and Statistics in Data Science using Python

Related Education Paths

Notable People in This Field

Mathematical Engineer
Mathematician

Related Books

Description

The lectures of this course are based on the first 11 chapters of Prof. Raymond Yeung’s textbook entitled Information Theory and Network Coding (Springer 2008). This book and its predecessor, A First Course in Information Theory (Kluwer 2002, essentially the first edition of the 2008 book), have been adopted by over 60 universities around the world as either a textbook or reference text.

At the completion of this course, the student should be able to: 1) Demonstrate knowledge and understanding of the fundamentals of information theory. 2) Appreciate the notion of fundamental limits in communication systems and more generally all systems. 3) Develop deeper understanding of communication systems. 4) Apply the concepts of information theory to various disciplines in information science.

Outline

Course Preliminaries
About this course
Grading Scheme

Chapter 1 Information Measures
Week 1 Introduction
Chapter 1

Chapter 2 Information Measures - Part 1
Chapter 2 - Section 2.1 A
Chapter 2 - Section 2.1 B
Chapter 2 - Section 2.1 C
Chapter 2 - Section 2.2
Chapter 2 - Section 2.3
Assignment 1 Video Preview
Homework Assignment 1 Solution

Chapter 2 Information Measures - Part 2
Week 2 Introduction
Chapter 2 - Section 2.4
Chapter 2 - Section 2.5
Chapter 2 - Section 2.6
Chapter 2 - Section 2.7
Chapter 2 - Section 2.8
Chapter 2 - Section 2.10
Assignment 2 Video Preview
Homework Assignment 2 Solution

Chapter 3 The I-Measure - Part 1
Week 3 Introduction
Chapter 3 - Section 3.1
Chapter 3 - Section 3.3 A
Chapter 3 - Section 3.3 B
Chapter 3 - Section 3.4
Chapter 3 - Section 3.5 A
Assignment 3 Video Preview
Homework Assignment 3 Solution

Chapter 3 The I-Measure - Part 2
Week 4 Introduction
Chapter 3 - Section 3.5 B
Chapter 3 - Section 3.6 A
Chapter 3 - Section 3.6 B

Chapter 4 Zero-Error Data Compression - Part 1
Chapter 4 - Section 4.1
Chapter 4 - Section 4.2 A
Assignment 4 Video Preview
Homework Assignment 4 Solution

Chapter 4 Zero-Error Data Compression - Part 2
Week 5 Introduction
Chapter 4 - Section 4.2 B
Chapter 4 - Section 4.2 C
Chapter 4 - Section 4.3

Chapter 5 Weak Typicality
Chapter 5 - Section 5.1
Chapter 5 -Section 5.2
Assignment 5 Video Preview
Homework Assignment 5 Solution

Chapter 6 Strong Typicality
Week 6 - Introduction
Chapter 6 - Section 6.1 A
Chapter 6 - Section 6.1 B
Chapter 6 - Section 6.2
Chapter 6 - Section 6.3 A
Chapter 6 - Section 6.3 B
Chapter 6 - Section 6.4
Assignment 6 Video Preview
Homework Assignment 6 Solution

Chapter 7 Discrete Memoryless Channels - Part 1
Week 7 Introduction
Chapter 7 - Section 7.1 A
Chapter 7 - Section 7.1 B
Chapter 7 - Section 7.2
Chapter 7 - Section 7.3 A
Chapter 7 - Section 7.3 B
Assignment 7 Video Preview
Homework Assignment 7 Solution

Chapter 7 Discrete Memoryless Channels - Part 2
Week 8 Introduction
Chapter 7 - Section 7.4 A
Chapter 7 - Section 7.4 B
Chapter 7 - Section 7.5
Chapter 7 - Section 7.6
Chapter 7 - Section 7.7
Assignment 8 Video Preview
Homework Assignment 8 Solution

Chapter 8 Rate-Distortion Theory - Part 1
Week 9 Introduction
Chapter 8 - Section 8.1
Chapter 8 - Section 8.2
Chapter 8 - Section 8.3 A
Chapter 8 - Section 8.3 B
Chapter 8 - Section 8.4
Assignment 9 Video Preview
Homework Assignment 9 Solution

Chapter 8 Rate-Distortion Theory - Part 2
Week 10 Introduction
Chapter 8 - Section 8.5 A
Chapter 8 - Section 8.5 B

Chapter 9 The Blahut-Arimoto Algorithms - Part 1
Chapter 9 - Section 9.1
Chapter 9 - Section 9.2 A
Chapter 9 - Section 9.2 B
Assignment 10 Video Preview
Homework Assignment 10 Solution

Chapter 9 The Blahut-Arimoto Algorithms - Part 2
Week 11 Introduction
Chapter 9 - Section 9.2 C
Chapter 9 - Section 9.3

Chapter 10 Differential Entropy - Part 1
Chapter 10 - Section 10.1 A
Chapter 10 - Section 10.1 B
Assignment 11 Video Preview
Homework Assignment 11 Solution

Chapter 10 Differential Entropy - Part 2
Week 12 Introduction
Chapter 10 - Section 10.2
Chapter 10 - Section 10.3 A
Chapter 10 - Section 10.3 B
Chapter 10 - Section 10.4
Chapter 10 - Section 10.5
Chapter 10 - Section 10.6
Assignment 12 Video Preview
Homework Assignment 12 Solution

Chapter 11 Continuous-Valued Channels - Part 1
Week 13 Introduction
Chapter 11 - Section 11.1
Chapter 11 - Section 11.2
Chapter 11 - Section 11.3 A
Chapter 11 - Section 11.3 B
Assignment 13 Video Preview
Homework Assignment 13 Solution

Chapter 11 Continuous-Valued Channels - Part 2
Week 14 Introduction
Chapter 11 - Section 11.4
Chapter 11 - Section 11.5
Chapter 11 - Section 11.6
Assignment 14 Video Preview
Homework Assignment 14 Solution

Chapter 11 Continuous-Valued Channels - Part 3
Week 15 Introduction
Chapter 11 - Section 11.7 A
Chapter 11 - Section 11.7 B
Chapter 11 - Section 11.8
Chapter 11 - Section 11.9
Assignment 15 Video Preview
Homework Assignment 15 Solution

Summary of User Reviews

Learn about information theory with this course on Coursera. Users have given this course high ratings and praise the in-depth coverage of the topic. However, some users have mentioned that the course can be quite challenging.

Key Aspect Users Liked About This Course

in-depth coverage of the topic

Pros from User Reviews

Comprehensive coverage of the subject matter
Engaging and knowledgeable instructors
Well-structured course material

Cons from User Reviews

Can be challenging for beginners
Some users found the pace of the course to be too fast
Lack of interaction with instructors
Not enough practical application of the concepts

Recommended for you

Sales Theory. How Sales Works And How To Sell More

Why customers buy and the sales techniques that sell to them effectively Learn the theory behind sales. Make more money and get more success....

Save Course

XML and XSD: a complete W3C-content based course (+10 hours)

The complete XML/XSD content from W3Schools, with much better explanations and practical examples with Eclipse IDE The World Wide Web Consortium's XML 1....

Save Course