A Mathematical Theory of Communication

Summary

Shannon established the mathematical foundations of information theory, modeling communication as the reproduction of a message selected from a set of possibilities across a (possibly noisy) channel. He introduced entropy as a quantitative measure of information and uncertainty, defined channel capacity, and proved coding theorems showing that reliable transmission is achievable up to (but not beyond) the channel capacity. The work unified the treatment of discrete and continuous sources and laid the groundwork for modern digital communication and data compression.

Key findings

• Defined information entropy as the measure of the average information produced by a source and the fundamental limit of lossless compression.
• Introduced channel capacity and proved that error-free communication is possible at any rate below capacity using suitable coding, even over a noisy channel (the noisy-channel coding theorem).
• Showed that source and channel can be treated separately, providing a general framework applicable to both discrete and continuous communication systems.

Subjects & keywords