A Mathematical Theory of Communication

A 1948 paper by Claude Shannon that defines information quantitatively and derives fundamental limits on transmitting messages through noisy channels.

Information theory: The mathematical study of information, its representation, and its communication under constraints. Shannon’s paper established the field by quantifying information and proving communication limits.
Shannon's source coding theorem: A theorem stating that lossless compression cannot asymptotically use fewer bits per symbol than a source’s entropy. It gives the paper’s fundamental lower bound for representing a source without losing information.
Claude Shannon: An American mathematician and electrical engineer who founded modern information theory. Shannon authored the paper and supplied its mathematical account of communication.
Error-correcting code: A code that adds structured redundancy so corrupted data can be detected or corrected. The channel coding theorem showed that redundancy can enable reliable transmission despite noise.
Semantic information: Information understood in relation to meaning, truth, or the content of a message. Shannon’s measure concerns uncertainty and transmission, not what a message means.
Entropy (information theory): A measure of the average uncertainty or information content of a random variable. Shannon uses entropy to quantify a source’s information rate and coding requirements.
Shannon-Hartley theorem: A formula for the capacity of a band-limited channel with additive white Gaussian noise. It translates the paper’s general capacity framework into a result for a common physical channel model.
Bell System Technical Journal: A technical journal published by Bell Labs from 1922 to 1983. The paper first appeared in this journal in 1948.
Digital communication: The transmission of information using discrete symbols represented by digital signals. Shannon’s limits provided a mathematical foundation for designing digital communication systems.
Rate-distortion theory: A theory describing the minimum coding rate needed to represent a source within a specified distortion. It extends the paper’s lossless source framework to representation that permits controlled errors.

Linked from 8 pages