Noisy-channel coding theorem

The theorem that information can be transmitted with arbitrarily low error over a noisy channel when the transmission rate is below the channel’s capacity.

Channel capacity: The maximum information rate at which a communication channel can transmit with arbitrarily low error. The theorem’s rate threshold is precisely the channel’s capacity.
Discrete memoryless channel: A channel with finite input and output alphabets whose outputs depend independently on current inputs. This standard model makes the theorem’s capacity and error claims explicit.
Claude Shannon: An American mathematician and electrical engineer who founded information theory. Shannon formulated the theorem in his foundational work on communication.
Shannon–Hartley theorem: A formula for the capacity of an ideal band-limited channel with additive white Gaussian noise. It gives a concrete capacity limit for a widely used physical channel model.
Finite blocklength information theory: The study of communication limits when codes have finite length and nonzero error probability. It quantifies the gap between asymptotic capacity and realistic finite transmissions.
Error-correcting code: A code that adds structured redundancy so transmitted messages can be recovered despite corruption. Codes provide the redundancy that makes the theorem’s low-error transmission possible.
Shannon entropy: A measure of the uncertainty in a probability distribution, expressed as an average information content. Entropy quantifies uncertainty in channel inputs and outputs.
A Mathematical Theory of Communication: Claude Shannon’s 1948 paper that established the foundations of information theory. The paper stated the coding theorem and defined the capacity framework.
Turbo code: A practical error-correcting code introduced in 1993 that can operate close to Shannon capacity. Turbo codes demonstrated that practical systems could approach the theorem’s limit.
Polar code: A recursively constructed code that achieves capacity for symmetric binary-input memoryless channels with suitable decoding. Polar codes give an explicit construction tied to Shannon’s capacity result.

Linked from 2 pages

Coding theoryRelated: It sets the fundamental limit that practical codes seek to approach.