Bayes' theorem

A rule for updating the probability of a hypothesis when evidence arrives: posterior probability is proportional to prior probability times likelihood of the evidence.

Prior probability: The probability assigned to a hypothesis before the new evidence is taken into account. The theorem's starting quantity, which the evidence updates into a posterior.
Thomas Bayes: English Presbyterian minister and amateur mathematician whose posthumously published 1763 essay contains the theorem. The theorem's namesake, though he stated only a special case.
Frequentist inference: Statistical inference that treats probability as long-run frequency and rejects prior probabilities for hypotheses. The dominant rival school; its refusal of priors is the deepest disagreement in statistics.
Medical testing: Interpreting diagnostic test results by weighing test accuracy against disease prevalence. The textbook setting: rare diseases make even accurate tests produce mostly false positives.
Subjective probability: The interpretation of probability as coherent degrees of belief held by an individual. The theorem made personal priors legitimate inputs, forcing this interpretation to the fore.
Likelihood function: The probability of observed data as a function of the unknown parameter or hypothesis. The bridge from evidence back to hypotheses: what carries all evidential weight in the update.
Pierre-Simon Laplace: French mathematician who independently derived the theorem and applied it to astronomical and judicial problems. He generalized Bayes' special case and put the theorem to systematic work from the 1770s.
Null hypothesis significance testing: The practice of judging results by p-values against a null hypothesis at a fixed significance level. What the theorem computes (posterior probability of a hypothesis) is exactly what p-values do not give.
Spam filtering: Classifying email by combining word frequencies with prior spam probabilities. Paul Graham's 2002 essay popularized a naive Bayes filter that reshaped practical spam defense.
Cromwell's rule: The injunction never to assign a hypothesis prior probability of exactly zero or one. Under the theorem, a zero prior can never be overturned by any amount of evidence.

Linked from 36 pages