Permutation test

A statistical hypothesis test that estimates a null distribution by rearranging observed data, relying on exchangeability under the null hypothesis.

Exchangeability: A property in which a joint distribution is unchanged when observations are permuted. The test is valid when the chosen rearrangements preserve the data’s distribution under the null.
Two-sample permutation test: A permutation test comparing two groups by reallocating observations between group labels. It tests group differences when observations are exchangeable under the null.
Null hypothesis: A statistical hypothesis that specifies a reference condition, commonly no effect or no association. The allowed rearrangements must represent data expected under this reference condition.
Parametric test: A statistical test whose null distribution is specified using a probability model with parameters. Permutation tests avoid choosing a parametric null distribution, but still require valid exchangeability.
Randomization distribution: The distribution of a test statistic generated by assignments or rearrangements allowed by a study design. It supplies the reference distribution against which the observed statistic is compared.
Paired permutation test: A test for paired data that permutes signs or swaps members within pairs under a specified null. Pairwise rearrangements preserve the dependence created by matched or repeated measurements.
Exchangeability under the null: The assumption that observations or labels have a joint distribution invariant to specified permutations when the null holds. It states precisely when rearranging the observed data yields a valid null distribution.
Bootstrap: A resampling method that estimates sampling uncertainty by drawing samples with replacement from observed data. Bootstrap resampling targets sampling variability, whereas permutation rearrangements typically impose a null hypothesis.
Test statistic: A numerical summary of sample data used to assess a statistical hypothesis. Each permitted rearrangement produces a value whose extremeness determines the p-value.
Randomization test: A test that derives its null distribution from treatment assignments permitted by an experiment’s randomization scheme. Its rearrangements follow the actual assignment mechanism rather than an assumed group-label symmetry.

Linked from 16 pages