Statistical power

Statistical power is the probability that a statistical test rejects a false null hypothesis under specified conditions. It equals one minus the probability of a Type II error.

Null hypothesis: A statistical claim, often of no effect or association, that a hypothesis test evaluates against evidence. Power is defined by how often a test rejects this hypothesis when it is false.
Power function: A function giving a test's rejection probability for each possible parameter value. It describes how power changes across the parameter values represented by alternatives.
Sample size determination: The process of choosing how many observations a study needs for a specified design and objective. Researchers commonly choose sample size to attain a target power for a meaningful effect.
P-value: The probability, assuming a specified null hypothesis and model, of data at least as extreme as observed. A p-value summarizes observed data under the null; power is a prospective rejection probability under alternatives.
Alternative hypothesis: A statistical claim that differs from the null hypothesis, specifying an effect or relationship a test can detect. Power concerns detecting conditions described by the alternative rather than the null.
Sampling distribution: The probability distribution of a statistic across repeated samples under specified conditions. Power is the alternative distribution's probability mass inside the test's rejection region.
A priori power analysis: A power calculation performed before data collection using anticipated effects and design assumptions. It translates a target detection probability into a planned sample size or study design.
Type I error: An error in which a statistical test rejects a true null hypothesis. Power concerns detecting false nulls, whereas Type I error concerns rejecting true ones.
Type II error: An error in which a statistical test fails to reject a false null hypothesis. Power is the complement of this error probability for specified conditions.
Critical region: The set of test-statistic outcomes for which a hypothesis test rejects its null hypothesis. The location and size of this region determine which alternative outcomes count as detections.

Linked from 29 pages