Null hypothesis significance testing

Null hypothesis significance testing is a procedure that compares data with a null hypothesis and uses a preset significance level to decide whether to reject it. A p-value below that threshold is called statistically significant.

Null hypothesis: A statistical claim that specifies no effect, difference, or relationship of interest. It supplies the reference claim that the test evaluates against observed data.
Statistical hypothesis: A claim about a probability model or population parameter that can be assessed using data. Significance testing is one procedure for evaluating such claims.
Bayesian inference: A framework that updates probability distributions for unknown quantities using data and prior information. It answers questions about parameter probabilities rather than relying on a null-test cutoff.
Multiple comparisons problem: The increased chance of false-positive findings when many statistical hypotheses are tested. Repeated thresholded tests can produce apparently significant results by chance.
Ronald Fisher: A British statistician who developed influential methods in experimental design and significance testing. Fisher established the use of p-values as measures of evidence against a null hypothesis.
Test statistic: A numerical summary of sample data used to evaluate a statistical hypothesis. The chosen statistic measures how far the data depart from the null model.
Statistical model: A mathematical description of how data are generated, including assumptions about probability. The null hypothesis and its sampling distribution depend on the model's assumptions.
Likelihood-ratio test: A test that compares the likelihoods of data under nested statistical models. It provides a model-comparison decision rule that need not be framed as a p-value threshold.
Multiple testing correction: A procedure that adjusts statistical thresholds or p-values to address multiple simultaneous tests. Corrections modify significance decisions when studies test many hypotheses.
Jerzy Neyman: A Polish-American mathematician and statistician who developed frequentist theories of testing and estimation. His decision-theoretic framework formalized error rates and planned testing rules.

Linked from 7 pages