Statistical hypothesis test

A statistical procedure that uses sample data to assess a claim about a population or model under a specified null hypothesis. It quantifies how compatible the data are with that null, rather than proving it true or false.

Null hypothesis: A claim or model treated as the reference assumption in a statistical test. The test evaluates sample evidence against this specified reference.
Sampling distribution: The probability distribution of a statistic across repeated samples drawn under specified conditions. Null-based conclusions depend on how the test statistic varies across samples.
Student's t-test: A family of tests that uses the t-distribution to assess means when variance is estimated from data. It tests mean differences when the population variance is unknown.
Bayesian hypothesis testing: An approach that compares hypotheses using prior probabilities and the likelihood of observed data. Unlike conventional null testing, it can quantify relative support for competing hypotheses.
Statistical significance: A designation indicating that data crossed a prespecified statistical decision threshold under a test. The label records a threshold crossing, not the importance or truth of a finding.
Alternative hypothesis: A claim that differs from the null hypothesis and represents departures the test is designed to detect. It states which departures from the null count as evidence of an effect.
Type I error: An incorrect rejection of a true null hypothesis. This is the error probability a significance level is designed to limit.
Analysis of variance: A statistical method that compares group means by partitioning variation into components. Its omnibus test assesses whether several population means are equal.
Bayes factor: The ratio of the marginal likelihoods of two hypotheses or models. It compares evidence between models rather than measuring tail probability under one null.
Publication bias: Systematic distortion of the research record when publication depends on study results or their statistical significance. Selective publication of significant tests can exaggerate apparent evidence.

Linked from 30 pages