Statistical hypothesis testing

Statistical hypothesis testing uses sample data to assess a claim by measuring how compatible the observations are with a specified null hypothesis.

Null hypothesis: A statistical proposition treated as the reference model in a hypothesis test. The test evaluates the data’s compatibility with this specified reference.
Type I error: The error of rejecting a true null hypothesis. The significance level bounds this error probability under the test’s assumptions.
Bayesian hypothesis testing: A framework that compares hypotheses using prior probabilities and the likelihood of observed data. Unlike conventional tests, it can quantify evidence for competing hypotheses through posterior odds.
Student's t-test: A family of tests that uses Student’s t distribution to assess means under specified assumptions. This familiar test illustrates how a statistic and reference distribution answer a focused mean question.
Statistical significance: A designation indicating that data meet a prespecified statistical decision threshold under a model. Its conventional threshold is often mistaken for evidence of practical importance or truth.
Alternative hypothesis: A statistical proposition that contrasts with the null hypothesis and represents effects or differences of interest. It states the possibility considered when evidence weighs against the null.
Type II error: The error of failing to reject a false null hypothesis. It captures missed effects that a test may fail to detect.
Likelihood-ratio test: A test that compares the maximum likelihoods under a null model and a broader alternative model. It is a specific testing method whose statistic measures relative fit of the two models.
Analysis of variance: A statistical method for comparing group means by partitioning variation into modeled components. It tests whether several group means differ without relying on separate pairwise tests.
Replication crisis: A recognized difficulty in reproducing findings across several scientific fields. Concerns about fragile findings brought testing practices and selective reporting under scrutiny.

Linked from 14 pages