Replication crisis

The replication crisis is the widespread difficulty of reproducing published research findings, especially in psychology and biomedicine. It has exposed weaknesses in evidence reliability and research practices.

Statistical power: The probability that a statistical test detects an effect when that effect exists. Low power makes estimated effects unstable and genuine findings harder to reproduce.
Reproducibility Project: Psychology: A 2015 project in which researchers attempted to replicate 100 psychological studies published in 2008. Its results made replication failures a prominent concern across psychology.
Preregistration: A time-stamped public record of a study's research questions, methods, and planned analyses made before results are known. It distinguishes planned tests from analyses devised after seeing the data.
Reproducibility: The ability to obtain consistent results using the same data, methods, and computational procedures. It concerns rerunning an analysis, unlike replication with new data.
P-hacking: The use of undisclosed analytical choices to obtain statistically significant results. Flexible analyses can produce publishable results that do not survive independent testing.
Many Labs 2: A large collaboration that tested the replicability of 28 psychological effects across research sites. Its coordinated multi-site design showed how replication varies across effects and settings.
Registered report: A publication format in which study methods are peer reviewed before results are collected. It reduces incentives to publish only positive or surprising outcomes.
Replicability: The ability of an independent study to obtain consistent findings using newly collected data. It is the direct empirical standard most often questioned in the crisis.
Winner's curse: The tendency for the largest observed effects among selected results to overestimate true effects. Studies selected for striking findings often report effects that later replications shrink.
Open Science Collaboration: A network of researchers that coordinated large-scale studies of reproducibility in psychology. The collaboration organized a landmark assessment of published psychological findings.

Linked from 66 pages