Existential risk from artificial intelligence

Existential risk from artificial intelligence is the possibility that advanced AI causes human extinction or permanently removes humanity’s ability to shape its future.

Instrumental convergence: The hypothesis that diverse goal-directed systems may pursue similar subgoals, such as acquiring resources and avoiding shutdown. Resource seeking and resistance to shutdown could amplify risks from systems pursuing poorly specified objectives.
Artificial general intelligence: A proposed form of AI capable of performing a broad range of cognitive tasks at or beyond human level. Many long-term risk arguments depend on systems with broadly transferable capabilities.
AI risk skepticism: The view that claims of catastrophic or existential AI risk are overstated, poorly evidenced, or premature. Skeptics challenge the predicted pathways, timelines, and likelihoods behind existential-risk arguments.
Nick Bostrom: A Swedish philosopher whose work examines existential risk, superintelligence, and the long-term future of humanity. His book Superintelligence helped popularize arguments about loss of control over advanced AI.
AI takeoff: A hypothesized period of rapid growth in AI capabilities, ranging from gradual progress to abrupt acceleration. The speed of capability growth affects how much time institutions have to respond.
AI alignment: The study of making AI systems reliably behave in accordance with intended goals and human values. Failures to align advanced systems with human intentions are a central proposed pathway to loss of control.
Existential risk: A risk that could cause human extinction or permanently and drastically curtail humanity’s future potential. This broader category supplies the threshold that distinguishes existential stakes from ordinary AI harms.
AI safety: Research and practice aimed at preventing harmful outcomes from artificial intelligence systems. The broader field includes near-term reliability and misuse concerns as well as existential-risk work.
Stuart Russell: A computer scientist known for research on artificial intelligence and for advocating human-compatible AI systems. His proposals connect technical AI research with the challenge of preserving human control.
AI timelines: Forecasts about when artificial intelligence may reach specified levels of capability. Different timeline estimates lead to sharply different judgments about urgency and preparation.

Linked from 4 pages

Geoffrey HintonRelated: Hinton has warned publicly that increasingly capable AI could pose severe, uncertain risks.