Research institute · Research orgs · Amsterdam, Netherlands · Est. 2021
Existential Risk Observatory (XRO) is a Dutch (Amsterdam-based) nonprofit foundation focused on reducing existential risks—especially those associated with advanced AI—by shifting public attention and policy discussion toward concrete AI safety governance proposals and evidence-based “risk communication.” XRO’s theory of impact treats public communication and advocacy as a neglected causal lever for increasing political priority, funding, and talent for existential-risk mitigation.
The organization’s work combines (1) public-facing research outputs (reports on public attitudes toward AI existential risk and on how to communicate existential risks to general audiences), (2) policy advocacy centered on international AI-safety coordination mechanisms, and (3) public events intended to bring recognized AI-safety and policy voices into higher-visibility public discussion.
No indexed openings right now. Check the careers page ↗.
Does your AI model know it’s being tested? While this sounds like a theoretical question, it’s a practical safety concern. If models can distinguish between evaluations and deployment, they might... The post Does Your AI Know It’s Being Tested? appeared first on Existential Risk
24% of the US public is now aware of AI xriskThe Existential Risk Observatory has been interested in public awareness of AI existential risk since its inception over five years ago. We started surveying public awareness in December 2022, including by asking the following... The post 24% of the US public is now aware of AI x
Hardening against AI takeover is difficult, but we should tryOver a decade ago, Eliezer Yudkowsky famously ran the AI box experiment, in which a gatekeeper had to keep a hypothetical ASI, played by him, inside a box, while the... The post Hardening against AI takeover is difficult, but we should try appeared first on Existential Risk Obser
AI Offense Defense Balance in a Multipolar WorldBy Otto Barten and Sammy Martin Executive summary We examine whether intent-aligned defensive AI can effectively counter potentially unaligned or adversarial takeover-level AI. This post identifies two primary threat scenarios:... The post AI Offense Defense Balance in a Multipol
Yes RAND, AI Could Really Cause Human ExtinctionLast month, think tank RAND published a report titled On the Extinction Risk from Artificial Intelligence and an accompanying blog post asking the question: “Could AI Really Kill Off Humans?”... The post Yes RAND, AI Could Really Cause Human Extinction appeared first on Existenti
AI has passed the Turing testIn 1950, Alan Turing asked himself the question: “can machines think?” Turing started, as mathematicians do, by defining “machines” and “think”. In search for an operational definition of thinking, he... The post AI has passed the Turing test appeared first on Existential Risk Ob
AI Safety Debate with Prof. Yoshua Bengio (9 Feb)Progress in AI has been stellar and does not seem to slow down. If we continue at this pace, human-level AI with its existential risks may be a reality sooner... The post AI Safety Debate with Prof. Yoshua Bengio (9 Feb) appeared first on Existential Risk Observatory .
Our proposal in TIME: a Conditional AI Safety TreatyLast week, we proposed the Conditional AI Safety Treaty in TIME Magazine as a solution to AI’s existential risks. Read the full piece here: There Is a Solution to AI’s... The post Our proposal in TIME: a Conditional AI Safety Treaty appeared first on Existential Risk Observatory
What does the breakdown of scaling laws imply?It is now public knowledge that multiple LLMs significantly larger than GPT-4 have been trained, but they have not performed much better. That means scaling laws have broken down. What... The post What does the breakdown of scaling laws imply? appeared first on Existential Risk O
The Future of AI – Too Much to Handle? (6 June)Artificial intelligence has advanced rapidly in the last years. If this rise will continue, it could be a matter of time until AI approaches, or surpasses, human capability level at... The post The Future of AI – Too Much to Handle? (6 June) appeared first on Existential Risk Obs
XRO published a YouTube video description for an AI Safety Debate featuring Yoshua Bengio and other panelists, framed as taking place the evening before a Paris AI Summit event.
Conditional AI Safety Treaty proposal published in TIMETIME published an article (as written by Otto Barten) describing XRO’s Conditional AI Safety Treaty concept, including the idea of pausing potentially unsafe training when AI-safety institutes determine loss-of-control risks are unacceptable.
XRO reports its 2024 CAIS merger (grantmaking.ai summary)Grantmaking.ai’s organization profile describes that the Campaign for AI Safety (CAIS) merged with XRO in 2024, expanding campaigning and public advocacy capacity.