Research institute · Research orgs · Amsterdam, Netherlands · Est. 2021
Existential Risk Observatory (ERO) is a small Amsterdam-based nonprofit foundation that aims to reduce existential risk—especially from advanced AI—by influencing how the issue is communicated to the public and incorporated into policymaking. Its central strategy is “awareness-first”: instead of treating existential risk communication as secondary, ERO frames effective public communication and policy advocacy as mechanisms for increasing downstream attention, talent, and political prioritization for AI safety.
ERO’s work is organized around three linked activities: (1) producing public-facing communication (e.g., media articles and opinion pieces) designed to explain existential risk clearly and responsibly; (2) publishing applied research on how such communication performs and how it should be improved; and (3) developing concrete governance proposals that can be evaluated and adopted by governments or international actors. In particular, ERO is known for advocating the “Conditional AI Safety Treaty” concept—an approach that conditions a pause on AI development on an external determination that risks are unacceptable (rather than an unconditional stop). ERO also coordinates and publicizes threat-model framing efforts in AI safety, including the release of TakeOverBench.com, a benchmark intended to operationalize progress toward AI “takeover” threat-model-relevant capabilities. Institutionally, ERO occupies a communications-and-policy niche within the broader AI safety ecosystem: many groups focus primarily on technical alignment or formal verification, while ERO focuses on the strategic communication channel and governance agenda-setting that precede (and can enable) those technical safety efforts.
No curated programs yet. Check the careers page ↗.
In response to the resignation of Jacob Coxon from OpenAI because of existential risk concerns, Gary Marcus recently discussed his counterarguments against these risks in his newsletter (“No, Anderson Cooper,... The post Yes, Gary Marcus, AI may well kill all humans by 2030 appea
Does Your AI Know It’s Being Tested?Does your AI model know it’s being tested? While this sounds like a theoretical question, it’s a practical safety concern. If models can distinguish between evaluations and deployment, they might... The post Does Your AI Know It’s Being Tested? appeared first on Existential Risk
24% of the US public is now aware of AI xriskThe Existential Risk Observatory has been interested in public awareness of AI existential risk since its inception over five years ago. We started surveying public awareness in December 2022, including by asking the following... The post 24% of the US public is now aware of AI x
Hardening against AI takeover is difficult, but we should tryOver a decade ago, Eliezer Yudkowsky famously ran the AI box experiment, in which a gatekeeper had to keep a hypothetical ASI, played by him, inside a box, while the... The post Hardening against AI takeover is difficult, but we should try appeared first on Existential Risk Obser
AI Offense Defense Balance in a Multipolar WorldBy Otto Barten and Sammy Martin Executive summary We examine whether intent-aligned defensive AI can effectively counter potentially unaligned or adversarial takeover-level AI. This post identifies two primary threat scenarios:... The post AI Offense Defense Balance in a Multipol
Yes RAND, AI Could Really Cause Human ExtinctionLast month, think tank RAND published a report titled On the Extinction Risk from Artificial Intelligence and an accompanying blog post asking the question: “Could AI Really Kill Off Humans?”... The post Yes RAND, AI Could Really Cause Human Extinction appeared first on Existenti
AI has passed the Turing testIn 1950, Alan Turing asked himself the question: “can machines think?” Turing started, as mathematicians do, by defining “machines” and “think”. In search for an operational definition of thinking, he... The post AI has passed the Turing test appeared first on Existential Risk Ob
AI Safety Debate with Prof. Yoshua Bengio (9 Feb)Progress in AI has been stellar and does not seem to slow down. If we continue at this pace, human-level AI with its existential risks may be a reality sooner... The post AI Safety Debate with Prof. Yoshua Bengio (9 Feb) appeared first on Existential Risk Observatory .
Our proposal in TIME: a Conditional AI Safety TreatyLast week, we proposed the Conditional AI Safety Treaty in TIME Magazine as a solution to AI’s existential risks. Read the full piece here: There Is a Solution to AI’s... The post Our proposal in TIME: a Conditional AI Safety Treaty appeared first on Existential Risk Observatory
What does the breakdown of scaling laws imply?It is now public knowledge that multiple LLMs significantly larger than GPT-4 have been trained, but they have not performed much better. That means scaling laws have broken down. What... The post What does the breakdown of scaling laws imply? appeared first on Existential Risk O
TIME publishes ERO’s Conditional AI Safety Treaty proposal as a solution to AI existential risk, presenting the idea as a governance intervention that is intended to be politically actionable.
Paris AI safety debate (February 2025) described in ERO funding/impact listingsARO/impact listing reports a February 2025 debate in Paris featuring Yoshua Bengio, with panelists including Jaan Tallinn and journalists from TIME and The Economist, positioned as an ERO-related policy communications initiative.