Forethought Research

One vision of space settlement is "galactic anarchy," where probes sent to distant stars are each free to govern the societies they found however they like. We argue this vision is dangerous: across millions of light-years, no centralized authority can punish violations after the

AI is increasingly being used to help with AI&D. Under certain conditions this feedback loop might be able to produce an intelligence explosion, with rapidly escalating AI capabilities. I explore the mathematics of the most explosive possibilities, with an eye to understanding wh

Governments will deploy frontier AI in areas where their powers are already least checked: military, intelligence, surveillance, and policing. Companies can't police those deployments, and legislatures largely can't see them. This could undermine democratic checks and balances, a

We make the case for training AIs to be risk-averse in resources — specifically, to treat resources as having diminishing marginal utility. We argue that risk aversion can preserve AIs’ usefulness in the event that they turn out aligned, and that it provides an extra line of defe

AI companies face a tangle of competing considerations when deciding what goes into a model spec. Should the AI be completely honest in every circumstance? Take proactive prosocial actions? Engage in whistleblowing? And so on. We lay out a checklist of plausible criteria for good

Google, SpaceX, Blue Origin, and others are betting that the next wave of AI data centers will be built in orbit. If they're right, it could reshape the economics of compute and create new headaches for AI governance. We dig into the technical and economic feasibility of orbital

Today's model specs are written for current and near-future versions of LLMs, and AI labs typically treat them as provisional. But what if the AI behaviors we set now stick around and end up governing far more capable future models by default? This piece traces four sources of "i

If humans and advanced AI systems are going to cooperate—to make honest deals and avoid negative-sum conflict—AIs will need reasons to trust us. By default, they won't have many: humans routinely lie to AIs in evaluations, and developers control much of what models see and believ

Will MacAskill presents a new theory of population ethics, the Saturation View. It is motivated mainly by the observation that virtually all existing population axiologies prefer a *homogeneous* universe, full of a vast amount of whatever sort of thing that axiology rates as best

As AI gets smarter, people will increasingly turn to it for advice on important decisions — and those who do may gain outsized influence. So the quality of that advice really matters. Tom Davidson discusses key scenarios where people might seek AI guidance, drafts a model spec fo

For better or worse, AI could reshape the way that people work out what to believe and what to do, with potentially enormous consequences for the decisions they go on to make. We explore how AI could enhance society's truth-tracking, or degrade it, whether unintentionally or deli

Near-term AI could make it dramatically easier for groups to find deals, resolve disputes, and hold each other accountable. But coordination tech is dual-use — it could also enable collusion and criminality. We sketch six concrete technologies, including AI-mediated negotiation a

Forethought posts research/policy ideas focused on responsible government deployment of AI (published 22 July 2026).

A Forethought-authored essay arguing that middle powers could be left behind as superintelligence is developed and deployed, and proposing a plan (published 27 May 2026).

Forethought and collaborators publish research laying out a draft “honesty policy” approach for credible communication with AI systems (published 6 May 2026).

Forethought publishes a mapping of dynamics for how AI could reshape “epistemics” (making sense of the world and figuring out what’s true), including uplifting, destabilizing, and malicious disruption dynamics (published 12 April 2026).

Forethought publishes a structured set of “concrete projects” it characterizes as neglected or under-discussed for AGI preparedness, including areas where Forethought states it is actively working (published 26 March 2026).

Forethought publishes an argument that AI “character” (e.g., obedience, honesty, cooperation) will have a big effect on society and future outcomes, and that the issue has received insufficient attention (published 23 March 2026).