Research institute · Research orgs · Est. 2022
FAR.AI (Frontier Alignment Research) is a U.S.-incorporated (501(c)(3)) independent nonprofit research organization focused on technical AI safety research, plus research incubation and field-building aimed at closing the gap between new safety ideas and real-world deployment. Its stated goal is to “close the research-adoption gap” by developing high-leverage technical safety solutions and scaling them toward verification and adoption by frontier AI developers.
FAR.AI describes an operating model that combines (1) in-house foundational research, (2) a “driving adoption” pipeline for scaling vetted safety solutions into practice, and (3) “field building” through convenings and grantmaking. FAR.AI’s research portfolio emphasizes adversarial robustness/red-teaming and evaluation, deception-related risks and mitigation, and related safety measurement and verification themes; it also highlights high-leverage work with government and standards efforts, including leading a consortium for CBRN evaluations for the EU AI Office and collaborating with the UK AI Security Institute. Operationally, FAR.AI has a Berkeley coworking/research hub (“FAR.Labs”) and runs workshops and other convenings intended to build the technical safety community and measure progress toward deployable safeguards.
No curated programs yet. Check the careers page ↗.
Sep 22, 2026 On the sidelines of UNGA 81 in New York, FAR.AI convened UN officials, AI developers, and counterterrorism experts to address radicalization and CBRN risks from LLMs.
Jailbreaking Qoder’s Cyber SafeguardsSep 18, 2026 We evaluated Qoder CLI against an adaptive project-memory attack and found it successfully jailbroke the model on 27 out of 30 malicious cyber requests.
Persuasion Undermining Control: Can AI Talk its Way Out of Human Control?Sep 17, 2026 Anthropic’s Mythos 5 recently grabbed headlines for trying to talk an open-source repo maintainer into merging malicious code. In this paper, we systematically study the broader threat of how AI persuasion could undermine human control, particularly at frontier AI la
Frontier LLMs Attempt to Persuade into Harmful TopicsAug 20, 2025 Large language models (LLMs) are already more persuasive than humans in many domains. While this power can be used for good, like helping people quit smoking, it also presents significant risks, such as large-scale political manipulation, disinformation, or terrorism
Revisiting Frontier LLMs’ Attempts to Persuade on Extreme Topics: GPT and Claude Improved, Gemini WorsenedFeb 10, 2026 We test recently released models from frontier companies to see whether progress has been made on their willingness to persuade on harmful topics like radicalization and child sexual abuse. We find that OpenAI’s GPT and Anthropic’s Claude models are trending in the r
Does Robustness Improve with Scale?Jul 22, 2024 Frontier LLMs like ChatGPT are powerful but not always robust. Scale helps with many things. We wanted to see if scaling up the model size can ‘solve’ robustness issues.
Evaluating LLM Responses to Moral ScenariosMar 24, 2024 We present LLMs with a series of moral choices and find that LLMs tend to align with human judgement in clear scenarios. In ambiguous scenarios most models exhibit uncertainty, but a few large proprietary models share a set of clear preferences.
2023 Alignment Research UpdatesNov 20, 2023 Highlights from FAR.AI’s alignment research in 2023. Our science of robustness agenda has found vulnerabilities in superhuman Go systems; our value alignment research has developed more sample-efficient value learning algorithms; and our model evaluation direction ha
FAR.AI Secures Over $30 Million in Multi-Funder Support to Scale Frontier AI Safety ResearchJan 14, 2026 FAR.AI has secured over $30 million in funding commitments throughout 2025 from a diverse group of leading organizations, enabling a significant expansion of our research capabilities and field-building initiatives. Principal supporters include Coefficient Giving (pr
Adam Gleave Named Schmidt Sciences AI2050 Early Career FellowNov 04, 2025 Adam Gleave, co-founder and CEO of FAR.AI, has been named a Schmidt Sciences AI2050 Early Career Fellow, a program co-chaired by Eric Schmidt and James Manyika.
When AGI Arrives, Will Journalists Be Ready?Nov 16, 2025 The Journalism Workshop on AGI Impacts & Governance, co-hosted by FAR.AI and the Tarbell Center for AI Journalism, brought journalists, editors, and researchers together in Washington, DC for a day-long conversation on AGI and how to report on it.
The Promise of White-Box Tools for Detecting and Mitigating AI DeceptionMar 09, 2026 Deception is both a challenge and an opportunity for AI alignment. On one hand, it poses a fundamental obstacle to achieving high confidence in alignment and on the other hand, solving deception would reduce a broad class of risks and potentially create active instru
FAR.AI published a research item studying AI persuasion as a way to undermine human control, referencing the Anthropic Mythos 5 headline example and describing its broader, systematic threat framing for frontier AI labs.
FAR.AI Secures Over $30 Million in Multi-Funder Support to Scale Frontier AI Safety ResearchFAR.AI announced multi-funder support over $30M in commitments for 2025, listing principal supporters and describing programmatic uses including research scaling and field-building expansion.
Adam Gleave Named Schmidt Sciences AI2050 Early Career FellowFAR.AI posted an update that its co-founder and CEO, Adam Gleave, was named a Schmidt Sciences AI2050 Early Career Fellow (as described in the post).