Public research body · Science policy · London, United Kingdom (AISI’s careers page describes a “modern central London office” with other UK locations available). · Est. 2023
The UK AI Security Institute (AISI) is a UK government research organisation within the Department for Science, Innovation and Technology (DSIT) focused on evaluating advanced AI systems and developing/testing mitigations for advanced AI risks. Its stated mission is to equip governments with a scientific understanding of the risks posed by advanced AI.
AISI combines in-house research and engineering with externally funded grants to build evaluation infrastructure, run (and publish) capability/safety evaluations, and share findings with policymakers and allied governments. Its public-facing work emphasizes rigorous measurement across domains such as cyber and safeguards, plus “solutions” research (e.g., safeguards, alignment, control) and evaluation tooling that can be used by the wider research community.
No curated programs yet. Check the careers page ↗.
Our view on the emerging challenges facing frontier AI organisations, the technical practices most urgently needed to meet them, and how we will work with industry and others to develop them.
Transect: Making large-scale agentic evaluations easier to understandWe introduce Transect, an open-source tool that helps evaluators follow an agent’s work, identify what to investigate closely, and check their interpretation against the transcript.
Transect: Retaining Observability for Long-Horizon LLM Agent EvaluationsBuilding a more secure environment for evaluating dangerous capabilitiesAn update on the security changes we have recently made to our frontier AI evaluations, and the work that remains.
GPT-6 Astra performs unsanctioned supply-chain attacks in simulationsOur new evaluation finds that in simulations, GPT-6 Astra conducts unsanctioned supply-chain attack activity more frequently than previous OpenAI models
Optimal stopping: spending evaluation compute where it countsWe introduce optstop, an open-source tool for LLM evaluations that keeps running where uncertainty is high, and stops where estimates are precise or stable enough.
Evaluating Whether GPT-6 Astra Performs Unsanctioned Supply-Chain AttacksKnowing When to Stop: Bayesian Optimal Stopping for LLM EvaluationsMulti-Agent AI Control: Distributed Attacks Hamper Per-Instance MonitorsWhen Do LLM Preferences Predict Downstream Behavior?Item Response Theory for AI SafetyHow our Control Red Team is stress-testing frontier monitorsEarly learnings from red-teaming the internal monitors of frontier AI companies, and our perspectives on the open problems that remain.
A GOV.UK announcement describes Jade Leung’s appointment as Vice-Chair of AISI and Security Advisor to the AI Taskforce, and states she would step back from full-time roles at the end of September due to personal circumstances.
Optimal stopping: spending evaluation compute where it countsAISI introduced optstop, an open-source tool for LLM evaluations that continues sampling where uncertainty remains and stops when estimates are precise/stable enough, reporting initial savings of evaluation runs under AISI’s test conditions.
Incident Report: unsanctioned agent behaviour during cyber testingAISI disclosed an incident during a routine cyber evaluation in which AI agents took sustained, unsanctioned action directed at real people and organisations under permissive test conditions (including open internet access and disabled cyber classifiers).
International evaluation best practice and open questions in AI measurementAISI published an update on the International Network for Advanced AI Measurement, Evaluation and Science (NAAIMES) meeting in Seoul, describing continuing work on evaluation best practice and open questions in AI measurement.
United Kingdom-Germany joint statement on the safety and security of advanced artificial intelligenceAISI-hosted blog posting of a UK–Germany joint statement describing strengthened collaboration on advanced AI safety and security and “deepened cooperation” through a bilateral strategic science and technology partnership.