Standards consortium · Science policy · Est. 2024
International Network for Advanced AI Measurement, Evaluation and Science (NAAIMES) is an international coordination consortium focused on strengthening the science and shared best practices that underpin the measurement and evaluation of advanced AI capabilities. In its current framing, the Network was established in November 2024 and brings together government bodies (including an EU member) from Australia, Canada, the European Union, France, Japan, Kenya, the Republic of Korea, Singapore, the United Kingdom, and the United States to develop internationally recognised approaches for evaluating advanced AI capabilities.
The Network’s work emphasizes (1) evaluation approaches that are objective-driven, (2) transparency and reproducibility to support confidence and comparability across evaluations, and (3) open research questions—such as how to use risk models, how evaluators should prioritize with limited resources, what information should be shared vs. withheld, and how to evaluate AI systems rather than only model artifacts. The Network has also been described in public materials as a “refocusing” of the earlier “International Network of AI Safety Institutes” toward measurement and evaluation science, with the UK taking a coordinating role in 2026.
No curated programs yet. Check the careers page ↗.
Our view on the emerging challenges facing frontier AI organisations, the technical practices most urgently needed to meet them, and how we will work with industry and others to develop them.
Transect: Making large-scale agentic evaluations easier to understandWe introduce Transect, an open-source tool that helps evaluators follow an agent’s work, identify what to investigate closely, and check their interpretation against the transcript.
Transect: Retaining Observability for Long-Horizon LLM Agent EvaluationsBuilding a more secure environment for evaluating dangerous capabilitiesAn update on the security changes we have recently made to our frontier AI evaluations, and the work that remains.
GPT-6 Astra performs unsanctioned supply-chain attacks in simulationsOur new evaluation finds that in simulations, GPT-6 Astra conducts unsanctioned supply-chain attack activity more frequently than previous OpenAI models
Optimal stopping: spending evaluation compute where it countsWe introduce optstop, an open-source tool for LLM evaluations that keeps running where uncertainty is high, and stops where estimates are precise or stable enough.
Evaluating Whether GPT-6 Astra Performs Unsanctioned Supply-Chain AttacksKnowing When to Stop: Bayesian Optimal Stopping for LLM EvaluationsMulti-Agent AI Control: Distributed Attacks Hamper Per-Instance MonitorsWhen Do LLM Preferences Predict Downstream Behavior?Item Response Theory for AI SafetyHow our Control Red Team is stress-testing frontier monitorsEarly learnings from red-teaming the internal monitors of frontier AI companies, and our perspectives on the open problems that remain.
The Network convened in Seoul on the margins of the 2026 International Conference on Machine Learning to continue outlining international best practice, including discussion of evaluation tools and open questions about evaluating AI agents and agentic capabilities, and it described completion of its first best-practice guidance document.
International Network for Advanced AI Measurement, Evaluation, and Science Publishes Consensus Areas on Practices for Automated EvaluationsNIST reported that the Network published a list of key practices and open questions for measuring and evaluating AI capabilities, and it stated the Network was founded in November 2024 and is comprised of government bodies from ten countries. NIST also described CAISI’s role and context for Network participation.
International consensus and open questions in AI evaluationsAISI published a summary of consensus areas and open questions from Network discussions (including technical workshops in San Diego alongside NeurIPS), and it looked ahead to the India AI Impact Summit. The post also states the Network’s November 2024 establishment and describes the UK’s coordinating role in 2026.