About

We develop quantitative risk models that translate AI capabilities into measurable harm. Our methodology breaks complex risks into quantifiable components and maps benchmark scores to probability estimates of real-world harm.

In the past, we have developed the first comprehensive AI risk management framework, adapting tried-and-tested practices from high-risk industries to AI systems. We’ve also partnered with Google DeepMind researchers on goal-directedness evaluations and pioneered quantitative AI risk estimation methodology using expert elicitation protocols.

Current focus: We've built nine risk models for AI-enabled cyberattacks spanning different threat actors, targets, and attack types. We're now expanding to include loss-of-control risks, CBRN threats, and harmful manipulation scenarios.

Impact: Our models help policymakers set evidence-based capability thresholds and enable the field to move from qualitative to quantitative assessments. National AI Safety Institutes are actively integrating our approaches into their evaluation frameworks.

We independently evaluate and rate how well leading AI companies manage risks. Our framework assesses companies across four dimensions: how they identify risks, assess their severity, implement safeguards, and govern their systems.

Current focus: Our public ratings create accountability through multiple channels: media coverage (featured twice in TIME and in Euractiv), market pressure from model deployers choosing safer systems, and investor decisions (Norway's $1.9 trillion sovereign wealth fund explicitly mentioned our framework as one of the external standards they were looking at).

Beyond ratings, we advise companies on developing Frontier Safety Frameworks. This advisory work is increasingly critical as California's SB53 requires companies to comply with their own frameworks, and the EU Code of Practice requires general-purpose AI companies to make a safety and security framework. We've contributed to frameworks for G42 and a major AI company.

Impact: We continue our work to increase accountability for AI companies, provide specific guidance on how their risk management can be improved, and create market incentives for better practices. We're scaling this work and exploring additional mechanisms, such as ESG frameworks, to reward responsible AI development.

We shape the technical foundations of AI regulation through leadership positions in key standards bodies and direct collaboration with governments. Our contributions to date include:

◆ James Gealy, our Standardization Lead, leads the high-risk AI systems risk management standard at CEN-CENELEC JTC 21 (the body writing technical specifications for EU AI Act compliance)

◆ James also serves as editor of the generative AI red teaming standard at ISO

◆ We participated in all four working groups developing the EU's Code of Practice for general-purpose AI models

◆ We were the only AI safety organization invited to the OECD taskforce working on the reporting framework of the Hiroshima AI Process

Current focus: We're contributing to the development of harmonized standards that will define how companies demonstrate AI Act compliance. We're also advising national AI Safety Institutes on implementing quantitative risk modeling.

Impact: Our standards work establishes the technical infrastructure that makes AI regulation enforceable. By codifying risk management requirements and evaluation methodologies, we're translating policy intent into concrete technical specifications that companies follow.