Anthropic
Risk assessment 44%
AI companies should do model evals and uplift experiments to determine whether models have dangerous capabilities or how close they are. They should also prepare to check whether models will act well in high-stakes situations.
Evals: domains, quality, elicitation
50%Evals: accountability
15%Adversarial evaluation for alignment
50%Model organisms
75%Scheming risk prevention 4%
AIs show signs that if they were more capable, they would sometimes scheme, i.e. fake alignment and subvert safety measures in order to gain power. AI companies should prepare for risks from models scheming, especially during internal deployment: if they can't reliably prevent scheming, they should prepare to catch some schemers and deploy potentially scheming models safely.
Safety case: process
10%Safety case: accountability
0%Internal deployment protocol
0%External deployment protocol
0%Plan for if an AI is caught scheming
0%Training: remove scheming capabilities
0%Training: adversarial training
0%Training: safe architecture
50%Boosting safety research 70%
AI companies should do (extreme-risk-focused) safety research, and they should publish it to boost safety at other AI companies. Additionally, they should assist external safety researchers by sharing deep model access and mentoring.
Publishing safety research
85%Deep access for external safety researchers
5%Mentoring external safety researchers
100%Misuse prevention 14%
AI companies should prepare to prevent catastrophic misuse for deployments via API, once models are capable of enabling catastrophic harm.
Safety case: process
25%Safety case: accountability
5%Removing dangerous capabilities
5%Emergency protocol
10%Prep for extreme security 3%
AI companies should prepare to protect model weights and code by the time AI massively boosts R&D, even from top-priority operations by the top cyber-capable institutions.
Plan for SL5
0%Red-team resilience
0%Practices: isolated network
10%Practices: secure developers' machines
10%Practices: multiparty controls
25%Practices: secure boot
10%Track record
0%Risk info sharing 35%
AI companies should share information on incidents, risks, and capabilities — but not share some capabilities research.
Incident reporting
0%Talk about extreme risks
60%Describe worst-case outcome
0%Don't publish some capabilities research
75%Planning 14%
AI companies should plan for the possibility that dangerous capabilities appear soon and safety isn't easy: both for evaluating and improving safety of their systems and for using their systems to make the world safer.