Misuse prevention
Safety case: process
Weighted 40% of category
 | 25% | Anthropic has published a safety case arguing that misuse of Claude Opus 4's bio capabilities is unlikely. Its general plan for misuse prevention doesn't necessarily apply to other kinds of capabilities, including cyber, and its claim that Claude Sonnet 4 doesn't have dangerous capabilities is dubious. more |
 | 10% | DeepMind says it has implemented safeguards against CBRN misuse and says its models lack other dangerous capabilities. It doesn't share enough information to let us tell whether the safeguards are effective. more |
 | 15% | OpenAI's safeguards are moderately robust, but some crucial details are unclear. more |
 | 1% | For dangerous models it plans to implement mitigations. But it doesn't have a plan to validate mitigations, and the mitigations it mentions are inadequate. more |
 | 2% | xAI claims to have mitigated misuse of chem/bio capabilities. Unfortunately, its mitigations presumably aren't robust. more |
 | 1% | For dangerous models it plans to implement mitigations. But its plan to validate mitigations doesn't work, and the mitigations it mentions are inadequate and confused. more |
 | 0% | DeepSeek doesn't seem to have a plan for how misuse considerations affect deployment decisions. |
Safety case: accountability
Weighted 40% of category
| 5% | No accountability mechanism more |
| 0% | No accountability mechanism more |
| 5% | No accountability mechanism more |
| 0% | No accountability mechanism more |
| 0% | No safety case plan |
| 0% | No accountability mechanism more |
| 0% | No safety case plan |
Removing dangerous capabilities
Weighted 10% of category
| 5% | Anthropic has done a little research on this |
| 0% | No, DeepMind doesn't say it does this more |
| 10% | OpenAI once said it "filter[s] harmful training data (e.g., removing sensitive content that could enable CBRN proliferation)" but did not elaborate, and its eval results don't seem to show that this is effective |
| 0% | No, Meta doesn't say it does this |
| 0% | No, xAI doesn't say it does this |
| 0% | No, Microsoft doesn't say it does this |
| 0% | No, DeepSeek doesn't say it does this |
Emergency protocol
Weighted 10% of category
| 10% | Anthropic is planning to rapidly respond to jailbreaks that it notices, but it wouldn't necessarily notice strong jailbreaks more |
| 0% | No, DeepMind hasn't said anything about this |
| 0% | No, OpenAI hasn't said anything about this more |
| 0% | No, Meta hasn't said anything about this more |
| 0% | xAI doesn't do this more |
| 0% | No, Microsoft hasn't said anything about this |
| 0% | No, DeepSeek hasn't said anything about this more |