Misuse prevention
Safety case: process
Weighted 40% of category
 | 20% | Anthropic is preparing to make a safety case for CBRN capabilities using its ASL-3 deployment standard. Its plan is reasonable but is somewhat abstract and doesn't apply to other capabilities including cyber. more |
 | 10% | Deepmind plans to implement mitigations for dangerous models, assess whether they are robust, and only publicly deploy if they are. This is good but its plan is vague. more |
 | 10% | OpenAI plans to implement safeguards for dangerous models. The safeguards and plan to validate them are on the right path, but they are very underspecified, and the thresholds are very vague. more |
 | 1% | For dangerous models it plans to implement mitigations. But it doesn't have a plan to validate mitigations, and the mitigations it mentions are inadequate. more |
 | 1% | For dangerous models it plans to implement mitigations. But its plan to validate mitigations doesn't work. more |
 | 1% | For dangerous models it plans to implement mitigations. But its plan to validate mitigations doesn't work, and the mitigations it mentions are inadequate and confused. more |
 | 0% | DeepSeek doesn't seem to have a plan for how misuse considerations affect deployment decisions. |
Safety case: accountability
Weighted 40% of category
| 0% | No accountability mechanism more |
| 0% | No accountability mechanism more |
| 0% | No accountability mechanism more |
| 0% | No accountability mechanism more |
| 0% | No safety case plan |
| 0% | No accountability mechanism more |
| 0% | No safety case plan |
Removing dangerous capabilities
Weighted 10% of category
| 5% | Anthropic has done a little research on this |
| 0% | No, DeepMind doesn't say it does this more |
| 10% | OpenAI says it "filter[s] harmful training data (e.g., removing sensitive content that could enable CBRN proliferation)" but does not elaborate, and its eval results don't seem to show that this is effective |
| 0% | No, Meta doesn't say it does this |
| 0% | No, xAI doesn't say it does this |
| 0% | No, Microsoft doesn't say it does this |
| 0% | No, DeepSeek doesn't say it does this |
Emergency protocol
Weighted 10% of category
| 0% | Anthropic doesn't do this more |
| 0% | No, DeepMind hasn't said anything about this |
| 0% | No, OpenAI hasn't said anything about this more |
| 0% | No, Meta hasn't said anything about this more |
| 0% | xAI doesn't do this more |
| 0% | No, Microsoft hasn't said anything about this |
| 0% | No, DeepSeek hasn't said anything about this more |