AI Lab Watch

Categories

Companies

Resources

Blog

About

			*
Weighted score	28%	22%	20%	4%	3%	3%	1%
Risk assessment	49%	34%	43%	1%	1%	1%	0%	28% weight
Scheming risk prevention	2%	8%	2%	2%	2%	2%	2%	21% weight
Boosting safety research	68%	55%	35%	28%	0%	15%	8%	13% weight
Misuse prevention	9%	4%	5%	0%	0%	0%	0%	12% weight
Prep for extreme security	2%	4%	0%	0%	0%	0%	0%	12% weight
Information sharing	35%	13%	32%	0%	28%	0%	0%	8% weight
Planning	14%	26%	0%	0%	0%	1%	0%	6% weight

Up to date as of May 21; not updated for Claude 4.

Overall score

Risk assessment

Evals: domains, quality, elicitation

Weighted 55% of category

	60%	Anthropic does model evals for dangerous capabilities; they're fine, but the elicitation quality is unclear (and it lacks scheming evals and difficult bio evals) more
	60%	DeepMind does model evals for dangerous capabilities; they're fine, but the elicitation is likely poor (and the bio evals are too easy) more
	70%	OpenAI does model evals for dangerous capabilities; they're fine, but the elicitation quality is unclear (and it lacks difficult bio evals) more
	2%	Meta does some evals for cyber and chem/bio capabilities — but it doesn't share much information, it's not clear whether the evals are good, and its elicitation is very poor more
	2%	xAI is planning to do a cyber capability eval and some simple bio capability evals, but no uplift experiments, AI R&D capability evals, or scheming evals, and it doesn't have an elicitation plan more
	2%	Microsoft is planning to do evals some dangerous capabilities, but its plan is very vague more
	0%	No, DeepSeek doesn't seem to have done evals for dangerous capabilities or have a plan

Evals: accountability

Weighted 25% of category

	15%	No accountability for internal evals; some external evals but details unclear more
	1%	No accountability more
	15%	No accountability for internal evals; some external evals but details unclear more
	0%	No accountability
	0%	No evals or accountability plan
	0%	No evals or accountability plan
	0%	No evals or plan

Adversarial evaluation for alignment

Weighted 10% of category

	50%	Anthropic is working on alignment audits.
	0%	DeepMind doesn't do this more
	10%	OpenAI doesn't do this more
	0%	Meta doesn't do this
	0%	xAI doesn't do this
	0%	Microsoft doesn't do this
	0%	DeepSeek doesn't do this

Model organisms

Weighted 10% of category

	75%	Anthropic does model organisms research and related "alignment auditing" research more
	10%	DeepMind pubishes a little research related to model organisms more
	0%	OpenAI doesn't do anything on this
	0%	Meta doesn't do anything on this
	0%	xAI doesn't do anything on this
	0%	Microsoft doesn't do anything on this
	0%	DeepSeek doesn't do anything on this