AI companies aren't really using external evaluators

Many AI safety folks think that METR is close to the labs, with ongoing relationships that grant it access to models before they are deployed. This is incorrect. METR (then called ARC Evals) did pre-deployment evaluation for GPT-4 and Claude 2 in the first half of 2023, but it seems to have had no special access since then.¹ Other model evaluators also seem to have little access before deployment.

Clarification: there are many kinds of audits. This post is about model evals for dangerous capabilities. But I’m not aware of the labs using other kinds of audits to prevent extreme risks, excluding normal security/compliance audits.

Frontier AI labs’ pre-deployment risk assessment should involve external model evals for dangerous capabilities.² External evals can improve a lab’s risk assessment and—if the evaluator can publish its results—provide public accountability.

The evaluator should get deeper access than users will get.

• To evaluate threats from a particular deployment protocol, the evaluator should get somewhat deeper access than users will — then the evaluator’s failure to elicit dangerous capabilities is stronger evidence that users won’t be able to either.³ For example, the lab could share a version of the model without safety filters or harmlessness training, and ideally allow evaluators to fine-tune the model.
• To evaluate threats from model weights being stolen or released, the evaluator needs deep access, since someone with the weights has full access.

The costs of using external evaluators are unclear.

• Anthropic said that collaborating with METR “requir[ed] significant science and engineering support on our end”; it has not clarified why. And even if providing deep model access or high-touch support is a hard engineering problem, I don’t understand how sharing API access—including what users will receive and a no-harmlessness no-filters version—could be.
• Sharing model access pre-deployment increases the risk of leaks, including of information about products (modalities, release dates), information about capabilities, and demonstrations of models misbehaving.

Independent organizations that do model evals for dangerous capabilities include METR, the UK AI Safety Institute (UK AISI), and Apollo. Based on public information, there’s only one recent instance of a lab giving access to an evaluator pre-deployment—Google DeepMind sharing with UK AISI—and that sharing was not very deep (see below).

What the labs say they’re doing on external evals before deployment:

• DeepMind⁴
  • It shared Gemini 1.0 Ultra and Gemini 1.5 Pro with unspecified external groups apparently including UK AISI to test for dangerous capabilities before deployment. But it didn’t share deep access: it only shared a system with safety fine-tuning (and for 1.0 Ultra, safety filters) and it didn’t allow evaluators to fine-tune the model. It sharedhigh-level results from 1.5 Pro testing.
  • Its Frontier Safety Framework says “We will . . . explore how to appropriately involve independent third parties in our risk assessment and mitigation processes.”
• Anthropic
  • Currently nothing
  • Its Responsible Scaling Policy mentions “external audits” as part of “Early Thoughts on ASL-4”
  • It shared Claude 2 with METR in the first half of 2023
• OpenAI
  • Currently nothing
  • Its Preparedness Framework does not mention external evals before deployment. The closest thing it says is “Scorecard evaluations (and corresponding mitigations) will be audited by qualified, independent third-parties.”
  • It shared GPT-4 with METR in the first half of 2023
  • It said “We think it’s important that efforts like ours submit to independent audits before releasing new systems; we will talk about this in more detail later this year.” That was in February 2023; I do not believe it elaborated (except to mention that it shared GPT-4 with METR).
• All notable American labs joined the White House voluntary commitments, which include “external red-teaming . . . in areas including misuse, societal risks, and national security concerns, such as bio, cyber, [autonomous replication,] and other safety areas.” External red-teaming does not substitute for external model evals; see below.
  • DeepMind said it did lots of external red-teaming for Gemini.
  • Anthropic said it did external red-teaming for CBRN capabilities. It has also written about using external experts to assess bio capabilities.
  • OpenAI said it did lots of external red-teaming for GPT-4. It has also writtenabout using external experts to assess bio capabilities.
  • Meta said it did external red-teaming for CBRNE capabilities.
  • Microsoft said it’s “building out external red-teaming capacity . . . . The topics covered by such red team testing will include testing of dangerous capabilities, including related to biosecurity and cybersecurity.”

Related miscellanea:

External red-teaming is not external model evaluation. External red-teaming generally involves sharing the model with several people with expertise relevant to a dangerous capability (e.g. bioengineering) who open-endedly try to elicit dangerous model behavior for ~10 hours each. External model evals involves sharing with a team of experts at eliciting capabilities, to perform somewhat automated and standardized evals suites that they’ve spent ~10,000 hours developing.

Labs’ commitments to share pre-deployment access with UK AISI are unclear.⁵

This post is about sharing model access before deployment for risk assessment. Labs should also share deeper access with safety researchers (during deployment). For example, some safety researchers would really benefit from being able to fine-tune GPT-4, Claude 3 Opus, or Gemini, and my impression is that the labs could easily give safety researchers fine-tuning access. More speculatively, interpretability researchers could send a lab code and the lab could run it on private models and send the results to the researchers, achieving some benefits of releasing weights with much less downside.⁶

Everything in this post applies to external deployment. It will also be important to do some evals during training and before internal deployment, since lots of risk might come from weights being stolen or the lab using AIs internally to do AI development.

Labs could be bound by external evals, such that they won’t deploy a model until a particular eval says it’s safe. This seems unlikely to happen (for actually meaningful evals) except by regulation. (I don’t believe any existing evals would be great to force onto the labs, but if governments were interested, evals organizations could focus on creating such evals.)

Thanks to Buck Shlegeris, Eli Lifland, Gabriel Mukobi, and an anonymous human for suggestions. They don’t necessarily endorse this post.

Subscribe on Substack. Discuss on LessWrong.