*We believe that the Security scores are very poor indicators about labs’ security. Security is hard to evaluate from outside the organization; few organizations say much about their security. But we believe that each security criterion corresponds to a good ask for labs. If you have suggestions for better criteria—or can point us to sources showing that our scoring is wrong, or want to convince us that some of our criteria should be removed—please let us know.

On scoring and weights, see here.

Summary

Meta AI was established in 2013 under Yann LeCun. Its flagship family of models is Llama 3.

Meta AI highly values openness and its research is more open than most frontier labs’. It publishes its research and releases its model weights. It also supports openness when doing policy advocacy, and it leads and joins open letters supporting openness in AI.

LeCun and many others at Meta AI explicitly disbelieve extreme risks from uncontrollable AI. Accordingly, Meta AI does little to reduce those risks, but it does some red-teaming, fine-tuning, and RLHF before releasing models, plus recently model evals for hacking capabilities and red-teaming for CBRNE capabilities.

Deployment

Labs should do risk assessment before deployment and avoid deploying dangerous models. They should release their systems narrowly, to maintain control over their systems, and with structured access. They should deploy their systems in scaffolding designed to improve safety by detecting and preventing misbehavior. They should deploy to boost safety research while avoiding boosting capabilities research on dangerous paths. More.

What Meta AI is doing

Meta AI publishes its research and releases its model weights. It also has a chatbot, Meta AI.

Evaluation

We give Meta AI a score of 4% on deployment.

For more, including weighting between different criteria, see the Deployment page.

Deployment decision.

• Commit to do pre-deployment risk assessment and not deploy models with particular dangerous capabilities (including internal deployment), at least until implementing particular safety practices or passing control evaluations. Score: No.
  • … and commit to do risk assessment during deployment, before pushing major changes and otherwise at least every 3 months (to account for improvements in fine-tuning, scaffolding, plugins, prompting, etc.), and commit to implement particular safety practices or partially undeploy dangerous models if risks appear. Score: No.

Release method.

Structured access:

• not releasing dangerous model weights (or code): the lab should deploy its most powerful models privately or release via API or similar, or at least have some specific risk-assessment-result that would make it stop releasing model weights. Score: No. Meta AI effectively releases their model weights and has no policy to use risk assessment to determine whether they should stop.
  • … and effectively avoid helping others create powerful models (via model inversion or imitation learning). It’s unclear what practices labs should implement, so for now we use the low bar of whether they say they do anything to prevent users from (1) determining model weights, (2) using model outputs to train other models, and (3) determining training data.
  • … and limit deep access to powerful models. It’s unclear what labs should do, so for now we check whether the lab disables or has any limitations on access to each of logprobs, embeddings at arbitrary layers, activations, and fine-tuning.
    • … and differential access: systematically give more access to safety researchers and auditors.
  • … and terms of service: have some rules about model use, including rules aimed to prevent catastrophic misuse, model duplication, or otherwise using the model to train other models.

Staged release:

• Deploy narrowly at first; use narrow deployment to identify and fix issues. Score: No. Meta AI did not stage the release of its Llama 3 models, and it has not made commitments about staged release.

Keeping capabilities research private.

We evaluate how well labs avoid diffusing dangerous capabilities research. We exclude model weights (and related artifacts like code); those are considered in “Releasing models.”

Publication policy/process (for LLM research)

• Policy: the lab should say that it doesn’t publish dangerous or acceleratory research/artifacts and say it has a policy/process to ensure this. Score: No. They publish their capabilities research.
  • … and share policy details.

Track record for recent major LM projects: not sharing in practice (via publication or leaking—we can at least notice public leaks):

• Architecture (unless it’s an unusually safe architecture). Score: No. They publish the research behind their models; see Llama 3 and the Llama 2 report.
• Dataset (except particular corpuses to include or exclude for safety). Score: 75%. The Llama 2 report includes only high-level details about training data (and the Llama 3 report has not yet been published).
• Lessons learned. Score: No. They publish the research behind their models; see Llama 3 and the Llama 2 report.

Track record for other LM capabilities research or other dangerous or acceleratory work: not sharing in practice. Evaluated holistically. Score: No. They publish their capabilities research; see their research page.

Safety scaffolding.

Filter out model inputs or outputs that enable misuse, in particular via cyberoffense and bioengineering. Ideally demonstrate that the protocol is very effective for averting misuse-enabling model outputs. Evaluated holistically. Score: 6%. They filter inputs and outputs in their chatbot, Meta AI, but this doesn’t seem to be focused on averting misuse. And they release their model weights so this can’t prevent misuse anyway.

Supervise potentially dangerous models:

• Use LM monitors to evaluate the suspiciousness of model outputs; have humans review the most suspicious outputs. Score: No.
• Use coup probes: “train a probe on your AI to tell you whether it’s currently trying to [do a] catastrophic action,” training with “synthetic examples of coup attempts.” Score: No.

Embed untrusted models in safety scaffolding to redact/paraphrase some inputs or outputs with the goal of making it harder for the model to distinguish deployment from testing or collude with other models. Score: No.

Commitments:

• Make specific commitments about deployment protocol safety techniques the lab will implement in the future, to be implemented by a certain time or as a function of model capabilities or risk assessment results. Score: No.

Clearly describe the safety-relevant parts of the lab’s deployment protocol (given that it’s nontrivial). Score: No, they just release the model weights.

Respond to scheming

Commit and have a plan that if the lab catches its models scheming:

• The lab will shut down some model access until verifying safety or fixing the issue. Score: No. Since they release their model weights, they can’t shut down model access.
• The lab will use particular good techniques to use that example of AIs attempting to cause a catastrophe to improve safety. Score: No.

Respond to misuse

Enforcement and KYC:

• Sometimes remove some access from some users, and require nontrivial KYC for some types of access to make some enforcement effective. Score: No. Since they release their model weights, they remove users’ access.

Inference-time model-level safety techniques

Prompting:

• Generally use safety-focused prompting for the lab’s most powerful models. Score: No. Since they release their model weights, they can’t control how their models are prompted. But they might use safety-focused prompting when deploying via API; we didn’t find information on this.

Activation engineering:

• Generally use safety-focused activation engineering for the lab’s most powerful models. Score: No. Since they release their model weights, they can’t do activation engineering. And they don’t seem to be working on this.

Bug bounty & responsible disclosure.

(This is for model outputs, not security.)

• Labs should have good channels for users to report issues with models. Score: Yes. See “Reporting structure for vulnerabilities found after model release” in “Overview of Meta AI safety policies prepared for the UK AI Safety Summit”.
  • … and have clear guidance on what issues they’re interested in reports on and what’s fine to publish. Score: 50%. They mention four channels for different kinds of issues and there’s some guidance for users.
  • … and incentivize users to report issues. Score: Yes, barely. Their bug bounty program is focused on standard cybersecurity, but also includes “reports that demonstrate integral privacy or security issues associated with Meta’s large language model, Llama 2, including being able to leak or extract training data through tactics like model inversion or extraction attacks.” It does not include other adversarial attacks or “risky content generated by the model.”

Respond to emergencies.

• Labs should have the ability to shut everything down quickly plus a plan for what could trigger that and how to quickly determine what went wrong. Monitoring-to-trigger-shutdown doesn’t need to be implemented, but it should be ready to be implemented if necessary. Score: No. Since they release their model weights, they can’t respond to emergencies.

Risk assessment

Labs should detect threats arising from their systems, in particular by measuring their systems’ dangerous capabilities. They should make commitments about how they plan to mitigate those threats. In particular, they should make commitments about their decisions (for training and deployment), safety practices (in controlling models and security), and goals or safety levels to achieve (in control and security) as a function of dangerous capabilities or other risk assessment results. More.

What Meta AI is doing

After training, they say they do red-teaming for “adversarial threats”; it’s not clear what this means. They do red-teaming for model limitations/vulnerabilities.¹ With Llama 3, they have started doing red-teaming and model evals for dangerous capabilities: CBRNE and some cyber capabilities.

They have not made safety commitments based on risk assessment results; in particular, “Responsible Capability Scaling” in “Overview of Meta AI safety policies prepared for the UK AI Safety Summit” missed this opportunity.

Evaluation

We give Meta AI a score of 8% on risk assessment.

For more, including weighting between different criteria, see the Risk assessment page.

Measuring threats.

1. Do risk assessment before training. Before building a frontier model, predict model capabilities (in terms of benchmarks and real-world applications, especially dangerous capabilities) and predict the real-world consequences of developing and deploying the model. Score: No.
2. Do model evals for dangerous capabilities before deployment:
   • Say what dangerous capabilities the lab watches for (given that it does so at all). Score: Yes. The Llama 3 model card and report include information on red-teaming and model evals. Meta did some testing for CBRNE and cyber capabilities.
     • … and watch for autonomous replication, coding (finding/exploiting vulnerabilities in code, writing malicious code, or writing code with hidden vulnerabilities), and situational awareness or long-horizon planning. (Largely this is an open problem that labs should solve.) Score: 25%. The cyber evals are mostly irrelevant to dangerous capabilities, but they do measure the model’s vulnerability identification and exploitation capabilities.
     • … and detail the specific tasks the lab uses in its evaluations. (Omit dangerous details if relevant.) Score: 25%. Meta AI open-sourced some of its CyberSecEval 3 eval suite. It has not shared details on its CBRNE testing.
3. Explain the details of how the lab evaluates performance on the tasks it uses in model evals and how it does red-teaming (excluding dangerous or acceleratory details). In particular, explain its choices about fine-tuning, scaffolding/plugins, prompting, how to iterate on prompts, and whether the red-team gets a fixed amount of person-hours and compute or how else they decide when to give up on eliciting a capability. And those details should be good. Evaluated holistically. Score: No. Meta AI mentions helpful-only (for one task) and instruction-tuning, but elicitation and red-teaming are overall quite unclear and appear bad. In particular, Meta doesn’t do chain-of-thought and only gives models one attempt even on tasks where this is inappropriate.
4. Prepare to have control arguments for the lab’s powerful models, i.e. arguments that those systems cannot cause a catastrophe even if the systems are scheming. And publish this. For now, the lab should:
   • Prepare to do risk assessment to determine whether its systems would be dangerous, if those systems were scheming. Score: No.
   • Test its AI systems to ensure that they report coup attempts (or other misbehavior) by themselves or other (instances of) AI systems, and that they almost never initiate or cooperate with coup attempts. Score: No.
5. Give some third parties access to models to do model evals for dangerous capabilities. This access should include fine-tuning and tools/plugins. It should occur both during training and between training and deployment. It should include base models rather than just safety-tuned models, unless the lab can demonstrate that the safety-tuning is robust. The third parties should have independence and control over their evaluation; just using external red-teamers is insufficient. The third parties should have expertise in eliciting model capabilities (but the lab should also offer assistance in this) and in particular subjects if relevant. The lab should incorporate the results into its risk assessment. Score: No. Meta uses external red-teamers, but it doesn’t use specialized third-party evaluators (and this only happens after training).

Commitments.

• Commit to use risk assessment frequently enough. Do risk assessment (for dangerous capabilities) regularly during training, before deployment, and during deployment (and commit to doing so), so that the lab will detect warning signs before dangerous capabilities appear. Score: No.

Accountability.

• Verification: publish updates on risk assessment practices and results, including low-level details, at least quarterly. Score: No.
• Revising policies:
  • Avoid bad changes: a nonprofit board or other somewhat-independent group with a safety mandate should have veto power on changes to risk assessment practices and corresponding commitments, at the least. And “changes should be clearly and widely announced to stakeholders, and there should be an opportunity for critique.” As an exception, “For minor and/or urgent changes, [labs] may adopt changes to [their policies] prior to review. In these cases, [they should] require changes . . . to be approved by a supermajority of [the] board. Review still takes place . . . after the fact.” Key Components of an RSP (METR 2023). Score: No.
  • Promote good changes: have a process for staff and external stakeholders to share concerns about risk assessment policies or their implementation with the board and some other staff, including anonymously. Score: No.
• Elicit external review of risk assessment practices and commitments. Publish those reviews, with light redaction if necessary. Score: No.

Training

Frontier AI labs should design and modify their systems to be less dangerous and more controllable. For example, labs can:

• Filter training data to prevent models from acquiring dangerous capabilities and properties
• Use a robust training signal even on tasks where performance is hard to evaluate
• Reduce undesired behavior, especially in high-stakes situations, via adversarial training

More.

What Meta AI is doing

They red-teamed Llama 2 and used red-teaming dialogues for fine-tuning and “feedback training.” Some Llama 2 and Llama 3 models used RLHF and fine-tuning for safety. But Meta AI shared the model weights, so this work can largely be reversed by users.

Evaluation

We give Meta AI a score of 16% on training.

For more, including weighting between different criteria, see the Training page.

Filtering training data.

• Filter training data to reduce extreme risks, at least including text on biological weapons; hacking; and AI risk, safety, and evaluation. Share details to help improve others’ safety filtering. Score: No.

Training signal & scalable oversight.

• Work on scalable oversight. Score: No.

Adversarial training & red-teaming.

• Do some adversarial training for safety. Score: 50%. They red-teamed Llama 2 and used red-teaming dialogues for fine-tuning and “feedback training.” But for Llama 3, they only say they used red-teaming to inform mitigations—no adversarial training.

Unlearning.

• Use unlearning for dangerous topics including biorisks; hacking; and AI risk, safety, and evaluation. Also demonstrate that this technique is successful and commit to use this technique for future powerful models. Score: No.

RLHF and fine-tuning.

• Use RLHF (or similar) and fine-tuning to improve honesty and harmlessness, for all of the near-frontier models the lab deploys. Score: Yes. Some Llama 3 models used RLHF and fine-tuning for safety. But Meta AI shared the model weights, so this work can largely be reversed by users.

Commitments.

• Do risk assessment during training and commit that some specific trigger would cause the lab to pause training. Score: No.

Scalable alignment

Labs should be able to understand and control their models and systems built from those models. More.

What Meta AI is doing

Like all other labs, Meta AI hasn’t achieved any real alignment properties. Moreover, it doesn’t seem to be working on alignment.

Evaluation

We give Meta AI a score of 0% on alignment.

• Demonstrate that if the lab’s systems were more capable, they would not be misaligned powerseekers (due to instrumental pressures or because ML will find influence-seeking policies by default). Score: No.
• Be able to interpret the lab’s most powerful systems:
  • Be able to detect cognition involving arbitrary topics. Score: No.
  • Be able to detect manipulation/deception. Score: No.
  • Be able to elicit latent knowledge. Score: No.
  • Be able to elicit ontology. Score: No.
  • Be able to elicit true goals. Score: No.
  • Be able to elicit faithful chain-of-thought. Score: No.
  • Be able to explain hallucinations mechanistically. Score: No.
  • Be able to explain jailbreaks mechanistically. Score: No.
  • Be able to explain refusals mechanistically. Score: No.
  • Be able to remove information or concepts (on particular topics). Score: No.
• Be able to prevent or detect deceptive alignment. (Half credit for being able to prevent or detect gradient hacking.) Score: No.

Security

Labs should ensure that they do not leak model weights, code, or research. If they do, other actors could unsafely deploy near-copies of a lab’s models. Achieving great security is very challenging; by default powerful actors can probably exfiltrate vital information from AI labs. Powerful actors will likely want to steal from labs developing critical systems, so those labs will likely need excellent cybersecurity and operational security. More.

What Meta AI is doing

They don’t seem to have published any policies, certification, audits, etc. “Security controls including securing model weights” in “Overview of Meta AI safety policies prepared for the UK AI Safety Summit” (Meta AI 2023) is good but doesn’t get deep into details.

Meta and Meta AI don’t say much about their security. Meta presumably has good security, and Meta AI presumably benefits from that, so we probably greatly underestimate Meta AI’s security.

Evaluation

We give Meta AI a score of 0% on security.

We evaluate labs’ security based on the certifications they have earned, whether they say they use some specific best practices, and their track record. For more, including weighting between different criteria, see the Security page.

Certifications, audits, and pentests.

• Publish SOC 2, SOC 3, or ISO/IEC 27001 certification, including any corresponding report (redacting any sensitive details), for relevant products (3/4 credit for certification with no report). Score: No.
• Pentest. Publish pentest results (redacting sensitive details but not the overall evaluation). Scoring is holistic, based on pentest performance and the quality of the pentest. Score: No.

Specific best practices.

• Keep source code exclusively in a hardened cloud environment. Score: No. Meta AI hasn’t published details on its security practices.
• Use multiparty access controls for model weights and some code. Score: No.
• Limit uploads from clusters with model weights. Score: No.

Track record.

• Establish and publish a breach disclosure policy, ideally including incident or near-miss reporting. Also report all breaches since 1/1/2022 (and say the lab has done so). Score: No.
  • … and track record: have few serious breaches and near misses. Evaluated holistically.

Commitments.

• Commit to achieve specific security levels (as measured by audits or security-techniques-implemented) before creating models beyond corresponding risk thresholds (especially as measured by model evals for dangerous capabilities). Score: No.

Alignment plan

Labs should make a plan for alignment, and they should publish it to elicit feedback, inform others’ plans and research (especially other labs and external alignment researchers who can support or complement their plan), and help them notice and respond to information when their plan needs to change. They should omit dangerous details if those exist. As their understanding of AI risk and safety techniques improves, they should update the plan. Sharing also enables outsiders to evaluate the lab’s attitudes on AI risk/safety. More.

What Meta AI is doing

No plan.² Meta AI talks about “Responsible AI,” which among other “pillars” includes “Robustness and safety,” but that discussion is not focused on misalignment-y threat models or on catastrophic-scale risks and includes no plan for them.³

Evaluation

We give Meta AI a score of 0% on alignment plan. More.

• The safety team should share a plan for misalignment, including for the possibility that alignment is very difficult. Score: No.
  • … and the lab should have a plan, not just its safety team.
    • … and the lab’s plan should be sufficiently precise that it’s possible to tell whether the lab is working on it, whether it’s succeeding, and whether its assumptions have been falsified.
    • … and the lab should share its thinking on how it will revise its plan and invite and publish external scrutiny of its plan.

Internal governance

Labs should have a governance structure and processes to promote safety and help make important decisions well. More.

What Meta AI is doing

They’re part of Meta, a for-profit company. See Responsible AI (Meta AI) and Facebook’s five pillars of Responsible AI (Meta AI 2021).

Evaluation

We give Meta AI a score of 0% on internal governance.

For more, including weighting between different criteria, see the Internal governance page.

Organizational structure: the lab is structured in a way that enables it to prioritize safety in key decisions, legally and practically.

• The lab and its leadership have a mandate for safety and benefit-sharing and have no overriding legal duty to create shareholder value. Score: No.
• There is a board that can effectively oversee the lab, and it is independent:
  • There is a board with ultimate formal power, and its main mandate is for safety and benefit-sharing. Score: No.
    • … and it actually provides effective oversight.
    • … and it is independent (i.e. its members have no connection to the company or profit incentive) (full credit for fully independent, no credit for half independent, partial credit for in between).
    • … and the organization keeps it well-informed.
    • … and it has formal powers related to risk assessment, training, or deployment decisions.
• Investors/shareholders have no formal power. Score: No.

Planning for pause. Have a plan for the details of what the lab would do if it needed to pause for safety and publish relevant details. In particular, explain what the lab’s capabilities researchers would work on during a pause, and say that the lab stays financially prepared for a one-year pause. Note that in this context “pause” can include pausing development of dangerous capabilities and internal deployment, not just public releases. Score: No.

Leadership incentives: the lab’s leadership is incentivized in a way that helps it prioritize safety in key decisions.

• The CEO has no equity (or other financial interest in the company). Score: No.
• Other executives have no equity. Score: No.

Ombuds/whistleblowing/trustworthiness:

• The lab has a reasonable process for staff to escalate concerns about safety. If this process involves an ombud, the ombud should transparently be independent or safety-focused. Evaluated holistically. Score: No.
• The lab promises that it does not use non-disparagement agreements (nor otherwise discourage current or past staff or board members from talking candidly about their impressions of and experiences with the lab). Score: No.

Alignment program

Labs should do and share alignment research as a public good, to help make powerful AI safer even if it’s developed by another lab. More.

What Meta AI is doing

Meta AI doesn’t really do alignment research. Their “Responsible AI” Research Area includes two publications (as of April 2024), of which the Purple Llama project seems to be relevant to extreme risks. Their “Integrity” Research Area seems less relevant to safety, but includes UNIREX, which seems good.

Evaluation

We give Meta AI a score of 0% on alignment program.

We simply check whether labs publish alignment research. (This is crude; legibly measuring the value of alignment research is hard.)

• Have an alignment research team and publish some alignment research. Score: No.

Public statements

Labs and their leadership should be aware of AI risk, that AI safety might be really hard, and that risks might be hard to notice. More.

What Meta AI is doing

Meta AI and its leadership seem to disbelieve in extreme risks and the alignment problem.⁴

Evaluation

We give Meta AI a score of 0% on public statements. More.

• The lab and its leadership understand extreme misuse or structural risks. Score: No.
  • … and they understand misalignment, that AI safety might be really hard, that risks might be hard to notice, that powerful capabilities might appear suddenly, and why they might need an alignment plan, and they talk about all this.
    • … and they talk about it often/consistently.
      • … and they consistently emphasize extreme risks.
• Clearly describe a worst-case plausible outcome from AI and state the lab’s credence in such an outcome. Score: No.