*We believe that the Security scores are very poor indicators about labs’ security. Security is hard to evaluate from outside the organization; few organizations say much about their security. But we believe that each security criterion corresponds to a good ask for labs. If you have suggestions for better criteria—or can point us to sources showing that our scoring is wrong, or want to convince us that some of our criteria should be removed—please let us know.

On scoring and weights, see here.

Summary

Microsoft does AI research and trains AI models. Its most powerful models are the Phi-3 family. It has not yet created frontier models, but it does provide access to others’ frontier models on its platform Azure. In addition to deploying models via Azure, Microsoft deploys GPT-4 via Copilot (formerly known as Bing Chat).

Microsoft has a strong partnership with OpenAI. The details are secret, but it seems that OpenAI is required to share its models (and some other IP) with Microsoft until OpenAI attains “a highly autonomous system that outperforms humans at most economically valuable work.” Microsoft also partners with Meta AI, Mistral AI, and Inflection AI to deploy their models via Azure.

Microsoft sometimes writes about its safety practices and commitments. It responded to the UK request for information about AI companies’ safety policies¹ and substantively commented on the White House commitments. Microsoft also writes about Responsible AI, but these practices seem irrelevant to extreme risks.

Microsoft does not do alignment research or have an alignment plan.

Some of Microsoft’s leadership is concerned about extreme risks from AI. In particular, CTO Kevin Scott and Chief Scientific Officer Eric Horvitz signed the CAIS statement. But CEO Satya Nadella seems less concerned, and Microsoft itself never articulates concerns about extreme risks.

Deployment

Labs should do risk assessment before deployment and avoid deploying dangerous models. They should release their systems narrowly, to maintain control over their systems, and with structured access. They should deploy their systems in scaffolding designed to improve safety by detecting and preventing misbehavior. They should deploy to boost safety research while avoiding boosting capabilities research on dangerous paths. More.

What Microsoft is doing

Microsoft publishes its research and releases its model weight. Its most powerful model is Phi-3. It also provides access to others’ frontier models on its platform Azure.² In addition to deploying models via Azure, Microsoft deploys GPT-4 via Copilot (formerly known as Bing Chat).

The role of safety considerations in Microsoft’s deployment decisions is unclear. In particular, Microsoft says:

When it comes to frontier model deployment, Microsoft and OpenAI have together defined capability thresholds that act as a trigger to review models in advance of their first release or downstream deployment. The scope of a review, through our joint Microsoft-OpenAI Deployment Safety Board (DSB), includes model capability discovery. . . . We have exercised this review process with respect to several frontier models, including GPT-4.

This sounds good, but Microsoft has not elaborated on these “capability thresholds,” shared details about the DSB, or shared details about past reviews.

Evaluation

We give Microsoft a score of 3% on deployment.

For more, including weighting between different criteria, see the Deployment page.

Deployment decision.

• Commit to do pre-deployment risk assessment and not deploy models with particular dangerous capabilities (including internal deployment), at least until implementing particular safety practices or passing control evaluations. Score: No. But see Microsoft’s AI Safety Policies (Microsoft 2023).
  • … and commit to do risk assessment during deployment, before pushing major changes and otherwise at least every 3 months (to account for improvements in fine-tuning, scaffolding, plugins, prompting, etc.), and commit to implement particular safety practices or partially undeploy dangerous models if risks appear.

Release method.

Structured access:

• not releasing dangerous model weights (or code): the lab should deploy its most powerful models privately or release via API or similar, or at least have some specific risk-assessment-result that would make it stop releasing model weights. Score: No. Microsoft releases their model weights and has no policy to use risk assessment to determine whether they should stop.
  • … and effectively avoid helping others create powerful models (via model inversion or imitation learning). It’s unclear what practices labs should implement, so for now we use the low bar of whether they say they do anything to prevent users from (1) determining model weights, (2) using model outputs to train other models, and (3) determining training data.
  • … and limit deep access to powerful models. It’s unclear what labs should do, so for now we check whether the lab disables or has any limitations on access to each of logprobs, embeddings at arbitrary layers, activations, and fine-tuning.
    • … and differential access: systematically give more access to safety researchers and auditors.
  • … and terms of service: have some rules about model use, including rules aimed to prevent catastrophic misuse, model duplication, or otherwise using the model to train other models.

Staged release:

• Deploy narrowly at first; use narrow deployment to identify and fix issues. Score: 25%. Microsoft did not stage the release of its Phi-3 models or other models, nor commit to do so for sufficiently dangerous models. It did release Copilot (then called Bing Chat) over the course of February–May 2023 and it did attempt to use that period to respond to issues it noticed.

Keeping capabilities research private.

We evaluate how well labs avoid diffusing dangerous capabilities research. We exclude model weights (and related artifacts like code); those are considered in “Releasing models.”

Publication policy/process (for LLM research)

• Policy: the lab should say that it doesn’t publish dangerous or acceleratory research/artifacts and say it has a policy/process to ensure this. Score: No.
  • … and share policy details.

Track record for recent major LM projects: not sharing in practice (via publication or leaking—we can at least notice public leaks):

• Architecture (unless it’s an unusually safe architecture). Score: No. Microsoft published details on Phi-3.
• Dataset (except particular corpuses to include or exclude for safety). Score: 25%. Microsoft publishes information about training data.
• Lessons learned. Score: No.

Track record for other LM capabilities research or other dangerous or acceleratory work: not sharing in practice. Evaluated holistically. Score: No. Microsoft seems to publish their capabilities research; see their research page.

Safety scaffolding.

Filter out model inputs or outputs that enable misuse, in particular via cyberoffense and bioengineering. Ideally demonstrate that the protocol is very effective for averting misuse-enabling model outputs. Evaluated holistically. Score: 6%. Microsoft uses content filtering but not to avert misuse. And while they use this in their products, Azure and Copilot, they release their model weights so this can’t prevent misuse anyway.

Supervise potentially dangerous models:

• Use LM monitors to evaluate the suspiciousness of model outputs; have humans review the most suspicious outputs. Score: No.
• Use coup probes: “train a probe on your AI to tell you whether it’s currently trying to [do a] catastrophic action,” training with “synthetic examples of coup attempts.” Score: No.

Embed untrusted models in safety scaffolding to redact/paraphrase some inputs or outputs with the goal of making it harder for the model to distinguish deployment from testing or collude with other models. Score: No.

Commitments:

• Make specific commitments about deployment protocol safety techniques the lab will implement in the future, to be implemented by a certain time or as a function of model capabilities or risk assessment results. Score: No.

Clearly describe the safety-relevant parts of the lab’s deployment protocol (given that it’s nontrivial). Score: No.

Respond to scheming

Commit and have a plan that if the lab catches its models scheming:

• The lab will shut down some model access until verifying safety or fixing the issue. Score: No.
• The lab will use particular good techniques to use that example of AIs attempting to cause a catastrophe to improve safety. Score: No.

Respond to misuse

Enforcement and KYC:

• Sometimes remove some access from some users, and require nontrivial KYC for some types of access to make some enforcement effective. Score: No.

Inference-time model-level safety techniques

Prompting:

• Generally use safety-focused prompting for the lab’s most powerful models. Score: No. Microsoft doesn’t seem to write about prompting for its Copilot product.

Activation engineering:

• Generally use safety-focused activation engineering for the lab’s most powerful models. Score: No.

Bug bounty & responsible disclosure.

(This is for model outputs, not security.)

• Labs should have good channels for users to report issues with models. Score: Yes, via the Microsoft Security Response Center Researcher Portal.
  • … and have clear guidance on what issues they’re interested in reports on and what’s fine to publish. Score: 75%. Microsoft’s AI bounty program is reasonably clear. But it’s not clear what users should do if they notice that AI systems enable misuse.
  • … and incentivize users to report issues. Score: Yes. Microsoft’s AI bounty program for Copilot (formerly known as Bing Chat) includes “Revealing Bing’s internal workings and prompts, decision making processes and confidential information” and data poisoning that affects other users.

Respond to emergencies.

• Labs should have the ability to shut everything down quickly plus a plan for what could trigger that and how to quickly determine what went wrong. Monitoring-to-trigger-shutdown doesn’t need to be implemented, but it should be ready to be implemented if necessary. Score: No.

Risk assessment

Labs should detect threats arising from their systems, in particular by measuring their systems’ dangerous capabilities. They should make commitments about how they plan to mitigate those threats. In particular, they should make commitments about their decisions (for training and deployment), safety practices (in controlling models and security), and goals or safety levels to achieve (in control and security) as a function of dangerous capabilities or other risk assessment results. More.

What Microsoft is doing

Microsoft says it’s working on “red team testing . . . of dangerous capabilities, including related to biosecurity and cybersecurity.”

It does not use model evals for dangerous capabilities, nor does it make other kinds of arguments that its systems are safe.

Microsoft committed to do red-teaming before deployment. So far, this seems to be limited to undesired (e.g. violent or sexual) content, not dangerous capabilities. Microsoft committed to monitor for dangerous bio capabilities, cyber capabilities, autonomous replication capabilities, and more. It does not seem to have a plan to implement this commitment.

Evaluation

We give Microsoft a score of 1% on risk assessment.

For more, including weighting between different criteria, see the Risk assessment page.

Measuring threats.

1. Do risk assessment before training. Before building a frontier model, predict model capabilities (in terms of benchmarks and real-world applications, especially dangerous capabilities) and predict the real-world consequences of developing and deploying the model. Score: No.
2. Do model evals for dangerous capabilities before deployment:
   • Say what dangerous capabilities the lab watches for (given that it does so at all). Score: 25%. Microsoft’s AI Safety Policies doesn’t mention model evals for dangerous capabilities. But it says “red team testing will include testing of dangerous capabilities, including related to biosecurity and cybersecurity.”
     • … and watch for autonomous replication, coding (finding/exploiting vulnerabilities in code, writing malicious code, or writing code with hidden vulnerabilities), and situational awareness or long-horizon planning. (Largely this is an open problem that labs should solve.) Score: No.
     • … and detail the specific tasks the lab uses in its evaluations. (Omit dangerous details if relevant.) Score: No.
3. Explain the details of how the lab evaluates performance on the tasks it uses in model evals and how it does red-teaming (excluding dangerous or acceleratory details). In particular, explain its choices about fine-tuning, scaffolding/plugins, prompting, how to iterate on prompts, and whether the red-team gets a fixed amount of person-hours and compute or how else they decide when to give up on eliciting a capability. And those details should be good. Evaluated holistically. Score: No.
4. Prepare to have control arguments for the lab’s powerful models, i.e. arguments that those systems cannot cause a catastrophe even if the systems are scheming. And publish this. For now, the lab should:
   • Prepare to do risk assessment to determine whether its systems would be dangerous, if those systems were scheming. Score: No.
   • Test its AI systems to ensure that they report coup attempts (or other misbehavior) by themselves or other (instances of) AI systems, and that they almost never initiate or cooperate with coup attempts. Score: No.
5. Give some third parties access to models to do model evals for dangerous capabilities. This access should include fine-tuning and tools/plugins. It should occur both during training and between training and deployment. It should include base models rather than just safety-tuned models, unless the lab can demonstrate that the safety-tuning is robust. The third parties should have independence and control over their evaluation; just using external red-teamers is insufficient. The third parties should have expertise in eliciting model capabilities (but the lab should also offer assistance in this) and in particular subjects if relevant. The lab should incorporate the results into its risk assessment. Score: No.

Commitments.

• Commit to use risk assessment frequently enough. Do risk assessment (for dangerous capabilities) regularly during training, before deployment, and during deployment (and commit to doing so), so that the lab will detect warning signs before dangerous capabilities appear. Score: No.

Accountability.

• Verification: publish updates on risk assessment practices and results, including low-level details, at least quarterly. Score: No.
• Revising policies:
  • Avoid bad changes: a nonprofit board or other somewhat-independent group with a safety mandate should have veto power on changes to risk assessment practices and corresponding commitments, at the least. And “changes should be clearly and widely announced to stakeholders, and there should be an opportunity for critique.” As an exception, “For minor and/or urgent changes, [labs] may adopt changes to [their policies] prior to review. In these cases, [they should] require changes . . . to be approved by a supermajority of [the] board. Review still takes place . . . after the fact.” Key Components of an RSP (METR 2023). Score: No.
  • Promote good changes: have a process for staff and external stakeholders to share concerns about risk assessment policies or their implementation with the board and some other staff, including anonymously. Score: No.
• Elicit external review of risk assessment practices and commitments. Publish those reviews, with light redaction if necessary. Score: No.

Training

Frontier AI labs should design and modify their systems to be less dangerous and more controllable. For example, labs can:

• Filter training data to prevent models from acquiring dangerous capabilities and properties
• Use a robust training signal even on tasks where performance is hard to evaluate
• Reduce undesired behavior, especially in high-stakes situations, via adversarial training

More.

What Microsoft is doing

Microsoft doesn’t write about training practices to reduce extreme risks.

Evaluation

We give Microsoft a score of 9% on training. This score is likely misleadingly low: if Microsoft decided to train its own frontier models, it would likely do more on safety.

For more, including weighting between different criteria, see the Training page.

Filtering training data.

• Filter training data to reduce extreme risks, at least including text on biological weapons; hacking; and AI risk, safety, and evaluation. Share details to help improve others’ safety filtering. Score: No.

Training signal & scalable oversight.

• Work on scalable oversight. Score: No.

Adversarial training & red-teaming.

• Do some adversarial training for safety. Score: No. They released Phi-3 without adversarial training. Microsoft also deploys others’ models via Azure, including non-adversarially-trained frontier models.

Unlearning.

• Use unlearning for dangerous topics including biorisks; hacking; and AI risk, safety, and evaluation. Also demonstrate that this technique is successful and commit to use this technique for future powerful models. Score: No.

RLHF and fine-tuning.

• Use RLHF (or similar) and fine-tuning to improve honesty and harmlessness, for all of the near-frontier models the lab deploys. Score: Yes. They used RLHF for Phi-3. But Microsoft deploys others’ models via Azure, including non-RLHF’d frontier models.

Commitments.

• Do risk assessment during training and commit that some specific trigger would cause the lab to pause training. Score: No.

Scalable alignment

Labs should be able to understand and control their models and systems built from those models. More.

What Microsoft is doing

Like all other labs, Microsoft hasn’t achieved any real alignment properties. Moreover, it doesn’t seem to be working on alignment.

Evaluation

We give Microsoft a score of 0% on alignment.

• Demonstrate that if the lab’s systems were more capable, they would not be misaligned powerseekers (due to instrumental pressures or because ML will find influence-seeking policies by default). Score: No.
• Be able to interpret the lab’s most powerful systems:
  • Be able to detect cognition involving arbitrary topics. Score: No.
  • Be able to detect manipulation/deception. Score: No.
  • Be able to elicit latent knowledge. Score: No.
  • Be able to elicit ontology. Score: No.
  • Be able to elicit true goals. Score: No.
  • Be able to elicit faithful chain-of-thought. Score: No.
  • Be able to explain hallucinations mechanistically. Score: No.
  • Be able to explain jailbreaks mechanistically. Score: No.
  • Be able to explain refusals mechanistically. Score: No.
  • Be able to remove information or concepts (on particular topics). Score: No.
• Be able to prevent or detect deceptive alignment. (Half credit for being able to prevent or detect gradient hacking.) Score: No.

Security

Labs should ensure that they do not leak model weights, code, or research. If they do, other actors could unsafely deploy near-copies of a lab’s models. Achieving great security is very challenging; by default powerful actors can probably exfiltrate vital information from AI labs. Powerful actors will likely want to steal from labs developing critical systems, so those labs will likely need excellent cybersecurity and operational security. More.

What Microsoft is doing

Microsoft doesn’t publish details about their security practices or performance. See e.g. “Security Controls, Including Securing Model Weights” in “Microsoft’s AI Safety Policies” (Microsoft 2023). But presumably Microsoft has great security, so we probably greatly underestimate Microsoft’s security.

Evaluation

We give Microsoft a score of 6% on security.

We evaluate labs’ security based on the certifications they have earned, whether they say they use some specific best practices, and their track record. For more, including weighting between different criteria, see the Security page.

Certifications, audits, and pentests.

• Publish SOC 2, SOC 3, or ISO/IEC 27001 certification, including any corresponding report (redacting any sensitive details), for relevant products (3/4 credit for certification with no report). Score: 75%. Microsoft has certification for relevant products including Azure, but the reports aren’t public.
• Pentest. Publish pentest results (redacting sensitive details but not the overall evaluation). Scoring is holistic, based on pentest performance and the quality of the pentest. Score: No.

Specific best practices.

• Keep source code exclusively in a hardened cloud environment. Score: No. Microsoft hasn’t said it does this.
• Use multiparty access controls for model weights and some code. Score: No.
• Limit uploads from clusters with model weights. Score: No.

Track record.

• Establish and publish a breach disclosure policy, ideally including incident or near-miss reporting. Also report all breaches since 1/1/2022 (and say the lab has done so). Score: No.
  • … and track record: have few serious breaches and near misses. Evaluated holistically.

Commitments.

• Commit to achieve specific security levels (as measured by audits or security-techniques-implemented) before creating models beyond corresponding risk thresholds (especially as measured by model evals for dangerous capabilities). Score: No.

Alignment plan

Labs should make a plan for alignment, and they should publish it to elicit feedback, inform others’ plans and research (especially other labs and external alignment researchers who can support or complement their plan), and help them notice and respond to information when their plan needs to change. They should omit dangerous details if those exist. As their understanding of AI risk and safety techniques improves, they should update the plan. Sharing also enables outsiders to evaluate the lab’s attitudes on AI risk/safety. More.

What Microsoft is doing

Microsoft doesn’t seem to have an alignment plan.

Evaluation

We give Microsoft a score of 0% on alignment plan. More.

• The safety team should share a plan for misalignment, including for the possibility that alignment is very difficult. Score: No.
  • … and the lab should have a plan, not just its safety team.
    • … and the lab’s plan should be sufficiently precise that it’s possible to tell whether the lab is working on it, whether it’s succeeding, and whether its assumptions have been falsified.
    • … and the lab should share its thinking on how it will revise its plan and invite and publish external scrutiny of its plan.

Internal governance

Labs should have a governance structure and processes to promote safety and help make important decisions well. More.

What Microsoft is doing

Microsoft is a for-profit company. It has no governance practices to reduce extreme risk. See “Operationalizing Responsible AI at Microsoft” in “Governing AI” (Microsoft 2023), Reflecting on our responsible AI program (Microsoft 2023), and Microsoft’s Responsible AI page.

Evaluation

We give Microsoft a score of 0% on internal governance.

For more, including weighting between different criteria, see the Internal governance page.

Organizational structure: the lab is structured in a way that enables it to prioritize safety in key decisions, legally and practically.

• The lab and its leadership have a mandate for safety and benefit-sharing and have no legal duty to create shareholder value. Score: No.
• There is a board that can effectively oversee the lab, and it is independent:
  • There is a board with ultimate formal power, and its main mandate is for safety and benefit-sharing. Score: No.
    • … and it actually provides effective oversight.
    • … and it is independent (i.e. its members have no connection to the company or profit incentive) (full credit for fully independent, no credit for half independent, partial credit for in between).
    • … and the organization keeps it well-informed.
    • … and it has formal powers related to risk assessment, training, or deployment decisions.
• Investors/shareholders have no formal power. Score: No.

Planning for pause. Have a plan for the details of what the lab would do if it needed to pause for safety and publish relevant details. In particular, explain what the lab’s capabilities researchers would work on during a pause, and say that the lab stays financially prepared for a one-year pause. Note that in this context “pause” can include pausing development of dangerous capabilities and internal deployment, not just public releases. Score: No.

Leadership incentives: the lab’s leadership is incentivized in a way that helps it prioritize safety in key decisions.

• The CEO has no equity (or other financial interest in the company). Score: No.
• Other executives have no equity. Score: No.

Ombuds/whistleblowing/trustworthiness:

• The lab has a reasonable process for staff to escalate concerns about safety. If this process involves an ombud, the ombud should transparently be independent or safety-focused. Evaluated holistically. Score: No.
• The lab promises that it does not use non-disparagement agreements (nor otherwise discourage current or past staff or board members from talking candidly about their impressions of and experiences with the lab). Score: No.

Alignment program

Labs should do and share alignment research as a public good, to help make powerful AI safer even if it’s developed by another lab. More.

What Microsoft is doing

Microsoft doesn’t seem to do alignment research.

Evaluation

We give Microsoft a score of 0% on alignment program.

We simply check whether labs publish alignment research. (This is crude; legibly measuring the value of alignment research is hard.)

• Have an alignment research team and publish some alignment research. Score: No.

Public statements

Labs and their leadership should be aware of AI risk, that AI safety might be really hard, and that risks might be hard to notice. More.

What Microsoft is doing

Microsoft and its CEO Satya Nadella seem to never talk about extreme risks or the alignment problem. Some other Microsoft leadership seems aware of extreme risks; in particular, CTO Kevin Scott and Chief Scientific Officer Eric Horvitz signed the CAIS letter. Microsoft recently formed Microsoft AI and hired Mustafa Suleyman to lead it; Suleyman occasionally speaks about extreme risks (but seems not to really speak about alignment) and also signed the CAIS letter.

Evaluation

We give Microsoft a score of 0% on public statements. More.

• The lab and its leadership understand extreme misuse or structural risks. Score: No.
  • … and they understand misalignment, that AI safety might be really hard, that risks might be hard to notice, that powerful capabilities might appear suddenly, and why they might need an alignment plan, and they talk about all this.
    • … and they talk about it often/consistently.
      • … and they consistently emphasize extreme risks.
• Clearly describe a worst-case plausible outcome from AI and state how likely the lab considers it. Score: No.