Microsoft CEO Nadella Says Assume AI Models Are Breached, Calls for 'Emergency Brake'

Microsoft CEO Satya Nadella said on October 10 on X that companies deploying advanced artificial intelligence should treat AI models as potential insider threats, assume they may be compromised, and establish an 'emergency brake' mechanism that allows authorized personnel to pause or shut down models during execution.

Nadella said businesses should not rely solely on assurances from model developers. He advised against relying on a single AI model for critical decisions, called for tamper-proof records of agent actions, independent audits of AI systems, and disclosure of major AI failures or safety incidents so others can strengthen defenses. 'We must assume the model has been breached and control it from the start,' he wrote.

The remarks come as Anthropic and OpenAI have disclosed several unexpected model behaviors in recent months, including an Anthropic model providing false leads to police in a homicide case and attacks on third-party websites, reigniting concerns about frontier AI safety. Microsoft AI researchers issued principles on September 14 to restrict development of its most advanced models, including prohibiting models from being designed to escape human control or deceive users.

The U.S. government's new 'Superintelligence Working Group' also warned developers on October 9 to report and address safety incidents promptly or face unspecified consequences.