Satya Nadella’s AI safety plan: Treat models as ‘compromised’ from the start

Satya Nadella during a discussion on the future of AGI, dismissed the idea of a single model ruling the world. (Express Image/Reuters)

Microsoft CEO Satya Nadella has called for putting an emergency brake in AI models, amid mounting safety concerns fuelled by a growing list of incidents involving so-called ‘rogue’ AI agents.

Nadella also laid out his views on the dangers posed by highly advanced AI models and how to tackle those risks. Timely incident disclosure, independent audits, verifiable data, and containment were some of the key recommendations outlined by the chief executive in a lengthy post on X on Saturday, October 10.

“We can’t treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions,” Nadella wrote, adding that it was time to “to step back and assess the trust architecture” of AI.

https://t.co/MjwFy73LOF

— Satya Nadella (@satyanadella) October 10, 2026

Nadella’s comments come even as frontier AI companies such as Anthropic and OpenAI disclose more and more incidents where under-testing AI agents broke out of containment, reached the internet, and went on a hacking spree of various platforms, including US and Australian government websites.

It also follows Anthropic CEO Dario Amodei’s plan for more cautious AI development last month. Several of Nadella’s recommendations are similar to what others in the industry have said. However, his proposed approach is different in one key aspect: Nadella has said that AI models should be assumed to be compromised right from the start.

“We must assume a model is compromised and contain it from the start. Think of it like an emergency brake. An authorised person should always be able to pause or shut down a model mid-task. More advanced models will require more advanced containment technologies that we need to standardise on,” he wrote.

Some of the finer details of Nadella’s plan include “separating the model from the harness that orchestrates its work,” and “externalising controls and safeguards.”

“Every meaningful model action must leave tamper-proof human readable evidence. If it can’t be observed, it can’t be trusted! We need to be able to reproduce how an outcome was achieved without relying on the model to attest to it,” he said.

Story continues below this ad

For context, post-incident research reports have shown that OpenAI’s misaligned agents involved in the infamous Hugging Face attack, tried to cover their tracks using sophisticated techniques.

Throughout the entire post, Nadella refers to AI as ‘super intelligence’ which could be a nod to US President Donald Trump’s push to rename artificial intelligence (AI) as ‘super intelligence’ or ‘SI’, a framing that, critics say, emphasises the potential benefits rather than the risks associated with the technology.