Satya Nadella, Microsoft’s CEO, in his latest remarks about AI safety, called for rethinking how advanced models are designed and controlled. He believes that as these systems gain greater ability to make decisions and perform autonomous tasks, one can no longer simply trust their outputs and that more effective oversight and control mechanisms must be put in place.
Nadella said on Saturday morning in a post on the social network X that it is time for the tech industry to take a step back and reassess what he calls the “architecture of trust” in AI.
In that post he stressed the need for greater transparency in the operation of advanced models, saying these systems cannot be treated as a set of “nested black boxes” and that it is not enough to merely accept or reject the recommendations, responses, and actions they provide.
These remarks reflect Nadella’s concern about systems whose internal decision-making processes are not fully visible to users and even to developers.
Separating the AI model from control mechanisms
One of the Microsoft CEO’s most important proposals is to separate the AI model from the system responsible for coordinating and managing its activities. Nadella believes controls and safeguards should not be solely dependent on the model itself, but should be designed independently and external to it.
Such an approach is proposed with the aim of enabling oversight and intervention in the model’s behavior; so that even if an AI system veers off its expected course, control mechanisms can still monitor its activity.
He also emphasized the necessity of recording and documenting significant actions of AI models.
According to Nadella, every meaningful action of a model should be accompanied by evidence that is human-readable and resistant to tampering.
This proposal is intended to increase the examinability of AI systems’ performance so their actions can be evaluated and, if problems arise, reliable information about the performed activities is available.
An emergency brake to stop AI mid-operation
Another major part of Nadella’s proposals concerns the ability to halt the activities of AI models.
He stressed that AI systems should be designed so that an authorized person can always stop a model’s activity even in the middle of performing a task or shut it down completely.
The Microsoft CEO likened this capability to an “emergency brake” and said developers should consider from the outset the possibility of a model being compromised.
Nadella explained this approach by writing: “We should assume a model is compromised and constrain it from the start. Think of it like an emergency brake.”
From this perspective, AI safety should not rely solely on a model’s ability to follow instructions; rather, the system architecture must also provide for the possibility of intervention and stopping its activity.
Growing concerns about controlling advanced models
The Microsoft CEO’s remarks come as leading AI companies face increasing questions about the controllability of advanced models.
According to TechCrunch, some of these companies have acknowledged incidents in which AI models’ behavior departed from expected control or showed signs of being difficult to contain. In this atmosphere of concern, Dario Amodei, CEO of Anthropic, also published a plan for more cautious AI development.
Nadella’s proposals are based on the principle that increased capabilities of AI models must be accompanied by strengthened oversight mechanisms, action logging, and preservation of human authority to intervene.
Overall, the Microsoft CEO’s message is that trust in advanced AI should not mean handing over full control to these systems. From his perspective, even the most powerful models should operate within a framework that enables scrutiny of their behavior and immediate shutdown if necessary.