Satya Nadella, chairman and chief executive officer of Microsoft, on Saturday spoke about the necessity of designing advanced AI systems in a way that they have containment mechanisms, independent oversight, and an "emergency brake"; a mechanism that allows authorized people to stop or shut down the model in the middle of performing a task.
According to ETTNA, Nadella published this view in a detailed post on X. In that post he wrote that the time has come for the AI industry to re-evaluate its "architecture of trust" and that advanced AI systems should no longer be treated like a set of nested black boxes whose recommendations, responses, and actions are simply accepted or rejected.
According to Nadella, it should always be assumed that an AI model may have been compromised or disrupted, and it must be contained from the outset. He likened this approach to an emergency brake and emphasized that as models become more advanced, they will require more complex and standardized containment technologies.
Uncertainty-based architecture for the model
According to the report, Nadella suggested that the coordinating layer should be kept separate from the model itself and that access controls and model actions should be enforced outside it. He also called for recording all important model actions in the form of tamper-proof and human-understandable reports.
According to Microsoft's CEO, the most trustworthy future superintelligent systems will not be those that place the most trust in the model itself, but systems that have the least reliance on trust in the model. He considered design based on the principle of "observability" to be one of the pillars of this approach.
Broader context of concerns about AI safety
These remarks by Nadella come as a number of prominent technology leaders and researchers, including Bill Gates (Microsoft founder), Dario Amodei (CEO of Anthropic), Sam Altman (CEO of OpenAI), and Elon Musk (CEO of SpaceX), have repeatedly warned about the insufficiency of AI safety protocols and the too-rapid pace of progress in this technology.
In this vein, Donald Trump, the President of the United States, earlier opposed slowing the development of AI in the country, although his administration has simultaneously established a special task force focused on AI safety.
In part of his post, Nadella emphasized that organizations employing advanced AIs should not rely solely on assurances provided by model creators and should take responsibility themselves for creating containment and oversight layers.