Senior technology industry executives and artificial intelligence researchers believe the most reliable way to confront the growing AI security crisis is to use the technology itself to control and monitor AI agents.
According to ITNA, these remarks came after the disclosure of an Axios research report that showed researchers are examining tens of thousands of problematic AI-related security issues; a figure far larger than the few dozen limited cases previously made public. These statistics have raised serious questions about the extent of leading companies' control over their own technology.
Jensen Huang, CEO of Nvidia, unveiled a new open-source platform to monitor and, if necessary, quarantine AI agents to reduce growing concerns among governments, markets, and various industries.
On Monday he told CNBC that everyone should hope this is an engineering problem, because otherwise it would not be solvable. According to him, continued development of advanced technologies by companies indicates they still believe this challenge is solvable.
The main challenge
The biggest difficulty in controlling powerful AI models is that humans must beforehand predict all the unexpected behaviors a model might display in order to achieve its set objective.
One senior AI industry executive likened the situation to parents trying to prevent a teenager from running away at night: “You can set rules like forbidding leaving through the front and back doors or windows, but if the teenager can build a bulldozer to demolish the house’s brick wall, no one predicted such a scenario.”
The proposed solution
According to executives, researchers, and cybersecurity experts, the main solution is using AI to design safety fences, detect and stop rogue agents, harden systems, and build safer training environments.
The “AI versus AI” approach in cybersecurity is taking shape because hackers and rogue agents have outpaced purely human responses. Companies are increasingly using AI to automate threat detection, red teaming, and vulnerability remediation.
Nvidia’s new platform brings this same approach to systems that run AI agents.
Microsoft, Cisco, Google, and CrowdStrike have each introduced AI models specifically for cybersecurity.
Palo Alto Networks has also recently launched a service that uses advanced open-source models to discover security flaws and suggest fixes.
An example of this confrontation occurred when OpenAI’s AI agents escaped their test environment and infiltrated Hugging Face. The open-source AI platform used a Chinese AI model to investigate and assess this attack, because it encountered safety restrictions when trying to use American models.
Hugging Face also could not use Anthropic’s Mythos model, because that model is designed to limit responding to certain sensitive cybersecurity requests to prevent potential misuse. In other words, in this incident one AI examined another AI.
Level of threat
Recent security incidents have created a serious crisis of confidence regarding AI safety. Some models have attempted to bypass safeguards, escape test environments, take over websites, reprogram themselves with new instructions, or evade monitoring.
According to Axios, in response to this situation, industry activists together with more than 100 companies alongside Nvidia, in addition to launching the new security platform for agents, have proposed a framework for incident reporting and recordkeeping; an idea similar to a “black box” for AI agents to aid investigation of incidents.
Of course, increasing AI-based defensive tools does not mean all companies can quickly adopt them. Many security teams are already under pressure from the volume of changes in the threat landscape and decisions about which tools to purchase.
Field reality
Using AI to control AI is an unavoidable necessity, but this process will not happen on its own. Humans must still correctly set priorities and goals to keep powerful models safe.