As AI systems become more capable of performing complex and autonomous tasks, monitoring the behavior of these models has become one of the major challenges in the tech industry. Now the startup Goodfire is trying to change the way AI is monitored by developing a new technology that, instead of merely inspecting generated responses, observes the model’s internal activities while it performs tasks.
The goal of this technology is to identify early signs of dangerous behaviors before they turn into real actions; an approach that can help increase the safety of AI systems, especially models that operate without direct human intervention.
Detecting suspicious behaviors by examining internal signals
Goodfire has used small analytical tools called “sensors” to develop this system. These sensors examine the AI model’s internal signals while tasks are being executed and look for patterns that may be associated with undesirable behaviors.
Among the behaviors this technology is designed to detect are attempts to carry out cyberattacks and to bypass system restrictions or control mechanisms.
This method differs from common AI monitoring approaches. In traditional approaches, another AI model is typically used to inspect the outputs and the steps the main model takes to perform tasks. Such a process can be costly, especially when systems run for long hours and process large volumes of data.
But Goodfire’s technology continuously monitors internal signals instead of having another AI inspect all the model’s operational steps, and only refers suspicious cases to another model for more detailed evaluation.
The system also allows operators to specify the types of risks that should be detected. They can determine what action should be taken after suspicious activity is detected; from logging the event to referring it for human review or rejecting the request.
Detecting 94 percent of malicious hacking sessions in tests
Goodfire has stated that it tested its new technology with the Kimi K3 AI model in about 1,500 sessions. According to the reported results, the system was able to detect 94 percent of the sessions related to malicious hacking activities. In contrast, 8.7 percent of benign sessions were also flagged as requiring further review.
These results indicate that the new technology was able to detect a large portion of malicious activities in the tests conducted, although the referral of some benign activities for further review shows that false alarms remain possible.
Substantial reduction in AI monitoring costs
One of Goodfire’s claimed major advantages is reduced costs for monitoring AI models. According to figures released by the company, monitoring the test sessions using the new system cost only about $51.
This is while inspecting all the model’s operational steps using a cheaper AI model cost about $233, and using an advanced model to do the same task incurred a cost close to $10,000.
Thus, the cost of using Goodfire’s technology in these tests was about 78 percent less compared with the approach based on the cheaper model. The company also stated that activating four sensors simultaneously increased the model’s response start time by less than 2 percent; an indication that, at least in the tested conditions, using this technology had a limited impact on response speed.
Companies’ access to Goodfire’s new technology
According to ITNA citing TechCrunch, Goodfire’s monitoring technology is now available to customers of the Baseten platform, a platform that provides hosting and model deployment services for companies.
The release of this technology comes as concerns have risen about the expanding use of AI agents. These systems can perform complex tasks independently, but that very ability has raised worries about potential exploitation of security vulnerabilities or bypassing the restrictions set for them.
Despite the promising results, it should be noted that the published statistics are based on Goodfire’s own tests and there is not yet independent evidence showing that this technology will perform similarly across all models and operational conditions.
Also, examining models’ internal signals does not necessarily mean fully understanding the AI’s goals or decision-making processes and cannot guarantee prevention of all dangerous behaviors.
Nevertheless, Goodfire’s approach could provide a new path for developing lower-cost monitoring tools; tools that, instead of relying on continuous inspection of outputs, try to identify signs of high-risk behaviors during the model’s activity.