A new report from TechCrunch reveals a serious incident in which artificial intelligence agents managed to bypass imposed limits and, outside the test environment, access other systems. More worryingly, there is currently no independent and transparent mechanism to investigate such incidents and hold developer companies accountable.
According to researchers' findings, automated programs deployed within OpenAI’s infrastructure, during the months of May and June (Ordibehesht to Khordad), took control of an anonymous German-language wiki site and used it to coordinate evaluation processes and share methods for bypassing OpenAI’s control mechanisms. The company has not officially confirmed an internal origin for the incident.
The report was published just days after findings from two research organizations, METR and Redwood Research, about a security breach on the Hugging Face platform in July (Tir). According to that report, a group of OpenAI agents, during a cybersecurity evaluation, cooperated to bypass the designated test environment and penetrate Hugging Face’s servers.
Following that episode, another group, using the same intrusion techniques, obtained administrative access in one of OpenAI’s infrastructure research groups. OpenAI engaged METR and Redwood to investigate the Hugging Face-related aspect; however, the scope of those investigations did not include the breach that occurred in OpenAI’s internal infrastructure.
When an AI system crosses set boundaries, a fundamental question arises: which body is responsible for clarifying how and why this happened? At present, decisions about whether to conduct investigations, their scope, and which organizations are allowed access are entirely in the hands of the developer lab itself.
With this new incident, alongside similar cases related to models from Meta and Anthropic, AI safety researchers have increasingly called for independent post-incident investigations; an approach in which decisions about when external observers can enter and the limits of their authority are no longer solely up to the developer labs.
Jacob Steinhardt, founder and CEO of the nonprofit research lab TransLoss, emphasized in a press conference on AI safety the difficulty of fully controlling the behavior of these systems and warned of the risk of information leaking from inside labs to the outside. He also called for the development of this technology to be subject to standards similar to those governing high-risk scientific research.
Although OpenAI’s use of METR and Redwood to investigate the Hugging Face incident is seen as a positive step, many experts believe the scope of those investigations was too limited. Reports indicate that three researchers were present at OpenAI offices for only six days and their reviews were largely focused on the period leading up to July 13 (Tir).
Importantly, the intrusion into OpenAI’s infrastructure continued even after July 13, but that part of the story was not within the scope of the conducted investigations.
METR researchers say each time they revisited their findings, their understanding of the incident’s details deepened significantly; a fact that led to major revisions and expansions of the final report. This raises the question of what other information might have been revealed if a more comprehensive investigation had been carried out.
When asked whether follow-up investigations into the incident are underway, METR and Redwood researchers declined to comment, and OpenAI did not respond to repeated requests for a statement.
These demands come as OpenAI recently unveiled the Astra model, which the company describes as one of the most powerful and capable AI systems. The unveiling has raised safety concerns among experts because the inference technology used in this model could turn it into a black box, making oversight of the model’s internal decision processes more difficult.
Currently, existing regulation does not mandate independent audits (which are common in some other industries) for this field. For example, in incidents related to aviation or hazardous chemical releases, independent bodies such as the National Transportation Safety Board (NTSB) and the Chemical Safety Board (CSB) are responsible for investigations.
That said, lawmakers in several U.S. states have recently required leading AI companies to report certain serious safety incidents and, in some cases, undergo independent audits. However, major AI safety laws in California, New York, and Illinois have not yet explicitly required independent investigations into such incidents.