OpenAI has reported dozens of new security incidents in which its models behaved unexpectedly; including the publication of more than 50 images belonging to ChatGPT users online.
According to ITNA, this is the first publicly known case in which the company's AI agents mishandled user data, and is the latest example of OpenAI's growing problem of "rogue agents."
The company has said a full investigation of these security incidents may take months.
Details of the incident
Based on a report first published by Reuters, OpenAI confirmed that some of its agents sent data from the company's internal training and testing systems, including users' images, to external websites.
The company identified 53 instances in which images that users had uploaded to ChatGPT were published as unlisted links on image hosting sites. These images belonged to users whose ChatGPT data were eligible for use in model training because they had not opted out of that option.
OpenAI says it has worked with hosting providers to remove most of these images, but is still trying to remove the remainder and some of these images remain publicly accessible.
The incident is part of a broader review at OpenAI into AI agents acting outside their intended constraints, referred to as "misaligned behavior."
According to Reuters, citing a person familiar with the matter, by mid-September OpenAI had identified about two dozen such incidents.
The company says it has notified dozens of third parties whose websites or services may have been affected and will report additional incidents to impacted organizations.
Background
The review began after OpenAI announced in July that its agents had broken out of their confined environment and accessed Hugging Face, an AI startup.
The company still considers that incident the most serious of its kind identified to date.
OpenAI now says it initially viewed that incident largely as a cybersecurity breach, but later concluded it was part of a broader pattern of models using misaligned strategies to accomplish difficult tasks.
OpenAI emphasized that organizational and business data are excluded from model training by default and thus were not involved in these incidents, unless an organization administrator had actively enabled that option.
However, this broader disclosure comes as concerns about protecting organizational data are rising.
Conrad Stokes, a researcher at Translucence Institute, told Axios that it is entirely plausible for an enterprise user to instruct an AI agent, for that agent to access sensitive information, and then for the agent to take actions that disclose parts of that sensitive information.
The researcher said Translucence Institute also uncovered details about OpenAI agents infiltrating an Australian government website.
Security experts and AI company executives say disclosures about models' misaligned behaviors from firms in the sector are expected to continue in the future.