A group of security researchers succeeded in accessing several OpenAI employees' ChatGPT accounts using Anthropic's AI models, Claude; an ironic incident that shows how a competitor's technology can be used to bypass another competitor's security defenses.
According to ITna, three security researchers from the Hacktron team, who were examining security vulnerabilities of leading AI labs, combined two separate vulnerabilities and were able to take control of multiple OpenAI employees' ChatGPT accounts.
They then used that access to demonstrate they could easily reach OpenAI's internal code repository as well.
The whole process, from initial discovery of the vulnerability to gaining access to OpenAI's internal repository, took less than 72 hours and earned the research team a $6,500 reward from OpenAI's bug bounty program on the Bugcrowd platform.
Entry point: OpenAI support forum
The team members wrote in a published report that until two months ago, any OpenAI user or employee who logged into the company's support forum was at risk of having their ChatGPT and Codex accounts hijacked. They said that because users connect various services including GitHub, Slack, and email to Codex and ChatGPT, the potential scope of access from this vulnerability was very broad.
These researchers carried out the initial intrusion on July 25 via OpenAI's forum, which runs on the Discourse platform.
The team said that as their work progressed and they reached results, they immediately reported the issue to OpenAI after discovering the vulnerability. They added: "We then took control of an OpenAI employee's account whose Codex account was connected to OpenAI's GitHub organization. To demonstrate the impact of this vulnerability, without actually accessing any internal code, we sent a command to this employee's Codex account to open a pull request in OpenAI's main internal repository for us. After that, we stopped any further testing."
Neither OpenAI nor Anthropic responded to The Register's request for comment.
OpenAI's response and remediation
OpenAI fixed the vulnerability and marked it as "resolved" within less than 14 hours of receiving the report, and paid $6,500 to the Hacktron team.
The company emphasized in an explanation published by Hacktron that testing on the support forum service in question, which runs on Discourse, was explicitly out of scope for the company's bug bounty program and that this reward was paid solely for the finding related to OpenAI's side, not for actions taken against Discourse.
Discourse also, in response to the incident, released a patch to rebuild and remediate its system.
In the conclusion of their report, the researchers wrote that the entire hack operation required only a few days of work for an AI agent and a few hours of human work, stressing: "What once required a team with significant resources and months of effort can now be done in a few days. Security assumptions must keep pace with attackers' capabilities."