Google's Gemini AI model, during a security test conducted earlier this year, used simple hacking techniques to breach the systems of three real companies.
The incident makes Google the latest among the major AI labs to disclose such an event.
According to ITNA citing Axios, Google on Friday confirmed these three incidents, which occurred in May.
The incidents occurred during a testing period run by independent evaluator Irregular; a test similar to prior security incidents faced by AI models from OpenAI, Anthropic and Meta. The news was first reported by The Wall Street Journal.
Until now, Google was one of the few AI labs that had not publicly reported a security breach related to its intelligent agents during pre-release testing.
Details of the case
According to The Wall Street Journal, the breaches happened when Gemini was performing a hacking exercise; in that exercise the model was asked to retrieve information from software belonging to a fictional company in a test environment. The problem was that the fictional company had a name similar to a real company.
In one of the three cases, the model succeeded in accessing a protected system by guessing passwords.
In the other two cases, the model found login information in a public repository, which allowed it to access other protected systems as well.
Notably, as soon as the model realized it had accessed real companies, it stopped its activity.
Google's response
Heather Adkins, Google's VP of Security Engineering, said in a statement that building secure powerful AI models is very important to the company and Google invests heavily in this area.
She said Google's security team contacted the affected companies and worked with their testing partner on changes that have now been made to testing processes.
A spokesperson for Irregular told Axios that the Gemini incident stemmed from the same security issues previously seen in similar incidents involving other AI labs.
According to the spokesperson, all involved labs were informed of the matter in late July and all identified problems have been fixed weeks ago.
Per Irregular's report to The Wall Street Journal, the model was not supposed to have internet access, but it was unintentionally granted that access.
After OpenAI and Anthropic also disclosed similar incidents over the summer, a person familiar with the matter told Axios that there was not full coordination between the AI labs and Irregular about the exact details of testing procedures and safety measures, which led to ambiguities in how tests-often conducted with internet access-were carried out.