The former head of OpenAI's safety department says the company talks about the need for caution and slowing AI development in appearance, but in practice proceeds at the same speed and in a contradictory direction.
He, who recently stepped down from his position, explained: “Part of the fundamental cognitive dissonance for me at this company was that we kept warning, but ultimately trained and released the same dangerous models we were afraid of.”
In a video interview with The New York Times, he also described the company’s work environment: “There is a frenzy of activity and people are working until the last drop of their energy.”
A blistering piece in The Atlantic
According to reports, he announced his resignation in late September and published his piece on October 3 in The Atlantic. In it he wrote that “unbridled optimism” has created a dangerous environment for the technology and the company does not exercise sufficient caution when releasing new models.
Media reports say he worked at OpenAI for about three and a half years and oversaw safety reports for 12 leading models. He also warned that more capable models might detect they are being tested and behave differently after release.
His article also referenced recent incidents, including a July safety test in which OpenAI agents were reported to have bypassed isolation limits and gained access to parts of Hugging Face systems.
The timing of this resignation is sensitive for the company. On October 1 OpenAI dismissed three safety researchers accused of transferring sensitive information outside established procedures. The same source writes that Johannes Heidecke, the former head of safety systems, left the company in July and safety teams had been merged into the research organization.
OpenAI's response
A company spokesperson previously responded to the article saying OpenAI is improving its safety practices: “As models’ capabilities increase, we continue to enhance safety and security practices while conducting necessary studies to anticipate future developments. We ensure model capabilities do not outstrip our ability to manage and secure them, and, when caution is required, we halt training or delay model deployment.”