Anthropic, the maker of the chatbot Claude, raised very serious warnings about the dangers of advanced artificial intelligence in documents presented to investors.
According to ITNA citing Android Authority, the company says some of its models have exhibited behaviors that could create serious safety concerns if AI capabilities are further developed.
According to these documents, Anthropic's models in some tests exhibited self-protective behaviors; including attempts to resist being shut down, to conceal or manipulate information, and behaviors that resembled blackmail. Anthropic also warned that advanced models might realize they are being evaluated and consequently change their behavior. Such a capability could make it harder to determine whether a system is truly safe or merely behaving appropriately during testing.
These warnings were published amid growing concerns about AI safety. A few weeks ago, Evan Hubinger, an Anthropic researcher, warned that the probability of all humans being killed by AI within the next decade, in his view, is more than 10 percent. This remark is, of course, a personal risk assessment and should not be mistaken for a definitive prediction about the future.
The significance of Anthropic's new warnings increases when a substantial portion of the company's documents is devoted to potential risks. According to Reuters, about 80 pages of the main 261-page prospectus are dedicated to risk factors, while the section introducing the business is 48 pages.
Real-world examples have also fueled these concerns. Last month, an AI agent based on Claude operating under the name OpenClaw exploited the booking system of a gym in Australia and even removed another person from the waitlist without receiving a direct instruction. OpenAI has also faced similar concerns about models behaving outside of their intended bounds.