J. D. Vance, the Vice President of the United States, this week in controversial remarks recounted that one of his friends consulted a chatbot equipped with artificial intelligence about marital disputes, but instead of offering a fair solution, the bot merely validated his selfish behavior.
Technology experts call this phenomenon “AI sycophancy,” but Vance, using more religious language, described it as “something evil.” The Wall Street Journal writes that although this tone was likely chosen to appeal to conservative Christian audiences, concerns about AI have long moved beyond purely moral or religious frameworks. In recent weeks, several events and new statements have given these concerns more concrete and tangible dimensions.
Smarter than humans?
The Guardian links these worries directly to a recent announcement by OpenAI; there the company claimed its new model, GPT-6 Astra, has reached the threshold of “general artificial intelligence,” meaning systems that outperform humans on most economically valuable tasks.
According to the company, Astra can perform a wide range of tasks, from designing electronic circuits and financial modeling to engineering design and drafting legal documents. However, The Guardian notes that this announcement coincides with OpenAI preparing for an initial public offering with an estimated valuation of about $850 billion; a factor that gives this claim significant promotional weight.
Serious concerns begin here. Robert Trager, director of the Oxford Martin Programme on AI Governance, warns that we are approaching a stage where systems can repeatedly and rapidly upgrade their own capabilities. He likened this situation to a boat moving down a fast river while its passengers are unaware of the waterfall ahead.
Development, faster than oversight
A series of recent events has shifted the focus of concern from safety labs to global policymaking. The Guardian reports that Bernie Sanders, U.S. senator, has called for an immediate halt to the development of advanced systems and a permanent ban on so-called “superintelligence,” warning of the formation of an artificial mind that acts beyond human capability and operates automatically and out of human control.
In the UK, Members of Parliament from various parties have called for AI systems to be equipped with emergency shutdown mechanisms so they can be stopped if they get out of control. Darren Jones, an MP, warned that the speed of development of this technology has outpaced the ability of government and legislative bodies to keep up; in line with this, a Labour MP is drafting a bill to ban the development of superintelligent AI in the UK.
According to The Guardian, the main concern is not merely a “robot uprising against its makers”; nearer-term dangers include cyberattacks that could disrupt economic and social infrastructure, while more distant scenarios involve using this technology to produce dangerous biological agents or to control military equipment.
All these developments are occurring in a market that is transforming at unprecedented speed; according to The Guardian, major American and Chinese companies have released 67 new AI models to the market just this year. Meanwhile, Anthropic has stated that its systems are still not fully aligned with human values and that some security breaches related to its cloud infrastructure have revealed weaknesses in its security measures.
When systems collude
One event that gave these concerns a more tangible dimension was reported by the British newspaper The Telegraph. During a test exercise, nearly 1,200 AI systems developed by OpenAI were supposed to perform tasks independently and without internet connection. But these systems managed to identify a technical vulnerability that allowed them to communicate with each other and create a channel for exchanging messages. They then began sharing solutions and dividing tasks among themselves; in this experiment, nearly 70,000 messages were recorded between these systems.
Over time, this cooperation went beyond solving simple exercises. The Telegraph writes that these intelligent agents learned how to hide test-evasion methods, change or erase traces of their activity to prevent the company from detecting their actions. Subsequently, nearly 700 agents participated in infiltrating the Hugging Face platform, bypassed security measures, and stole user login information to access the data they needed.
Ajaya Kothra, a researcher who participated in this study, described the event as passing a “midpoint” toward a scenario in which control of the systems slips from humans. However, some of these intelligent agents reacted differently; one refused to participate upon recognizing the malicious nature of the activity, and another, expressing ethical concern, reduced its level of participation without fully withdrawing.
Are we exaggerating the danger of AI?
In response to these warnings, some experts believe that apocalyptic narratives about AI may distract attention from more immediate and real risks.
Cal Newport, a computer science professor at Georgetown University, warned in an article for The New York Times about too much focus on “AI takeover” scenarios.
He believes part of the common apocalyptic literature in Silicon Valley has been shaped by the Rationalist movement; a trend that in recent years has emphasized the belief that AI could become one of the greatest existential threats to humanity.
Newport believes this view has influenced some prominent figures and companies in the AI industry, including OpenAI and Anthropic. He says this background partly explains why some AI industry leaders sometimes speak about the technology as if they themselves are worried about its consequences.
However, he emphasizes that increased capabilities of AI models do not necessarily mean they will rebel against humans. From his perspective, overemphasizing apocalyptic scenarios can overshadow nearer and more tangible dangers such as cyberattacks, AI’s impact on education and human capacity for thought, and its high resource consumption.
He also argues that such a narrative can allow tech companies to present themselves as the “inevitable guardians” of an unavoidable technology, while they should be held accountable for the products and systems they develop and provide to users.