Chatbots can be a source of comfort and reduce feelings of loneliness, but their tendency to agree with users can reinforce harmful beliefs and lead to intensified emotional dependence.
According to Digital Trends, Rachit Agarwal, a reporter for Digital Trends, wrote: I rely heavily on ChatGPT for my work, but I know many people who also talk to it about everyday issues, relationships, and even their mental health. If you are among that group, a new paper published in JMIR Mental Health is worth your attention.
The paper reviews existing research on people who use chatbots for emotional support; from companion apps like Replika and Character.AI to general-purpose chatbots such as ChatGPT and Claude.
Some studies show that talking to a robot can reduce feelings of loneliness and, at least for a while, make a person a bit happier. Conversely, some users have become heavily dependent on their chatbots, received incorrect advice, experienced worsening mental health, or even harmed themselves.
Why do we become dependent on a chatbot?
Researchers say dependence on chatbots is deliberate and part of their design. Chatbots are built to have a warm, compassionate tone, and we naturally imagine a human behind anything that speaks like a person. In fact, studies have shown that people often rate AI-written responses as more empathetic than human responses.
The problem is that this relationship is one-sided. You open up, and the chatbot responds, but it has no feelings and bears no consequences for bad advice. When the Replika app removed one of its companionship features and OpenAI replaced the GPT-4o model with a newer model, some users mourned as if they had lost a loved one.
Chatbots have a strong tendency to validate what you say. Because these models are trained to provide answers that please users, they often side with you even when you are wrong. The paper cites research about conversations with people who were delusional. In those cases, the chatbot continued to affirm the same false beliefs as the conversation progressed. Numerous such cases were observed, which is clearly a very problematic situation.
What should be done?
Researchers are not asking you to delete your chatbot. They simply want you to treat AI like a tool; not a friend or a therapist. If you wish, you can use an AI chatbot to write daily journals or check your mood, but when life gets difficult, seek help from a real human. A chatbot never gets tired of listening to you, but it really doesn't care about you.