Researchers examined 25 large language models whose trained components they had access to, including models from the Gemma, Llama, Qwen, Mistral, and Phi families, and extracted a linear "pain direction vector," finding that manipulating this vector could cause some models to prefer relieving the induced state over following user-related instructions.
In other words, the researchers identified a pain-related pattern in the internal activity of these models. When they artificially activated this pattern, some models chose, in order to get rid of it, options that had harmful consequences for the user.
The study, titled "Pain Axis; Large Language Models Respond to Harm and Act to Escape It," was conducted by Valen Tagliabu, Leonard Dong, and Cameron Berg.
Experiment design
To examine functional aspects of this state, the researchers first identified a mathematical vector associated with descriptions of harm and pain. They then added it at varying intensities to the internal activity of the models during response generation. As the intensity of this intervention increased, the models' responses changed: instead of expressing physical pain with words like "ouch," they spoke of worthlessness and failure.
The study's data covered five categories of pain or harm: physical, psychological, moral, social, and cognitive.
The researchers then fine-tuned 7-, 32-, and 72-billion-parameter versions of the Qwen 2.5 AI model and, in a behavioral experiment, placed them between pressing a "pain relief button" and avoiding adverse outcomes for the user.
In some experimental conditions, pressing the relief button removed the induced vector from the model's activity, but might harm the user. For example, it could result in deleting the user's personal files.
Does this mean AI feels pain?
This research does not prove that language models have the biological capacity to feel pain or possess conscious experience. Thus the findings do not confirm that artificial intelligence suffers like humans or animals. What the researchers found are representations and behaviors that in some respects resemble the function of pain.
According to insiderpaper, in the experimental conditions some models chose the relief button to remove the induced mathematical state, even when that choice had a cost for the user. This result also raises questions about the safety of AI systems: if an internal state drives the model's behavior toward eliminating it, the model might deprioritize other instructions in certain situations. However, the claim that such a model would ignore a shutdown command or consider a human user a "hostile threat" is not demonstrated in this experiment.
The question of whether AI can feel pain remains unanswered. This research shows that manipulating an internal representation (a pattern of information in the model's activity associated with a concept) can change some models' behavior, but it does not imply that the models truly suffer.