A study found that texts summarized by artificial intelligence can distort the memories of people who are shown those texts.
The new research shows that AI systems not only commonly summarize material shown to them incorrectly, but these mistakes can deceive people and lead them to hold incorrect information.
The study was conducted amid a rise in the use of AI-generated summaries in everything from transcribed meeting notes to sensitive situations like police image analysis. Researchers at Georgetown University Medical Center examined how these summaries can change people’s perception of the truth.
Researchers first examined how well AI systems actually summarize information in practice and found that these systems repeatedly produced incorrect information. The most common error found was that the summaries omitted important details of the material they were summarizing, and researchers found that across different prompts and models, the system omitted 51.6 percent of important information.
They then examined how these summaries might affect the memory and understanding of people who rely on them. Researchers found that if participants were shown an AI-generated summary that misleadingly described events (which the participants had actually seen earlier), they could not accurately recall those events.
Researchers found that this occurred regardless of whether people thought the summary had been written by an AI or a human, and it did not appear to depend on whether people trusted or were familiar with AI.
In the new study, 331 participants watched an animated video showing a car approaching an intersection. At that intersection there was a sign telling the driver either to stop or to proceed. The car then turned at the intersection and collided with a pedestrian crossing the street.
The researchers then waited 24 to 48 hours and gave participants an AI-generated summary that was either accurate or contained incorrect details. Participants were then asked to recall the event and say who was at fault, and the study found that those who had read the misleading AI summary were significantly less able to recall the event correctly.
The researchers noted that such summaries are used across a wide range of applications including reviewing police images. They also warned that the finding suggests these applications may be misleading even if none of the people involved in the process have malicious intent.