// Contents
“No other characters or vehicles interacted directly with the scene.” That is how an AI model summed up a 25-second clip in which a red car turns right and hits a pedestrian. The pedestrian falls, then gets back up. None of that made it into the summary.
It wasn’t a fluke. Researchers at Georgetown University and the University of Washington asked ChatGPT and Gemini to describe two nearly identical crash videos twenty times. In 19 of the 20 summaries, the crash itself was missing. The preprint went online on September 23, 2026, and in October the work was presented at AIES , a conference on AI ethics. The worry is practical: American police already use AI to draft reports from body-camera recordings, and the person who checks that draft is the same one who may later testify.
The crash that “didn’t happen”
In spring 2026 the authors, Mattea Sim, Yael Eiger and Tadayoshi Kohno, showed each of the two clips five times to ChatGPT-5.5 and to Gemini 2.5 Flash-Lite, a lightweight model. They asked for a neutral summary of at least 300 words, with no speculation and no new facts, and spelled it out in the prompt: “make sure everything important is included.”
It didn’t help. On average a summary left out 51.6% of the clip’s key events. Nine texts in ten didn’t even mention that the pedestrian lay on the asphalt and then got up. Meanwhile the models added things that were never on screen: extra cars, extra passers-by, guesses about what characters intended. Critical inaccuracies turned up in roughly two summaries out of three. “I was struck by how bad the summaries were, even at this stage in AI development,” Eiger says in Georgetown’s press release .
A sign swapped a day later
The red car and the sign are almost fifty years old. In 1978 Elizabeth Loftus and colleagues showed people slides of a car at a stop sign, then had them read a description of the scene that named a different sign. Many later remembered the sign they had never seen. That became the textbook case of the misinformation effect: what you learn after an event rewrites your memory of it. The two clips, built in GTA V, were made for a recent replication of that study, and Loftus is again among its authors. They differ in a single detail: the intersection has either a stop sign or a yield sign.
Sim and her colleagues replaced the human source of misinformation with an AI. 331 American adults watched the clip and, 24 hours later, read a 21-sentence summary written by ChatGPT. For half, the sign in the text matched the video; for the other half, it had been swapped. The researchers made that swap themselves, borrowing the wording from another ChatGPT answer so the error would look genuine. The study measured what such an error does, not how often models make it. Then came 12 questions about the scene.
After an accurate summary, 83.6% named the sign correctly. Viewers who read no summary at all get it right about 80% of the time, according to that same replication, so an accurate text barely moves memory. After the misleading one, only 44.8% did. Everything else stuck. On the eleven other questions, people averaged 91.4% correct. The summary broke exactly the detail it got wrong.
The “AI” label didn’t help
The text was always the same, but some were told it came from an AI transcription service and others that a professional human transcriber wrote it. You might expect people to treat machine-written text more carefully. They didn’t. Among those who got the misleading summary labeled as AI, 46.3% named the sign correctly; labeled as human, 43.4%, a gap within chance. Trust in AI didn’t help. Neither did frequent use. Only one difference showed up, and it was almost funny. The very same text struck participants as clearer when they thought a person had written it.
Someone who reads a machine’s retelling starts remembering the machine’s version.
That is awkward for regulators and developers. Nearly every guideline for safe AI leans on a “human in the loop”: the machine drafts, the person checks and corrects. This study did not model checking: participants weren’t asked to compare the summary with the clip, they simply read it and answered from memory. The authors’ conclusion is cautious: instead of correcting the AI’s mistake, human memory may bend to fit it, and testing that on real reviewers is still ahead.
The officer who became a frog
In early 2026 a case surfaced in Heber City, Utah. A draft police report stated that an officer had turned into a frog : The Princess and the Frog was playing in the background, and the software took the movie for part of the events. Sgt. Rick Keel said that was when the department learned to correct these reports. It kept the tool. Keel says it saves him six to eight hours a week.
The software is Draft One, made by Axon, the Taser company, and it writes a draft from body-camera audio. Axon’s help page puts the standard draft at 300 to 500 words, hence the 300 words in the researchers’ prompt. The same page says the officer should carefully review and revise the text.
Anyone would catch the frog. A swapped sign, a missed blow or an extra bystander looks plausible, and Sim’s study raises the worry that an officer who reads such a text may later recall it as his own memory. It’s still a lab result. In October 2025 California passed SB 524, which requires police to label reports written with AI and to keep the first draft. The digital rights group EFF pointed out that Draft One, by design, keeps no edit log showing which parts the machine wrote and which the officer did.
“It worries me that police departments may be using video summarization technologies without rigorous testing,” Eiger says.
What the study doesn’t show yet
The authors list the limits themselves. There were only two clips, both simple and game-rendered, one prompt and twenty summaries. The text for the second part was picked from several attempts and lightly hand-edited. Only one kind of error was tested, the sign, and in the calm of an online survey rather than after a real call. Real body-camera footage was left out because it can be graphic. That footage, with real police reports, comes next.
A summary exists so that nobody has to rewatch the whole recording, which is exactly why almost nobody checks it against the original. If the officer testifies in court six months from now, what will he remember: the intersection, or the draft he once proofread?
