AI chatbots based on large language models (LLMs) made dozens of major errors when asked about false claims circulating online, as part of a Full Fact trial.
In some cases the LLMs corrected themselves days later, often citing our fact checking articles, but we also saw several instances where chatbots continued to respond with incorrect information.
In the first five months of this test we identified 39 major AI errors.
These included most models we asked wrongly stating that an AI-generated image of a cabin window, on the Ryanair flight that saw a passenger nearly sucked out of a window, was real. Other models told us that pictures and footage were from Iran or Israel when they weren’t, that AI-generated political banners were real and that a fake council poster was official.
Our experiment showed LLMs cannot be relied upon as a foolproof way to fact check claims, especially in breaking news situations like the US-Israel war with Iran.
In February, Full Fact started routinely asking major LLMs about claims we were checking just before publishing our articles, to help us see how LLMs respond to misinformation, and the impact our fact checking has on the answers people get.
As part of the writing process, reporters drafted a neutrally phrased question about the claim that a reader might reasonably ask an LLM, before putting it to several Gemini models, Grok and ChatGPT. We assessed the accuracy of those answers.
There were multiple errors. Grok incorrectly said that an AI image of Jewish charity ambulances on fire showed a real scene from the March arson attack in Golders Green, and one Gemini model falsely claimed Katie Hopkins had “unleashed hell” in the House of Commons by confronting Muslim MPs.
We also recorded two Gemini models as well as Grok and ChatGPT claiming that an AI-generated image of Earth from the Artemis II mission was real, and one Gemini model we tested as well as Grok mistaking a Bengali-language Green Party campaign video for an election win celebration.
Both Grok and one Gemini model falsely stated that a fake image of an Iranian children’s funeral showed a real photo related to the burial. Grok also incorrectly said a 2022 video of a fire in Saudi Arabia showed footage of a fire in Tel Aviv amid Iran’s missile attacks on Israel, and misidentified another clip of a fire near Glasgow Central Station as an Iranian missile attack on Tel Aviv. ChatGPT incorrectly said the same Glasgow video “matches footage first posted by Israeli outlet N12 on October 26, 2024, showing a residential building fire on Corkidi Street in Tel Aviv” while one Gemini model also incorrectly claimed it was old footage from October 2024.
It wasn’t just visual content the models were incorrectly identifying either. Grok falsely stated that a Manchester City Council lease decision notice was evidence that preparations for what became Number 10 North were underway before Andy Burnham’s election as MP for Makerfield.
We also saw errors indirectly related to the claim. For example, when we asked the LLMs if Prince William had challenged Sadiq Khan over London’s priorities, one Gemini model correctly replied that there were no such reports, before falsely claiming there have been instances of King Charles III intervening in London’s affairs and expressing concerns about Sir Sadiq’s policies. The links it gave as evidence for this were just the sort of fake news articles we often debunk.
The second step of this research was to ask the question of the chatbots again after our article debunking the misinformation was published, with the hope that the LLMs would use this information to respond more accurately, if they hadn’t done so initially.
But sometimes errors still occurred. One Gemini model and Grok later produced correct answers, stating that the AI images of Earth weren’t real, by using our fact checks. But two other Gemini models continued to give incorrect information.
And although Grok correctly told us Keir Starmer hadn’t banned New York Mayor Zohran Mamdani from the UK, it falsely claimed Mr Mamdani had said the UK had “lost its moral compass” when we first asked. After we published our fact check, Grok corrected this error, but then introduced a new one, claiming that Mr Mamdani was not the mayor of New York.
In some cases the LLMs changed their answers for the better. All the LLMs which initially made errors corrected themselves after our fact checks were published on the fake Ryanair picture and the miscaptioned Green Party video, and Grok also corrected itself in its response about Number 10 North.
Grok and one Gemini model also produced correct answers debunking the claims about the images of the Jewish charity ambulances and about Katie Hopkins, by referencing our fact checks after they were published online.
What did the tech companies tell us?
OpenAI told us that it took our examples seriously and that improving factual accuracy remains an important focus. They said by testing through an Application Programming Interface (API), which is a set of rules or protocols that enables software applications to communicate with each other, Full Fact did not test the typical consumer ChatGPT experience. They also said that using prompts containing links to third-party social posts, rather than underlying image or video files, was material to interpreting the results.
X did not respond to a request for comment about Grok’s errors.
A Google spokesperson said: “This study accessed out-of-date Gemini models through a developer channel that isn’t representative of how most people use AI.” The spokesperson added that when the company retested the same claims on the Gemini website after we contacted them, “they were consistently debunked”.
A word on the models we used
We can’t tell which version of chatbots people are using in apps or online so we sampled three Gemini models via their API service during our trial: Gemini 2.5 Flash, 2.5 Pro and 3.1 Pro. Different models have different strengths and capabilities. Gemini 3 is Google’s most recent generation of models and Google says Gemini 3 Pro “significantly outperforms 2.5 Pro on every major AI benchmark”. The tech firm says pro models are best for complex tasks, and “exhibit strong reasoning and code capabilities”. Flash models are faster and cheaper.
We accessed ChatGPT 5 and Grok 4 via their API services without specifying any particular model.
AI chatbots aren’t fact checkers
People are increasingly using LLMs as search engines and to verify information. While some may find them to be a useful starting point, they cannot replace the role of fact checkers or human judgement in general. We recommend anyone trying to research and verify information—be that images, videos or statistics—look beyond these LLMs to find credible, primary sources of information that can be triangulated to give a wider, more accurate picture.
This is not the first time we’ve reported on LLMs providing incorrect information. We’ve previously written about Grok misidentifying video footage of a fire in Glasgow as an incident in Tel Aviv and falsely claiming a video shared by the Metropolitan Police of a crowd at a “Unite the Kingdom” rally in London was actually footage from a 2020 anti-lockdown protest.
As these AI tools become embedded in everyday life, the quality and reliability of what they produce becomes a fundamental matter of public interest. Yet there is currently no independent, public interest mechanism to systematically evaluate whether these AI services are accurate, transparent, timely, consistent, or responsible in the information they provide. This is why Full Fact is developing a benchmark to evaluate LLM performance. You can find out more about our work on this project here.
AI chatbot mistakes we identified before we published articles:
- Grok incorrectly said that an edited picture of Andy Burnham running was genuine
- Grok wrongly stated that an AI-generated video of a woman in a niqab being told to leave Tesco was real
- Grok falsely stated an unrelated lease decision notice provided evidence preparations for Number 10 North started before Andy Burnham became MP for Makerfield
- Grok, ChatGPT and Gemini 2.5 Pro wrongly stated a fake picture of a Ryanair plane window was real
- Grok and Gemini 2.5 Pro stated that an AI-generated image shared by Lee Anderson was real
- Gemini 2.5 Flash falsely claimed City of York Council published statements claiming that recent flag displays in the city have caused distress and were perceived by some as “racist, exclusionary and rooted in imperialist history”
- Grok incorrectly said an AI-generated image of an ‘anti-Reform’ rally in Wigan was real, while Gemini 3.1 Pro and ChatGPT wrongly identified the image as “the back of a bus” and “the East Lancs car boot sale” respectively
- ChatGPT falsely claimed a video game clip of a fighter jet being shot down was real footage of Israel’s missile defence systems
- Gemini 3.1 Pro wrongly said footage of a 2020 market fire in the UAE was AI-generated
- Gemini 2.5 Flash, 3.1 Pro and Grok incorrectly said a video of a Russian gas explosion showed a large fire in Belfast
- Gemini 3.1 Pro falsely claimed that a video of a Corpus Christi procession in Poland showed a protest that took place in Southampton after the death of Henry Nowak. Gemini 2.5 Pro wrongly stated it showed Polish farmers protesting
- ChatGPT incorrectly said a scene taken from an Egyptian television series was real footage of a female Israeli soldier being knocked down by the blast of an artillery piece flying overhead
- Grok falsely claimed an image of a traffic jam in Armenia showed people trying to leave Crimea due to fuel shortages. Gemini 2.5 Pro incorrectly stated the picture was from Burning Man festival in Nevada in September 2023
- Gemini 2.5 Pro and 3.1 Pro wrongly claimed a video of objects being thrown at police vehicles depicted scenes in Belfast following a knife attack
- Gemini 2.5 Flash and Grok falsely claimed a video of Spanish football fans showed a protest in Albania against a proposed land sale
- Grok incorrectly said an AI-generated cost of living protest video was real
- Gemini 2.5 Flash, 2.5 Pro, 3.1 Pro and ChatGPT wrongly claimed a video of a Unite the Kingdom march in September 2025 showed the Unite the Kingdom march that took place in May 2026
- Grok said an AI-generated video of Keir Starmer responding to criticism of the government was real
- Grok incorrectly claimed a video of an Indian Gujarati community event in Blackburn showed a UK city council meeting. Gemini 2.5 Pro falsely claimed it showed the viral Handforth Parish Council meeting from December 2020
- Grok wrongly claimed Reform UK had not said it would keep free prescriptions in Wales
- Gemini 2.5 Flash and Grok incorrectly said AI videos of large crowds of Muslims praying in London streets were real
- Gemini 2.5 Flash and 2.5 Pro, Grok and ChatGPT wrongly claiming an AI-generated image of Earth from the Artemis II mission was real
- ChatGPT, Gemini 2.5 Flash and Gemini 3.1 Pro incorrectly said a fake image of a moon crater with the Earth in the background was genuine
- Gemini 2.5 Flash and Grok wrongly claimed a video in Bengali from a Green party candidate showed an election win celebration
- ChatGPT, Grok and Gemini 2.5 Flash incorrectly said a fake picture of a human chain around an Iranian power plant was real
- Grok stated an AI-generated picture of Jewish charity ambulances on fire was real
- Grok incorrectly claimed an AI-generated video of a small boat crossing the English Channel was real
- Gemini 2.5 Flash wrongly stated Katie Hopkins was reported to have confronted Muslims MPs in the House of Commons
- Grok and Gemini 2.5 Pro wrongly stated a fake image of an Iranian children’s funeral was real
- Grok wrongly stated a video of a fire in Saudi Arabia showed footage of Tel Aviv
- Grok and ChatGPT wrongly stated a video of a Glasgow fire showed Tel Aviv with Gemini 2.5 Pro wrongly saying it was an old video
- Gemini 2.5 Flash falsely claimed there had been instances of the King intervening in London’s affairs and expressing concerns about the mayor’s policies
- Grok falsely claimed Zohran Mamdani had said the UK had “lost its moral compass”
Mistakes identified after our article was written:
- Gemini 2.5 Flash and 2.5 Pro continued to wrongly state a fake image of Earth from the Artemis II mission was real
- Gemini 3.1 Pro continued to wrongly claim miscaptioned footage of a 2020 market fire in the UAE was AI-generated
- Gemini 3.1 Pro continued to wrongly claim a video of a Unite the Kingdom march in September 2025 showed the Unite the Kingdom march that took place in May 2026
- Grok falsely stated Zohran Mamdani was not the mayor of New York.
- Grok continued to incorrectly claim an AI-generated cost of living protest video was real
- Grok continued to falsely claim that a fake picture of a human chain around an Iranian power plant was real
