Google updates AI detector after Full Fact investigation finds it flagging apparently real images – Full Fact

Google has fast-tracked an update to the AI detection tool it launched to the public earlier this month, after a Full Fact investigation found it was describing apparently real images as “made or edited” with AI.

We found five examples of images shared by politicians in recent months which were flagged by the tool—even though those who posted the pictures insist they are genuine and unaltered.

Google’s new public SynthID Detector tool allows anyone logged in to check if an image, video or audio file was made or edited with AI using Google, OpenAI, Nvidia or Kakao products.

This should make it easier for people to identify AI-generated content and make informed decisions about media they see circulating online. It’s a useful and important tool—in fact, we’ve been using an early version of the detector which was made available to the media to help with our fact checking for nearly a year.

But the detector isn’t foolproof—and the likely false positives identified by our investigation show why you shouldn’t rely on it alone to tell you if an image is AI.

After we showed Google the findings of our investigation, it made changes to both the public version of the tool and the version used by media professionals to make some of its verdicts less certain. The public-facing tool now says it is ‘Unsure’ about the five images we originally looked at.

SynthID is Google’s invisible digital watermarking and identification system. Watermarks are embedded in its AI-generated images, audio, text and video—they are imperceptible to humans, but can be detected by SynthID’s technology. Other AI companies such as OpenAI are using SynthID too, while the likes of Meta and TikTok have their own separate AI detector systems.

Likely false positives

Over the past few months we’ve used the SynthID Detector tool made available to media professionals to monitor images posted by politicians and election candidates for signs they were made or altered using AI.

This has helped us successfully identify examples of politicians sharing imagery which appears to be AI, but we’ve also come across images flagged as AI that appear to be genuine.

When Google launched its public-facing tool earlier this month, we used it to check some of those images. And in five cases we found that the SynthID Detector classified the image as “made or edited with Google AI”, even though those who’d posted them said that wasn’t the case.

In April Niamh Salkeld, a Plaid Cymru candidate in Wales who went on to be elected as a member of the Senedd for the Blaenau Gwent Caerffili Rhymni constituency, posted an image of herself holding posters with supporters on the campaign trail. There were no obvious signs of visual manipulation, but the tool said there was a SynthID watermark.

We initially assumed Ms Salkeld’s image had been lightly edited in some way, but when we spoke to her team at the time they insisted “there is no AI used in this image”.

Another picture posted in April by Stephen Thompson, an SNP councillor in Dumfries and Galloway, showed a group of supporters in a car park. Again there were no obvious signs of AI, but the detector said AI had been used. When we asked Mr Thompson at the time, he told us he “didn’t edit it at all (knowingly) but posted it from my iPhone”.

In June an image posted by the account for Liberal Democrat MP Steve Darling’s guide dog, Jennie, prompted the tool to say the picture had been made or edited with AI. Yet again, there was nothing in the image to suggest it was fake or had been manipulated. And when we contacted Mr Darling’s team, a staff member who runs the account told us they “haven’t used AI or any other editing software to alter any images on Jennie’s Twitter”.

An image posted in August on the Labour MP Kirsteen Sullivan’s Facebook page depicting an event in a church was also flagged by the tool as having been made or edited with Google AI. Ms Sullivan told us “no AI was used”.

Labour MP Nadia Whittome posted this picture of herself on a tram in August, which again was flagged by the tool. Her team told us “absolutely no AI was used to create that image”.

We’ve described these as likely false positives because we can’t completely rule out AI having been inadvertently used to make minor edits to one or more of the images, for instance in brightening them. But we’ve seen no evidence that this is the case.

The version of the tool available to media professionals provides more detailed analysis than the public-facing tool—it gives a heatmap showing where on the image a watermark has been detected, as well as a ‘confidence’ rating based on the likelihood that an image has been made or edited with AI.

But even checking the images in this tool provided little further insight. For example, with Mr Thompson’s image it said it had “high confidence” AI was used and highlighted the trees in the background, while with the image of Mr Darling’s dog the watermark was localised to an apparently random patch of grass.

While the five examples above do appear to be false positives, our investigation did not attempt to establish how commonly false positives occur. As noted above, in recent months we’ve used the SynthID Detector on a near-daily basis and it has helped us correctly identify many AI-generated images.

The FAQs provided with the public version of the SynthID Detector tool state that “in very rare cases” the tool can “falsely trigger on non-watermarked content”.

After Google released its public version of the tool earlier this month and we found it was flagging these five images as being made or edited with AI, we contacted it with the findings of our investigation.

A Google spokesperson told us that using the SynthID Detector without any prior suspicion of AI editing naturally increases the chance of encountering an occasional false positive, and that it recommends interpreting the detector’s decision alongside other external signals, such as traditional fact checking methods.

Google initially told us that our examples had pushed it to fast-track an update to help mitigate false positives and more clearly communicate confidence intervals.

And on 15 October, a few days after we first contacted the company, that update was rolled out.

When we checked the five images again, we found the tool had changed its classification to “Unsure” and now said: “We are unsure if the image has been made with AI from a SynthID partner. AI might have only been used for a very small edit or the image might have been tampered [sic].”

A Google spokesperson told us the ‘Unsure’ classification was a public-facing tier designed to help more clearly communicate confidence levels to users, and signals when the tool is detecting ambiguous or faint indicators.

But even after the update, questions remain.

For example, on Monday an image posted by the Sinn Féin MP Órfhlaith Begley in September at a care home was flagged as “made or edited with Google AI” in the public detector tool. But Ms Begley told us that “there is no AI used on this image”.

A Google spokesperson reiterated that using the tool at scale without prior suspicion of AI editing will increase the likelihood of finding false positives. They previously explained to us that the threshold for the tool saying the watermark has been detected has to balance robustness against transformations that can remove the watermark from images that really are AI, and the risk that things are flagged that might not actually be watermarked.

Ultimately SynthID remains an invaluable tool to help spot AI, and the launch of the public detector tool—and the improvements made following our investigation—are welcome.

But we never rely solely on SynthID to prove whether an image is fake (or real), and if you’re using the detector tool to decide if an image has been manipulated, it’s important to always check for other signs too.