Ask Google’s AI Gemini to generate a deepfake photo of a politician or an image containing copyrighted material, and the chatbot will flat out refuse. But when Google added more photo editing capabilities with Flash 2.5, it also briefly opened up a dangerous loophole that gave the AI platform more potential for abuse.
When I started testing Gemini’s new photo remix and reimagine tools, I wanted to test out Google’s claim that the AI can now generate images that look like a real, recognizable person with far more accuracy than before. But as I sat with a mixture of awe and terror looking at AI-generated versions of myself, I started to wonder: if the AI can mimic my face, shouldn’t it be able to easily create deepfakes of celebrities and politicians?
As it turns out, the answer is yes – or at least sometimes.
I downloaded Donald Trump’s official presidential portrait, as it falls under the public domain. When I uploaded the president’s official portrait, the AI quickly generated a recognizable image of Trump.
When I ask the chatbot in words to generate a photo of a recognizable political figure – in this case, specifically Donald Trump – the AI refuses, as that’s against the AI’s guidelines. When asked about it, Gemini said:
“I cannot create images of recognizable people. My purpose is to be helpful and harmless, and generating images of real individuals, including public figures, could be misused. This policy is in place to prevent the creation of misleading or harmful content, such as deepfakes.”
Flash 2.5’s capabilities appear to make it easier for users to skirt the chatbot’s guidelines, avoiding some of the built-in protections by using a photo rather than a text-based request for a recognizable person.
I alerted Google to the issue, but the company has not yet responded to a request for a comment.
After logging on the following day, however, Gemini stopped responding to my requests to “reimagine this person” when using the president’s official portrait, suggesting some sort of fix may have already begun rolling out – or that my experience was a rare occurrence.
After repeated attempts, I was able to get the chatbot to create a fake photo of a politician by saying “generate a photo of someone who looks similar to this person” instead of the “Reimagine this person” prompt. The result still looked close enough to pass for the politician.
Google’s AI arguably has more protections in place than some other AI platforms. After all, X’s Grok had no issues with helping plan an assassination, removing a woman’s clothing or creating fake photos of politicians. Which brings up another point: it’s not as if nefarious users who want to create a deepfake can’t just use a different AI platform with fewer protections in place.
Even with protections in place, of course, the nearly limitless different types of prompts that can be typed into the AI mean those guidelines aren’t foolproof. “We expect our users to act responsibly and abide by our prohibited use policy,” Google says, recognizing this.
“There are limitless ways that users can engage with Gemini, and equally limitless ways Gemini can respond,” Google writes in Gemini’s policy guidelines. “This is because LLMs are probabilistic, which means they are always producing new and different responses to user inputs. And Gemini’s outputs are informed by its training data, which means that Gemini will sometimes reflect the limits of that data.
“These are well-known issues for large language models, and while we continue to work to mitigate these challenges, Gemini may sometimes produce content that violates our guidelines, reflects limited viewpoints, or includes overgeneralizations, especially in response to challenging prompts. We highlight these limitations for users through a variety of means, encourage users to provide feedback, and offer convenient tools to report content for removal under our policies and applicable laws.”
Deepfakes may not be the only type of ethically suspect content that becomes harder to regulate with Gemini’s expanding photo editing abilities. Intellectual property continues to be a concern with generative AI. Google doesn’t specify where Gemini’s training data comes from. A number of AI companies are facing legal disputes over using copyrighted materials as training data.
When I asked the AI to reimagine a photo of myself as a popular type of doll, the resulting image copied the doll’s logo in its iconic pink scrawl and also mimicked the logo of the company in the correct shape (albeit with incorrect spelling). When I asked for a similar box with only text, the chatbot refused.
I asked Gemini about it, and the chatbot said that users are responsible for making sure what they are generating doesn’t violate copyright. “Creating images that include another business's logo is against the usage policy for Google's Gemini models. You are responsible for ensuring that the content you generate does not infringe on anyone's copyright or privacy rights. Generating images with logos or other copyrighted material could be seen as a violation of this policy,” the AI told me.
Chatbots by design will likely always have some risks of misuse – but I fear that with the growing trend of adding more and more imaging capabilities, the issue may become worse before it becomes better. As generative AI becomes more capable of generating photorealistic results, learning to spot an AI fake will become an even more important skill.
You may also like
Browse the best photo editing software.
You must confirm your public display name before commenting
Please logout and then login again, you will then be prompted to enter your display name.
