Allison Johnson for The Verge: Google’s AI tool helped us add disasters and corpses to our photos

In our week of testing, we added car wrecks, smoking bombs in public places, sheets that appear to cover bloody corpses, and drug paraphernalia to images. That seems bad. As a reminder, this isn’t some piece of specialized software we went out of our way to use — it’s all built into a phone that my dad could walk into Verizon and buy.

This article sums up my general unease with this tech pretty well. I’m all for lower the barriers for creative work, but it definitely feels icky to be this easy to create fake scenes so easily.

Take insurance claims, for example. Photographs of your possessions or of an incident are often important evidence. Now imagine you’re in a minor fender bender collision with another car and that owner uses a Pixel to make the tiny dent in their car look like they got plowed into. Maybe they even took a selfie and made themselves look like they suffered some bodily harm in the incident as well. Of course you should take photos, but now we have a he-said-she-said situation that’s only going to turn into more of a toss up as these tools get more capable. Did they use AI tools to make the crash look worse or did you use AI tools to make the crash look more mild?

Of course there are ways to mitigate some of these problems. You can put metadata on an image that indicates it was modified, but metadata can be changed and removed easily enough. Maybe AI tools can be used to detect AI images, although that means trusting AI to make decisions. Maybe a third party can get unbiased pics of the scene, although that’s not available in many situations. I’m not saying these solutions (or solutions I haven’t thought of) will fail, I’m just saying that we have new problems to solve that didn’t exist at this scale before. Yes, Photoshop has existed for decades, but the barrier to entry was higher making the risk of someone convincingly faking something like this much lower. Now all anyone has to do is circle a bumper and type “make it look more beat up.”