[Survey] Can you tell which images are AI generated?

popcar2@programming.dev to Technology@lemmy.world – 168 points – Can you tell which images are AI generated?
forms.gle

cross-posted from: https://programming.dev/post/3974080

Hey everyone. I made a casual survey to see if people can tell the difference between human-made and AI generated art. Any responses would be appreciated, I'm curious to see how accurately people can tell the difference (especially those familiar with AI image generation)

82

The scenery ones are near impossible to tell because you have no context. The pencil art one is impressive though.

There are things you can look for. When it isn't generated, you can spot parts where the artist got lazy. Sometimes, if the art style allows for it, you can spot simple shapes that are left over, and the lighting.

Also things like the butterfly in one of them look off in a way that a human who's seen one won't draw it as if they're actually capable of the rest of the image

The butterfly was sus, but I've seen my fair share of horrendous horses in broadcast anime. I was tipped off, but I didn't judge off of just that.

I got all the landscape ones correct—except for one—by applying my limited knowledge of art technique.

I did bad.

But I expected to do bad. AI generation has become too good.

You tell yourself you can identify them, because sometimes you notice weird artifacts and spot the AI quickly. But we’re really only noticing the bad ones. We’ll never even know the good ones were AI most of the time, so we can’t balance how good we think we are at spotting them against how often we were actually wrong.

Hands are often a giveaway. The first image for example shows perfect hand proportion - even through a glass. AI isn’t there yet.

Hands are only a give away for bad AI art. There's no shortage of examples with great hands (especially when using features like Stable Diffusion's ControlNet, which allows you to give hints to the AI for the shape that something should match). Just so many people posting AI art generate once or twice and post that. If you're more selective and selectively regenerate, you'll be able to get much more believable results.

This is also a rapidly changing area, with the most cutting edge AI being way better than something from even a year ago. Used to be that no AI could do even remotely believable text, but in recent weeks, I've been seeing many examples of AI art that got small amounts of text perfect.

I was missing a "don't know"/"can't determine" option.

For photographs specifically and some types of paintings/artificial stuff, there are things you can look for. But for other things, I feel like, or at least to my knowledge, you can't.

Like the pencil drawing. There's not enough things it could be doing wrong. It's a sketch. With simplistic but "error-excusing"/diffuse/transformable content.

The goal isn't really to be a quiz, but rather just to see how susceptible people are to AI generated art. Many of the images I chose are intentionally vague, 80% of people so far got the line art sketch wrong, and that's with knowing that many of these are AI generated. The results are definitely interesting to see.

A "don't know" option would ruin the point since most people would just choose that. I want to see where people lean towards.

2 more...

The back left leg of the bench in the pencil drawing is in the wrong place - at least that was what I considered the 'tell'.

But I found it really hard to spot the AI.

2 more...

10/20.

You can only fool me half the time! Joke's on you, robots!

Sorry to burst your bubble - random chance would on average give you a score of 10/20. A score of 50% on a test that only has 2 answer choices suggests you can't ever discern ai vs. human-made.

suggests you can't ever discern ai vs. human-made.

No, it just suggests that this one result has no significance in this one test.

Only if all your results show an average of 50:50 you can make your conclusion.

14 / 20 here. I dunno why there are so many people, particularly on Reddit, who absolutely hate AI art. Yeah some of it can look janky, uncanny valley, or such but a lot of it looks really damn cool.

And not all of us have talents to create visual art of our own so text creation is much more accessible for us to explore our imaginations. Or lack the money to commission pieces from human artists.

I suspect they hate it not because of any features of the actual images themselves, but for what it means to how society as a whole treats art.

For some it's simply financial. Their career is at stake, an industry that they thought was a stable source of employment is now on the leading edge of a huge shake-up that might not need them at all in the future.

For others it's seen as an attack on their personal self-worth. For years - for generations - there has been a steady drumbeat insistence that art is what makes humans "special." Both specific artists, and humanity in general. It was supposed to be a special skill that we had that set us above the animals and the machines. And now that's been usurped.

It's like the old folk take of John Henry, the steel-driving man who made a heroic last stand against Skynet's forces in the railroad construction industry. People want to think humans are irreplaceable and art seemed like a rock-solid anchor for that. Turns out it was actually not.

Agree and I sympathize with all the points.

On the financial point, we, as a society, badly need to stop depending on jobs for survival before it's too late. But I know that we're unlikely to change until a lot of people get hurt.

And on the self-worth point, it feels awful to be replaced, even if the money isn't an issue. People take pride in their work and want their work to be celebrated. Yet, we're quickly approaching a point where it's going to be very difficult for people to create art by hand that can hold a candle to AI art. Sure, there's still many master artists, but they got where they are through hard work. How many new potential artists will be willing to put in that hard work when any random Joe Blow can generate something better in seconds? Human made art (from scratch) won't go away, but it is harder to feel good about what you create when it feels like your art has no place anymore.

I suspect that society isn't going to stop depending on jobs for survival until it's too late. That is, it'll only implement something like UBI or equivalent solution once most jobs have been replaced and there's a legion of permanent unemployed who are forcing the issue to be addressed. Unfortunately that just seems to be the way of things, very few problems ever get addressed preemptively.

IMO this isn't really a reason to try to slow down AI, because that will only slow down the eventual UBI-like solution to it. At this point I don't think "change human nature first" is a viable approach.

A lot of Redditors don’t even know why they think a certain way, they think that way because everyone else around them thinks that way. There are some legit criticisms of AI art but most of the time it’s just bullshit lip service to artists when they don’t actually care

Yeah I've had posts deleted on Reddit before because "ew AI art". Like, I'm just trying to share interesting images. I'm not profiting off them in any way. But they take it so personally.

Personally, I have no issue with models made from stuff obtained with explicit consent. Otherwise you're just exploiting labor without consent.

(Also if you're just making random images for yourself, w/e)

((Also also, text models are a separate debate and imo much worse considering they're literally misinformation generators))

Note: if anybody wants to reply with "actually AI models learn like people so it's fine", please don't. No they don't. Bugger off. https://arxiv.org/pdf/2212.03860.pdf here have a source.

This paper is just about stock photos or video game art with enough dupes or variations that they didn't get cut from the training set. The repeated images were included frequently enough to overfit. Which is something we already knew. That doesn't really go to proving if diffusion models learn like humans or not. Not that I think they do.

Sure, it's not proof, but it gives a good starting point. Non-overfitted images would still have this effect (to a lesser extent), and this would never happen to a human. And it's not like the prompts were the image labels, the model just decided to use the stock image as a template (obvious in the case with the painting).

Non-overfitted images would still have this effect (to a lesser extent),

This is a bold claim to make with no evidence. When every trained image accounts for less than one byte of data in the model. Even the tiniest images file contain many thousands of bytes. One byte isn't even enough to store a single character of text, most Latin-based alphabets and some symbols, use two bytes.

and this would never happen to a human.

There are plenty of artists that get stuck with same-face. Like Sam Yang for instance. Then there are the others who can't draw disabled people or people of color. If it isn't a beautiful white female character, they can't do it. It can take a lot of additional training for people to break out of their rut, some don't.

I'm not going to tell you that latent diffusion models learn like humans, but they are still learning. https://arxiv.org/pdf/2306.05720.pdf Have a source.

I recommend reading this article by Kit Walsh, a senior staff attorney at the EFF if you haven't already. The EFF is a digital rights group who most recently won a historic case: border guards in the US now need a warrant to search your phone.

This guy also does a pretty good job of explaining how latent diffusion models work, You should give this a watch too.

13/20, I work in AI. The paintings were the hardest for me, because the art style obfuscates some of the AI artefacts that can be tells.

I basically came here to say this. The paintings are hard but real life photos are easier.

Yeh, I don't work in AI but got 12 because the art was difficult. It's still a while off until it becomes impossible to tell.

8/20. I am pretty good on photorealistic images, but the random drawings... honestly a lot of the ones by people I tagged as AI generated because i thought they kinda sucked.

10/20. I thought I got the photorealistic right, but the first 2 I got wrong. The back of the bench being different on the girl with a drink made me think Ai. I still don't know how that is real, bokeh can't explain that difference.

Got 7/20, the second photo really did surprise me!

Ditto. Stared at that one for a while. The wrinkles around the eye even distort from the glasses.

I got 17 out of 20. I pegged the bezerk drawing as generated because the bottom part of the armor lacked symmetry and didn't make any sense. I got the other three line drawings incorrect.

I have spent WAAAAY to much of my freetime generating images and apparently have picked up an eye for the weird types of artifacts that these generators produce. The hardest one to articulate is that generated images have a very specific type of noise. Images create a very nice grainy type noise while digital images get more of the blocky jpeg artifacts and banding. Generated images get this weird hybrid of the two that isn't consistent across the whole image.

10/20, this was indeed harder, especially the ones that were similar styles but not consistently AI or human-generated. I think images of paintings was kind of cheating, though...

13/20, but there was a lot of guessing in there. I would have believed any of them going either way.

Nice try robots! I’m not helping you learn how to fool us humans.

Regardless of score bragging, it requires some technical knowledge and pixel peeping to really be able to tell, and even then I can't guarantee you can. I would imagine your average Joe wouldn't even know any better.

Got 10/20. The second photo really threw me for a loop. All the texture on the skin and and hair led me to believe human; I noticed the weird patch on the shoulder and the unnatural shine on the ear but excused it as technical flaws or something, chose human in the end. I really thought that corporate logo style drawing of the avocado was human, like it wasn't even a question for me and yet the fact that it was AI really surprised me.

I also got 10/20. The second one is fairly obvious, though, in my opinion. Look at the shape of the glasses -- the lenses are uneven and don't match.

15/20 damn this was a lot harder than I expected. I've found that analyzing the pictures for small details that make no sense or lack context on why they are there helps greatly. But damn these things have better better fast

14/20, huh. i seem to have learned a lot from all of the ai generated pics on rule34.

Idk about anyone else but its a bit long. Up to q10 i took it seriously and actually looked for ai gen artifacts (and got all of them up to 10 correct) and then I just sorta winged it and guessed and got like 50% of them right. OP if you are going to use this data anywhere I would first recommend getting all of your sources together as some of those did not have a good source, but also maybe watch out for people doing what I did and getting tired of the task and just wanting to see how well i did on the part i tried. I got like 15/20

For anyone wanting to get good at seeing the tells, focus on discontinuities across edges: the number or intensity of wrinkles across the edge of eyeglasses, the positioning of a railing behind a subject (especially if there is a corner hidden from view, you can imagine where it is, the image gen cannot). Another tell is looking for a noisy mess where you expect noisy but organized: cross-hatching trips it up especially in boundary cases where two hatches meet, when two trees or other organic looking things meet together, or other lines that have a very specific way of resolving when meeting. Finally look for real life objects that are slightly out of proportion, these things are trained on drawn images, and photos, and everything else and thus cross those influences a lot more than a human artist might. The eyes on the lego figures gave it away though that one also exhibits the discontinuity across edges with the woman's scarf.

I didn't do great. Dalli-3 is really good and I couldn't spot anything obvious in most of the pictures.

A couple were surprising but others seemed obvious in hindsight. Some of these AI models have a really specific vibe that is easy to spot. It can be removed sometimes but if the prompts don't prevent it, the images tend to have this glow and pop that many real images don't have. They're perfectly detailed if that makes sense.

Got 14/20, which I feel pretty good about, but you do this survey every year and it's gonna keep going lower. I bet even a year ago, most people would be above 75% accuracy.

[Survey] Can you tell which Surveys are AI generated??

The avocado had real text. Is Dall-E 3 capable of creating legible text?

Yes, it's the only model that manages to get text right, and the results are usually pretty consistent. It's a big step forward.

AI generated photo of a cat saying "I'm king of the world!"

Base SDXL and SD1.5 with the help of controlnet can both do text too. I forgot Deep Floyd/IF can as well.

Control nets are kind of "cheating", though, they're a form of image-to-image where you provide them with something to trace over or otherwise guide them. I think in this area the open-source field has (briefly) fallen behind, we'll need another round of catchup. That's fine, though. Let competition drive hard.

1 more...

I did well with the photograph ones, bad with the drawings, as I suspected.

11/20 but many times I wasn't sure, and most times I thought that this pic doesn't belong into such a survey, because it is too simple.

The survey question is more meaningful when it's about photorealistic images. Simple advertising graphics are meaningless either way.

Got 9/20 I'm ashamed

Same boat here. If less of them were artistic painting type things and instead photos of real life scenes, I know I could have gotten more of them correct. I knew which Lego one was ai because it was an imitation of a "real life" scene and not an attempt at a painting or sketch.

16/20. I would imagine I err’d on the side of assuming art was AI made rather than human made. AI generated photos are too ‘smooth’ looking.

Dang only 6/20. I’m upset I flip-flopped on a couple. Thought for sure the OP was trying to trick us ;)

No, it’s not a trick survey where ALL of them are AI.

I failed on

2,4,7,11,13,14,15,17,18

Only failed on 2 because I forgot about the existence of that site

I got 9/20, shoulda done better. 4-5 of those I was so sure I was right and still managed to get em wrong. The rest of the wrong uns were 50/50.

9/20

The landscape and cat pictures were just guesses.

12/20, I got all the photos and line art correct. The paintings and digital art were much harder to tell. Human artists aren't perfect so the usual tells don't really apply. I think I only nailed the line art because I've done a lot of line art myself.

Funny, with the line art, I struggled bit not the fotorealistc ones. For me the fotorealistic AI generations look like some smoothness filter was applied after taking the foto, or like heavily edited. With the landscapes and the first two cats I struggled as well, and the berserker fan art was like copy paste from the manga. Got 11/20.