A courts reporter wrote about a few trials. Then an AI decided he was actually the culprit.
When German journalist Martin Bernklautyped his name and location into Microsoft’s Copilot to see how his articles would be picked up by the chatbot, the answers horrified him. Copilot’s results asserted that Bernklau was an escapee from a psychiatric institution, a convicted child abuser, and a conman preying on widowers. For years, Bernklau had served as a courts reporter and the AI chatbot had falsely blamed him for the crimes whose trials he had covered.
The accusations against Bernklau weren’t true, of course, and are examples of generative AI’s “hallucinations.” These are inaccurate or nonsensical responses to a prompt provided by the user, and they’re alarmingly common. Anyone attempting to use AI should always proceed with great caution, because information from such systems needs validation and verification by humans before it can be trusted.
But why did Copilot hallucinate these terrible and false accusations?
I don't think they should be liable for what their text generator generates. I think people should stop treating it like gospel. At most, they should be liable for misrepresenting what it can do.
If these companies are marketing their AI as being able to provide "answers" to your questions they should be liable for any libel they produce.
If they market it as "come have our letter generator give you statistically associated collections of letters to your prompt" then I guess they're in the clear.
It's like that aeroplane company who had a chatbot serve answers, and then tried to weasel out of it when the chatbot informed the customer about a refund policy that didn't actually exist.
If they're presenting it as an authoritative source of information, then they should be held to the standard they claim.
So you don’t think these massive megacompanies should be held responsible for making disinformation machines? Why not?
because when you provide computer code for money you don't want there to be any liability assigned
Which is why, in many cases, there should be liability assigned. If a self-driving car kills someone, the programming of the car is at least partially to blame, and the company that made it should be liable for the wrongful death suit, and probably for criminal charges as well. Citizens United already determined that corporations are people....now we just need to put a corporation in prison for their crimes.
I want them to have more warnings and disclaimers than a pack of cigarettes. Make sure the users are very much aware they can't trust anything it says.
If they aren’t liable for what their product does, who is? And do you think they’ll be incentivized to fix their glorified chat boxes if they know they won’t be held responsible for if?
Their product doesn't claim to be a source of facts. It's a generator of human-sounding text. It's great for that purpose and they're not liable for people misusing it or not understanding what it does.
So you think these companies should have no liability for the misinformation they spit out. Awesome. That’s gonna end well. Welcome to digital snake oil, y’all.
I did not say companies should have no liability for publishing misinformation. Of course if someone uses AI to generate misinformation and tries to pass it off as factual information they should be held accountable. But it doesn't seem like anyone did that in this case. Just a journalist putting his name in the AI to see what it generates. Nobody actually spread those results as fact.
If we’ve learned any lesson from the internet, it’s that once something exists it never goes away.
Sure, people shouldn’t believe the output of their prompt. But if you’re generating that output, a site can use the API to generate a similar output for a similar request. A bot can generate it and post it to social media.
Yeah, don’t trust the first source you see. But if the search results are slowly being colonized by AI slop, it gets to a point where the signal-to-noise ratio is so poor it stops making sense to only blame the poor discernment of those trying to find the signal.