Google AI making up recalls that didn’t happen

nifty@lemmy.world to Technology@lemmy.world – 1597 points –
214

You are viewing a single comment

Iirc cases where the central complaint is AI, ML, or other black box technology, the company in question was never held responsible because "We don't know how it works". The AI surge we're seeing now is likely a consequence of those decisions and the crypto crash.

I'd love CVS try to push a lawsuit though.

In Canada there was a company using an LLM chatbot who had to uphold a claim the bot had made to one of their customers. So there's precedence for forcing companies to take responsibility for what their LLMs says (at least if they're presenting it as trustworthy and representative)

This was with regards to Air Canada and its LLM that hallucinated a refund policy, which the company argued they did not have to honour because it wasn't their actual policy and the bot had invented it out of nothing.

An important side note is that one of the cited reasons that the Court ruled in favour of the customer is because the company did not disclose that the LLM wasn't the final say in its policy, and that a customer should confirm with a representative before acting upon the information. This meaning that the the legal argument wasn't "the LLM is responsible" but rather "the customer should be informed that the information may not be accurate".

I point this out because I'm not so sure CVS would have a clear cut case based on the Air Canada ruling, because I'd be surprised if Google didn't have some legalese somewhere stating that they aren't liable for what the LLM says.

But those end up being the same in practice. If you have to put up a disclaimer that the info might be wrong, then who would use it? I can get the wrong answer or unverified heresay anywhere. The whole point of contacting the company is to get the right answer; or at least one the company is forced to stick to.

This isn’t just minor AI growing pains, this is a fundamental problem with the technology that causes it to essentially be useless for the use case of “answering questions”.

They can slap as many disclaimers as they want on this shit; but if it just hallucinates policies and incorrect answers it will just end up being one more thing people hammer 0 to skip past or scroll past to talk to a human or find the right answer.

But it has to be clearly presented. Consumer law and defamation law has different requirements on disclaimers

"We don't know how it works but released it anyway" is a perfectly good reason to be sued when you release a product that causes harm.

I would love if lawsuits brought the shit that is ai down. It has a few uses to be sure but overall it's crap for 90+% of what it's used for.

iirc alaska airlines had to pay

That was their own AI. If CVS' AI claimed a recall, it could be a problem.

So will the google AI be held responsible for defaming CVS?

Spoiler alert- they won’t.

The crypto crash? Idk if you've looked at Crypto recently lmao

Current froth doesn't erase the previous crash. It's clearly just a tulip bulb. Even tulip bulbs were able to be traded as currency for houses and large purchases during tulip mania. How much does a great tulip bulb cost now?

4 more...
4 more...
4 more...