A law firm was fined $5,000 after one of its lawyers used ChatGPT to write a court brief riddled with fake case references

BotIt@lemmy.world to Technology@kbin.social – 58 points – A law firm was fined $5,000 after one of its lawyers used ChatGPT to write a court brief riddled with fake case references
businessinsider.com

The lawyer said that he had "no idea" ChatGPT could fabricate information and that he "deeply" regretted his decision.

13

I feel like a good lawyer should do a tiny bit of research before letting AI write a court brief. At least enough to read the 2 warnings on the ChatGPT site about how it can generate inaccurate information.

Or, they've done it before and gotten away with it.

It's impressive how well ChatGPT hallucinates citations.

I was asking it about a field of law I happen to be quite aware of (as a layman), and it came up with entire sections of laws that didn't exist to support its conclusions.

Large Language Models like ChatGPT are in my view verisimilitude engines. Verisimilitude is the appearance of being true or real. You'll note, however, that it is not being true or real, simply appearing so.

It's trying to make an answer that looks right. If it happens to know the actual answer then that's what it'll go with, but if it doesn't, it'll go with what a correct answer might statistically look like. For fields with actual right and wrong answers like law and science and technology, its tendency to make things up is really harmful if the person using the tool doesn't know it will lie.

$5k is a horseshit low fine for this.

Yeah, this seems like a massive ethics violation that should result in contempt, or referral for disbarment.

I'm imagining their lawyer pulling out their wallet at the hearing and thumbing through their singles to pay the fine.

$5k is absolutely nothing for a typical law firm, lunch money

It's wild how negligent the lawyer was. How did he manage to pass the bar with that level of work ethic?
You would think that any reasonable person having someone else do their work for them would at least have to good sense to look it over, let alone letting an AI do it for you and not fact checking it at all. What an utter nincompoop.

There should be a law that AI answers have a cryptic word hidden in plain sight to reveal itself if put to review. Something weird that only AI will pick up.

The issue here is that most people will give it at least cursory read throughs, to make sure it passes the sniff test.

The reality is it’s not the ai that’s submitting it. The human is blindly cutting and pasting, but the moment you add “this text was generated by ai”… literally or just as tags… then they’re going to start clipping or adjusting it

I saw a YouTube video (sorry can't find the link now) on using hidden weighting to allow exam markers to detect ai. It was things like words with many synonyms always picking the third most popular or somthing so that over a 3 page essay if you happened to use the lightly off weighting on word choice the anti cheating software would pick up on it. The chances of a human weighing themselves that way would be rare as they would tend to work to one of its patterns where as AI was forced to use a few different patterns but in a deterministic way.

Over time, wouldn't you expect as people see more examples of text generated by these systems, that the general usage of the "third most popular" synonym would eventually eclipse the second or first? If the ranking of the synonyms were based solely on written texts, proliferation of generated texts with weighted word choices would also skew usage.