Over just a few months, ChatGPT went from correctly answering a simple math problem 98% of the time to just 2%, study finds

leninmummy@lemmy.ml to Technology@lemmy.ml – 430 points –
Over just a few months, ChatGPT went from correctly answering a simple math problem 98% of the time to just 2%, study finds
finance.yahoo.com
71

You are viewing a single comment

It can probably still write boilerplate code, but I wouldn't currently trust it for algorithmic design.

I've tried to use it for debugging by copying code into it, and it gives me the same code back as the corrected version. I was wondering why it's been getting worse

My guess is they've been trying to make it cheaper by decreasing the amount of time it spends on each response or by decreasing the amount of computing power that goes into the instance you're speaking to. Coding and math are products of high-level cognition and arise emergently out of neural networks that are very sophisticated, but take just a bit of power out and the abilities degenerate rapidly.

I also experienced this issue last week. I asked for a specific correction and got unchanged code back. Sometimes it does update, though. Maybe like 50-70% of requests.