ChatGPT gets code questions wrong 52% of the time

☆ Yσɠƚԋσʂ ☆@lemmy.ml to Technology@lemmy.ml – 196 points –
ChatGPT gets code questions wrong 52% of the time
theregister.com
66

You are viewing a single comment

It actually doesn't have to be. For example the way I use Github Copilot is I give it a code snippet to generate and if it's wrong I just write a bit more code and the it usually gets it right after 2-3 iterations and it still saves me time.

The trick is you should be able to quickly determine if the code is what you want which means you need to have a bit of experience under your belt, so AI is pretty useless if not actively harmful for junior devs.

Overall it's a good tool if you can get your company to shell out $20 a month for it, not sure if I'd pay it out of my own pocket tho.

It... it was a joke. I was implying that 52% was better than me.

Ah ok I guess I misread that. My point is that by itself it's not gonna help you write either better or shittier code than you already do.

GitHub Copilot is just intellisense that can complete longer code blocks.

I’ve found that it can somewhat regularly predict a couple lines of code that generally resemble what I was going to type, but it very rarely gives me correct completions. By a fairly wide margin, I end up needing to correct a piece or two. To your point, it can absolutely be detrimental to juniors or new learners by introducing bugs that are sometimes nastily subtle. I also find it getting in the way only a bit less frequently than it helps.

I do recommend that experienced developers give it a shot because it has been a helpful tool. But to be clear - it’s really only a tool that helps me type faster. By no means does it help me produce better code, and I don’t ever see it full on replacing developers like the doomsayers like to preach. That being said, I think it’s $20 well spent for a company in that it easily saves more than $20 worth of time from my salary each month.