The study said 86.66% of the generated software systems were "executed flawlessly."
But...
Nevertheless, the study isn't perfect: Researchers identified limitations, such as errors and biases in the language models, that could cause issues in the creation of software. Still, the researchers said the findings "may potentially help junior programmers or engineers in the real world" down the line.
So… they failed 13.34% of their own unit tests?
That’s a B+! Fire all our engineers immediately.
some tech CEO, somewhere
Better than CyberPunk at release.
🎵🎵 99 little bugs in the code, 99 bugs in the code, Fix one bug, compile it again, 101 little bugs in the code. 101 little bugs in the code, 101 bugs in the code, Fix one bug, compile it again, 103 little bugs in the code. 🎵🎵
But...
So… they failed 13.34% of their own unit tests?
That’s a B+! Fire all our engineers immediately.
Better than CyberPunk at release.
🎵🎵 99 little bugs in the code, 99 bugs in the code, Fix one bug, compile it again, 101 little bugs in the code. 101 little bugs in the code, 101 bugs in the code, Fix one bug, compile it again, 103 little bugs in the code. 🎵🎵