ChatGPT wants to be a coding aid/reference material. A better baseline would be the top rated answer for the question on stackoverflow or whether the answer exists on the first 3 Google search results.
Or a textbook's explanation
Cause people arent looking at chatgpt as an accurate simple code generator, rather a junior dev.
Why would we compare it against an average coder?
ChatGPT wants to be a coding aid/reference material. A better baseline would be the top rated answer for the question on stackoverflow or whether the answer exists on the first 3 Google search results.
Or a textbook's explanation
Cause people arent looking at chatgpt as an accurate simple code generator, rather a junior dev.