AI is actually the king of bullshit. It might give you some code, but will it compile? Probably about as well as an AI art knee would actually bear your weight.
And the author clearly has no idea what she’s talking about, or the impact of AI on CS.
I use chatgpt regularly to build outlines and boilerplate to code I want to write. Yes it’s code I can’t trust and almost entirely rewrite, but simply the act of it naming the variables saves me time.
And even if it did get to the point of good code, you have to be a developer to know what it created is what you wanted in the first place.
This is fearmongering targeting tech people by claiming their jobs are at the same level of risk of disruption by ai as other white collar jobs (like hers!).
We are nowhere near AI writing our software unattended. Not even close. People really over estimate the state of AI.
I'm an AI nerd and yes, nowhere close. AI can write code snippets pretty well, and that'll get better with time, but a huge part of software development is translating client demands into something sane and actionable. If a CEO of a 1-man billion dollar company asks his super-AI to "build the next Twitter", that leaves so many questions on the table that the result will be completely unpredictable. Humans have preferences and experiences which can inform and fill in those implicit questions. They're generally much better suited as tools and copilots than autonomous entities.
Now, there was a paper that instantiated a couple dozen LLMs and had them run a virtual software dev company together which got pretty good results, but I wouldn't trust that without a lot more research. I've found individual LLMs with a given task tend to get tunnel vision, so they could easily get stuck in a loop trying the same wrong code or design repeatedly.
Now, there was a paper that instantiated a couple dozen LLMs and had them run a virtual software dev company together which got pretty good results
Dude, you need to take a closer look at that paper you linked, if you consider that "pretty good results". They have a github repo with screenshots of some of the "products", which should give you some idea https://github.com/OpenBMB/ChatDev/tree/main/misc .
Not to mention the terrible decision making of the fake company (desktop app you have to download? no web/mobile version? for a virtual board game?)
(Also the paper never even tried to prove its main hypothesis, that all this multi agent song and dance would somehow reduce hallucinations and improve performance. There is a lot of good AI stuff coming out daily, but that particular paper - and the articles reporting on it - was pure garbage.)
True, as of today. On the other hand, future advancements could very easily change that. On the other other hand, people have been saying the same about self driving cars 10 years ago, and while they do basically work, and are coming eventually, progress there has been a lot slower than predicted.
So who knows. Could go either way.
It’s almost a philosophical question of whether I can replace us though. Because for it to be anything more than a tool it needs real intelligence, compassion, etc. Basically it would need a conscious.
I’m certain it’ll replace some jobs without that, just because being a tool it’ll make us more efficient and that efficiency will eliminate jobs. But I’m not seeing it replace or assimilate entire industries at this stage.
Yea…anyone who has asked chatgpt to help them fix a piece of code or write one would know it requires a lot of human editing/good prompting. And a lot of time, what I was trying to accomplish still wouldn’t work.
True. But if AI makes people more productive it could make it really hard to find work. Especially if you're straight out of college with zero experience.
finding job as a junior is already a bit harder than it was because so many developers are working remote, which is way harder to do when you are a junior developer.
You can't write this kind of thing if you understand what a programmer does. The biggest part of the job is finding a good way to break down a problem into executable steps, not just actually writing the code.
Executable and maintainable
AI generated code can’t, as of yet, go in and fix a bug
software developers with access to GitHub’s Copilot chatbot were able to finish a coding task 56 percent faster than those who did it solo
Are these competent developers, or the kind who already take 4 or 5 times longer to do a task than their peers?
Part of the problem with telling people "learn to code" is that a lot of them are bad at it. There may be some diamonds in the rough, but there is a lot of rough out there.
Author seems to think that starting salary for developers working for Google is representative as well. The average computer science graduate does not get a job at Google.
People who learn to code because it means job security are not the ones we look to hire. We look for people who are passionate about it, whose interest in the subject is deeper than skin deep.
Not looking for people who live and breathe code, but you need to like to solve problems and like to learn new things.
We look for people who are passionate about it, whose interest in the subject is deeper than skin deep.
Doesn't it hurt those people a lot more when their project nearly inevitably gets shut down?
I'm still bitter about the project I worked on that got killed at my company three years ago.
Where I work we haven’t really shut down any projects in the last six years.
We’ve had some smaller projects which got parked due to shifting priorities, but other than that we’ve shipped everything else.
But inevitably, over a career in software there will be projects that don’t make it to production for one reason or another.
Personally I’m very pragmatic about it, but I know people who get very attached to the code they write.
I’m the kind of guy that is passionate about what I’m doing when I’m doing it, not necessarily for all eternity. I’ve written stuff that I’d be more than happy for someone to come and replace, but the thing about revenue generating systems (most people say “legacy”, but I prefer this term) is that they aren’t always easy to replace.
I know we’re not all wired that way, and some people find it harder to see an older system get retired. A consultant I use is more attached to my code than I am, for instance.
I am a very competent developer. Copilot makes me a lot faster with net new code and tests because a lot of that stuff is very close to boilerplate so Copilot can build 95% of it for me. Declarative stuff like HCL is so much faster. Copilot doesn’t necessarily speed me up for things like bug fixes because a lot of that is code reading. Refactoring? Hell yeah. Way faster.
Here’s the study. If you look at the actual prompt (near the end), it’s exactly the kind of thing Copilot kicks ass at: something that’s super fucking common all over GitHub (a toy JavaScript server). I really don’t think my job is in jeopardy yet.
Are these competent developers
Here's an example - a few minutes ago I wrote this line of code:
// date
... and a split second later copilot auto completed exactly the seven lines of code that I would have typed. I read the code, tested it, and moved on to the next block of code.
Yes, I could have written those seven lines. They were pretty basic - read a value from the database, transform it to a string human form, and send that the user. CoPilot types a lot faster than me (words per second, instead of per minute) and it makes fewer typos.
It's also more familiar than I am with all the major libraries. I find I'm spending a lot less time reading documentation or searching the web these days.
But the real kicker... I work on a small team. My project is full of code that I didn't write and it isn't as well documented as I'd like it to be. It's also not publicly documented, so I can't use Google or Stack Overflow to find answers. CoPilot has indexed the project, and it knows how to read the date from the database. It also knows what human readable date string format has been used elsewhere in the user interface.
This feels like a more cogent opinion piece on AI's potential impact on programming:
This is a much better article. OP's article just shows the author's surface understanding of how coding works and how well an LLM can actually code. There's way more that goes into a programming task than just coding.
I see LLMs as having the potential of being almost like a super library. I can prompt GPT, Claude, etc. to write me a custom function that I copy, paste, test, scrutinize, and almost certainly change. It's a tool that will make someone a more productive programmer. It won't completely subsume a human's ability to be creative and put the pieces together.
At the absolute worst over the next decade, I could see programming changing from writing and debugging code to prompting, stitching together, and debugging.
It's the same with CAM software in CNC, like sure, If you set it up right (which is a skill in and of itself) it can spit out a decent toolpath, but there's tons of magic to be done by hand and understanding the way the underlying G code works allows you to make small changes on the fly.
Stakeholders struggle to give accurate requirements most of the time, they're not gonna be programming with ChatGPT soon. AI can really improve a good developer's output though.
Haven't found a usecase for it yet where it doesn't shit out gift wrapped garbage
You have to give it very specific instructions and small, targeted things to do. I've used it to write a lot of terraform, I hate writing IaC.
Holy shit the speed increase with HCL is fucking nuts
The article discusses how the rise of AI may impact computer science careers going forward. While coding jobs have long been seen as stable career paths, chatbots can now generate code in various languages. Developers are using AI tools like Copilot to accelerate routine coding tasks. Within a decade, coding bots may be able to do much more than basic tasks. However, programmers will still be needed to guide AI toward productive solutions. Teaching coding is also becoming more challenging, as students could use chatbots to cheat. Conceptual problem-solving skills will remain important for programmers to apply their expertise where AI falls short. The future may belong to those who can think entrepreneurially about how technology solves problems.
In the end, what students study may matter less than their ability to apply knowledge to technology challenges.
This comment was generated by a bot. Send comments and complaints via private message.
:::
However, programmers will still be needed to guide AI toward productive solutions
So it would still be safe, they'd just be doing different work from what they do now. Same as how other advances in tech stacks made it so we do things differently now than 30 years ago.
People are very adaptable
Indeed. Do people still use emacs to code, for example?
Technologies evolve. People coding today in COBOL or Fortran are few and far between (but very well compensated).
Yeah sure, i use Emacs to code. In evil mode. A lot of Emacs users use it to code. Why would you think otherwise?
No, all the cool kids use Vim.
Do people still use emacs to code, for example?
Sure. Why wouldn't they?
Hell yea we do use emacs!
Indeed. Do people still use emacs to code, for example?
Not sure if that's a serious question. Yes. They do. And many use it effectively. I use (neo)vim though because it works for me
Yes, that's the key. I haven't written assembly code since the 1990s, I use higher-level abstractions to get to the goal more quickly now. AI-generated code is just yet another layer of abstraction away from machine language.
lol
I just want to mention the clever graphic design of the Illustration by Ben Kothe
It's a multilayered visual pun. A visual punion, if you will.
There is still going to be a need for programmers and don't think "AI" is going to be on its own anytime soon. Even using it as an assistant you need to know what you want and have some understanding of the code. Feel like most of what we see of "AI" right now is just propaganda so investors will throw money at them.
AI is actually the king of bullshit. It might give you some code, but will it compile? Probably about as well as an AI art knee would actually bear your weight.
And the author clearly has no idea what she’s talking about, or the impact of AI on CS.
I use chatgpt regularly to build outlines and boilerplate to code I want to write. Yes it’s code I can’t trust and almost entirely rewrite, but simply the act of it naming the variables saves me time.
And even if it did get to the point of good code, you have to be a developer to know what it created is what you wanted in the first place.
This is fearmongering targeting tech people by claiming their jobs are at the same level of risk of disruption by ai as other white collar jobs (like hers!).
We are nowhere near AI writing our software unattended. Not even close. People really over estimate the state of AI.
I'm an AI nerd and yes, nowhere close. AI can write code snippets pretty well, and that'll get better with time, but a huge part of software development is translating client demands into something sane and actionable. If a CEO of a 1-man billion dollar company asks his super-AI to "build the next Twitter", that leaves so many questions on the table that the result will be completely unpredictable. Humans have preferences and experiences which can inform and fill in those implicit questions. They're generally much better suited as tools and copilots than autonomous entities.
Now, there was a paper that instantiated a couple dozen LLMs and had them run a virtual software dev company together which got pretty good results, but I wouldn't trust that without a lot more research. I've found individual LLMs with a given task tend to get tunnel vision, so they could easily get stuck in a loop trying the same wrong code or design repeatedly.
(I think this was the paper, reminiscent of the generative agent simulacra paper, but I also found this)
Dude, you need to take a closer look at that paper you linked, if you consider that "pretty good results". They have a github repo with screenshots of some of the "products", which should give you some idea https://github.com/OpenBMB/ChatDev/tree/main/misc .
Not to mention the terrible decision making of the fake company (desktop app you have to download? no web/mobile version? for a virtual board game?)
(Also the paper never even tried to prove its main hypothesis, that all this multi agent song and dance would somehow reduce hallucinations and improve performance. There is a lot of good AI stuff coming out daily, but that particular paper - and the articles reporting on it - was pure garbage.)
True, as of today. On the other hand, future advancements could very easily change that. On the other other hand, people have been saying the same about self driving cars 10 years ago, and while they do basically work, and are coming eventually, progress there has been a lot slower than predicted.
So who knows. Could go either way.
It’s almost a philosophical question of whether I can replace us though. Because for it to be anything more than a tool it needs real intelligence, compassion, etc. Basically it would need a conscious.
I’m certain it’ll replace some jobs without that, just because being a tool it’ll make us more efficient and that efficiency will eliminate jobs. But I’m not seeing it replace or assimilate entire industries at this stage.
Yea…anyone who has asked chatgpt to help them fix a piece of code or write one would know it requires a lot of human editing/good prompting. And a lot of time, what I was trying to accomplish still wouldn’t work.
True. But if AI makes people more productive it could make it really hard to find work. Especially if you're straight out of college with zero experience.
finding job as a junior is already a bit harder than it was because so many developers are working remote, which is way harder to do when you are a junior developer.
You can't write this kind of thing if you understand what a programmer does. The biggest part of the job is finding a good way to break down a problem into executable steps, not just actually writing the code.
Executable and maintainable
AI generated code can’t, as of yet, go in and fix a bug
Are these competent developers, or the kind who already take 4 or 5 times longer to do a task than their peers?
Part of the problem with telling people "learn to code" is that a lot of them are bad at it. There may be some diamonds in the rough, but there is a lot of rough out there.
Author seems to think that starting salary for developers working for Google is representative as well. The average computer science graduate does not get a job at Google.
People who learn to code because it means job security are not the ones we look to hire. We look for people who are passionate about it, whose interest in the subject is deeper than skin deep.
Not looking for people who live and breathe code, but you need to like to solve problems and like to learn new things.
Doesn't it hurt those people a lot more when their project nearly inevitably gets shut down?
I'm still bitter about the project I worked on that got killed at my company three years ago.
Where I work we haven’t really shut down any projects in the last six years.
We’ve had some smaller projects which got parked due to shifting priorities, but other than that we’ve shipped everything else.
But inevitably, over a career in software there will be projects that don’t make it to production for one reason or another.
Personally I’m very pragmatic about it, but I know people who get very attached to the code they write.
I’m the kind of guy that is passionate about what I’m doing when I’m doing it, not necessarily for all eternity. I’ve written stuff that I’d be more than happy for someone to come and replace, but the thing about revenue generating systems (most people say “legacy”, but I prefer this term) is that they aren’t always easy to replace.
I know we’re not all wired that way, and some people find it harder to see an older system get retired. A consultant I use is more attached to my code than I am, for instance.
I am a very competent developer. Copilot makes me a lot faster with net new code and tests because a lot of that stuff is very close to boilerplate so Copilot can build 95% of it for me. Declarative stuff like HCL is so much faster. Copilot doesn’t necessarily speed me up for things like bug fixes because a lot of that is code reading. Refactoring? Hell yeah. Way faster.
Here’s the study. If you look at the actual prompt (near the end), it’s exactly the kind of thing Copilot kicks ass at: something that’s super fucking common all over GitHub (a toy JavaScript server). I really don’t think my job is in jeopardy yet.
Here's an example - a few minutes ago I wrote this line of code:
... and a split second later copilot auto completed exactly the seven lines of code that I would have typed. I read the code, tested it, and moved on to the next block of code.
Yes, I could have written those seven lines. They were pretty basic - read a value from the database, transform it to a string human form, and send that the user. CoPilot types a lot faster than me (words per second, instead of per minute) and it makes fewer typos.
It's also more familiar than I am with all the major libraries. I find I'm spending a lot less time reading documentation or searching the web these days.
But the real kicker... I work on a small team. My project is full of code that I didn't write and it isn't as well documented as I'd like it to be. It's also not publicly documented, so I can't use Google or Stack Overflow to find answers. CoPilot has indexed the project, and it knows how to read the date from the database. It also knows what human readable date string format has been used elsewhere in the user interface.
This feels like a more cogent opinion piece on AI's potential impact on programming:
ChatGPT Isn't Coming for Your Coding Job
AI is coming for your shitty "journalism" jobs writing about AI taking your jobs.
This is a much better article. OP's article just shows the author's surface understanding of how coding works and how well an LLM can actually code. There's way more that goes into a programming task than just coding.
I see LLMs as having the potential of being almost like a super library. I can prompt GPT, Claude, etc. to write me a custom function that I copy, paste, test, scrutinize, and almost certainly change. It's a tool that will make someone a more productive programmer. It won't completely subsume a human's ability to be creative and put the pieces together.
At the absolute worst over the next decade, I could see programming changing from writing and debugging code to prompting, stitching together, and debugging.
It's the same with CAM software in CNC, like sure, If you set it up right (which is a skill in and of itself) it can spit out a decent toolpath, but there's tons of magic to be done by hand and understanding the way the underlying G code works allows you to make small changes on the fly.
Stakeholders struggle to give accurate requirements most of the time, they're not gonna be programming with ChatGPT soon. AI can really improve a good developer's output though.
Haven't found a usecase for it yet where it doesn't shit out gift wrapped garbage
You have to give it very specific instructions and small, targeted things to do. I've used it to write a lot of terraform, I hate writing IaC.
Holy shit the speed increase with HCL is fucking nuts
::: spoiler TL;DR for the linked article
This comment was generated by a bot. Send comments and complaints via private message. :::
So it would still be safe, they'd just be doing different work from what they do now. Same as how other advances in tech stacks made it so we do things differently now than 30 years ago.
People are very adaptable
Indeed. Do people still use emacs to code, for example?
Technologies evolve. People coding today in COBOL or Fortran are few and far between (but very well compensated).
Umm. Yes.
Yeah sure, i use Emacs to code. In evil mode. A lot of Emacs users use it to code. Why would you think otherwise?
No, all the cool kids use Vim.
Sure. Why wouldn't they?
Hell yea we do use emacs!
Not sure if that's a serious question. Yes. They do. And many use it effectively. I use (neo)vim though because it works for me
Yes, that's the key. I haven't written assembly code since the 1990s, I use higher-level abstractions to get to the goal more quickly now. AI-generated code is just yet another layer of abstraction away from machine language.
lol
I just want to mention the clever graphic design of the Illustration by Ben Kothe
It's a multilayered visual pun. A visual punion, if you will.
There is still going to be a need for programmers and don't think "AI" is going to be on its own anytime soon. Even using it as an assistant you need to know what you want and have some understanding of the code. Feel like most of what we see of "AI" right now is just propaganda so investors will throw money at them.