Wow, the text generator that doesn't actually understand what it's "writing" is making mistakes? Who could have seen that coming?
I once asked one to write a basic 50-line Python program (just to flesh things out), and it made so many basic errors that any first-year CS student could catch. Nobody should trust LLMs with anything related to security, FFS.
Nobody should trust LLMs with anything
ftfy
also any inputs are probably scrapped and used for training, and none of these people get GDPR
also any inputs are probably scraped
ftfy
Let's hope it's the bad outputs that are scrapped. <3
Eh, I'd say mostly.
I have one right now that looks at data and says "Hey, this is weird, here are related things that are different when this weird thing happened. Seems like that may be the cause."
Which is pretty well within what they are good at, especially if you are doing the training yourself.
you are part of the problem
That is about the most generic statement possible, with nearly zero knowledge of what I'm doing on yours.
So... What problem? Feel free to enlighten me.
I wish we could say the students will figure it out, but I've had interns ask for help and then I've watched them try to solve problems by repeatedly asking ChatGPT. It's the scariest thing - "Ok, let's try to think about this problem for a moment before we - ok, you're asking ChatGPT to think for a moment. FFS."
Critical thinking is not being taught anymore.
Has critical thinking ever been taught? Feel like it’s just something you have or you don’t.
Critical thinking is essentially learning to ask good questions and also caring enough to follow the threads you find.
For example, if mental health is to blame for school shootings then what is causing the mental health crisis and are we ensuring that everyone has affordable access to mental healthcare? Okay, we have a list of factors that adversely impact mental health, what can we do to address each one? Etc.
Critical thinking isn't hard, it just takes time, effort.
I have the impression that most people (or maybe it's my faith in Humanity that's at an all time low and it's really just "some people") just want pre-chewed explanations given to them rather than spend time and energy figuring things out themselves - basically baby pap as ideas food rather than cooking their own ideas food out of raw ingredients.
Certainly that would help explain the resurgence of Populist sloganeering and continued popularity of Religion (with it's ever popular simple explanations of "Deity did it" and "it's the will of Deity")
In my mind, it's really just entitlement. Something along the lines of, "well, I don't know the answer and why should I have to know if someone else is going to figure it out."
In a tired way, I understand it. Everyday I just want some of my time back for myself. If I'm always the one who has to work through all the problems for my ideas just to be ignored then I'm just going to be perpetually frustrated. So if my ideas are half baked and the solutions i barf up aren't to your liking, well, figure it out yourself.
Not to say that I am this way. I don't get frustrated when my ideas are ignored. I do get frustrated, though, when others eat up half baked ideas knowing they are just that.
Sorry, if what I've wrote so far has gotten a bit confusing. I'll wrap it up and say, it's entitlement. People don't want to think for themselves because it's time consuming. They think the world should order itself in a way that fulfills their needs with minimal effort on their part. Except, to understand how the world would be ordered for that to be reality, they can't comprehend because no one has really figured that one out. So they fall back on god and gods an easy out because, duh, he's god.
Critical thinking, especially Skepticism, does not make for good Consumers or mindless followers of Political Tribes.
British primary schools used to have something called 'problem solving' it was usually a simple maths problem described in words that required some degree of critical thinking to solve. e.g. A frog is at the bottom of a 30m well, it climbs 7m each day but in the night it slides 3m back down in its sleep. You can't just calculate 30/(7-3) because it doesn't account for the day the frog gets over the top and thus doesn't slide back down in its sleep.
Not the most complex problem but pretty good for kids under 10 to start getting the basics.
I had a chat w/ my sibling about the future of various careers, and my argument was basically that I wouldn't recommend CS to new students. There was a huge need for SW engineers a few years ago, so everyone and their dog seems to be jumping on the bandwagon, and the quality of the applicants I've had has been absolutely terrible. It used to be that you could land a decent SW job without having much skill (basically a pulse and a basic understanding of scripting), but I think that time has passed.
I absolutely think SW engineering is going to be a great career long-term, I just can't encourage everyone to do it because the expectations for ability are going to go up as AI gets better. If you're passionate about it, you're going to ignore whatever I say anyway, and you'll succeed. But if my recommendation changes your mind, then you probably aren't passionate enough about it to succeed in a world where AI can write somewhat passable code and will keep getting (slowly) better.
I'm not worried at all about my job or anyone on my team, I'm worried for the next batch of CS grads who chatGPT'd their way through their degree. "Cs get degrees" isn't going to land you a job anymore, passion about the subject matter will.
Outsourcing killed a lot of the junior and even mid-level career level opportunities in CS and AI seems on track to do the same.
The downside is that going into CS now (and having gone into CS in the last decade or so, especially in English-speaking countries) was basically the career equivalent of just out of the starting line running full speed into a brick wall.
The upside is that for anybody who now is a senior techie things have never been this good because there are significantly fewer people at that level than there is need for such people, since in the last decade or so a lot of people haven't had the chance to progress in their careers to that point.
Whilst personally this benefits me, I'm totally against this shit and what it has done to the kids entering my career.
Yup, and that's why I'll discourage people from entering my career, not because it's a bad gig and it's going away, but because the bar for competency is about to go up. Do it if you're passionate and you'll probably do well for yourself, but don't do it if you're just looking for a good job. If you just want a good job, go into nursing, accounting, or the trades.
I think it's even worse than just the bar for competency going up: even for a coding wizard going into the career, it's a lot harder to squeeze through the bottleneck which is getting an entry level position nowadays unless they have some public proof out on the Net of how good they're at coding (say, commits in open source projects, your own public projects, or even Youtube videos about it).
This is something that will negativelly impact perfectly capable young developers who have an introvert personality type (which are most of them in my experience, even in domains such as Hacking) since some of the upsides of Introversion are a greater capacity for really focusing on on things and for detailed analysis - both things that make for the best programmers - and self publicising isn't a part of the required skillset for good developers (though sooner or later the best ones will have to learn some "image management" if they end up in the Corporate world)
I'm a bit torn on this since on one side salesmanship being more of a criteria determining one's chances of getting a break at the start of one's career as a developer is bad news (good coding and good salesmanship tend to be inverselly correlated) but on the other side a junior developer with some experience actually working with other people on real projects with real users (because they contributed to existing open source projects) has already started learning what we have to teach fresh-out-of-Uni developers to make them professionals.
it’s a lot harder to squeeze through the bottleneck
Eh, I think that's overblown. As someone involved in hiring, we go through a ton of crappy candidates before finding someone half-decent, and when we see someone who actually knows what they're doing, we rush them through the process. The problem is that we're not a big tech company, we're in manufacturing, but we do interesting things w/ software. So getting on at one of the big tech companies may be challenging, but if you broaden the scope a little, there are tons of jobs waiting. We've had junior positions open for months because the hiring pool is so trash, but when we see a good candidate, we can get an offer to them by the end of the week.
We don't care too much about broader visibility (though I will look at your code if you provide a link), we expect competency on our relatively simple coding challenges, as well as a host of technical questions. We also don't mind hiring immigrants, we've sponsored a number of immigrants on our team.
introversion
As an introvert myself, I totally get it. I got my job because a recruiter reached out to me, not because I was particularly good at following up with applications. And that's why I tend to tell people to not get into CS. I encourage them to take CS classes if they're offered, but not to make it a career choice, and this is for two reasons:
manage expectations of the future of CS - junior jobs are likely to contract a bit w/ AI
thin the field so it's easier to find the good candidates - we have to go through 5-10 candidates before we find someone we like
I see. That does change the idea I had about things a bit.
It's been a while since I was last hiring.
I wasn't aware that the problem nowadays in the West (or at least the US) was an excess of people who don't really have a natural skill for it choosing software development as a career.
That kind of thing was one of the main problems with outsourcing to India maybe a decade ago: the profession was comparatively very well paid for the country so it attracted far too many people without the right skills resulting in a really low average quality of the programmers there - India had really good programmers just like everywhere else but then had a ton of people also working as programmers who should never had gone into it, so the experience of those having to deal with outsourced programming in India usually was pretty bad (I remotelly was a technical lead for a small outsourced team in India from London, and they were really bad whilst, curiously, the good programmers from the Indian Subcontinent I worked with had emigrated from there and were working in London and New York).
Yeah, there was a huge spike in demand for software engineers in the mid-2010s or so, and a massive explosion during COVID, so a lot of less qualified people were handed jobs as long as they could write very basic Python/JavaScript. But after the drop in tech demand after COVID, companies realized they overhired, so they did a ton of layoffs, mostly shucking those low-skilled employees (at least in the first round or two).
Maybe it's different in my area, but we have a ton of applicants whose only dev experience is a bootcamp program, and they apply for a full-time position when many aren't even qualified for a part-time intern position. They would utterly fail to answer most of our questions, and not make any progress on our (relatively simple) programming challenges. We have a mix of theory (i.e. OO principles, ACID, etc) and practical questions (e.g. concurrency vs parallelism in JavaScript or Python, duck typing, etc). CS grads with little practical experience would fly through the theory, but fail on the practical questions. Bootcamp "grads" would fail miserably at theory, and maybe get half of the practical questions, but then fail on the challenge. Dedicated hobbyists and passionate CS/bootcamp grads would do okay at theory (esp. if we change terminology) and fly through the practical questions and coding challenge.
It's pretty easy to weed a lot of those out in our first round, but the sheer volume of terrible applicants makes hiring super time-consuming. I mostly do second round interviews now (my boss, the director, has the thankless first round job), but I've done my fair share of first round interviews as well. I was involved in interviews almost 10 years ago, and I remember there being a lot fewer bad applicants then (i.e. of 5 applicants, 2 were hirable; now it's like 1 in 10, on a good week).
To be fair, I don't work for a tech company, so we're not really anyone's first pick. My company manufacturers things, and our software wing is pretty new and IMO really interesting (we do a lot of complex modeling), but it's not a place most would think to apply for, which is probably why we attract more desperate people. Then again, my last company was similar and much less visible (had something like 30-40 employees, current company has hundreds locally and thousands globally), yet we got better applicants. The main difference here is time, the CS programs at local universities have a ton more enrollment (some are actually turning away people now), whereas when I went it wasn't very popular, and I don't recall bootcamps really being a thing.
As for India, they have a lot of great talent and their IT/programming programs are super competitive (i.e. something like 5% of applicants get in, and only 40-60% of grads get jobs). However, the common thread I've seen is that Indian developers are very reliant on requirements, and they'll build pretty much exactly what you specify (i.e. if something seems off, they won't raise concerns). A lot of this is cultural, and I've heard horror stories from my Indian coworkers about managers telling them to do ridiculous things because of what a client said instead of pushing back. For example, my coworker spent well over a week re-implementing a standard Android behavior because the client specified something slightly outside what was possible through standard APIs, but probably would've preferred the 5 min solution. An American dev would just ask the customer and probably end up doing the 5-minute solution. Maybe these are isolated incidents, but I've seen similar behavior in different projects. If you know this upfront, you can get a good result with regular checkins and small adjustments as you go, but a lot of people don't understand that.
As it stands, since we can't hire good devs here in the US, we've been forced to hire outside firms to fill our ranks. We now have a team in Europe and another in India because we can't find the proper talent here, and not for lack of trying. And I don't think our expectations are out of whack, I just think the good devs already have good jobs and the less qualified devs are the ones getting laid off. We have hired a few good devs in the last few years (not rockstars, just solid devs), so it's not like everyone who lost their jobs aren't qualified, there just seems to be a lot of unqualified people.
Thanks for that very complete view of things.
Things are quite different since I last was doing hiring, which was pre-COVID.
Yeah, my experience leading a remoted team in India also showed the importance of cultural awareness and good requirements: I ultimately got into the habit of, after the big meeting with the boss were all the work was given to the various teams, get my guys individually on the phone (so that they feared not "losing face") and carefully coach out of them any questions or doubts since otherwise they wouldn't voice them and just end up implementing something they misunderstood or which wasn't explained correctly and indeed they also needed very detailed requirements which was a problem because the senior guys on the other side who ended up having to write said requirements could pretty much have done the job themselves in that time.
This was a big Investment Bank and some top level manager in NY decided to create a division in India to outsource work to, but it definitely didn't get the cream of the crop over there and the career structuring there was so shit that the few good techies we got would quickly end up as (bad) managers - their pay scales followed the stupid idea that "nobody can be paid more than management" so good mid level techies had to become junior managers to earn more, and they invariably were crap as managers.
Your experience echoes my own with our team. That said, I think we got a pretty good set of devs on our team, and I think a big part of that is my boss being Indian and worked at the org we hired, so he understands the culture there and how to get a better deal. The net result is that the teams we hired are pretty reasonable and work alongside mine (we actually hired two teams and had them "compete," and we kept the better one).
And that's not even getting into how flooded the sector is with the hundreds of thousands being laid off for the past few years
And that's what I'm blaming the low quality of applicants on recently. We looked for almost two years for a FE lead, and then they ended up being super toxic a few months in (they blew up in a meeting w/ some remote teams that came to town to visit). Even decent junior devs are hard to find it seems.
So it seems a lot of these layoffs are cutting out the less skilled devs, but given that we've been able to hire a few great people in the last year, there is some good talent getting caught in the cross-fire as well.
Altering the prompt will certainly give a different output, though. Ok, maybe "think about this problem for a moment" is a weird prompt; I see how it actually doesn't make much sense.
However, including something along the lines of "think through the problem step-by-step" in the prompt really makes a difference, in my experience. The LLM will then, to a higher degree, include sections of "reasoning", thereby arriving at an output that's more correct or of higher quality.
This, to me, seems like a simple precursor to the way a model like the new o1 from OpenAI (partly) works; It "thinks" about the prompt behind the scenes, presenting only the resulting output and a hidden (by default) generated summary of the secret raw "thinking" to the user.
Of course, it's unnecessary - maybe even stupid - to include nonsense or smalltalk in LLM prompts (unless it has proven to actually enhance the output you want), but since (some) LLMs happen to be lazy by design, telling them what to do (like reasoning) can definitely make a great difference.
And that's why I'm the one that fixes the PC when it breaks... because even good programmers may even consider the pc to be magicboxes if they've never turned a screwdriver in their life...
All the while it gets further and further from the requirements. So you open five more conversations, give them the same prompt, and try pick which one is least wrong.
All the while realising you did this to save time but at this point coding from scratch would have been faster.
I interviewed someone who used AI (CoPilot, I think), and while it somewhat worked, it gave the wrong implementation of a basic algorithm. We pointed out the mistake, the developer fixed it (we had to provide the basic algorithm, which was fine), and then they refactored and AI spat out the same mistake, which the developer again didn't notice.
AI is fine if you know what you're doing and can correct the mistakes it makes (i.e. use it as fancy code completion), but you really do need to know what you're doing. I recommend new developers avoid AI like the plague until they can use it to cut out the mundane stuff instead of filling in their knowledge gaps. It'll do a decent job at certain prompts (i.e. generate me a function/class that...), but you're going to need to go through line-by-line and make sure it's actually doing the right thing. I find writing code to be much faster than reading and correcting code so I don't bother w/ AI, but YMMV.
An area where it's probably ideal is finding stuff in documentation. Some projects are huge and their search sucks, so being able to say, "find the docs for a function in library X that does..." I know what I want, I just may not remember the name or the module, and I certainly don't remember the argument order.
AI is fine if you know what you're doing and can correct the mistakes it makes (i.e. use it as fancy code completion)
I'm not a developer and i havent touched code for over 10 yrs, but when i heard about my company pushing AI tools on the devs, i thought exactly what you said. It should be a tool for experienced devs who already know what they're doing....
Lo and behold they did the opposite... They fired all the senior people and pushed AI on the interns and new grads.... and then expected AI to suddenly make the jr devs work like the expensive Sr devs they just fired...
Wtf
Yeah, it makes no sense. AI is at best a replacement for junior devs and interns.
AI is like having an intern you can delegate to. If you give it a simple enough task with clear direction, it can come up with something useful, but you need to check.
That sums up my experience too, but I have found it good for discussing functions for SQL and Powershell. Sometimes, it’ll throw something into its garbage code and I’ll be like “what does this do?” It’ll explain how it’s supposed to work, I’ll then work out its correct usage and solve my problem. Weirdly, it’s almost MORE helpful than if it just gave me functional code, because I have to learn how to properly use it rather than just copy/paste what it gives me.
That's true. The mistakes actually make learning possible!
Man, designing CS curriculum will be easy in future. Just ask it to do something simple, and ask your CS students to correct the code.
AI created 17 Security Corporation™️s in response to this comment.
I like using it like a rubber ducky. I even have it respond almost entirely in quacks.
Note: it's a local model running for free. Don't pay anyone for this slop.
What llm did you use, and how long ago was it? Claude sonnet usually writes pretty good python for smaller scripts (a few hundred lines)
It was ChatGPT from earlier this year. It wasn't a huge deal for me that it made mistakes, because I had a very specific use case and just wanted to save some time; I knew I'd have to troubleshoot grafting it into my function, but even after I pointed out that it was using depreciated syntax (and how to correct it), it just spat out the code again with even more errors and still using depreciated syntax.
All LLMs will fail like this in some way, because they don't actually understand what they're generating (i.e. they have no mechanism for self-evaluating the veracity of their statements).
This is a very simple one, but someone lower down apparently had issue with a script like this:
I tested the code, it works. If I was gonna change anything, probably move matplotlib import to after else so it's only imported when needed to display the image.
I have a lot more complex generations in my history, but all of them have personal or business details, and have much more back and forth. But try it yourself, claude have a free tier. Just try to be clear in the prompt what you want. It might surprise you.
I appreciate the effort you put into the comment and your kind tone, but I'm not really interested in increasing LLM presence in my life.
I said what I said, and I experienced what I experienced. Providing me an example where it works is in no way a falsification of the core of my original comment: LLMs have no place generating code for secure applications apart from human review, because they don't have a mechanism to comprehend or proof their own work.
I'd also add that, depending on the language, the ways you can shoot yourself in the foot are very subtle (cf C++/C, which are popular languages for "secure" stuff).
It's already hard to not write buggy code, but I don't think you will detect them by just reviewing LLM code, because detecting issues during code review is much harder than when you're writing code.
Oh, and I assume it'll be tough to get an LLM to follow MISRA conventions.
It's already hard to not write buggy code, but I don't think you will detect them by just reviewing LLM code, because detecting issues during code review is much harder than when you're writing code.
Definitely. That's what I was trying to drive at, but you said it well.
Oh wait, every single senior developer who is currently railing against their moron AI-bandwagoning CEOs.
Middle and upper management are like little children - they'll only learn that fire hurts by putting their hand in it.
But are the shareholders pleased?
I've been laughing at this quote for 5 minutes straight
It's so good
He knows he's right
Also: I code sometimes, and all of my code is of masterpiece quality. I cannot debug my own code, I ask for outside help and we have to dismantle the NT kernel to find out what's gone wrong
Good. This is digital Darwinism at its finest. Weeds out the companies who thought they could save money by relying on a digital monkey instead of actual professionals.
Lmao my job announced layoffs a few months back. They continue to parade their corporate restructuring plan in front of us like we give a fuck if shareholders make money. My output has dropped significantly as I search for another role. Whatever code I do write now is always just copy pasted from AI (which is getting harder to use...fuck you Copilot). I give zero fucks about this place anymore. Maybe if people had some small semblance of investment in their company's success (i.e.: not milked by shareholders and beaten to dust by shitty profit driven metrics that take away from the core business), the employees might give enough fucks to not copy paste shitty third party code.
Additionally, this is a training issue. Don't offload the training of your people onto the universities (which then trap the students into an insurmountable debt load leading them to take jobs they otherwise wouldn't want to take just to eat and have a roof over their heads). The modern corporate landscape has created a perfect shitstorm of disincentives for genuine effort and diligence. Then you expect us to give a shit about your company even though the days of 40 years and a pension are now gone. We're stuck with 401k plans and social security and the luck of the draw as to whether we can retire or not. Work your whole life for what? Fuck you. I'm gonna generate that AI code and enjoy my 30s and 40s.
A workforce trapped by debt, forced to prioritize job security and paycheck size over passion or purpose. People end up in roles they don't care about, working for companies they have no investment in, simply to keep up with loan payments and the ever increasing cost of living.
"Why is my organization falling apart!?" Fucking look up from the stupid fucking metrics that don't actually tell you anything you dumb fucks. Make an actual human decision and fix the wealth inequality. It's literally always wealth inequality.
"People work in roles they don't care about, for companies they have no investment in, to pay loans they shouldn't have."
That sounds like a fight club quote lol. I know you didn't say "loans they shouldn't have" but the cost of college is just stupidly high. It doesn't have to be free but come on.
It doesn't have to be free but come on.
I beg to differ! My degree was free for all intents and purposes, and no, it didn't take away from the challenge or the quality of education. I cried blood tears in order to graduate but it was worth it.
Chuck Palahniuk leaking into my writing like the carrot out of the protagonist's ass in Guts.
15 years ago I got a job where I wasn't allowed to do anything. I hated it. I wanted to learn and be valuable and be valued. I left that job.
I worked for a bank and then Red Hat and I loved what I did and burned myself out trying to make them happy. Only to find out they still didn't value me.
I switched jobs two years ago and increased my pay 30% overnight and back to a job doing nothing. And I'm totally fine with it now. I have a family and I focus on them and during work, if they don't have anything for me to do I make my own happiness.
Fuck corporations. I'll take your money, I'll never again kill myself as I'll never be valued anyway. Jobs aren't worth it. People are.
I told my manager that I've been burned and can't make myself work hard for another company again. She's leaving so there's no vested interest in the company for her. But yeah, fuck these cunts.
Similar trajectory for me, but I'm now being micromanaged on the daily. We got a new CIO recently who is micromanaging his direct reports and our culture has evaporated overnight. The shit is indeed rolling down hill and the writing is on the wall to leave. I know it's not just me either. There will be an exodus when rates get cut and hiring picks up again. This place is fucked.
But that's the key. If you can find something and lay low with minimal annoyance, hang onto that for as long as you can.
You are my spirit animal.
Are you also finding copilot to be less helpful of late? The other day it couldn't follow the simplest of instructions
For me it's the "Stop responding" button. Sometimes I'll neglect something in my prompt, such as the fact that I'm stuck on ES5 javascript in my job (ServiceNow). It'll spit out ES6+ with let declarations or something like that, and I have to go back and qualify my limitations. So I click stop responding. What used to happen was that it would stop and allow for additional prompting. Now it's just like a client side trick. It hides the output but the server is still returning shit in the background, so if I try to re-prompt or add context it finishes what it was originally saying first, then tacks the new answer onto the old one without pause, separation, or human readable formatting that would indicate that there is a new output. It's an awful experience.
I've been using perplexity.ai but my company thinks its agreements will stop Microsoft from training their AIs on our proprietary data, so I have to be more careful with perplexity than Copilot.
Me and my team take our site down the old fashioned way. Code copied from some rando on the internet.
Reminds me of the time that I took down the corporate website by translating the entire website into German. I'd been asked to do this but I hadn't realized that the auto translation Plug-In actually rewrote code into German, I thought it was just going to alter the HTML with JavaScript at runtime, but nope. It actually edited the files.
It also translated the password into German which was fun because it was just random characters so I have no idea what it translated into.
That's fucking hilarious
Same happened with people using the Cloud To Butt extension which replaced every 'cloud' with 'butt' even for codes. Hilarity ensued.
I do have that extension installed. Never been bit so far. I don't copy and paste anymore than a couple of lines at a time.
It's pretty much the same as AIs do - copy and past random code from Stackoverflow - but they do it automatically.
Good old curl|sh
Copy pasting random snippets from search results and chatgpt until something works is how I do my job.
“until something works"
At least you're doing a better job than some people.
Some leave it at will ai told me so. And they don't know better and put that into prod!
“When asked about buggy AI, a common refrain is ‘it is not my code,’ meaning they feel less accountable because they didn’t write it.”
That's... That's so fucking cool...
Amazing
This must sound terrible. So high pitched
And none of the forced tech support "AI" replacements work. And the companies don't give a shit.
I've had this argument with them a few times at work. They are definitely going to replace this all with AI. Probably within the next year and no amount of us pointing out that it won't work and they'll end up having to bring us back, at 3x the rate, seems to have any effect on them.
I'm probably going to have to listen to a lot of arguments about this strawberry thing tomorrow.
Anyway whatever, severance is severance.
I was once in a similar position: company merger and they decided to move support offshore. We got 6 months lead notice and generous severance paid out as long as we stayed to the end. Fast forward a year and they took 85% customer approval to 13%. We got hired back at 1.5x our old pay rate, so not quite the 3x you mentioned. Hoping this works out similar for you in the end.
As stated in the article, this has less to do with using AI, more to do with sloppy code reviews and code quality enforcement. Bad code from AI is just the latest version of mindlessly pasting from Stack Overflow.
I encourage jrs to use tools such as Phind for solving problems but I also expect them to understand what they’re submitting and be ready to defend it no differently to any other PR. If they’re submitting code they don’t understand that’s incredibly unprofessional and I would come down very hard on them. They don’t do this though because we don’t hire dickheads.
Shift-left eliminated the QA role.
Now we have AI generated shit code, with devs that don't understand the low level details of both the language, and the specifics of the generated code.
So we basically have content entry (ai inputs) and extremely shitty QA bundled into the "developer" role.
As a 20 year veteran of the industry, people keep asking me if I think AI will make developers obsolete. I keep telling them "maybe some day, but today's LLMs are not it. The AI bubble is going to burst, and a few legit use cases will make it through"
Bad code from AI is just the latest version of mindlessly pasting from Stack Overflow.
Humans literally can not scan all of SO to make a huge copypasta.
It takes much more time, effort, and thought to find various solutions on SO and patch them together into something that works well.
Yeah but... i asked chatgpt once how to style something in asciidoctors style.yml. It proposed me html syntax (some inline stuff can be done with html tags in asciidoctor, if output is html). After the usual apology, it suggested some wrong yaml. Third try, because formatting was wrong, it mixed them both.
I mean, sure, some niche usecase in a somewhat obscure (lots of moving parts) lightweight markup. But still, this was a lesson.
this has less to do with using AI, more to do with sloppy code reviews and code quality enforcement.
They are the same picture.
More specifically: the same kind of decision makers are behind both.
We used to have these shit developers and I accepted a lot of bad code back then -- if it actually worked -- because otherwise "code review" is full-on training, which is an entire other job from the one I was hired to do.
The client ditched that contracting firm, and the devs I work with now are worth putting in time on code review with -- but damn, we got hella shit code in our codebase to deal with now. Some of it got tossed, some of it ... we live with.
Computer write shite code and the human still gets blamed.
edit: we have become gods
The human turned the code in. They deserve 100% of the blame.
If I was still in a senior dev position, I’d ban AI code assistants for anyone with less than around 10 years experience. It’s a time saver if you can read code almost as fluently as you can read your own native language but even besides the A.I. code introducing bugs, it’s often not the most efficient way. It’s only useful if you can tell that at a glance and reject its suggestions as much as you accept them.
Which, honestly, is how I was when I was first starting out as a developer. I thought I was hot shit and contributing and I was taking half a day to do tasks an experienced developer could do in minutes. Generative AI is a new developer: irrationally confident, not actually saving time, and rarely doing things the best way.
I've found they're great as a learning tool where decent docs are available. Or as an interactive docs you can ask follow up questions to.
We mostly use c# and it's amazing at digging into the MS docs to pull out useful things from the bcl or common patterns.
Our new juniors got up to speed so fast by asking it to explain stuff in the existing codebases. Which in turn takes pressure off more senior staff.
I got productive in vuejs in a large codebase in a couple days that way.
Using to generate actual code is insanely shit haha It is very similar to just copy pasting code and hacking it in without understanding it.
You make a good point about using it for documentation and learning. That’s a pretty good use case. I just wouldn’t want young developers to use it for code completion any more than I’d want college sophomores to use it for writing essays. Professors don’t have you write essays because they like reading essays. Sometimes, doing a task manually is the point of the assignment.
What setup are you using for junior devs to be able to query your codebase?
I was talking more generally about LLMs here
Even worse than it being wrong, is that by nature of the tool it looks right.
Eh, I'm a senior dev, and I don't ban it (my boss, the director, does that for me lol; he's worried about company secrets leaking).
In fact, we had an interview for a senior dev position, and the applicant asked if they could use AI, and I told them to use whatever tools they normally would for development. It shouldn't come as a surprise that they totally botched the programming challenge because of it (introduced the same bug twice, then said they were very confident in the correctness of the code...), and that made it so much easier to filter them out from our hiring pool. If you're going to use a tool in an interview, you better feel confident with it. If that dev had solved the problem significantly faster than our other applicants, I would've taken that to my boss to have the team experiment with it. We target budget 30 min for our challenges, and our seniors generally finish in under 20, and it took them more than our allotted time to get the code to actually run properly (and that's with us pointing out certain mistakes the AI generated).
But no, I haven't seen an actually productive use of AI for software development, beyond searching for docs online (which you can totally do w/ Bing or Google w/o involving our codebase). You may feel more productive because more code is appearing on the screen, but the increase in bugs likely reduces overall productivity. We're always looking for ways to improve, but when I can solve the same problem in my bare-bones editor (vim) faster than my more junior colleagues can with their fancy IDEs, I really don't think AI is going to be the thing that improves our productivity, actually understanding logic will. If someone demonstrates that AI does save time, I'll try it out and campaign for it.
Anyway, that's my take as someone who has been in the industry for something like 15 years. Knowing your tools is more important, IMO, than having more tools.
I had my suspicions before but the moment I realized for certain Elon Musk couldn’t run a software company was when he judged people by lines of code written.
Ew, I would hate to be in charge of code reviews at an org like that.
The proper metric is success of the actual product. We have our engineers give estimates, then hold them to those estimates and evaluate based on consistency of on-time releases and number of production bugs. At the end of the day, predictable, high quality delivery is usually more valuable than faster time to market, unless you're in a startup or something and just need to get early adopters on-board. Judge QA by defects discovered in production and devs by defects found by QA and in production. It's really not that hard.
The one time some manager voiced such an idea, I very overtly in front of everybody offered to make "loop unrolling" software working at the source level (compilers already do it at the Assembly level in some cases for performance) for me and my colleagues to really boost that code line count (while totally screwing maintenability).
Mind you, all devs in that meeting were loudly against measuring performance by code lines, but I like to think that suggestion of mine really hammered down the coup the grace on that "brilliant" idea.
Not trying to defend him, but I thought the reasoning behind doing that was to get the least obedient people to leave the company so that there won't be a delayed push back from the employees.
In my experience working for almost 3 decades in software development, passive-agressive shit from upper management just causes the best people to leave (as they're the ones who easilly find better jobs) leaving behind mainly a mix of the incompetent and those who never worked anywhere else (who are either already incompetent or will become so, as only ever having worked in just one company is far too narrow professional experience for anything beyond junior/mid level - you need to have seen more than one way of doing things to understand certain higher level concerns and choices in software development).
Yeah and I'd say these people left are exactly those Elon wants, he doesn't want white guys in their 50s, he wants obedient young guys.
Sound like a variant of the good old saying "pay peanuts, get monkeys" only using a stick and threats instead of payment.
Mind you, it does sound like the kind of think somebody with his kind of personality - narcissistic shameless and dishonest salesman - would think it's a great idea.
I've worked as a freelancer (specifically as a Contractor) in Software Development for over a decade and more often than not I ended up having to work with some existing code base, having to deal with the design choices, coding style and bugs of somebody else, often multiple somebody elses.
There's nothing quite as "entertaining" as having to deal with 3+ different code and design styles in the same code base because all previous developer thought their own way of doing things was the superior way so just added one more layer of their style (not just coding but, worse, software design) on top of what was already there increasing the mess, rather than work within the existing structure and style and doing some refactoring.
Anyway, in my experience having to read, understand and work with existing code that you yourself did not made is way more time costly and less pleasant than actually doing your stuff from scratch.
See? AI creates jobs! Granted, it's specialized mop up situations, but jobs!
It'll be even more interesting in the future! Every now and then a T1000 will lose all hydraulic fluids right out it's prosthetic anus and they'll need someone there with a mop and bucket! Our economy lives on...
If by economy you mean some of us are needed to mop up hydraulic ass-juices at gunpoint I suppose you're technically correct. At least they have to feed us, right?
..right?
Having spent most of my career working as a senior contractor, which often meant landing on code bases with 3+ layers of fuckups, I can only imagine how painful it will be to end up having to clean and fix AI generated code, since that doesn't even have a consistent coding style or pattern of design errors and bugs.
How come the hallucinating ghost in the machine is generating code so bad the production servers hallucinate even harder and crash?
I’m not sure how AI supposed to understand code. Most of the code out there is garbage. Even most of the working code out there in the world today is garbage.
Heck, I sometimes can’t understand my own code. And this AI thing tries to tell me I should move this code over there and do this and that and then poof it doesn’t compile anymore. The thing is even more clueless than me.
Randomly rearranging non working code one doesn’t understand… sometimes gets working code, sometimes doesn’t fix the bug, sometimes it won’t even compile anymore? Has no clue what the problem is and only solves it randomly by accident?
Sounds like the LLM is as capable as me /s
Sometimes you even get newer and more interesting bugs!
As a senior dev, this sounds like job security. :)
You know you’re Sr. when it doesn’t even bother you anymore. It amuses you.
My boss comes to me saying we must finish feature X by date Y or else.
Me:
We're literally in this mess right now. Basically, product team set out some goals for the year, and we pointed out early on that feature X is going to have a ton of issues. Halfway through the year, my boss (the director) tells the product team we need to start feature X immediately or it's going to have risk of missing the EOY goals. Product team gets all the pre-reqs finished about 2 months before EOY (our "year" ends this month), and surprise surprise, there are tons of issues and we're likely to miss the deadline. Product team is freaking out about their bonuses, whereas I'm chuckling in the corner pointing to the multiple times we told them it's going to have issues.
There's a reason you hire senior engineers, and it's not to wave a magic wand and fix all the issues at the last minute, it's to tell you your expectations are unreasonable. The process should be:
product team lists requirements
some software dev gives a reasonable estimate
senior dev chuckles and doubles it
director chuckles and adds 25% or so to the estimate
if product team doesn't like the estimate, return to 1
we release somewhere between 3 and 4
If you skip some of those steps, you're going to have a bad time.
In my experience, the job of a sr. revolves around expectations. Expectations of yourself, of the customer, of your bosses, of your juniors and individual contributors working with you or that you're tasking. Managing the expectations and understanding how these things go to protect your guys and gals and trying to save management from poking out their own eyes.
And you may actually have time to do some programming.
Yup. I actually only take a 50% workload because half of my time is spent in random meetings telling people no, or giving obscenely high estimates that essentially amount to "no." The other half of my time is fixing problems from when they didn't listen when I said "no."
Such is life I guess. But occasionally, I get to work on something new. And honestly, that's fine, I've long since stopped caring about my name showing up on things.
Not all heroes wear capes. You're saving their butts, and they don't know it.
Can confirm. At our company, we have a tech debt budget, which is really awesome since we can fix the worst of the problems. However, we generate tech debt faster than we can fix it. Adding AI to the mix would just make tech debt even faster, because instead of senior devs reviewing junior dev code, we'd have junior devs reviewing AI code...
LLMs are not supposed to understand, they are supposed to pretend to understand.
You have to be hallucinating to understand.
I’ve licked the frog twice! How many does it take?
A-one. A-two-hoo. A-three... *Crumch*
I take it that frog hadn't been de-boned.
"AI" is just good for simple code snippets.
(Which it stole from Github repos).
This whole ai bs needs to die already, and the people who lie about it held accountable.
Sounds like the Sirius cybernetics corporation:
The fundamental design flaws are obscured by the superficial design flaws.
The point of the article isn't that AI is outright useless as a coding tool but that it lulls programmers into a false sense of security regarding the quality and security of their code. They aren't reviewing their work as frequently because of this new reliance on AI as a time saver, and as such are more likely to miss any mistakes that they or the AJ made.
The point of the article isn’t that AI is outright useless as a coding tool but that it lulls programmers into a false sense of security regarding the quality and security of their code.
Lulling them into a false sense of security is half of what makes it useless. The fact that it makes shitty code is the other half.
But the job of a software developer is not to write good code, it is to deliver features. People have been writing bad code without any AI for decades. Businesses often prioritize speed over quality, rewarding teams that deliver features quicker.
A computer lets you make more mistakes faster than any other invention with the possible exceptions of handguns and Tequila.
Now Even Faster™ with no exceptions thanks to "AI"
Now now, AJ may not know everything, but he'll learn
It basically just turns coders into debuggers.
Devs care to debug code only if they believe in its quality. Otherwise they write the code again from scratch. This is also cheaper than debugging.
I dare to say it: 70% of the devs are not quality focused to start with. They are already happy if something, somewhat, sort of, works. And then not even ship a unit test with it.
Everyone is QA now.
AI can be a useful tool, but it’s not a substitute for actual expertise. More reviews might patch over the problem, but at the end of the day, you need a competent software developer who understands the business case, risk profile, and concrete needs to take responsibility for the code if that code is actually important.
AI is not particularly good at coding, and it’s not particularly good at the human side of engineering either. AI is cheap. It’s the outsourcing problem all over again and with extra steps of having an algorithm hide the indirection between the expertise you need and the product you’re selling.
Debugging and maintenance was always the hardest aspect of large code bases... writing the code is the easy part. Offloading that part to AI only makes the hard stuff harder
I have a lot of empathy for a lot of people. Even ones, who really don't deserve it. But when it comes to people like these, I have absolutely none. If you make a chatbot do your corporate security, it deserves to burn to the ground
Also it is pure junk. Chat-GPT code may come out fast on the screen but it's garbage. I tried python and c++ both just pure garbage. Sure I got it to do what I wanted but only after a day of hair pulling repetitive madness. Simple task, open an image and invert it . Then we'll it opened the image but didn't invert. Or maybe it's upside down. Can you open the image right side up and invert it....fuck fuck, why is the window full screen? Did I ask for full screen, shit heavens no! Anyway it's a fuckin idiot just rambling code at me.
I use it for Ansible, so not for code, and just to reduce the time my brain is exposed to Ansible.
If all you said to me was open an image and invert it, I would probably turn it upside down as well. What are you trying to get it to do?
Probably make the bright pixels dark and the dark pixels bright.
So what they should have said is to make the image negative.
That's being a standard image editing function since the days of film but you have to use the correct terminology.
It's just an example. I did get useful code from all this effort but usually the first prompt gets the closest. Everything else is like a bad genie story. Exactly like this: https://youtu.be/lM0teS7PFMo?si=yMtEaVkpSrn9q5Ap
Open it how using what at what size what codec where, for how long, for what purpose, using what data structures, use what libraries, what versions. You sound like my PO trying to request an update to software they have no comprehension of.
ahahahaha...
Good. Maybe if the stuff trashes enough of our infrastructure somebody somewhere will actually figure out that it's bad and get rid of it forever.
I know, it'll never happen. But a man can dream.
trashes enough of our infrastructure somebody somewhere will actually figure out that it’s bad and get rid of it forever
thinking Neoliberlism.
The thing I dislike most about code assisting tools is that they're geared to answering your questions instead of giving advice. I'm sure they also give bad recommendations but I've seen LLMs basically double down on bad code.
No they’re giving you exactly what you’re asking for. Problem is you’re not asking for advice. Your asking to “build a thing” and expecting it to read your mind.
No sh*t, this is what I predicted from day one.
We should have looked to melroy
Thank you! That is indeed a valid point. I was hoping more people came up with this valid remark. Do you have any other questions or predictions you would like to know? So that we don't get "surprises" in the field of technology again?
Please hit me with some predictions :D
Sure!
More and more (AI) spyware / malware is getting injected into projects and operating systems. Without the user consent. Mobile phones, laptops, desktop PCs, smart devices, etc. This comes from companies, but also from governments (no, not just China, but also US and EU).
AI bubble itself will burst for the "normal users" and most companies who won't really benefit from AI / LLMs as they thought they will. This will be apparenty only after several years. Where the highly skilled developers left the companies, and you are left with software engineers using AI tools which generates wrong code. The damage LLM (like AI Code generation) is doing and will be continue to do in the upcoming years is very untransparent, but it won't be nice.
We suddently are not getting AGI.
More research and efforts will be put into alternative computers, like computers based on biology. Like using living cells. After all nature is so much more efficient then our current technologies. This could fix the energy demand issues we now see with AI.
Biology computer will then also create huge moral issues. Since, how do we know the cells are not becoming aware? How do we know it won't feel pain or the cells are feeling trapped? After all, we, humans, don't even know how conscious really works and self aware.
Users & companies want to get back in control over 5 or 15 years from now. So their could be a big move back from "Cloud" to on-prem. You are already seeing this now with the fediverse.
The internet becomes too much centralized and controlled by goverments. Blocking public DNS IPs. Overruling them. The only answer would be is to create a much more decentralized internet alternative, so over 20 or 30 years from now (so we can still talk which each other about issues in the goverments par example). The current internet is just too fragile. And the root of the problem is already DNS. Meaning you need to basically start from scratch.
Over 80 years Windows might only be used by corporate businesses. Most people might only use Android or any Linux based distro. This mainly depends on how fast we change our education process, so young people learn about alternatives. And schools should stop promoting and forcing people to use Microsoft products only. If schools won't change, then we might have a huge issue, and this topic won't be valid.
Google will be split into multiple companies.
Microsoft might be split later as well into multiple companies, but only much later, after Google.
So you predicted that security flaws in software are not going to vanish with AI?
All software has bugs. I prefer the human-generated bugs, they're much easier to diagnose and solve.
My point exactly, now you have genAI code written by AI, who doesn't know what it is doing. Instructed by a developer, who doesn't understand the programming language. Reviewed by a co-worker, who doesn't know what is doing on. It's madness I tell you!
I predicted that introducing AI on software engineer (especially juniors) will result in overall worse code, since apparently people don't feel responsible for the genAI code. While I believe the responsibility is still fully at the humans who try to deliver code. And on top of that, most devs are not doing good code reviews in general (often due to lack of time or .. skill issue). And now we have AI that generates code which are too easily accepted on top of reviewers who blindly accept code.. And no unit tests or integration tests.. And then we have this current situation. No wonder this would happen. If you are in software engineering, you would know exactly where I'm talking about. Especially if you would work at larger companies.
Lmao this is precisely how I thought this would turn out.
as opposed to human-generated code
But at least that crappy bug-riddled code has soul!
no common sense allowed in this thread, sir. only AI hate bandwagon please.
and here's me learning C programming language from a selfhosted AI :/
It's a great launching pad to learn how a language works, but beyond simple things, it get bad very fast.
I also use AI to look for terms in specific domains, which is really helpful as well.
Where’s the articles about humans doing the exact same shit for the last 40-50 fucking years and no one bats an eye. Looks at the prompts from people complaining about ai responses and see they don’t know how to use this shit any better than my grandparents can use a touchtone phone.
“Build an app”
Fails
“This ai is shit”.
Just like ever other piece of technology. Garbage in garbage out. If you can’t reliably describe what you want then no one is going to be able to do it. AI just blatantly points out your descriptive failures.
I've yet to see generative AI make an error that a human couldn't make. Maybe that's why people seem so hateful of it; they were expecting it to be superhuman but instead it's too much like us.
Ai llms have learned from us. Good and bad. It doesn't know the difference between good and bad unless you tell it.
So you have to know what's good or bad from the get go before using it and trusting it yet.
And some blindly trust ai already... Which its far from that level of trust
That's on them though. The other ones making the claim that it's supposed to be The Culture, but I don't think anyone at the companies is saying that it is.
Wow, the text generator that doesn't actually understand what it's "writing" is making mistakes? Who could have seen that coming?
I once asked one to write a basic 50-line Python program (just to flesh things out), and it made so many basic errors that any first-year CS student could catch. Nobody should trust LLMs with anything related to security, FFS.
ftfy
also any inputs are probably scrapped and used for training, and none of these people get GDPR
ftfy
Let's hope it's the bad outputs that are scrapped. <3
Eh, I'd say mostly.
I have one right now that looks at data and says "Hey, this is weird, here are related things that are different when this weird thing happened. Seems like that may be the cause."
Which is pretty well within what they are good at, especially if you are doing the training yourself.
you are part of the problem
That is about the most generic statement possible, with nearly zero knowledge of what I'm doing on yours.
So... What problem? Feel free to enlighten me.
I wish we could say the students will figure it out, but I've had interns ask for help and then I've watched them try to solve problems by repeatedly asking ChatGPT. It's the scariest thing - "Ok, let's try to think about this problem for a moment before we - ok, you're asking ChatGPT to think for a moment. FFS."
Critical thinking is not being taught anymore.
Has critical thinking ever been taught? Feel like it’s just something you have or you don’t.
Critical thinking is essentially learning to ask good questions and also caring enough to follow the threads you find.
For example, if mental health is to blame for school shootings then what is causing the mental health crisis and are we ensuring that everyone has affordable access to mental healthcare? Okay, we have a list of factors that adversely impact mental health, what can we do to address each one? Etc.
Critical thinking isn't hard, it just takes time, effort.
I have the impression that most people (or maybe it's my faith in Humanity that's at an all time low and it's really just "some people") just want pre-chewed explanations given to them rather than spend time and energy figuring things out themselves - basically baby pap as ideas food rather than cooking their own ideas food out of raw ingredients.
Certainly that would help explain the resurgence of Populist sloganeering and continued popularity of Religion (with it's ever popular simple explanations of "Deity did it" and "it's the will of Deity")
In my mind, it's really just entitlement. Something along the lines of, "well, I don't know the answer and why should I have to know if someone else is going to figure it out."
In a tired way, I understand it. Everyday I just want some of my time back for myself. If I'm always the one who has to work through all the problems for my ideas just to be ignored then I'm just going to be perpetually frustrated. So if my ideas are half baked and the solutions i barf up aren't to your liking, well, figure it out yourself.
Not to say that I am this way. I don't get frustrated when my ideas are ignored. I do get frustrated, though, when others eat up half baked ideas knowing they are just that.
Sorry, if what I've wrote so far has gotten a bit confusing. I'll wrap it up and say, it's entitlement. People don't want to think for themselves because it's time consuming. They think the world should order itself in a way that fulfills their needs with minimal effort on their part. Except, to understand how the world would be ordered for that to be reality, they can't comprehend because no one has really figured that one out. So they fall back on god and gods an easy out because, duh, he's god.
Critical thinking, especially Skepticism, does not make for good Consumers or mindless followers of Political Tribes.
British primary schools used to have something called 'problem solving' it was usually a simple maths problem described in words that required some degree of critical thinking to solve. e.g. A frog is at the bottom of a 30m well, it climbs 7m each day but in the night it slides 3m back down in its sleep. You can't just calculate 30/(7-3) because it doesn't account for the day the frog gets over the top and thus doesn't slide back down in its sleep.
Not the most complex problem but pretty good for kids under 10 to start getting the basics.
I had a chat w/ my sibling about the future of various careers, and my argument was basically that I wouldn't recommend CS to new students. There was a huge need for SW engineers a few years ago, so everyone and their dog seems to be jumping on the bandwagon, and the quality of the applicants I've had has been absolutely terrible. It used to be that you could land a decent SW job without having much skill (basically a pulse and a basic understanding of scripting), but I think that time has passed.
I absolutely think SW engineering is going to be a great career long-term, I just can't encourage everyone to do it because the expectations for ability are going to go up as AI gets better. If you're passionate about it, you're going to ignore whatever I say anyway, and you'll succeed. But if my recommendation changes your mind, then you probably aren't passionate enough about it to succeed in a world where AI can write somewhat passable code and will keep getting (slowly) better.
I'm not worried at all about my job or anyone on my team, I'm worried for the next batch of CS grads who chatGPT'd their way through their degree. "Cs get degrees" isn't going to land you a job anymore, passion about the subject matter will.
Outsourcing killed a lot of the junior and even mid-level career level opportunities in CS and AI seems on track to do the same.
The downside is that going into CS now (and having gone into CS in the last decade or so, especially in English-speaking countries) was basically the career equivalent of just out of the starting line running full speed into a brick wall.
The upside is that for anybody who now is a senior techie things have never been this good because there are significantly fewer people at that level than there is need for such people, since in the last decade or so a lot of people haven't had the chance to progress in their careers to that point.
Whilst personally this benefits me, I'm totally against this shit and what it has done to the kids entering my career.
Yup, and that's why I'll discourage people from entering my career, not because it's a bad gig and it's going away, but because the bar for competency is about to go up. Do it if you're passionate and you'll probably do well for yourself, but don't do it if you're just looking for a good job. If you just want a good job, go into nursing, accounting, or the trades.
I think it's even worse than just the bar for competency going up: even for a coding wizard going into the career, it's a lot harder to squeeze through the bottleneck which is getting an entry level position nowadays unless they have some public proof out on the Net of how good they're at coding (say, commits in open source projects, your own public projects, or even Youtube videos about it).
This is something that will negativelly impact perfectly capable young developers who have an introvert personality type (which are most of them in my experience, even in domains such as Hacking) since some of the upsides of Introversion are a greater capacity for really focusing on on things and for detailed analysis - both things that make for the best programmers - and self publicising isn't a part of the required skillset for good developers (though sooner or later the best ones will have to learn some "image management" if they end up in the Corporate world)
I'm a bit torn on this since on one side salesmanship being more of a criteria determining one's chances of getting a break at the start of one's career as a developer is bad news (good coding and good salesmanship tend to be inverselly correlated) but on the other side a junior developer with some experience actually working with other people on real projects with real users (because they contributed to existing open source projects) has already started learning what we have to teach fresh-out-of-Uni developers to make them professionals.
Eh, I think that's overblown. As someone involved in hiring, we go through a ton of crappy candidates before finding someone half-decent, and when we see someone who actually knows what they're doing, we rush them through the process. The problem is that we're not a big tech company, we're in manufacturing, but we do interesting things w/ software. So getting on at one of the big tech companies may be challenging, but if you broaden the scope a little, there are tons of jobs waiting. We've had junior positions open for months because the hiring pool is so trash, but when we see a good candidate, we can get an offer to them by the end of the week.
We don't care too much about broader visibility (though I will look at your code if you provide a link), we expect competency on our relatively simple coding challenges, as well as a host of technical questions. We also don't mind hiring immigrants, we've sponsored a number of immigrants on our team.
As an introvert myself, I totally get it. I got my job because a recruiter reached out to me, not because I was particularly good at following up with applications. And that's why I tend to tell people to not get into CS. I encourage them to take CS classes if they're offered, but not to make it a career choice, and this is for two reasons:
I see. That does change the idea I had about things a bit.
It's been a while since I was last hiring.
I wasn't aware that the problem nowadays in the West (or at least the US) was an excess of people who don't really have a natural skill for it choosing software development as a career.
That kind of thing was one of the main problems with outsourcing to India maybe a decade ago: the profession was comparatively very well paid for the country so it attracted far too many people without the right skills resulting in a really low average quality of the programmers there - India had really good programmers just like everywhere else but then had a ton of people also working as programmers who should never had gone into it, so the experience of those having to deal with outsourced programming in India usually was pretty bad (I remotelly was a technical lead for a small outsourced team in India from London, and they were really bad whilst, curiously, the good programmers from the Indian Subcontinent I worked with had emigrated from there and were working in London and New York).
Yeah, there was a huge spike in demand for software engineers in the mid-2010s or so, and a massive explosion during COVID, so a lot of less qualified people were handed jobs as long as they could write very basic Python/JavaScript. But after the drop in tech demand after COVID, companies realized they overhired, so they did a ton of layoffs, mostly shucking those low-skilled employees (at least in the first round or two).
Maybe it's different in my area, but we have a ton of applicants whose only dev experience is a bootcamp program, and they apply for a full-time position when many aren't even qualified for a part-time intern position. They would utterly fail to answer most of our questions, and not make any progress on our (relatively simple) programming challenges. We have a mix of theory (i.e. OO principles, ACID, etc) and practical questions (e.g. concurrency vs parallelism in JavaScript or Python, duck typing, etc). CS grads with little practical experience would fly through the theory, but fail on the practical questions. Bootcamp "grads" would fail miserably at theory, and maybe get half of the practical questions, but then fail on the challenge. Dedicated hobbyists and passionate CS/bootcamp grads would do okay at theory (esp. if we change terminology) and fly through the practical questions and coding challenge.
It's pretty easy to weed a lot of those out in our first round, but the sheer volume of terrible applicants makes hiring super time-consuming. I mostly do second round interviews now (my boss, the director, has the thankless first round job), but I've done my fair share of first round interviews as well. I was involved in interviews almost 10 years ago, and I remember there being a lot fewer bad applicants then (i.e. of 5 applicants, 2 were hirable; now it's like 1 in 10, on a good week).
To be fair, I don't work for a tech company, so we're not really anyone's first pick. My company manufacturers things, and our software wing is pretty new and IMO really interesting (we do a lot of complex modeling), but it's not a place most would think to apply for, which is probably why we attract more desperate people. Then again, my last company was similar and much less visible (had something like 30-40 employees, current company has hundreds locally and thousands globally), yet we got better applicants. The main difference here is time, the CS programs at local universities have a ton more enrollment (some are actually turning away people now), whereas when I went it wasn't very popular, and I don't recall bootcamps really being a thing.
As for India, they have a lot of great talent and their IT/programming programs are super competitive (i.e. something like 5% of applicants get in, and only 40-60% of grads get jobs). However, the common thread I've seen is that Indian developers are very reliant on requirements, and they'll build pretty much exactly what you specify (i.e. if something seems off, they won't raise concerns). A lot of this is cultural, and I've heard horror stories from my Indian coworkers about managers telling them to do ridiculous things because of what a client said instead of pushing back. For example, my coworker spent well over a week re-implementing a standard Android behavior because the client specified something slightly outside what was possible through standard APIs, but probably would've preferred the 5 min solution. An American dev would just ask the customer and probably end up doing the 5-minute solution. Maybe these are isolated incidents, but I've seen similar behavior in different projects. If you know this upfront, you can get a good result with regular checkins and small adjustments as you go, but a lot of people don't understand that.
As it stands, since we can't hire good devs here in the US, we've been forced to hire outside firms to fill our ranks. We now have a team in Europe and another in India because we can't find the proper talent here, and not for lack of trying. And I don't think our expectations are out of whack, I just think the good devs already have good jobs and the less qualified devs are the ones getting laid off. We have hired a few good devs in the last few years (not rockstars, just solid devs), so it's not like everyone who lost their jobs aren't qualified, there just seems to be a lot of unqualified people.
Thanks for that very complete view of things.
Things are quite different since I last was doing hiring, which was pre-COVID.
Yeah, my experience leading a remoted team in India also showed the importance of cultural awareness and good requirements: I ultimately got into the habit of, after the big meeting with the boss were all the work was given to the various teams, get my guys individually on the phone (so that they feared not "losing face") and carefully coach out of them any questions or doubts since otherwise they wouldn't voice them and just end up implementing something they misunderstood or which wasn't explained correctly and indeed they also needed very detailed requirements which was a problem because the senior guys on the other side who ended up having to write said requirements could pretty much have done the job themselves in that time.
This was a big Investment Bank and some top level manager in NY decided to create a division in India to outsource work to, but it definitely didn't get the cream of the crop over there and the career structuring there was so shit that the few good techies we got would quickly end up as (bad) managers - their pay scales followed the stupid idea that "nobody can be paid more than management" so good mid level techies had to become junior managers to earn more, and they invariably were crap as managers.
Your experience echoes my own with our team. That said, I think we got a pretty good set of devs on our team, and I think a big part of that is my boss being Indian and worked at the org we hired, so he understands the culture there and how to get a better deal. The net result is that the teams we hired are pretty reasonable and work alongside mine (we actually hired two teams and had them "compete," and we kept the better one).
And that's not even getting into how flooded the sector is with the hundreds of thousands being laid off for the past few years
And that's what I'm blaming the low quality of applicants on recently. We looked for almost two years for a FE lead, and then they ended up being super toxic a few months in (they blew up in a meeting w/ some remote teams that came to town to visit). Even decent junior devs are hard to find it seems.
So it seems a lot of these layoffs are cutting out the less skilled devs, but given that we've been able to hire a few great people in the last year, there is some good talent getting caught in the cross-fire as well.
Altering the prompt will certainly give a different output, though. Ok, maybe "think about this problem for a moment" is a weird prompt; I see how it actually doesn't make much sense.
However, including something along the lines of "think through the problem step-by-step" in the prompt really makes a difference, in my experience. The LLM will then, to a higher degree, include sections of "reasoning", thereby arriving at an output that's more correct or of higher quality.
This, to me, seems like a simple precursor to the way a model like the new o1 from OpenAI (partly) works; It "thinks" about the prompt behind the scenes, presenting only the resulting output and a hidden (by default) generated summary of the secret raw "thinking" to the user.
Of course, it's unnecessary - maybe even stupid - to include nonsense or smalltalk in LLM prompts (unless it has proven to actually enhance the output you want), but since (some) LLMs happen to be lazy by design, telling them what to do (like reasoning) can definitely make a great difference.
And that's why I'm the one that fixes the PC when it breaks... because even good programmers may even consider the pc to be magicboxes if they've never turned a screwdriver in their life...
My experience with ChatGPT goes like this:
All the while it gets further and further from the requirements. So you open five more conversations, give them the same prompt, and try pick which one is least wrong.
All the while realising you did this to save time but at this point coding from scratch would have been faster.
I interviewed someone who used AI (CoPilot, I think), and while it somewhat worked, it gave the wrong implementation of a basic algorithm. We pointed out the mistake, the developer fixed it (we had to provide the basic algorithm, which was fine), and then they refactored and AI spat out the same mistake, which the developer again didn't notice.
AI is fine if you know what you're doing and can correct the mistakes it makes (i.e. use it as fancy code completion), but you really do need to know what you're doing. I recommend new developers avoid AI like the plague until they can use it to cut out the mundane stuff instead of filling in their knowledge gaps. It'll do a decent job at certain prompts (i.e. generate me a function/class that...), but you're going to need to go through line-by-line and make sure it's actually doing the right thing. I find writing code to be much faster than reading and correcting code so I don't bother w/ AI, but YMMV.
An area where it's probably ideal is finding stuff in documentation. Some projects are huge and their search sucks, so being able to say, "find the docs for a function in library X that does..." I know what I want, I just may not remember the name or the module, and I certainly don't remember the argument order.
I'm not a developer and i havent touched code for over 10 yrs, but when i heard about my company pushing AI tools on the devs, i thought exactly what you said. It should be a tool for experienced devs who already know what they're doing....
Lo and behold they did the opposite... They fired all the senior people and pushed AI on the interns and new grads.... and then expected AI to suddenly make the jr devs work like the expensive Sr devs they just fired...
Wtf
Yeah, it makes no sense. AI is at best a replacement for junior devs and interns.
AI is like having an intern you can delegate to. If you give it a simple enough task with clear direction, it can come up with something useful, but you need to check.
That sums up my experience too, but I have found it good for discussing functions for SQL and Powershell. Sometimes, it’ll throw something into its garbage code and I’ll be like “what does this do?” It’ll explain how it’s supposed to work, I’ll then work out its correct usage and solve my problem. Weirdly, it’s almost MORE helpful than if it just gave me functional code, because I have to learn how to properly use it rather than just copy/paste what it gives me.
That's true. The mistakes actually make learning possible!
Man, designing CS curriculum will be easy in future. Just ask it to do something simple, and ask your CS students to correct the code.
AI created 17 Security Corporation™️s in response to this comment.
I like using it like a rubber ducky. I even have it respond almost entirely in quacks.
Note: it's a local model running for free. Don't pay anyone for this slop.
What llm did you use, and how long ago was it? Claude sonnet usually writes pretty good python for smaller scripts (a few hundred lines)
It was ChatGPT from earlier this year. It wasn't a huge deal for me that it made mistakes, because I had a very specific use case and just wanted to save some time; I knew I'd have to troubleshoot grafting it into my function, but even after I pointed out that it was using depreciated syntax (and how to correct it), it just spat out the code again with even more errors and still using depreciated syntax.
All LLMs will fail like this in some way, because they don't actually understand what they're generating (i.e. they have no mechanism for self-evaluating the veracity of their statements).
This is a very simple one, but someone lower down apparently had issue with a script like this:
https://i.imgur.com/wD9XXYt.png
I tested the code, it works. If I was gonna change anything, probably move matplotlib import to after else so it's only imported when needed to display the image.
I have a lot more complex generations in my history, but all of them have personal or business details, and have much more back and forth. But try it yourself, claude have a free tier. Just try to be clear in the prompt what you want. It might surprise you.
I appreciate the effort you put into the comment and your kind tone, but I'm not really interested in increasing LLM presence in my life.
I said what I said, and I experienced what I experienced. Providing me an example where it works is in no way a falsification of the core of my original comment: LLMs have no place generating code for secure applications apart from human review, because they don't have a mechanism to comprehend or proof their own work.
I'd also add that, depending on the language, the ways you can shoot yourself in the foot are very subtle (cf C++/C, which are popular languages for "secure" stuff).
It's already hard to not write buggy code, but I don't think you will detect them by just reviewing LLM code, because detecting issues during code review is much harder than when you're writing code.
Oh, and I assume it'll be tough to get an LLM to follow MISRA conventions.
Definitely. That's what I was trying to drive at, but you said it well.
Oh geez…who could have seen this coming?
Oh wait, every single senior developer who is currently railing against their moron AI-bandwagoning CEOs.
Middle and upper management are like little children - they'll only learn that fire hurts by putting their hand in it.
But are the shareholders pleased?
I've been laughing at this quote for 5 minutes straight
It's so good
He knows he's right
Also: I code sometimes, and all of my code is of masterpiece quality. I cannot debug my own code, I ask for outside help and we have to dismantle the NT kernel to find out what's gone wrong
Good. This is digital Darwinism at its finest. Weeds out the companies who thought they could save money by relying on a digital monkey instead of actual professionals.
Lmao my job announced layoffs a few months back. They continue to parade their corporate restructuring plan in front of us like we give a fuck if shareholders make money. My output has dropped significantly as I search for another role. Whatever code I do write now is always just copy pasted from AI (which is getting harder to use...fuck you Copilot). I give zero fucks about this place anymore. Maybe if people had some small semblance of investment in their company's success (i.e.: not milked by shareholders and beaten to dust by shitty profit driven metrics that take away from the core business), the employees might give enough fucks to not copy paste shitty third party code.
Additionally, this is a training issue. Don't offload the training of your people onto the universities (which then trap the students into an insurmountable debt load leading them to take jobs they otherwise wouldn't want to take just to eat and have a roof over their heads). The modern corporate landscape has created a perfect shitstorm of disincentives for genuine effort and diligence. Then you expect us to give a shit about your company even though the days of 40 years and a pension are now gone. We're stuck with 401k plans and social security and the luck of the draw as to whether we can retire or not. Work your whole life for what? Fuck you. I'm gonna generate that AI code and enjoy my 30s and 40s.
A workforce trapped by debt, forced to prioritize job security and paycheck size over passion or purpose. People end up in roles they don't care about, working for companies they have no investment in, simply to keep up with loan payments and the ever increasing cost of living.
"Why is my organization falling apart!?" Fucking look up from the stupid fucking metrics that don't actually tell you anything you dumb fucks. Make an actual human decision and fix the wealth inequality. It's literally always wealth inequality.
"People work in roles they don't care about, for companies they have no investment in, to pay loans they shouldn't have."
That sounds like a fight club quote lol. I know you didn't say "loans they shouldn't have" but the cost of college is just stupidly high. It doesn't have to be free but come on.
I beg to differ! My degree was free for all intents and purposes, and no, it didn't take away from the challenge or the quality of education. I cried blood tears in order to graduate but it was worth it.
Chuck Palahniuk leaking into my writing like the carrot out of the protagonist's ass in Guts.
15 years ago I got a job where I wasn't allowed to do anything. I hated it. I wanted to learn and be valuable and be valued. I left that job.
I worked for a bank and then Red Hat and I loved what I did and burned myself out trying to make them happy. Only to find out they still didn't value me.
I switched jobs two years ago and increased my pay 30% overnight and back to a job doing nothing. And I'm totally fine with it now. I have a family and I focus on them and during work, if they don't have anything for me to do I make my own happiness.
Fuck corporations. I'll take your money, I'll never again kill myself as I'll never be valued anyway. Jobs aren't worth it. People are.
I told my manager that I've been burned and can't make myself work hard for another company again. She's leaving so there's no vested interest in the company for her. But yeah, fuck these cunts.
Similar trajectory for me, but I'm now being micromanaged on the daily. We got a new CIO recently who is micromanaging his direct reports and our culture has evaporated overnight. The shit is indeed rolling down hill and the writing is on the wall to leave. I know it's not just me either. There will be an exodus when rates get cut and hiring picks up again. This place is fucked.
But that's the key. If you can find something and lay low with minimal annoyance, hang onto that for as long as you can.
You are my spirit animal.
Are you also finding copilot to be less helpful of late? The other day it couldn't follow the simplest of instructions
For me it's the "Stop responding" button. Sometimes I'll neglect something in my prompt, such as the fact that I'm stuck on ES5 javascript in my job (ServiceNow). It'll spit out ES6+ with let declarations or something like that, and I have to go back and qualify my limitations. So I click stop responding. What used to happen was that it would stop and allow for additional prompting. Now it's just like a client side trick. It hides the output but the server is still returning shit in the background, so if I try to re-prompt or add context it finishes what it was originally saying first, then tacks the new answer onto the old one without pause, separation, or human readable formatting that would indicate that there is a new output. It's an awful experience.
I've been using perplexity.ai but my company thinks its agreements will stop Microsoft from training their AIs on our proprietary data, so I have to be more careful with perplexity than Copilot.
Me and my team take our site down the old fashioned way. Code copied from some rando on the internet.
Reminds me of the time that I took down the corporate website by translating the entire website into German. I'd been asked to do this but I hadn't realized that the auto translation Plug-In actually rewrote code into German, I thought it was just going to alter the HTML with JavaScript at runtime, but nope. It actually edited the files.
It also translated the password into German which was fun because it was just random characters so I have no idea what it translated into.
That's fucking hilarious
Same happened with people using the Cloud To Butt extension which replaced every 'cloud' with 'butt' even for codes. Hilarity ensued.
I do have that extension installed. Never been bit so far. I don't copy and paste anymore than a couple of lines at a time.
It's pretty much the same as AIs do - copy and past random code from Stackoverflow - but they do it automatically.
Good old
curl|shCopy pasting random snippets from search results and chatgpt until something works is how I do my job.
“until something works" At least you're doing a better job than some people.
Some leave it at will ai told me so. And they don't know better and put that into prod!
That's... That's so fucking cool...
Amazing
This must sound terrible. So high pitched
And none of the forced tech support "AI" replacements work. And the companies don't give a shit.
I've had this argument with them a few times at work. They are definitely going to replace this all with AI. Probably within the next year and no amount of us pointing out that it won't work and they'll end up having to bring us back, at 3x the rate, seems to have any effect on them.
I'm probably going to have to listen to a lot of arguments about this strawberry thing tomorrow.
Anyway whatever, severance is severance.
I was once in a similar position: company merger and they decided to move support offshore. We got 6 months lead notice and generous severance paid out as long as we stayed to the end. Fast forward a year and they took 85% customer approval to 13%. We got hired back at 1.5x our old pay rate, so not quite the 3x you mentioned. Hoping this works out similar for you in the end.
As stated in the article, this has less to do with using AI, more to do with sloppy code reviews and code quality enforcement. Bad code from AI is just the latest version of mindlessly pasting from Stack Overflow.
I encourage jrs to use tools such as Phind for solving problems but I also expect them to understand what they’re submitting and be ready to defend it no differently to any other PR. If they’re submitting code they don’t understand that’s incredibly unprofessional and I would come down very hard on them. They don’t do this though because we don’t hire dickheads.
Shift-left eliminated the QA role.
Now we have AI generated shit code, with devs that don't understand the low level details of both the language, and the specifics of the generated code.
So we basically have content entry (ai inputs) and extremely shitty QA bundled into the "developer" role.
As a 20 year veteran of the industry, people keep asking me if I think AI will make developers obsolete. I keep telling them "maybe some day, but today's LLMs are not it. The AI bubble is going to burst, and a few legit use cases will make it through"
Humans literally can not scan all of SO to make a huge copypasta.
It takes much more time, effort, and thought to find various solutions on SO and patch them together into something that works well.
Yeah but... i asked chatgpt once how to style something in asciidoctors style.yml. It proposed me html syntax (some inline stuff can be done with html tags in asciidoctor, if output is html). After the usual apology, it suggested some wrong yaml. Third try, because formatting was wrong, it mixed them both.
I mean, sure, some niche usecase in a somewhat obscure (lots of moving parts) lightweight markup. But still, this was a lesson.
They are the same picture.
More specifically: the same kind of decision makers are behind both.
We used to have these shit developers and I accepted a lot of bad code back then -- if it actually worked -- because otherwise "code review" is full-on training, which is an entire other job from the one I was hired to do.
The client ditched that contracting firm, and the devs I work with now are worth putting in time on code review with -- but damn, we got hella shit code in our codebase to deal with now. Some of it got tossed, some of it ... we live with.
Computer write shite code and the human still gets blamed.
edit: we have become gods
The human turned the code in. They deserve 100% of the blame.
If I was still in a senior dev position, I’d ban AI code assistants for anyone with less than around 10 years experience. It’s a time saver if you can read code almost as fluently as you can read your own native language but even besides the A.I. code introducing bugs, it’s often not the most efficient way. It’s only useful if you can tell that at a glance and reject its suggestions as much as you accept them.
Which, honestly, is how I was when I was first starting out as a developer. I thought I was hot shit and contributing and I was taking half a day to do tasks an experienced developer could do in minutes. Generative AI is a new developer: irrationally confident, not actually saving time, and rarely doing things the best way.
I've found they're great as a learning tool where decent docs are available. Or as an interactive docs you can ask follow up questions to.
We mostly use c# and it's amazing at digging into the MS docs to pull out useful things from the bcl or common patterns.
Our new juniors got up to speed so fast by asking it to explain stuff in the existing codebases. Which in turn takes pressure off more senior staff.
I got productive in vuejs in a large codebase in a couple days that way.
Using to generate actual code is insanely shit haha It is very similar to just copy pasting code and hacking it in without understanding it.
You make a good point about using it for documentation and learning. That’s a pretty good use case. I just wouldn’t want young developers to use it for code completion any more than I’d want college sophomores to use it for writing essays. Professors don’t have you write essays because they like reading essays. Sometimes, doing a task manually is the point of the assignment.
What setup are you using for junior devs to be able to query your codebase?
I was talking more generally about LLMs here
Even worse than it being wrong, is that by nature of the tool it looks right.
Eh, I'm a senior dev, and I don't ban it (my boss, the director, does that for me lol; he's worried about company secrets leaking).
In fact, we had an interview for a senior dev position, and the applicant asked if they could use AI, and I told them to use whatever tools they normally would for development. It shouldn't come as a surprise that they totally botched the programming challenge because of it (introduced the same bug twice, then said they were very confident in the correctness of the code...), and that made it so much easier to filter them out from our hiring pool. If you're going to use a tool in an interview, you better feel confident with it. If that dev had solved the problem significantly faster than our other applicants, I would've taken that to my boss to have the team experiment with it. We target budget 30 min for our challenges, and our seniors generally finish in under 20, and it took them more than our allotted time to get the code to actually run properly (and that's with us pointing out certain mistakes the AI generated).
But no, I haven't seen an actually productive use of AI for software development, beyond searching for docs online (which you can totally do w/ Bing or Google w/o involving our codebase). You may feel more productive because more code is appearing on the screen, but the increase in bugs likely reduces overall productivity. We're always looking for ways to improve, but when I can solve the same problem in my bare-bones editor (vim) faster than my more junior colleagues can with their fancy IDEs, I really don't think AI is going to be the thing that improves our productivity, actually understanding logic will. If someone demonstrates that AI does save time, I'll try it out and campaign for it.
Anyway, that's my take as someone who has been in the industry for something like 15 years. Knowing your tools is more important, IMO, than having more tools.
I had my suspicions before but the moment I realized for certain Elon Musk couldn’t run a software company was when he judged people by lines of code written.
Ew, I would hate to be in charge of code reviews at an org like that.
The proper metric is success of the actual product. We have our engineers give estimates, then hold them to those estimates and evaluate based on consistency of on-time releases and number of production bugs. At the end of the day, predictable, high quality delivery is usually more valuable than faster time to market, unless you're in a startup or something and just need to get early adopters on-board. Judge QA by defects discovered in production and devs by defects found by QA and in production. It's really not that hard.
The one time some manager voiced such an idea, I very overtly in front of everybody offered to make "loop unrolling" software working at the source level (compilers already do it at the Assembly level in some cases for performance) for me and my colleagues to really boost that code line count (while totally screwing maintenability).
Mind you, all devs in that meeting were loudly against measuring performance by code lines, but I like to think that suggestion of mine really hammered down the coup the grace on that "brilliant" idea.
Not trying to defend him, but I thought the reasoning behind doing that was to get the least obedient people to leave the company so that there won't be a delayed push back from the employees.
In my experience working for almost 3 decades in software development, passive-agressive shit from upper management just causes the best people to leave (as they're the ones who easilly find better jobs) leaving behind mainly a mix of the incompetent and those who never worked anywhere else (who are either already incompetent or will become so, as only ever having worked in just one company is far too narrow professional experience for anything beyond junior/mid level - you need to have seen more than one way of doing things to understand certain higher level concerns and choices in software development).
Yeah and I'd say these people left are exactly those Elon wants, he doesn't want white guys in their 50s, he wants obedient young guys.
Sound like a variant of the good old saying "pay peanuts, get monkeys" only using a stick and threats instead of payment.
Mind you, it does sound like the kind of think somebody with his kind of personality - narcissistic shameless and dishonest salesman - would think it's a great idea.
I've worked as a freelancer (specifically as a Contractor) in Software Development for over a decade and more often than not I ended up having to work with some existing code base, having to deal with the design choices, coding style and bugs of somebody else, often multiple somebody elses.
There's nothing quite as "entertaining" as having to deal with 3+ different code and design styles in the same code base because all previous developer thought their own way of doing things was the superior way so just added one more layer of their style (not just coding but, worse, software design) on top of what was already there increasing the mess, rather than work within the existing structure and style and doing some refactoring.
Anyway, in my experience having to read, understand and work with existing code that you yourself did not made is way more time costly and less pleasant than actually doing your stuff from scratch.
See? AI creates jobs! Granted, it's specialized mop up situations, but jobs!
It'll be even more interesting in the future! Every now and then a T1000 will lose all hydraulic fluids right out it's prosthetic anus and they'll need someone there with a mop and bucket! Our economy lives on...
If by economy you mean some of us are needed to mop up hydraulic ass-juices at gunpoint I suppose you're technically correct. At least they have to feed us, right?
..right?
Having spent most of my career working as a senior contractor, which often meant landing on code bases with 3+ layers of fuckups, I can only imagine how painful it will be to end up having to clean and fix AI generated code, since that doesn't even have a consistent coding style or pattern of design errors and bugs.
How come the hallucinating ghost in the machine is generating code so bad the production servers hallucinate even harder and crash?
I’m not sure how AI supposed to understand code. Most of the code out there is garbage. Even most of the working code out there in the world today is garbage.
Heck, I sometimes can’t understand my own code. And this AI thing tries to tell me I should move this code over there and do this and that and then poof it doesn’t compile anymore. The thing is even more clueless than me.
Randomly rearranging non working code one doesn’t understand… sometimes gets working code, sometimes doesn’t fix the bug, sometimes it won’t even compile anymore? Has no clue what the problem is and only solves it randomly by accident?
Sounds like the LLM is as capable as me /s
Sometimes you even get newer and more interesting bugs!
As a senior dev, this sounds like job security. :)
You know you’re Sr. when it doesn’t even bother you anymore. It amuses you.
My boss comes to me saying we must finish feature X by date Y or else.
Me:
We're literally in this mess right now. Basically, product team set out some goals for the year, and we pointed out early on that feature X is going to have a ton of issues. Halfway through the year, my boss (the director) tells the product team we need to start feature X immediately or it's going to have risk of missing the EOY goals. Product team gets all the pre-reqs finished about 2 months before EOY (our "year" ends this month), and surprise surprise, there are tons of issues and we're likely to miss the deadline. Product team is freaking out about their bonuses, whereas I'm chuckling in the corner pointing to the multiple times we told them it's going to have issues.
There's a reason you hire senior engineers, and it's not to wave a magic wand and fix all the issues at the last minute, it's to tell you your expectations are unreasonable. The process should be:
If you skip some of those steps, you're going to have a bad time.
In my experience, the job of a sr. revolves around expectations. Expectations of yourself, of the customer, of your bosses, of your juniors and individual contributors working with you or that you're tasking. Managing the expectations and understanding how these things go to protect your guys and gals and trying to save management from poking out their own eyes.
And you may actually have time to do some programming.
Yup. I actually only take a 50% workload because half of my time is spent in random meetings telling people no, or giving obscenely high estimates that essentially amount to "no." The other half of my time is fixing problems from when they didn't listen when I said "no."
Such is life I guess. But occasionally, I get to work on something new. And honestly, that's fine, I've long since stopped caring about my name showing up on things.
Not all heroes wear capes. You're saving their butts, and they don't know it.
Can confirm. At our company, we have a tech debt budget, which is really awesome since we can fix the worst of the problems. However, we generate tech debt faster than we can fix it. Adding AI to the mix would just make tech debt even faster, because instead of senior devs reviewing junior dev code, we'd have junior devs reviewing AI code...
LLMs are not supposed to understand, they are supposed to pretend to understand.
You have to be hallucinating to understand.
I’ve licked the frog twice! How many does it take?
I take it that frog hadn't been de-boned.
"AI" is just good for simple code snippets. (Which it stole from Github repos).
This whole ai bs needs to die already, and the people who lie about it held accountable.
Sounds like the Sirius cybernetics corporation:
The fundamental design flaws are obscured by the superficial design flaws.
The point of the article isn't that AI is outright useless as a coding tool but that it lulls programmers into a false sense of security regarding the quality and security of their code. They aren't reviewing their work as frequently because of this new reliance on AI as a time saver, and as such are more likely to miss any mistakes that they or the AJ made.
Lulling them into a false sense of security is half of what makes it useless. The fact that it makes shitty code is the other half.
But the job of a software developer is not to write good code, it is to deliver features. People have been writing bad code without any AI for decades. Businesses often prioritize speed over quality, rewarding teams that deliver features quicker.
Now Even Faster™ with no exceptions thanks to "AI"
Now now, AJ may not know everything, but he'll learn
It basically just turns coders into debuggers.
Devs care to debug code only if they believe in its quality. Otherwise they write the code again from scratch. This is also cheaper than debugging.
I dare to say it: 70% of the devs are not quality focused to start with. They are already happy if something, somewhat, sort of, works. And then not even ship a unit test with it.
Everyone is QA now.
AI can be a useful tool, but it’s not a substitute for actual expertise. More reviews might patch over the problem, but at the end of the day, you need a competent software developer who understands the business case, risk profile, and concrete needs to take responsibility for the code if that code is actually important.
AI is not particularly good at coding, and it’s not particularly good at the human side of engineering either. AI is cheap. It’s the outsourcing problem all over again and with extra steps of having an algorithm hide the indirection between the expertise you need and the product you’re selling.
What a surprise, right?
Eaxctly my point as well here.. https://kbin.melroy.org/m/technology@lemmy.world/t/461269/-/comment/4095583
Debugging and maintenance was always the hardest aspect of large code bases... writing the code is the easy part. Offloading that part to AI only makes the hard stuff harder
I have a lot of empathy for a lot of people. Even ones, who really don't deserve it. But when it comes to people like these, I have absolutely none. If you make a chatbot do your corporate security, it deserves to burn to the ground
Also it is pure junk. Chat-GPT code may come out fast on the screen but it's garbage. I tried python and c++ both just pure garbage. Sure I got it to do what I wanted but only after a day of hair pulling repetitive madness. Simple task, open an image and invert it . Then we'll it opened the image but didn't invert. Or maybe it's upside down. Can you open the image right side up and invert it....fuck fuck, why is the window full screen? Did I ask for full screen, shit heavens no! Anyway it's a fuckin idiot just rambling code at me.
I use it for Ansible, so not for code, and just to reduce the time my brain is exposed to Ansible.
If all you said to me was open an image and invert it, I would probably turn it upside down as well. What are you trying to get it to do?
Probably make the bright pixels dark and the dark pixels bright.
So what they should have said is to make the image negative.
That's being a standard image editing function since the days of film but you have to use the correct terminology.
It's called invert in Photoshop:
https://www.adobe.com/creativecloud/photography/discover/invert-colors.html
It's just an example. I did get useful code from all this effort but usually the first prompt gets the closest. Everything else is like a bad genie story. Exactly like this: https://youtu.be/lM0teS7PFMo?si=yMtEaVkpSrn9q5Ap
Open it how using what at what size what codec where, for how long, for what purpose, using what data structures, use what libraries, what versions. You sound like my PO trying to request an update to software they have no comprehension of.
ahahahaha...
Good. Maybe if the stuff trashes enough of our infrastructure somebody somewhere will actually figure out that it's bad and get rid of it forever.
I know, it'll never happen. But a man can dream.
thinking Neoliberlism.
The thing I dislike most about code assisting tools is that they're geared to answering your questions instead of giving advice. I'm sure they also give bad recommendations but I've seen LLMs basically double down on bad code.
No they’re giving you exactly what you’re asking for. Problem is you’re not asking for advice. Your asking to “build a thing” and expecting it to read your mind.
No sh*t, this is what I predicted from day one.
We should have looked to melroy
Thank you! That is indeed a valid point. I was hoping more people came up with this valid remark. Do you have any other questions or predictions you would like to know? So that we don't get "surprises" in the field of technology again?
Please hit me with some predictions :D
Sure!
@Eheran@lemmy.world @RagingRobot@lemmy.world
#it #software #ai #predictions
So you predicted that security flaws in software are not going to vanish with AI?
All software has bugs. I prefer the human-generated bugs, they're much easier to diagnose and solve.
My point exactly, now you have genAI code written by AI, who doesn't know what it is doing. Instructed by a developer, who doesn't understand the programming language. Reviewed by a co-worker, who doesn't know what is doing on. It's madness I tell you!
I predicted that introducing AI on software engineer (especially juniors) will result in overall worse code, since apparently people don't feel responsible for the genAI code. While I believe the responsibility is still fully at the humans who try to deliver code. And on top of that, most devs are not doing good code reviews in general (often due to lack of time or .. skill issue). And now we have AI that generates code which are too easily accepted on top of reviewers who blindly accept code.. And no unit tests or integration tests.. And then we have this current situation. No wonder this would happen. If you are in software engineering, you would know exactly where I'm talking about. Especially if you would work at larger companies.
Lmao this is precisely how I thought this would turn out.
as opposed to human-generated code
But at least that crappy bug-riddled code has soul!
no common sense allowed in this thread, sir. only AI hate bandwagon please.
and here's me learning C programming language from a selfhosted AI :/
It's a great launching pad to learn how a language works, but beyond simple things, it get bad very fast.
I also use AI to look for terms in specific domains, which is really helpful as well.
Where’s the articles about humans doing the exact same shit for the last 40-50 fucking years and no one bats an eye. Looks at the prompts from people complaining about ai responses and see they don’t know how to use this shit any better than my grandparents can use a touchtone phone.
“Build an app”
Fails
“This ai is shit”.
Just like ever other piece of technology. Garbage in garbage out. If you can’t reliably describe what you want then no one is going to be able to do it. AI just blatantly points out your descriptive failures.
I've yet to see generative AI make an error that a human couldn't make. Maybe that's why people seem so hateful of it; they were expecting it to be superhuman but instead it's too much like us.
Ai llms have learned from us. Good and bad. It doesn't know the difference between good and bad unless you tell it.
So you have to know what's good or bad from the get go before using it and trusting it yet.
And some blindly trust ai already... Which its far from that level of trust
That's on them though. The other ones making the claim that it's supposed to be The Culture, but I don't think anyone at the companies is saying that it is.