Backdoor found in widely used Linux utility breaks encrypted SSH connections

Technology@lemmy.world – 667 points – 7 months ago

Backdoor found in widely used Linux utility breaks encrypted SSH connections

The malicious changes were submitted by JiaT75, one of the two main xz Utils developers with years of contributions to the project.

“Given the activity over several weeks, the committer is either directly involved or there was some quite severe compromise of their system,” an official with distributor OpenWall wrote in an advisory. “Unfortunately the latter looks like the less likely explanation, given they communicated on various lists about the ‘fixes’” provided in recent updates. Those updates and fixes can be found here, here, here, and here.

On Thursday, someone using the developer's name took to a developer site for Ubuntu to ask that the backdoored version 5.6.1 be incorporated into production versions because it fixed bugs that caused a tool known as Valgrind to malfunction.

“This could break build scripts and test pipelines that expect specific output from Valgrind in order to pass,” the person warned, from an account that was created the same day.

One of maintainers for Fedora said Friday that the same developer approached them in recent weeks to ask that Fedora 40, a beta release, incorporate one of the backdoored utility versions.

“We even worked with him to fix the valgrind issue (which it turns out now was caused by the backdoor he had added),” the Ubuntu maintainer said.

He has been part of the xz project for two years, adding all sorts of binary test files, and with this level of sophistication, we would be suspicious of even older versions of xz until proven otherwise.

Dude seems like a foreign asset

Jia Tan, University of Hong Kong in China. He’s been the sole maintainer of the package for almost two years.

Looks like he'd done a lot for various US companies on his LinkedIn.

I would not be surprised if he was previously legit but pressured into doing this by the CCP.

Maybe he wasn't sloppy by accident if he was indeed coerced by someone. I don't think we'll ever find out the backstory of this though.

I’ve watched a rundown of what the backdoor does. It’s impossible that this was an accident. It hides a compiled library in test data and injects that into the ssh binary to override code there.

They didn't mean the backdoor was (or was not) an accident. They meant the backdoor was implemented sloppily enough to be discovered and maybe that was not an accident (as in, he wanted it to be found, but still wanted to plausibly be seen as trying his best to keep those coercing him appeased)

It would make more sense to compromise developers in trusted positions, or steal their credentials, than going through the time and effort of building trusted users and projects only to burn them with easily spotted vulnerabilities.

This wasn't easily spotted. They use words like sloppy, but it all started with someone digging in because starting ssh season was about a half second slower that it used to be. I could easily imagine 99.99% of people shrugging and deciding just something in the chain of session startup took a bit longer for a reason not worth digging into.

Also, this was a maintainer that just started two years ago. xz is much older than that, just he took over.

foreign to whom?

his mother. he hasn't called in a week, he doesn't write.

Maybe she's a bitch.

From the article...

Will Dormann, a senior vulnerability analyst at security firm Analygence, said in an online interview. “BUT that's only because it was discovered early due to bad actor sloppiness. Had it not been discovered, it would have been catastrophic to the world.”

Is auditing for security reasons ever done on any open source code? Is everyone just assuming that everyone else is doing it, and hence no one is really doing it?

EDIT: I'm not attacking open source, I'm a big believer in open source.

I'm just trying to start a conversation about a potential flaw that needs to be addressed.

Once the conversation was started I was going to expand the conversation by suggesting an open source project that does security audits on other open source projects.

Please put the pitchforks away.

Edit2: This is not encouraging.

You're making a logical fallacy called affirming the consequent where you're assuming that just because the backdoor was caught under these particular conditions, these are the only conditions under which it would've been caught.

Suppose the bad actor had not been sloppy; it would still be entirely possible that the backdoor gets identified and fixed during a security audit performed by an enterprise grade Linux distribution.

In this case it was caught especially early because the bad actor did not cover their tracks very well, but now that that has occurred, it cannot necessarily be proven one way or the other whether the backdoor would have been caught by other means.

Also they are counting the hits and ignoring the misses. They are forgetting that sneaking a backdoor into an open source project is extremely difficult because people are reviewing the code and such a thing will be recognized. So people don't typically try to sneak back doors in. Also, backdoors have been discovered in an amazing amount of closed source projects where no one was even able to review the code.

They are forgetting that sneaking a backdoor into an open source project is extremely difficult because people are reviewing the code and such a thing will be recognized.

Everyone assumes what you have stated, but how often does it actually happen?

How many people, and how often, and how rigorous, are code reviews actually done? Especially with large volume projects?

Depends on the project, but for a lot of projects code review is mandatory before merging. For XZ the sole maintainer can do whatever they want.

Depends on the project, but for a lot of projects code review is mandatory before merging. For XZ the sole maintainer can do whatever they want.

I've done plenty of code reviews in my time, and I know one thing, the more busy you are, the faster you go through code reviews, and the more chance things can be missed.

I would hope that for the real serious shit (like security) the code reviews are always thorough and complete, but I know my fellow coding brethren, and we all know that's not always the case. Time is a precious resource, and managers don't always give you the time you need to do the job right.

Personally I use a distro backed indirectly by a corporation and hope that each release gets the thorough review that it needs, but human nature is always a factor in these things as well, and honestly, there are times when everyone thinks everyone else is doing a certain task, and the task falls between the cracks.

Reviewing the code was irrelevant in this case because the back door only existed in the binaries.

It's maybe possible, but perhaps even unlikely still.

Overwhelmingly thorough security review is time consuming and expensive. It's also not perfect, as evidenced by just how many security issues accidentally live long enough to land Even in enterprise releases. That's even without a bad actor trying to obfuscate the changes. I think this general approach had several aspects that would made it likely to pass scrutiny:

It was in XZ, which was likely not perceived as a security critical library. A security person would recognize any thing as potentially security critical, but they don't always have the resources and so are directed to focus on obviously security related and historically security incident magnets.
it was carried out by someone who spent years building up an innocuous reputation. Investigation may even show previous "test samples" to be malicious but not caught, or else it was a red herring to get people used to random test samples getting placed in the project.
The only "source code" he touched was "just build scripts". Even during a security audit, build shell scripts are likely going to be ignored, they are just build scripts and maybe you run some tests on all scripts, but those tests aren't going to catch this sort of misbehavior.
The actual runtime malicious code was delivered as portions of ostensibly throw away test sample xz files. The malicious code is applied by binary patch of the build output. A security audit won't be thinking too hard about a sea of binary files that are just throwaway samples as fodder for test.

So while I see the point about logical fallacy about it accidentally not getting far enough to see if the enterprise release process would have caught it, I think we know track records well enough to deem this approach likely to get through. Now that it has been caught, I could see some changes that may mitigate this in the future. Like package build scripts deleting all test samples and skipping tests when building for release, as well as more broad scrutiny.

There's also the reality that a lot of critical applications deem themselves too cool to settle for "old crusty enterprise distributions". They think that approach is antiquated and living on the edge is better. Admittedly I doubt theyd go as far as arch, tumbleweed, or rawhide, but this one could have easily made it to Debian testing, fedora release, or an Ubuntu release.

I think we know track records well enough to deem this approach likely to get through.

That was my concern, and why I brought up my point.

Human nature, especially when volunteer work versus paid work is being done, as well as someone who purposely over the long-term is trying to be devious, could be a potent combination for disaster.

I still wonder if there should be an actual open source project that does nothing but security audits of all other open source projects, hence my original question as an opener to a conversation that I never got to elaborate on because I was getting attacked almost immediately by people who are very sensitive about bringing any criticisms/concerns about open source out in the open.

The issue is that it implies that open source has a problem due to volunteers that is not found in closed source, which is not really the reality.

You can look at a closed source vendor like Cisco and see backdoors, generally left over from developer access, yet open for abuse. The nature of those is so blatantly obvious any open review would have spotted it instantly, yet there it was

With this, you had a much more device obfuscated attack that probably would have passed through even serious security audits unnoticed, yet it was caught because someone was curious about a slight performance degradation and investigated. Having been in the closed source world,I can tell you that they never would have caught someone like this. Anyone even vaguely saying they wanted to spend some time investigating a session startup delay of half a second would be chastised for wasting time.

Further, open source projects are also the fodder for security researchers to build their resumes. Hard to prove your mettle without works, and catching vulnerabilities in OSS code is a popular feather in their cap.

It also implies that open source is strictly a volunteer affair. Most commercial applications of a Linux platform involve paid employees doing some enablement, and that differs place to place. There's of course red hat paying for security research, Google, Microsoft also. I know at least one company that distrusts everything and repeats a whole bunch of security audits, including paying external companies to audit open source code. I would wager that folks downstream of say centos stream or certain embedded platforms can feel pretty good about audits. Of course all bets are off when you go grab yarballs, npm, pip, etc.

The issue is that it implies that open source has a problem due to volunteers that is not found in closed source, which is not really the reality.

I (partially) disagree. Fundamentally, my belief is that someone who gets paid to do the work is more rigorous doing the work than someone who does it on a volunteer basis, a human nature thing. Granted, I'm speaking very generally, and what I stated is not always true, but still.

Also, corporations that write close source programs are much more legally adverse to being sued if their product fails (there's a reason why we're seeing so many corporations slapping in arbitration clauses into their agreements these days; risk-averse).

Open source projects tend to just be more careful about their code base not being tainted, and write in disclaimers ("As-is") to protect themselves legally for the failure of the product scenario, and call it a day (again, very generally speaking (I use Fedora specifically for a reason)).

And speaking of Fedora, I do agree with your point that some open source projects are actually done by paid coders. I just believe that's more of the outlier, than the norm, though. Some of that work is done by corporate employees, but still on a volunteer basis.

Not dismissing at all, I am thankful for corporations that actually spend time letting their employees do open source work, even if it's just for their own direct benefit, as it also benefits everyone else.

Having worked with closed source, whatever they project externally, internally they are generally lazy and do the bare minimum. If there is a security review, it might just be throwing it at something like bdba that just checks dependencies against cve. Maybe a tool like coverity or similar code analysis. That's about as far as a moderately careful closed source so goes. It is exceedingly rare for them to fund folks to endlessly fiddle with the code looking for vulnerabilities, and in my experience actively work to rationalize away bugs if possible, rather than allocating time to chasing root cause and fix.

There may be paragons of good closed source development, but there are certainly bad ones. Same with open source.

I also think most open source broadly is explicitly employee work nowadays. Not just hobbyist, except for certain niches.

internally they are generally lazy and do the bare minimum.

Day to day, and with a lazy manager who is not technically knowledgeable, I would agree, and they do existence in corporations.

But if you work for one who knows what they're doing and gets a mandate from their boss to make sure the code doesn't leave the corporation legally exposed, then not so much.

Also special events like Y2K also gets extra scrutiny for legal reasons way up and above the normal level scrutiny thing production code gets.

I've worked it both types throughout my career.

The same argument can be made about open source, some projects are very carefully and festidously managed, and others not so much.

Main difference is with closed source, it's hard to know which sort of situation your are dealing with, and no option for an interested third party to come along and fix a problematic project.

https://hachyderm.io/@drmorr/112193633428121537

That link doesn't prove whatever you think it's proving.

The open source ecosystem does not rely (exclusively) on project maintainers to ensure security. Security audits are also done by major enterprise-grade distribution providers like Red Hat Enterprise. There are other stakeholders in the community as well who have a vested interest in security, including users in military, government, finance, health care, and academic research, who will periodically audit open source code that they're using.

When those organizations do their audits, they will typically report issues they find through appropriate channels which may include maintainers, distributors, and the MITRE Corporation, depending on the nature of the issue. Then remedial actions will be taken that depend on the details of the situation.

In the worst case scenario if an issue exists in an open source project that has an unresponsive or unhelpful maintainer (which I assume is what you were suggesting by providing that link), then there are several possible courses of action:

Distribution providers will roll back the package to an earlier compatible version that doesn't have the vulnerability if possible
Someone will fork the project and patch the fix (if the license allows), and distribution providers will switch to the fork
In the worst case scenario if neither of the above are possible, distribution providers will purge the vulnerable package from their distributions along with any packages that transitively depend on it (this is almost never necessary except as a short-term measure, and even then is extremely rare)

The point being, the ecosystem is NOT strictly relying on the cooperation of package maintainers to ensure security. It's certainly helpful and makes everything go much smoother for everyone if they do cooperate, but the vulnerability can still be identified and remedied even if they don't cooperate.

As for the original link, I think the correct takeaway from that is: If you have a vested or commercial interest in ensuring that the open source packages you use are secure from day zero, then you should really consider ways to support the open source projects you depend on, either through monetary contributions or through reviews and code contributions.

And if there's something you don't like about that arrangement, then please consider paying for licenses on closed-source software which will provide you with the very reassuring "security by sticking your head in the sand", because absolutely no one outside the corporation has any opportunity to audit the security of the software that you're using.

That link doesn’t prove whatever you think it’s proving.

That link strengthens my argument that we're assuming because it's open source that the code is less likely to have security issues because it's easier to be audited, when in truth it really just depends on the maintainer to do the proper level of effort or not, since it's volunteer work.

When someone suggested a level of effort to be put on code checked in to prevent security issues from happening, the maintainer pushed back, stating that they will decide what level of effort they'll put in, because they're doing the work on a volunteer basis.

And my rebuttal is three-fold:

Security does not depend entirely on the maintainer, and there is recourse even in the worst case scenario of an uncooperative or malicious maintainer.
The maintainer you quoted said he would be open to complying with requests if the requesters were willing to provide monetary support. You are intentionally misrepresenting their position.
The alternative of closed source software doesn't actually protect you from security issues, it just makes it impossible for any users to know if the software has been compromised. For all you know, a closed source software product could be using one of the hypothetical compromised open source software project that you're so afraid of, and you would never actually know.

If you're willing to pay a license for a private corporation's closed source software so you get the pleasure of never being able to know your security posture, then why would you be unwilling to financially support open source developers so they have the resources they need to have the level of security that you'd like from them?

You are intentionally misrepresenting their position.

No I'm not. Or you're assuming my position incorrectly.

You're either intentionally misrepresenting the post or you failed to understand them correctly. I'll let you take your pick for which is less embarrassing for you.

You’re either intentionally misrepresenting the post or you failed to understand them correctly.

You're incorrectly seeing more into what I'm saying than I'm actually saying, probably because you are very invested in defending Linux, and interpret what I'm saying as an attack on Linux.

For what its worth, I'm not attacking Linux. I use Linux as my daily driver (Fedora/KDE).

The key sentence in the post you linked which constituted more than 50% of the words being stated by the poster and yet you somehow conveniently missed which completely negates the whole narrative that you're trying to promote:

Speaking as an open source maintainer, if a tech company would like to pay me to do ~anything for my open source project, we can sit down and talk about my rates.

Which means this person is NOT simply a volunteer as you insinuated here:

When someone suggested a level of effort to be put on code checked in to prevent security issues from happening, the maintainer pushed back, stating that they will decide what level of effort they'll put in, because they're doing the work on a volunteer basis.

but in fact is available to be paid a fair rate for the labor they perform. In fact your entire description of the post is mischaracterizing what is being said in the post.

I don't know how you could have accidentally missed or misinterpreted one of the two sentences being said by the poster, and the longer of the two sentences at that. It was also the first sentence in the poster's statement. It seems more likely to me that you missed that on purpose rather than by accident. Maybe you're just so eager to find evidence to match your narrative that your brain registered the entire point of the post incorrectly. Allow me to reframe what's being said to simplify the matter:

As a self-employed contractor, if you demand that I perform free labor for you, I will decline that request.

Now just add a much more frustrated tone to the above and you get the post you linked.

Which means this person is NOT simply a volunteer as you insinuated here:

You're missing this part of what they said, take a second look (bolded part)...

if a tech company would like to pay me to do ~anything for my open source project, we can sit down and talk about my rates.

That means they haven't been paid yet, they're doing volunteer work, and they're soliciting publicly for pay to do the work that we would all expect volunteers to do already anyways, making sure their code is secure, which, is my point.

And the rest of that quote...

Otherwise they can fuck right off and I'm going to do what I want with my project.

They're signaling publicly that since they're not getting paid to do the work they can do any level of effort, not just the required (security wise) effort.

We shouldn't assume that full diligent effort is being done to secure the code, just because it's open source and easily readable by anyone. Doesn't matter if there's easy access if no one ever actually looks at it.

I'm not saying it's never done, I'm just saying we should not assume it's always being done (my bet would be more often than not, it's not) and that is a real problem, as this story/situation demonstrates. Capitalism, human nature, and volunteer versus paid work efforts, based on available hours to do the job correctly.

I really wish you would just stop trying to defend Linux and open source development, and listen to the concept/opinion I'm actually stating, because it's really important for all of us that depends on open source efforts to be aware of it and act on it, not just stick our heads in the sand about it.

Your interpretation is simply not supported by the literal words being said by the person. "we can sit down and talk about my rates" implies that this person already has rates that they charge for the labor they do.

You're projecting a meaning into the person's words that simply aren't there because you want it to fit a narrative that has is not commensurate with reality.

You brought up your credentials earlier so now I'll bring up mine: My full time job, which I get paid a very competitive salary for, is to develop exclusively open source software. I have many collaborators in the industry, both at my same organization and from others (some non profits, some academic labs, some government agencies, but mostly private for-profit organizations) who contribute to open source projects either full time or part time.

I don't have one single collaborator who is the mythical unreliable open source volunteer you're talking about. Every single person I've worked with has a commercial or professional (i.e. academic, mission-driven) interest in the developmental health of open source software. When we decide what dependencies we use, we rule out anything that looks like a pet project or something with amateur maintenance because we know if the maintainer slacks off or goes rogue then that's going to be our problem.

The xz case is especially pernicious. This is a person who by all initial appearances was a respected professional doing respectable work. He/they (perhaps there was a team involved) went to great lengths to quietly infiltrate the ecosystem. I guarantee someone could do the same thing at a private company, but admittedly they're less likely to have as broad of an impact as they can by targeting the open source ecosystem.

I really wish you would just stop trying to defend Linux and open source development, and listen to the concept/opinion I'm actually stating

I am listening, and I'm telling you that you're wildly misunderstanding the nature of the open source industry. You, like many many other software developers, are ignorant about the vast bulk of widely used open source software gets developed.

5 more...

You’re making a logical fallacy called affirming the consequent where you’re assuming that just because the backdoor was caught under these particular conditions, these are the only conditions under which it would’ve been caught.

No, I'm actually making that comment based on a career as a software developer, who has actually worked on a few open source projects before.

Your credentials don't fix the logical fallacy.

Experience matters.

9 more...

14 more...

Having once worked on an open source project that dealt with providing anonymity - it was considered the duty of the release engineer to have an overview of all code committed (and to ask questions, publicly if needed, if they had any doubts) - before compiling and signing the code.

On some months, that was a big load of work and it seemed possible that one person might miss something. So others were encouraged to read and report about irregularities too. I don't think anyone ever skipped it, because the implications were clear: "if one of us fails, someone somewhere can get imprisoned or killed, not to speak of milder results".

However, in case of an utility not directly involved with functions that are critical for security - it might be easier to pass through the sieve.

I don’t think anyone ever skipped it, because the implications were clear: “if one of us fails, someone somewhere can get imprisoned or killed, not to speak of milder results”.

However, in case of an utility not directly involved with functions that are critical for security - it might be easier to pass through the sieve.

I've actually seen people checking in code that doesn't get reviewed properly on mission critical apps before (like in the health industry).

My understanding is basically the same as yours, and in theory I agree with you. However, the problem is we all tend to hand-wave away any possibility of bad things happening, because it's open source, and don't take into account human nature, especially when it comes to volunteer versus paid work.

Auditing can be done only on open source code. No code = no audit. Reverse engieneering doesn't count.

True, but does it actually get done, or just everyone just assuming gets done, because it's open source?

The answer is the same as closed source software: sometimes.

But that's beside the point, a security audit is not perfect. Plenty of audited codebases are the source of security vulnerabilities in the wild. We know based on analysis that the malicious actor's approach would have a high chance of successfully hiding from a typical security audit.

Oh I know security audits are not perfect, I'm just wondering if they actually get done, or everyone just assumes they get done because of "Open Source", but they don't.

There are security researchers looking for vulnerabilities constantly, but they're inconsistent and informal. Issues usually get caught eventually, but sometimes that's after a vulnerability in the wild.

14 more...

Thankfully this was discovered before hitting stable distros but I'm hoping it increases scrutiny across the board. We dodged a bullet on this one.

This is why I run debian oldstable.

Or maybe it's because I'm too lazy to do a dist-upgrade.

2 more...

Long game supply chain attacks, pretty much going to be state actors. And I wouldn't chalk it up to the usual malicious ones like China and Russia. This could be the NSA just as easily.

I honestly think the NSA has changed. If you look at the known backdoors they haven't got caught making any new backdoors since like 2010. Their MO also seems to be more hardware and encryption (more of an observational charter) than manipulation.

There's also evidence US Congress acted to stop the NSA from doing these underhanded tacits at least once https://www.wired.com/story/nsa-backdoors-closed/

They're not idiots, lots of smart people there that surely understand the risk of something like this to US national security interests. It's not the NSA that's been asking for encryption to be broken in recent years. They've been warning about quantum threats and ... from what I'm aware of actually been taking on the defensive role they were conducted to perform https://gizmodo.com/nsa-plans-to-act-now-to-ensure-quantum-computers-cant-b-1757038212

This seems like something that could actually be weaponized against predominantly western technology companies so I'd be very surprised if it was them and very surprised if they used someone that appears to be a Chinese born resident to do it.

I really can't believe they've stopped. Their mentality is "national security has no morals". They'll do everything they can do to facilitate that mission, though not getting caught is a big part of the facade they need to put on to keep or renovate their image to do this.

Maybe they're being more careful, and doing simple things like putting in timestamps that emulate working hours in other timezones are certainly the first thing they're going to think about. That one has always cracked me up, security researchers point to it like it's proof of something, which is ridiculous. Just like our people are smart, I don't think the foreign actors are dumb either.

And before you say it, I'd be all over not being paranoid if it hadn't been proven to me time and again that these agencies won't change, that they don't give a shit about what's right if it gets in the way of their mandate. The only thing that might change is how well they hide things now and intimidate their people into staying quiet. Because potential whistleblowers have seen the examples that have been made.

Personally I suspect they're getting all the information they care about via subpoenas on big data and social media companies. They don't have a need to compromise security on a technical level anymore because the justice system itself is compromised. That means backdoors only benefit national enemies at this point, so the NSA of today would rather those not exist at all.

Of course that's not to say anyone should trust those agencies at their word on anything.

Backdoors at a mation-state level are a double edged sword. In order to successfully implement a backdoor, you need to ensure that you are more clever than your adversaries, because those same backdoors can be used against you. You must assume that they will eventually discover them, and be able to leverage them against you. Then you must be able to identify that it had been compromised, and then "responsibly disclose" the vulnerability before too much damage is done.

Much better to be on the defensive. Discover 0days first, either accidental or intentional, and then use them until someone else discloses them and they get patched to hell.

In order to successfully implement a backdoor, you need to ensure that you are more clever than your adversaries, because those same backdoors can be used against you.

In this instance, that's not the case. Only those in possession of the right key can use the backdoor. Also, discovering infected systems from the outside, appears to be impossible - the backdoor simply does not do anything to reveal itself if you don't have the key.

I must be mistaken then. I suppose keys have never, ever been compromised. Nobody has ever taken sensitive information without authorization, either. Especially not from the NSA!

You were talking about adversaries discovering the backdoor. That's something entirely different from compromised keys. So your sacrasm is quite misplaced here.

Columbus "discovered" America.

That's not true, Shadow broker leaks for example contained 0-day found by the NSA well after 2010. And that's only what got published, there's probably more !

There is a difference from finding something you can take advantage of and putting it there though, no? This sounds like the former.

But still, it's a good point, thanks.

Ah sorry, english is not my native language so I'm not sure I fully got what you meant, your point was that they stopped inserting backdoors and instead concentrated on getting access by finding vulnerabilities ?

Basically two points, they stopped inserting backdoors and their backdoors seem to have only ever been to show them what's going on (so this just doesn't look like them to me).

I didn't really comment on "what they do now" as much. I think they do continue to spy, finding preexisting vulnerabilities is definitely one way to spy. I wouldn't be surprised if they report the worst ones in NATO systems to be repaired and keep the others for themselves.

They also tap into weak points like Google and Apple's notification services where things aren't end to end encrypted to gather information. I believe this was revealed recently.

Snowden I recall saying the modern NSA is more interested in metadata than what's actually in the message as well.

In general, I think they still do some shady stuff, but I don't think they do shady stuff that risks compromising a system. This exploit is quite literally a system compromise as (if I understand it correctly) it allows bypassing sshd authentication.

It's not the NSA that's been asking for encryption to be broken in recent years.

I remember 2013 backdoored crypto by NSA. If they get caught less doesn't mean they make less backdoors.

EIDT: it was discovered in 2007 and revoked as standard in 2014

Also they owned corporation that made backdoored crypto algos till 2018. And the only reason they stopped is FOIA.

I don't know man. Imagine you could have ssh access to every Debian and fedora server on the planet, and all you had to do was write tests for some compression library for 2 years and sneak in a clever patch. I'd guess such an exploit is worth millions. You wouldn't work 2 years for millions of dollars?

This is sophisticated but it doesn't have to be a state actor.

Yup. I think it's an independent hacker, probably hired by a state actor, but not a state actor themselves.

My understanding is that state actors generally look for exploits, not create them. I also think they'd be a little more clever than this.

If you throw enough money at the right person you can get shit done.

I think you are greatly underestimating FSB incompetense.

There are no known reports of those versions being incorporated into any production releases for major Linux distributions

...

A stable release of Arch Linux is also affected.

... BTW.

The malicious code is only thought to have affected deb/rpm packaging (i.e the backdoor only included itself with those packaging methods). Additionally, arch doesn't link ssh against liblzma which means this specific vulnerability wasn't applicable to arch. Arch may have still been vulnerable in other ways, but this specific vulnerability targeted deb/rpm distros

I liked the joke, but ya arch is not compromised. Check out this user's detailed comment.

https://feddit.de/comment/8782369

This is really bad.

The backdoor has existed for a month at least. Yikes.

https://www.openwall.com/lists/oss-security/2024/03/29/4

The backdoor appears to specifically target RSA public key authentication, so they must have had a target in mind that they know uses RSA keys.

I would highly recommend Curve25519, etc., just because such keys are faster and less common than RSA public-private keys in today's world. RSA 2048-bit keys are considered weak today, while the Curve25519 256-bit keys remain stronger. Also, the ChaCha20-Poly1305 cipher has an interesting backstory and doesn't necessarily need hardware acceleration (which, in theory, could be borked by the HW-vendor) to obtain good performance.

Unfortunately, some SSH front-ends don't play nice with Curve25519 public-private keys yet... (I'm pointing at the putty SSH client, but that may have improved from the last time I had to use it)

It would be Ed25519 for SSH:

$ ssh-keygen -t ed25519

Would another less complex answer simply be that many (most?) people and organizations use RSA because it was first and elliptic signing is not yet as prevalent?

Going with Occam’s Razor here…

Woosh? (Probably)

I genuinely don’t understand if I missed something here, and would love more explanation.

I'm assuming the original post you replied to was meant to be a joke, since, like you pointed out, many or most people use RSA. I assume (using Occam's Razor) that is more likely than them not knowing that and intending their post at face value.

It’s the 46 upvotes that have me concerned that many people do not in fact see it as a joke

A stable release of Arch Linux is also affected. That distribution, however, isn't used in production systems.

Shots fired!

It seems WSL Ubuntu and Kali are safe with versions 5.2.5 and 5.4.4 installed respectfully.

Damn, I installed mine disrespectful.

I thought about the same (disrespectful) as I typed it. I've already laughed with you!

Is that when you install it in a VM, as if to show it that it’s not a good enough little distro to boot directly on your hardware?

I suddenly feel so guilty!

Don't forget about openSUSE Tumbleweed! It's actually affected AFAIK.

I think the AI that wrote the article misunderstood.

Arch doesn't build from release tar balls, but straight from git. Arch also doesn't link sshd against liblzma. So while they've shipped the dirty version of xz utils, at least sshd is not affected.

It's possible that the dirty version affected some of the other things that link liblzma. Like a handful of kde components for example.

the AI that wrote the article

The linked article is by Dan Goodin from Ars Technica. He's not immune to mistakes, but he's been writing good articles about security for years.

Can we please not accuse everybody of being AI just because they made a mistake?

Can we please not accuse everybody of being AI

Suspicious. /s

Well, he's credited as the editor overseeing security stuff. Reading between the lines I'd say he's just taking responsibility for the articles correctness.

This article in particular is just so poorly written that you'd forgive me for assuming it wasn't man-written.

Arch didn't build from git until March 28th. See commit history for xz and lib32-xz packages.

https://gitlab.archlinux.org/archlinux/packaging/packages/xz/-/commit/881385757abdc39d3cfea1c3e34ec09f637424ad

https://gitlab.archlinux.org/archlinux/packaging/packages/lib32-xz/-/commit/9d411814d763dfaba969c4c16adc0ce9a131bb2e

Also, the malicious code only activated if it detected being built as dpkg or rpm.

Here is a more detailed FAQ about what happened:

https://www.tenable.com/blog/frequently-asked-questions-cve-2024-3094-supply-chain-backdoor-in-xz-utils

This is also a good summary of the timeline: https://boehs.org/node/everything-i-know-about-the-xz-backdoor

Please help me as a novice Linux user- is this something that comes preinstalled with Mint Cinnamon? And if so, what can I do about it?

As the other person said it’s likely that xz is already installed on your system, but almost certainly a much older version than the compromised one. It’s likely that no action is required on your part assuming you’ve not been downloading tarballs of bleeding edge software.

As the other person said, just keep doing updates as soon Mint recommends them (since it’s based on Ubuntu LTS, it’s a lot less likely to have these bleeding edge vulnerabilities).

Much appreciated, thank you.

You're good. Even if you do use xz and ssh the version with the vulnerability only made it's way to rolling release distros or beta version of distros like fedora 40

Welp, time to go check my version of fedora

40 is still in beta. 39 doesn't have the vulnerability

made its* way to

Hahaha

Thanks!

The library itself is very common and used by a lot of things (in this case it seems that the payload only activated when used by specific programs, like SSH).

What you can do about it is keep your system up-to-date using your distribution update mechanisms. This kind of thing, when found out, is usually fixed quickly in security updates. In Mint (which I don't use, but I believe is based on either debian or ubuntu, which uses dpkg/apt) security updates are flagged differently anc can be installed automatically, depending on your configuration.

tl;dr: keep your system up-to-date, it will keep known vulnerabilities away as much as it can;

In this case though the backdoor was added recently so updating could do the opposite of help here. Luckily I don't think any stable distros added the new version.

It was added recently, but at this point in the timeline, fixes are available for most mainstream distro at least. Except for rare cases where a fix can't be made available quickly, this kind of publicity is only done when a fix is broadly available. There are extreme cases of course, but in this case, it's fixed.

Thanks. I do my best to regularly update, so here's hoping it will not be a problem for me before an update fixes it!

2 more...

dpkg --list | grep xz

should return what version of xz package is on your system. Likely 5.4, in which case you should be okay.

It says "5.2.5-2ubuntu1." So I'll have to see about updating it.

EDIT: However, this says I should be safe: https://forums.linuxmint.com/viewtopic.php?t=416756

The affected versions are 5.6.0 to 5.6.1

ii xz-utils 5.2.5-2ubuntu1 amd64 XZ-format compression utilities On my latest Mint install.

which isn't an effected version, so you should be okay.

2 more...

openssh does not directly use liblzma. However debian and several other distributions patch openssh to support systemd notification, and libsystemd does depend on lzma.

It’s all systemds fault, got it.

Do it all? Then you did it all, motherfucker.

Had redhat's patch write READY=1 into $NOTIFY_SOCKET instead linking to libsystemd to do this, this backdoor would not be possible.

Technically it breaks libsystemd encryption, which is not used in upstream openssh. There are unofficial redhat patches that use this library instead of reading one enviroment variable and writing to one file.

This is the best summary I could come up with:

Researchers have found a malicious backdoor in a compression tool that made its way into widely used Linux distributions, including those from Red Hat and Debian.

An update the following day included a malicious install script that injected itself into functions used by sshd, the binary file that makes SSH work.

So-called GIT code available in repositories aren’t affected, although they do contain second-stage artifacts allowing the injection during the build time.

In the event the obfuscated code introduced on February 23 is present, the artifacts in the GIT version allow the backdoor to operate.

“This could break build scripts and test pipelines that expect specific output from Valgrind in order to pass,” the person warned, from an account that was created the same day.

The malicious versions, researchers said, intentionally interfere with authentication performed by SSH, a commonly used protocol for connecting remotely to systems.

The original article contains 810 words, the summary contains 146 words. Saved 82%. I'm a bot and I'm open source!

So-called GIT code available in repositories aren’t affected

I wonder what convinced the model to treat git as an acronym

I imagine many aren't familiar with British slang and therefore assume git must stand for something, especially considering software devs love their acronyms.

It was like that in the original article. Or are you saying that the original was written by an AI too (it might be).

Who wants to bet he received a nice lump sum deposit of cash from a five eyes state to make an "accident"...

1 more...

And that's why you cannot trust open source software blindly.

And yet with closed-source software you have no choice but to trust it blindly. At least open source software has people looking at the code.

You are an idiot. It's not blind. That's how it was found.

Not having world accessible SSH is the real fix here.

Yeah I nearly panicked for a second there, then I remember noone's getting near that anyway. Back to my relaxing weekend.

You are an idiot. It’s not blind. That’s how it was found.

From the article...

Will Dormann, a senior vulnerability analyst at security firm Analygence, said in an online interview. “BUT that's only because it was discovered early due to bad actor sloppiness. Had it not been discovered, it would have been catastrophic to the world.”

The fact that it was discovered early due to bad actor sloppiness does not imply that it could not have also been caught prior to wide spread usage via security audits that take place for many enterprise grade Linux distributions.

I think it does though. They call it sloppy, I call it sophisticated. Same reason they major distro is running checking shit out right now.

Opensource = fast detection

Opensource + sloppiness = faster detection

Closedsource = never detected

Closedsource + sloppiness = maybe detected

You can put the pom-poms/rifle down, I'm not attacking open source, not in the slightest. I'm a big believer open source.

But I also know that volunteer work is not always as rigorous as when paid for work is being done.

The only point I'm trying to make in this conversation is getting confirmation if security audits are actually done, or if everyone just thinks they're done because of "Open Source" reasons.

As opposed to what? If you said "thats why you cannot trust any software blindly" it would have been not that wrong.

This is one hell of a take.

Certainly one of the takes of all time.

Imagine trying to make a helpdesk of a proprietary company take your "it's taking 0.5 seconds longer to login" complaint seriously...

So many vulnerabilities were found due to time to login that one of the security features was to take longer to respond to a bad login so they couldn't tell what part failed. Here's an article I found about one such vulnerability.

s/open source software/anything/

Reflections on Trusting Trust

- paper by Ken Tompson, 1984(not that one)

You can trust blindly whatever software you like. Most of us, even those that can code, trust blindly whatever software we use because we have other priorities. But what you can do only with open source software, is open your eyes if you choose.

Ftfy: And that's why you cannot trust people blindly.

Just because we cant observe the code of proprietary software and is sold legally doesn’t mean its all safe. Genuinely I distrust anything with a profit incentive.