Funny how CrowdStrike already sounds like some malware’s name.
It literally sounds like a DDoS!
Botnet if you will
Not too surprising if the people making malware, and the people making the security software are basically the same people, just with slightly different business models.
Reminds me of the tyre store that spreads tacks on the road 100m away from their store in the oncoming lanes.
People get a flat, and oh what do you know! A tyre store! What a lucky coincidence.
Classic protection racket. "Those are some nice files you've got there. It'd be a shame if anything happened to them..."
It sounds like the name of a political protest group.
This is, in a lot of ways, impressive. This is CrowdStrike going full "Hold my beer!" about people talking about what bad production deploy fuckups they made.
You know you’ve done something special when you take down somebody else’s production system.
*production systems around whole world
Few people can put that into their CVs, that a real achievement!
I'm volunteering to hold their beer.
Everyone remember to sue the services not able to provide their respective service. Teach them to take better care of their IT landscape.
Typically auto-applying updates to your security software is considered a good IT practice.
Ideally you'd like, stagger the updates and cancel the rollout when things stopped coming back online, but who actually does it completely correctly?
Applying updates is considered good practice. Auto-applying is the best you can do with the money provided. My critique here is the amount of money provided.
Also, you cannot pull a Boeing and let people die just because you cannot 100% avoid accidents. There are steps in between these two states.
you cannot pull a Boeing and let people die
You say that, but have you considered the savings?
People are temporary. Money is forever.
I have. They are not mine. The dead people could be.
Edit: I understand you were being sarcastic. This is a topic where I chose to ignore that.
That's totally fair. :)
I work at a different company in the same security space as cloudstrike, and we spend a lot of time considering stuff like "if this goes sideways, we need to make sure the hospitals can still get patient information".
I'm a little more generous giving the downstream entities slack for trusting that their expensive upstream security vendor isn't shipping them something entirely fucking broken.
Like, I can't even imagine the procedureal fuck up that results in a bsod getting shipped like that. Even if you have auto updates enabled for our stuff, we're still slow rolling it and making sure we see things being normal before we make it available to more customers. That's after our testing and internal deployments.
I can't put too much blame on our customers for trusting us when we spend a huge amount of energy convincing them we can be trusted to literally protect all their infrastructure and data.
You seem knowledgable. I’m surprised that it’s even possible for a software vendor to inject code into the kernel. Why is that necessary?
The kernel is responsible for managing hardware and general low-level system operations. Anything that wants to do those things needs to get itself into kernel mode one way or another.
The typical way you do this is called a "driver" and no one thinks about them as being kernel code. Things like graphics cards and the like.
Things that want to do actions like monitor network traffic or filesystem activity system wide or in a lower level capacity than the normal tools provide also need to be kernel level.
In a security context, that specifically would include things that want to monitor raw packets rather than the parsed content that assumes the packet is well formed in a way that a malicious one might not be.
Cloudstrike does the same thing on Linux, and the typical tools for network management or advanced security are also either compiled in or loadable kernel modules.
It's easy to forget that ip/ebtables and selinux and friends are kernel level software frequently distributed as kernel modules, in the case of the firewalls, or compiled in with a special framework and not just user mode software.
Not who you asked, but did you ever hear of Valiant and their kernel level anti cheat.
This is not a 1:1 comparison but anticheat software running in the kernel has the ability to monitor all other processes due to its permission levels. It can monitor all scheduled tasks and infer from that information.
Drivers need similar access but for different reasons, they need access to os functionality a user would absolutely never be granted. This is because they interface directly with hardware and means when drivers crash, they generally don't do it gracefully. Hence the BSOD loop and the need for booting windows without drivers (i.e. safe mode) and the deletion of the misconfiguration file.
TL;DR: Because the underlying OS is garbage.
Whatever CrowdStrike's "features" are should already be core security features of the kernel itself, or be exposed/extracted into user space.
NT was supposed to be a micro kernel. That this tool injects itself into the kernel immediately compromises the kernel. Edit: I should point out that it seems that CS injects drivers into the Linux kernel too, it might just be that Linux handles a driver crash more elegantly.
No different to the gaming anti-cheat kernel crap.
Having a "security" tool immediately compromise your actual security is absurd.
I'd love to know how you plan to do user mode packet filtering. Keep in mind that on Linux, the designated API is inherently kernel mode. https://netfilter.org/
This isn't one of the cases where we're talking about Linux being superior to windows. Any OS will be fucked if you give it a mangled kernel module. In this case, it's just that only one got one.
Your perception that anything that touches the kernel is an intrinsic security risk is unfounded.
I, too, work in a similar type of company, and can confirm from experience that Linux can get just as absolutely fucked up by a bad kernel module as windows.
And it's not just changes to the module that can cause things to go wrong.
For example, the kernel released alongside the latest Ubuntu LTS included a change that conflicted with our module behaviour, so machines with that kernel or newer would panic on boot.
It was a super minor change, but when you're deep in the weeds, it's really easy for these things to be brittle. But that's just an inherent consequence of the fact that this sort of stuff is intrinsically low-level interaction with the OS itself.
I can put the blame to your customers. If I make a contract with a bank they are responsible for my money. I don't care about their choice of infrastructure. They are responsible for this. They have to be sued for this. Same for hospitals. Same for everyone else. Why should they be exempt from punishment for not providing the one service they were trusted to provide? Am I expected to feel for them because they made the "sensible choice" of employing the cheapest tools?
This was a business decision to trust someone external. It should not be tolerated that they point their fingers elsewhere.
Can't fault you for feeling that way. I definitely don't think anyone should be exempt from responsibility, I meant blame in the more emotional "ugh, you jerk" sense.
If someone can't fulfill their responsibilities because someone they depended on failed them, they're still responsible for that failure to me, but I'm not blaming them if that makes any sense.
Power outage or not, the store owes me an ice cream cake and they need to make things even between us, but I'm not upset with them for the power outage.
You can be reasonable in your choice of words, but there are heads that need to roll. In this case it is not the one pushing the final button, but all those that created this system. Developers, Project Managers, Team Leaders, all the way up to the CEO. If the space to work in is so limited that the possibility of such pushes seems like a tolerable idea, then everything leading to this is broken. And people need to invest to make this right. Therefore there needs to be incentives, good and bad. To steer out of the current course there need to be very unfavorable incentives.
You can mock my argument by giving a ridiculous example. Once people die it will be too late. It's why there was a time where people thought it to be a good idea to employ giant generators to keep the power in a hospital running even in case of a power outage. Or to have redundant systems in an airplane.
There is a need for adequate standards in the software world. Trusting businesses to create them will evidently kill people. Creating something like certificates for personal skills and products is severely lacking.
I wasn't mocking your argument, I was agreeing with you and clarifying that my feeling was about who I'm most "irritated" with, not about responsibility or legal culpability.
My example was for simplicity, not mockery.
The power going out is the power companies fault, so I'm most mad at them. The store didn't have a generator because they trusted the power company, so my cake got ruined. I'm still mad at them but less so because they weren't the cause of the problem, even though they could have done more to prevent this from impacting me.
Culpability wise, I can only make demands of the store and hope that enough other people do so that they in turn demand answers from the power company.
There are actually a fair number of certifications, including ones from government agencies, relating to software development, deployment, and related practices. That so many organizations didn't have the ones relating to protection from supply chain issues is distressing, to say nothing of it slipping through quality control in the first place.
Please, if you think we're in a place in this thread where I'd be mocking you, re-read it with the understanding that I agree with you entirely on legal and structural issues, and at most just have a different opinion about where the balance of "fuck you"s go. I think I put more scorn towards the vendor because doing the thing is worse than failing to prevent the thing. Also, I work at a parallel company and so I'm more familiar with exactly how much you have to be fucking up for this to happen because I spent the last three days dealing with the more minor controls that prevent this from happening. Everyone has outages because you can't prevent 100% of errors, but it's on the vendor to build to the spec of their most sensitive customer and ensure that outages don't keep a doctor from patient records.
I wasn't mocking your argument, I was agreeing with you and clarifying that my feeling was about who I'm most "irritated" with, not about responsibility or legal culpability.
Okay, sorry for that. It happens to me sometimes to be mocked without me seeing prior cause for this. Thank you for clarifying that.
If a shop can't sell me cakes, then it's inconvenient. If a hospital is not able to keep people alive, that's where things get intolerable. Them not having access to their PCs is a hospital thing. If they cannot use them they should not use them. If it's a cost saving measure at the cost of people's lives, then I want heads to roll. Literally, preferably.
For the icecream, yes. If I want icecream and the shop doesn't have any because of a power grid failure, then I blame the power company more. The generator would be overkill, as it needs constant maintanance and checkups; immense running costs. This would not be justifiable for something like ice cream.
The hospital needs to be way more thorough with their supply chains. This discrepancy of responsibilities towards patients/customers is why I thought I was mocked, sorry again for that.
I called the certification processes "lacking" because they are very often out of date, if at all applied, like you said. The timeframe for product certifications needs to be drastically reduced for software products. I am aware that those checks need time the developers often don't have, but that doesn't matter. If that is a crucial issue, then they should stay the fuck away from critical infrastructure.
I'm actually willing to believe that CrowdStrike was actually compromised by a bad actor that realised how fragile CS was.
Can't get hacked if your machine isn't running.
You’re hired!
Idk boss, people weren't too happy the last time we tried that.
The answer is obviously to require all users to change their passwords and make them stronger. 26 minimum characters; two capitals, two numbers, two special characters, cannot include '_', 'b' or the number '8', and most include Pi to the 6th place.
Great! Now when I brute force the login, I can tell my program to not waste time trying '_', 'b' and '8' and add Pi to the 6th place in every password, along with 2 capitals, 2 numbers and 2 other special characters.
Furthermore, I don't need to check passwords with less than 26 characters.
Sorry, I don’t understand. Do you mean there have to be 6 digits of Pi in there, or the sixth character must be π? I’m down either way.
We won't tell you, and the rule gets re-rolled every 14 seconds. It may stay the same or it may change.
Also, there are requirements we check for that we don't tell you about! 🤭
The modern direction is actually going the other way. Tying identity to hardware, preventing access on unapproved or uncompliant hardware. It has the advantage of allowing biometrics or things like simple pins. In an ideal world, SSO would ensure that every single account, across the many vendors, have these protections, although we are far from a perfect world.
SSO means you only need to compromise one piece of hardware to get access to everything.
Effectively, the other option is passwords, and people are really, really, bad at passwords. Password managers help, but then you just need to compromise the password manager. Strong SSO, backed by hardware, at least makes the attack need to be either physical, or running on a hardware approved by the company. When you mix that with strong execution protections, an EDR, and general policy enforcement and compliance checking, you get protection that beats the pants off 30 different passwords to 30 different sites, or more realistically, 3 passwords to 30 different sites.
Yes, much better than 3-30 passwords.
But I view SSO as enterprise password manager with a nice UI. I don't trust it for anything super important.
What’s the saying about dying a hero or becoming the villain?
Maybe this is a case of hindsight being 20/20 but wouldn't they have caught this if they tried pushing the file to a test machine first?
It's not hindsight, it's common sense. It's gross negligence on CS's part 100%
Well, it is hindsight 20/20... But also, it's a lesson many people have already learned. There's a reason people use canary deployments lol. Learning from other people's failures is important. So I agree, they should've seen the possibility.
I saw one rumor where they uploaded a gibberish file for some reason. In another, there was a Windows update that shipped just before they uploaded their well-tested update. The first is easy to avoid with a checksum. The second...I'm not sure...maybe only allow the installation if the windows update versions match (checksum again) :D
Windows has beta channels for their updates
It's a sequence of problems that lead to this:
The kernel driver should have parsed the update, or at a minimum it should have validated a signature, before trying to load it.
There should not have been a mechanism to bypass Microsoft's certification.
Microsoft should never have certified and signed a kernel driver that loads code without any kind signature verification, probably not at all.
Many people say Microsoft are not at fault here, but I believe they share the blame, they are responsible when they actually certify the kernel drivers that get shipped to customers.
A real Anakin arc right here.
Now threat actors know what EDR they are running and can craft malware to sneak past it. yay(!)
Smart threat actors use the EDR for distribution. Seems to be working very well for whoever owned Solar Winds.
SHOULD'VE USED OPENBSD LMAO
Yes but the difference is one of them is also going to help you fix it.
Netflix knew they were going to move from DVD rentals to streaming over the Internet. It is right in their name.
CrowdStrike knew they were eventually going to _________. It is right in their name.
ItS NoT A wInDoWs PrObLeM -- Idiots, even on Lemmy
I genuinely can't tell at whom you are addressing this. Those claiming it is a Windows problem or those that say otherwise?
You can not like windows, and also recognize that CrowdStrike isn't from Microsoft - so a problem that CrowdStrike caused isn't the fault of Windows.
If that makes me a idiot by holding two different ideas in my head, so be it, but you are spending time with us, so thank you for elevating us!
I'm sorry, but distinguishing between different concepts is forbidden here. You go straight to jail.
I'm waiting for the post mortem before declaring this to not be anything to do with MS tbh. It's only affecting windows systems and it wouldn't be the first time dumb architectural decisions on their part have caused issues (why not run the whole GUI in kernel space? What's the worst that could happen?)
I agree it's possible. But if you're a software as a service vendor, it is your responsibility to be in the alpha and beta release channels, so if there is a show stopping error coming down the pipeline you can get in front of it.
But more tellingly, we have not seen Windows boot loop today from other vendors, only this vendor. Right now the balance of probabilities is in the direction of crowd strike
Ah yes the classic presumption of guilt, a keystone of justice
I'm not sure how to break this to you but this is just an internet forum, not a court of law
The reason courts use it is because they value having true opinions. But you're welcome to not value that indeed
The reason courts have rules of how convinced one must be to declare guilt is because they dread punishing an innocent over allowing a guilty person free
We aren't in a position to hurt the probably guilty party so it doesn't matter a bit of we jump to conclusions unfairly
Hi, idiot here. Can you explain how it is a windows problem?
It's a problem affecting Windows, not problem caused by Windows I guess.
That's what I was thinking so I was curious what the argument would be
If you patch a security vulnerability, who's fault is the vulnerability? If the OS didn't suck, why does it need a 90 billion dollar operation to unfuck it?
Crowdstrike completely screwed the pooch with this deploy but ideally, Windows wouldn’t get crashed by a bas 3rd party software update. Although, the crashes may be by design in a way. If you don’t want your machine running without the security software running, and if the security software is buggy and won’t start up, maybe the safest thing is to not start up?
Are we acting like Linux couldn't have the same thing happen to it? There are plenty of things that can break boot.
CrowdStrike also supports Linux and if they fucked up a Windows patch, they could very well fuck up a linux one too. If they ever pushed a broken update on Linux endpoints, it could very well cause a kernel panic.
Yeah, it's a crowd strike issue.
The software is essentially a kernel module, and a borked kernel module will have a lot of opportunities to ruin stuff, regardless of the OS.
Ideally, you want your failure mode to be configurable, since things like hospitals would often rather a failure with the security system keep the medical record access available. :/.
If they're to the point of touching system files, you're pretty close to "game over" for most security contexts unfortunately. Some fun things you can do with hardware encryption modules for some cases, but at that point you're limiting damage more than preventing a breach.
Architecture wise, the windows hybrid kernel model is potentially more stable in the face of the "bad kernel module" sort of thing since a driver or module can fail without taking out the rest of the system. In practice.... Not usually since your video card shiting the bed is gonna ruin your day regardless.
Like this is some highbrow collection of geniuses here?
No just statistically we are all Arch (btw) Linux users who hate Windows.
Because it isn't. Their Linux sensor also uses a kernel driver, which means they could have just as easily caused a looping kernel panic on every Linux device it's installed on.
There's no way of knowing that, though. Perhaps their Linux and Darwin drivers wouldn't have paniced the system?
Regardless, doing almost anything at the kernel level is never a good idea
It's not impossible. Crowdstrike has done it recently to linux machines.
Also, it's less about "their" drivers and more about what a kernel module can do.
Saying "there's no way to know" doesn't fit, because we do know that a malformed kernel module can destabilize a linux or mac system.
"Malformed file" isn't a programming defect or something you can fix by having a better API.
Having the data exposed to userspace via an API would avoid having to have a kernel module at all... Which when malformed wouldn't compromise the kernel.
I mean, sure. But typically operating systems don't expose that type of information to user space, instead providing a kernel interface with user mode configuration.
It's why they use the same basic approach on mac and Linux.
Security operations being one of the things that is often best done at the kernel level because of the need to monitor network and file operations in a way you can't in user mode.
Funny how CrowdStrike already sounds like some malware’s name.
It literally sounds like a DDoS!
Botnet if you will
Not too surprising if the people making malware, and the people making the security software are basically the same people, just with slightly different business models.
Reminds me of the tyre store that spreads tacks on the road 100m away from their store in the oncoming lanes.
People get a flat, and oh what do you know! A tyre store! What a lucky coincidence.
Classic protection racket. "Those are some nice files you've got there. It'd be a shame if anything happened to them..."
It sounds like the name of a political protest group.
This is, in a lot of ways, impressive. This is CrowdStrike going full "Hold my beer!" about people talking about what bad production deploy fuckups they made.
You know you’ve done something special when you take down somebody else’s production system.
*production systems around whole world
Few people can put that into their CVs, that a real achievement!
I'm volunteering to hold their beer.
Everyone remember to sue the services not able to provide their respective service. Teach them to take better care of their IT landscape.
Typically auto-applying updates to your security software is considered a good IT practice.
Ideally you'd like, stagger the updates and cancel the rollout when things stopped coming back online, but who actually does it completely correctly?
Applying updates is considered good practice. Auto-applying is the best you can do with the money provided. My critique here is the amount of money provided.
Also, you cannot pull a Boeing and let people die just because you cannot 100% avoid accidents. There are steps in between these two states.
You say that, but have you considered the savings?
People are temporary. Money is forever.
I have. They are not mine. The dead people could be.
Edit: I understand you were being sarcastic. This is a topic where I chose to ignore that.
That's totally fair. :)
I work at a different company in the same security space as cloudstrike, and we spend a lot of time considering stuff like "if this goes sideways, we need to make sure the hospitals can still get patient information".
I'm a little more generous giving the downstream entities slack for trusting that their expensive upstream security vendor isn't shipping them something entirely fucking broken.
Like, I can't even imagine the procedureal fuck up that results in a bsod getting shipped like that. Even if you have auto updates enabled for our stuff, we're still slow rolling it and making sure we see things being normal before we make it available to more customers. That's after our testing and internal deployments.
I can't put too much blame on our customers for trusting us when we spend a huge amount of energy convincing them we can be trusted to literally protect all their infrastructure and data.
You seem knowledgable. I’m surprised that it’s even possible for a software vendor to inject code into the kernel. Why is that necessary?
The kernel is responsible for managing hardware and general low-level system operations. Anything that wants to do those things needs to get itself into kernel mode one way or another.
The typical way you do this is called a "driver" and no one thinks about them as being kernel code. Things like graphics cards and the like.
Things that want to do actions like monitor network traffic or filesystem activity system wide or in a lower level capacity than the normal tools provide also need to be kernel level.
In a security context, that specifically would include things that want to monitor raw packets rather than the parsed content that assumes the packet is well formed in a way that a malicious one might not be.
Cloudstrike does the same thing on Linux, and the typical tools for network management or advanced security are also either compiled in or loadable kernel modules.
It's easy to forget that ip/ebtables and selinux and friends are kernel level software frequently distributed as kernel modules, in the case of the firewalls, or compiled in with a special framework and not just user mode software.
Not who you asked, but did you ever hear of Valiant and their kernel level anti cheat.
This is not a 1:1 comparison but anticheat software running in the kernel has the ability to monitor all other processes due to its permission levels. It can monitor all scheduled tasks and infer from that information.
Drivers need similar access but for different reasons, they need access to os functionality a user would absolutely never be granted. This is because they interface directly with hardware and means when drivers crash, they generally don't do it gracefully. Hence the BSOD loop and the need for booting windows without drivers (i.e. safe mode) and the deletion of the misconfiguration file.
TL;DR: Because the underlying OS is garbage.
Whatever CrowdStrike's "features" are should already be core security features of the kernel itself, or be exposed/extracted into user space.
NT was supposed to be a micro kernel. That this tool injects itself into the kernel immediately compromises the kernel. Edit: I should point out that it seems that CS injects drivers into the Linux kernel too, it might just be that Linux handles a driver crash more elegantly.
No different to the gaming anti-cheat kernel crap.
Having a "security" tool immediately compromise your actual security is absurd.
I'd love to know how you plan to do user mode packet filtering. Keep in mind that on Linux, the designated API is inherently kernel mode. https://netfilter.org/
This isn't one of the cases where we're talking about Linux being superior to windows. Any OS will be fucked if you give it a mangled kernel module. In this case, it's just that only one got one.
Your perception that anything that touches the kernel is an intrinsic security risk is unfounded.
I, too, work in a similar type of company, and can confirm from experience that Linux can get just as absolutely fucked up by a bad kernel module as windows.
And it's not just changes to the module that can cause things to go wrong.
For example, the kernel released alongside the latest Ubuntu LTS included a change that conflicted with our module behaviour, so machines with that kernel or newer would panic on boot.
It was a super minor change, but when you're deep in the weeds, it's really easy for these things to be brittle. But that's just an inherent consequence of the fact that this sort of stuff is intrinsically low-level interaction with the OS itself.
I can put the blame to your customers. If I make a contract with a bank they are responsible for my money. I don't care about their choice of infrastructure. They are responsible for this. They have to be sued for this. Same for hospitals. Same for everyone else. Why should they be exempt from punishment for not providing the one service they were trusted to provide? Am I expected to feel for them because they made the "sensible choice" of employing the cheapest tools?
This was a business decision to trust someone external. It should not be tolerated that they point their fingers elsewhere.
Can't fault you for feeling that way. I definitely don't think anyone should be exempt from responsibility, I meant blame in the more emotional "ugh, you jerk" sense.
If someone can't fulfill their responsibilities because someone they depended on failed them, they're still responsible for that failure to me, but I'm not blaming them if that makes any sense.
Power outage or not, the store owes me an ice cream cake and they need to make things even between us, but I'm not upset with them for the power outage.
You can be reasonable in your choice of words, but there are heads that need to roll. In this case it is not the one pushing the final button, but all those that created this system. Developers, Project Managers, Team Leaders, all the way up to the CEO. If the space to work in is so limited that the possibility of such pushes seems like a tolerable idea, then everything leading to this is broken. And people need to invest to make this right. Therefore there needs to be incentives, good and bad. To steer out of the current course there need to be very unfavorable incentives.
You can mock my argument by giving a ridiculous example. Once people die it will be too late. It's why there was a time where people thought it to be a good idea to employ giant generators to keep the power in a hospital running even in case of a power outage. Or to have redundant systems in an airplane.
There is a need for adequate standards in the software world. Trusting businesses to create them will evidently kill people. Creating something like certificates for personal skills and products is severely lacking.
I wasn't mocking your argument, I was agreeing with you and clarifying that my feeling was about who I'm most "irritated" with, not about responsibility or legal culpability.
My example was for simplicity, not mockery.
The power going out is the power companies fault, so I'm most mad at them. The store didn't have a generator because they trusted the power company, so my cake got ruined. I'm still mad at them but less so because they weren't the cause of the problem, even though they could have done more to prevent this from impacting me.
Culpability wise, I can only make demands of the store and hope that enough other people do so that they in turn demand answers from the power company.
There are actually a fair number of certifications, including ones from government agencies, relating to software development, deployment, and related practices. That so many organizations didn't have the ones relating to protection from supply chain issues is distressing, to say nothing of it slipping through quality control in the first place.
Please, if you think we're in a place in this thread where I'd be mocking you, re-read it with the understanding that I agree with you entirely on legal and structural issues, and at most just have a different opinion about where the balance of "fuck you"s go. I think I put more scorn towards the vendor because doing the thing is worse than failing to prevent the thing. Also, I work at a parallel company and so I'm more familiar with exactly how much you have to be fucking up for this to happen because I spent the last three days dealing with the more minor controls that prevent this from happening. Everyone has outages because you can't prevent 100% of errors, but it's on the vendor to build to the spec of their most sensitive customer and ensure that outages don't keep a doctor from patient records.
Okay, sorry for that. It happens to me sometimes to be mocked without me seeing prior cause for this. Thank you for clarifying that.
If a shop can't sell me cakes, then it's inconvenient. If a hospital is not able to keep people alive, that's where things get intolerable. Them not having access to their PCs is a hospital thing. If they cannot use them they should not use them. If it's a cost saving measure at the cost of people's lives, then I want heads to roll. Literally, preferably.
For the icecream, yes. If I want icecream and the shop doesn't have any because of a power grid failure, then I blame the power company more. The generator would be overkill, as it needs constant maintanance and checkups; immense running costs. This would not be justifiable for something like ice cream.
The hospital needs to be way more thorough with their supply chains. This discrepancy of responsibilities towards patients/customers is why I thought I was mocked, sorry again for that.
I called the certification processes "lacking" because they are very often out of date, if at all applied, like you said. The timeframe for product certifications needs to be drastically reduced for software products. I am aware that those checks need time the developers often don't have, but that doesn't matter. If that is a crucial issue, then they should stay the fuck away from critical infrastructure.
I'm actually willing to believe that CrowdStrike was actually compromised by a bad actor that realised how fragile CS was.
Can't get hacked if your machine isn't running.
You’re hired!
Idk boss, people weren't too happy the last time we tried that.
The answer is obviously to require all users to change their passwords and make them stronger. 26 minimum characters; two capitals, two numbers, two special characters, cannot include '_', 'b' or the number '8', and most include Pi to the 6th place.
Great! Now when I brute force the login, I can tell my program to not waste time trying '_', 'b' and '8' and add
Pi to the 6th placein every password, along with 2 capitals, 2 numbers and 2 other special characters.Furthermore, I don't need to check passwords with less than 26 characters.
Sorry, I don’t understand. Do you mean there have to be 6 digits of Pi in there, or the sixth character must be π? I’m down either way.
We won't tell you, and the rule gets re-rolled every 14 seconds. It may stay the same or it may change.
Also, there are requirements we check for that we don't tell you about! 🤭
The modern direction is actually going the other way. Tying identity to hardware, preventing access on unapproved or uncompliant hardware. It has the advantage of allowing biometrics or things like simple pins. In an ideal world, SSO would ensure that every single account, across the many vendors, have these protections, although we are far from a perfect world.
SSO means you only need to compromise one piece of hardware to get access to everything.
Effectively, the other option is passwords, and people are really, really, bad at passwords. Password managers help, but then you just need to compromise the password manager. Strong SSO, backed by hardware, at least makes the attack need to be either physical, or running on a hardware approved by the company. When you mix that with strong execution protections, an EDR, and general policy enforcement and compliance checking, you get protection that beats the pants off 30 different passwords to 30 different sites, or more realistically, 3 passwords to 30 different sites.
Yes, much better than 3-30 passwords.
But I view SSO as enterprise password manager with a nice UI. I don't trust it for anything super important.
What’s the saying about dying a hero or becoming the villain?
Maybe this is a case of hindsight being 20/20 but wouldn't they have caught this if they tried pushing the file to a test machine first?
It's not hindsight, it's common sense. It's gross negligence on CS's part 100%
Well, it is hindsight 20/20... But also, it's a lesson many people have already learned. There's a reason people use canary deployments lol. Learning from other people's failures is important. So I agree, they should've seen the possibility.
I saw one rumor where they uploaded a gibberish file for some reason. In another, there was a Windows update that shipped just before they uploaded their well-tested update. The first is easy to avoid with a checksum. The second...I'm not sure...maybe only allow the installation if the windows update versions match (checksum again) :D
Windows has beta channels for their updates
It's a sequence of problems that lead to this:
Many people say Microsoft are not at fault here, but I believe they share the blame, they are responsible when they actually certify the kernel drivers that get shipped to customers.
A real Anakin arc right here.
Now threat actors know what EDR they are running and can craft malware to sneak past it. yay(!)
Smart threat actors use the EDR for distribution. Seems to be working very well for whoever owned Solar Winds.
SHOULD'VE USED OPENBSD LMAO
Yes but the difference is one of them is also going to help you fix it.
Who says it was accidental?
Netflix knew they were going to move from DVD rentals to streaming over the Internet. It is right in their name.
CrowdStrike knew they were eventually going to _________. It is right in their name.
ItS NoT A wInDoWs PrObLeM -- Idiots, even on Lemmy
I genuinely can't tell at whom you are addressing this. Those claiming it is a Windows problem or those that say otherwise?
You can not like windows, and also recognize that CrowdStrike isn't from Microsoft - so a problem that CrowdStrike caused isn't the fault of Windows.
If that makes me a idiot by holding two different ideas in my head, so be it, but you are spending time with us, so thank you for elevating us!
I'm sorry, but distinguishing between different concepts is forbidden here. You go straight to jail.
I'm waiting for the post mortem before declaring this to not be anything to do with MS tbh. It's only affecting windows systems and it wouldn't be the first time dumb architectural decisions on their part have caused issues (why not run the whole GUI in kernel space? What's the worst that could happen?)
I agree it's possible. But if you're a software as a service vendor, it is your responsibility to be in the alpha and beta release channels, so if there is a show stopping error coming down the pipeline you can get in front of it.
But more tellingly, we have not seen Windows boot loop today from other vendors, only this vendor. Right now the balance of probabilities is in the direction of crowd strike
Ah yes the classic presumption of guilt, a keystone of justice
I'm not sure how to break this to you but this is just an internet forum, not a court of law
The reason courts use it is because they value having true opinions. But you're welcome to not value that indeed
The reason courts have rules of how convinced one must be to declare guilt is because they dread punishing an innocent over allowing a guilty person free
We aren't in a position to hurt the probably guilty party so it doesn't matter a bit of we jump to conclusions unfairly
Hi, idiot here. Can you explain how it is a windows problem?
It's a problem affecting Windows, not problem caused by Windows I guess.
That's what I was thinking so I was curious what the argument would be
If you patch a security vulnerability, who's fault is the vulnerability? If the OS didn't suck, why does it need a 90 billion dollar operation to unfuck it?
Redhat is VALUED at less than that.
https://pitchbook.com/profiles/company/41182-21
It's a fucking windows problem.
Sure, but they weren't patching a windows vulnerability, windows software, or a security issue, they were updating their software.
I'm all for blaming Microsoft for shit, but "third party software update causes boot problem" isn't exactly anything they caused or did.
You also missed that the same software is deployed on Mac and Linux hosts.
Hell, they specifically call out their redhat partnership: https://www.crowdstrike.com/partners/falcon-for-red-hat/
Crowdstrike completely screwed the pooch with this deploy but ideally, Windows wouldn’t get crashed by a bas 3rd party software update. Although, the crashes may be by design in a way. If you don’t want your machine running without the security software running, and if the security software is buggy and won’t start up, maybe the safest thing is to not start up?
Are we acting like Linux couldn't have the same thing happen to it? There are plenty of things that can break boot.
CrowdStrike also supports Linux and if they fucked up a Windows patch, they could very well fuck up a linux one too. If they ever pushed a broken update on Linux endpoints, it could very well cause a kernel panic.
Yeah, it's a crowd strike issue. The software is essentially a kernel module, and a borked kernel module will have a lot of opportunities to ruin stuff, regardless of the OS.
Ideally, you want your failure mode to be configurable, since things like hospitals would often rather a failure with the security system keep the medical record access available. :/. If they're to the point of touching system files, you're pretty close to "game over" for most security contexts unfortunately. Some fun things you can do with hardware encryption modules for some cases, but at that point you're limiting damage more than preventing a breach.
Architecture wise, the windows hybrid kernel model is potentially more stable in the face of the "bad kernel module" sort of thing since a driver or module can fail without taking out the rest of the system. In practice.... Not usually since your video card shiting the bed is gonna ruin your day regardless.
How the fuck did my Fedora just bluescreen?? Crowdstrike!
*audience laughter, freeze-frame*
Are the Mac and Linux machines having BSOD (-style) issues and trouble booting?
No, because CrowdStrike didn't bork the drivers for those systems. They could have, though.
Nope, because they only shipped a corrupted windows kernel module.
It's dumb luck that whatever process resulted in them shipping a broken build didn't impact the other platforms.
They've got Paid BSOD, I've got FreeBSD, we're not the same.
isn't XNU more decoupled than Windows kernel?
"even on Lemmy"
Like this is some highbrow collection of geniuses here?
No just statistically we are all Arch (btw) Linux users who hate Windows.
Because it isn't. Their Linux sensor also uses a kernel driver, which means they could have just as easily caused a looping kernel panic on every Linux device it's installed on.
There's no way of knowing that, though. Perhaps their Linux and Darwin drivers wouldn't have paniced the system?
Regardless, doing almost anything at the kernel level is never a good idea
It's not impossible. Crowdstrike has done it recently to linux machines.
Kernel panic observed after booting 5.14.0-427.13.1.el9_4.x86_64 by falcon-sensor process:
https://access.redhat.com/solutions/7068083
Paywalled, unfortunately
Also, it's less about "their" drivers and more about what a kernel module can do.
Saying "there's no way to know" doesn't fit, because we do know that a malformed kernel module can destabilize a linux or mac system.
"Malformed file" isn't a programming defect or something you can fix by having a better API.
Having the data exposed to userspace via an API would avoid having to have a kernel module at all... Which when malformed wouldn't compromise the kernel.
I mean, sure. But typically operating systems don't expose that type of information to user space, instead providing a kernel interface with user mode configuration.
It's why they use the same basic approach on mac and Linux.
Security operations being one of the things that is often best done at the kernel level because of the need to monitor network and file operations in a way you can't in user mode.