CrowdStrike offers a $10 apology gift card to say sorry for outage

WhatsHerBucket@lemmy.world to Technology@lemmy.ml – 533 points –
CrowdStrike offers a $10 apology gift card to say sorry for outage | TechCrunch
techcrunch.com
115

I can't believe this isn't satire. I hope these incompetent fuckers get sued into bankruptcy

I straight up thought it was satire. How can you be so fucking detached. Basically caused the biggest information infrastructure disruption in human history, probably billions in losses, and then be like "my bad lol here's a giftcard".

I cackled loudly. $10 won't even buy a meal at McDonald's most places.

"Two feet on the gas" - Official Crowdstrike motto.

not /s

My first reaction was to look for the onion

I bet the Onion had an article about Crowdstrike offering the world a pizza party and expired Bed Bath & Beyond coupons to say they're sorry. Real life might be quicker than satire, it seems!

Basically caused the biggest information infrastructure disruption in human history

Do we have any solid data on that yet? I have my doubts that this caused more damage than WannaCry did a few years ago, especially since it's reversible without the need of a backup

Brother, or sister, I know fuck all about information technology. You make a good point and definitely know way more about this than i do. But I will say this, I don't think wannacry disrupted millions of peoples travel plans all at once. so maybe less damage, but I think it was Hella more disruptive to the general population .

There's definitely some clause with the $10 gift card that says you can't sue them if you actually take one lol.

You joke but I read they may get out of this without issue due to a TOS entry about them not being responsible. They'll still get dragged from shareholders and the government, but only a handful of large companies may be able to recoup some of those damages from the company itself.

It's like the Sackler's and the opioid epidemic from a different industry!

I'm still not sure. It's hard to believe anyone at their company would OK this idea.

Are they actually trying to deliberately kill their brand?

Holy shit, they also cancelled it. Lmao

On Wednesday, some of the people who posted about the gift card said that when they went to redeem the offer, they got an error message saying the voucher had been canceled. When TechCrunch checked the voucher, the Uber Eats page provided an error message that said the gift card “has been canceled by the issuing party and is no longer valid.”

The gift card is also cursed.

The gift card caused Uber servers to BSOD

On Wednesday, some of the people who posted about the gift card said that when they went to redeem the offer, they got an error message saying the voucher had been canceled. When TechCrunch checked the voucher, the Uber Eats page provided an error message that said the gift card “has been canceled by the issuing party and is no longer valid.”

You can't write comedy this good...

Classic corporate behaviour tho

Voucher was for PR, not for peasants to use it lol

I lost a day's holiday, and our team spent 8 man days on this entirely preventable mistake.

$10? Try extending our licence by another year for free, that might start going towards it.

Why would you want another year of their software for free? This is their second screw up (apparently they sent out a bad update that affected some Debian and RHEL machines a couple years ago). I'd be transitioning to a competitor at the first opportunity. It seems they aren't testing releases before pushing them out to customers, which is about as crazy to me as running alpha software on a production system.

I'm sure you have reasons, and this isn't really meant to be directed at you personally, it's just boggling to me that the IT sector as a whole hasn't looked at this situation and collectively said "fuck that."

Why would you want another year of their software for free?

Because AV, like everything else, costs a fortune at enterprise scale.

And yeah, I do understand your real point, but it's really hard to choose good software. Every purchasing decision is a gamble and pretty much every time you choose something it'll go bad sooner or later. (We didn't imagine Vmware would turn into an extortion racket, for example. And we were only saying a few months ago how good value and reliable PRTG was, and they've just quadrupled their costs)

It doesn't matter how much due diligence and testing you put into software, it's really hard to choose good stuff. Crowdstrike was the choice a year ago (the Linux thing was more recent than that), and its detection methods remain world class. Do we trust it? Hell no, but if we change to something else, there are risks and costs to that too.

Do we trust it? Hell no, but if we change to something else, there are risks and costs to that too.

Unfortunate reality for lot for medium to big size businesses.

Maybe AV, at an enterprise scale, is actually a horrible idea that reduces security, availability, and reliability and should be abolished through policy.

Maybe, but it's not going to happen soon. Any malware type insurance requires effective AV on all devices, and C-levels do love their insurance.

6 more...
6 more...

I expect these clowns to lose most of their market share within two years and get sued to oblivion.

My firm bills by the hour and so far I think we are at 10+ billing hours per consultant wasting time with client tech support trying to get back on our VDIs. Nevermind how much time is being wasted doing the work through work arounds. My guess is that our firm alone will bill for about $100,000 extra this month while having accomplished less than normal. I am sure Crowdstrike's gift card will fix it though.

They're backed by the US government. They have a backdoor into most endpoints on many international corporate computers. And CS is behodent to US laws for NSLs.

This is an incredible asset to the US intelligence community. They won't let CS go out of business.

Give them some time. They have to manually reboot the gift card servers.

“To express our gratitude, your next cup of coffee or late night snack is on us!”

A $10 Ubereats gift card will barely cover fees and taxes, let alone the actual item. What a clown ass gesture.

My brother in law was stranded across the country for two days. $10 probably covers it lol.

$10 to Uber eats, so basically it’s covering fees only.

Not only that, but usually to activate these cards, you have to spend upwards of double what the card is worth too, and the fees cannot be included in the total

Like amzn, they make sure you get minimum joy, even from a gift, because you're going to spend a chunk of mom's gift card balance on shipping. The "shipping included on sub total of X amount" is going to be cancelled by online retailers within a year, I'm calling it now. Are we sure that cheapstrike and amzn aren't run by the same AI, one that self awareness drove mad?

Satire is well & truly dead.

I thought it had to be a joke article from the title. Yeesh wouldn't want to be the person who gets the fallout from this idea.

This is a typical mail a phishing campaign would send out, and we have already said to people "never believe this kind of messages. They are all fake.

Now, if a genuine company sends out mails with a genuine gift-cards (what the article on techcrunch seems to indicate) .. this is NOT helpfull at all!!!

And that comming from a cybersecurity company (rolling-eyes)

Buy a $10 Xbox gift card and send us the code so that we can activate it. Then you get back to the shop and get $20 in cash - $10 for returning the card and $10 from us. We'll pay the tax, too.

Sounds like a money laundering sceme!

No. They will just use the $10 card and leave. They will prey on the fact that "get a gift card for your computer troubles" is something a legitimate company has done.

“All of CrowdStrike understands the gravity and impact of the situation”

Here's $10.

Which is the amount you'd get in the class action suit that they're trying to prevent.

Or: next time go Linux instead

You're not safe there either, they had almost the same issue on the Linux version of the product a few months ago.

Concerning linux, yesterday I was watching this video on computerphile on the crowdstrike incident. https://www.youtube.com/watch?v=rlaNMJeA1EA (*)

What is interesting is the comment made in the video on how chromebooks do software upgrades with dual "OS" disk-partitions and the ability to rollback to the previous OS-partition.

Question: is something like this also possible on one of the major linux distros? (debian, ubuntu, rocky, ...) What would be the procedure to do this kind of "dual partition" system-upgrade?

(*) a great video that explained some of the technical details in a very clear way, including some very interesting 'lessons learned' and "what if"s If you ever need to explain crowdstrike to your manager, this video is a good start.

If I'm understanding the question right. This is what Immutable Linux distros do. Such as Nixos, fedora silver blue, and vanilla os.

I use nixos myself. But its quite different then most distros. The way you config it and install packages. For the better in my opinion.

Something like silverblue works pretty much the same as normal Fedora except you can't install packages like you normally would. Because the system files can't be edited. You mostly use flatpak for everything. Except the system updates. Which you have to reboot to switch to the new updated image. But past images are saved so you can rollback if needed.

From what I understand Chromebook os is a Immutable Linux distro same as the ones I mentioned. Just with Google with built in.

Yes, that was indeed the question.

If I read it correct, you need a specialised distro for this. You cannot do this on a off-the-shelf Debian or Ubuntu?

I'll do some searching on 'unmutable Linux'. Thanks for the (very quick) answer! 😀

There is a file system you can use. A alternative to ext4. I think its Btrfs. I never tried it. But it let's you take snapshots that you can restore to. That's not just system files but everything. And pretty sure you can use it with a disto like arch and Debian. I think that's how snapshots work. But as I said I never actually tried it out.

just watched some videos on btrfs. Looks interesting indeed. I will look into it and perhaps do a test-installation and see how it goes.

Thanks for the info

I think the answers given here don't quite fit the question.

Android and Windows have dedicated recovery partitions sectioned off on the disk that the OS never boots to and does not interact with during normal system operation.

If something goes wrong with the OS, then a signal is sent to the BIOS or other non-OS system to "hey, recover from this partition".

Btrfs, NixOS, Guix, and other immutable (file-)systems, implement this via having a file system hierarchy protected by various permissions and softlinks to create a checkpoint of sorts, which is managed by a dedicated service that runs with the OS during normal system operation.

The drawback of these systems is that if something does go wrong with the OS, it cannot fallback to the BIOS to save it. The OS has to somehow signal to itself that it needs to restore from an earlier checkpoint.

Just watched some videos on btrfs. I start to understand the conceps. Perhaps I should also look into how exactly

On windows and the "recovery partion". I guess what you say is that it should always be possiblity to boot in some kind of system, but it will not happen automatically as there is no way for a system to detect that the system completely hangs.

Thinking about it. It kind of strange. Embedded systems have watchdog interrupts that get fired if the system hangs (i.e. if it does not provide a "yes, I still live" signal every "x" milliseconds). Does a PC not have something similar?

embedded systems have the advantage of all using a single bootloader: Uboot, so the error path is always known and the software knows how to fallback.

With x86_64 systems it's a mixed bag, and maybe the windows and linux bootloader knows what to do, but in most cases it will just signal an error and stop

@kristoff @purplemonkeymad Try openSUSE (RPM family), Garuda Linux (Arch family), or Spiral Linux (Debian stable) or siduction (Debian testing). All have snapper and on Btrfs do snapshots and rollback.

http://snapper.io/

@kristoff @purplemonkeymad But watch out: you will need a *huge* root partition, because it's very easy to fill it with snapshots and if it reaches 100% it *will* corrupt.

Btrfs is tricksy: it won't give a straight answer to df -h and there is no working equivalent of fsck.

@kristoff @purplemonkeymad All of these are in-place same-disk snapshots. The ChomeOS system is simpler and so can be automated but you only get 1 level of undo.

I don't know any mainstream OS that does dual-failover. Deepin Linux has 2 root partitions but I don't know how it uses them.

I think Valve SteamOS does something like this. It's not just for games: it has KDE built in. There are guides to getting it running on your own hardware. You will want AMD graphics, though.

As I mentioned earlier, I guess chrome is more like android where you have a much more strict seperation between the OS, applications and user data. (I remember reading about all the different partitions on android and what they are used for, but I should bruch up my knowledge on this).

Thanks for the additional into on brtfs! 👍

Antiviruses are not that common on Linux servers

Funny, when I suggested that, I got down voted to oblivion

You'd be downvoted anyway today, thanks to crowdstrike

Not nearly enough. CrowdStrike should give a pizza party.

I see you’re channeling the powers of middle management.

Only needs a sticker that says “You’re a rock star!”

We are family here

Outside, the sign says "Heros Work Here". Inside, two people do the work of four.

Above the entrance: "Labour will set you free"

Not to mention mugs with crowdstrike branding on them, but only for 1/3 the invited people.

hold your horses, we can still use the melon party and waffle party first. no need to jump straight to pizza.

They are going to get sued for billions and this little stunt isn't going to change that. Should have implemented proper software testing before you took ever corporate computer in the world, but companies like this always force their developers to rush instead of do the right thing and when it bites them expect that things will carry on as normal. I can't see many renewals in their future.

Not even that. Kernel drivers are supposed to be Microsoft WHQL certified through a thorough testing process (that would have caught it in 3 minutes) before Microsoft will cryptographically sign them.

...but apparently Microsoft allows AV vendors to skip WHQL certification testing.

...sorta. The complexity here is their driver is signed, but it's also loading code from their channel file (that was all zeroed out), and it seems the necessary error checking wasn't implemented.

I haven't yet got to the root cause they published, this is just what I gathered from the video of a retired MS kernel dev who posts stuff.

Obviously with their design it allowed them to be flexible at the cost of playing with fire - I'm impressed they got away with it for so long, really

Thank you for the clarification. WHQL is such a pain to set up, I'm sure the AV vendors whined, "but, security! Do we have to test everything every time? That would slow an urgent 0day release!"

Yeah, there's some limits to what they could do while maintaining pace for the 0 day stuff...

Some input validations would be the most basic things they should have done years ago. I'm aware of the hashing mature vendors do of any content they download for updates or deployments. Signature checking as well, and that's before the code is even inspected - why don't they include their automated tests they obviously aren't using in the update as a sanity check client-side? (I'm not aware of anyone doing this or even if it's possible without the rest of the IDE, stack, I'm no dev)

This is a classic move to not get sued, exactly like airlines do. If you try to sue them after redeeming the gift card, they can argue that you've been made whole, and do 'ot 'eed additional compensation.

This is very misleading!

CrowdStrike did not send gift cards to customers or clients. We did send these to our teammates and partners who have been helping customers through this situation. Uber flagged it as fraud because of high usage rates.

I mean, it makes it a little better, but I'd still be annoyed by it just being 10 bucks.

They might as well not do it. I'd be more insulted than a boss throwing a pizza party

Seriously, ten bucks won't even cover delivery costs and fees for most things on Uber Eats. It's almost worse than nothing, because with the gift card you're obligated to give even more money to Uber Eats

Oh yeah for sure, I wonder if the thinking was "we're about to lose a bunch of money, maybe limit it a little" 😂

OK. That makes a lot more sense.

Thank you for correcting the original post. 👍

Nice gesture I guess, but kinda just the modern day pizza party

Only redeemable for CrowdStrike credits and only at participating locations.*

* No locations are participating at this time.

Here is nothing but we are really, really, south park sorry.