Major IT outage affecting banks, airlines, media outlets across the world

rxxrc@lemmy.ml to Technology@lemmy.world – 1186 points –
Live: Major IT outage affecting banks, airlines, media outlets across the world
abc.net.au

All our servers and company laptops went down at pretty much the same time. Laptops have been bootlooping to blue screen of death. It's all very exciting, personally, as someone not responsible for fixing it.

Apparently caused by a bad CrowdStrike update.

Edit: now being told we (who almost all generally work from home) need to come into the office Monday as they can only apply the fix in-person. We'll see if that changes over the weekend...

548

You are viewing a single comment

never do updates on a Friday.

You posted this 14 hours ago, which would have made it 4:30 am in Austin, Texas where Cloudstrike is based. You may have felt the effect on Friday, but it's extremely likely that the person who made the change did it late on a Thursday.

Never update unless something is broken.

This is fine as long as you politely ask everyone on the Internet to slow down and stop exploiting new vulnerabilities.

I think vulnerabilities found count as "something broken" and chap you replied to simply did not think that far ahead hahah

For real - A cyber security company should basically always be pushing out updates.

Exactly. You don't know what the vulnerabilities are, but the vendors pushing out updates typically do. So stay on top of updates to limit the attack surface.

Major releases can wait, security updates should be pushed as soon as they can be proven to not break prod.

always pushing out updates

Notes: Version bump: Eric is a twat so I removed his name from the listed coder team members on the about window.

git push --force

leans back in chair productive day, productive day indeed

git commit -am "Fixed" && git push --force

That's advice so smart you're guaranteed to have massive security holes.

This is AV, and even possible that it is part of definitions (for example some windows file deleted as false positive). You update those daily.

10 more...