Lemmy.ml ~~is down right now~~ (back online, was unreachable for about 6 hours)

BitOneZero @ .world@lemmy.world to Fediverse@lemmy.world – 164 points –
isitdownorjust.me

lemmy.world had a DDOS in the past 24 hours, Lemmy.ml was showing problem, but now it is entirely unreachable. lemmy.world was showing "Error!" page (server still reachable), but that seems to have only lasted for a hour or so.

56

Ok so I have learned that it is always good to have a backup account or two if I am intent on getting my memes

lemm.ee has always been solid for me. At this point, I'm debating whether to just make it my primary instance instead.

Nice! I've more or less switched lemdro.id

Same but it’s tough to keep favorites synced and blocks and filters.

I wrote a tool that will help: https://github.com/CMahaff/lasim

Let's you copy your subs, blocks, and profile settings between Lemmy accounts.

You can run it as many times as you like and will keep adding new things.

At the minute it never removes things though, so you'd have to unsubscribe multiple places.

True true. Hopefully some of the upcoming 3rd party apps can address some of the issues like this.

Ya if they have cloud user data.

Cloud? I'm making one, and doing everything locally. Your phone can hold a small database - from there it's all data management. It's

I haven't started synching accounts yet, but it's in the works. Eventually, I want to do custom feeds and stitch feeds from multiple servers/accounts together

I neither want nor need your data though... It's harder to do it all on the device, but it's very doable

That’s what I just did. Also found a tool to subscribe me to all the communities.

lemmy.world is back up now (I'm posting this on it). I'm looking forward to @Ruud posting a postmortem.

This information clarified so much of this morning for me.

Was troubleshooting on my phone because I thought it was my kids using all the bandwidth for YouTube (kids)) and Roblox on a slower than normal Sunday.

Lemmy.world is now responding some at least, and some of the comments from remote servers came into this posting just now.

That feel when I miss all the fun in Australia because I was asleep

World was down for me too in the last 10 minutes.

Lemmy.world down now for over 30 minutes, Lemmy.ml over an hour (I created this post on Lemmy.world before it went down). Distant Early Warning Sign: https://www.youtube.com/watch?v=wrDj5XvZXX4

It's weird, I can access Lemmy.world through sh.itjust.works but I can't log in. Are they updating maybe?

If you mean that you can see content from lemmy.world from sh.itjust.works, that might be stuff that federated over before lemmy.world went down.

But aren't we talking and communicating on lemmy.world right now? Maybe I don't understand it correctly and we're communicating on sh.itjust.works even though it says lemmy.world.

Interesting, I didn't notice that this community was hosted on lemmy.world. Yeah, my understanding of how ActivityPub worked was that lemmy.world would have to act as a "relay" for these message to make it out, even though you're on sh.itjust.works and I'm on kbin.social. I could easily be wrong about that, or perhaps lemmy.world is managing to stay just functional enough in the background to keep traffic flowing.

Edit: Never mind, lemmy.world seems to be back up. I just visited their front page and it loaded fine.

I know there's a lemmy.world support community, I'll try and see if they know what's going on.

Edit: On your edit, I still can't log in and it's acting the same as it was before. Nvmd, it's just slow to load.

You're on sh.itjust.works

This community is hosted on lemmy.world, but it doesn't disappear if lemmy.world goes down.

Yes, but we're communicating on it where? If lemmy.world is down, why can we interact with each other, where is the info being stored?

@pelespirit @Madbrad200

The comments/posts stay in your local instance until the lemmy.world will be up again and then it will sychronise via ActivityPub again.

I'm not on works or world, but I see your 4m old comment. Maybe it works like file sharing where there are seeds and peers? And without seeds, we can still see some content?

Well that makes more sense. That explains how you can see beehaw.org stuff and even comment but they never get updated. I think I might be understanding this stuff a little now, thanks.

lemmy.world is reachable, but no database connections, it has an "Error" page on every post I try to load. Which this post itself on Lemmy.world serves as example: https://lemmy.world/post/1578844

Sorry, not very techy, what does this mean?

lemmy.world is reachable, but no database connections,

Does that mean he's being attacked again?

most likely, yes. There was no notice of a planned outage, and the timing of them both going down within 15 minutes of each other. Lemmy server programming code isn't very robust when it comes to performance, it is possible to busy up the database to a point that the site become unresponsive. I don't know what is going on with lemmy.ml being entirely unreachable, the devs who run that server don't normally work on weekends... and normally nginx responds with an error at minimum. Nothing at all is loading

Well... there has to be something lemmy does right I guess

Why tf aren’t they using Cloudflare for DDoS protection? The site isn’t run for-profit so it would be covered under the free tier. My homelab is proxied through Cloudflare and would be able to handle an attack better than this.

How did you make that Title?!

You can use markdown codes in titles. Bold, italic, and in my case strikethrough. Two tildes before and after the text.

Beehaw.org was down for over 30 minutes, came back for 15 minutes, now down again another 30 minutes.

6 hours down for lemmy.ml is really going to have a lot of items queued on lemmy.world, beehaw, and other popular servers to delivery to lemmy.ml - causing resources to climb on all those servers.