(De)centralisation of Lemmy communities in numbers

Ada@lemmy.blahaj.zone to Fediverse@lemmy.ml – 45 points –

cross-posted from: https://kbin.social/m/fediverse/t/893150

Lemmy instances by number of communities with over 1 thousand (5 thousands, 10 thousands) of monthly users:

Lemmy.world: 122, 30 over 5K, 12 over 10K, 1 over 20K (Technology, Lemmy Shitpost, News, Politics, World News, Memes, Comic Strips, Microblog Memes, Ask Lemmy, Political Memes, Linux Memes, No Stupid Questions)
Lemmy.ml: 30, 7, 4 (Memes, World News, Ask Lemmy, Linux)
sh.itjust.works: 11, 5, none (White People Twitter, Games, Greentext, Funny, NonCredibleDefense)
Hexbear, 12, none, none (Chapo Trap House, The Dunk Tank, generic named communities)
Lemmy NSFW 10
Sopuli 8, 1 (Memes, Ukraine, Steam Deck, Meow_IRL, AnarchyChess, Aneurysm Posting, NYT gift articles, Map Enthusiasts)
Lemm.ee 7 (Movies and TV, Cyanide and Happiness, Artporn, YUROP, Movies, BrainWorms, Conservative)
Beehaw 6 (Technology, Gaming, FOSS, Politics, World News, Humor)
Lemmy.ca 6 (PC Gaming, Canada, Nostalgia, Cool Guides, Offbeat, Men's Liberation)
Midwest.social 5, 1 (The Onion, Lord of the memes, Memes, The Right Can't Meme, Religious Cringe)
SLRPNK 5, 1 (Climate, Memes, Solarpunk, Antiwork, tombstone of „TwoXChromosomes”)
/0 4, 1, 1 (Piracy, ADHD memes, Lefty Memes, Stable Diffusion Art)
Feddit.de 4, 1 (Europe, ich_iel, DACH, Deutschland)
Mander 3, 1, 1 (Science Memes, Science, Astronomy)
Programming.dev 3, 1 (Programmer Humor, Programming, Comics)
Lemmy.zip 3 (Global News, Technology, Gaming)
Feddit.uk 3 (UK, UK politics, Casual UK)
Lemmygrad 3 (GenZedong, World News, Comradeship)
Blåhaj Lemmy 1, 1, 1 (196)
Star Trek Website 2 (RISA, Star Trek)
ani.social 2 (Anime, Animemes)
Lemmon 1, 1 (Tails)
TTRPG Network 1, 1 (RPGMemes)
Lemdroid 1 (Android)
Reddthat 1 (Memes)
KDE 1 (KDE)
Futurology.today 1 (Futurology)
Aussie Zone 1 (Australia)
Lemmy.one 1 (Privacy Guides)
Lemmy Fan 1 (Weird News)
Diagon Lemmy 1 (The Leaky Cauldron)
tchncs 1 (Right to Repair)
Smeargle 1 (mirror of Hacker News)

  • Lemmy.world has got a plurality of 1K/mo Lemmy communities. (122 vs 140) Lemmy.world and Lemmy.ml together make a majority of 1K/mo Lemmy communities (152)
  • Lemmy.world has got a majority of 5K/mo Lemmy communities (30 vs 22)
  • Lemmy.world has got a majority of 10K/mo Lemmy communities (12 vs 7)
  • Lemmy.world has got only 20K/Mo Lemmy community
  • Only 1K/mo national "sublemmies" are Canadian, Australian, British and German. Only non-English 1K/mo "sublemmies" are German.
20

Relevant: https://reddthat.com/comment/8316861 tl;dr. The current centralisation results in a lemmy-verse theoretical maximum for of 1 activity per 0.3 seconds, or 200 activities per minute. As total transfer of packets is just under 0.3 seconds between EU -> AU and back.

Edit: can't math when sleepy

Oh, now that's fascinating, and explains some of the issues we're having. Our hosting provider had some issues, so all of our queues were delayed. It took us a while to recover, but we got through all of the outstanding queues, except for lemmy.world, which still has a huge amount outstanding. And I'm guessing that's because we're processing them only slightly faster than new ones enter the queue!

Since the 11th @ 9am UTC, LW has seen a 2 fold increase of activities. If my insider knowledge is right (and math) it's 7req/s average up from 3req/s.

Lucky for both of us we are not subbed to every community on LW but I think we are subbed just enough to be affected.

They ignored lemmy.dbzer0.com

I think they included it as "/0"

Oh right. That's an interesting shortcut I didn't think to look for. Or maybe it was edited in? I remember looking for piracy

Thanks for sharing, and happy to see two of the communities I'm posting to on Lemm.ee are mentioned there (!yurop@lemm.ee and !movies@lemm.ee )

The centralization on Lemmy.world can create performance issues. Reddthat has been having issues since LW upgraded to 19.3: https://grafana.lem.rocks/d/edf3gjrxxnocgd/federation-health?orgId=1&var-instance=lemmy.world&var-remote_instance=reddthat.com&var-remote_software=All&from=now-24h&to=now, while they have no issues with SJW: https://grafana.lem.rocks/d/edf3gjrxxnocgd/federation-health?orgId=1&var-instance=sh.itjust.works&var-remote_instance=reddthat.com&var-remote_software=All&from=now-24h&to=now

@ticoombs@reddthat.com, our admin, is trying to figure out a solution, but it might require important rework of the Lemmy codebase

I'm thinking about asking a few communities I mod on LW if they could consider moving to another instance to spread the load.

Sadly still stuck on the OG lemmy.ml. Still waiting impatiently to be able to migrate my account to an another smaller instance.

What do you mean waiting to migrate?

Post history migration will never be implemented. But you can already use apps to move over your subscribe lists etc.

With 0.19, you can even move your subscribe list in two clicks from the settings (you get a JSON file that you can import on another instance) without external apps

What’s so bad about lemmy.ml?

I assume the usual complaint, which is that it's the one of the only major servers where tankies are prevalent.

That's, of course, not to suggest that even a majority of its users are tankies or that there aren't plenty of great communities there, but the users who are tankies seem to be almost entirely split between lemmy.ml and lemmygrad.

The biggest problem with "federation" is the control given to bullshit admins in some of the biggest and how big into hiding their problems they are. One of the biggest instances has shoved two of their admins under the rug, without openly acknowledging why they did it or the drama there clearly is behind it, giving the impression that the problems those admins were creating have been addressed on one side and on the other just trying to dismiss it as "no big deal, they are just taking time off".

So keep in mind joining some of those instances is just joining a good paint job. Sometimes, the rust breaks through, sometimes the rusted pieces get removed, but the people painting over the rust are still there.

It matters because when communities crowd around large instances, those admins have ultimate control over participation within them. And while easy to bypass, it will still affect the reputation your comment history would have represented.