mastodon) hasn't even attempted to solve; groups and what happens when they get popular.

Technology@beehaw.org – 42 points – 1 years ago

#lemmy/#kbin has a problem that #mastodon hasn't even attempted to solve; groups and what happens when they get popular.

#Communities, #groups, #magazines, whatever they are called are implemented as #Actors in #ActivityPub. They are basically just *very* popular users who boost a *lot*.

You can't just distribute them across instances the way normal actors do. Whichever server hosts @technology@lemmy.ml or @technology@beehaw.org is going to get HOSED on the regular.

You are viewing a single comment

View all comments Show the parent comment

Lots of traffic, lots of posts, lots of comments, ... That's going to need more storage, more bandwidth, more CPU power, higher running costs. The original instance hosting the community bears a higher load than the instances that duplicate it.

Ideally, there would be a way to more evenly distribute this load across instances according to their resources, but from my (currently limited) knowledge, I don't think Lemmy/ActivityPub is really geared for that kind of distributed computing, and currently I don't believe that there's a way to move subs between instances to offload them (although I believe some people may be working on that).

Perhaps the Lemmy back-end could use a distributed architecture for serving requests and storage, such that anyone could run a backend server to donate resources without necessarily hosting an instance.

For example, I currently have access to a fairly powerful spare server. I'm reluctant to host a Lemmy instance on it as I can't guarantee its availability in the long term (so any communities/user accounts would be lost when it goes down), but while it's available I'd happily donate CPU/storage/bandwidth to a Lemmy cloud, if such a thing existed.

There are pros and cons to this approach, but it might be worth considering as Lemmy grows in popularity.

I don’t think it’s a problem. If you weren’t using activity pub and just something like reddit then if you were reddit (the sysadmin) you’d also deal with having to scale if your community gets really popular

Stuff that gets linked to also has the same problem

https://www.jwz.org/blog/2022/11/mastodon-stampede/

(Btw I don’t like jwz but he mentions it here)

Funny how you say it's not a problem, then go on to describe the problem that needs to be dealt with. Dealing with scaling is a problem, and it's a problem that costs money.

Posts like this: https://lemm.ee/post/58472 suggest it is a problem. The rise in traffic seen by Lemmy in the last few days is absolutely tiny compared to a site like reddit, and already instances are struggling to cope. The recent growth in user registrations represents only about 0.007% of reddit's active user base. (~60K new Lemmy users vs 861,000,000 active monthly reddit users). A site like reddit costs millions to run.

There are 190+ Lemmy instances last time I checked, yet almost all the brunt of this load has been borne by a handful of servers, which see an inordinate amount of traffic while 100+ other servers sit around idle. Why should a handful of "lucky" servers have to pay all the hosting costs? What if a volunteer-run instance explodes to reddit-like levels of popularity? It will simply fold, unless the volunteer has serious money to throw at the problem.