Addressing the Exponential Growth of Communities
Over the past few days, I've witnessed a remarkable surge in the number of communities on browse.feddit.de. What started with 2k communities quickly grew to 4k, and now it has reached an astonishing 8k. While this exponential growth signifies a thriving platform, it also brings forth challenges such as increased fragmentation and the emergence of echo chambers. To tackle these issues, I propose the implementation of a Cross-Instance Automatic Multireddit feature within Lemmy. This feature aims to consolidate posts from communities with similar topics across all federated instances into a centralized location. By doing so, we can mitigate community fragmentation, counter the formation of echo chambers, and ultimately foster stronger community engagement. I welcome any insights or recommendations regarding the optimal implementation of this feature to ensure its effectiveness and success.
I honestly wouldn't want that, a feature like multi-reddit would be much better IMO.
I personally don't want to be "automatically" subscribed to all tech communities for example just because I joined one, nor I want to be flood by an immense feed because all communities of the same type are put all together, that takes away individual choices IMO.
We had exactly the same problem on reddit, but multi-reddit solved that very well by leaving the choice to individuals instead of being forced by admins.
EDIT: for those who don't know, multi-reddit is a reddit feature that allows you to create different "labels" into which you can combine different subreddits, which label to create and which subs to combine is totally a user choice, those labels become "tabs" into your UI that you can use as they were individual subs.
So for example, I can create a label/tab called "linux" and use it to combine r/linux + r/linuxmx + r/xfce, etc., than I can create another label called "games" and combine r/MMORPG + r/wow + r/guildwars2, etc., and so on.
multi-reddits can be private, that is only the user who created them can see them, or they can be made public, so if some user doesn't want to create their own, they can use multis created by other people.
join a meme sub
get joined to 15 meme subs
every meme is posted 15 times
surprised pikachu face
ftfy
I like this idea, however it would need to be intuitive to use and clearly advertised as a feature with a plain explanation up front. I say this because I'd never heard of this feature before and I used reddit for over 15 years (had to Google how it worked after seeing your comment).
Yes, users need ofc explanations, but that can be easily done, as much as people here are trying their best now to explain people the fediverse, that is quite more difficult to grasp IMO than a multi-reddit feature.
I like this idea better, also it shouldn’t be as hard to implement?
I am in this picture and I don't like it. Lmao I literally had a multi-reddit called Linux.
Exactly. Just because I join a US politics sub doesn't mean I want to join all US politics subs. The fact they are separate means I can find one that fits me. If this was reddit, joining /r/moderatepolitics would automatically sub you to /r/politics. Fuck that.
How about a mod option to voluntarity merge another community into their community?
This would be great
Damn, this is a lot of discussion and I don't see a single person actually volunteering to actually go code the feature. It's open source, you know? If anyone cares about the feature, go learn rust (like I'm trying to do now) and code it up.
EDIT: In case anyone reads this, please look at entitlement in open source. It's an eye-opening read for those not familiar with the headaches involved.
I'm honestly more confused than I was before. With so many opinions, I don't know how this could ever be implemented in a way that satisfies people.
It'll be solved the same way anything gets solved in open source: those that can code make the final decisions.
Give me the few years I need to grasp how Rust actually works lmao
Open-source? More like open head from the SKULL SPLITTING HEADACHE!
Instructions unclear: do i put in the skeletons mouth or not
Thanks for the article. I really like the overall message about maintainers setting strong boundaries and being able to walk away.
I love open source. And it is so sad when good projects hit snags because of a toxic user-maintainer relationship.
"An ounce of planning is worth a pound of cure."
Nothing wrong with rushing into projects when you're learning a new language, but on big OSS projects it's a good idea to make sure you're working on something that the maintainers are willing to merge. Getting community consensus is a good thing.
I also started to learn rust for the same reason.
"Just go learn Rust"
Yeah that applies to a handful of people.
The underlying problem here is the lemmy community being spread out across many instances, and this solution doesn't really fix the underlying problem.
This is just speculation, but I think eventually 1-4 instances will grow much bigger than the rest. I think when this happens, communities will become much less fragmented and the problem will solve itself.
tl;dr while this is a good idea, I think if we just leave everything the way it is the problem will solve itself.
Isn't a few large communities eating up the others like the opposite of what Lemmy is trying to do?
i keep hearing people call for this like its going to happen and be the only way things will be. Look at reddit, look at the history of some of these subs.
there will always be multiple copies of various communities. what software gives us the ability to do is sort and filter and tag (we need to add this) to our hearts content so instance admins and users have control over what comes across thier feeds.
Joined communities will have many of the same centralization problems reddit has now. I've seen this call mostly from users who were on reddit long after it was large. It seems many have no idea that almost every topic on reddit has 4-6 subs around it usually.
Indeed - what you describe is why I'm really not worried about fragmentation. Federation means you'll be able to see all of the relevant communities, and you can decide to subscribe to any or all of them.
In a community with 5m+ members, everything moved too fast
If people are satisfied with them, I think that's OK, and more efficient than having a zillion.
Problems will happen if we go too low, and bigger instances start de-federating. Some might be tempted to start monetizing like Reddit.
I think it's only a problem if it congregates to 1 instead of 4 or so. If one of the 4 goes rogue or disappointing its users, people can easily just jump on a different one. Most servers will suck and that's ok. Good ones will attract users.
The example of federation most people have experience with is email. There will almost certainly be gmails and yahoos emerging over time, but they will have limited control compared to reddit, because if you don't like the filtering/advertizing/whatever of one you're free to leave for another
The email analogy breaks down when you consider that most email servers are run by big tech and cost a lot of money to upkeep
Or you can run your own email server for yourself and a few of your family.
There's almost nothing in between gmail and some random person's self hosted email server.
In terms of the fediverse,who the heck is willing to host a lemmy server for 1 million complete strangers? Not many people i think
If Lemmy takes off I wouldn't be at all surprised if tech companies hosted instances that they monetize through advertising, and many people would be willing to have a home instance that showed them ads in exchange for high stability and potentially more user-friendly clients
They'd still have to release the source for their modded versions with ads, thus, ads can be mitigated from the instance client/app side.
Not necessarily, there are several ways they could release a proprietary app: either code it from scratch so they own the copyright, use OSS code that has a commercial-friendly license (eg. MIT), use an OSS library that allows them to link with their proprietary code (eg. LGPL).
But even if they did release the source code, I think they could still be profitable. Their main customers would be people who want something that "just works", and a lot of those people would rather see a few ads than deal with downloading a modified version of the official client. People who hate ads and are willing to tinker are more likely to run their own insurance, IMO.
They'd still have to use the Lemmy API, thus, recognizing ads and/or reversing code should be fairly easy (when you actually know how everything communicates).
Just as a side note (am kinda curious to be honest) I always ran the official Reddit app (don't mod anything, so... didn't see the point in using 3rd party apps) and I never EVER saw a single ad in the app. Maybe it's because I don't live in the US, IDK, but would like to hear an explanation as to why ads weren't served on my client... not that it bothered me, lol 😂.
They would only need to use the API to communicate with other fediverse instances. They could make a proprietary API for their mobile client, which would display ads.
But that's beside my point, which is that no one will bother to reverse engineer their app if there are easier ways to avoid ads (like setting up your own instance). Their users will be people who want a turn-key solution. People who are allergic to ads (like me) will be better off setting up their own server or using a smaller server, or paying for "premium" access to a commercial server.
At least, that's how I see the fediverse evolving. I've only been here for about a week, though, so take my opinion with a big grain of salt. It'll be interesting to see how this plays out.
It's GPL licensed. If they mod the Lemmy source to comunicate with their app, they have to release the source.
Unless they write something from scratch (or take MIT/BSD licensed source, as you pointed out) and just implement the API to communicate with other instances, yes, in that case they don't have to release the source. But that is a very unlikely scenario to be honest, it's just too much work.
Yeah, been around a week here as well, still learning the ropes, lol.
Yeah, but we are social animals. Everyone* want's to join the big group!
The goal of implementing this feature is to leverage the benefits of federation. If we wait until there is only a few big communities, the purpose of having federation becomes irrelevant. When an instance hosting one of those large communities shuts down, the community would have to migrate to the next major community.
By proactively implementing this feature, Lemmy can harness the advantages of federation while actively mitigating the challenges posed by community fragmentation and echo chambers. It provides a centralized hub that encourages cross-pollination of ideas, fosters community engagement, and ensures that valuable content is accessible to all users, regardless of the size or popularity of individual communities.
I disagree on two points:
I don't think it'll ever be perfect either. The setup Lemmy has just means it'll be more resilient to breaking down entirely because there's no single point of failure. So yeah hopefully it stabilizes more over time.
see I'm not sure I see that as a problem. There are lots of reasons to spawn a new but similar community (bad community mods, bad server admins). There are lots of subreddits I avoided because they were just too big to get into any real info or discussion, just the same beginner questions asked over and over again.
The proposal does not necessarily imply merging all small communities with others. The implementation can provide an optional choice to community moderators, allowing them to decide whether they want their community to be included in the multireddit. This approach respects the autonomy of individual communities and acknowledges the reasons why new but similar communities may emerge, such as issues with community mods or server admins. By offering this flexibility, the feature can cater to the diverse needs and preferences of different communities while still providing the benefits of consolidating posts from communities with similar topics.
Who would control the multi-community, and how could a potential bad actor/community be dealt with?
I agree. Time will show which instance most people will go to. The smaller instances will slowly quiet down, while the larger ones will gain in popularity. The issue will definitely sort itself out over time. I'm not worried at all.
Whatever y'all decide, please just remember what McLuhan said: "The medium is the message." What's the message implicit in your decision? Be deliberate.
reddit also had that a bunch of places, for example /r/gaming /r/games /r/truegaming etc. etc.
I feel as others had suggested that client side multi
redditscommunities would be ideal so you could set up what groups you like to peruse yourself.I prefer sublemmys and multilemmys
I would kill to just have some help/pointers figuring out how to navigate this... Fediverse?
I've made a couple posts, on one, maybe two, um, Instances? In the communities there?
I don't know. All this change is coming at like, the WORST time in my personal/professional life and learning a whole new world is just... Daunting. (waahhhhhh 😭)
I'm new too, but here's what I've learned in the last week:
You're a user of and logged into @beehaw.org. This post (and the community it was posted to) exists on the @lemmy.world instance. You can see and post to it from your beehaw.org instance, because @lemmy.world also exists in the Fediverse.
My instance is @lemmy.world, so this community/post is "local" to my instance, but in practice that's not super important. All that tells you is where I enter the fediverse, from there we're able to see and post in communities from across instances. For example, I can see communities/posts from @beehaw.org, where you are. I am subbed to a few communities there.
It's possible that a community like /c/games exists on @beehaw.org and on @lemmy.world. You would see them as games@beehaw.org and games@lemmy.world, and they are separate communities (despite having the same community name) so you can sub to one or both. OP is basically suggesting a feature to group (for example) games@beehaw.org and games@lemmy.world so that it just looks like one big community.
More experienced Lemmy and Fediversers, please correct any errors I may have made it this post!
I think that's generally true, however it's worth noting that what you can see from other (non-local) instances is dependent on the admin of your instance. They choose with other instances to federate (exchange data, e.g. communities, posts, comments, etc) with. If they choose not to federate with a specific instance, you won't see content from that instance.
There are already examples of this, but I don't know the details well enough to be confident in expanding on it.
Excellent point!
And as if to illustrate my point, Beehaw is no longer federated with Lemmy.world, so @mountainmycelium@beehaw.org won't be getting updates to this thread any more.
Yeah I just read that news and thought of this comment trail! My description did not age well.
Hopefully this is just due to sudden and rapid uptake of users to the fediverse and not something that happens regular once things have settled down a little.
Yeah, it's not immediately obvious what the implications of an account on a particular instance are. I think over time it should become more obvious, for example meta data about instances on lemmy instance directory sites, so users know what they're signing up for. I also think things will settle down as communities find an instance that works for them, and tools improve for more granular control over federation.
What do you need to know? (As I was typing this g0nz0li0 posted a reply, I'll try to mention different things)
At a low level it's not too bad, if you've gotten this far I think you've gotten through the worst of it. Things get a bit messy when you follow a link or leave your instance though. (Link replacement seems to be the #1 complaint and is extremely easy to fix though. I'd expect it to come quickly)
But for the most part just browsing the various front page at Beehaw should do you. If you're interested in communities going to https://beehaw.org/search and changing it to all will help you find them. Just be wary because they open links that aren't made for Beehaw lol. But you can hit subscribe there and then see them under communities > Subscribed and your front page will populate with them if you goto beehaw's front page (Beehaw.org) and click subscribed instead of local or all. (Which local is communities that were made ON beehaw but anyone can see those if they search for them, and All is posts from every server on the fediverse. So they could be from anything (NSFW seems to be going crazy right now so that's most of my all feed)) There's also a bug right now that new posts get added to the top so the front page experience is a bit frustrating right now.
I like your username and profile pic. Were you an r/unclebens enthusiast by chance?
Hopefully those guides get moved over to Lemmy. I'm in Colorado where we just legalized psychedelics so I've been curious about trying to grow.
I like multis and I think discoveribility is a bottleneck, but I'm very wary of this idea. If you merge communities together like this, you essentially multiply the users in that community. Moderation isn't 4 small instances anymore - it's one large one with 4 separate mod teams each handling a quarter of the posts
I think this is more likely to lead to polarization and eventually echo chambers than if you kept them separate - outrage drives engagement more than anything else, and explosive growth is a great way for a fraction of the group to dominate the first few pages of comments, which turns off moderate voices, which works like confirmation bias to make the outraged believe they're the prevailing voice of the community, which again drives them to post more incendiary comments, and the whole thing spirals
If you want to avoid echo chambers, the best way is to throw a small group together and make them get along through mods that are involved in the community
But then you'd probably end up with most members of one community slowly joining the rest, which is a healthier growth model, but still not great
My intuition is that the ideal solution involves encouraging users to join a single smaller group, but being exposed to top posts from sister groups to avoid fomo. Possibly through something like the way Reddit handled crossposts, where you get the post but not the comments, and a small link to the discussion in other communities. It could be automated if the post crossed a certain threshold of votes, keyed to a certain deviation above the daily average of the original group and optionally with a minimum up/down vote ratio.
This would help keep moderation ahead of participation, and hopefully build a tighter knit community - people are less willing to be jerks to people they recognize than strangers you get in a larger population. By encouraging users into one small random group instead of shopping around for the one that best fits their view, I think we could resist natural grouping by beliefs.
To go further, if this works we could consider a mechanism for "mitosis", a splitting of a group when the mod team feels the culture of the group is getting past their ability to manage in a nuanced way
The goal is decentralization after all, not distributed centralized groups
Make it user specific. Feeds are combined solely from the individual user's perspective. Consumption would be easier but submissions are still federated.
I think this was the proposal - but problem is still doing this automatically.
It's not posts I'm worried about, it's comments. Comments are where the discussion takes place and the culture develops
Why not just allow the option to subscribe to them as a multi or subscribe to them individually, leave it up to the user to decide?
Because humans are barely sapient animals with limited understanding of ourselves and little to no awareness of the long-term consequences of our actions.
We don't operate in our own best interest or the best interest of the group, we're built on the assumption that the environment and our local community will moderate our actions. There's natural limits to physical actions, natural repercussions to social ones when everyone knows each other
Technology doesn't have these limits. Things made of code can scale past human comprehension in seconds. And it changes it's users
Part of the ethics of software development is to carefully consider the ramifications of what you bring into the world.
The public can't make an informed choice, because they lack both the nuanced understanding of the tech, and every choice has a cognitive load. It's up to you to make it safe and healthy or to inform them of the consequences, and you can't just put up a 26 research papers on the psychological and solciological considerations for hitting a button... No one is going to read that.
You also can't have booby traps - anything a user can do inadvertently or accidentally shouldn't have serious consequences.
There's some room for debate, but it all comes down to this: you're responsible for how an average user is going to use your technology. You should do all you can to make it easy to use the tech safely, you should add covers over the buttons that do something with consequences, and things with deeper ramifications should only be available to power users who presumably have the technical knowledge to make an informed choice themselves.
So onto this situation. Say you make this button "sub to /c/_____ and all sister communities". That's not really a choice - it's like you go to McDonald's and order a burger, and they say "for the same price, I'll give you 3 additional burgers with different options". Some people would say no, but they wanted a thing and you offer them more of the thing. If they haven't tried them before, there's fomo - what if one of the other burgers is better? And it's not like they couldn't just stop eating.
The majority will accept 4 burgers, because they don't see the hidden consequences. There's no world where the average person sits down with 4 burgers and eats less than they would if they had 1 - it's human nature, studied and documented... Giving someone more food leads to them eating more, because we judge the amount we're eating in large part visually. Put it on a larger plate or pile it higher, and we underestimate how much we've eaten. Put it on a small plate, and we eat less.
Sure, there's people who understand this - those of us who've struggled with weight or food scarcity are either not going to accept the burgers, or we'll set 3 aside for later. There's people who might benefit from eating 4 burgers - someone who burns 10k calories a day needs that kind of intake (even though they'd be better off with more variety).
Good or bad, you've increased consumption based on how you've presented this choice. The outcome was a statistical certainty, but technically it was a choice. It's just a choice that every human would naturally answer the same way if they went in blind - do you want only the thing you asked for, or that plus more free stuff.
So if you make this a button, it'll overwhelm the single sub option. And there's a game theory aspect to this - I'd likely hit the button too, because individual choices here don't matter, it's a matter of speed and volume of users subbing and unsubbing
Why not for local communities, you still display / require the local @instance name after the community name.
So /c/foo on Lemmy.world even when I am on that instance is still displayed as
foo@lemmy.world
Whereas if I search for or subscribe to “foo” by itself it displays content from the “foo” communities from all federated instances?
Some way to search other instances without knowing their name like that would be great.
It would be really nice if the search showed all results from federated communities instead of just the ones members are already subbed to
I think it would be really nice to have a "fediverse map" for each server, to show where they're connected to and what instances are endorsed back.
Would make finding new servers/communities easier too
There is a list of federated and blocked instances on the bottom of the page. https://latte.isnot.coffee/instances
Also this script is to see which instances block or are blocked by the instance you choose.
Oh great thats really helpful. Thanks!
should understand lemmy.world is just a backwater refugee camp. fediverse is much more https://fediverse.observer/map this map needs an update https://fediverse.party/
lemmy issue tracking
Why not make this purely client-side? Give me the option to merge what I see as like-minded feeds into one feed. Label it and be able to scroll it as one feed. All without the need for admins or instances to do more work?
That's how multi-reddit works, totally client-side, much better IMO than "forcing" the choice upon everyone at admin level.
Agree - Mastodon has lists, would like this too for Lemmy (even though I've not used them so far)
Why don't you read the issue? It's in lemmy-ui, so it's clearly client-side. So just because you want to waste your time going through hundreds of instances to find similar communities, do we have to force everyone else to do the same?
Oh woah, dial it back, friend
Maybe don't take disagreement so personally?
I too would like to do this myself and not have AI or anyone else decide for me what content gets lumped together.
I predict that this is also an issue that will slowly resolve itself over time, as critical masses of users gradually coalesce around one community, or more...but only if the extras are distinct in some way...which would very specifically be made more difficult by the sort of programming you're proposing.
I'm not saying there's no merit in your suggestion, only that it may not be the one-size-fits-all solution that you seem to think it is
Not taking it personally.
I would also appreciate the ability to customize, but it would be helpful to begin with a curated list of instances for each topic.
But who gets to curate that? How who has to sift through all of the 8000 instances and figures out the topic of each of likely thiusands of communities?
Automated sounds as though it's purely based off of the community name? How does it figure out the difference between table top gaming or video gaming or even slots/gambling?
What about football? American or the rest of the world?
What about politics? Is it left wing or right wing?
Seems like a cool idea at first, but when you get into the weeds it becomes a pretty complicated issue.
OptimusPrime is a Deception here, apparently.
I tend to agree with your take on this. I'm getting serious FOMO over here and over-subscribing because I don't know which sub will be the one to "take off."
I read on another post in a different community that some servers have neo-nazis running them? If that is the case, no thanks, I don’t want that.
It's possible, it's one of the downsides of being federated. The instances are moderated independendently, there is no central authority that enforces law or morals. It's great to avoid things like what happened to Reddit, but stuff like that can and will show up. You can always block instances to avoid seeing them, if you want.
How does one block an instance as a user? I thought only instances can block other instances.
Seconding this, I keep coming across Portuguese things and I don't speak Portuguese.
Maybe this is your sign to learn Portuguese...
Mas não conheço ninguém que fale português para conversar. Se eu quisesse aprender um idioma, deveria escolher um local para realmente usá-lo.
Holy shit that was fast. I didn't know Lemmy had a language extension feature.
My bad, I meant community. You can block communities individually. For blocking instances as a user there's an open issue on Lemmy's GitHub, but I believe it's not going to be implemented anytime soon.
That's easy to block those individual instances if needed (assuming they exist).
How do you block an instance as a user? I know how to block a community from an instance, and i know an instance owner can defederate another instance, but how do I as a user block a whole instance?
Actual neo-nazis or people you disagree with?
We have the real deal in my state.
They don't care what you want..
We need both options. Some systems like USENET use a global groups list (rec.radio.amateur.misc is the same group everywhere). Federated communities need a similar option.
Sure, let me create my own c/gaming if I want, but also give the option to… merge? combine? cross-federate? Not sure what term fits here.
!gaming@me, !gaming@you, and !gaming@them can be 3 separate, distinct, and independent communities (like it is today).
!gaming@me, !gaming@you, and !gaming@them could also be the same !gaming community, replicated and synced across all 3 servers.
Here’s an idea. Add another name to the community designation. So you could have !gaming#context@instance. (Or whatever separator makes sense. You could even just use a subdomain like !gaming@context.instance, but that might be harder).
In this model, #context refers to a shared view of the world that instances can choose to participate in. As the instance admin (or maybe a mod??), I choose to join #context1 but not #context2. When I do, All the !communities under #context1 become available for me. I still choose the ones that are appropriate for my instance. This would mean that when a new instance joins the federation, it acquires the shared set of #contexts that the federation publishes. A different federation of instances could still have different contexts.
(All of this still feels clunky. USENT’s simple hierarchy still makes so much sense, but it unfortunately places all the control at the group level, not the instance/user level.)
I recently suggested @global for consolidated communities. There would have to be some kind of consensus on who and how communities would consolidate. I agree that having that does not get rid of all the other permutations.
I'd say just let the cross-federated communities redirect to each other.
For what it’s worth, I’m VERY much for this.
One of the pain points for those coming into this to fill the Reddit void is fragmentation. Beyond being a huge improvement in usability, information would be shared much more easily this way. For someone who spends a lot of time in IT/tech related subs, that’s very important to me.
I think you perfectly described my issues with comment sections on Reddit for the last few years. That attempt to appeal to an audience rather than further the discussion.
I used to love comment sections as much as, if not more than, the actual post on Reddit. It felt more like a conversation that had insight and humor. It got too big for it's britches and became that soulless monolith.
I get an almost nostalgic vibe from this place. It's nice.
Well, it's the nature of almost any online community. Say the 'popular' thing and you're lauded, even if a slightly less popular point is more valid / has better evidence.
There really is no good way to discourage this other than fostering a community which values the discourse over 'popular' thing. That's difficult to do even offline.
I'm not sure it's even possible to discourage it really. If you have any sort of user-user engagement system, whether up/downvotes or comments/shares or whatever, you're going to have particular sentiments that are popular with particular audiences and get more of that engagement. If you take those features out, you're going to lose engagement, pretty much definitionally.
I've always thought about creating some metric to weight users who create comments with the most engagement as higher. That leads to the most controversial or dividing comments rising though.
Some impartial judgement via mod points and or community awards to weigh valuable users would be nice.
The issue is any of these would be gamed, it might be possible today to use an AI model like ChatGPT but that's got its own biases.
So for the moment I can't think of a better system than upvote downvote.
Yeah, weighting 'engagement' higher is basically the youtube algorithm problem: you'd be attracting trolls most of all. You could probably devise something smarter, like weighting it to include all of most upvotes, fewest downvotes, and most comments; adding comments to it helps identify people who post positive but engaging things, but again that can lead to an echo chamber. Plus, it then under-weights new users compared to established ones, which can be unfortunate.
Yep. It turns out there is no such thing as a 'balanced' social network.
Which is analogous to life, depressingly enough.
I absolutely agree with you on this point. Comments used to be about commenting and carrying a conversation. Then they suddenly became monetized and sought after for the likes. Funny overrode useful and now comments are a trash fire that make one think twice before starting to engage with it.
The amount of times I've had to scroll and scroll for answers is way too high. Everyone thinks they're a comedian and that's always the top comment. So frustrating.
I see very little discussion about the implications of this for moderation, and it feels to me like they get very sticky. With traditional human-curated multi-reddits, you as a subscriber must engage with the idea that you are choosing to aggregate multiple communities into a single feed, which is intuitive enough, the subscribed feed already works that way.
But by making it automatic, the software hides the fact that it's pulling together discrete feeds from communities with different rules and different moderators. This feels very awkward to me. I'm all in favor of traditional multi-reddits, which can be used to create this sort of feed for yourself. I'm still on the train of "duplicate communities will sort themselves out if community discovery is made much easier and popular communities reliably show up at the top community searches, mostly irrespective of what instance the search was performed on" (obviously defederation takes precedence here).
Honestly, we've all seen this happen in every community that fragments for some reason. Whenever reddit banned a subreddit, 1,000 mini-subs would spawn, eventually coalescing into 1 or 2 main communities. Whenever a migration happens (Tumblrinos moving to reddit, redditors moving to various reddit-like things, forums moving to reddit, fark migrations, usenet, IRC, etc) they always fragment initially only to end up with a relatively few coalesced groups that hold similar ideas.
c/news might end up with 10,000 news subreddits (even one for each local newspaper!) but eventually it comes out to something like: c/news, c/leftnews c/rightnews c/ihateallnews with the most active (by far) being one of the generic ones. The others might technically exist, but if you search "news" and get one that has 10,000,000 subs and another that has... 3, well, you're just not going to bother.
I think it would need to be voluntary; mods on different instances have to agree to be a joint mod group. If an instance decides to break up, they take their mods with them, etc. Might he complicated, but it seems doable.
If the mods were willing to agree to this, why would they not also agree to simply merge the communities? Frequently, when mods choose to maintain overlapping/competing communities/subs... it's because there's conflict about culture or moderation policies. This mechanism feels very duplicative of existing patterns, and limited in much the same ways when cooperation breaks down between mods of different communities.
There’s an advantage to splitting the load amongst multiple instances. If community@host1.com has issues, the subscribers to community can still access community@host2.com.
Further, preventing centralization is a core focus of the fediverse, with all the pros and cons that come with it. The reddit migration is a testament to that philosophy.
If an in-fight happens between two instances’ communities, host1 won’t be able to boot host2’s entire existence, just their agreement to be in sync. Voluntary syncing of communities seems to me a natural extension of the federated philosophy.
I don't have links handy, but the devs have said multiple times that federation replication load is not a limiting factor. It's the browsing load that matters, which is already spread across the instances that subscribers hail from. There isn't a performance issue here to fix, at least around the current network size.
Again, user-account federation already provides this. If the host where the community resides is down, I can still read existing posts, and post and comment with users on my local instance. I don't see significant further benefits in splitting the community hosting.
I don't disagree with this, the challenge is that the federated philosphy makes everything it touches very complicated. A sprinkle of federation in the core of an ecosystem brings enormous resilience against the ecosystem getting coopted against the will of its users. A heaping spoonful of federation makes it an unusable mess. The complexity of federated design needs to bring very big benefits to "pull its weight" in complexity. The tradeoff looks positive for me for federated replication even though the cost in complexity is large, but I don't for meta-communities. They seem like cosmetic improvements over the existing capabilities.
But fair enough, I see where you're coming from. It's not madness, on-balance it's just not something I see providing value in proportion to its conceptual complexity.
Can you help me understand how that all works? I ran into this today where an instance wouldn’t let me submit due to performance issues, and I jumped instances to be able to provide some (potentially) important breaking news to “community.” If they were fully in-sync and one instance didn’t “own” the community, when things stabilized, my content would have replicated instead of being stranded on a lonely instance. As it stands, that content will never be part of the “real” community.
Right on. I’m still new to the fediverse and am probably missing some foundational concepts still. Love where it’s headed though.
I'll preface this by saying I haven't read the code and it's certainly possible there are errors in my mental model, but my belief is:
It's possible I'm mistaken about 3-5, but if you had trouble posting... my guess would be that your own user-instance was struggling and slow. But if you've rules that out maybe the update behavior when the community-instance is down works differently than I think it does. The reading in step 2 I'm quite certain about though.
I'm pretty sure you can't post to community-instance if they're down/having issues to the local DB. The instance I'm on is very stable and every time I think it may be an issue, it turns out to be the other instance.
It sounds like a lot of the "sync" behavior I was thinking of is already built-in if we can just expand on it a bit.
I can't disprove this. My experience when
lemmy.mlwas struggling was that I could comment-reply and vote consistently even though many of the communities I'm active in are on that struggling server. But maybe I was just lucky. I can't explain the behavior you observed, and it may be that certain operations are more directly passed through to the community-hosting instance than I previously believed.But irrespective of the details, plain federation does provide some significant resilience to transient instance failure, and there's probably room to tune it further in at least some ways.
Interesting discussion, though.
Ah, yeah I should clarify that I was able to comment on existing posts, but could not submit a new post. Fun discussion, agreed!
That's the difference then, I do A LOT more commenting than posting. Perhaps there's some essential reason they're handled differently or hopefully an accidental one and posts could be made to behave more like comments someday.
on one of my other accounts i am collecting communities that all have the same name across instances just so that whichever place people end up moving to for that is consistent with the others. having them all roll into one would be a massive amount less work and is pretty much absolutely ideal for what i'm trying to do.
there absolutely needs to be an opt-out or even an exclude feature tho. as someone pointed out, some instances are run by not very nice people (tm) and there are also regional-based instances like my native one the aussie zone. yeh sure it'd be great if all the "australia" communities were merged (or most of them at least) but 2nd or 3rd for size is the melbourne community, and there's also a melbourne in florida and people from neither one particularly enjoy being mistaken for the other. region-based communities need to be able to maintain their own exclusivity for community reasons, yeah? another community on the aussie zone is called "gigs" and it's not about rock concerts that happen in new york city if you get my drift. you have a great idea, but it needs to be optional, and easy to withdraw from if desired.
I like the general idea of merging communities, but I'm not sure if I like the idea of it being automatic. What if instead communities could apply "hashtags" for their community, and then you could efficiently browse multiple communities at once. For example, I'm subscribed to a few different TTRPG communities across a few different instances, but what if each of those communities was tagged "#ttrpg" and then I could browse #ttrpg instead of browsing any of those individual communities.
I think conflicts can arise with hashtags just as easily as with community names, so it might be better to have an updatable and moderatable link instead.
Can you explain how conflicts can arrive with hashtags? I thought the whole idea is that they just let you tag content with its topics
Oh, I thought it was about tagging the communities and merging them based on those tags. Are you suggesting tagging the posts instead, and displaying all posts tagged the same from all communities across all federated instances in the same location?
But this can already be accomplished with the search feature. You only need to select 'Local' or 'All' and search for a word. People shouldn't be forced to hashtag every post, so the result would be the same as it is now.
It could be done at the community or post level. I think because of the Mastodon backend Lemmy already supports hashtagging posts. But yeah my idea is that tagging communities would make it easy to merge communities with similar scopes
There could be conflicts between hashtags, as a hashtag for one community may not have the same meaning for another community. This would result in mixing topics and potential confusion.
I suppose that's possible, but I don't really view it as a serious problem that sometimes words overlap.
But one thing to consider is how divided it would be. Let's say I wanted to browse martial arts so I go to #martialarts. Now what about people adding stuff like #MMA or #karate. None of these would show up in #martialarts. Seems like hashtags would be even more divided than communities to me
The idea is that one community could apply many tags. So if you have karate communities from several Lemmy instances, browsing a multi-feed of #karate would show all of the communities you e subscribed to where the mods tagged their community with #karate. But the mods could also additionally tag their karate sub with #martialarts, and then it would show up in multi-feeds filtering on that hashtag as well
I would definitely consider that a serious potential issue, if for no other reason than so many communities will likely find a use for tags based on the nature of the community structure.
For example, I could see a ton of communities having tags for things like modposts, new member intros, meta topics, memes, questions, reviews, how-to's/tutorials, guides, etc. and that's just for broad post types that would apply to thousands of communities.
I think letting users manually make their own multi-lems, perhaps with the ability for communities to sort of team up to make uber-lems of closely related communities to help users discover more of them...but sub, unsub, multi, and un-multi as they see fit...is likely the best approach.
I'm not talking about communities having tags for posts, I'm suggesting communities could apply a hashtag to the community itself so that it would be easy to combine view of many communities with the same tag.
To use a Reddit example, imagine if r/gaming, r/games, and r/patientgamers all had "#videogames" applied to them. Then if you tried to view a multi based on which subreddits had #videogames, all three would show up in that multi.
I am working on a multi lemmy manager that Id love to get some alpha testers to.
As a web fronted, or mobile app?
I'm using jerboa (android app) and would find an equivalent of reddit's 'multi' system useful. I'd likely switch to an app that provided that. I wonder if it could be implemented solely within an app, or if it would need backend support from lemmy.
I'm interested to test it!
I don't really know what that would entail but I might be interested.
Another day another "When will the thing not meant to be Reddit be Reddit" post
Hmm, I don't disagree with the fragmentation but that's the nature of any new social platform. It's also been proven out that eventually one or two communities for a topic will become the dominant one with the others falling into disuse.
Attempting to merge communities early or artificially will cause moderator strife as minor disagreements balloon. Especially in a multireddit community where no one mod(team) has absolute control.
I don't have a reason from a technical point of view, but from a social one. Forcing communities and instances together early will only cause strife. After a few years where two communities have a track record and proven 'behavior' would the multireddit not cause issue.
Every moderator would have control only over the content displayed on their instances, and not on everyone else, as it should be. The argument about having one or two large communities is a recurring one. There is no reason to have federation if we are going to centralize communities in a couple of instances. Then, if one of those instances shuts down, everyone in those communities would have to migrate. The main benefit of federation is decentralization.
How will this help the posters reach the fragmented communities? Will they just pray that everyone is using the the aggregator?
Lol this is a very good point. I'll give it some thought.
Cross-instance "multireddits", that are also automatic and topic-based. #1113
TL;DR: The suggestion is to implement an automatic multireddit feature in Lemmy that displays all posts from communities with the same name across federated instances. It aims to promote decentralization, avoid echo chambers, and ensure high availability. Community moderators would have the option to opt-in or opt-out their communities from being displayed. There are discussions about potential issues such as community name collisions, duplicates, abuse, and practical implementation. Some propose using a new link format, while others suggest providing users with a list of related communities.
I can definitely see name-collisions being an issue, where communities on different instances have the same community "ID", but aren't actually about the same thing. I'm still overall in favor of the basic idea though.
I think what I'd prefer is a more supported version of Reddit crossposting... I'm brand new so stop me if Lemmy already does something like this. For example, if someone has a vegetarian recipe community, a more general vegetarian community might automate a feature to crosspost their content, with clear linkage to the source... But a new discussion thread as the default in the crossposting community.
This allows the different related communities to choose their own moderation and regulation, but can also allow communities to be content aggregators.
I don’t think I’m understanding this right cause it sounds like you’re trying to make it more fun by adding more rules. If there are 20 groups that are all about pickles that’s fine they each like running things their own way. Eventually one group gets popular and that’s where the majority goes. I think your frustration could better be solved with something like tags where groups could choose to associate certain tags words that makes search easier like tag: pickles-fermenting-homemade-cucumbers and that could clear up search from people just wanting to share pickle Rick memes.
Yes the fragmentation is a feature not a bug. There are dozens of reasons why people might want to leave a community and create their own alternative version. With blackjack and hookers.
Combining communities should be a front end feature... Allow users to merge their views if they want. But it should not be enforced at the backend or federation level.
Eventually there will be third party apps which can do this merging in their interface if someone wants it.
I agree with this. The grouping should be a front-end feature based on hashtags, as someone else mentioned, instead of the community names. Alternatively, there could be lists that you can simply copy and paste to create your own multireddit, eliminating the need for hashtags. However, considering that the original issue was already on the lemmy-ui, I'm not sure why you brought up the backend.
One thing I do agree with is that discoverability should be better. Can't remember the url now but there's a site which crawls all instances and creates a searchable list of all communities. This should seriously be built in. Have all instances generate a list of all communities which is broadcast, cached by all, and searchable by all.
This "paste a url of a community you already found into the search box" concept is not good enough.
Yeah, couldn't agree more.
https://browse.feddit.de/
This seems like a devisive topic.
The fragmentation is not a feature to me, but something I view as detrimental.
I like the idea that tags could facilitate a sort of one-to-many and many-to-one relationship structure, where a pickle community could say it is related to "Canning" and "Cucumbers" among others, and potentially populate in the feed of someone interested in prepper stuff or in the feed of someone interested in vegetable gardening. I'm sure I'm not using the DB terms correctly but they feel indicative of the modular structure this could allow.
I like this flexibility better than the idea of a consolidation of subs into one themed silo, but I could be misunderstanding the structure of the mulit-reddit proposal if it allows for this.
The thing is that say i want to see all the pickle content across the lemmy-verse, i want to just be able to subscribe to an umbrella "pickles" category and get all the c/pickles content from any instance my instance federates with without worrying that i missed a community or something. If there are 20 groups but i only know of 1 or 2, odds are that i'll miss the biggest ones with a bunch of pickle content that i want. And I don't want to have to manually go through, search for all the biggest instances, and subscribe to each pickle community one by one. Plus, say a new instance is started and it's pickles community blows up. I'll miss it because i already searched through and subscribed to all the pickles communities that were available when i joined. I'd rather default to subscribing to all the c/pickles communities my instance sees, and then if one instance is posting stuff i don't want to see i could manually exclude them
Tldr I guess it depends what you think will take more effort to do manually. I think having to manually find and subscribe to every community i want from across the lemmyverse (and periodically run the search again to find new communities) would be way more work than subscribing to every c/pickles my instance can see and manually excluding the instances or instance-specific-communities i don't want content from
I'm new to the whole concept of federation, so please let me know if I'm missing something fundamental, but with your proposal (subscribing to all c/pickles) works for when each instance has the same general rules for posts, but what about when one c/pickles on some server is actually about... dildos? (like r/trees was about pot, and r/marijuanaenthusiasts was about trees)?
Wouldn't your feed be polluted with pics and reviews of dildos when you want wholesome and healthy pickle recipes?
That's where manually being able to block specific instances would come in. So if you subscribe to every c/pickles your instance federates with but then didlos.lemmy/c/pickles starts causing problems, you could manually block that one instance. Or maybe you want to see dildos but not when you're looking at pickles, you could choose to manually exclude didlos.lemmy/c/pickles from your c/pickles subscription
That makes sense, but sounds like it'd be a constant spam battle. Maybe I'm overthinking it, or just thinking about how I'd go about breaking/attacking something to try to figure out how to fix it.
Yeah it could be, but keep in mind it would only include communities from instances that your instance federates with. So if your instance has blocked another instance it wouldn't be included in the group. This issue actually describes it really well and is better thought out than what i've described https://github.com/LemmyNet/lemmy-ui/issues/1113
Or just make it user-side. Let users create their own feed combinations. They'd still have to select a specific instance for posts.
Feeds would be consolidated but posts and comments will still be federated. And one user will be unaffected by how another user organizes their feed.
That seems like a good solution. Let me subscribe to a half dozen different game comms and put them together into one "games" list that shows up with pretty much the same interface as a single community, so I can browse just "games" content that I have subscribed to.
having subscribed to multiple games communities, the issue then becomes content duplication; the same trailer or article will get posted in three different communities, and I don't actually want to see it three times in my feed. I'm not sure there's a good way to solve that, though.
(I'm subscribed to multiple communities b/c I'm not sure which one will have the largest comments sections, and those are what I'm really interested in.)
I personally like the duplicates because different communities have different comments.
Later on there could potentially be an "exclude duplicate links" option which will automatically filter out the duplicate posts which contain the exactly the same link and show only the post with the highest interaction count. Unsure if that would be valuable enough to implement as a feature though.
The issue I then see with that is well, which duplicate instance gets to be the Highlander? 😋 Does the system automatically show you the post with the most comments? The one that's had its link clicked most? The one from the largest community?
Also, how do you prevent this from being hijacked? Say I post some bright, shiny new OC meme, only to have some bot immediately repost it with exactly the same title, etc...
Not that I have any solutions for any of this, mind...
Yeah these are all great points. It would definitely need to be workshopped to see if there are solutions to some of this or if other approaches are better.
My first thought would be that the top duplicate in the feed would be the one that would be shown (so based on up/down votes I guess). The others would be lower in the list anyway so you would only see those duplicates after the top one if they were there. Simply filtering them out and maybe having a way to show a list of duplicates on the visible one might be good. Then you could choose the comment section of the smaller duplicates of you want to read more comments or whatever. Still lots of unknowns but I feel like it could possibly work.
Of course, it's a whole new platform, there's bound to be all sorts of things that need ironing out. Tbh I just think it's a good sign already that these are things that are being discussed (or are open for discussion at all, for that matter) and not just unilaterally decided.
Also, I hope my comment didn't come across as being disparaging or anything... All my criticism was intended to be constructive, I promise
Nah, all good! This is the kind of discussion I have at work when we're deciding what feature to implement or the nuances in the implementation so I definitely don't see it in a negative way. In a week or so when I have more time, I'm going to see if I can put some time into helping fix bugs or improve the android app. It's exciting to potentially be part of the group to shape the platform at these early stages!
I don't know why you are bitching about rules and frustration.
I believe the best approach would be to have these multireddits automatically created for convenience. However, users should have the option to choose whether they want to see only the content from their instance's community or from any number of communities, instead of being forced to view all of them.
I think that approach is needlessly complicated and would confuse people more than help them. There should be a way for individual users to merge the feeds from multiple communities in a multilemmy if they specifically choose to.
But I don't see any way to create such multireddits/multilemmys automatically because there is no objective categorization. Everyone is going to have a different opinion about which communities fit together and which ones are similar but different enough to keep separate. Instead of forcing a generalist solution that makes everyone unhappy, just give people the tools to build their own solution that works best for them.
I would prefer something pre-made for convenience but that can be modified by each user to adjust to their preference. I'd rather have a generalist solution forced on me than have to spend countless hours grouping communities from hundreds of instances.
I think you're vastly overestimating how many communities actually matter. At least 90% of communities will be ghost towns. It's just too early to tell right now because the platform is basically in its first week of existence. You're planning a solution to a problem that won't exist.
I get it, some of the first comments I made when I joined were about how to combine communities across multiple instances. As I've spent more time here, I've begun to understand that it's not actually a problem. The easy solution is to subscribe to the biggest community and/or one on your home instance.
Besides that, I don't see much of a difference between manually going through hundreds of communities to add the ones you want, or going through hundreds of communities on a pre made list to remove the ones you don't want.
The problem is not a technical nor architectural problem but a user/usability issue.
Look at the workflow how people create a new community. They are registered on one instance, probably fixed to that bubble and probably don’t interact with other instances at all (subscribing to other communities is a pain. Other problem.) They might (if at all) search for a similar community on their instance. If they don’t find one, they’ll create a new one. Searching every single community is not implemented in this flow. You need to call up feddits search to do so.
My suggestion: Either do a name (fuzzy) check when creating a community, listing the ones already existent on other instances. Or at least implement the search feature from feddit.
I don't see any problem. If the problem is the server admin is not able to moderate the communities, he can disable automatic community creation and define an approval process. He can also close dead communities, apoint moderators, merge proposals...
Sounds like something. I don’t know enough about the tech to provide a meaningful comment but I can see the problems you’ve mentioned and something doing these lines sounds like it could be a good solution.
I think a community needs to add the option of a topic hashtag, so communities can group together in that way if they want to, and people can subscribe to these hashtags if they want to (with the ability to remove single instances of they want to)
The theme here is "if they want to" giving choice to both the communities and the subscribers
Agree, would be a great mix!
Communities are sort of already hashtags when using the microblog post type. Seems like an interesting approach.