Just an explorer in the threadiverse.
I thought that was the first rule of rendering web content? Or was it protocol parsers?
I remember, it was first rule of video game character creation screens:
Tests go in !test@lemmy.ml
The terms-of-use on every lemmy server I've seen would be considered underdeveloped by any lawyer I've ever met. Pragmatically:
Overall, I'd say most of the lemmyverse has underbaked policy frameworks. The de-facto results function ok pragmatically anyway for what lemmy does on its own. Any scraping/reuse of content from Lemmy would have to navigate a very complex, confusing, and ambiguous licensing landscape. Probably 10y from now, if the Lemmyverse continues to grow, TOU's will be more common and more clear about open-licensing content or leaving it all-rights-reserved but giving lemmy a perpetual irrevocable non-exclusive right to distribute whatever you post here (the latter of which is more or less what's implicitly happening today).
Have you considered modifying this to be a pull-request for lemmy-ui? It's cool to have an extension, but Lemmy being open-source makes it possible to merge the improvement upstream and make it better for everyone.
I blame unfederated subscriber counts. If you look up any community from an account on lemmy.world and there is a local version and a remote version... the local version LOOKS bigger when it's about half the size because the remote version only shows subscribers from lemmy.world whereas the local version shows subs fediverse-wide.
If sub counts were apples to apples for remote and local communities, people would much more frequently sub to the bigger remote comminity. But lemmy.world is so big, that when people are subbing locally because they're confused about which is bigger... the lemmy.world community actually becomes bigger very quickly. So it's winning the community scaling races consistently on pure confusion. The resulting community centralization is not all that healthy and they often overtake better run and more established communities for no meaningful reason.
Two tips:
I have not tried running WINE yet but I plan on doing so soon.
Steam "just works" on Linux, you can install it via flatpak (which I use) or from their deb repo. It includes "Proton", which is a fancy bundle of wine and some extra open source valve sauce to make it nice and easy to use. Any game that runs on the steam deck also runs on Linux via proton, and there's no messing around at all. It looks and feels just like steam on Windows, and thousands of games just work with no setup or config beyond clicking the big blue and green buttons to install and run. Not EVERY games works, but tons do. I'd heavily recommend this over raw wine to a beginner.
The second tip is not to ask what you can do on Linux. The answer, to a first approximation, is that you can do everything on Linux that you can do on Windows or OSX. I daily drive all three, and mostly do the same stuff on them. Instead, ask YOURSELF what you WANT to do on Linux. Then Google and ask us HOW to do it... or what the nearest approximation is if the precise thing you want to do doesn't work on Linux.
11,263 lbs, huh? It's not a kind estimate, but not unrealistic either.
They're different communities, just like /r/tech and /r/technology, /r/DnD and /r/dndnext, or the million different aita subs that popped up last month.
There is a GitHub issue for the Lemmy equivalent of a multireddit which would allow you to create a compound feed of several communities. Others have gone further and requested some kind of automatic merging, which strikes me as a pretty terrible idea... they're different communities with different rules and different mods and maybe different cultures. Sometimes they exist separately because the mods don't like each other or have very different ideas about what the culture should be. Transparent merging in such cases is awkward and creates confusion.
My advice is to consider the server name as if it were part of the sub/community name so that !this@that.com
is just a different thing from !this@there.com
. Dupe subs have always been a thing on Reddit, they're a thing here too. They will get better with time as community discovery improves and people aggregate in the active/well-moderated ones and the abandoned ones die off.
I think the issue is that .world has put itself forward as some sort of super lemmy.
Citation needed. All the admins of lemmy world ever purported to do was host a well-run general-purpose (aka not topic-oriented) lemmy instance. It was and remains that, and part of being a well-run general purpose instance is managing legal risk when a small subset of the community generates an outsized portion of it.
Being well run meant that they scaled up and remained operational during the first reddit migration wave. People appreciated that, but continuing to function does not amount to a declaration of being a super lemmy.
World also has kept signups open through good times, and more recently bad. Other instances at various times shut down signups or put irritating steps and purity tests along the way. Keeping signups open is a pretty bare-minimum bar for running a service though, it is again not a declaration of being a super-lemmy.
Essentially lemmy world just... kept working (until recently when it has done a pretty poor job of that). I dunno where you found a declaration that lemmy world is a super-lemmy, but it's not coming from the lemmy world admins, it's likely randos spouting off.
Not anymore, at least for (an unspecified) while. See https://lemmy.world/post/149743.
In short, beehaw is having trouble moderating there communities, and defederated several of the biggest Lemmy instances just to cut down on moderation volume. It's was an incredibly short-sighted thing to do, and will make the lemmyverse a much more confusing place for the foreseeable future, but it's within their power to do.
Is it necessary to add an instance to the allowed list? If federation is enabled, isn’t an instance ’allowed’ by default?
Your understanding is correct, and making your "allowed" list non-empty is a big deal because it implicitly defederates you with every instance that isn't in that list. I rather suspect that OP doesn't understand what the allowed list does and is trying to find ways to promote small Instances while lacking knowledge about how things work.
Defederating from all major instances would almost certainly relegate small instances doing so to irrelevance.
This is a terrible idea, and borderline irresponsible. One of the key reasons that Lemmy doesn't subscribe by default is to avoid forcing servers with many communities to waste time/CPU delivering messages to servers where no one will read those messages. By subscribing to everything, you're telling all those overloaded servers to waste time sending content to your server that you'll never even see.
It is much MUCH better to just hit lemmyverse.net and subscribe to 10-100 communities you care about. If script accepted a list of community-urls and automated subscribing to those, that would be super nice. Subscribing to the entire lemmyverse is terrible for your server, for your hosting liability, and for the lemmyverse's performance.
The upsides are that you control your defederation list and you're your own admin so you're in control of whether your instance goes down and what it's policies are.
The downsides are:
Others have answered the crux of your questions, which is that it's basically donations... either from the admins by providing free access to their server, or by the community through Patreon or whatever.
But to put into context how much money we're talking about...
Now, if you count admin/mod time and expertise, of course... those costs would be huge. But those people either volunteer or get a bit of money from non-profits. But the hardware costs are modest.
My money is also on IO. Outside of CPU and RAM, it's the most likely resource to get saturated (especially if using rotational magnetic disks rather than an SSD, magnetic disks are going to be the performance limiter by a lot for many workloads), and also the one that OP said nothing about, suggesting it's a blind spot for them.
In addition to the excellent command-line approaches suggested above, I recommend installing netdata on the box as it will show you a very comprehensive set of performance metrics without having to learn to collect each one on the CLI. A downside is that it will use RAM proportional to the data retention period, which if you're swapping hard will be an issue. But even a few hours of data can be very useful and with 16gb of ram I feel like any swapping is likely to be a gross misconfiguration rather than true memory demand... and once that's sorted dedicating a gig or two to observability will be a good investment.
... advertisement and push they did on sites like reddit...
The lemmy world admins advertised on Reddit? Can you link an example?
... their listing on join-lemmy.org...
Until recently EVERY lemmy instance was listed on join-lemmy.
And with the name Lemmy.world they did nothing to dissuade anyone from thinking that.
They run a family of servers under the world tld, including at least mastodon, lemmy, and calckey. They're all named similarly.
I also saw nothing from .world not claiming to be the bigger instance(super lemmy)
They ARE the biggest instance, but that happened organically. It's not based on any marketing claims from the admin team about being a flagship/super/mega/whatever instance. People just joined, and the admins didn't stop them (nor should they). It's not a conspiracy to take over lemmy. It's just an instance that... until recently... happened to work pretty well when some were struggling.
I'm mostly in the pro-written word camp myself, but I have sought out video tutorials in cases where written docs seem to assume something I don't know. When I'm learning something new, a written doc might have a 3-word throwaway clause like "... add a user and then...". But I've never added a user and don't know how. If it's niche open-source software with a small dev team, this may not be covered in the docs either. I'll go fishing for videos and just seeing that they go to a web-ui or config-file or whatever sets me on the path to figure out the rest myself.
That is to say, video content that shows someone doing a thing successfully often includes unspoken visual information that the author doesn't necessarily value or even realize is being communicated. But the need to do the thing successfully on-screen involves documenting many small/easy factoids that can easily trip someone inexperienced up for hours.
I'm as annoyed as anyone when I want reference material and find only videos, and I generally prefer written tutorials as well. But sometimes a video tutorial is the thing that gets me oriented enough to understand the written worthy I wasn't ready to process previously.
Edit: The ubiquity of video material probably has little to do with it's usefulness though, and everything to do with how easy it is to monetize on YouTube.
I posted a comparison a short while ago: https://lemmy.world/post/1452988
I recently decided on headscale as a coordination server with tailscale apps/clients for my setup. My rationale was:
Others have pointed out that docker containers aren't idiomatically like VMs and you interact with them differently. Some workflow tips:
The other issue in play here is that the fundamentals necessary to understand how docker containers run aren't actually docker fundamentals. They're:
These things aren't trivial to learn, a thorough understanding of these things is the difference between a junior sysadmin and a senior one, and you WILL get exposed to them when things break. But step one in learning more is definitely to recognize that the hards parts of docker are rarely docker itself. Rather, the hard parts are their own thing. Figure out what Linux system you're struggling with and start learning about that and how docker uses it rather than narrowly focusing your research on docker itself. Anything focusing on the docker piece must necessarily gloss over the real foundations which are often very complex... so this will start you expose you to deeper material that you can assemble in your own mind into an understanding of your own specific docker setup.
The earth and sun and milky way are temporary. Like these things, the Beehaw defederation is ongoing and has no predetermined end.
Thank you for stepping in to bring this under control. I'm subbed here for the admin announcements and meta discussions, and the free for all here in recent days has been irritating.
Another thought... I feel like the community name lemmy.world
kind of invites confusion as to being about "the whole world Lemmy related topics". Hopefully once the mess is cleaned up the recent posts will make the topic obvious... but other instances often call their equiv community meta
or something similar that invites less confusion. Adopting a more specific name might reduce the confusion and moderation load. If that happened, I dunno what happens to this community. Maybe just shut it down, its free for all form seems unsustainable.
You misunderstand what the Hot rank is doing. It's not balancing newness vs hotness, it's scaling hotness according to community size. This might feel like newness if you're focused on vote counts as a proxy for post age, but it's a different approach. See https://github.com/LemmyNet/lemmy/issues/3622 for details.
There's a couple ways to think about this:
At any rate, this preference toward smaller communities in hot is a recent change and deliberate. While they might further tweak the scaling factors, I wouldn't expect it to be drastically different. It sounds to me like what you want is Top, Active, or Most Comments. All these are unscaled according to community size and will get you top posts by their absolute metric rather than posts that are doing well relative to their community size.
I don't have a link handy, but I saw this reported before and when someone went to lemmynsfw to check the posts, they actually had been properly marked as nsfw.
My unsubstantiated theory is that federation is working pretty crappily right now and so lots of federation messages get dropped. When a new post or comment gets dropped you don't notice because... well... you don't see anything. But when a post makes it through and an edit gets dropped, that can be more visible.
So likely what's happening is that on some small percent but medium absolute number of lemmynsfw posts, someone makes an honest mistake and fails to tag it. They either notice themselves and fix it, or a mod asks them to and they comply... so on lemmynsfw it looks right. But the federation message with the edit gets dropped by your instance, and for you and others viewing from there it forever remains sfw in error, even though all the right steps were taken. I've also heard mods discussing this happening with post removal moderation actions, the post gets removed on the community's home instance, but the federation message containing the removal gets dropped by some big instance and the post blows up there anyway where the mods are actually unable to shut it down.
I don't think there's anything anyone can do about this other than maybe the lemmynsfw admins tuning their federation worker counts (though they may have already and that may no longer be needed in recent Lemmy versions), or the Lemmy devs working on federation scalability so a larger percentages of federation messages get reliably delivered.
I use k8s at work and have built a k8s cluster in my homelab... but I did not like it. I tore it down, and currently using podman, and don't think I would go back to k8s (though I would definitely use docker as an alternative to podman and would probably even recommend it over podman for beginners even though I've settled on podman for myself).
Overall, the simplicity and lightweight resource consumption of podman/docker are are what I value at home. The extra layers of abstraction and constraints k8s employs are valuable at work, where we have a lot of machines and alot of people that must coordinate effectively... but I don't have those problems at home and the overhead (compute overhead, conceptual overhead, and config-overhesd) of k8s' solutions to them is annoying there.
Is there a point where there are so many instances that propagating all that data is too taxing
Yes, that becomes a concern as the network size grows and the amount of aggregate replication traffic increases. Mastodon has like 10x the server count of Lemmy, though... so that's hopeful. They do use ActivityPub differently though, it possible that federation scales differently between them.
This GitHub issue has a lot of good (but rough and high-level) thoughts on future scaling techniques: https://github.com/LemmyNet/lemmy/issues/3062
Do I need to set up NGINX on a VPS (or similar cloud based server) to send the queries to my home box?
A proxy on a VPS is one way to do this, but not the only way and not necessarily the best one... depending on your goals.
Do I need to purchase a domain (randomblahblah.xyz) to use as the main access route from outside my house?
Not for tailscale, and I don't think for Cloudflare tunnel. Yes for a VPS proxy.
I've run a VPS for a long while and use multiple techniques for different services.
Because of the federation, your votes are not technically anonymous on Lemmy. At least, I think.
I was a little skeptical of this assertion without any sources, but 10m of source scanning does seem to support it:
I haven't looked for APIs to extract this data, it might only be available to an instance admin... but yeah Lemmy does not seem to aggregate vote histories, but rather stores them on a per-user basis.
I use Headscale, but Tailscale is a great service and what I generally recommend to strangers who want to approximate my setup. The tradeoffs are pretty straightforward:
Tailscale is great, and there's no compelling reason that should prevent most self-hosters that want it from using it. I use Headscale because I can and I'm comfortable doing so... But they're both awesome options.
https://www.quora.com/Is-there-an-open-source-alternative-to-Blackboard
The above link looks like a pretty reasonable answer to this question to me. In short, Moodle, Canvas, Sakai, and OpenEdX are all open-source e-classroom solutions. They are not really targeted toward casual self-hosters though. These packages are typically run by full-time engineering staff on multiple beefy servers at schools where the setup serves thousands of students. If you're quite strong technically, it sounds like Moodle might be at the easier end of things, but I don't commonly see individual teachers standing up their own Moodle servers, but I also don't hang out with technically oriented teachers so maybe I'm not in the right crowd.
There may be something oriented toward more casual self-hosting, but these are what I'm aware of. I haven't used them though.
Thanks for your work debugging and fixing this for everyone.
Rule 3, this is not a community for Lemmy support. There are Lemmy support and Lemmy admin communities, try asking there.
Another user posted the blog where they discuss their speedup techniques: https://tailscale.com/blog/more-throughput/
It's likely that the kernel version can use similar techniques to surpass the performance of the userspace version that tailscale uses, but no one has put in the work to to make the kernel implementation as sophisticated as the userspace one.
Let them post in another community. I care about the server admin announcements here that I cannot get anywhere else, not so much about hotdognoodlememes which I already can't open my browser without getting smacked in the face with.
I don't know if lemmy supports sub-paths, I've never seen a lemmy hosted at one though. If it doesn't nginx definitely supports vhosts. So a single server/vm/container can definitely respond both at my.host.com and lemmy.host.com with appropriate content.
So in my case I am the only user on my instance so I am certainly not going to be hammering a bunch of instances just to send me updates of whatever total number of communities I'm subscribed to.
You don't understand how federated replication works. It doesn't occur on-demand when you read a post, it occurs when the instance hosting the community gets a post, comment, or vote. The federation load you place on other servers has nothing to do with how many users are on your instance or how much they read... it has everything to do with how many communities they subscribe to. This script is literally signing you up to proactively receive the firehose of every post and comment in the lemmyverse, without regard for what you actually look at.
I completely understand the idea of the app, and your confusion about how much load it generates is exactly why it's such an irresponsible idea. If you want to fill the timeline of your small instance, do so by subscribing to specific communities you're interested in until your timeline becomes active enough for you. Subscribing to 100 communities you care about will result in a very lively feed of stuff that is interesting to you, while generating a tiny percentage of the federation load this approach does. Carpet bombing the entire lemmyverse with subscriptions you cannot read is madness. It's like writing a reddit app that downloads everything ever posted to reddit to your phone to save you the trouble of picking subreddits to follow. It's bad for reddit, bad for your phone, bad for your isp, and a bad idea all around. If I were running a large instance, I'd defederate with any tiny instance I observed subscribing indiscriminately via this script. It's abuse.
Are you using the Jerboa app on Android? The votes in the two places are the same, but Jerboa doesn't update the feed when you vote in the post. If you refresh it will show your vote in the feed. It's a small bug.
Do you realize that you're advising OP to ignore a mod (in a reply to said mod) of this community who is being perfectly chill while informing OP about the published rule 3, which the community mods have stated they're being relaxed about while so many new Lemmings are joining, but not giving up on forever?
Rule 3 is quoted below, and helpfully directs people to active communities that are dedicated to support:
Not regarding using or support for Lemmy: context, find help in the list of support alternatives below
Have a read through https://tailscale.com/blog/how-nat-traversal-works/
You, and many commenters are pretty confused about out tailscale/Headscale work.
It's nice to see this officially hosted for lemmy.world
users. I've been curious about wefwef but there was no way I was going to enter my creds into a third-party proxy. It feels much better to do so via an instance hosted by the world admins where my Lemmy account is though.
FYIW, when you save this as a PWA via Firefox mobile, the name is just "voyager", which I assume makes it hard to distinguish from the voyager instance hosted by its devs. I don't think this can be changed as a user (unless I'm too dumb to figure it out). If the PWA app name can be changed server side, might be good to call it Voyager World or something.
Folks should not use lemmony to bootstrap their subscription count. It's not that hard to hit lemmyverse.net and just manually sub a bunch of stuff you're actually interested in, or to visit a big instance and browse their all feed unauthenticated.
But if you really want to automate community bootstrapping, lemmony is the worst of the scripts that doit because it defaults to subscribing to EVERYTHING, including all the porn, piracy, and hate communities on the most absent-admin'ed under-modded instances in the lemmyverse. Then your instance will mirror all those questionably legal communities and re-serve them to the public unauthenticated internet, creating hosting liability for you. Not to mention being a bad fediverse citizen and creating massive amounts of federation load on the instances forwarding you posts and comments from 20k communities that you don't read.
These two subscription bootstrapping scripts limit you to top subs by default... So you're more likely to be in well-modded territory and just the number of subs is smaller you you can review them and back out of anything sketchy. Subscriber-bot's docs do a good job of explaining the risks and problems of mass-subscription so you know what you're getting into.