I run my own instance, the benefit is privacy and reliability. Everything is controlled on your own server. You also aren’t reliant on someone else running an instance that could go down at any time, either permanently or an outage. Been a problem with Lemmy.ml recently.
You also aren’t reliant on someone else running an instance that could go down at any time, either permanently or an outage.
You have to worry about it yourself though.
A balancing act for sure. I’m torn on the topic. With some much excitement right now but so little history there’s a lot of uncertainty where to “plant your flag”. Part of me wants to setup my own instance simply so I maintain control of my identity should .world suddenly disappear. On the other hand now I have the responsibility of making sure I don’t make myself disappear. The mental debate will continue.
I think for now it's so early it doesn't matter all that much. Just have fun! You can make multiple accounts so why not
Yea but if you host with a major cloud server provider you're basically having 100% uptime because they very very rarely go down and so the only issues would be stuff you'd have to deal with.
I was asking myself a question, if you comment like you did here
Is it saved in the server on which the original post is, or is it saved on your server?
Kind of both. His server has a mirror of the community. When he comments it gets saved on his server and the his server communicates with the original server. In turn the original server also communicates his comment with other federated servers.
If data is migrated from server to server, as the community grows in size, the data to be maintained on each server also grows in size? Also i've seen some servers allow the creation of new users/communities, but some don't... whats the point of that if the data is just replicated anyway?
Yes that's right, an instance is constantly accumulating data over time, however instances that aren't the origin instance have the option of going back and deleting old posts (manually in the DB) but then their users wouldn't be able to see them anymore. I do get the concern though, if things really start to pickup and we get insane volume, I'm afraid even my instance wouldn't be able to pick up. I'd have to unsubscribe from everything. On the other hand, I think people will come up with solutions as things scale. There's a lot of unknowns right now, too many to build a solution. Just take a look at mastodon and how its model has changed over time.
If data is migrated from server to server, as the community grows in size, the data to be maintained on each server also grows in size? Also i've seen some servers allow the creation of new users/communities, but some don't... whats the point of that if the data is just replicated anyway?
I believe it is saved first on the instance you're signed up for, then gets pushed around the network using the Activity pub protocol. So it eventually ends up being stored across many instances of it has far enough reach.
How is your RAM/storage usage? I'm interested in setting up my own instance (no communities, just a username that will always be here) but don't want to upgrade my VPS again. I already had to do that spinning up a Mastodon server.
I’m up to about 300MB of disk usage after a day of hosting my own. Curious to see how it grows.
haha better than the 12GB and rising of my single-user Mastodon instance. And this is with deleting my media cache every night.
Mastodon is aggressive with caching media. Akkoma is more lightweight
Up to 400MB after two days here. I took a look at the code and it looks like Lemmy keeps all ActivityPub JSON for 6 months. It would be nice if it was possible to shorten that.
I'm still happy that I'm hosting my own instance, but I hope this thing doesn't get too big!
The pictures folder on my instance is at 1.3GB after two days. It's just me and my friend. About how many communities are you subscribed to?
Some of those are lemmy.ml and not a lot of comments, etc have synced yet.
My raspberry pi 4 is using 810mb of RAM and 11gb of file system space.
That's interesting to read. Could an instance be added to an existing setup? (Debian OS)
So long as you can run Docker, I would think that you could setup an instance. You just need to make sure that the image you use for lemmy and lemmy-ui are compatible with your platform. I had to alter the provided docker-compose.yml file to use arm64 versions for my RaspberryPi.
I mentioned the total disk usage for the sake of setting up a pi. I don't know what the space requirements are for lemmy separate from the bulkiness of an Ubuntu 22.04 install.
For personal use, even a Raspberry Pi is sufficient.
I think it's a matter of personal preference.
I've been running my own Mastodon instance for several months now, and I've enjoyed it. I don't have to rely on someone else, either, which is nice. I'm in control of everything on that instance.
As for Lemmy, I just started my own instance today, and am currently writing you from it. What made me decide to setup my own instance was some performance issues I was seeing with Lemmy.world, although that might have been an UI problem. Anyway, I enjoy doing this stuff, so I'm running my own instance for the sake of doing it.
On the flip side, it's more expensive and time consuming, and I'm the one who has to worry about backing up data, etc. Like I said, though, I enjoy doing it, so it's no big deal.
On the flip side, it's more expensive
Can you go into more detail on this?
Sure.
I run my own instance at a cloud provider, and thus have monthly expenses I wouldn't normally incur, if I were using a public instance.
Oh, gotcha. I guess if using a personal local server, then the only recurring cost would be electricity.
Yes, there is electricity.
I think Internet connectivity could also be an issue, unless you have an ISP that's friendly to you running a publicly accessible server on your Internet connection at home.
Could you run a local server on your desktop that you only turn on whenever as a client? I don't really understand the Fediverse's architecture yet, but as far as I saw instances being down are not a big problem beyond not being able to log in if it's your home instance, and communities fracturing to separate discussions in other instances' local federation caches that only get resynced when the thing comes back up.
What prevents me from running my own instance as a very heavy client? Discounting the public DNS + static globally routable IP part as those can be solved IMO.
I've wondered that myself, and I don't know, to be honest, but there are some issues you'd certainly encounter. For example, if you posted any media it would need to be somewhere "always on" or remote instances and users might not be able to see it unless they managed to cache it on time. It means that your posts URLs wouldn't be accessible, and would only be available on servers to which it has already federated. There may be other issues, too, such as queues only keeping undelivered messages for so long, etc.
I'm sure someone with a good understanding of ActivityPub could explain whether or not this is possible.
Yes you could, but if your instance is down often, it might be that there is some timeout in ActivityPub when your instance will be marked as gone for others and might not get new content once it is up again.
The fun part is that you can run it on the same server as your mastodon is already running with no additional costs.
Sure, one could do that, but I prefer to keep things separate.
As someone who likes having control over their data and especially backups, and someone who normally enjoys self-hosting things, I honestly might do it. I'm not sure if I'd want to host a lemmy instance or kbin instance though, since I know they all federate together anyway. I may also end up waiting until the software is more mature too before looking into it.
That's awesome! Running my own social media instances has become a hobby for me.
Having my own Lemmy instance has felt fairly seamless versus using Lemmy.world, but there have been some kinks. For example, when attempting to subscribe to a new community, the server has to pull a bunch of data first. This takes several seconds, but the UI simply says "not found" -- and then after several seconds, the UI updates with the community you want to follow. I figured this out by tailing the logs.
Also, the installation was pretty damn easy, especially when compared to Mastodon.
I'd maybe be interested in trying out self-hosting Mastodon at some point too, good to hear that Lemmy was easy to install though. I'm not too worried since I have quite a bit of Linux experience, I figure it probably won't be too bad to setup whatever social media instances I'm interested in checking out.
Yeah, if you've got a decent amount of Linux experience, I don't think you'll have any issues. Mastodon's installation is well-documented and works. My only criticism is that it's a bit long and you have to be careful not to miss anything.
On the other hand, I recall installing Pixelfed back several months ago and having a difficult time. The documentation was lacking, and it required me to use Arch Linux, which I had never used. I was able to get it working, but eventually terminated the instance after a while because I was never using it.
Oof, yeah, requiring someone to use Arch definitely seems like a steep requirement lol
LOL
It wasn't bad -- I just wasn't familiar with it.
Wait. I'm looking at the Pixelfed installation guide right now and it doesn't tell me to use Arch at all? Isn't it just a PHP app?
That's awesome. This was several months ago when there was a link to some specific Arch Linux documentation that @dansup@mastodon.social mentioned was the most complete. Sounds like it's been cleaned up.
Yeah, I think it is a PHP app.
mm. Might have been a getting started from the ground up kind of setup guide and the author might be most familiar with that. But happily it seems like there's docker files in the main repo, including docker compose files, so that should make it quite easy to set up.
Yea, but then they can say "I use arch btw". That's what really matters in life.
#goals
Personally, I went with Lemmy because it seems to be quite a bit further along in it's development. It also doesn't look like kbin's developer is recruiting much outside help, if you look at the repos of the two projects on GitHub/Codeberg, Lemmy has tons of contributions from people while kbin is mostly just the one guy with a few commits here and there. Not to mention that Lemmy's way less of a resource hog because it's written in Rust whereas kbin is implemented in PHP. Also, as far as I know federation is still currently broken on kbin.
The reason why I'd consider kbin is because I almost think I might prefer the interface, also I'm more of a PHP developer myself so that is tempting in and of itself too lol
I may wait a little bit before trying out either one tbh and maybe I'll give both of them a try.
As someone that has spent the better part of the week mucking with it.... the kbin build docs have multiple gaps in the documentation and are functionally broken unless you have some better understanding of the setup. I WAS able to get the system built, but could never get it online. Best i got was 500 errors where the UI was up but there was a break somewhere in Redis, Postgres, Nginx etc. All the logs were clean though. This was with the docker method and build from source method on both Ubuntu 22.04 and Debian 11 (which are what he specifically referenced)
Lemmy was much easier to setup using the ansible method. I have an instance online. Though im still working out the federation thing and some other kinks. I figured it would just reach out to Activity pub and federate with everyone but now it seems I have to build a static list...If if search for an instance i know exists I get a
404: couldnt_find_community
So there are some gaps but it seems much more mature. For example you cant mark your instance private AND have federation enabled. If you do that and restart the instance will fail to come up, but theres no warning or error in the UI.
I like the kbin dev better as people. But the lemmy code is definately more polished, even if the devs are turd sammiches.
Well that's interesting, and somewhat like I expected but not really, I am surprised the setup process for kbin is that broken, although I guess it does make some sense since kbin is that much newer.
Yeah. I like the kbin dev as a person much better. And I can empathize with what he is going through. Its why i wanted to setup an instance and help spread the load. But I just couldnt get it working and didnt have the time to really dig in (I have a lot going on otherwise with work).
He did announce he is onboarding a sysadmin/netadmin to handle the stability and scaling issues so he can focus on commits which is good. Ill keep an eye out for updates and maybe migrate then.
I run my own Mastodon instance, but for Lemmy it seemed more logical to join an existing instance that aligned with my interests.
I wouldn't be adverse to abandoning my self-hosted Mastodon for a shared instance, but I would prefer a small instance run by and for people I know, rather than one of the huge ones.
What might make you want to ditch your self-hosted Mastodon instance?
With Lemmy, I didn't feel a need to pick any specific instance because I can follow communities from anywhere, and it seems to work pretty well.
One downside I've encountered with my own Lemmy instance is that post and comment history in the communities I follow begins when I started following them on my new instance. New posts and comments are federated my way, going forward, but I don't have the ability to go back and view as much history as one would on lemmy.world or lemmy.ml, for example.
I have experienced the same with mastodon and pixelfed.
I didn't have this problem with Mastodon, but totally had it with Pixelfed. I don't think Pixelfed, at least at that time, supported relays. I scraped around pixelfed.social to find people to follow because I had an account there. It didn't seem possible at the time to see profiles on public servers, without having an account, so it was hard finding people. It was something I was used to do doing on Mastodon. In the end, I didn't have a positive experience running my own Pixelfed instance, and just decided to use pixelfed.social.
I do follow the developer and he's been making a lot of great progress. I've got the mobile app, and it's quite decent.
I really like Pixelfed too, the developer is very nice and very responsive.
Basically just the hastle of maintaining and hosting it. My ideal situation would be an instance with a few people, where we can share some of the burden, and perhaps cost. But maybe that has its own headaches when there is a falling out etc.
There are also other drawbacks with your own Mastodon instance in terms of discovering new people, as a lot of those tools are geared towards the server scope, and Mastodon prohibits a full index search.
I actually don't know what the Lemmy policy is on indexing, but a way to search the entire Fediverse (or at least large parts of it) would help tremendously in popularizing it, I think. I understand why indexing would be blocked, but that seems a lot like security by obscurity to me, which I don't think works very well.
Basically just the hastle of maintaining and hosting it. My ideal situation would be an instance with a few people, where we can share some of the burden, and perhaps cost. But maybe that has its own headaches when there is a falling out etc.
All very good points. I wouldn't mind sharing the costs and burden with some folks, but I'm pretty happy just maintaining it myself. Again, for me, it's something I enjoy doing.
There are also other drawbacks with your own Mastodon instance in terms of discovering new people, as a lot of those tools are geared towards the server scope, and Mastodon prohibits a full index search.
I never really had that problem, but then I started out on other Mastodon instances and just migrated my account around until I ended up on my own personal instance. I also participate in several relays, which helps a lot. In the end, I've also spent time looking at the public feeds on other servers and browsing their profiles to find people. Another thing I did was participate in conversations, which was a good way to get mutual followers.
With that being said, I don't follow a ton of people either. I read my entire timeline, chronologically, so I keep it pretty tailored. I disable boosts and mute/unfollow people often.
I actually don’t know what the Lemmy policy is on indexing, but a way to search the entire Fediverse (or at least large parts of it) would help tremendously in popularizing it, I think. I understand why indexing would be blocked, but that seems a lot like security by obscurity to me, which I don’t think works very well.
Lemmy indexes everything on the server, as far as I know, which means you should be able to find local content and content federated to the instance.
how much does your mastadon instance cost per month? what do you expect for lemmy?
I'm using Linode, and their prices are publicly available.
For Mastodon, I'm using the Linode 4 GB while the Lemmy server runs on the Linode 2 GB option. Both are under the Shared CPU pricing -- not dedicated.
thanks for all of the information you've shared in this thread.
i was put off by the cost and effort involved in maintaining a mastadon instance when i looked into that a while back, but i'm glad to hear that lemmy could be cheaper.
Can you tell me a bit about the process you went through to create your own instance? I'd like to make one myself.
You're talking about Lemmy, right?
I provisioned an Ubuntu 22.02 server at Linode. I chose their 2 GB Shared CPU instance type. Once I configured the server to my liking, I ran through the Lemmy-Ansible instructions. (They have other methods, so check the documentation.)
Essentially, you install Ansible on your workstation. I'm on macOS and installed it via Homebrew. You then download their git repository, create the necessary configuration files, and then have Ansible configure the server. It was fairly simple.
I may go that route. I was wanting to host my own server but I feel like it would be easier to just use a cloud server
I have a lab at home and do host some stuff for myself from there in a small DMZ (ie: Miniflux RSS readers, Plex through Reverse proxy etc).
But I used a linode for my lemmy/kbin stuff. Reason being is that the code is fairly new and there may be exploits bugs and
I dont want to deal with my ISP made an instance is exploited and becomes some type of C2 box or spews out spam. Kbin specifically already has PRs to fix XSS and Sql injection stuff, the former of which is usually avoidable if you just follow some pretty basic principles. So its a concern.
Linode has better bandwidth than my non-symmetrical ISP uplink and is on its own quota.
It sounds like linode is the way to go then and their prices seem reasonable. The funny thing is I've heard of linode before because Computer Clan uses it as a sponsor, but ever since I started using sponsor block I haven't really heard about it. I didn't actually know what they did.
I’ve run linodes for years. My blog runs on them. I still host a variety of other services on them. They are good for everything from gaming servers to a blog etc.
They did get bought out by akamai a while back. And have raised their prices but they are still solid.
Nanodes are awesome deals frankly.
Is the 2GB the one you use for your Lemmy server? I'm getting ready to purchase one to see if I can figure it out.
Im currently on the 4GB dedicated. However heres an htop of it.
I am currently the only user. Im considering opening it up to limited users but not really having communities once i get a lot of the instances cached and indexable.
Others like @leopardboy@netmonkey.tech are running on a 2GB shared just fine. I will likely move to that if i choose to keep it solo for sure, or under 100 users and no communities.
I dont have the time to really moderate others or content on the instance. So i dont think I plan to host any communities at all. I do wish you could federate/sync specific communities to your instance to make searching/subscribing easier.
I do wish you could federate/sync specific communities to your instance to make searching/subscribing easier.
You mean something that populates your server with a history of posts and comments to communities before your subscribe to them?
Correct. Connect to for example connect to lemmy.ml and pull their communities so it shows in your communities page locally. Dont have to sync the posts etc. Just the base stats (subs, post, comments. Basically exactly what this is doing. https://lemmyverse.net/communities
note: I hadnt seen that page until after my comment... But im getting a lot of 404's on specific communities, so i have to put in their ! name in search...spam that, click to open the community and subscribe.
That's exactly what I'm wanting to do. Just have myself and maybe a few others with limited communities. Would you recommend using ansible to setup a Lemmy on Linode?
Yes the ansible config worked fine for me. I worked for days to get an kbin instance up. Ansible worked first go.
I have yet to get email working but otherwise its solid. Linode will block email btw if you account is new (and frankly may be blocking mine now). You just have to put in a case and justify and it should be fine. My account should be old enough to be exempt but I will likely do it anyhow. Their support is pretty good.
Getting federation crawled and communities added is a bit slow. Mostly because the other instances are a bit slow.
A few pointers if you havent done admin yet.
Put nothing in the federation allow list unless you want to go whitelist only. Over time as other instances hit yours and you search others, the linked list of instances will grow. Just use the blocklist if you want to block certain instances. I havent found a good way to block the growing number of instances in case they have some illegal content like CSAM. So...i may just go whitelist anyhow
Searching for instances seems to be CPU heavy on mine. Its not a problem though. You just cant simply plug in a URL of a community in another instance if you havent linked. You will get a 404 if you do. So you have to go to search, looking for that community by hitting search a few times until it shows up, then you can join and it will start crawling
I have no idea what "Private instance" does other than i believe it will keep your instance form starting in the future if you have it checked AND federation turned on. I saw some logs in dockers startup when i did it but nothing in the UI.,
Thank you for the information. I've signed up for a 2GB shared linode and it's running Debian 11. I assume I need to ssh into it, then run the ansible stuff. I have very basic knowledge of Linux and command line. I am not entirely sure how to fill out the config but I will cross that bridge when I come to it.
It's personal preference, but I find it easier, for sure.
I would assume it would be more secure as well
Yeah, you generally wouldn't want to run a public Internet server on the same network as your personal systems.
With cloudflare tunnel it is ok
mmm. thats debateable.
If theres vulnerabilities in the software, like RCE's or SQL Injections that can lead to access...Cloudflare wont do much for you. For example Kbin has already have PRs for SQL injections and even XSS vulns.
These will get flushed out with time and more people maintaining them of course. But I dont know if I would want that on my personal network even if on a DMZ. If for no other reason than if your instance starts spamming outbound traffic and you get flagged by your ISP.
Heck I had one of my domains flagged by my works Cisco Umbrella instance and the dang thing wasnt even in prod yet.
Wow, I had no idea.
Thanks for your input.
Sure, it would keep the origin from being publicly accessible on the Internet, but you would want to put that server on its own network, anyway.
I feel that speed is the biggest benefit. I was on kbin.social and in the beginning everything was fine, but after a while when they got more and more users it was terrible. Every second click led me to cloudflare sometimes even with the capcha.
On my own instance now since yesterday everything is so fast! I chose lemmy because it's written in Rust and I have the feeling that it will be more resourceful and with less bugs than /kbin because of that ^^
Yeah, I'm running on an instance of just me and my wife, biggest downside is needing to subscribe to communities before we get content, but its sooo much faster.
I almost think this is a blessing because you don't get so overwhelmed with stuff you don't care about and only see what you're looking for. But yes the UI for it is not very good yet.
I've taken to just blocking the more popular communities I don't care about, like sports stuff
Once I subbed to a few things now it seems my all feed gets content from servers and communities I'm not subscribed too. Just took a bit. Mostly from smaller foreign ones right now tho.
Once I subbed to a few things now it seems my all feed gets content from servers and communities I'm not subscribed too. Just took a bit. Mostly from smaller foreign ones right now tho.
Deleted, I figured out you were not conplaining, but responding to another comment
From what I've seen and read, server to server traffic is less taxing on instances than client to server. So even if your instance is JUST you, it would be your instance talking to everything else so it would have some net benefit on the federation. But it would take a lot of users self-hosting solo instances for this to help in any noticeable way, I'd think.
There is certainly no downside to running a solo instance, if you're even slightly interested I would say go for it!
It's only less taxing if it's multiple ppl on an instance.
If every solo user spun up an instance just for themselves there wouldn't be a benefit over all those users just signing up directly to an existing instance.
Eg 5 users on instance b trying to access instance a
Is better then 5 users each with their own instance trying to access instance a.
Wouldn't it still be a bit less taxing even with only one user? If I'm not mistaken then your instance only initially requests a community/post/comment from another instance when you specifically search for it. After that your server gets updates through activities pushed by the other instance. So if you refresh a post multiple times those requests only go to your instance. It somewhat acts like a cache, while the other instance can push activities at it's own pace instead of being hammered with requests.
Of course multiple users per instance would still be better.
It acts like a cache, yes, but now the other instance has to send a push to keep every user's instance updated, whereas if there were only a few instances with lots of users only a few instances needed to get updated.
If every solo user spun up an instance just for themselves there wouldn’t be a benefit over all those users just signing up directly to an existing instance.
Not quite true... There's a ton of sql queries that go on to show you the correct content for your user. Showing the correct subscriptions, applying the correct "bans" etc... Further sessions management of the user logging in itself... Direct messages, inbox management... etc...
Just the raw content being sent to sync is just minimal text data that's effectively broadcasted. The stress of that is virtually nothing in comparison.
The break even is that a ton of data might sync that you never look at... but it's a broadcast of sorts. So that's a bit of a non issue.
I would doubt this. Your server is essentially another client
I did. The benefits as I see them:
I can still use Lemmy if the instance I would have used as my "home instance" ever went down.
Even if a public instance doesn't go down, all this extra load is making strange bugs surface that I don't encounter (I still have the live refresh bug everyone has, but not this one).
I have full control over my account.
If I ever want to get to customizing my UI later, I can.
Content I create originates on my instance, and I have full control over it. I can't stop other instances from caching what I post publicly, but this still gives me more data governance.
I can curate my "All" tab to only show stuff I actually want to see, instead of trying to figure out how to block communities (not sure if that's possible for regular users).
I get a custom domain which I think is pretty neat.
Regular users can block communities and instances both.
I wonder if it's possible to migrate those Reddit datasets and import them into our own Lemmy instance
I’ve wondered the same. I know I’m not the only one with the 1.8 TB Reddit data dump. It would be cool to import all that into a Lemmy instance.
Running my own instance using AWS's free tier for now, though I think I'll keep it after. It makes scaling soo easy and simple if my instance ever takes off. Which I don't know if it ever will lol. The reason why I even created one is to actually use my domain name for something rather than keep paying for a domain that I'll never use for anything.
Which of the free AWS ones did you choose. I just had a look and there are lots of different ones which is best for lemmy?
I’m using a t2.micro instance which I think is best for a small instance like mine, but I think there are other options you can look into with different RAM and CPU.
I was discussing this with other lemmings on matrix and it seems there is not much help if you dont have a community to build on your own instance. Now if you do host for yourself then you can federate with other instances to subscribe and pull from their communities which does reduce the total load on those services but that is about it.
Communities are going to Win/Loose based on personalities and critical mass, and the people hosting those communities will just have to increase their hosting needs.
Now if you do host for yourself then you can federate with other instances to subscribe and pull from their communities which does reduce the total load on those services but that is about it.
That is the main thing I'm doing, personally.
Communities are going to Win/Loose based on personalities and critical mass, and the people hosting those communities will just have to increase their hosting needs.
Speaking of hosting, I got to thinking what might happen when a community needs to move to another server. I wonder if some day we'll see a solution similar to Mastodon's where users can move their accounts and/or entire communities between Lemmy instances.
If you host your own, do you need to establish federation with all other instances or only with the ones you want to use communities from?
If I only federate with lemmy.world, would I be able to see comments on /c/selfhosted@lemmy.world on my instance made by a user from lemmy.ml?
Would a user that reads /c/selfhosted@lemmy.world on lemmy.ml see my comments, if I only federate with lemmy.world?
only the ones that host communities you subscribe to, in fact you can specifically whitelist certain instances
yes, comments and posts made from other instances are forwarded on to subscribers
yes, for the same reason
I did it. So far I've noticed a few things, for example you have to populate/federate the communities yourself, and it can take a long time. It took hours to retrieve and catch up all the lemmy.world posts. I expect it to be an ongoing thing. When you first connect to a community, it downloads the first 20 posts, but all the comments are empty.
The plus side though is it is very fast for me. And nobody can delete my profile.
Do the comments ever load reliably? For me that would be a dealbreaker...
You gotta remember, The blackout brought us refugees I don't think lemmy planned for this. I think the updates that are coming will address all of this. Reddit is decades old. Lemmy is new to all of us. We just gotta wait and eventually it will become second nature and we will be as good as Reddit
Oh totally. It wasn’t a knock at the software at all. In fact. I’m surprised by how well this works as a drop-in replacement for Reddit for me and both Lemmy and Kbin are solid.
The reason I asked was that, with my single-user Mastodon instance, likes/boosts and comments are nearly always incomplete on my server just because of the way federation works. I was just wondering if that was something smaller instances had to deal with in perpetuity or if it was just a one-off issue that happened at the start.
The OP commented below saying that comments appeared to be loading instantaneously after that initial hiccup.
Cool. Hope you get it resolved
As far as I can tell yes. Once the federation catches up the backlog, comments and posts seem to be near instantaneous.
I like running my own too, and agree it’s a bit of a chore to get content.
I feel the biggest hurdle is finding communities. I wish the instance would automatically index communities from federated instances. I don’t mean track posts/comments, just keep a record of the communities existence.
Looking them up elsewhere, then searching locally, waiting for it to work, then subscribing is a bit of work and I’ve only bothered with about 10 so far.
I'm selfhosting Lemmy and its SUPER fast. Just think of it more of a personal caching layer than anything else.
Oh yeah, your instance is fast. That's nice! What hardware are you running on?
Just the cheapest Digital Ocean instance that is on a 2 core CPU. It helps that its just me on there, so I don't have to share with anyone... yet.
I host my own instance, but I like selfhosting as a hobby. I really love having all my account/login stuff on my own hardware and my own backups. If I break something, its just my own stuff that broken, no harm done to any other users. I don't really plan on hosting any of my own communities, just participating in others. I'm an avid seflhoster, and my setup is pretty exotic compared to the vast majority of hosters, but I got the whole thing working perfectly over a day. Much better than when I tried to get mastodon working. More power to more fediverse stuff though.
I've been looking to do the same for the many pros I've seen posted here, but maybe someone can give me some clarity on a very big downside to me.
From my understanding most instances are pretty liberal with federating anyone, then blacklisting bad actots or problematic instances. However as adoption grows is there not the potential for larger instances to move towards a whitelist, and possibly move towards only federating with known, established instances or ones with established conditions? Possibly flat out banning personal instances due to moderation overhead?
Perhaps my understanding is incorrect, but seems to me that there could be a big future risk your personal server turns into an island and all of your past engagement is no longer in your control.
I'm with you. I think the comparison to email that a lotnof people have been doing works well. I can see lots of hoops to jump through to maintain federation.
The analogy works to some extent, but it is a gross oversimplifications in most regards. But yeah, keeping up with maintaining a small mail server if you expect not to continually end up in SPAM is a royal pain.
Will be interesting to see how it develops. Could see a movement towards RBL type block lists, but with the lack of tools available at the moment I think most admins are going to end up having to take some pretty drastic actions at times.
Appreciate if you guys shared some guides on setting it up. I’m not new to selfhosting but tried setting it up and failed with strange errors all day long :(
Not sure if you got it sorted or not, but if you were following the docker-compose method documented by the devs, there were a couple hurdles I ran into. The one that may be relevant here is that at some point their docker-compose.yml did not expose the Lemmy backend to the Internet, and so all federation was failing. That said, I checked just now and they seem to have fixed that issue upstream. So you should be able to re-pull their docker-compose.yml and it should work.
I will try that, thank you.
I would suggest joining us on the Lemmy matrix space, particularly the "Lemmy Instance Admin" channel. It's much easier to help in semi-real time.
No worries, this is a fairly complex piece of software. If you really want to though, try copy pasting your errors into chatgpt if you can. While not perfect, it usually can give you an idea of what steps are going wrong. Some of us can help you too, but it's no pressure. Being a part of the community is more than just the machines, it's the friends we made along the way.
I think it would be better if @marsta@beehaw.org would post the errors so the solution to his problems is documented in case someone else has a similar problem.
This way we could google problems and just add lemmy instead of reddit to find a proper solution.
Of course, the most important aspect of fixing a problem, is writing down how you fixed it. Never know what a different, but similar problem might crop up again. Sometimes getting a quick answer, or having a human readable explanation for an error goes a long way. Often enough I'll have complex logs entries that I can't parse what the problem actually is, I can quick copy paste and get a normal people paragraph. Sometimes "os: error (-4) aborting" actually just means you forgot to change user permissions on your volume folder. In a perfect world, all program's logs would be perfectly readable by machines and people at the same time. :P
I started my own instance and do currently not intend to open it for others (besides, maybe, close friends and family).
My intention are
to learn more about the concepts
evaluate how reliable the replication of comments and posts works
maybe create my own pseudo-community just for myself, as kind of a simplified blog
Reading other posts in this sub, I saw it is still seen as offloading the main servers, as the replication of the data is a low load compared to serving the UI. Maybe one of these motivations apply to you, too? Or you find another one? At the end of the day, host your own instance if you want to :-)
Is there a tutorial or something out there? I've got no background in any of this but would like to maybe give it a go myself at some point. Would you be willing to say how much it costs as well?
In my navigation of the fediverse the last few days I've noticed a few people running instances of mastodon and lemmy with just themselves or 1 or 2 other people etc.
If you into tinkering and selfhosting, why not :D Means you're in full control of what other instances you federate with/can see etc.
less thing to worries like you dont need an email to use it from single user instance, lemmy now dont have 2nd authentication like totp at the moment and may it have risk to get pawned and leak your email address so yeah it is to run your own single user instance
So if I made my own Lemmy instance, and subscribed to !selfhosted, does that mean if Lemmy.world went down, the !selfhosted community is still up?
Yesterday I saw a beehaw (I think) community thread which got locked by the beehaw mods but because it was federated ppl on other instances could still comment. I think !selfhosted would be still be up on your instance if lemmy.world went down.
I believe that any posts and comments that were pulled and stored on your server will remain but new posts or comments will no longer federate. I'm actually not sure if new posts would be possible at all but you could always test it by setting up 2 servers.
Everything is under your control, you'll never be collateral damage. Cooler username?
I run my own instance because I have the resources, I intend to create communities and it is much more private this way.
What are the privacy advantages of self hosting lemmy? as in what kind of data are we otherwise sharing when we use someone else's instance?
If I create a private community for all my family members it remains private while I selfhost. If I create it on someone else's server it ain't that private because the admin could be eavesdropping.
Same! Community building is fun, although in my case slow…
I'm in the process of upgrading my Plex server, once that's done I plan hosting my own Lemmy instance on the old server to test it out for personal use.
You absolutely don't need to be hosting any communities and can use it just for your own control over your access to the fediverse.
Okay that is pretty tempting, now I am gonna have to look into hosting my own at some point lol
For me the benefit is uniformity (not sure if thats a word) i can have a matrix account, a mastodon account, a lemmy account, all sorts of fediverse accounts all under my own domain.
This comes next to the already mentioned benefits ofcourse :)
Pros: you [sort of] own your Fediverse identity; you can make any changes to your instance you want (if you know how to do it); you’re in control of whom you peer with.
Cons: maintenance burdens (especially if you make any changes); content discovery complexity; possibly slightly less privacy (as you’re the only user of the instance, whatever is visible about it can be directly attributed to your activity). All solvable, of course.
Yes, big instances are not ready to handle the traffic and could go down - see lemmy.ml
Besides all the good reasons already given you can also get a hostname that fits you the best :)
Been trying to find out how; kinda funny googling it brings up reddit links on a protesting reddit sub.
I run my own instance, the benefit is privacy and reliability. Everything is controlled on your own server. You also aren’t reliant on someone else running an instance that could go down at any time, either permanently or an outage. Been a problem with Lemmy.ml recently.
You have to worry about it yourself though.
A balancing act for sure. I’m torn on the topic. With some much excitement right now but so little history there’s a lot of uncertainty where to “plant your flag”. Part of me wants to setup my own instance simply so I maintain control of my identity should .world suddenly disappear. On the other hand now I have the responsibility of making sure I don’t make myself disappear. The mental debate will continue.
I think for now it's so early it doesn't matter all that much. Just have fun! You can make multiple accounts so why not
Yea but if you host with a major cloud server provider you're basically having 100% uptime because they very very rarely go down and so the only issues would be stuff you'd have to deal with.
I was asking myself a question, if you comment like you did here Is it saved in the server on which the original post is, or is it saved on your server?
Kind of both. His server has a mirror of the community. When he comments it gets saved on his server and the his server communicates with the original server. In turn the original server also communicates his comment with other federated servers.
If data is migrated from server to server, as the community grows in size, the data to be maintained on each server also grows in size? Also i've seen some servers allow the creation of new users/communities, but some don't... whats the point of that if the data is just replicated anyway?
Yes that's right, an instance is constantly accumulating data over time, however instances that aren't the origin instance have the option of going back and deleting old posts (manually in the DB) but then their users wouldn't be able to see them anymore. I do get the concern though, if things really start to pickup and we get insane volume, I'm afraid even my instance wouldn't be able to pick up. I'd have to unsubscribe from everything. On the other hand, I think people will come up with solutions as things scale. There's a lot of unknowns right now, too many to build a solution. Just take a look at mastodon and how its model has changed over time.
If data is migrated from server to server, as the community grows in size, the data to be maintained on each server also grows in size? Also i've seen some servers allow the creation of new users/communities, but some don't... whats the point of that if the data is just replicated anyway?
I believe it is saved first on the instance you're signed up for, then gets pushed around the network using the Activity pub protocol. So it eventually ends up being stored across many instances of it has far enough reach.
How is your RAM/storage usage? I'm interested in setting up my own instance (no communities, just a username that will always be here) but don't want to upgrade my VPS again. I already had to do that spinning up a Mastodon server.
I’m up to about 300MB of disk usage after a day of hosting my own. Curious to see how it grows.
haha better than the 12GB and rising of my single-user Mastodon instance. And this is with deleting my media cache every night.
Mastodon is aggressive with caching media. Akkoma is more lightweight
Up to 400MB after two days here. I took a look at the code and it looks like Lemmy keeps all ActivityPub JSON for 6 months. It would be nice if it was possible to shorten that.
I'm still happy that I'm hosting my own instance, but I hope this thing doesn't get too big!
The pictures folder on my instance is at 1.3GB after two days. It's just me and my friend. About how many communities are you subscribed to?
My raspberry pi 4 is using 810mb of RAM and 11gb of file system space.
That's interesting to read. Could an instance be added to an existing setup? (Debian OS)
So long as you can run Docker, I would think that you could setup an instance. You just need to make sure that the image you use for lemmy and lemmy-ui are compatible with your platform. I had to alter the provided docker-compose.yml file to use arm64 versions for my RaspberryPi.
I mentioned the total disk usage for the sake of setting up a pi. I don't know what the space requirements are for lemmy separate from the bulkiness of an Ubuntu 22.04 install.
For personal use, even a Raspberry Pi is sufficient.
I think it's a matter of personal preference.
I've been running my own Mastodon instance for several months now, and I've enjoyed it. I don't have to rely on someone else, either, which is nice. I'm in control of everything on that instance.
As for Lemmy, I just started my own instance today, and am currently writing you from it. What made me decide to setup my own instance was some performance issues I was seeing with Lemmy.world, although that might have been an UI problem. Anyway, I enjoy doing this stuff, so I'm running my own instance for the sake of doing it.
On the flip side, it's more expensive and time consuming, and I'm the one who has to worry about backing up data, etc. Like I said, though, I enjoy doing it, so it's no big deal.
Can you go into more detail on this?
Sure.
I run my own instance at a cloud provider, and thus have monthly expenses I wouldn't normally incur, if I were using a public instance.
Oh, gotcha. I guess if using a personal local server, then the only recurring cost would be electricity.
Yes, there is electricity.
I think Internet connectivity could also be an issue, unless you have an ISP that's friendly to you running a publicly accessible server on your Internet connection at home.
Could you run a local server on your desktop that you only turn on whenever as a client? I don't really understand the Fediverse's architecture yet, but as far as I saw instances being down are not a big problem beyond not being able to log in if it's your home instance, and communities fracturing to separate discussions in other instances' local federation caches that only get resynced when the thing comes back up.
What prevents me from running my own instance as a very heavy client? Discounting the public DNS + static globally routable IP part as those can be solved IMO.
I've wondered that myself, and I don't know, to be honest, but there are some issues you'd certainly encounter. For example, if you posted any media it would need to be somewhere "always on" or remote instances and users might not be able to see it unless they managed to cache it on time. It means that your posts URLs wouldn't be accessible, and would only be available on servers to which it has already federated. There may be other issues, too, such as queues only keeping undelivered messages for so long, etc.
I'm sure someone with a good understanding of ActivityPub could explain whether or not this is possible.
Yes you could, but if your instance is down often, it might be that there is some timeout in ActivityPub when your instance will be marked as gone for others and might not get new content once it is up again.
The fun part is that you can run it on the same server as your mastodon is already running with no additional costs.
Sure, one could do that, but I prefer to keep things separate.
As someone who likes having control over their data and especially backups, and someone who normally enjoys self-hosting things, I honestly might do it. I'm not sure if I'd want to host a lemmy instance or kbin instance though, since I know they all federate together anyway. I may also end up waiting until the software is more mature too before looking into it.
That's awesome! Running my own social media instances has become a hobby for me.
Having my own Lemmy instance has felt fairly seamless versus using Lemmy.world, but there have been some kinks. For example, when attempting to subscribe to a new community, the server has to pull a bunch of data first. This takes several seconds, but the UI simply says "not found" -- and then after several seconds, the UI updates with the community you want to follow. I figured this out by tailing the logs.
Also, the installation was pretty damn easy, especially when compared to Mastodon.
I'd maybe be interested in trying out self-hosting Mastodon at some point too, good to hear that Lemmy was easy to install though. I'm not too worried since I have quite a bit of Linux experience, I figure it probably won't be too bad to setup whatever social media instances I'm interested in checking out.
Yeah, if you've got a decent amount of Linux experience, I don't think you'll have any issues. Mastodon's installation is well-documented and works. My only criticism is that it's a bit long and you have to be careful not to miss anything.
On the other hand, I recall installing Pixelfed back several months ago and having a difficult time. The documentation was lacking, and it required me to use Arch Linux, which I had never used. I was able to get it working, but eventually terminated the instance after a while because I was never using it.
Oof, yeah, requiring someone to use Arch definitely seems like a steep requirement lol
LOL
It wasn't bad -- I just wasn't familiar with it.
Wait. I'm looking at the Pixelfed installation guide right now and it doesn't tell me to use Arch at all? Isn't it just a PHP app?
That's awesome. This was several months ago when there was a link to some specific Arch Linux documentation that @dansup@mastodon.social mentioned was the most complete. Sounds like it's been cleaned up.
Yeah, I think it is a PHP app.
mm. Might have been a getting started from the ground up kind of setup guide and the author might be most familiar with that. But happily it seems like there's docker files in the main repo, including docker compose files, so that should make it quite easy to set up.
Yea, but then they can say "I use arch btw". That's what really matters in life.
#goals
Personally, I went with Lemmy because it seems to be quite a bit further along in it's development. It also doesn't look like kbin's developer is recruiting much outside help, if you look at the repos of the two projects on GitHub/Codeberg, Lemmy has tons of contributions from people while kbin is mostly just the one guy with a few commits here and there. Not to mention that Lemmy's way less of a resource hog because it's written in Rust whereas kbin is implemented in PHP. Also, as far as I know federation is still currently broken on kbin.
The reason why I'd consider kbin is because I almost think I might prefer the interface, also I'm more of a PHP developer myself so that is tempting in and of itself too lol
I may wait a little bit before trying out either one tbh and maybe I'll give both of them a try.
As someone that has spent the better part of the week mucking with it.... the kbin build docs have multiple gaps in the documentation and are functionally broken unless you have some better understanding of the setup. I WAS able to get the system built, but could never get it online. Best i got was 500 errors where the UI was up but there was a break somewhere in Redis, Postgres, Nginx etc. All the logs were clean though. This was with the docker method and build from source method on both Ubuntu 22.04 and Debian 11 (which are what he specifically referenced)
Lemmy was much easier to setup using the ansible method. I have an instance online. Though im still working out the federation thing and some other kinks. I figured it would just reach out to Activity pub and federate with everyone but now it seems I have to build a static list...If if search for an instance i know exists I get a
So there are some gaps but it seems much more mature. For example you cant mark your instance private AND have federation enabled. If you do that and restart the instance will fail to come up, but theres no warning or error in the UI.
I like the kbin dev better as people. But the lemmy code is definately more polished, even if the devs are turd sammiches.
Well that's interesting, and somewhat like I expected but not really, I am surprised the setup process for kbin is that broken, although I guess it does make some sense since kbin is that much newer.
Yeah. I like the kbin dev as a person much better. And I can empathize with what he is going through. Its why i wanted to setup an instance and help spread the load. But I just couldnt get it working and didnt have the time to really dig in (I have a lot going on otherwise with work).
He did announce he is onboarding a sysadmin/netadmin to handle the stability and scaling issues so he can focus on commits which is good. Ill keep an eye out for updates and maybe migrate then.
I run my own Mastodon instance, but for Lemmy it seemed more logical to join an existing instance that aligned with my interests. I wouldn't be adverse to abandoning my self-hosted Mastodon for a shared instance, but I would prefer a small instance run by and for people I know, rather than one of the huge ones.
What might make you want to ditch your self-hosted Mastodon instance?
With Lemmy, I didn't feel a need to pick any specific instance because I can follow communities from anywhere, and it seems to work pretty well.
One downside I've encountered with my own Lemmy instance is that post and comment history in the communities I follow begins when I started following them on my new instance. New posts and comments are federated my way, going forward, but I don't have the ability to go back and view as much history as one would on lemmy.world or lemmy.ml, for example.
I have experienced the same with mastodon and pixelfed.
I didn't have this problem with Mastodon, but totally had it with Pixelfed. I don't think Pixelfed, at least at that time, supported relays. I scraped around pixelfed.social to find people to follow because I had an account there. It didn't seem possible at the time to see profiles on public servers, without having an account, so it was hard finding people. It was something I was used to do doing on Mastodon. In the end, I didn't have a positive experience running my own Pixelfed instance, and just decided to use pixelfed.social.
I do follow the developer and he's been making a lot of great progress. I've got the mobile app, and it's quite decent.
I really like Pixelfed too, the developer is very nice and very responsive.
Basically just the hastle of maintaining and hosting it. My ideal situation would be an instance with a few people, where we can share some of the burden, and perhaps cost. But maybe that has its own headaches when there is a falling out etc.
There are also other drawbacks with your own Mastodon instance in terms of discovering new people, as a lot of those tools are geared towards the server scope, and Mastodon prohibits a full index search.
I actually don't know what the Lemmy policy is on indexing, but a way to search the entire Fediverse (or at least large parts of it) would help tremendously in popularizing it, I think. I understand why indexing would be blocked, but that seems a lot like security by obscurity to me, which I don't think works very well.
All very good points. I wouldn't mind sharing the costs and burden with some folks, but I'm pretty happy just maintaining it myself. Again, for me, it's something I enjoy doing.
I never really had that problem, but then I started out on other Mastodon instances and just migrated my account around until I ended up on my own personal instance. I also participate in several relays, which helps a lot. In the end, I've also spent time looking at the public feeds on other servers and browsing their profiles to find people. Another thing I did was participate in conversations, which was a good way to get mutual followers.
With that being said, I don't follow a ton of people either. I read my entire timeline, chronologically, so I keep it pretty tailored. I disable boosts and mute/unfollow people often.
Lemmy indexes everything on the server, as far as I know, which means you should be able to find local content and content federated to the instance.
how much does your mastadon instance cost per month? what do you expect for lemmy?
I'm using Linode, and their prices are publicly available.
https://www.linode.com/pricing/
For Mastodon, I'm using the Linode 4 GB while the Lemmy server runs on the Linode 2 GB option. Both are under the Shared CPU pricing -- not dedicated.
thanks for all of the information you've shared in this thread.
i was put off by the cost and effort involved in maintaining a mastadon instance when i looked into that a while back, but i'm glad to hear that lemmy could be cheaper.
Can you tell me a bit about the process you went through to create your own instance? I'd like to make one myself.
You're talking about Lemmy, right?
I provisioned an Ubuntu 22.02 server at Linode. I chose their 2 GB Shared CPU instance type. Once I configured the server to my liking, I ran through the Lemmy-Ansible instructions. (They have other methods, so check the documentation.)
Essentially, you install Ansible on your workstation. I'm on macOS and installed it via Homebrew. You then download their git repository, create the necessary configuration files, and then have Ansible configure the server. It was fairly simple.
I may go that route. I was wanting to host my own server but I feel like it would be easier to just use a cloud server
I have a lab at home and do host some stuff for myself from there in a small DMZ (ie: Miniflux RSS readers, Plex through Reverse proxy etc).
But I used a linode for my lemmy/kbin stuff. Reason being is that the code is fairly new and there may be exploits bugs and
I dont want to deal with my ISP made an instance is exploited and becomes some type of C2 box or spews out spam. Kbin specifically already has PRs to fix XSS and Sql injection stuff, the former of which is usually avoidable if you just follow some pretty basic principles. So its a concern.
Linode has better bandwidth than my non-symmetrical ISP uplink and is on its own quota.
It sounds like linode is the way to go then and their prices seem reasonable. The funny thing is I've heard of linode before because Computer Clan uses it as a sponsor, but ever since I started using sponsor block I haven't really heard about it. I didn't actually know what they did.
I’ve run linodes for years. My blog runs on them. I still host a variety of other services on them. They are good for everything from gaming servers to a blog etc.
They did get bought out by akamai a while back. And have raised their prices but they are still solid.
Nanodes are awesome deals frankly.
Is the 2GB the one you use for your Lemmy server? I'm getting ready to purchase one to see if I can figure it out.
Im currently on the 4GB dedicated. However heres an htop of it.
https://imgur.com/a/NpEsw4t
I am currently the only user. Im considering opening it up to limited users but not really having communities once i get a lot of the instances cached and indexable.
Others like @leopardboy@netmonkey.tech are running on a 2GB shared just fine. I will likely move to that if i choose to keep it solo for sure, or under 100 users and no communities.
I dont have the time to really moderate others or content on the instance. So i dont think I plan to host any communities at all. I do wish you could federate/sync specific communities to your instance to make searching/subscribing easier.
You mean something that populates your server with a history of posts and comments to communities before your subscribe to them?
Correct. Connect to for example connect to lemmy.ml and pull their communities so it shows in your communities page locally. Dont have to sync the posts etc. Just the base stats (subs, post, comments. Basically exactly what this is doing. https://lemmyverse.net/communities
note: I hadnt seen that page until after my comment... But im getting a lot of 404's on specific communities, so i have to put in their ! name in search...spam that, click to open the community and subscribe.
That's exactly what I'm wanting to do. Just have myself and maybe a few others with limited communities. Would you recommend using ansible to setup a Lemmy on Linode?
Yes the ansible config worked fine for me. I worked for days to get an kbin instance up. Ansible worked first go.
I have yet to get email working but otherwise its solid. Linode will block email btw if you account is new (and frankly may be blocking mine now). You just have to put in a case and justify and it should be fine. My account should be old enough to be exempt but I will likely do it anyhow. Their support is pretty good.
Getting federation crawled and communities added is a bit slow. Mostly because the other instances are a bit slow.
A few pointers if you havent done admin yet.
Put nothing in the federation allow list unless you want to go whitelist only. Over time as other instances hit yours and you search others, the linked list of instances will grow. Just use the blocklist if you want to block certain instances. I havent found a good way to block the growing number of instances in case they have some illegal content like CSAM. So...i may just go whitelist anyhow
Searching for instances seems to be CPU heavy on mine. Its not a problem though. You just cant simply plug in a URL of a community in another instance if you havent linked. You will get a 404 if you do. So you have to go to search, looking for that community by hitting search a few times until it shows up, then you can join and it will start crawling
I have no idea what "Private instance" does other than i believe it will keep your instance form starting in the future if you have it checked AND federation turned on. I saw some logs in dockers startup when i did it but nothing in the UI.,
Thank you for the information. I've signed up for a 2GB shared linode and it's running Debian 11. I assume I need to ssh into it, then run the ansible stuff. I have very basic knowledge of Linux and command line. I am not entirely sure how to fill out the config but I will cross that bridge when I come to it.
It's personal preference, but I find it easier, for sure.
I would assume it would be more secure as well
Yeah, you generally wouldn't want to run a public Internet server on the same network as your personal systems.
With cloudflare tunnel it is ok
mmm. thats debateable.
If theres vulnerabilities in the software, like RCE's or SQL Injections that can lead to access...Cloudflare wont do much for you. For example Kbin has already have PRs for SQL injections and even XSS vulns.
These will get flushed out with time and more people maintaining them of course. But I dont know if I would want that on my personal network even if on a DMZ. If for no other reason than if your instance starts spamming outbound traffic and you get flagged by your ISP.
Heck I had one of my domains flagged by my works Cisco Umbrella instance and the dang thing wasnt even in prod yet.
Wow, I had no idea.
Thanks for your input.
Sure, it would keep the origin from being publicly accessible on the Internet, but you would want to put that server on its own network, anyway.
I already have a nomad cluster in my homelab, running Lemmy is a no brainer.
I feel that speed is the biggest benefit. I was on kbin.social and in the beginning everything was fine, but after a while when they got more and more users it was terrible. Every second click led me to cloudflare sometimes even with the capcha.
On my own instance now since yesterday everything is so fast! I chose lemmy because it's written in Rust and I have the feeling that it will be more resourceful and with less bugs than /kbin because of that ^^
Yeah, I'm running on an instance of just me and my wife, biggest downside is needing to subscribe to communities before we get content, but its sooo much faster.
I almost think this is a blessing because you don't get so overwhelmed with stuff you don't care about and only see what you're looking for. But yes the UI for it is not very good yet.
I've taken to just blocking the more popular communities I don't care about, like sports stuff
Once I subbed to a few things now it seems my all feed gets content from servers and communities I'm not subscribed too. Just took a bit. Mostly from smaller foreign ones right now tho.
Once I subbed to a few things now it seems my all feed gets content from servers and communities I'm not subscribed too. Just took a bit. Mostly from smaller foreign ones right now tho.
Deleted, I figured out you were not conplaining, but responding to another comment
From what I've seen and read, server to server traffic is less taxing on instances than client to server. So even if your instance is JUST you, it would be your instance talking to everything else so it would have some net benefit on the federation. But it would take a lot of users self-hosting solo instances for this to help in any noticeable way, I'd think.
There is certainly no downside to running a solo instance, if you're even slightly interested I would say go for it!
It's only less taxing if it's multiple ppl on an instance.
If every solo user spun up an instance just for themselves there wouldn't be a benefit over all those users just signing up directly to an existing instance.
Eg 5 users on instance b trying to access instance a Is better then 5 users each with their own instance trying to access instance a.
Wouldn't it still be a bit less taxing even with only one user? If I'm not mistaken then your instance only initially requests a community/post/comment from another instance when you specifically search for it. After that your server gets updates through activities pushed by the other instance. So if you refresh a post multiple times those requests only go to your instance. It somewhat acts like a cache, while the other instance can push activities at it's own pace instead of being hammered with requests. Of course multiple users per instance would still be better.
It acts like a cache, yes, but now the other instance has to send a push to keep every user's instance updated, whereas if there were only a few instances with lots of users only a few instances needed to get updated.
Not quite true... There's a ton of sql queries that go on to show you the correct content for your user. Showing the correct subscriptions, applying the correct "bans" etc... Further sessions management of the user logging in itself... Direct messages, inbox management... etc...
Just the raw content being sent to sync is just minimal text data that's effectively broadcasted. The stress of that is virtually nothing in comparison.
The break even is that a ton of data might sync that you never look at... but it's a broadcast of sorts. So that's a bit of a non issue.
I would doubt this. Your server is essentially another client
I did. The benefits as I see them:
Regular users can block communities and instances both.
I wonder if it's possible to migrate those Reddit datasets and import them into our own Lemmy instance
I’ve wondered the same. I know I’m not the only one with the 1.8 TB Reddit data dump. It would be cool to import all that into a Lemmy instance.
Running my own instance using AWS's free tier for now, though I think I'll keep it after. It makes scaling soo easy and simple if my instance ever takes off. Which I don't know if it ever will lol. The reason why I even created one is to actually use my domain name for something rather than keep paying for a domain that I'll never use for anything.
Which of the free AWS ones did you choose. I just had a look and there are lots of different ones which is best for lemmy?
I’m using a t2.micro instance which I think is best for a small instance like mine, but I think there are other options you can look into with different RAM and CPU.
I was discussing this with other lemmings on matrix and it seems there is not much help if you dont have a community to build on your own instance. Now if you do host for yourself then you can federate with other instances to subscribe and pull from their communities which does reduce the total load on those services but that is about it.
Communities are going to Win/Loose based on personalities and critical mass, and the people hosting those communities will just have to increase their hosting needs.
That is the main thing I'm doing, personally.
Speaking of hosting, I got to thinking what might happen when a community needs to move to another server. I wonder if some day we'll see a solution similar to Mastodon's where users can move their accounts and/or entire communities between Lemmy instances.
If you host your own, do you need to establish federation with all other instances or only with the ones you want to use communities from?
If I only federate with lemmy.world, would I be able to see comments on /c/selfhosted@lemmy.world on my instance made by a user from lemmy.ml?
Would a user that reads /c/selfhosted@lemmy.world on lemmy.ml see my comments, if I only federate with lemmy.world?
I did it. So far I've noticed a few things, for example you have to populate/federate the communities yourself, and it can take a long time. It took hours to retrieve and catch up all the lemmy.world posts. I expect it to be an ongoing thing. When you first connect to a community, it downloads the first 20 posts, but all the comments are empty.
The plus side though is it is very fast for me. And nobody can delete my profile.
Do the comments ever load reliably? For me that would be a dealbreaker...
You gotta remember, The blackout brought us refugees I don't think lemmy planned for this. I think the updates that are coming will address all of this. Reddit is decades old. Lemmy is new to all of us. We just gotta wait and eventually it will become second nature and we will be as good as Reddit
Oh totally. It wasn’t a knock at the software at all. In fact. I’m surprised by how well this works as a drop-in replacement for Reddit for me and both Lemmy and Kbin are solid.
The reason I asked was that, with my single-user Mastodon instance, likes/boosts and comments are nearly always incomplete on my server just because of the way federation works. I was just wondering if that was something smaller instances had to deal with in perpetuity or if it was just a one-off issue that happened at the start.
The OP commented below saying that comments appeared to be loading instantaneously after that initial hiccup.
Cool. Hope you get it resolved
As far as I can tell yes. Once the federation catches up the backlog, comments and posts seem to be near instantaneous.
I like running my own too, and agree it’s a bit of a chore to get content.
I feel the biggest hurdle is finding communities. I wish the instance would automatically index communities from federated instances. I don’t mean track posts/comments, just keep a record of the communities existence.
Looking them up elsewhere, then searching locally, waiting for it to work, then subscribing is a bit of work and I’ve only bothered with about 10 so far.
I'm selfhosting Lemmy and its SUPER fast. Just think of it more of a personal caching layer than anything else.
Oh yeah, your instance is fast. That's nice! What hardware are you running on?
Just the cheapest Digital Ocean instance that is on a 2 core CPU. It helps that its just me on there, so I don't have to share with anyone... yet.
I host my own instance, but I like selfhosting as a hobby. I really love having all my account/login stuff on my own hardware and my own backups. If I break something, its just my own stuff that broken, no harm done to any other users. I don't really plan on hosting any of my own communities, just participating in others. I'm an avid seflhoster, and my setup is pretty exotic compared to the vast majority of hosters, but I got the whole thing working perfectly over a day. Much better than when I tried to get mastodon working. More power to more fediverse stuff though.
I've been looking to do the same for the many pros I've seen posted here, but maybe someone can give me some clarity on a very big downside to me.
From my understanding most instances are pretty liberal with federating anyone, then blacklisting bad actots or problematic instances. However as adoption grows is there not the potential for larger instances to move towards a whitelist, and possibly move towards only federating with known, established instances or ones with established conditions? Possibly flat out banning personal instances due to moderation overhead?
Perhaps my understanding is incorrect, but seems to me that there could be a big future risk your personal server turns into an island and all of your past engagement is no longer in your control.
I'm with you. I think the comparison to email that a lotnof people have been doing works well. I can see lots of hoops to jump through to maintain federation.
The analogy works to some extent, but it is a gross oversimplifications in most regards. But yeah, keeping up with maintaining a small mail server if you expect not to continually end up in SPAM is a royal pain.
Will be interesting to see how it develops. Could see a movement towards RBL type block lists, but with the lack of tools available at the moment I think most admins are going to end up having to take some pretty drastic actions at times.
Appreciate if you guys shared some guides on setting it up. I’m not new to selfhosting but tried setting it up and failed with strange errors all day long :(
Not sure if you got it sorted or not, but if you were following the docker-compose method documented by the devs, there were a couple hurdles I ran into. The one that may be relevant here is that at some point their
docker-compose.ymldid not expose the Lemmy backend to the Internet, and so all federation was failing. That said, I checked just now and they seem to have fixed that issue upstream. So you should be able to re-pull theirdocker-compose.ymland it should work.I will try that, thank you.
I would suggest joining us on the Lemmy matrix space, particularly the "Lemmy Instance Admin" channel. It's much easier to help in semi-real time.
No worries, this is a fairly complex piece of software. If you really want to though, try copy pasting your errors into chatgpt if you can. While not perfect, it usually can give you an idea of what steps are going wrong. Some of us can help you too, but it's no pressure. Being a part of the community is more than just the machines, it's the friends we made along the way.
I think it would be better if @marsta@beehaw.org would post the errors so the solution to his problems is documented in case someone else has a similar problem.
This way we could google problems and just add lemmy instead of reddit to find a proper solution.
Of course, the most important aspect of fixing a problem, is writing down how you fixed it. Never know what a different, but similar problem might crop up again. Sometimes getting a quick answer, or having a human readable explanation for an error goes a long way. Often enough I'll have complex logs entries that I can't parse what the problem actually is, I can quick copy paste and get a normal people paragraph. Sometimes "os: error (-4) aborting" actually just means you forgot to change user permissions on your volume folder. In a perfect world, all program's logs would be perfectly readable by machines and people at the same time. :P
It was mainly around this error: https://github.com/LemmyNet/lemmy/issues/2278
I started my own instance and do currently not intend to open it for others (besides, maybe, close friends and family).
My intention are
Reading other posts in this sub, I saw it is still seen as offloading the main servers, as the replication of the data is a low load compared to serving the UI. Maybe one of these motivations apply to you, too? Or you find another one? At the end of the day, host your own instance if you want to :-)
Is there a tutorial or something out there? I've got no background in any of this but would like to maybe give it a go myself at some point. Would you be willing to say how much it costs as well?
In my navigation of the fediverse the last few days I've noticed a few people running instances of mastodon and lemmy with just themselves or 1 or 2 other people etc.
If you into tinkering and selfhosting, why not :D Means you're in full control of what other instances you federate with/can see etc.
less thing to worries like you dont need an email to use it from single user instance, lemmy now dont have 2nd authentication like totp at the moment and may it have risk to get pawned and leak your email address so yeah it is to run your own single user instance
So if I made my own Lemmy instance, and subscribed to !selfhosted, does that mean if Lemmy.world went down, the !selfhosted community is still up?
Yesterday I saw a beehaw (I think) community thread which got locked by the beehaw mods but because it was federated ppl on other instances could still comment. I think !selfhosted would be still be up on your instance if lemmy.world went down.
I believe that any posts and comments that were pulled and stored on your server will remain but new posts or comments will no longer federate. I'm actually not sure if new posts would be possible at all but you could always test it by setting up 2 servers.
Everything is under your control, you'll never be collateral damage. Cooler username?
I run my own instance because I have the resources, I intend to create communities and it is much more private this way.
What are the privacy advantages of self hosting lemmy? as in what kind of data are we otherwise sharing when we use someone else's instance?
If I create a private community for all my family members it remains private while I selfhost. If I create it on someone else's server it ain't that private because the admin could be eavesdropping.
Same! Community building is fun, although in my case slow…
I'm in the process of upgrading my Plex server, once that's done I plan hosting my own Lemmy instance on the old server to test it out for personal use.
You absolutely don't need to be hosting any communities and can use it just for your own control over your access to the fediverse.
Okay that is pretty tempting, now I am gonna have to look into hosting my own at some point lol
For me the benefit is uniformity (not sure if thats a word) i can have a matrix account, a mastodon account, a lemmy account, all sorts of fediverse accounts all under my own domain.
This comes next to the already mentioned benefits ofcourse :)
Pros: you [sort of] own your Fediverse identity; you can make any changes to your instance you want (if you know how to do it); you’re in control of whom you peer with.
Cons: maintenance burdens (especially if you make any changes); content discovery complexity; possibly slightly less privacy (as you’re the only user of the instance, whatever is visible about it can be directly attributed to your activity). All solvable, of course.
Yes, big instances are not ready to handle the traffic and could go down - see lemmy.ml
Besides all the good reasons already given you can also get a hostname that fits you the best :)
Been trying to find out how; kinda funny googling it brings up reddit links on a protesting reddit sub.