wpuckering

You shouldn't be charged for unauthorized requests to your buckets. Currently if you know any person's bucket name, which is easily discoverable if you know what you're doing, that means you can maliciously rack up their bill just to hurt them financially by spamming it with anonymous requests.

I just stood up a selfhosted Invidious instance the other day, and I replaced YouTube ReVanced with Clipious (an Invidious client for Android) on my phone. No ads, SponsorBlock built-in, no need for a YouTube/Google account to create subscriptions, playlists, etc. And it's highly performant since I run it behind a reverse proxy with some custom caching configuration for things like thumbnail images, static assets, etc.

Clipious can also be installed on an Android TV (has an actual Android TV interface). I'm going to end up installing it on mine, but I'm also using SmartTubeNext at the moment, which does require a YouTube/Google account for subscriptions, playlists, etc, but does have no ads, built-in SponsorBlock, and a slew of other great features. I'll be keeping both around, since I do sometimes like to cast to my TV, and SmartTubeNext allows for that (Clipious does not, at least at this time).

Unless YouTube somehow starts dynamically splicing in ads as part of the actual video stream, there's always going to be a way to block ads, unless they do something pretty elaborate. But that's probably not worth the effort on their end to go that far, since the vast, vast majority of people won't know what to do to get around that, nor will they probably care enough to try. But I think it's clear that DNS blocking using services such as AdGuard Home, PiHole, etc, are going to become less effective over time.

Atheist here. Extraordinary claims require extraordinary evidence. Atheism is merely about trusting what's been proven, or has some evidence backing the claim that can be verified without doubt. Being agnostic is being indecisive about everything, even things that are completely made up.

I'll update to a newer Postgres version and report back. It would be nice to know what the minimum supported version is, maybe that should be added to the documentation.

EDIT: Upgrading to Postgres 15 resolved my issue, but not without some pain since the migration scripts had already tried to run on my Postgres 13 database. So after dumping the 13 database, I had to make some modifications after the fact to satisfy the migration scripts. It was a pretty janky process but I seem to be in a good place now.

I would highly advise communicating to people that they should upgrade past Postgres 13 before trying to upgrade Lemmy to 0.18.3 or higher, or you're gonna have a bad time.

Same here, I've never had this problem, ever. I don't even get how it's possible to not know where your files are being saved if you are the least bit techsavvy.

It's awesome to see Lemmy getting lots of love, and choice in the mobile app space is great for everyone. But some part of me also kind of wishes that rather than spreading so much development effort out over so many mobile apps, that more developers would jump in and contribute to polishing up the official open source Lemmy mobile app, Jerboa. I can't help but feel that it would be nice to see a focused effort somewhere in bringing that one in particular up to snuff, as a sort of "reference" app. And have a few others floating around out there just for some diversity and testing innovative ideas.

Maybe it's already that way, I don't know. It kind of feels like there's a new Lemmy mobile app announced every couple of days.

How does this work? At what URL will this be hosted at?

Morons!

Ran into this issue with database migrations:

Failed to run 2023-07-08-101154_fix_soft_delete_aggregates with: syntax error at or near "trigger"', crates/db_schema/src/utils.rs:221:25

Will open an issue on GitHub.

There's nothing stopping instance owners from incorporating their own security measures into their infrastructure as they see fit, such as a reverse proxy with a modern web application firewall, solutions such as Cloudflare and the free captcha capabilities they offer, or a combination of those and/or various other protective measures. If you're hosting your own Lemmy instance and exposing it to the public, and you don't understand what would be involved in the above examples or have no idea where to start, then you probably shouldn't be hosting a public Lemmy instance in the first place.

It's generally not a good idea to rely primarily on security to be baked into application code and call it a day. I'm not up to date on this news and all of the nuances yet, I'll look into it after I've posted this, but what I said above holds true regardless.

The responsibility of security of any publicly hosted web application or service rests squarely on the owner of the instance. It's up to you to secure your infrastructure, and there are very good and accepted best practice ways of doing that outside of application code. Something like losing baked in captcha in a web application should come as no big deal to those who have the appropriate level of knowledge to responsibly host their instance.

From what this seems to be about, it seems like a non-issue, unless you're someone who is relying on baked in security to cover for your lack of expertise in properly securing your instance and mitigating exploitation by bots yourself.

I'm not trying to demean anyone or sound holier than thou, but honestly, please don't rely on the devs for all of your security needs. There are ways to keep your instance secure that doesn't require their involvement, and that are best practice anyways. Please seek to educate yourself if this applies to you, and shore up the security of your own instances by way of the surrounding infrastructure.

I've never seen this before, someone here said it's a Lemmy UI loading indicator? I guess my selfhosted instance is working really well then!

The ending of Soma.

With all due respect, it seems like a janky solution to have a bot post public comments on request with transformed links specific to a given user's own instance (that no other users would be likely to care about), just so that they can refresh the page and click on them... If something like this went into widespread use, threads would just become cluttered with comments containing transformed links, and I could see that being really annoying to other users who are trying to properly participate in discussion.

Back on Reddit, I always thought the !remindme bot was pretty dumb. Certain threads would just be spammed with comments for the bot to pick up to remind that specific user on some date to come back and check the thread. We can do better than that here. It was a janky solution to something that was a problem best left to the end-user to manage separately (just set a reminder in your own calendar...).

This is best left to client-side code in the form of a browser addon, or ideally, the Lemmy frontend itself.

It should be trivial to make an enhancement to the official Lemmy frontend such that links to any other Lemmy communities/posts/comments/etc are transformed to the context of the user's home instance. It could be a togglable setting in the user's own settings, or maybe both the original link and the transformed link could be presented to the user on click (to accommodate both desktop and mobile browsing).

I'm actually really surprised this isn't already implemented given how simple it is to do.

I use Clipious, an Android client for Invidious, on my phone. I selfhost my own Indivious instance so this is perfect in that my phone never connects to YouTube directly, and I can save all my subscriptions in one place without a YouTube account.

On my Android TV I use Smart Tube Next. If I really need to cast, I also have YouTube ReVanced on my phone for just that, but I barely use it.

As soon as Clipious gets a proper Android TV interface, I'll be set, as both devices can just connect to Invidious and let it do all the work.

I've struggled with trying to find an alternative to Google Photos that actually works well enough, and reliably enough, for me to feel comfortable enough to fully replace it. I've tried everything on the Awesome Selfhosted list that would be a potential competitor, but nothing comes close to Google Photos. It's honestly just such a solid product it's really hard to find an open source/selfhosted replacement that works at least as well. And Google Photos is just so convenient when it comes to shared albums, it's just slick.

My ideal solution would be to have Google Photos remain the source of truth, but have something else as a secondary backup. I looked into the idea of using Rclone to mount Google Photos and another backend (ie. Wasabi), and just replicating periodically from Google Photos to another location. But unfortunately at this time (and maybe forever), the Google Photos API doesn't allow you to access photos/videos in their original form, only compressed. But I want the originals of course, so this doesn't fly. The next thing I'll be looking into when I have more time is automating Google Takeout periodically to fetch the original quality photos/videos, then upload to a backup location. But it's such a janky idea and it rubs me the wrong way... But it might be the only way. Rclone would have been so perfect if only it could get the original quality content, but that's on Google not enabling that capability.

The Linux distro concept is a great analogy.

I know it's been around for a long time, but I just heard about Real Debrid. My current setup is Wasabi + Rclone + Jellyfin, plus all the *arr services. What's the benefit of Real Debrid over this setup, aside from cached torrents?

Wasabi is great and cheap for S3-compatible object storage.

I have a single ASUS Chromebox M075U I3-4010U which I use as a Docker host. It's neatly and inconspicously tucked away under my TV, and it's quiet even when the fan's on full if a heavy workload is running.

Main Specs:

• Processor: Intel Core i3-4010U 1.7 GHz
• Memory: 4 GB DDR3 1600 (I upgraded this to 16 GB)
• Storage: 16 GB SSD (I upgraded this to 64 GB)
• Graphics: Intel HD Graphics 4400
• OS: Google Chrome OS (Currently running Ubuntu 22.04)

Full Specs: https://www.newegg.ca/asus-chromebox-m075u-nettop-computer/p/N82E16883220591R

I started off with a single-node Kubernetes cluster (k3s) a few years ago for learning purposes, and ran with it for quite a long time, but have since gone back to Docker Compose for a few reasons:

• Less overhead and more light-weight
• Quicker and easier to maintain now that I have a young family and less time
• Easier to share examples of how to run certain stacks with people that don't have Kubernetes experience

For logs, I'm only concerned with container logs, so I use Dozzle for a quick view of what's going on. I'm not concerned with keeping historical logs, I only care about real-time logs, since if there's an ongoing issue I can troubleshoot it then and there and that's all I need. This also means I dont need to store anything in terms of logs, or run a heavier log ingestion stack such as ELK, Graylog, or anything like that. Dozzle is nice and light and gives me everything I need.

When it comes to container updates, I just do it whenever I feel like, manually. It's generally frowned upon to reference the latest tag for a container image to get the latest updates automatically for risk of random breaking changes. And I personally feel this holds true for other methods such as Watchtower for automated container updates. I like to keep my containers running a specific version of an image until I feel it's time to see what's new and try an update. I can then safely backup the persistent data, see if all goes well, and if not, do a quick rollback with minimal effort.

I used to think monitoring tools were cool, fun, neat to show off (fancy graphs, right?), but I've since let go of that idea. I don't have any monitoring setup besides Dozzle for logs (and now it shows you some extra info such as memory and CPU usage, which is nice). In the past I've had Grafana, Prometheus, and some other tooling for monitoring but I never ended up looking at any of it once it was up and "done" (this stuff is never really "done", you know?). So I just felt it was all a waste of resources that could be better spent actually serving a better purpose. At the end of the day, if I'm using my services and not having any trouble with anything, then it's fine, I don't care about seeing some fancy graphs or metrics showing me what's going on behind the curtain, because my needs are being served, which is the point right?

I do use Gotify for notifications, if you want to count that as monitoring, but that's pretty much it.

I'm pretty proud of the fact that I've got such a cheap, low-powered little server compared to what most people who selfhost likely have to work with, and that I'm able to run so many services on it without any performance issues that I myself can notice. Everything just works, and works very well. I can probably even add a bunch more services before I start seeing performance issues.

At the moment I run about 50 containers across my stacks, supporting:

• AdGuard Home
• AriaNG
• Bazarr
• Certbot
• Cloudflared
• Cloudflare DDNS
• Dataloader (custom service I wrote for ingesting data from a bunch of sources)
• Dozzle
• FileFlows
• FileRun
• Gitea
• go-socks5-proxy
• Gotify
• Homepage
• Invidious
• Jackett
• Jellyfin
• Lemmy
• Lidarr
• Navidrome
• Nginx
• Planka
• qBittorrent
• Radarr
• Rclone
• Reactive-Resume
• Readarr
• Shadowsocks Server (Rust)
• slskd
• Snippet-Box
• Sonarr
• Teedy
• Vaultwarden
• Zola

If you know what you're doing and have enough knowledge in a variety of areas, you can make a lot of use of even the most basic/barebones hardware and squeeze a lot out of it. Keeping things simple, tidy, and making effective use of DNS, caching, etc, can go a long way. Experience in Full Stack Development, Infrastructure, and DevOps practices over the years really helped me in terms of knowing how to squeeze every last bit of performance out of this thing lol. I've definitely taken my caching strategies to the next level, which is working really well. I want to do a write-up on it someday.

How do people like this even make it far enough to get this type of job?

It's actually easy, here's an explanation for one simple way you could do it.

On my instance, this post has the URL: https://lm.williampuckering.com/post/171615

On the instance the post originated on, the URL is: https://lemmings.world/post/175809

So on my instance, the post has the ID: 171615

On the originating instance, the ID is: 175809

In the database on my instance, this query will retrieve the post: select * from post where id = 171615

Also in the database on my instance, this query will retrieve the post: select * from post where ap_id = 'https://lemmings.world/post/175809'

Using the second query and finding the post by URL, I can see if the post is federated to my own instance or not. If so, I can look at the id field in the database and merely swap it out with the originating instance's ID, and form the URL to access the post as it exists on my own instance. If the post isn't federated on my own instance, then of course this won't work. But that makes total sense, since you won't be able to transform links for external instances to the corresponding entity on your own instance, because it doesn't exist there.

tl;dr - You can look up local entities by ID, and you can lookup remote entities by original URL. Then you just need to swap the ID in the URL to the ID (primary key in the table) in the database, if it exists, to convert a remote link to a local link. If a link can't be converted, you can just leave it as-is.

The capability needed to add this functionality is already there. Someone just needs to decide how to handle it on the frontend elegantly from a UI perspective, and decide how the backend will pass what's required to the frontend to drive the functionality. But the plumbing is already there.

One practical way to go about this would be to add one or more API endpoints to transform remote entities (URLs) to local entities, if they exist. Whenever posts/comments/whatever are loaded into the client's browser, Lemmy UI can have code that takes any links that match patterns for Lemmy entities, and use the API endpoints to transform the remote URLs to local URLs, if it can be done. For those that can be done, swap the remote URLs on the frontend for the local ones (at this point it's essentially just find/replace). That's one quick and simple way to do it that shouldn't be all that performance-impacting. There might be more elegant and efficient ways I can think of if I put more effort into thinking about this, but that for sure would work and be a decent first-cut solution. You could even add a caching mechanism (or maybe even a new database table) to persist the mapping as it happens so that you don't need to do it on each request for a given entity, only the first time. Also, doing it this way allows for content that is not yet federated to work if one day it becomes federated (ie. try to do this mapping or each entity, everytime, if it never works, until one day it does).

You're seeing that toast about versions since backend version 0.18.0 switched away from using a websockets-based API to a REST API, and the Jerboa client app is (in a not-so-descriptive way) warning you that the backend you are connected to isn't aligned with the app version in terms of what it expects of the backed. This should go away pretty soon as more servers update their backend version and the Jerboa app update hits more devices.

What a sorry state Canada is in when people are hired out of desperation without proper vetting to ensure they are suited to their jobs, even if there is a nurse shortage.

EDIT: Ah thought I was in a Canada community, my mistake. But I guess there are worldwide problems in healthcare these days.

It may make things simpler for the user, but at the cost of storage and performance of every instance in the index, which won't scale well as more instances are added over time. I personally think it's better the way it is. As long as you are educated enough to know how to federate with other instances you choose to federate with, you can keep your own instance minimally connected to only the instances and communities you actually care about.

Maybe a good compromise would be for such an idea as a globally replicated index, to be optional, so individual instances could keep it disabled if they wanted to. If you choose to enable it as an instance owner, the pain points for your end users go away, at the cost of performance and other potentially negative side-effects. If you choose to keep it disabled, you can still federate with any instances you want, but you won't participate in the index. Or maybe your instance would be listed and replicated to other instances' indexes, but your own instance won't receive updates as the global index continues to grow. Since it would just be for convenient discoverability, there's not really any problem with that. No functionality would be lost for your or any other instance.

I still have an active Netflix account (for the odd thing I haven't yet added to my self-hosted Jellyfin instance), and actually downgraded from the Premium tier to the Basic tier a few months ago when they started cracking down on password-sharing here in Canada.

Even though the Basic tier is "only 720p", I barely notice the difference in quality since my TV (and a lot of other modern TVs) has built-in upscaling that works surprisingly well. And I'm the type that is really picky about picture quality, particular about codecs and encoding methods, and all that jazz. But I'm really happy that I managed to get onto the Basic tier before they removed it. I was prepared for a clear drop in quality in return for cost-savings, and I was okay with that, but was delighted to find nothing had noticeably changed after switching over.

The Basic tier checks all of my boxes, verifiably:

• 1 screen at a time is enough
• The end result of the video quality I can perceive is perfect
• Cheapest plan without ads

Sometimes I even wonder if my TV is even actually upscaling from 720p, or if Netflix is just quietly serving 1080p in reality, but was just continuing to advertise 720p to deter people from the cheaper plan to get them onto a more expensive one, with plans all along to phase this tier out.

My parents, who were previously sharing my account when I was on the Premium tier, ended up getting their own account also on the Basic tier. The net result is that Netflix makes less money off of the two of our accounts combined now compared to the single Premium account we shared before. So in the end, they ended up losing money, and we lost nothing of perceivable value.

I'll probably end up cancelling our account at some point entirely, and my parents can continue to use their own account without being affected, so the forced split actually saved us all money and made our situations more future-proofed.

Contrary to popular belief, I actually think that the Basic tier could have ended up seeing more uptake in the long-run had people who only needed a single screen and wanted to save money, decided to try it, and notice that the picture quality was more than satisfactory enough, either due to the stream quality being better than advertised in reality, or due to the pretty good upscaling ability of modern TVs. I'm sure word would have gotten around from technically-minded people to the masses at some point, and we would have seen more people switching.

I'm sure Netflix did away with the Basic tier because they knew it could realistically put a dent in their profits at some point.

Whoever thought it was good at coding? That's not what it's designed for. It might get lucky and spit out somewhat functional code sometimes based on the prompt, but it never constructed any of that itself. Not truly. It's conceptually Googling what it thinks it needs, copying and pasting together an answer that seems like it might be right, and going "Here, I made this". It might be functional, it might be pure garbage. It's a gamble.

You're better off just writing your own code from the beginning. It's likely going to be more efficient anyways, and you'll properly understand what it does.

There's a lot of things that factor into the answer, but I think overall it's gonna be pretty random. Some instances are on domains without "Lemmy" in the name, some don't include "Lemmy" in the site name configuration, and in the case of some like my own instance, I set the X-Robots-Tag response header such that search engines that properly honor the header won't crawl or index content on my instance. I've actually taken things a step further with mine and put all public paths except for the API endpoints behind authentication (so that Lemmy clients and federation still work with it), so you can't browse my instance content without going through a proper client for extra privacy. But that goes off-topic.

Reddit was centralized so could be optimized for SEO. Lemmy instances are individually run with different configuration at the infrastructure level and the application configuration level, which if most people leave things fairly vanilla, should result in pretty good discovery of Lemmy content across most of these kinds of instances, but I would think most people technical enough to host their own instances would have deviated from defaults and (hopefully) implemented some hardening, which would likely mess with SEO.

So yeah, expect it to be pretty random, but not necessarily unworkable.

The main advantage to me is that I can work with Invidious as a backend, and whatever I configure there will reflect in Clipious as a client. So as I subscribe to new channels in Invidious, add or update playlists, etc, Clipious will reflect these changes accordingly. Advantages of selfhosting Invidious that indirectly benefit Clipious are of course built-in adblocking by virtue of how Invidious works, SponsorBlock support, and the ability to cache static assets, such as video thumbnails for faster load times, using a reverse proxy (Nginx is what I use). There's a lot more we could dive into beyond this, such as no Google account requirement (for enhanced privacy).

One area where the SmartTubeNext and YouTube ReVanced combo has the advantage is the convenience of being able to cast from your handheld device to your TV. Clipious/Invidious has no casting ability. But I can totally live without that.

I have all my Nginx files separated and using include statements for organization, so I can't quickly and easily post an example, but a good place to start looking is at the various proxy_cache directives.

To be more accurate, I actually self-host Vaultwarden, which is a Bitwarden-compatible server built in Rust. I highly recommend it, it's quick and easy to setup, light-weight, and works with all of the Bitwarden apps, browser extensions, etc.

However, that's come with other tradeoffs in useability, speed, and fediration experience.

Like what? If properly configured none of the things listed should negatively impact hosting a Lemmy instance.

sure I'll be adding an exception/rule for that, but it's not a straight forward task.

It honestly should be to someone who would be hosting any public web application using Cloudflare. Cloudflare makes all of this quite easy, even to those with less experience.

Heck, the removal of websockets will require quite a few changes in my Cloudflare config.

What config are you referring to? In the Cloudflare console? For websockets changing to a REST API implementation there should be nothing at all you need to do.

Sure, someone truly concerned with security knows to do this, but that's definitely not going to be everyone

And it shouldn't have to be everyone, only those who take on the responsibility of hosting a public web application such as a Lemmy instance.

No matter the capabilities inherent in what you choose to host, the onus rests on the owner of the infrastructure to secure it.

Everyone should be free to host anything they want at whatever level of security (even none) if that's what they want to do. But it's not reasonable nor appropriate to expect it to be done for you by way of application code. It's great if security is baked in, that's wonderful. But it doesn't replace other mitigations that according to best practices should rightfully be in place and configured in the surrounding infrastructure.

In the case of the captcha issue we're discussing here, there's more than enough appropriate, free solutions that you can use to cover yourself.

I know right? The free tier would be enough to handle most anything and would take a tremendous load off of the origin server with proper Cache Rules in place. I can't remember which instance it was, but one of the big ones started to use Cloudflare but then backtracked because of "problems". When I saw that, I couldn't help but think that they just didn't know what they were doing. My instance is currently behind Cloudflare, and I've had no problem whatsoever with anything.

Oh yeah for sure, everyone should work on whatever they want without restriction or obligation to be focusing on what someone else wants. And more often than not a pet project is a way to learn a new language or framework with the goal of self-development. That's a great thing.

It's just a thought I selfishly have sometimes when I see many apps in development for the same platform, I can't help but wonder "if all of this effort were focused across fewer apps, could each of those be better than any of these current ones are individually today?" Of course the number of devs contributing to a project has no direct correlation when it comes to the quality or maturity of the product. That's down to the management, skillset of the devs, etc. I'm well aware of all of that, and the pros and cons of the differences in scenarios.

Just thought I'd share the thought out there. In any case, Lemmy getting all of this attention will no doubt lead to the rise of at least a few solid mobile apps that will stick around and not fizzle out into development neglect within a couple of months.

Oh yeah for sure. It's just a matter of time. Out of all of these options, some will just fade away (some already have within weeks of initial release), and some will remain and just continue to get better as the development community continues to get a better picture of where all of the action is, and starts to want to be a part of it.

I keep all secrets and passwords in a selfhosted Bitwarden instance. I don't maintain any kind of "documentation" since my deployment files and scripts are clean and tidy, so I can tell what's going on at a glance by looking at them directly. They're also constantly changing as I continuously harden services according to ever-changing standards, so it's more efficient for me to just continue keeping my codebase clean than it is to maintain separate documentation that I myself will likely never read again once I've published it.

I'm the only one that needs to know how my own services are deployed and what the infrastructure looks like, and it's way faster for me to just look at the actual content of whatever deployment files or scripts I have.

It's a different story for things I work with professionally, because you can't trust someone else working to maintain the same things as you has the same level of knowledge to "just know where to go and what to do". But that doesn't apply to personal projects where I just want to get things done.

Ah sorry, just remembered I put my entire instance behind authentication except for the API endpoints required for federation. The comment I was linking to is in this thread. Just describes how all the info you need to properly transform the links is right there in the database records of the entities you want to transform, so this functionality can easily be added without much work.

Conduit is a great Matrix home server. So quick and easy to get up and running with very little fiddling around. It's a no-brainer to deploy.

The only way to guarantee you can see exactly what you want, without being at the mercy of anyone else, is to run your own instance.

It is actually trivial. Here's a walkthrough with one solution for how to do it: https://lm.williampuckering.com/comment/723311

I run all of my services in containers, and intentionally leave my Docker host as barebones as possible so that it's disposable (I don't backup anything aside from data to do with the services themselves, the host can be launched into the sun without any backups and it wouldn't matter). I like to keep things simple yet practical, so I just run a nightly cron job that spins down all my stacks, creates archives of everything as-is at that time, and uploads them to Wasabi, AWS S3, and Backblaze B2. Then everything just spins back up, rinse and repeat the next night. I use lifecycle policies to keep the last 90 days worth of backups.