What's your "base" stack of choice?

kopper [they/them]@lemmy.blahaj.zone to Selfhosted@lemmy.world – 57 points –

How do you set up a server? Do you do any automation or do you just open up an SSH session and YOLO? Any containers? Is docker-compose enough for you or are you one of those unicorns who had no issues whatsoever with rootless Podman? Do you use any premade scripts or do you hand craft it all? What distro are you building on top of?

I'm currently in process of "building" my own server and I'm kinda wondering how "far" most people are going, where do y'all take any shortcuts, and what do you spend effort getting just right.

84

I'm a lazy piece of shit and containers give me cancer, so I just keep iptables aggressive and spin up whatever on an Ubuntu box that gets upgrades when I feel like wasting a weekend in my underwear.

An honest soul

I get paid to do shit with rigor; I don't have the time, energy, or help to make something classy for funsies. I'm also kind of a grumpy old man such that while I'll praise and embrace Python's addition of f-strings which make life better in myriad ways, I eschew the worse laziness of the all the containers attitude that we see for deployment.

Maybe a day shall come when containers are truly less of a headache than just thinking shit through the first time, and I'll begrudgingly adapt and grow, but that day ain't today.

I use debian VMs and create rootless podman containers for everything. Here's my collection so far.

I'm currently in the process of learning how to combine this with ansible... that would save me some time when migrating servers/instances.

Thanks for sharing. There’s some great stuff in the repo.

After many years of tinkering, I finally gave in and converted my whole stack over to UnRAID a few years ago. You know what? It's awesome, and I wish I had done it sooner. It automates so many of the more tedious aspects of home server management. I work in IT, so for me it's less about scratching the itch and more about having competent hosting of services I consider mission-critical. UnRAID lets me do that easily and effectively.

Most of my fun stuff is controlled through Docker and VMs via UnRAID, and I have a secondary external Linux server which handles some tasks I don't want to saddle UnRAID with (PFSense, Adblocking, etc). The UnRAID server itself has 128GB RAM and dual XEON CPUs, so plenty of go for my home projects. I'm at 12TB right now but I was just on Amazon eyeing some 8TB drives...

Proxmox, then create LXC for everything (moslty debian and a bit of alpine), no automation, full yolo, if it break I have backup (problems are for future me eh)

Mostly the same. Proxmox with several LXC, two of which are running docker. One for my multimedia, the other for my game servers.

I used to do the same, but nowadays I just run everything in docker, within a single lxc container on proxmox. Having to setup mono or similar every time I wanted to setup a game server or even jellyfin was annoying.

Debian + nginx + docker (compose).

That's usually enough for me. I have all my docker compose files in their respective containers in the home directory like ~/red-discordbot/docker-compose.yml.

The only headache I've dealt with are permissions because I have to run docker as root and it makes a lot of messy permissions in the home directories. I've been trying rootless docker earlier and it's been great so far.

edit: I also use rclone for backups.

I use the following procedure with ansible.

  1. Setup the server with the things I need for k3s to run
  2. Setup k3s
  3. Bootstrap and create all my services on k3s via ArgoCD

People like to diss running kubernetes on your personal servers, but once you have enough services running in your servers, managing them using docker compose is no longer cut it and kubernetes is the next logical step to go. Tools such as k9s makes navigating as kubernetes cluster a breeze.

Right now, I just flash ubuntu server to whatever computer it is, ssh and yolo lmao. no containers, no managers, just me, my servers, and a vpn, raw dogging the internet lmao. The box is running a nas, jellyfin, lemmy, and a print server; the laptop a minecraft server, and the pi is running a pihole, and a website that controls gpio that controls the lights. In the pictured setup i dont have access to the apartment complex's router, so i vpn through a openvpn server i setup in a digitalocean server.

I just have a pi 4 running OpenMediaVault with docker and portainer. 😅

I run unraid on my server box with a few 8tb hdd and nvme for cache. From there it is really easy to spin up Docker containers or stacks using compose, as well as VMs using your iso of choice.

For automation, I use Ansible to run one click setup machines; it is great for any cloud provider work too.

I love unraid! Definitely wait between updates though to let them stabilize.

I've set up some godforsaken combination of docker, podman, nerdctl and bare metal at work for stuff I needed since they hired me. Every day I'm in constant dread something I made will go down, because I don't have enough time to figure out how I was supposed to do it right T.T

I use NixOS on almost all my servers, with declarative configuration. I can also install my config in one command with NixOS-Anywhere

It allows me to improve my setup bit by bit without having to keep track of what I did on specific machines

I run unraid on my server box with a few 8tb hdd and nvme for cache. From there it is really easy to spin up Docker containers or stacks using compose, as well as VMs using your iso of choice.

For automation, I use Ansible to run one click setup machines; it is great for any cloud provider work too.

About two years ago my set up had gotten out of control, as it will. Closet full of crap all running vms all poorly managed by chef. Different linux flavors everywhere.

Now its one big physical ubuntu box. Everything gets its own ubuntu VM. These days if I can't do it in shell scripts and xml I'm annoyed. Anything fancier than that i'd better be getting paid. I document in markdown as i go and rsync the important stuff from each VM to an external every night. Something goes wrong i just burn the vm, copy paste it back together in a new one from the mkdocs site. Then get on with my day.

I use Unraid and their docker and VM integration, Works great for me as a home user with mixed drives. Most of the dockers i want already have unraid templates so require less configuration. Does everything i want and made it a bit easier for me with less configuration and the mixed drive support.

A bunch of old laptops running Ubuntu Server and docker-compose. Laptops are great; built in screen, keyboard, and UPS (battery), and more than capable of handling the kind of light workloads I run.

I use a heterogeneous environment with some things hosted in various cloud providers and others locally. Often times, I can usually find the package I need - but if I can't, I usually go for Docker and docker-compose. This is often the case in Oracle Linux on OCI - where docker just makes things so much easier.

For my static stuff I just use Cloudflare Pages and forget about it.

On my homelab it is Arch Linux with my own set of scripts. I used to do VFIO gaming a lot (less now), so I had the host only be a hypervisor and used a separate Arch VM to host everything in a docker-compose stack. The VM makes my server operations a lot more tidy.

My RPI is using dietpi and is natively running the pihole software and a couple other things.

I know some folks swear by UnRaid and Proxmox, but I've always found those platforms limited me vs building things my way. Also borking my own system unintentionally on occasion is a thrilling opportunity to learn!

I setup my bare metal boxes and vms with ansible. Then I use ansible to provision docker containers on those.

For personal Linux servers, I tend to run Debian or Ubuntu, with a pretty simple "base" setup that I just run through manually in my head.

  • Setup my personal account.
  • Upload my SSH keys.
  • Configure the hostname (usually after something in Star Trek 🖖).
  • Configure the /etc/hotss file.
  • Make sure it is fully patched.
  • Setup ZeroTier.
  • Setup Telegraf to ship some metrics.
  • Reboot.

I don't automate any of this because I don't see a whole of point in doing it.

Super interesting to me that you swap between Debian and Ubuntu. Is there any rhyme or reason to why you use one over the other?

I tend to prefer installing Debian on a server, but recently I did install Ubuntu's recent LTS on a box because I was running into an issue with the latest version of Debian. I didn't want to revert to an earlier version of Debian or spend a bunch of time figuring out the problem I was having with Python, so I opted to use Ubuntu, which worked.

Ubuntu is based on Debian, so it's like using the same operating system, as far as I'm concerned.

If doing a fresh server external, I'd go for debian as base(don't need to update it too often + stable)

For apps it's mostly docker-compose to set up portainer/nginx-proxy then from ther just manage the rest from portainer/nginx-proxy web-ui. ony log on the server for the occasional docker updates / pruning for space.

I see a lot of guys going the full kubernetes route and it's something I'm hoping to get into at some point but it seems like a lot to unpack for now.

raspberry pi, arch linux, docker-compose. I really need to look up ansible

Ansible and docker compose.

I just use Ansible + docker_container. Does it all beautifully!

i do this, mixed with a little docker run inside of Makefiles. i store all my ansible playbooks in a repo, along with other repos for different projects and purposes. i store all of those in git repos that i clone via ssh from a server that acts as a NAS backed by zfs.

Debian + docker-compose

I'm currently Ubuntu, but if I were to start from scratch this would be it, simple, basic, does everything I need.

I have a base Debian template with a few tweaks I like for all my machines. Debating setting up something like terraform but I just don't spin up VMs frequently enough to wan tto do that. I do have a few Ansible playbooks I run on a fresh server to really get it to where I want though.

I'm all in on docker-compose + rootless podman. Definitely not no issues, but I've got the hang of the kinds of issues it presents at this point. They're mostly around SELinux and networking, though generally the networking only gets problematic on exotic compose setups - jitsi was a huge pain for me.

Raw server with SSH and an immutable OS too. I'm using fedora IOT for my homeserver, and apart from some initial issues with GPU drivers because of layering issues (now working) that's been basically flawless.

I was on OpenSuse MicroOS, but I had huge problems with BTRFS and decided to give it up in favour of EXT4 + XFS. That necessitated moving distro, because MicroOS uses BTRFS snapshots as the basis for its auto-updating/green/blue system. Fedora IOT uses rpm-ostree instead, and works on any filesystem.

Fedora-server with Podman and Quadlet on btrfs drives. Although I must admit I often use rootful mode in Podman as it works better with Containers made for Docker. Ah and you might want to turn off SElinux in the beginning as it can get frustrating fast.

I have a git repository with all my compose files sorted neatly into directories, i.e. my "stack". Portainer allows adding stacks using a repository, so it's essentially one click deployment once the compose file is on a remote git server.

Xen on Gentoo with Gentoo VMs. I've scripted the provisioning in bash, it's fairly straightforward - create lvm volume, extract latest root, tell xen whick kernel to boot.

Ideally would like to netboot a readonly root off nfs and apply config from some source. Probably bash :D

Some things like opnsense are much more handcrafted because they're a kind of unicorn compared to the rest of the stuff.

That’s impressive effort for a home lab.

Hi jago,

Sorry for the delayed response. I do also see the inconsistency between looking at your post directly vs lemmy.ml. I have noticed, however, that every now and the lemmy.ml throws a bad gateway error, which would imply it's getting overloaded again. That might create situations where lemmy.ml has all comments marked as federated, while some of them were actually dropped mid transit. Same applies to lemmy.one.

I don't know of any workarounds for that, unfortunately. Feels a lot like a bug.

In regards to subscriptions - you're right, the pending state does seem to actually impact federation. Some of my subscriptions to beehaw have been pending since day one but I can see the content just fine. I've written this off for another bug in the software.

I had a look at your profile - I can definitely see the posts you've created as well as the comments. I've noticed some UI bits fail to get refreshed - things like notification status, etc. I found forcing a page refresh helps with that.

Sorry I couldn't be of more help.

Generally, it’s Proxmox, debían, then whatever is needed for what I’m spinning up. Usually Docker Compose.

Lately I’ve been playing some with Ansible, but it’s use is far from common for me right now.

Usually Debian as base, then ansible to setup openssh for accessandd for the longest time, I just ran docker-compose straight on bare metal, these days though, I prefer k3s.

I run unraid on my server box with a few 8tb hdd and nvme for cache. From there it is really easy to spin up Docker containers or stacks using compose, as well as VMs using your iso of choice.

For automation, I use Ansible to run one click setup machines; it is great for any cloud provider work too.

I have a single desktop running Proxmox with a TrueNAS VM for handling my data and a Debian VM for my Docker containers which accesses the NAS data through NFS.

Up until now I've been using docker and mostly manually configuring by dumping docker compose files in /opt/whatever and calling it a day. Portainer is running, but I mainly use it for monitoring and occasionally admin tasks. Yesterday though, I spun up machine number 3 and I'm strongly considering setting up something better for provisioning/config. After it's all set up right, it's never been a big problem, but there are a couple of bits of initial with that are a bit of a pain (mostly hooking up wireguard, which I use as a tunnel for remote admin and off-site reverse proxying.

Salt is probably the strongest contender for me, though that's just because I've got a bit of experience with it.

Debian netinst via PXE, SSH/YOLO, docker + compose (formerly swarm), scripts are from my own library, Debian.

I do the same except I boot a usb installer instead of PXE.

I can never find a USB drive when I need one, thus my PXE server was born. lol

Proxmox and shell scripts. I have everything automated from base install to updates.

All the VMs are Debian which install with a custom seed file. Each VM has a config script that will completely setup all users, ip tables, software, mounts, etc. SSL certs are updated on one machine with acme.sh and then pushed out as necessary.

One of these days I’ll get into docker but half the fun is making it all work. I need some time to properly set it up and learn how to configure it securely.

For a while I tried to run k8s (k3s mostly), then I did run nomad for a while. Now I am just running docker compose on Ubuntu (still have one box running Proxmox, but that will be decommissioned eventually, and mostly just runs one VM running Ubuntu).

I am building a few things to solve specific problems I have with this:

  • Some basic ansible scripts to set up ssh, users, basic packages, etc
  • Docker label-based service discovery/announcement that traefik can consume. (currently working!)
  • Deployment: getting the compose files, config files, and docker images to the right machine and getting them running. (in progress)
  • At some point I will probably get around to automating deployment of the rest of the above via Ansible when it is more stable.

Most of my server hardware is oriented toward having a bunch of disks plugged into them (I am 100% guilty of being a data hoarder), and I am running gluster to glue that all together, so that is something I install onto the servers to share their physical disks and/or mount the logical disks.

I have a stupid overcomplicated networking script that never works. So every time i set up a new server I need to fix a myriad of weird issues I've never seen before. Usually I setup a server with a keyboard and mouse because SSH needs networking, if it's a cloud machine its the QEMU console or hundreds of reboots.

A series of VPSes running AlmaLinux, I have a relatively big Ansible playbook to setup everything after the server goes online. The idea is that I can at any time scrape the server off, install an OS, put in all the persistent data (Docker volumes and /srv partition with all the heavy data), and run a playbok.

Docker Compose for services, last time I checked Podman, podman-compose didn't work properly, and learning a new orchestration tool would take an unjustifiable amount of time.

I try to avoid shell scripts as much as possible because they are hard to write in such a way so that they handle all possible scenarios, they are difficult to debug, and they can make a mess when not done properly. Premade scripts are usually the big offenders here, and they are I nice way to leave you without a single clue how the stuff they set up works.

I don't have a selfhosting addiction.

Probably the odd one here with Arch Linux + docker compose with still a lot of manual labor
updating it after maximum 4 weeks is enough, container more often

I usually set up SSH keys and disable password login.

Then I git-pull my base docker-compose stack that sets up:

  • Nginx proxy manager
  • Portainer
  • Frontend and backend networks

I have a handful of other docker-compose files that hook into that setup to make it easy to quickly deploy various services wherever in a modular way.

NixOS instances running Nomad/Vault/Consul. Each service behind Traefik with LE certs. Containers can mount NFS shares from a separate NAS which optionally gets backed up to cloud blob storage.

I use SSH and some CLI commands for deployment but only because that’s faster than CICD. I’m only running ~’nomad run …’ for the most part

The goal was to be resilient to single node failures and align with a stack I might use for production ops work. It’s also nice to be able to remove/add nodes fairly easily without worrying about breaking any home automation or hosting.

For years over done an Ubuntu LTS base with docker, but I've just recently started using debian base. Moved to debian for my workstation as well.

I'd like to use rootless podman, but since I include zerotier in my containers, they need access to the tunnel device and net_admin, so rootless isn't an option right now.

Podman-compose works for me. I'd like to learn how to use Ansible and Kubernetes, but right now, it's just my Lemmy VPS and my Raspberry Pi 4, so I don't have much need for automation at the moment. Maybe some day.

You can add net_admin to the user running podman, I have added it to the ambient capability mask before, which acts like an inherited override for everything the user runs.

Super interesting to me that you switch between Debian and Ubuntu. Is there any rhyme or reason to when you use one over the other?

I've recently switched my entire self hosted infrastructure to NixOS, but only after a few years of evaluation, because it's quite a paradigm shift but well worth it imho.

Before that I used to stick to a solid base of Debian with some docker containers. There are still a few of those remaining that I have yet to migrate to my NixOS infra (namely mosquitto, gotify, nodered and portainer for managing them).

I use SSH to manage docker compose. I'm just using a raspberry pi right now so I don't have room for much more than Syncthing and Dokuwiki.

Don't underestimate a pi! If you have a 3 or up, it can easily handle a few more things.

I forgot to mention I also have a samba share running on it and it's sooooooo sloooooow. I might need to reflash the thing just to cover my bases but it's unusable for large or many files.

Cloud vps with debian. Then fix/update whatever weird or outdated image my vps provider gave me (over ssh). Then setup ssh certs instead of password. I use tmux a lot. Sometimes I have local scripts with scp to move some files around.

Usually I'm just hosting mosquitto, maybe apache2 webserver and WordPress or Flask. The latter two are only for development and get moved to other servers when done.

I don't usually use containers.

I'm better at hardware development than all this newfangled web stuff, so mostly just give me a command line without abstractions and I'm happy.

I have a bunch of different stuff, a dedicated server with Debian, 4 raspberry Pis + 1 micro computer that acts as a LB/Router/DHCP/DNS for the Pis.

In general I would say that my logic is as follows:

  • Every OS change is done through Ansible. This sometimes is a pain, you want to just apt install X and instead you might need to create a new playbook for it, but in the long term, it paid off multiple times. I do have some default playbook that does basic config (user, SSH key provisioning, some default packages) and hardening (SSH config, iptables).
  • I then try to keep the OS logic to a minimum, and do everything else as code. On my older dedicated server I run mostly docker-compose with Systemd + templated docker-compose files dropped by Ansible. The Pis instead run Kubernetes, with flux and all my applications are either directly managed via Flux or they have Helm in between. This means I can destroy a cluster, create another way, point it to my flux repository and I am pretty much back where I started.

Sounds cool. ansible could never convince me, though, because playbook writing is so annoying.

Oh, I am there with you on that. I got used in my previous job, where everything was done with Ansible, but I still find myself copy pasting and changing most of the times. I actually like way more a declarative approach a-la-terraform.

Overall though there is a lot of community material, and once the playbooks are written it's quite good!

I guess if I would automate my base setups with ansible so that I have a good foundation and have learned the tool properly, I would stick to it, but it was one of the cases were I was pushed away right from the start.