Why not do both? I run proxmox on my physical hardware, then have guest VMs within proxmox that run k8s.
Advantages of proxmox:
- Proxmox makes it easy to spin up VMs for non self host purposes (say I want to play with NixOS)
- Proxmox snapshots make migrations and configuration changes a bit safer (I recently messed up a postgres 15 migration and was able to roll back in a button press)
You can then just run docker images through Proxmox, but I like k8s (specifically k3s) because:
Advantages of k8s:
- Certmanager means your HTTP services automatically get assigned TLS certs essentially for free (once you've set up cert manager for the first time, anyway)
- I find k8s' YML-based configuration easier to track and manage. I can spin my containers up fresh just from my config, without worrying about stray environment settings I might not have backed up.
- k8s makes it easy for me to reason about which services are exposed internally to each other, and which are exposed on the host outside of my k8s cluster.
- k8s services get persistent DNS and IPs within the cluster, so configuring nodes to talk to each other is very easy.
And yeah, this way I get to learn two technologies rather than one 😁
Voice / video requires a separate TURN server, IIRC.