how much backing up would you do of a media server?

DavidDoesLemmy@aussie.zone to Selfhosted@lemmy.world – 81 points –

I have a nas with 2x10tb drives. I mostly just have music, movies and tv shows on it.

People talk about raid not being a backup, but is that relevant for non-original data? I mean I can always get the media again if need be. It would just be an inconvenience.

What would you do?

76

I only backup data that I either can't replace or would have to spend significant effort to replace. Most of what's on a media server doesn't fall into that category.

Media Server? No content backup at all.

If you lose everything, just download new stuff you want to watch, or redownload a few TV series/movies.

Music? There are streaming services.

Only backup configurations and maybe application data, so that the reinstall will be easy. Those few kB/MB could sit anywhere. I'm using GitLab for this purpose.

Edit: Images! If you have your photos on there, back them up! They can't be replaced!

The streaming services wont work if you have no access to interner lol.

At my last job I had to travel to my work dailly for over an hour in one way, for almost the whole travel I didn't have any network or phone reception.

Will much rather just have music on a media server and a client that allows me to locally download some of my favouritr music for such situations like navidrome and synfonium than pay for spotify premium to allow me to do that.

Streaming services let you just mark playlists for offline use, I have my whole spotify library offline.

Most streaming services have that under a paywall, which in that case I will much rather just make my own if I have a system to do it.

What music streaming service is even usable without paying?

Spotify is the only one that I know of that has a free plan and it's (supposed to be) terrible

What I'm saying is that if I already have hardware to make one my self, why pay for it?

Edit: also some people just can't afford to payfor streaming services

Sure you can, but I very much prefer the experience of something like Spotify. It's very easy to find songs and to just listen to them at any time.

And some obviously don't like piracy (I don't care much since I have around 8 TBs of films, shows and other shit with Plex.)

We're talking about replacing lost content here though. And as such you can use the streaming services as a "backup" by re-ripping your whole collection if you lose it.

I'm actually doing this now as part of a library cleanup. Zotify + beets are a great combo to pull down vast quantities of music and properly sort and tag it.

Then I stream it to my phone in my truck using ampache and ultrasonic, which does have a local buffering option.

However if you have some exotics that you ripped from rare discs, demos or prerelease, live recordings with sentimental value etc. I would suggest keeping those properly backed up. I don't have many of these, but the ones I do have are backed up both cloud and offsite.

Do people really have so much music that it's get's hard to just keep it backed up?

I personally never went over 1gb in size of my music library,

Personally I live in a very rural location and I farm, so I can spend a lot of time on the road or in my tractor. 1gb wouldn't get me through a day in the field, so I have a pretty big collection with a lot of variety. We don't even have reliable FM radio here, so it's bring your own music or listen to the diesel roar.

I grew up on a farm, still help out sometimes. And same our fm radio doesnt work on most ny routes.

My songs were almost always just highly compressed mp3's I would get years ago si ce back then spotify wasnt in Croatia so my only way was yt.

I'm probably an outlier, but I have a full 3-2-1 backup. Over 100Tb myself, with it all backed up. I have a safe off-site I back everything up to weekly and then annually I do a full backup to LTO tapes.

I lost my media once. I don't want to go through that again.

Wow!

Given your previous experience, your approach is understandable.

I have an old raid setup on which the card died, and Crashplan deleted my Backups when the array went offline (yea, I was pissed)l.

One of these days I'll find a card on ebay, recover everything, and back it up again.

If I'd had a second backup...

Pretty sure something like 10 years ago crashplan deleted a bunch of customer data in a deduplication job gone wrong.

What’s the cost on the drives and tapes? (Roughly)

Tapes aren't bad, I can get few dozen TB off eBay for a couple hundred. Drive was crazy though. Dropped 2 grand on it and it still isn't that good of a drive.

Oof.

Yeah, last time I looked the drives are stupid money (I feel like the should be much cheaper).

I only backup data I've generated myself, nothing that was autogenerated or downloaded from somewhere else.

This goes far beyond backing up since not a long ago I had to deal with emptying the house of a deceased person that had been locked for a while and got to the conclusion the only things worth keeping are original ones (photos, handwritten letters and so...). Anything that could have been bought somewhere else, no matter the antique it was resulted to be almost worthless, not just to me but also to pawn shops, as it seems to be easy to find the exact same thing somewhere else.

So I took that as a life learning and apply the same concept to my data :)

You think you can get the media again if need be.

Depending on how large your collection is, would you remember every item in it? How much effort did you put into organizing it?

IME it's far more of an inconvenience and expense rebuilding data from scratch than properly backing it up. And the peace of mind from a robust, tried and true DR process is golden.

You think you can get the media again if need be.

Well that's my usual approach however we now live in the world of censored tv shows by netflix meaning some of the new media you may get might not be the original thing. :(

If you woke up and all of that data was gone tomorrow but you didn’t care, then there is no reason to back it up IMO.

Hell, I download things multiple times sometimes just to spite Comcast.

I can always get the media again if need be.

Doesn't that mean you already have backup? It may not be the easiest to restore, but it is a backup nevertheless.

I wouldn’t really class that as a backup, that’s like saying you have a spare tyre because you can always buy one from a garage!

It’s different because when you need a tire, you need it now. When you need a movie it can wait 5 minutes.

I only back up my music collection because I put extra effort into organising and tagging everything, plus some of it is rips of CDs not available anywhere. As for movies and TV shows, I only back up configurations and catalogues of the relevant apps, the contents themselves are 1) too big to be feasible to back up and 2) 99% of the time available to re-download.

This is how I do it. I have all my media on a raid 5 tho. Broken hdds are unavoidable in the long run, so I want protection for that. If something goes sideways at rebuilding, so be it. Most of the movies and shows I'm wondering what I save them for anyways. My music collection is worth more to me and backed up properly. Same goes for my personal stuff.

I have the equivalent of RAID 5 too, but mind the usual "RAID is not a backup" - if you deliberately delete something (or something goes wrong with an app managing your media), hardware redundancy won't save you in any way; it only helps if the data is intact and you want to remedy a hardware failure.

My brother also has his own NAS at his house. We sync our media between both of our servers to both share it and to serve as an off-site backup.

Everything else on my nas gets backed up to a cloud provider.

Like you said, it could be replaced it'd just be inconvenient, and media is kinda bulky so cloud storage for all of it would get a little pricy.

"RAID is not a backup" just means that the entire RAID disk counts as one copy of the files.

Non original media doesn't matter unless you think you have old obscure things that aren't even on Internet archive or private torrent groups, or it has some sentimental value like a VTR recording of something you watched as a kid. Most you can download again and likely in better definition.

Focus first on getting at least 2 separate backups of the most important stuff: your family photos and videos. Then records, then work stuff.

That saying also means something else (and imo more important): RAID doesn't protect against accidental or malicious deletion/modification. It only protects against data loss due to hardware fault.

If you delete stuff or overwrite it then RAID will dutifully duplicate/mirror/parity-check that action, but doesn't let you go back in time.

Thats the same reason why just syncing the data automatically to another target also isn't the same as a full backup.

Very true. Also I like your username :)

The key concept here is how valuable your time is to rebuild your collection. I have a ~92TB (8x16 radiz2) array with about 33TB of downloaded data that has never been backed up as it migrated from my original cluster of 250GB drives through to today. I think part of the key is to have a spare drive on hand and ready to go when you do lose a drive, to be swapped in as soon as a problem shows up, plus having email alerts when a drive goes down so you're aware right away.

To add a little more perspective to my setup (and nightmare fuel for some people), I have always made my clusters from used drives, generally off ebay but the current batch comes from Amazon's refurbished shop. Plus these drives all sit externally with cables from SAS cards. The good news is this year I finally built a 3D-printed rack to organize the drives, matched to some cheap backplane cards, so I have less chance of power issues. And power is key here, my own experience has shown that if you use a cheap desktop power supply for external drives, you WILL lose data. I now run a redundant PS from a server that puts out a lot more power than I need, and I haven't lost anything since those original 250GB drives, nor have I had any concerns while rebuilding a failed drive or two. At one point during my last upgrade I had 27 HDDs spun up at once so I have a lot of confidence in this setup with the now-reduced drive count.

My goodness, used drives are a nightmare indeed for me

It definitely comes down to the specific line. My last 6TB drives were my first jump into SAS drives but that series was terrible and I had a bunch of failures. I really should check google more often before jumping on what looks like a good deal.

Living life on the edge currently, but thats because I dont have a means to backup my media at the moment

I'm currently running mine on Windows and use SnapRAID and DrivePool as my defense against drive failures. I think I have 7 data drives and 2 parity at this point (totalling around 90TB). Beyond that I copy the Snapraid whatchamacallit to a separate backup drive along with my OS drive. This isn't really a 'backup' but in the scenario where I have several failures and no way to restore, I still have radarr/sonarr keeping track of my library and a membership to several private trackers.

I wouldn't worry too much about losing media files as most can just be downloaded again. I find it more beneficial to make use of all the storage space you can rather than trying to do a 1:1 backup, which gets pretty absurd once you start getting up there in movie/TV count.

I do n+2 of my media. It's overkill but I have the space. You might want local n+1 for convenience of restoring, but it's not necessary. You could absolutely consider the ability to torrent something as one of your backups.

So technically I have n+3. If my house burns down (n+1), and my off-site storage (+1) explodes, I can always still torrent (+1).

I only backup what I can’t redownload, ie personal media. Everything else would be annoying but probably also a “great filter” if it all got lost and I’d have to make choices about what I really wanted in the first place.

I just need my config files. The media I'm ok with losing it. I've been losing it for years.

Very specific media like rare or modified Rips gave an extra copy on an archive folder. All my cloud storage and personal backups also go to the archive folder. That folder then gets backed up to local raid 6 NAS, and then the qnap software syncs that up to backblaze once a week.

I only back up things that would make me sad if I lost it or cause me a lot of time-sensitive work. Personal data files and configuration files. Media? I wouldn't sweat it if my media drive got corrupted by malware or a hack or a lightning strike. I'd just live with a smaller library until I get things re-download again. And I'd be ok if I can't find a handful of the rarer things. Pictures of my family? Backed up locally and on a remote server with immutable backups. Configuration files? Synced with a remote git repository.

I mirror it 1:1
If you can't afford to mirror it, backup the files that were and are currently the most difficult to get atm.

I use Linode object storage for backups using Restic. 500gb of storage for 5eur/month.

I don't backup logs, backups made by an app, cache, thumbnails and other stuff. I backup actual container data, so I can reatore it, restart docker and it works like nothing happened.

I’ve lost my music collection twice. Once when I gave away all my cds in a fit of minimalism, once when our house got broken into and they took all our cds.

It’s farking annoying and takes forever to get all your music again. At the very least make sure you have a list of albums so you can remember what you had.

I was just thinking about this recently. For my original data I already have multiple copies: 2 desktop PCs, home and office, synced with a home NAS, adding a server in the office soon too, laptop has everything but photos (which is a lot since I am into photography and timelapses). My non original media has only one copy, but will soon have a second copy in the server at my office.

But I can't count on using my office at my job as a long term thing. For my original data, I have been planning on getting something like Backblaze for a full professional off-site copy. For all my non original media, well... It would be ok to lose it I suppose, but I would rather not. Would this be a good use case for some sort of other stable media? I forgot what it was called, but I recently saw a post about some high density disk (like some sort of multi TB blu ray disk thing?) That seems like a decent solution, better to lose 1 year of piracy instead of 20 years of piracy haha. I have lots of obscure stuff that would be hard to get again, curated by and copied from cinephile and audiophile friends, rare movies I ripped from university library DVD discs and even VHS tapes!

Maybe I need to start learning about some alternative storage media for that stuff. Anyone have suggestions? Some sort of tape or disc for this kind of large but immutable media?

i have ~24tb (6x4) unraided x3 on separate nas, one of which is only plugged in and turned on every few months. if i lose a drive, i can clone the whole thing quickly from one of the other 2 backups.. i dont have to worry about failed raid arrays and i get a bit more useful storage..

in the ~5 years i've had this setup going i think ive only lost one drive, and it started throwin smart errors long before it died

i guess im using a 'redundant array of inexpensive nas' = RAIN! is that a thing? can i make it a thing?

Curious what was the model of your drive failure? I have 6 years now on a bunch of 8TB WD Elements/EasyStore drives as well as some 10TB-14TB WD MyBook, Elements, and refurbished WD drives from serverpartdeals in the preceding years. Still no failures yet but I'm expecting one eventually.

I just want to add, you may not need to backup what you think you can redownload easily.

There are cases of series or movies being really difficult to find or it takes you a long time to find a working torrent. For those it definitely should make sense to back them up, but its a bit subjective feeling which ones those might be.

Another thing in my case, I got movies and seriea for my child on the server. He expects that is accessible, always, and is too young to understand that data can get lost. So I am also backing that up, it could be easily replaced but it would take me time and this would result in my child being unhappy. Not worth it!

I have 3x 14 TB in a raidz1 setup on TrueNAS. Would take awhile to redownload but isn't critical in any way, so I feel like that's a good compromise.

I back up everything. I use Stablebit Drivepool with duplication for all of my source code, media, photos, documents, music, books, laptop backups, etc. I back that up periodically to a Drobo DAS and 8 Bay USB enclosure setup under Drivepool. I also have off site backup (working on a new NAS which will be accessed over a VPN). I don't want to spend the time worrying about loosing anything I have put time and effort into. Been there and done that. Drives are relatively inexpensive but can fail without warning.

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

Fewer Letters More Letters
Git Popular version control system, primarily for code
NAS Network-Attached Storage
Plex Brand of media server package
RAID Redundant Array of Independent Disks for mass storage
RPi Raspberry Pi brand of SBC
SBC Single-Board Computer
SSD Solid State Drive mass storage
VPN Virtual Private Network

[Thread #357 for this sub, first seen 16th Dec 2023, 03:45] [FAQ] [Full list] [Contact] [Source code]

I have a total of 48 TB across 4 HDD, plus the system SSD on my old desktop, now just server. It's unfortunately running windows, because I haven't had the time to reinstall everything from scratch yet even though I've intended to for years.

Anyway, I just run Backblaze for backup on it, except one drive which is important photos/files and that gets an extra backup to a RPi using Syncthing. The other drives are all downloaded media that could be replaced given enough time, but Backblaze backs up my whole PC and hasn't complained about the size yet. I thought for sure they'd cut me off or something, but nope still just paying $99/year for a single PC backup.

I backup my music, photos, docker settings and that's about it. Daily backups to one external HDD, but recently setup a second backup that's runs weekly juuuuust in case. The music is only because it's taken me a long time to build upy library, and that would be painful to lose. TV, movies, meh.

I have some rare media that I know would be extremely difficult to replace, so I back that up, but the general stuff is less important.

However, with rights holders constantly trying to move away from the idea of permanent physical ownership, some media will become harder and harder to find in their best or purest forms, disks will go out of print and the used market will start to slowly die as media ages and rots.

Yeah. I only have 6TB. Most of it is just movies and shows for plex. I don’t have backup at all. I could always redownload what i want to watch. I noticed that i rarely rewatch what i download. It’s pointless to backup it.

I have it replicated to a backup server then to one in the dc

You have a data center?

I have a similar setup.

I have a 16tb USB HDD that syncs to my NAS whenever my workstation is idle for 20 mins.

I have all my spare drives pooled together into a frankenNAS system in a spare Fractal R5 case. Whatever media fits gets a backup on there (in order of personal importance). Otherwise I will reacquire all my ISO's should disaster strike.