Well the drives arrived on the past Saturday (4-4TB Ironwolves).
The server shipped today (not sure if that was in the US or China), Amazon is still predicting a wild range of dates (11-20th) and DHL isn’t yet aware of the tracking #. The AMD Aooster talked about in the previous thread (16 GB Ram and 512GB SSD).
Current storage utitlized is about 6 TB. The vast majority being the video library, the minor being photo reservoir, and with hopes of being the primary repo of what is currently in cloud storage (very minor: Mostly articles written by me or novel attempts).
My mind has wandered all over the place, from TrueNAS to Unraid and finally arriving at the notion that for the content described above something very simple would be best. I’m currently favoring a RAID5 BTRFS volume (though there are all sorts of articles about Raid5 being risky on btrfs, but most are more than a year old and apparently progress has been made?). For the short term, I will be keeping (powered off) the USB drives that currently contain things as an offline archive/safety net. The btrfs (if it doesn’t implode) has the advantage of being low overhead. I am thinking to utilize the raw drives and not even partition and run Arch on the server, as it would seem best to stay on a current system (vs an antique like Debian) with recommendations on reddit/btrfs being to run a current kernel (>6.3), plus it lessens the distro news I need to keep updated on. I will make the effort to make a very minimal install. I did run the previous server on ENos for a year, so I know what I’m asking with regards to maintaining an Arch server. I did not take the trouble to minimize the install (lessening the updates) on the previous server.
Please tell me I’m basically insane (probably true). Your experience will be weighed.
Besides, if you do not have a separate backup it is recommended to use RAID6 instead.
Reason: Typically a RAID is setup with similar drives. Similar in model and age. Because of that similarity, it is more likeley that if one drive fails a second drive will fail shortly after the first.
Imagine you have one broken drive. The rebuild of a RAID5 array with one new drive can take very long. During that time you have a higher chance that a second drive fails and all your data will be lost. A RAID6 protects you from that.
You can find several articles on the web talking about that.
Well, although I very much like endeavouros or arch linux. I would not use it for a server that needs to run 24/7. I have had that in the past and was not happy with it. It depends on which services you want to offer with your server.
In my use case, I am providing a private nextcloud to my familiy so that everybody can sync files from iphone to the server. The nextcloud has some strong dependencies on php and other stuff. Arch linux was changing these components too often so that my nextcloud broke several times.
Now I am back to a debian server and that is stable and rock solid. This is what debian is made for. This is not what arch linux is made for.
In my case, it’s only me, my files. The services are simply jellyfin media server, DNS, time at the moment, until I find something else I might want it to do (possibly NFS very very seldom).
The risk factor (if not like 100%, because I understand it’s edge cases that are issues?) is possibly acceptable, because A) photos not likely to change and are backed up elsewhere B) media can always be re-ripped and/or restored from the USB drives that are currently in use. There will never be content that exists only on this drive array (at least that’s my thought now).
Yes, nextcloud has far more dependencies than jellyfin does. I am currently running alpine on the current server, because of exactly what you said (too many updates and at the time, too unstable zfs modules with regard to kernel version). But if I update somewhat infrequently and use a minimal load, I think it’s probably ok.
If you feel btrfs is simply going to rot in six months (on the server), then yes, I shouldn’t use it. My read was that the btrfs issues were with regards to commits and power failures. Since my use case is primarily read-only, I didn’t see an issue. Is that correct? I really don’t know, most comments are more than a year old (and maybe 5) with regards to raid. I don’t want to do Raid 6, because that would take up 1/2 my drives…might as well do Raid 1 if I was to do 6.
I could still continue to use zfs or use an mdadm raid, just seemed overkill for what I’m doing.
I can not answer that. I am not a btrfs expert. Last year I asked about the btrfs RADI5 status on the btrfs github page.
Based on that reply and on everything else you can find on the web, incl. the btrfs developers warning to use it, I would not not use it. But you are free to ignore that.
Zfs has certainly been serviceable, but it’s just kinda busy for a single user system (consuming ram as if it were going out of style for caching that doesn’t help anyone). I might try using mdadm which is certainly a mature technology.
Well it is good (in a manner of speaking) to see a dev respond as late as August 2024. I wonder if there is any place they want issues opened lol?
mdadm is certainly good. I used it for many years before I moved to zfs. But be aware, if you use mdadm with btrfs you will not be able to use the btrfs data recovery options. Simply because btrfs only sees the one mdadm device, e.g /dev/md127. One device is not enough for btrfs to repair data. With mdadm+btrfs there is not redundancy on btrfs level. And in that scenario you also need to figure out how to do scrubs. You probably need to scrub twice: first scrub with mdadm and then with btrfs.
Yes, I saw that mdadm with btrfs is not recommended, in fact they said just use btrfs (which is why I got where I got in direction). I also saw a one month old mailing list reference where it seems to say Raid5 is fine, but Raid6 hasn’t yet gotten the love that 5 did with a patch. Also random people on reddit say they’ve been running raid 5 or 6 for years without issue.
That’s why I just don’t know
Actually no, I’m not a beginner, except at RAID (a technology I always wanted to mess around with, but never had a symmetrical set of drives). I’ve been an IT person specializing in Unix, networking, and security all my life with a MS in Computer Science.
How many NAS sytems at home did you have? How often did you actually perform a disaster data recovery at home with your data? Knowing the theory is just that - theory.
I’m a home environment with a single user (me). I stated I don’t have data that lives in a single place. I don’t think I need a DR plan (and yes I’ve had to make them before for work).
More than anything I like to learn stuff (so that’s a focus here).
I’ve had 3 physical NAS systems at home before. The latter of which was home brew with ZFS for the last 4 years. Actually I’ve had the system for 7 or 8 years…so probably at least 7 and I just don’t remember the years
Use docker, docker gets around all the instability and runs containerized (from the few I use, it seems to be mainly Alpine Linux as the base).
I think the main problem is access from the real world, but I can access everything from a command line ssh using Tailscale, I can even watch movies using, Emby on android and the Tailscale address:
Just wanted to share my experience, before the thread closes down.
Ok, so after 6 days of not being able to track the shipment, I contacted the Aoostar rep who had made contact with me (through Amazon) over the weekend, and today the amazon tracking # was updated. The package (which I suspect was found or reshipped) is still in Hong Kong. However, at least I now will have some idea where it is and hopefully know the day it will hit my porch as it gets closer.
I wonder if two will arrive (lol)? I’ll be happy with just one.
Ok, apparently they have now completed the server build and it’s actually going somewhere (or at least DHL says they have it). Predicted delivery is Monday Feb 17.
I’m leaning toward general purpose server vs specialized like TrueNAS or Unraid, though still gathering information (when I can’t do anything but…since it’s not here yet)
Thanks everyone.
Let us know how you set this up then as a general purpose server. I’m also interested. I have very limited knowledge of TruNas or Unraid and i don’t tend to save a lot of data myself but i have an interested party who is a professional photographer who would be interested in setting up one of these. He also is a graphic gaming artist for a major gaming entity. So he has lots of data and was waiting for Synology to come out with something new.
Well this is (for now until I see how it goes for a while) purely one of those, ‘Don’t do as I do, but do as I say’ situations. I might recommend it to others if it pans out after at least six months and doesn’t have filesystem warp core breach (Star Trek reference)!
With my low usage (me), the primary be-all-end-all is to contain a collection of non-critical data where it doesn’t TRULY matter if I lose it, due to being able to recreate it. So that being said, I’m taking some risks for something that will most economically (like power usage) fit my needs.
I’m going to put my 4 identical drives into a Raid5 configuration with BTRFS (edge case issues there…so they say), purely for pooling the drives into one storage array. If I have catastropic failure I may revert to using ZFS, but I don’t want to, because I don’t like the out-of-kernel nature of ZFS (currently using ZFS on the old server) with it lagging behind new kernels.
The OS configuration with be Arch (because I’ve been there before with ENOS being the OS, but I’m gonna trim it down to reduce updates) and that’s mostly because A) I like up to date software B) I can gain some synergy with the desktop also being an Arch derivative (ie common issues and problems). This is of course totally contrary to everyone else that would install Debian or other relatively static distro on a server (and I would too if it were my job on the line vs my personal toy).
It will (from OS perspective) be a DNS cache server, NTP server, and jellyfin media server. There will also be a photo archive, though that could fall under the nextcloud section below. The media and photos will be on the raid array.
With additional storage, cpu, and memory compared to my old server, I’m contemplating a nextcloud server as well (or other similar solution)…maybe just a samba server? Hmm, I wonder what it would take to have my android phone do NFS ?
This basically is a solution that only a linux tinkerer would consider viable (in my opinion), however, that makes me totally giddy
I too would probably opt to use btrfs. Everything is well above my knowledge level as just don’t have experience using these type of devices. Not that i can’t… but i just haven’t. I’m kind of in the same category as I don’t have to worry if i lose what i store. Currently i have no data for that purpose. I’d also like to try having the os (EOS) installed on the same device and use it for both purposes since the drives for storage are separate anyway.
I’d also like to try having the os (EOS) installed on the same device and use it for both purposes since the drives for storage are separate anyway.