Maybe I'm losing my mind, but I either didn't know, or I had forgotten that Toshiba still made 3.5" hard drives.
I have come to think of them as a laptop drive only manufacturer.
I presume they are enterprise models? Because most hard drives - especially high capacity ones - are enterprise drives intended for servers these days.
I wonder how they compare to current versions of Seagates EXOS line and WD's Gold line.
My twelve 16TB Exos 18's are still going strong, at close to 23,500 hours, are still under warranty for another 2.5 years, and will likely last for a long time before they have to be replaced.
That, and I am only at 48% capacity, so I won't have to replace them for more capacity any time soon either.
I'm kind of hoping that the next time I have to replace them, that SSD's (preferably SATA) will be affordably enough to do it, but that is probably a ways off. Given that I am only at 48%, I could get away with much smaller drives than my 16's, but while ZFS allows expanding by replacing a drive at a time and growing a pool, going the other way (shrinking) is not possible. I'd have to rebuild the entire pool.
Doing this might actually not be a bad idea, though, as this pool has been active and grown dynamically since mid-2012, and there are some architectural choices I probably would do differently if I did it again today...
(Notably I'd probably do native ZFS encryption, to make encrypted off-site backups easier. Back in 2012 I never envisioned backing the whole shebang up to an offsite server, and didn't think I needed encryption. I felt it would only slow me down. There are some ways to do this after the fact, but none of them are really good)
I guess there is a lesson here for being forward looking when you create something that might live a long time.
Either way, this is really not an option yet. Even if I could move to 8TB SSD's (which would require pruning a lot of data, or leaving an uncomfortably small amount of free space) it would still be prohibitively expensive... The cheapest 8TB drive I can find is like $645, and I'd need 12 of them. That's $7740
That said, I do have 12 empty slots in the server, so I could also do 24x 4TB SSD's, but even that would be pricy. The cheaper 4TB SSD's are some really sketchy brands. If I sort by price, by the time I get into something I somewhat trust, we are at the ~$240 level. That's $5760
That said, seeing that I have 12 free slots, I could create a secondary smaller pool only for stuff that matters if it is faster using 12 cheaper SSD's, and keep the hard drives around for stuff where speed really doesn't matter (massive drive image dumps, media library, etc.)
It's too bad ZFS doesn't natively support storage tiering. (other than special VDEV's, but those really don't count, as it is not true storage tiering)
Worth thinking about for the future for sure.
