add arrow-down arrow-left arrow-right arrow-up authorcheckmark clipboard combo comment delete discord dots drag-handle dropdown-arrow errorfacebook history inbox instagram issuelink lock markup-bbcode markup-html markup-pcpp markup-cyclingbuilder markup-plain-text markup-reddit menu pin radio-button save search settings share star-empty star-full star-half switch successtag twitch twitter user warningwattage weight youtube

Storage setup, should I add another HDD?

Theme57
  • 18 months ago

I am in the works for building a PC and am unsure if I should get another 4TB HDD when I build it or later when I run out of space as well as if I should have both drives in Raid 1.

I plan on editing videos at 1080p between 10 - 60 minutes. However, I may step into the realm of 2k later down the road. This includes files from a video capture card and camera all with files at 1080p 60pfs, I am unsure of the bitrate. I am not a professional video editor by vocation, rather, more as a side hobby.

The setup if the following:

120GB SSD for video cache 250G SATA M.2 for OS/Programs 1TB SATA M.2 for main files and games 4TB 3.5 HDD (WD Blue) for mass storage 1TB 2.5 HDD (WD Blue) for important internal backup 1TB External WD Passport for important external backup

Comments

  • 18 months ago
  • 1 point

Some suggestions:

I find it painful to have a bunch of small logical drives, since you will constantly have to juggle files around. If you're buying parts afresh, I would recommend getting a single SSD for OS and files; I don't see why they have to be on separate physical drives.

One can take this further by combining the SSD and bulk storage HDD into a single cached logical volume; ideally the system will cache recently-used data on the SSD and it will perform nearly as well as a raw SSD. On Linux, two options for this are bcache and lvmcache. I'm less familiar with Windows, but one option I'm aware of is AMD's StoreMI.

Also, instead of two mirrored HDDs, consider getting a larger number of smaller HDDs in RAID-6 (double parity). Advantages over mirrors: you can now survive 2 drive failures instead of only 1; you can use a higher fraction of raw storage space than 50%. Disadvantages: complexity; need room for more physical drives. I would use ZFS to do this on Linux. Windows has a similar system called Storage Spaces (you might need to buy one of the professional editions of Windows for that, though).

I would strongly recommend getting backup disks large enough to back up all your data, to avoid having to decide what data is important enough to back up. Keep it simple and just back up everything. Only you can decide whether the rest of your data is important enough to justify the extra cost, but if it were me, I'd find it painful to lose a portion of my data, even if it's "just a hobby". (Side note: even if you go with RAID, you still need backups. Redundancy protects against simple hardware failures. Backups also protect against a power surge frying all your drives at once, and accidental deletions.)

  • 18 months ago
  • 1 point

Thank you for the awesome reply.

Funny enough, I've done the research and based of what Puget Sounds Recommends including the Adobe forums, you do want at least three drives (pref. SSD). The smallest SSD won't be use for any real data storage, the OS/Programs won't be used often, while the larger SSD will be more my main files as the larger storage won't be accessed that often.

  • 18 months ago
  • 1 point

Could you link me to the Puget recommendation? I'm curious what reasons they provide, since their recommendation does not comport with my intuition.

What makes you say the OS drive won't be used often? Unless you buy a ridiculous amount of RAM and never reboot your computer, I expect an OS drive to be heavily used.

  • 18 months ago
  • 1 point
  • 18 months ago
  • 1 point

Three SSDs doesn't make any sense (unless you are just reusing old ones, which I suspect is the case). Also, do you already own the 1TB HDDs? The price difference between 1TB and 3TB isn't all that much: is the next 2TB of data really worth less than $20 (per drive)?

Raid 1 doesn't increase space. It just means you can keep going while you buy another drive. And remember to back things up to those backup drives (and seriously considering bumping them up to 3TB if these aren't ones you already have). Raid 0 increases space (and possibly speed of bulk transfer) but is half as safe as regular drives (plus additional ways for your OS to corrupt it). RAID 5 is the best of both worlds (redundancy plus increased storage), but still doesn't work with StoreMI (make sure your motherboard "supports" RAID5 before expecting to use it in Windows (although higher-end copies of Windows should have better means of storage). I'm fairly sure that getting a tiering system working is going to be more important than getting a RAID system working: you'll either need a tiering system compatible with your RAID or manually sloshing data from HDD+SDD to RAID.

StoreMI works with one fast and one big drive, so fancy drive arrays won't work (I haven't really dug into other tiering options, but you might want to check them. Also backups become even more critical when having more and more things that go wrong and corrupt your data). That said, I really expect you want a tiering system for working on video: lighting fast load/saves of whatever you are working on, while not having to manually slosh data from SSD to HDD.

If you really want to go down the rabbit hole: https://www.reddit.com/r/DataHoarder/ will take it to any extreme.

  • 18 months ago
  • 1 point

Based off what the Adobe forums and Puget Sounds have found, three drives for an editing rig is pretty normal, some have as high as four or five. Reason being is the smaller cache drive isn't used for real storage, the OS/Programs have their own drive (not used as often) while the other larger SSD is for the main media files that are fast with the HDD's as larger storage.

I own the 1TB 2.5" HDD already, while the other drives not so much. It's there as somewhat of a prebackup to the external drive since they are both the same size.

  • 18 months ago
  • 1 point

If the OS/Programs drive isn't used often and the smaller drive isn't used for storage, you would still be better off combining that into a roughly 512GB (480?) drive. It will simply be able to access any of the data faster, with possibly and exception if it is accessing both the cache drive and the OS/Programs drive at the same time (and will still likely be faster than the 128GB drive). Buying one drive should be cheaper, faster, and easier to manage than two separate drives. You can get the best of both worlds (performance of bigger drives, management of separate drives) by simply partitioning the drive.

Combining SSD drives past 512GB probably doesn't buy you much (performance doesn't increase), and even less if you often access both at the same time. I can also imagine filling up a case with a HDD array (I'd be equally temped to stuff them in an 8GB DDR3 craiglist special and run FreeNAS on it, but that would be hard to cache). Past that I'd still expect that most of the "lots of SSD" is thanks to buying SSD everytime the price dropped or you filled one up.

So two SSDs (OS/Programs + Cache) (1TB Mass storage). Three HDDs (4TB bulk storage [cached]) (internal backup HDD) (external backup HDD). Sounds like "4 or 5" to me (depending on if you count the external one or not).

The OS/Programs might get a bit cramped if you start using this computer much outside of video editing, but there certainly will be plenty of room to stuff the overflow.

Sort

add arrow-down arrow-left arrow-right arrow-up authorcheckmark clipboard combo comment delete discord dots drag-handle dropdown-arrow errorfacebook history inbox instagram issuelink lock markup-bbcode markup-html markup-pcpp markup-cyclingbuilder markup-plain-text markup-reddit menu pin radio-button save search settings share star-empty star-full star-half switch successtag twitch twitter user warningwattage weight youtube