What hardware combination should I consider for a 20TB file server?

Question

I want to build a file server for my home media. The server will only be used to serve files to my Media PC. Essentially all it needs to do is be able to expose around 20TB of shared folders.

I don't really have a clue on server hardware, or even if this can be done with desktop hardware. I need some of the following considerations answered please:

1. What motherboards are able to hold 20 SATA connections? I've decided to go for 1TB drives. Obviously I won't be buying these upfront, but add them as my media collection grows. I've never seen a motherboard with 20 SATA connections, so I'm guessing you can get controller cards to handle more HDDs.

2. I've heard of something called a SAN; would it be in my best interests to try and find a cheap second hand SAN, or should I try and find a cheap second hand file server? Alternatively would it be better to try and rig a desktop to perform the task? I mainly want reliability, but price is also a governing factor.

3. Knowing all this there is another solution. I have an old laptop that has USB ports. I could also buy an external USB drive and then have these running all the time connected via USB to this laptop. However, I'm not wild about having 20 USB drives connected to 1 laptop (not even sure if this would work), but the power cables required to power 20 USB drives would be insane!

Any recommendations are most welcome.

Answer 1

Interesting project. You need to nail down your requirements and your budget, and that will help you nail down what hardware you need for your design.

Step back from the "1TB drive" decision for the moment; a year or two from now that will seem rather limited. Also consider how badly you want the ability to use 20 drives in this system -- that's a lot of power. The number of spindles your plan calls for will impact your case and power supply choices.

1. 20 SATA connections will be tough to find on a single controller, but there are some options. You can get motherboards that provide 6 or 8 SATA ports, then add PCIe SATA controllers for another 6 or 8 SATA ports per card.
   
   The other option is to use a special port multiplier backplane to turn one SATA port into many. Read through the Backblaze pod architecture that Journeyman Geek's answer links to for an example use. They caution that SATA port multiplication can be tricky, and that their solution works well because they use Silicon Image chipsets on both the backplane and the PCIe cards.

2. SAN is one tech design; NAS is another. In a nutshell, NAS provides file sharing; SAN provides block storage (see SAN vs NAS for some of the gory details). From a computer's point of view, NAS provides file shares across (your existing) network, while SAN provides hard disks across (a separate) network. See the difference? The computer can't partition or reformat a NAS share, but SAN storage acts just like local drives.
   
   SAN equipment is typically very expensive because they use enterprise-level components -- 10k or 15k RPM drives, using superfast Fibre Channel or SAS interconnects. Building your own SAN is not for the faint of heart, and probably not a good starter project for someone new to computer hardware.
   
   On the other hand, building a NAS-style server can be done on the cheap, with consumer-grade equipment, and will be a good introduction to system building.

3. Yeah, no. Just don't. USB isn't a real solution here. A USB bus can handle that many devices connected, but performance will crawl if you try to access more than one or two drives at a time. If you want to play around with the laptop and a couple of USB drives, just to try setting up a very basic network file server, feel free -- I guarantee you'll learn something. But that's completely different project than the project you've outlined above.

Answer 2

Well this is a LITTLE bigger than what you want, but the design/architecture, on the hardware side would be close to what you need. Finding a 20 drive, or even a 15 drive case is gonna be tricky though

Answer 3

1 TB drives are pretty much history when it comes to mass storage (and i mean MASS storage :), 1.5 TB is currently the sweet sport for cost per GB and 2 TB drives aren't too far off either.

Get yourself some rack-mounting enclosures (like the one below, it can hold 5 3.5" HDDs), you can populate them as your collection grows.

if you prefer good looks

and here's a (industrial) Rackmount Chassis for 20x hot-swappable SATA drives ($350):

Here's an excellent illustrated review at WeGotServed.com that you may find helpful.

This solves the problem finding a "motherboard with "20 SATA connections" (the transfer rate up to 6 G will certainly suffice :) and you can connect them to any regular computer you wish to use as your Home/NAS server.

Answer 4

There are several Adaptec, 3Ware and Areca RAID controllers with 24 SATA ports available. Keep in mind that agreggating 20 drives mean you're having 20 times more chance of a catastrophic failure, so RAID-5 or RAID-6 really is mandatory.

You can also add drives to a RAID array with those professional RAID controllers. So my advice is to start with one of these and 8 drives, set up as a RAID6 array, that will provide you about 5.5 TB of effective space. Then grow as you go.

Keep in mind that mixing different drives (even of the same brand) may cause problems and kill performance.

Last but not least, the "desktop drives" have serious vibration interaction when put together in large stacks. Either use the more expensive "enterprise" drives, or use less bigger drives ( 8 x 2TB for instance).

Yes, there definitely have real reasons why storage enclosures, enterprise drives and RAID controllers are expensive. So be prepared to spend some serious money if you don't want to be riddled with troubles.

Answer 5

From Tom's Hardware : 2.5 TB HDD Expected in 2010 (August 7, 2009)

A recent roadmap presented by TDK Corporation--a manufacturer of read/write heads for many hard disk drive (HDD) suppliers--revealed that 640 GB 2.5-inch and 2.5 TB 3.5-inch HDDs should become available around January 2010. For now, Seagate and WD provide HDDs with the largest capacities: 2 TB.

Since you wish to start slow, I would start with one or two 1.5TB USB disks.

Or if you wish, you can get a server with 6-8 internal disk bays, and populate two bays with the 1.5TB disks, then wait. By the time you have need for more, 2.5TB disks will be much cheaper and more available. This is by far the simplest solution and with the best performance.

Answer 6

There are some full tower cases that fit 20 drives, with these. You can of course buy hot-swap bays, but those cost a lot more and in home environment it is unlikely that powering the server down for a drive swap would be an issue. Of course using those hot-swap bays that fit 5 drives in 3 5.25" slots can help to fit 20 drives to more cases.

As for the sata ports, there are a lot of cheap raid cards around that you can use without the raid feature. Just buy a reliable motherboard with enough pcie/pci slots.

To address the points Console made: all this is obviously balancing between cost and convenience. It would be really nice to just get Sun Engineers to put couple of Thumpers in the basement, but the cost is too much for most consumers.

However, if you are willing to put in the time and the effort building your own nas will in most cases be a lot cheaper and often also quieter and more reliable than ready made boxes. As a sidebenefit you also learn a lot about OpenSolaris/Linux/WSH/W2008/FreeNas/hardware/etc. The situation may change in the future if NAS manufacturers start to offer 20TB solutions to consumers at reasonable prices. YMMV.

Answer 7

I'm going to go a little against the grain and recommend buying a WIndows Home Server box.

I have a 4TB server that is primarily doing EXACTLY what he's talking about - serving huge media files up to a Media center PC (and a couple of XBox 360s).

If 20TB is his 'sweet spot', then one HP's WHS servers gives him room for 7TB internal (one 1TB that comes with it and the remaining 3 drawers can be filled with 2TB drives), with an eSata that can be used for immediate expansion.

Drive pooling is a godsend - no worries about remembering what files were put on which drive. Just add a drive to the 'pools' and PRESTO - your 'drive' just got 2TB bigger. My 'Media' folder on my WHS box is about 3TB on it's own.

File copying - poor man's RAID. Makes sure that there are redundant copies of each file so that no single drive going south will result in the loss of any files.

Network backups - will allow him to have fast bare-metal restores of his Media Center PC if THAT goes south (or any other PC on his LAN, up to 10).

Extender support, remote access and a web server are other plusses that you may never use but it's nice to have them.

I wouldn't worry about USB not handling the load, if he went in that direction since he's only going to be streaming one file at a time. Even if he gets a couple more Media Center PCs, it's just not that much of a demand on USB 2.0. (Though the wiring could get messy)

A device like the one reviewed here: http://www.wegotserved.com/2008/09/21/sans-digital-tr4m-b-4-bay-sata-das-enclosure-review/ would do for expansion on the eSata port and you can probably find similar devices for USB. With the external ports provided, you're looking at about 32TB of external expansion available on the HP from the 3 USBs and 1 eSata port.

Answer 8

I don't recommend a custom built fulltower solution, even though that might be cheaper you will get issues with cooling, power, noise and whatnot. A few drives failed due to overheating or power spikes and you are soon wasting more money than you think.

I recommend a simple, cheap computer of your choice with a couple of eSATA connectors on it acting as the file server, and then some external multi-drive enclosures like this one or why not even the much-praised drobo, which will allow you to grow by adding drives when needed. Separate enclosures might cost more upfront but it will be way more convenient, clean and reliable in the long run.