What are challenges with Linux Storage Servers having 7 or 15 drives?

Question

I'm evaluating options for image webservers for Tralev (in the case it grows).

I was looking into Hetzner's storage line.

They offer storage servers with 7 and 15 drives.

7 drives with software raid and 15 drives with hardware raid.

Assuming they have good hardware raid controllers for 15 drives (good monitoring), what are challenges of choosing such servers rather than commodity hardware?

Possible problems:

• partition sizes with ubuntu 12.04 with possible max limit 2GB (they still don't offer ubuntu 14.04 LTS)
• since having backup in another data center is a good idea, the problem is availability of similar backup servers by other companies for having backups, any idea about this one?
• since I'll be serving images and assuming HD has read transfer rates of 60MB, which is around 500MBps the server could be hitting 1Gbit network port with anything higher than 4 drives in raid 10
• assuming 400KB average image size and the outbound of 500Mbit, the server can serve 156 images per second, which is about 5 users per second (excluding robots traffic)
• if in average consuming 300MBit outbound, it looks like the server woould generate 88TB of traffic monthly - using formula (300000000 / 8 * 60 * 60 * 24 * 30) / (1024 * 1024 * 1024 * 1024)
• since Google Cloud Storage is about 20$ per TB, storing 52TB of backup would cost about 1100$ which makes cloud storage services much expensive than having server replica

So it looks like 15 drive servers could hit 1Gbit network bottleneck (assuming that images of poor quality which are never read are removed from the system). In the case there are

So my analyzes shows that there are less risk in going with commodity hardware (i.e. 4x2TB):

• less risk in hitting network limit
• cheaper backup to another commodity hardware in other datacenter

Anything I missed?

Answer 1

hat are challenges of choosing such servers rather than commodity hardware?

Äh? Sorry, but Hetzner is not some miracle shop. ALL their server run commodity hardware. These raid cards they use are commodity hardware.

Tehy likely go hardware raid because in a 15 disc setup they likely run a SAS backplane and using a motherboard with SAS connector limis them.

So it looks like 15 drive servers could hit 1Gbit network bottleneck

Are you a hobbyist? A 1 drive machine can hit the paltry 1gbit network bottlenech - every SSD on this planet dlivers more than 100 megabyte per second.

What you do miss is a reality check.

"Commodity Hardware" in your world seems to be "low end stuff that home users buy".

Google gives:

Commodity computing is to use large numbers of already available computing components for parallel computing to get the greatest amount of useful computation at low cost. It is computing done in commodity computers as opposed to high-cost supermicrocomputers or boutique computers. ...

SAS backplanes, hardware raid, SAS discs are all commodity hardware. THe definition of commodity hardware is "In wide distributoin and easy to get over regular channels". Now, if you ignore the small corner computer shop and check some larger outputs of goods (Amazon, newegg) you will find all those things there. The definition of commoditxy is not "stuff build for home use and low end".

And your challenges - with all respect - are on a very low level. 2 gigabyte partitions? SERIOUS? IOs linux really that far behind? I thought ZFS etc. are top notch.

Since I'll be serving images and assuming HD has read transfer rates of 60MB, which is around 500MBps the server could be hitting 1Gbit network port with anything higher than 4 drives in raid 10 assuming 400KB average image size and the outbound of 500Mbit, the server can serve 156 images per second, which is about 5 users per second (excluding robots traffic)

Let's hope yo uare not an admin. You talk MB/S like the disc is a SSD where random access does not matter. The moment you hit 20 requests per second you MAY be lucky to get 10MB/S out of those discs - maybe a lot less. moving the head costs. Random image access is head movement.

since Google Cloud Storage is about 20$ per TB, storing 52TB of backup would cost about 1100$ which makes cloud storage services much expensive than having server replica

Yes. How did you not realize that earlier? ;) Anything cloud is quite expensivbe when used non stop. That said, AMazon Glacier may work out cheaper.... but nothing beats getting your hands on some local discs for backup.