4

I am looking to buy HDDs for a single server which will be used as development server. The server will use ESXi as hypervisor and will have some but not too many virtual machines (web server, database server, file server)

We have already taken a Samsung 950 Pro 256 GB for the virtual machines but I struggle in finding an answer to the question:

In a RAID 10: Are lots of slow disks (5400/5900 rpm) faster than fewer fast disks (7200 rpm)? For example 8x 5400rpm vs 4x 7200rpm

Would the 8 slow be a better choice compared to the 4 fast?

My choice was between 1TB WD Red which run at 5400rpm vs 2TB WD Se 7200rpm.

Noctalin
  • 49
  • 1
  • 5
  • 5
    Given the tremendous price differences and the fact your developers work for free, I understand your reluctance to go all SSD. Except your developres do not work for free and a 1tb SSD does not cost a lot. Your setup has low performance and high maintenance. Go with SSD. – TomTom Mar 31 '16 at 09:01
  • Thank you for your idea TomTom, I forgot to say that the company is not very big (12 persons) so I wasn't thinking of going straight to SSD for this scenario. It is supposed to hold backups here in our office of our main servers located at a provider, therefore I thought of going with lots of space and some redundancy with RAID – Noctalin Mar 31 '16 at 09:28
  • Especialyl dev servers are often hit with heavy IO operations. Saving there is ismply wasted. My dev server is all SSD and I experience bottlenecks at times. – TomTom Mar 31 '16 at 09:29
  • 2
    @TomTom I completely agree, but would note that it's axiomatic that devs (however many there are) will always exhaust the server (however big it is). *Especially* if there are databases involved. – MadHatter Mar 31 '16 at 09:37
  • We will use a database as well but the idea was to have a VM take care of backups of our provider server and of the other VM running on the server. There is not much redundancy except I put SSD in RAID10 if that even makes sense. – Noctalin Mar 31 '16 at 09:48
  • @TomTom Don't you need another performance profiling server with HDDs if your DB dev server is all SSD and you plan to support HDDs? – Eugene Ryabtsev Mar 31 '16 at 12:17
  • 3
    Generally no. My production server is all SSD too - there is nothing cheaper than SSD for a database server because unless you love dead data you ARE limited by IOPS. – TomTom Mar 31 '16 at 12:21
  • @TomTom How do you keep your data safe from drive failures? Only through backups? – Noctalin Mar 31 '16 at 15:42
  • 2
    @Noctalin [Are SSD drives as reliable as mechanical drives (2013)?](http://serverfault.com/q/507521/58408) (And yes, you *have* to keep backups. Unless of course you want those 12 people to stop working for several days and lose Who Knows How Much time getting back to where they were.) – user Mar 31 '16 at 15:43
  • @Noctalin Depends. Tempdb: Raid 10 (of SSD). Storage: Multiple Raid 5 of 3 SSD at the moment. Logging: Raid 1 of SSD. Backup every 5 minutes. – TomTom Mar 31 '16 at 15:43
  • 3
    Since when do 7200 rpm spinners count as "fast" in a modern multi-user environment? – user Mar 31 '16 at 15:44
  • @TomTom This setup is quite interesting but we try to keep it low with the costs. EDIT: How about using main SSD for the OS of the VMs, have 1 or 2 SSD for storage (RAID?) and a Synology with 4 HDD for backups? – Noctalin Apr 05 '16 at 09:53
  • Keeping COsts low is not smart the moment you do not get the performance out that you need/want. Now, HDD for backup - doable. I back up to 2TB SSD (largest I can get with 2.5") and performance there is acceptable (but then I have 16 of them in 2 Raid 6 at the moment). – TomTom Apr 05 '16 at 10:40

4 Answers4

10

This is hard to answer directly, because the answer is very much 'it depends on your caching'.

You see - hard disks are slow. Very slow. But actually, most of the time you don't notice, if you have a decent cache, and a reasonably predicable access pattern.

Read caching means recently accessed things will be coming off cache. Write caching means that recently written things can be deferred and coalesced into full stripe writes.

But when it comes to measuring speed - there's two elements to it. Write performance ... actually probably doesn't matter too much, because of the caching - you can always cache writes, and so as long as your sustained write IO doesn't exceed that of your combined RAID group, it's irrelevant.

But read IO it's much more of a problem - you cannot service a read until you've got your data, and to get your data you have to wait for the disk if it's not already in cache. This is where you see the performance cost - it shows up in read latency.

And that's why slower disks are a problem - because it doesn't actually matter how fast the combined RAID group is, when the data is at rest on one of two drives - you have to access that drive to service the read.

So - at risk of sounding a bit wishy washy - "it depends" a lot on your workload. You can usually get by with some really slow disks and some awesome caching if you want really large amounts of terabytes cheap.

However, I'd only bother doing so when we start talking at 100+TB or so, which is where the cost of rack space, cooling, floor footprint etc. start to become significant.

For your application - I'd say buy the fastest drives you can afford, because it's a lot easier to buy more drives later if you need them, than to realise you can't because you've filled your drive bays and need a new shelf.

But I'd strongly suggest also considering SSDs - their price-per-gig isn't amazing, but their price-per-IOP really is. And whilst no one seems to care about IOPs, that's the thing they really do care about when it comes to 'being a bit slow'.

Sobrique
  • 3,697
  • 2
  • 14
  • 34
  • SSDs would do the job but we are going to create a VM to backup our remote servers and also our local data on our computers. That is why I opted for HDD RAID. – Noctalin Mar 31 '16 at 10:02
  • 1
    You're creating a VM to act as a backup server? Why? Why not just run a backup server on the physical tin? If performance _isn't_ a consideration, then your whole question is moot - just buy the cheapest per TB, and don't even bother with RAID 10 in the first place. – Sobrique Mar 31 '16 at 10:05
  • I will actually do it that way and get a separate machine for backups. A QNAP/Synology with Mirror should do the work? Use the 950 Pro for the VM and an additional big SSD as storage. – Noctalin Mar 31 '16 at 10:11
  • +1 for hard disks are slow. Another mental +1 for the rest because I can't vote up twice. – user Mar 31 '16 at 15:46
  • 1
    If you are looking at storage solutions in the multiple TB and up range, you also have to consider how much of that is your *active* data set. My guess is that many such workloads are more like write-and-archive than they are read-heavy *across the whole data set*. When you've got large bulk storage of data that is mostly at rest, and a smaller working set that is accessed a lot, you can look into things like having a large but slow set of spinners fronted by some SSDs. ZFS' L2ARC makes that easy to maintain (but needs time to prime the cache, so uptime), but there are other options as well. – user Mar 31 '16 at 20:20
5

Based on calculations from RAID Performance Calculator, eight 5400rpm disks will yield about 304 IO/s. Whereas four 7200rpm disks will yield about 210 IO/s.

Frank Perez
  • 193
  • 4
  • 4
    The irony is - that's about the same as a single 15K SAS drive, and no where even close to even a single SSD. – Sobrique Mar 31 '16 at 09:28
  • 1
    @Sobrique, why is that ironic? – Ash Burlaczenko Mar 31 '16 at 12:19
  • 1
    @AshBurlaczenko: Because the poinf of using HDD's over SSD's is to save money. But here, the HDD solution has less IOPS for more $. – MSalters Mar 31 '16 at 15:33
  • 1
    @MSalters, I can't see any pricing on this page so how can you say its more. Where can I buy 8TB of SSD storage for less than than 8TB of HDD? – Ash Burlaczenko Mar 31 '16 at 15:44
  • @AshBurlaczenko: Don't compare capacity. The constraint here was IOPS, see TomTom's comment on the question and Sobrique's answer. This answer would suggest 8 HHD's for 304 IOPS. A **single** Samsung 950 Pro will do in excess of 30.000 IOPS. – MSalters Mar 31 '16 at 15:51
  • 5
    When it comes to enterprise storage: capacity is cheap. Gigs at rest cost negligible amounts. What's important - and expensive - is how fast you can move it around. But too many people argue about whether a bus or a truck is better, when they should be thinking about whether they actually need a motorbike. – Sobrique Mar 31 '16 at 17:16
  • An interesting fact about the numbers you provided is that (210/304) IO/s yields nearly same ratio as ((7200 rpm * 4 hdd) / (5400 rpm * 8 hdd)) – MonkeyZeus Mar 31 '16 at 20:31
1

The main difference between your two choices is 3 year warranty vs 5 years and SOHO usage area vs. certification for datacenters. In terms of raw I/O performance, there is not much of a difference between 5400/5900 and 7200 rpm S-ATA disks nowadays (that was different a few years ago).

In the end therefore the array with more disks will very likely show a superior performance, as RAID10 scales with the number of disks.

What I would guess is more important in this setup, then the number of drives is, get a proper hardware RAID controller with BBU+Cache if you can afford it - or even better go with SSD only from the start on.

Also please be aware, that ESXi natively only supports hardware RAID! While there are workarounds to use software RAID with ESXi, I personally wouldn't advice using these, if you are looking for a decent performance.

Protip: If you are on a really tight budget to self-build this storage server and we are talking about a non mission-critical system, then go fetch yourself an OEM version of a proper RAID controller (IBM M5015, DELL PERC H710, etc.) on ebay.

s1lv3r
  • 1,155
  • 1
  • 14
  • 24
  • Well we code our sites and mobile apps currently on remote server and would rather have the development happen here on a local server. It is not supposed to be accessible from outside the office. So a hardware RAID with HDD is what you recommend? – Noctalin Mar 31 '16 at 09:59
  • 3
    The problem is, that if you want to run ESXi you need a hardware RAID controller anyway, as ESXi doesn't support software RAID (also no motherboard onboard "fake" RAID). Besides that a proper hardware RAID controller will influence the performance of the array far more then the choice between 5400 vs. 7200 rpm disks. – s1lv3r Mar 31 '16 at 10:03
  • At home I use FreeNAS as VM to create a ZFS RAID and then pass these to ESXi to manage. Is that not a good solution to use? – Noctalin Mar 31 '16 at 10:05
  • 1
    It certainly works, but of course you have a lot of overhead (iSCSI stack etc.) and a more complex system setup. Personally I wouldn't recommend it, if you are looking for performance. – s1lv3r Mar 31 '16 at 10:11
  • 1
    Does it really matter whether ESXi supports SW RAID? It just needs to expose the disks to the guest VMs, and _they_ can manage their software RAIDs. Sure, not as CPU-efficient as HW RAID, but the problem for HW RAID is that CPU performance is cheap. – MSalters Mar 31 '16 at 15:56
  • @MSalters It's clearly possible. The downside is that you can only passthrough complete disks using raw device mapping (which severely limits your flexibility as you need atleast two disks dedicated *per* VM), or you pass all the disks to a single storage VM and then provide this storage as a datastore back to ESXi via iSCSI (which has the limitation of a lot of overhead & added complexity). Personally, due to this limitations, I wouldn't consider one of this solutions, if I were to setup anything but a proof-of-concept system. YMMV. :) – s1lv3r Mar 31 '16 at 19:16
  • @s1lv3r So how about a software RAID through motherboard, create and use the VMDKs for the different VMs. This solution wouldn't create a big overhead and a complex system. Afterwards setup a Storage VM as NAS for the people in the company to store their data which should be accessible through our network. – Noctalin Apr 05 '16 at 09:52
  • @Noctalin That's what I meant in my original post with `ESXi natively only supports hardware RAID` - if you create a RAID through your motherboard, ESXi simply won't see it/ won't be able to use it. It lacks the drivers. Generally almost all onboard S-ATA controllers are "unsupported"/ not on the ESXi HCL. They just happen to fortunately work through the generic ahci driver - but only in AHCI and not in RAID mode. – s1lv3r Apr 05 '16 at 10:11
  • @s1lv3r This limits me then to using a storage VM with lots of Overhead (Would have to see if this is even such a big problem in this environment), not using RAID with SSDs, only use one for OSes and one for storage, the HDDs in a separate NAS for backups, or get hardware RAID which would be an overkill for this situation imo. – Noctalin Apr 05 '16 at 10:20
  • 1
    @Noctalin Yeah, looks like those are your options. Maybe you also want to consider to switch to Hyper-V or Xen. They both offer similiar features while supporting soft RAID. ESXi always sucks for hardware compatility anyway (I think they want to enforce use of certified hardware only) ... – s1lv3r Apr 05 '16 at 10:50
  • @s1lv3r This is actually very good to know, I had the impression VMware had the best hardware compatibility, so much for that then. Edit: XenServer is this reliable? – Noctalin Apr 05 '16 at 11:03
  • @Noctalin while it's OpenSource, commercial support is provided by Citrix (which is a billion dollar company) and it runs some of the biggest clusters in the world. So. Yeah. Pretty reliable according to my standards. Choosing between the big three (ESXi, XenServer, Hyper-V) is a matter of personal choice I would say. ;) – s1lv3r Apr 05 '16 at 12:37
0

8x 5400 RPM disks will generally outperform 4x 7200 RPM disks, save for single-threaded sequential read/write.

As your use case (virtual machines) is rich of small random reads/writes, you should prefer the solution with more spindles (ie: the more the disks, the better).

Anyway, considered how cheap are 7200 RPM disks, I strongly suggest to use 8x such disks ;)

shodanshok
  • 44,038
  • 6
  • 98
  • 162