RAID10 without write-back cache = horrible write performance?

Question

I have just provisioned a dedicated server on singlehop.

I'm running it through some tests to know what to expect performance-wise. On the I/O side (with 4 1TB disks in RAID 10) I get:

write-cache disabled
200 MB/s read throughput
30  MB/s write throughput

I thought that was really low compared to my desktop HD which gets 150-150 or so. So I had a chat with them and they suggested enabling the write cache. New results:

write-cache enabled
280 MB/s read
260 MB/s write

which is great and all but means I'd have to add a BBU for an additional monthly cost.

Is it normal for the write throughput to be 1/4 of a regular drive on RAID10, if you don't have write cache? It almost feels like its intentionally bad to force you to pony up for the BBU. I'd be happy with normal non-raid performance of 150/150.

UPDATE: They're have a look at it now and seeing whether there is something wrong. I'm gonna give ahamat the accepted answer as he broke down when this 8x drop-off would affect the server workload and when it would not. Will update again if I get any more data. +1 for other answers. Thanks.

UPDATE2: There was something wrong with the hardware it seems. Moved to a new machine with identical specs and getting 80MB/s writes without write-back cache. 250MB/s with the cache on. so 3x drop-off and reasonable throughput without it.

Please take a moment to fix your units. I guarantee that they're not measured in `mb`, they're either `Mb` or `MB` and they are different. — MDMarra, Oct 26 '12 at 15:39
Using this tool: http://thesz.diecru.eu/content/parkdale.php — Jamona Mican, Oct 26 '12 at 15:49
I'm going to assume that your 150MB/s write speed figure on your desktop is using write-back as well, just with no battery to back it. — MDMarra, Oct 26 '12 at 15:54
Could be MDMarra. Still a HD should be able to get way more than 30MB/s write, no? Which means RAID10 without write-back cache has worse write performance than a single drive without it? — Jamona Mican, Oct 26 '12 at 15:59
Keep in mind that your 100-150 MBps rate for the desktop setup is with the Drive's Cache enabled. The RAID HBA will flush the member drives' cache on every write when write caching is disabled. You're comparing apples and oranges unfortunately. — Chris S, Oct 26 '12 at 17:27

score 2 · Answer 1 · edited Apr 13 '17 at 12:14

2

Yeah, you want a battery-backed cache unit. Write speed is typically poor without it. If your application requires that type of performance, you'll need to pay...

edited Apr 13 '17 at 12:14

Community

1

answered Oct 26 '12 at 15:51

ewwhite

194,921
91
434
799

So you think the drop-off is typical? 8x with vs without? – Jamona Mican Oct 26 '12 at 15:58
It's a big drop-off, but this isn't necessarily an enterprise setup either. Are the drives SATA? – ewwhite Oct 26 '12 at 16:10
Yep Sata... they're gonna run a check against a similar set-up. thanks for the link. I just only noticed it. will +1 when I have enough points – Jamona Mican Oct 26 '12 at 18:14

score 1 · Accepted Answer · answered Oct 26 '12 at 17:23

The performance in real world applications will vary, due to the nature of the application.

Asynchronous writes will go to RAM, while you have any available for write buffering. Obviously writing to RAM will be significantly faster than to disk. This is the default for most (all?) modern operating systems. If you have enough RAM to writes until they are flushed to disk then all writes will appear extremely fast. Although, there is a time period where loss of power results in data loss. Battery backed on-disk write buffering reduces (but doesn't completely eliminate) this period.

Synchronous writes must be committed to disk before the write can return and is significantly slower. This is the default mode for NFS and some other applications. For synchronous writes battery backed on-disk write buffering significantly increases apparent write performance and eliminates the risk of data loss due to power failures. Take note though, that the writes are still going to volatile memory, it's just been moved to disk PCB's memory rather than main memory. ZFS has solved this a different way by use of an SSD ZIL which will commit to the SSD and return the write operation, then later move it to spinning disks.

So you really need to look at your application. Are the majority of your writes synchronous or asynchronous? For asynchronous you can get away with just having gobs of RAM. For synchronous you'll need the battery backed write cache (but ZFS may be able to provide a cheaper solution).

In any case, you need write caching.

Thanks for the breakdown. The workload would be coming from MongoDB. I'll have a look into whether it uses asynchronous writes or not. — Jamona Mican, Oct 26 '12 at 18:13
This is a hosted system. The discussion about ZFS, ZIL seems to be off-topic, as we're talking about a single managed system. — ewwhite, Oct 26 '12 at 18:41

score 0 · Answer 3 · answered Oct 26 '12 at 17:22

On a RAID10 scenario it is really slow even without write-back cache.
Which tool do you try to use to measure? I recommend Phoronix for this.
On RAID10 1TB SATA disks a sequential write should be at least 80-100MB/sec, and in random it should not drop below 50 (of course it depends on the concurrency too)

RAID10 without write-back cache = horrible write performance?

3 Answers3

Linked