Raid Controller Write-Back Cache Behavior

Question

I am trying to understand how it is that the write-back cache on a raid controller can continue to yield a benefit while you're writing a huge random dataset. Let me first say that I understand how a write-back cache works: the OS gets an I/O completion on a write when the data hits the controller cache rather than the slower underlying medium, and the controller then writes the data to the medium as fast as it can be written. So with that being the case, if you're very quickly writing data faster than the backing medium can write it, I would expect to blow the controller cache and the performance gain of having a write-back to go away, and you'd end up with performance equivalent to a write-through setup?

What I described above is what I'd expect, but it's not what I'm seeing. The write-back cache backed server consistently has at least 4x higher IOPS and throughput, and sustains it; as compared to an identical server that I put into write-through mode. Any ideas on how to explain this behavior?

And yes I am writing way more than enough data to saturate the filesystem cache, and I'm writing it very quickly.

Answer 1

Write-back cache allows the OS to better organize the data before writing, thus the performance gain. If you do a lot of relatively small writes, then simply putting the data from several consecutive writes together and writing it to the media all at once will improve performance significantly.

To a lesser extent, write-back cache may help the OS to match the cluster size during writes. For example, if you write 10000 bytes at a time, and the cluster size is 8192 bytes, caching will prevent the OS from writing 1808 bytes a partially filled cluster (which would require reading the cluster first).