0
I'm testing the performance of ext3/ext4 on a raid1 system (but the same goes for a non raid system). The results from bonnie++ give a distorted image because they circumvent the delayed allocation on ext4. So i wanted to test the performance with dd.
dd bs=20M count=1024 if=/dev/zero of=/mountpoint/test conv=fdatasync
These are the results from a bunch of dd runs on ext4:
21474836480 bytes (21 GB) copied, 177.162 s, 121 MB/s
21474836480 bytes (21 GB) copied, 181.148 s, 119 MB/s
21474836480 bytes (21 GB) copied, 181.873 s, 118 MB/s
21474836480 bytes (21 GB) copied, 183.865 s, 117 MB/s
21474836480 bytes (21 GB) copied, 189.333 s, 113 MB/s
21474836480 bytes (21 GB) copied, 190.711 s, 113 MB/s
21474836480 bytes (21 GB) copied, 195.004 s, 110 MB/s
21474836480 bytes (21 GB) copied, 197.961 s, 108 MB/s
21474836480 bytes (21 GB) copied, 202.645 s, 106 MB/s
21474836480 bytes (21 GB) copied, 204.505 s, 105 MB/s
21474836480 bytes (21 GB) copied, 210.022 s, 102 MB/s
21474836480 bytes (21 GB) copied, 215.073 s, 99.8 MB/s
21474836480 bytes (21 GB) copied, 221.405 s, 97.0 MB/s
21474836480 bytes (21 GB) copied, 226.552 s, 94.8 MB/s
21474836480 bytes (21 GB) copied, 236.038 s, 91.0 MB/s
21474836480 bytes (21 GB) copied, 242.735 s, 88.5 MB/s
21474836480 bytes (21 GB) copied, 252.866 s, 84.9 MB/s
21474836480 bytes (21 GB) copied, 263.934 s, 81.4 MB/s
21474836480 bytes (21 GB) copied, 277.89 s, 77.3 MB/s
21474836480 bytes (21 GB) copied, 295.957 s, 72.6 MB/s
21474836480 bytes (21 GB) copied, 320.406 s, 67.0 MB/s
21474836480 bytes (21 GB) copied, 310.082 s, 69.3 MB/s
21474836480 bytes (21 GB) copied, 180.104 s, 119 MB/s
21474836480 bytes (21 GB) copied, 178.806 s, 120 MB/s
21474836480 bytes (21 GB) copied, 184.888 s, 116 MB/s
21474836480 bytes (21 GB) copied, 184.223 s, 117 MB/s
21474836480 bytes (21 GB) copied, 189.156 s, 114 MB/s
21474836480 bytes (21 GB) copied, 191.854 s, 112 MB/s
21474836480 bytes (21 GB) copied, 195.005 s, 110 MB/s
21474836480 bytes (21 GB) copied, 199.11 s, 108 MB/s
21474836480 bytes (21 GB) copied, 203.337 s, 106 MB/s
21474836480 bytes (21 GB) copied, 206.47 s, 104 MB/s
21474836480 bytes (21 GB) copied, 211.136 s, 102 MB/s
21474836480 bytes (21 GB) copied, 216.129 s, 99.4 MB/s
21474836480 bytes (21 GB) copied, 221.425 s, 97.0 MB/s
21474836480 bytes (21 GB) copied, 228.962 s, 93.8 MB/s
21474836480 bytes (21 GB) copied, 238.214 s, 90.1 MB/s
21474836480 bytes (21 GB) copied, 245.252 s, 87.6 MB/s
21474836480 bytes (21 GB) copied, 254.058 s, 84.5 MB/s
21474836480 bytes (21 GB) copied, 267.616 s, 80.2 MB/s
21474836480 bytes (21 GB) copied, 282.603 s, 76.0 MB/s
21474836480 bytes (21 GB) copied, 298.554 s, 71.9 MB/s
21474836480 bytes (21 GB) copied, 322.213 s, 66.6 MB/s
21474836480 bytes (21 GB) copied, 282.227 s, 76.1 MB/s
21474836480 bytes (21 GB) copied, 179.186 s, 120 MB/s
21474836480 bytes (21 GB) copied, 180.16 s, 119 MB/s
21474836480 bytes (21 GB) copied, 185.174 s, 116 MB/s
21474836480 bytes (21 GB) copied, 184.693 s, 116 MB/s
21474836480 bytes (21 GB) copied, 188.318 s, 114 MB/s
21474836480 bytes (21 GB) copied, 192.305 s, 112 MB/s
21474836480 bytes (21 GB) copied, 195.372 s, 110 MB/s
21474836480 bytes (21 GB) copied, 200.711 s, 107 MB/s
21474836480 bytes (21 GB) copied, 204.003 s, 105 MB/s
21474836480 bytes (21 GB) copied, 206.468 s, 104 MB/s
21474836480 bytes (21 GB) copied, 213.033 s, 101 MB/s
21474836480 bytes (21 GB) copied, 217.02 s, 99.0 MB/s
21474836480 bytes (21 GB) copied, 222.62 s, 96.5 MB/s
21474836480 bytes (21 GB) copied, 227.523 s, 94.4 MB/s
As you can see, the performance starts out as expected, around 120MB/s but drops to around 65MB/s, than jumps back to 120MB/s, and so on...
At the moment, I have no clue whatsoever about the cause of these weird results. The results from my ext3 test where as expected (all around 100MB/s). Would someone be able to enlighten me ?
Here are some more system specs:
mdadm controlled raid1
ext4 with default options (mkfs.ext4 /dev/sdb1)
mounted with default options (mount /dev/sdb1 /mountpoint)
disks: WD5003ABYX
os: custom debian kernel 2.6.36 x86
using oflag=direct/sync both also circumvent the delayed allocation on ext4, do they not ? And the tests are done on an idle system. I'm gonne check iowait in a second, as that could explain something. How can powersave influence this test ? The disks are constantly being written, so they can't go to power save. And the os isn't heavily stressed, so even if it would go to power save, it still should perform just fine. And lastly the jumps back to 120MB/s are still very strange. – Minion91 – 2013-03-14T14:43:42.393
direct
should enableO_DIRECT
and hence mballoc. Not 100% sure about delayed allocation, repeat the tests with and without the mount optionnodelalloc
to confirm. Powersave is something of a guess, there are different levels of powersave (possibly eco and low-noise modes). – mr.spuratic – 2013-03-14T15:20:58.500