RAID 6 array much faster than my RAID 10 array?

Question

How come my RAID-6 outperforms my RAID-10?

Docker container on Synology with 8x4TB RAID-10:

[root@centos Files]# bonnie++ -d /Files/tmp -r 16G -s 32G -n 1 -f -b -u root
Using uid:0, gid:0.
Writing intelligently...done
Rewriting...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.97       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
centos          32G           454106  19 177752  12           336051  11 174.9   9
Latency                         637ms     437ms               271ms    1199ms
Version  1.97       ------Sequential Create------ --------Random Create--------
centos              -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                  1    19   1 +++++ +++    24   2    19   1 +++++ +++    23   3
Latency               401ms      63us     177ms     177ms      19us     122ms
1.97,1.97,centos,1,1503010478,32G,,,,454106,19,177752,12,,,336051,11,174.9,9,1,,,,,19,1,+++++,+++,24,2,19,1,+++++,+++,23,3,,637ms,437ms,,271ms,1199ms,401ms,63us,177ms,177ms,19us,122ms

FreeNAS 6x4TB RAID-Z2, RAID-6 from a jail:

root@testjail:/ # bonnie++ -d /Files/tmp -r 16G -s 32G -n 1 -f -b -u root
Using uid:0, gid:0.
Writing intelligently...done
Rewriting...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.97       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
testjail        32G           1158187  63 850172  70           3130981  97  7888  90
Latency                         173ms   79669us             13285us     197ms
Version  1.97       ------Sequential Create------ --------Random Create--------
testjail            -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                  1 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency             19728us      41us    2506us     102us      20us     154us
1.97,1.97,testjail,1,1503021586,32G,,,,1158187,63,850172,70,,,3130981,97,7888,90,1,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,,173ms,79669us,,13285us,197ms,19728us,41us,2506us,102us,20us,154us
root@testjail:/ #

Specs: 16GB DDR4 (2133 vs 2400)

Synology RS3617RPxs - Intel Xeon D-1521 - 8 x WDD Red 4TB in Raid 10 - Quad Core 2,2 (base) / 2,7 (turbo) GHz - 2 x 8 GB DDR4 ECC UDIMM
FreeNAS VMware VM - 1 x 16GB allocated RAM (out of 2x16GB) - 16GB NVMe bootdisk only - 6x4TB desktop drives - Was difficult to get working.

My expectations were ~400MB/sec write on a good day for the RAIDZ2 as some drives are really old and ~750MB/sec for the RAID10 array.

But the raidz2 seems impossibly fast and the raid10 seems very slow.

I would recommend using deskspeed.exe to be sure that you are confident about numbers which are you sharing with us. — Net Runner, Aug 21 '17 at 11:09
Clarification needed: Is the synology benchmark across the network (what speeds and topology if so) and the FreeNAS bench local to VM? — Jeff Burns, Aug 21 '17 at 12:38

score 6 · Accepted Answer · answered Aug 21 '17 at 11:16

6

I would recommend using deskspeed.exe to be confident about numbers which you displaying to the community.

This article should help you at tunning Synology box

https://www.starwindsoftware.com/blog/synology-diskstation-ds916-turbocharged-with-starwind-virtual-san

answered Aug 21 '17 at 11:16

Net Runner

5,626
11
29

I find this a poor recommendation. Bonnie++ is a good cross-platform tool and we should benchmark with non-OS specific tools so that we can be as platform agnostic and run as close to bare-metal as possible. Else we are skewing numbers with all kinds of layers. – pbrunnen Apr 23 '21 at 18:25

RAID 6 array much faster than my RAID 10 array?

1 Answers1