LVM vs /dev/md0 performance problem

Question

I am trying to tune my NAS, running openfiler, and wondering why I'm getting relatively poor read performance from 4 WD RE3 drives in RAID 5.

When I run hdparm on the meta device I get the levels of performance I'd expect, drop to the volume and it's a third the speed !

Any one any idea why ? Is LVM that bad ?

[root@nas2 etc]# hdparm -t /dev/md0

/dev/md0:
 Timing buffered disk reads:  524 MB in  3.01 seconds = 174.04 MB/sec

[root@nas2 etc]# hdparm -t /dev/mapper/vg1-vol1

/dev/mapper/vg1-vol1:
 Timing buffered disk reads:  200 MB in  3.01 seconds =  66.43 MB/sec

Answer 1

No, LVM shouldn't perform that bad. Maybe your RAID striping doesn't mach LVM chunking? It's not enough to have LVM chunk size divided evenly by RAID chunk size. For starters you might want to look at this mailing list about aligning RAID and LVM.

And, as hdparm is not a performance tool, I might suggest any testing to be done with e.g. Bonnie++.