3

I'm renting two dedicated servers from a hosting company. Here are the specs:

server1:
HP ProLiant DL165 G7
2x AMD Opteron 6164 HE 12-Core
40 GB RAM
HP Smart Array P410 RAID controller
2x Samsung 830 256 GB SSD

server2:
HP ProLiant DL120 G7
Intel Xeon E3-1270
16 GB RAM
HP Smart Array P410 RAID controller
2x Samsung 830 128 GB SSD

Setup is the same on both servers:

  • Debian 6.0.
  • No swap.
  • File systems use ext3 with no special mount options (only rw) and I'm quite certain the partitions are properly aligned.
  • Using noop scheduler.
  • RAID 1.
  • RAID controller has BBU.
  • Drive Write Cache has been enabled in the RAID controllers.
  • Read / Write cache ratio is 25% / 75% on both RAID controllers.

I'm currently trying to figure out how to get the most out of the disks in these servers starting with sequential reads/writes. Here are the speeds I'm seeing at the moment:

Writes:
server1:~# dd if=/dev/zero of=tempfile bs=1M count=1024 conv=fdatasync,notrunc
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 5.05089 s, 213 MB/s

server2:~# dd if=/dev/zero of=tempfile bs=1M count=1024 conv=fdatasync,notrunc
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 4.09768 s, 262 MB/s

Reads:
server1:~# echo 3 > /proc/sys/vm/drop_caches
server1:~# dd if=tempfile of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 4.14051 s, 259 MB/s

server2:~# echo 3 > /proc/sys/vm/drop_caches
server2:~# dd if=tempfile of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 3.33901 s, 322 MB/s

First of all, can anyone explain the big difference between these servers?

Second, should I expect more than this? When I've read about the Samsung 830 SSD I've seen write speeds of over 300 MB/s and read speeds of over 500 MB/s using the same benchmarking method (dd). But then there's no RAID controller involved. Is the penalty of RAID this high or is it a config issue?

UPDATE:

I've did some tests using iozone instead of dd and the results I'm getting make a lot more sense. No big difference between the two servers (server1 is slightly faster now) and I'm getting quite close to the speeds rated on these drives. So I guess I shouldn't have used dd. Lesson learned!

I'll be using noop with nr_requests and read_ahead_kb set at the defaults (128 and 128) to start with. Setting read_ahead_kb higher seems to hurt the random read performance too much on server2. Hopefully I'll get time to revisit this once I've used the servers in production for a while when I have a clearer picture of the usage patterns.

ntherning
  • 265
  • 2
  • 7
  • I'd test again with the [deadline scheduler](http://en.wikipedia.org/wiki/Deadline_scheduler). – ewwhite Sep 24 '12 at 17:01
  • 1
    It's not clear whether you intend to create the R1 mirror in hardware or software but you'd probably see a write penalty using software RAID, so avoid unless you have a good reason to. – Chopper3 Sep 24 '12 at 17:24

3 Answers3

7

There's a lot to cover here.

If you want more performance (in order from greatest to least impact):

  • Add another pair of disks and expand to RAID 1+0. This will provide the greatest benefit.
  • Tune your filesystem (noatime, journaling mode, remove write barriers, etc.) and/or move to a higher-performance filesystem like XFS or even ext4.
  • Go back to a deadline elevator. It will perform better under realistic workloads than your noop scheduler.
  • Upgrade the firmware of your HP Smart Array P410 controllers (and the server, as well)
  • Consider some more advanced tuning techniques.
  • Improve your benchmarking technique. dd is not an appropriate way to measure I/O performance. Try purpose-built applications like iozone, bonnie++, etc. and tweak them to your desired read/write patterns.
  • For purely sequential reads/writes, regular SAS drives aren't too bad a choice either...

As far as compatibility, I use non-HP disks with HP RAID controllers and servers often. Sometimes, things don't work, but if your SSD is connected, reports proper temperature and doesn't display any errors in the HP Array Configuration Utility, you're fine.

You are using HP Management Agents on the server, aren't you?

Edit:

I ran the same on one of my systems, featuring the same controller with four SATA SSD's, tuned XFS, deadline elevator, etc.

[root@Kitteh /data/tmp]# dd if=/dev/zero of=tempfile bs=1M count=1024 conv=fdatasync,notrunc
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 3.1985 s, 336 MB/s

[root@Kitteh /data/tmp]# echo 3 > /proc/sys/vm/drop_caches
[root@Kitteh /data/tmp]# ll
total 1048576
-rw-r--r-- 1 root root 1073741824 Sep 24 14:01 tempfile

[root@Kitteh /data/tmp]# dd if=tempfile of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 1.60432 s, 669 MB/s
Community
  • 1
ewwhite
  • 194,921
  • 91
  • 434
  • 799
  • Just upgraded the firmware on the RAID controller but there's no change. Then I tried the tuning techniques you linked to and managed to get the read speeds up to over 500 MB/s! There's still around 50 MB/s difference between the servers which I have no explanation for but I'll save that for another day. Thanks! – ntherning Sep 24 '12 at 19:24
  • Excellent. Which specific tunables did you modify? – ewwhite Sep 24 '12 at 23:45
  • I'm playing with nr_requests and read_ahead_kb. I've also found that with iozone instead of dd I get quite different numbers and the differences between the servers seem to vanish. – ntherning Sep 25 '12 at 08:08
  • 2
    @ntherning that's because dd isn't a benchmark tool, plus that PURE sequential reads/writes isn't something you'll find in the real world.. there's always some random I/O. I'd stay with iozone if I were you. – pauska Sep 25 '12 at 15:35
0

Using non-HP drives with HP SmartArray controllers, you should not expect any particular performance or that it will even work. HP has made it abundantly clear that if their SA controllers happen to work with non-HP drives it's completely unintentional and unsupported. If you want good performance from those drives, get a HBA that supports them (LSI and Adaptec, who actually make HP's SA controllers, tend to be excellent choices).

Chris S
  • 77,337
  • 11
  • 120
  • 212
0

First, Adaptec do not manufacture Smartarray, but in some cases HP has used SCSI chipset from adaptec in their Smararrays.

Second, read this, contain importat info about smartarray tuning with SSD.

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&taskId=120&prodSeriesId=3802118&prodTypeId=329290&objectID=c02963968

Manfri
  • 1