12

As a test of the Opteron processor family, I bought a HP DL385 G7 6128 with HP Smart Array P410i Controller - no memory.

The machine has 20GB ram
2x146GB 15k rpm SAS + 2x250GB SATA2, both in Raid 1 configurations.
I run Vmware ESXi 4.1.

Problem:
Even with one virtual machine only, tried Linux 2.6/Windows server 2008/Windows 7, the VMs' feel really sluggish. With windows 7, the vmware converter installation even timed out. Tried both SATA and SAS disks and SATA disks are nearly unsusable, while SAS disks feels extremely slow.

I can't see a lot of disk activity in the infrastructure client, but I haven't been looking for causes or even tried diagnostics because I have a feeling that it's either because of the cheap raid controller - or simply because of the lack of memory for it.

Despite the problems, I continued and installed a virtual machine that serves a key function, so it's not easy to take it down and run diagnostics.
Would very much like to know what you guys have to say of it, is it more likely to be a problem with the controller/disks or is it low performance because of budget components?

Thanks in advance,

ewwhite
  • 194,921
  • 91
  • 434
  • 799
3molo
  • 4,340
  • 5
  • 30
  • 46
  • Inside of a virtual machine on the SAS raid, I get roughly 60 MB/s write using 'dd if=/dev/zero of=somefile bs=1M count=1024'. I guess that's what is to be expected, so perhaps all tests with extremely low performance really only regards the SATA disks. The sata disks are 5.4k rpm, but I still think that one (or a raid 1) 5.4k rpm sata disk should be reasonably OK for a windows 7 guest. – 3molo Jan 12 '11 at 13:00
  • So I want to rephrase the question; Would I win anything in either buying a better controller, or buying memory for my P410i? (Either way, I intend to get 4 more SAS disks) – 3molo Jan 12 '11 at 13:01

4 Answers4

26

The HP Smart Array P410 is a fine controller, but you will get poor performance out of it if you don't have the battery-backed or flash-backed cache units installed. The cache makes a tremendous difference in that writes are buffered by the cache memory before being committed to disk. You get the write confirmation to the application without having to incur the latency of the physical disk drives.

Here's a 4GB dd on a similarly-spec'd system (DL380 G7 with 24GB RAM and a p410 with 2 x SAS disks and 1GB Flash-Backed Write Cache). The RAM helps a lot in a test like this, but you get the idea...

[root@xxxx /]# dd if=/dev/zero of=somefile bs=1M count=4096
4096+0 records in
4096+0 records out
4294967296 bytes (4.3 GB) copied, 3.70558 seconds, 1.2 GB/s

But realistically, your write performance with two SAS drives in a RAID 1 on that controller with the appropriate cache should be between a sustained 130-170 megabytes/second. A quick iozone test on the above server configuration shows:

[root@xxxx /]# iozone -t1 -i0 -i1 -r1m -s16g
Write
Avg throughput per process                      =  166499.47 KB/sec
Rewrite:
Avg throughput per process                      =  177147.75 KB/sec

Since you're using ESXi, you can't run online firmware updates. You should download the Current Smart Update Firmware DVD, burn it to disk and make sure your system is patched to a relatively recent level.

Here are the controller's quickspecs: http://h18004.www1.hp.com/products/quickspecs/13201_na/13201_na.html

You will want to purchase one of the following, ranging from $350-$600 US:
512MB BBWC
512MB Flash Backed Write Cache
1G Flash Backed Write Cache

To answer your question, the cache solution will help the most. Additional disks won't make much of a difference until you handle the caching situation.

*Note for other users. If you have cache memory on recent HP controllers with up-to-date firmware, there is a write cache override available if you have RAM on the controller but no battery unit. It's slightly risky, but can be an intermediate step in testing what performance would be like on the way to buying a battery or flash unit.

ewwhite
  • 194,921
  • 91
  • 434
  • 799
  • 2
    +1, having a fast system like this without write cache is stupidity. Buy controller cache and battery! – pauska Jan 12 '11 at 13:28
  • Yes yes yes yes yes yes yes. +1 a lot. – mfinni Jan 12 '11 at 14:00
  • The more cache the happier you'll be. The 512 BBWC module is the best bang for your buck. – ErnieTheGeek Jan 12 '11 at 14:23
  • More is better, but any write-caching at all is supremely better than none, for highly interactive (citrix/ts) or VM systems. – mfinni Jan 12 '11 at 14:29
  • 3
    I don't understand why HP even sells the p410 or certain server configs without any cache. A LOT of people run into this issue. – ewwhite Jan 12 '11 at 14:38
  • @ewwhite My sentiments exactly. It's *so* easy to miss when configuring up a new server. We got a pair of DL5xx w/o realizing they changed things and forgot to get cache for those RAID cards. Educational. – sysadmin1138 Jan 12 '11 at 15:20
  • 4
    @ewwhite: your dd command may not be representative of your disk throughput, as it is very likely being cached at the vmfs layer. You should append "oflag=direct" to your dd command to make it use O_DIRECT when opening the file, which will bypass the vmfs cache. 1.2GB/s is far too high for a two-disk SAS array :) – Daniel Lawson Jan 12 '11 at 17:47
  • Right. That's why I posted the more realistic iozone results. But the dd flag does come in handy elsewhere. – ewwhite Jan 12 '11 at 18:12
3

Check controller firmware version and update if possible.

I've experienced similar issues with Smart Array 410i rebuilding array very slowly after disk failure (over 2 days for hardly idle 2 SAS drives). In my case te problem was in outdated firmware. From my experience, the firmware is surprisingly often the answer for any problems with HP storage components, so I'd recommend checking it first and if it doesn't work - open support case in HP (this is what they are for ;).

pitr
  • 567
  • 3
  • 6
0

I would run more drives that are striped somehow. The write cache will make a significant difference. A couple mirrored SAS drives isn't fast enough for any of our stuff. Some of the (much) pricier stuff can actually cache all your VMs in memory.

JOTN
  • 1,727
  • 1
  • 10
  • 12
0

As other says, for local or remote/shared storage operations with VMWare virtualization there is almost a requirement to have raid controller or array with BBWC cache (battery backup cache), other way storage will perform extremely and unusable slow with SAS/SATA spindle disks, only with SSD you can eventually use VMWare ESXi 3/4/5 without write/read cache and may have not so bad performance (also I recommend write cache for SSD too).

slm
  • 7,355
  • 16
  • 54
  • 72
NTShad0w
  • 1