1
I have a ProLiant DL380 Gen9 server with a HP Smart Array P440ar controller and HP MO0400JEFPA 400GB SAS SSD drives.
The server is meant to host a write intensive database so I'm particularly interested in the write speed.
We have 4 arrays: 3 raid-1 arrays with 2 drives each and a raid-10 array with 16 drives that will be used to store the data.
I've been measuring the write speed by using dd:
time sh -c "dd if=/dev/zero of=test.tmp bs=8k count=200000000 conv=fdatasync"
On raid-1 arrays I get around 600MB/s ( close to the theoretical 700MBPS of the drive ). But on the raid-10 array I only get 1.5GB/s. I understand that with a 16 drive raid-10 array I should theoretically be getting 8x write speed. And I'm getting just 3x. Should I expect more from it? Or is real performance this far away from theory? (on read speed I also get a 3x instead of the theoretical 16x).
This is the configuration of the array I get from hpssacli:
Smart Array P440ar in Slot 0 (Embedded)
Bus Interface: PCI
Slot: 0
Serial Number: xxxxxxxxxxxx
Cache Serial Number: xxxxxxxxxxxx
RAID 6 (ADG) Status: Enabled
Controller Status: OK
Hardware Revision: B
Firmware Version: 4.52
Rebuild Priority: High
Expand Priority: Medium
Surface Scan Delay: 3 secs
Surface Scan Mode: Idle
Parallel Surface Scan Supported: Yes
Current Parallel Surface Scan Count: 1
Max Parallel Surface Scan Count: 16
Queue Depth: Automatic
Monitor and Performance Delay: 60 min
Elevator Sort: Enabled
Degraded Performance Optimization: Disabled
Inconsistency Repair Policy: Disabled
Wait for Cache Room: Disabled
Surface Analysis Inconsistency Notification: Disabled
Post Prompt Timeout: 15 secs
Cache Board Present: True
Cache Status: OK
Cache Ratio: 0% Read / 100% Write
Drive Write Cache: Enabled
Total Cache Size: 2.0 GB
Total Cache Memory Available: 1.8 GB
No-Battery Write Cache: Disabled
SSD Caching RAID5 WriteBack Enabled: True
SSD Caching Version: 2
Cache Backup Power Source: Batteries
Battery/Capacitor Count: 1
Battery/Capacitor Status: OK
SATA NCQ Supported: True
Spare Activation Mode: Activate on physical drive failure (default)
Controller Temperature (C): 55
Cache Module Temperature (C): 45
Number of Ports: 2 Internal only
Encryption: Disabled
Express Local Encryption: False
Driver Name: hpsa
Driver Version: 3.4.14
Driver Supports HPE SSD Smart Path: True
PCI Address (Domain:Bus:Device.Function): 0000:03:00.0
Negotiated PCIe Data Rate: PCIe 3.0 x8 (7880 MB/s)
Controller Mode: RAID
Controller Mode Reboot: Not Required
Latency Scheduler Setting: Disabled
Current Power Mode: MaxPerformance
Host Serial Number: xxxxxxxxxxx
Sanitize Erase Supported: True
Primary Boot Volume: None
Secondary Boot Volume: None
And for the raid-10 array I get:
Array: D
Interface Type: Solid State SAS
Unused Space: 0 MB (0.0%)
Used Space: 5.8 TB (100.0%)
Status: OK
MultiDomain Status: OK
Array Type: Data Spare Type: dedicated
HPE SSD Smart Path: disable
Logical Drive: 4
Size: 2.9 TB
Fault Tolerance: 1+0
Heads: 255
Sectors Per Track: 32
Cylinders: 65535
Strip Size: 256 KB
Full Stripe Size: 2048 KB
Status: OK
MultiDomain Status: OK
Caching: Disabled
Unique Identifier: xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Disk Name: /dev/sdd
Mount Points: /data 2.9 TB Partition Number 2
OS Status: LOCKED
Logical Drive Label: xxxxxxxxxxxxxxxxxxxxxxxxxxx
Mirror Group 1:
physicaldrive 1I:1:7 (port 1I:box 1:bay 7, Solid State SAS, 400 GB, OK)
physicaldrive 1I:1:8 (port 1I:box 1:bay 8, Solid State SAS, 400 GB, OK)
physicaldrive 1I:1:9 (port 1I:box 1:bay 9, Solid State SAS, 400 GB, OK)
physicaldrive 1I:1:10 (port 1I:box 1:bay 10, Solid State SAS, 400 GB, OK)
physicaldrive 1I:1:11 (port 1I:box 1:bay 11, Solid State SAS, 400 GB, OK)
physicaldrive 1I:1:12 (port 1I:box 1:bay 12, Solid State SAS, 400 GB, OK)
physicaldrive 1I:1:13 (port 1I:box 1:bay 13, Solid State SAS, 400 GB, OK)
physicaldrive 1I:1:14 (port 1I:box 1:bay 14, Solid State SAS, 400 GB, OK)
Mirror Group 2:
physicaldrive 1I:1:15 (port 1I:box 1:bay 15, Solid State SAS, 400 GB, OK)
physicaldrive 1I:1:16 (port 1I:box 1:bay 16, Solid State SAS, 400 GB, OK)
physicaldrive 1I:1:17 (port 1I:box 1:bay 17, Solid State SAS, 400 GB, OK)
physicaldrive 1I:1:18 (port 1I:box 1:bay 18, Solid State SAS, 400 GB, OK)
physicaldrive 1I:1:19 (port 1I:box 1:bay 19, Solid State SAS, 400 GB, OK)
physicaldrive 1I:1:20 (port 1I:box 1:bay 20, Solid State SAS, 400 GB, OK)
physicaldrive 1I:1:21 (port 1I:box 1:bay 21, Solid State SAS, 400 GB, OK)
physicaldrive 1I:1:22 (port 1I:box 1:bay 22, Solid State SAS, 400 GB, OK)
Drive Type: Data
LD Acceleration Method: All disabled
physicaldrive 1I:1:7
Port: 1I
Box: 1
Bay: 7
Status: OK
Drive Type: Data Drive
Interface Type: Solid State SAS
Size: 400 GB
Drive exposed to OS: False
Native Block Size: 4096
Firmware Revision: HPD2
Serial Number: xxxxxxxxxx
Model: HP MO0400JEFPA
Current Temperature (C): 26
Maximum Temperature (C): 36
Usage remaining: 99.74%
Power On Hours: 6982
Estimated Life Remaining based on workload to date: 111600 days
SSD Smart Trip Wearout: False
PHY Count: 2
PHY Transfer Rate: 12.0Gbps, Unknown
Drive Authentication Status: OK
Carrier Application Version: 11
Carrier Bootloader Version: 6
Sanitize Erase Supported: False
(all drives are configured exactly the same way).
Is there any problem with the raid controller? Anything miss-configured? Should I be happy with a 3x write speed? Or should I look for the bottleneck somewhere else?
ikusimakusi
Posted 2017-02-28T12:38:07.310
Reputation: 13
M.2 SSD speeds are only in theory. I have 2 of them and they are outperformed by an ancient OCZ 60GB. – Overmind – 2017-02-28T12:53:45.420
Two notes: 1: Be careful just which M.2 (NVME) SSDs are used. Those Samsungs you mentioned are quite fast, but there are also NVME SSds which are barely faster than a SATA based SSD. 2: Ware of thermal throttling for some M.2 SSD's – Hennes – 2017-02-28T12:53:56.037