I've inherited a setup using Linux Ubuntu 14, a megaraid_sas, and ZFS.
We're getting some performance problems (never being able to fully utilize the 6GB bandwidth from the raid) and I'm curious if it's related to the ZFS setup - which seems a little unusual.
The question really are:
Is this setup (see below) problematic? At best, it seems unnecessarily complex (why use ZFS instead of just controlling data space size at the raid an mounting directly?)
We do not seem to be utilizing the maximum read rates from the raid. Could this setup be why?
SETUP
The RAID has a number of RAID 5 and 6 virtual disks which present to the Linux server as local devices. I.e.
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 14,6T 0 disk
├─sda1 8:1 0 14,6T 0 part
└─sda9 8:9 0 8M 0 part
sdb 8:16 02,6G 0 disk
├─sdb1 8:17 0 94,6G 0 part /
├─sdb2 8:18 0 1K 0 part
└─sdb5 8:21 0 128G 0 part [SWAP]
sdc 8:32 0 14,6T 0 disk
├─sdc1 8:33 0 14,6T 0 part
└─sdc9 8:41 0 8M 0 part
sdd 8:48 0 10,9T 0 disk
├─sdd1 8:49 0 10,9T 0 part
└─sdd9 8:57 0 8M 0 part
sd<N> 8:64 0 7,3T 0 disk
├─sd<N>1 8:65 0 7,3T 0 part
└─sd<N>9 8:73 0 8M 0 part
These are then all strapped together again as a zpool. I.e.
# zpool status
pool: zpool
state: ONLINE
scan: scrub repaired 0 in 84h36m with 0 errors on Tue Aug 29 00:48:43 2017
config:
NAME STATE READ WRITE CKSUM
zpool ONLINE 0 0 0
sd<N> ONLINE 0 0 0
sda ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
errors: No known data errors
...which are then divided up into datasets. I.e.
# df -h
Filesystem Size Used Avail Use% Mounted on
udev 63G 12K 63G 1% /dev
tmpfs 13G 23M 13G 1% /run
/dev/sdb1 94G 13G 76G 15% /
zpool/dataset1 13T 11T 2,3T 82% /common/share
zpool/dataset1/archive 3,9T 1,6T 2,3T 41% /common/share/archive
zpool/dataset2 6,6T 4,3T 2,3T 66% /h2t
zpool 5,2T 2,9T 2,3T 56% /zpool
zpool/dataset3 12T 8,8T 2,3T 80% /zpool/backup
zpool/dataset4 2,4T 28G 2,3T 2% /zpool/homes
zpool/dataset4/extern 2,7T 317G 2,3T 12% /zpool/homes/externstaff
zpool/homes/students 14T 12T 2,3T 84% /zpool/homes/students
zpool/temp 2,4T 50G 2,3T 3% /common/temp