Our server has a raid6 array (10+2 disks, each disk is 6TB/SATA/7200rpm). We plan to make it an NFS storage shared with other clients. However, the writing speed is very slow:
[root@Server ~]# mkfs.xfs /dev/sdd1
meta-data=/dev/sdd1 isize=512 agcount=55, agsize=268435455 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=0, sparse=0
data = bsize=4096 blocks=14640739328, imaxpct=1
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=521728, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
[root@Server ~]# mount /dev/sdd1 /share/
[root@Server ~]# mkdir -p /share/aa/bb/cc/dd/ee
[root@Server ~]# cd /share/aa/bb/cc/dd/ee
[root@Server ~]# dd if=/dev/zero of=tempfile bs=1M count=30000 conv=fsync
30000+0 records in
30000+0 records out
31457280000 bytes (31 GB) copied, 1186.26 s, 26.5 MB/s
[root@Server ~]# cd /share
[root@Server ~]# dd if=/dev/zero of=tempfile bs=1M count=30000 conv=fsync
30000+0 records in
30000+0 records out
31457280000 bytes (31 GB) copied, 109.495 s, 287 MB/s
It is weird that writing to /share
is much faster (287MB/s) than writing to deeper subfolders (26MB/s). I tested for many times, and the results were similar.
I also tried formating it to other file systems, such as ext4 and brtfs, but both of them behaved normally in subfolders (592MB/s and 226MB/s).
ext4:
[root@Server ~]# mkfs.ext4 /dev/sdd1
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
915046400 inodes, 14640739328 blocks
732036966 blocks (5.00%) reserved for the super user
First data block=0
446800 block groups
32768 blocks per group, 32768 fragments per group
2048 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848, 512000000, 550731776, 644972544, 1934917632,
2560000000, 3855122432, 5804752896, 12800000000
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
[root@Server ~]# mount /dev/sdd1 /share/
[root@Server ~]# mkdir -p /share/aa/bb/cc/dd/ee
[root@Server ~]# cd /share/aa/bb/cc/dd/ee
[root@Server ~]# dd if=/dev/zero of=tempfile bs=1M count=30000 conv=fsync
30000+0 records in
30000+0 records out
31457280000 bytes (31 GB) copied, 53.1054 s, 592 MB/s
btrfs:
[root@Server ~]# mkfs.btrfs /dev/sdd1
btrfs-progs v4.9.1
See http://btrfs.wiki.kernel.org for more information.
Label: (null)
UUID: 32aa3677-7898-4f14-9158-b698bd11abbc
Node size: 16384
Sector size: 4096
Filesystem size: 54.54TiB
Block group profiles:
Data: single 8.00MiB
Metadata: DUP 1.00GiB
System: DUP 8.00MiB
SSD detected: no
Incompat features: extref, skinny-metadata
Number of devices: 1
Devices:
ID SIZE PATH
1 54.54TiB /dev/sdd1
[root@Server ~]# mount /dev/sdd1 /share/
[root@Server ~]# mkdir -p /share/aa/bb/cc/dd/ee
[root@Server ~]# cd /share/aa/bb/cc/dd/ee
[root@Server ~]# dd if=/dev/zero of=tempfile bs=1M count=30000 conv=fsync
30000+0 records in
30000+0 records out
31457280000 bytes (31 GB) copied, 139.226 s, 226 MB/s
I googled for several weeks, but found no solution. Could anyone here give me some hints?
PS: the OS is CentOS 7.4