FreeNAS + LAGG Slow Transfer?

Question

I built a new file server that should be way overbuilt for its purpose. It has 2 Intel(R) Xeon(R) CPU E5-2660 v3, with 128GB of ram and it is running LCAP through an Intel X540T2 network card to a Netgear GS752TP switch. Our router is a Ubiquiti EdgeMax

When I tried to transfer 1GB of data to it, the speed stays around 12MBps. I have seen it hit up to 25MBps, but has even dropped down to 500kBps. Both in short bursts.

I ran two iperf tests, one using "-u -b 1000m" on the end and the other with nothing. I will post the results below.

My question is why would this transfer be so slow with all GB equipment and being hard wired? Could there be something miss configured?

I appreciate any advice you have for me.

Test 1 results form computer using "-u -b 1000m"

    [ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec   109 MBytes   918 Mbits/sec  14012
[  4]   1.00-2.00   sec  94.2 MBytes   790 Mbits/sec  12061
[  4]   2.00-3.00   sec   104 MBytes   871 Mbits/sec  13293
[  4]   3.00-4.00   sec   108 MBytes   906 Mbits/sec  13827
[  4]   4.00-5.00   sec   113 MBytes   952 Mbits/sec  14527
[  4]   5.00-6.00   sec   114 MBytes   953 Mbits/sec  14545
[  4]   6.00-7.00   sec   111 MBytes   930 Mbits/sec  14197
[  4]   7.00-8.00   sec   109 MBytes   918 Mbits/sec  14014
[  4]   8.00-9.00   sec   114 MBytes   956 Mbits/sec  14582
[  4]   9.00-10.00  sec   103 MBytes   868 Mbits/sec  13244
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec  1.06 GBytes   906 Mbits/sec  0.044 ms  50745/138072 (37%)
[  4] Sent 138072 datagrams

iperf Done.

Test 1 results from server using "-u -b 1000m"

[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec  65.2 MBytes   547 Mbits/sec  0.047 ms  4776/13117 (36%)
[  5]   1.00-2.00   sec  68.4 MBytes   574 Mbits/sec  0.062 ms  3333/12089 (28%)
[  5]   2.00-3.00   sec  68.3 MBytes   573 Mbits/sec  0.060 ms  4448/13192 (34%)
[  5]   3.00-4.00   sec  67.8 MBytes   569 Mbits/sec  0.021 ms  4993/13675 (37%)
[  5]   4.00-5.00   sec  68.6 MBytes   575 Mbits/sec  0.053 ms  5901/14676 (40%)
[  5]   5.00-6.00   sec  68.3 MBytes   573 Mbits/sec  0.088 ms  5883/14629 (40%)
[  5]   6.00-7.00   sec  68.0 MBytes   570 Mbits/sec  0.037 ms  5455/14155 (39%)
[  5]   7.00-8.00   sec  67.6 MBytes   567 Mbits/sec  0.037 ms  5659/14306 (40%)
[  5]   8.00-9.00   sec  68.4 MBytes   573 Mbits/sec  0.029 ms  5581/14330 (39%)
[  5]   9.00-10.00  sec  68.5 MBytes   574 Mbits/sec  0.161 ms  4522/13288 (34%)
[  5]  10.00-10.05  sec  3.29 MBytes   600 Mbits/sec  0.044 ms  194/615 (32%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.05  sec   682 MBytes   570 Mbits/sec  0.044 ms  50745/138072 (37%)  receiver

Test 2 results form computer

[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  26.8 MBytes   224 Mbits/sec
[  4]   1.00-2.00   sec  29.2 MBytes   246 Mbits/sec
[  4]   2.00-3.00   sec  29.8 MBytes   250 Mbits/sec
[  4]   3.00-4.00   sec  30.2 MBytes   254 Mbits/sec
[  4]   4.00-5.00   sec  30.0 MBytes   251 Mbits/sec
[  4]   5.00-6.00   sec  29.8 MBytes   250 Mbits/sec
[  4]   6.00-7.00   sec  30.0 MBytes   252 Mbits/sec
[  4]   7.00-8.00   sec  28.0 MBytes   235 Mbits/sec
[  4]   8.00-9.00   sec  28.8 MBytes   241 Mbits/sec
[  4]   9.00-10.00  sec  30.0 MBytes   252 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec   292 MBytes   245 Mbits/sec                  sender
[  4]   0.00-10.00  sec   292 MBytes   245 Mbits/sec                  receiver

iperf Done.

Test 2 results from server

[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  26.5 MBytes   222 Mbits/sec
[  5]   1.00-2.00   sec  29.4 MBytes   247 Mbits/sec
[  5]   2.00-3.00   sec  29.6 MBytes   248 Mbits/sec
[  5]   3.00-4.00   sec  30.3 MBytes   254 Mbits/sec
[  5]   4.00-5.00   sec  29.9 MBytes   251 Mbits/sec
[  5]   5.00-6.00   sec  29.7 MBytes   249 Mbits/sec
[  5]   6.00-7.00   sec  30.0 MBytes   252 Mbits/sec
[  5]   7.00-8.00   sec  28.2 MBytes   236 Mbits/sec
[  5]   8.00-9.00   sec  28.7 MBytes   241 Mbits/sec
[  5]   9.00-10.00  sec  30.0 MBytes   252 Mbits/sec
[  5]  10.00-10.02  sec   128 KBytes  51.2 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.02  sec   292 MBytes   245 Mbits/sec                  receiver

rjt · Answer 1 · 2020-01-01T20:52:49.463

If i understand, on one of the desktops, rsync, Windows Explorer or robocopy.exe is reporting 12Mbps. Thirty-seven percent of your packets on at least one connection are going into the bitbucket. I am amazed it is still able to get over half-a-gigabit. That is phenomenal. From the host with the slow transfer, run a ping with very large packets, ping -s 60000 on Linux or ping -t -l 60000 on Windows. Need to determine whether it is the ethernet connection on the client or on the NAS. Also look on the switch(es) for dropped frames statistics. Was the slow machine connected over WiFi?

[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.05  sec   682 MBytes   570 Mbits/sec  0.044 ms  50745/138072 (37%)  receiver

The lowest speed in your data is about 222Mbps. Where in that data does it indicate 12MegaBytesPerSecond / 96MegaBitsPerSecond?

The 12MegaBytesPerSecond / 96MegaBitsPerSecond is when I was transferring either a large file from my computer to the server. That is what the average speed was while transferring that file. What is weird to me or at least stands out is the 40% loss of packets. That seems to be really high — laxer, Jan 01 '20 at 20:21
When I use ping -l 6000 it says `Request timed out.`. The maximum packet size It will send and accept is 25152. Anything more, even 25153 will say `Request timed out.` — laxer, Jan 02 '20 at 17:48

FreeNAS + LAGG Slow Transfer?

1 Answers1