I have connected two rooms via a 10Gbit fiber connection. The connection has one MikroTik switch with four SFP+ on one end (room A) and an Ubiquiti Unifi 24 ports with two SFP+ (room B).
On Room A, I have a Mac mini 2018 with a 10Gbit ethernet connected to the MikroTik switch using an SFP+-RJ45 transceiver.
On Room B, I have a Supermicro machine with an SFP+ running ESXi with several VM's. One of those is a FreeNAS (TrueNAS) and another one is a Ubuntu 20.04 server.
All those machines are connected to a VLAN (id 45) that is used mostly for storage.
I wanted to test the speed of the network using iperf3. I am a bit disappointed, as I am getting a bit less than gigabit speeds from the Mac to the FreeNAS. I tested it against the Ubuntu server, and the speed was around 9Gbit, which seems more like it should be.
Upstream was a bit different: getting around 1.45Gbit from the FreeNAS to the Mac mini, and around 3.5Gbit from the Ubuntu to the Mac mini.
I also tested between then FreeNAS and the Ubuntu server, and it was around 9Gbit from the first to the last one, and around 2.3Gbit the other way around. Once I had jumbo frames enabled, and it was way faster both ways, but it didn't work well with the Mac, so I went back to MTU 1500.
Then I decided to boot the Mac mini with Ubuntu to see if this was an OS-related issue, and indeed, From Ubuntu client (Mac) to my Ubuntu server, I got up and down speeds of 9Gbit. But then I got around 2.Gbit from the FreeNAS, and 9Gbit speed the other way around:
FreeNAS | Ubuntu Server | |
---|---|---|
From Mac (macOS) | 0.9 | 9 |
To Mac (macOS) | 2.3 | 3.5 |
From Mac (ubuntu) | 9 | 9 |
To Mac (ubuntu) | 2.3 | 9 |
(speeds are in Gbits/s)
I realise now that the hardware is working as it should (except for the Mac that doesn't seem to play well with MTU 9000 with this MikroTik transceiver) - using it gets be 0Gbits from the other machines (though I haven't tested with Ubuntu).
My questions are: Is there a way to perform a test that is a bit more consistent between OS'es? And does it mean that my Mac will get slower transfers to/from the FreeNAS, or is it just iperf3 that's behaving inconsistently?