I have two VMs running on an ESXi 7.0U3g host.
Both VMs are installed on an NVMe datastore (FYI for comparison below, note "usbspeed" vs "nvmespeed" tests), and have identical configuration:
- CPU: 8
- Memory: 16GB
- SCSI Controller: LSI Logic Parallel
- USB Controller: USB 3.1
VM #1: Debian 11.4.0 (5.10.0-18-amd64)
# dmesg | grep usb
[237086.992370] usb 2-1: new SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[237087.013420] usb 2-1: New USB device found, idVendor=1058, idProduct=264d, bcdDevice=30.12
[237087.013422] usb 2-1: New USB device strings: Mfr=2, Product=3, SerialNumber=1
[237087.013423] usb 2-1: Product: easystore 264D
[237087.013424] usb 2-1: Manufacturer: Western Digital
[237087.013425] usb 2-1: SerialNumber: [redacted]
[237087.046318] usb-storage 2-1:1.0: USB Mass Storage device detected
[237087.046483] scsi host33: usb-storage 2-1:1.0
[237087.046547] usbcore: registered new interface driver usb-storage
[237087.048081] usbcore: registered new interface driver uas
# lsusb -t
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M
|__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M
|__ Port 1: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 12M
# lspci | grep -i usb
13:00.0 USB controller: VMware USB3 xHCI 1.0 Controller
# dd if=/dev/zero of=/mnt/easystore/usbspeed bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.84469 s, 157 MB/s
# dd if=/dev/zero of=/nvmespeed bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.664862 s, 1.6 GB/s
VM #2: Ubuntu 22.04.1 (5.15.0-43-generic)
# dmesg | grep usb
[ 8175.488648] usb 2-1: new SuperSpeed USB device number 2 using xhci_hcd
[ 8175.509805] usb 2-1: New USB device found, idVendor=1058, idProduct=264d, bcdDevice=30.12
[ 8175.509808] usb 2-1: New USB device strings: Mfr=2, Product=3, SerialNumber=1
[ 8175.509810] usb 2-1: Product: easystore 264D
[ 8175.509811] usb 2-1: Manufacturer: Western Digital
[ 8175.509812] usb 2-1: SerialNumber: [redacted]
[ 8175.540057] usb-storage 2-1:1.0: USB Mass Storage device detected
[ 8175.540532] scsi host33: usb-storage 2-1:1.0
[ 8175.541089] usbcore: registered new interface driver usb-storage
[ 8175.542779] usbcore: registered new interface driver uas
# lsusb -t
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M
|__ Port 1: Dev 3, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M
|__ Port 1: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 12M
# lspci | grep -i usb
0b:00.0 USB controller: VMware USB3 xHCI 1.0 Controller
# dd if=/dev/zero of=/mnt/easystore/usbspeed bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.567066 s, 1.9 GB/s
# dd if=/dev/zero of=/nvmespeed bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.636782 s, 1.7 GB/s
I've run the same tests multiple times and gotten similar results every time, so it's not like a one-time or cache related thing.
Why is Debian so much slower when writing to the USB3 disk?
Edit 1: added output from lspci | grep -i usb
