1

The block-device of a USB mass storage does not show up in Ubuntu 16.04, while I positively can see it in Ubuntu 14.04. Question: How can I make it show up in Ubuntu 16.04?

NOTES:

  1. Both machines are virtual on the same hardware, so it is not cabling, wiring, power, connection, blah blah... Virtualization of the USB is identical and I have triple-checked it is not the problem.
  2. In fact... the USB layer is properly detected both in Ubuntu 14.04 and Ubuntu 16.04 (see the logs below).
  3. The block device /dev/sdb shows up in Ubuntu 14.04 (as expected), but does not show up in Ubuntu 16.04 (topic of this question).
  4. The problem seems to be located in something named usb-storage which in the Ubuntu 14.04 is bridging the USB into the SCSI but seems to be missing in my installation of the Ubuntu 16.04.

Context

I have two Vagrant/VirtualBox machines. One with Ubuntu 14.04 and the other with Ubuntu 16.04. In the host, I have a JMicron-based dock with a 1T HD plugged in (chip product JMS56x).

The drive is not formatted and has no partitions, I plan to use the raw-block device in a pool fo ZFS.

I want to do the usage of the ZFS from inside Ubuntu 16.04.

In Ubuntu 14.04

I use the vagrant image config.vm.box = "ubuntu/trusty64" and I already have usbutils installed:

root@vagrant-ubuntu-trusty-64:/home/vagrant# apt list --installed | grep usb

WARNING: apt does not have a stable CLI interface yet. Use with caution in scripts.

libusb-0.1-4/trusty,now 2:0.1.12-23.3ubuntu1 amd64 [installed]
libusb-1.0-0/trusty,now 2:1.0.17-1ubuntu2 amd64 [installed]
usbutils/trusty,now 1:007-2ubuntu1 amd64 [installed]

I tell VirtualBox to connect the JMicron and I successfully get the block-device as a SCSI disk.

Before connecting

Before connecting the disk, I can see sda and its unique partition sda1:

root@vagrant-ubuntu-trusty-64:/home/vagrant# ls -l /dev/sd*
brw-rw---- 1 root disk 8, 0 Nov 26 13:06 /dev/sda
brw-rw---- 1 root disk 8, 1 Nov 26 13:06 /dev/sda1

After connecting

After connecting, I can see both sda and sdb (in this case, sdb with no partitions, as expected):

root@vagrant-ubuntu-trusty-64:/home/vagrant# ls -l /dev/sd*
brw-rw---- 1 root disk 8,  0 Nov 26 13:06 /dev/sda
brw-rw---- 1 root disk 8,  1 Nov 26 13:06 /dev/sda1
brw-rw---- 1 root disk 8, 16 Nov 26 16:56 /dev/sdb

fdisk gives coherent results with the /dev/sd* listings.

So Ubuntu 14.04 behaves exactly as expected, as the the block device /dev/sdb shows up.

In Ubuntu 16.04

I use the vagrant image config.vm.box = "gbarbieru/xenial" and I don't have the usbutils installed. I install them via apt install usbutils and after that I get:

root@vagrant:/home/vagrant# apt list --installed | grep usb

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libusb-0.1-4/xenial,now 2:0.1.12-28 amd64 [installed]
libusb-1.0-0/xenial,now 2:1.0.20-1 amd64 [installed,automatic]
usbutils/xenial,now 1:007-4 amd64 [installed]

I tell VirtualBox to connect the JMicron and I do not get any extra block device and this is the main problem.

Before connecting

Before connecting the disk, I can see sda and its several partitions sda1, sda2 and sda5:

root@vagrant:/home/vagrant# ls -l /dev/sd*
brw-rw---- 1 root disk 8, 0 Nov 26 10:55 /dev/sda
brw-rw---- 1 root disk 8, 1 Nov 26 10:55 /dev/sda1
brw-rw---- 1 root disk 8, 2 Nov 26 10:55 /dev/sda2
brw-rw---- 1 root disk 8, 5 Nov 26 10:55 /dev/sda5

After connecting

After connecting, I cannot see in Ubuntu 16.04 any sdb as I could see with the Ubuntu 14.04:

root@vagrant:/home/vagrant# ls -l /dev/sd*
brw-rw---- 1 root disk 8, 0 Nov 26 10:55 /dev/sda
brw-rw---- 1 root disk 8, 1 Nov 26 10:55 /dev/sda1
brw-rw---- 1 root disk 8, 2 Nov 26 10:55 /dev/sda2
brw-rw---- 1 root disk 8, 5 Nov 26 10:55 /dev/sda5

fdisk gives coherent results with the /dev/sd* listings.

So Ubuntu 16.04 does not behave as expected because the block device /dev/sdb does not show up. Nevertheless, the low-level USB device is in fact properly detected. See the logs:

Traces and logs

In Ubuntu 14.04 I get following traces in tail -f /var/log/kern.log when connecting:

Nov 26 16:14:34 vagrant-ubuntu-trusty-64 kernel: [11313.532808] usb 1-1: new high-speed USB device number 3 using xhci_hcd
Nov 26 16:14:34 vagrant-ubuntu-trusty-64 kernel: [11313.553057] usb 1-1: New USB device found, idVendor=152d, idProduct=9561
Nov 26 16:14:34 vagrant-ubuntu-trusty-64 kernel: [11313.553060] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=5
Nov 26 16:14:34 vagrant-ubuntu-trusty-64 kernel: [11313.553062] usb 1-1: Product: JMS56x Series
Nov 26 16:14:34 vagrant-ubuntu-trusty-64 kernel: [11313.553064] usb 1-1: Manufacturer: JMicron
Nov 26 16:14:34 vagrant-ubuntu-trusty-64 kernel: [11313.553065] usb 1-1: SerialNumber: 00000000000000000000
Nov 26 16:14:34 vagrant-ubuntu-trusty-64 kernel: [11313.557954] usb-storage 1-1:1.0: USB Mass Storage device detected
Nov 26 16:14:34 vagrant-ubuntu-trusty-64 kernel: [11313.557991] scsi2 : usb-storage 1-1:1.0
Nov 26 16:14:35 vagrant-ubuntu-trusty-64 kernel: [11314.557915] scsi 2:0:0:0: Direct-Access     WDC WD10 EAVS-00D7B1      0105 PQ: 0 ANSI: 6
Nov 26 16:14:35 vagrant-ubuntu-trusty-64 kernel: [11314.558304] sd 2:0:0:0: Attached scsi generic sg1 type 0
Nov 26 16:14:35 vagrant-ubuntu-trusty-64 kernel: [11314.564883] sd 2:0:0:0: [sdb] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
Nov 26 16:14:35 vagrant-ubuntu-trusty-64 kernel: [11314.566221] sd 2:0:0:0: [sdb] Write Protect is off
Nov 26 16:14:35 vagrant-ubuntu-trusty-64 kernel: [11314.566225] sd 2:0:0:0: [sdb] Mode Sense: 47 00 10 08
Nov 26 16:14:35 vagrant-ubuntu-trusty-64 kernel: [11314.571284] sd 2:0:0:0: [sdb] No Caching mode page found
Nov 26 16:14:35 vagrant-ubuntu-trusty-64 kernel: [11314.572366] sd 2:0:0:0: [sdb] Assuming drive cache: write through
Nov 26 16:14:35 vagrant-ubuntu-trusty-64 kernel: [11314.579291] sd 2:0:0:0: [sdb] No Caching mode page found
Nov 26 16:14:35 vagrant-ubuntu-trusty-64 kernel: [11314.579651] sd 2:0:0:0: [sdb] Assuming drive cache: write through
Nov 26 16:14:36 vagrant-ubuntu-trusty-64 kernel: [11315.056332]  sdb: unknown partition table
Nov 26 16:14:36 vagrant-ubuntu-trusty-64 kernel: [11315.066919] sd 2:0:0:0: [sdb] No Caching mode page found
Nov 26 16:14:36 vagrant-ubuntu-trusty-64 kernel: [11315.067273] sd 2:0:0:0: [sdb] Assuming drive cache: write through
Nov 26 16:14:36 vagrant-ubuntu-trusty-64 kernel: [11315.070669] sd 2:0:0:0: [sdb] Attached SCSI disk

Instead, in Ubuntu 16.04 I get those traces in tail -f /var/log/kern.log when connecting the device:

Nov 26 11:14:59 vagrant kernel: [ 1178.761744] usb 1-1: new high-speed USB device number 4 using xhci_hcd
Nov 26 11:14:59 vagrant kernel: [ 1179.088626] usb 1-1: New USB device found, idVendor=152d, idProduct=9561
Nov 26 11:14:59 vagrant kernel: [ 1179.088632] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=5
Nov 26 11:14:59 vagrant kernel: [ 1179.088636] usb 1-1: Product: JMS56x Series
Nov 26 11:14:59 vagrant kernel: [ 1179.088638] usb 1-1: Manufacturer: JMicron
Nov 26 11:14:59 vagrant kernel: [ 1179.088641] usb 1-1: SerialNumber: 00000000000000000000

The first missing line is:

usb-storage 1-1:1.0: USB Mass Storage device detected

So comparing both logs, it seems that "something named usb-storage" is not present. Probably the usb-storage thing is the bridge to the SCSI.

Questions

  1. Is usb-storage still the way to go in 2016? Or shoud I use another bridge?
  2. How do I get the usb-storage thing active in the Ubuntu 16.04 so it continues the process, exposes the storage to the SCSI and I get the /dev/sdb block device listed?
Xavi Montero
  • 295
  • 3
  • 16

2 Answers2

1

Try to run modprobe usb-storage or modprobe usb_storage before connecting the USB disk. Does it change anything?

shodanshok
  • 44,038
  • 6
  • 98
  • 162
  • I start to see... usb_storage is a kernel module... Thanks for the tip. It is present in the `14.04` but it is not present in the `16.04` that I am using. `ls -l /lib/modules/*/kernel/drivers/usb/storage/` displays `usb-storage.ko` in the `14.04` but is missing in the `16.04`. Is it possible to install kernel modules **without** recompiling (for example from apt-get or so?) - Or I necessarily **have** to compile? – Xavi Montero Nov 27 '16 at 14:21
  • Can you post the output of `zgrep -i usb /proc/config.gz` (or any other kernel config file) ? – shodanshok Nov 27 '16 at 18:19
  • I don't have `/proc/config.gz` but I have `/boot/config*`. I have dumped them in this `gist`: https://gist.github.com/anonymous/29c13e97165d955ca07d6f983132f228 - see snippet1 & snippet2 for the `ubuntu/trusty64` and `gbarbieru/xenial` vagrant boxes respectively. Seems they both have `CONFIG_USB_STORAGE=m`, instead in the `/lib/modules/*` directory I can't see the `.../kernel/drivers/usb/*` in the xenial, only in the trusty - snippets 3 and 4 of the same gist. So... may I dynamically add the "module" in the filesystem at `/lib/modules/*` and have my kernel load the `*.ko` without recompiling? – Xavi Montero Nov 27 '16 at 23:24
  • I just checked on a Ubuntu 16.04.01 and I can confirm that `usb-storage.ko` *is present* under `/lib/modules//kernel/drivers/usb/storage/`. If you have no modules, try installing a newer/different kernel via `apt-get` – shodanshok Nov 28 '16 at 10:24
1

The problem is that you use just generic kernel without usb-storage module. To have usb storage working just install linux-image-extra package

E.g. on ubuntu 16.04: 

apt-get install linux-image-extra-$(uname -r) -y

And then load usb-storage module or disconnect/connect your device

root@ubuntu-xenial:~# modprobe usb-storage
root@ubuntu-xenial:~# dmesg | tail
[  698.649461] scsi host3: usb-storage 1-1:1.0
[  698.650107] usbcore: registered new interface driver usb-storage
[  699.667145] scsi 3:0:0:0: Direct-Access              USB DISK 2.0     PMAP PQ: 0 ANSI: 6
[  699.668667] sd 3:0:0:0: Attached scsi generic sg2 type 0
[  699.679897] sd 3:0:0:0: [sdc] 30289920 512-byte logical blocks: (15.5 GB/14.4 GiB)
[  699.687947] sd 3:0:0:0: [sdc] Write Protect is off
[  699.687951] sd 3:0:0:0: [sdc] Mode Sense: 45 00 00 00
[  699.695974] sd 3:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[  699.753448]  sdc: sdc1
[  699.798466] sd 3:0:0:0: [sdc] Attached SCSI removable disk
Stas
  • 11
  • 2