1

We have two hosts running VMWare ESXi 5.1.0. Both are connected to a RAID 5 array via iSCSI.

The lun is an existing lun with vm on it but it is not reconized by the hosts after the reboot.

it is visible in vSphere Client in Configuration > Storage > Devices but not in Configuration > Storage > Datastores

Somebody have an idea how can we fix this ?

These are some commande we tried on the host

~ # voma -m vmfs -f check -d /vmfs/devices/disks/naa.60080e50002e0074000003975193cbf3
Checking if device is actively used by other hosts
Running VMFS Checker version 0.9 in check mode
Initializing LVM metadata, Basic Checks will be done
         ERROR: Missing LVM Magic. Disk doesn't have valid LVM Device
         ERROR: Failed to Initialize LVM Metadata
   VOMA failed to check device : Not a Logical Volume

Total Errors Found:           0

With this we can see the LUN.

~ # esxcfg-scsidevs -c
Device UID                            Device Type      Console Device                                            Size      Multipath PluginDisplay Name
mpx.vmhba0:C0:T0:L0                   CD-ROM           /vmfs/devices/cdrom/mpx.vmhba0:C0:T0:L0                   0MB       NMP     Local TSSTcorp CD-ROM (mpx.vmhba0:C0:T0:L0)
naa.600605b004802b101840d2513e1e6628  Direct-Access    /vmfs/devices/disks/naa.600605b004802b101840d2513e1e6628  1140592MB NMP     Local IBM Disk (naa.600605b004802b101840d2513e1e6628)
naa.60080e50002dff340000019750b4fd4e  Direct-Access    /vmfs/devices/disks/naa.60080e50002dff340000019750b4fd4e  20MB      NMP     IBM Serial Attached SCSI Disk (naa.60080e50002dff340000019750b4fd4e)
naa.60080e50002dff340000044d53d7d13d  Direct-Access    /vmfs/devices/disks/naa.60080e50002dff340000044d53d7d13d  123121MB  NMP     IBM Serial Attached SCSI Disk (naa.60080e50002dff340000044d53d7d13d)
naa.60080e50002e0074000003975193cbf3  Direct-Access    /vmfs/devices/disks/naa.60080e50002e0074000003975193cbf3  614400MB  NMP     IBM Serial Attached SCSI Disk (naa.60080e50002e0074000003975193cbf3)

partition :

~ # partedUtil getptbl /vmfs/devices/disks/naa.60080e50002e0074000003975193cbf3
gpt
78325 255 63 1258291200

This is the Log file: from /var/log/vmkernel.log

2014-08-01T15:59:33.327Z cpu16:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:33.337Z cpu16:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:33.351Z cpu16:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:33.356Z cpu16:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:33.361Z cpu16:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:33.366Z cpu16:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:33.372Z cpu16:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:33.377Z cpu16:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:33.388Z cpu16:9200)FSS: 4972: No FS driver claimed device 'mpx.vmhba0:C0:T0:L0': Not supported
2014-08-01T15:59:33.396Z cpu16:9200)Vol3: 692: Couldn't read volume header from control: Not supported
2014-08-01T15:59:33.396Z cpu16:9200)Vol3: 692: Couldn't read volume header from control: Not supported
2014-08-01T15:59:33.396Z cpu16:9200)FSS: 4972: No FS driver claimed device 'control': Not supported
2014-08-01T15:59:33.433Z cpu16:9200)VC: 1547: Device rescan time 27 msec (total number of devices 8)
2014-08-01T15:59:33.433Z cpu16:9200)VC: 1550: Filesystem probe time 80 msec (devices probed 6 of 8)
2014-08-01T15:59:58.817Z cpu4:9190)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:58.821Z cpu4:9190)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:58.830Z cpu4:9190)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:58.838Z cpu4:9190)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:58.843Z cpu4:9190)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:58.848Z cpu4:9190)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:58.854Z cpu4:9190)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:58.859Z cpu4:9190)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T15:59:58.867Z cpu4:9190)FSS: 4972: No FS driver claimed device 'mpx.vmhba0:C0:T0:L0': Not supported
2014-08-01T15:59:58.880Z cpu0:9190)Vol3: 692: Couldn't read volume header from control: Not supported
2014-08-01T15:59:58.880Z cpu0:9190)Vol3: 692: Couldn't read volume header from control: Not supported
2014-08-01T15:59:58.880Z cpu0:9190)FSS: 4972: No FS driver claimed device 'control': Not supported
2014-08-01T15:59:58.911Z cpu0:9190)VC: 1547: Device rescan time 22 msec (total number of devices 8)
2014-08-01T15:59:58.911Z cpu0:9190)VC: 1550: Filesystem probe time 79 msec (devices probed 6 of 8)
2014-08-01T16:00:01.440Z cpu9:15384)VmkAccess: 637: lanusbcfg: access denied:: dom:appDom(2), obj:appDom(2), mode:inet_socket_bind(17) port no: (68)
2014-08-01T16:00:01.440Z cpu9:15384)VmkAccess: 207: ipAddr = 0.0.0.0, port = 68
2014-08-01T16:01:14.089Z cpu2:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.097Z cpu2:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.142Z cpu2:9200)Vol3: 692: Couldn't read volume header from control: Not supported
2014-08-01T16:01:14.142Z cpu2:9200)Vol3: 692: Couldn't read volume header from control: Not supported
2014-08-01T16:01:14.142Z cpu2:9200)FSS: 4972: No FS driver claimed device 'control': Not supported
2014-08-01T16:01:14.149Z cpu0:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.153Z cpu0:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.158Z cpu0:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.164Z cpu0:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.169Z cpu0:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.174Z cpu0:9200)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.185Z cpu0:9200)FSS: 4972: No FS driver claimed device 'mpx.vmhba0:C0:T0:L0': Not supported
2014-08-01T16:01:14.187Z cpu0:9200)VC: 1547: Device rescan time 21 msec (total number of devices 8)
2014-08-01T16:01:14.187Z cpu0:9200)VC: 1550: Filesystem probe time 82 msec (devices probed 6 of 8)
2014-08-01T16:01:14.377Z cpu14:9201)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.385Z cpu14:9201)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.420Z cpu14:9201)Vol3: 692: Couldn't read volume header from control: Not supported
2014-08-01T16:01:14.420Z cpu14:9201)Vol3: 692: Couldn't read volume header from control: Not supported
2014-08-01T16:01:14.420Z cpu14:9201)FSS: 4972: No FS driver claimed device 'control': Not supported
2014-08-01T16:01:14.430Z cpu14:9201)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.434Z cpu14:9201)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.440Z cpu14:9201)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.445Z cpu14:9201)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.450Z cpu14:9201)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.456Z cpu14:9201)<3>ata1.00: bad CDB len=16, scsi_op=0x9e, max=12
2014-08-01T16:01:14.464Z cpu14:9201)FSS: 4972: No FS driver claimed device 'mpx.vmhba0:C0:T0:L0': Not supported
2014-08-01T16:01:14.466Z cpu14:9201)VC: 1547: Device rescan time 14 msec (total number of devices 8)
2014-08-01T16:01:14.466Z cpu14:9201)VC: 1550: Filesystem probe time 74 msec (devices probed 6 of 8)
2014-08-01T16:01:53.503Z cpu14:8206)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x1a (0x412402e61dc0, 0) to dev "mpx.vmhba0:C0:T0:L0" on path "vmhba0:C0:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
2014-08-01T16:01:53.503Z cpu14:8206)ScsiDeviceIO: 2316: Cmd(0x412402e61dc0) 0x1a, CmdSN 0x263e from world 0 to dev "mpx.vmhba0:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2014-08-01T16:02:01.455Z cpu12:15700)VmkAccess: 637: lanusbcfg: access denied:: dom:appDom(2), obj:appDom(2), mode:inet_socket_bind(17) port no: (68)
2014-08-01T16:02:01.455Z cpu12:15700)VmkAccess: 207: ipAddr = 0.0.0.0, port = 68
baaron
  • 111
  • 1
  • 1
  • 4
  • 2
    "after the reboot" - which reboot? What caused this condition in the first place? Seems like the filesystem on that device has been destroyed; the errors in the vmkernel log are just for the CD-ROM drive. – Shane Madden Aug 01 '14 at 16:27
  • After rebooting the esx servers. We had to reboot them for maintenance. – baaron Aug 01 '14 at 22:03

1 Answers1

2

Please Refer to the following KB article from vmware. You probably have a stale lock on it.

vmkfstools -B /vmfs/devices/disks/naa.00000000000000000000000000:1

where naa.00000000000000000000000000:1 is replaced with the LUN NAA ID and partition to be checked. Note the ":1"

FYI: naa is not always the prefix. Mine are a fiber lun that starts with eui.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2036767

Andrew Williams
  • 667
  • 8
  • 20
Parallogix
  • 11
  • 3