XenServer, mapping SAN LUN to VDI

Question

I have a few XenServer 6.2/6.5 hosts, and ZFS SAN datastore, providing iSCSI exports for VM storage. It's not an optimal solution to export huge 1-2 Tb LUN as XenServer LVMoiSCSI SR, because: a) no thin provisioning on XenServer storage b) unefficiency of ZFS snapshot/send on SAN side

I found that question as probably worked case of directly map SAN LUN to VM VDI: Mapping VDIs directly to software iSCSI LUNs in XenServer

But it not worked in my case, because created VDI has 'managed=false' property and cannot be attached to VM:

[root@kst0-hv01 ~]# xe vbd-plug uuid=f1d66282-b8f0-3776-f913-7ea731c260df
This operation cannot be performed because the system does not manage this VDI
vdi: f1d66282-b8f0-3776-f913-7ea731c260df

So I call 'xe vdi-forget' and recreated it by 'xe vdi-introduce' with the same params, and I have VDI:

[root@kst0-hv01 ~]# xe vdi-param-list uuid=b0840d42-7bc1-66d7-a94d-e639cc9b3818
uuid ( RO)                    : b0840d42-7bc1-66d7-a94d-e639cc9b3818
              name-label ( RW):
        name-description ( RW):
           is-a-snapshot ( RO): false
             snapshot-of ( RO): <not in database>
               snapshots ( RO):
           snapshot-time ( RO): 19700101T00:00:00Z
      allowed-operations (SRO): forget; destroy; copy; snapshot
      current-operations (SRO):
                 sr-uuid ( RO): b192fe5c-c32b-9ce9-9d4e-1d2222269120
           sr-name-label ( RO): san
               vbd-uuids (SRO):
         crashdump-uuids (SRO):
            virtual-size ( RO): 68719476736
    physical-utilisation ( RO): 0
                location ( RO): b0840d42-7bc1-66d7-a94d-e639cc9b3818
                    type ( RO): User
                sharable ( RO): false
               read-only ( RO): false
            storage-lock ( RO): false
                 managed ( RO): true
                  parent ( RO): <not in database>
                 missing ( RO): false
            other-config (MRW):
           xenstore-data (MRO):
               sm-config (MRO): LUNid: 0; type: raw; SCSIid: 26537373734393364
                 on-boot ( RW): persist
           allow-caching ( RW): false
         metadata-latest ( RO): false
        metadata-of-pool ( RO): <not in database>
                    tags (SRW):

And when I created VBD and attached it to VM there is error:

Error code: SR_BACKEND_FAILURE_46
Error parameters: , The VDI is not available,

I can see VBD attached in 'xe vbd-list' as /dev/xvde, but can't see it from VM:

[root@kst0-hv01 ~]# xe vbd-param-list uuid=f8963685-b4d9-cdbb-d1af-47c7f2b54cb8
uuid ( RO)                        : f8963685-b4d9-cdbb-d1af-47c7f2b54cb8
                     vm-uuid ( RO): 251e8ca9-72c9-c220-a53c-b783acd9935f
               vm-name-label ( RO): backup
                    vdi-uuid ( RO): b0840d42-7bc1-66d7-a94d-e639cc9b3818
              vdi-name-label ( RO):
          allowed-operations (SRO): pause; unpause; attach; unplug_force; unplug
          current-operations (SRO):
                       empty ( RO): false
                      device ( RO): xvde
                  userdevice ( RW): 4
                    bootable ( RW): false
                        mode ( RW): RW
                        type ( RW): Disk
                 unpluggable ( RW): true
          currently-attached ( RO): true
                  attachable ( RO): true
                storage-lock ( RO): false
                 status-code ( RO): 0
               status-detail ( RO):
          qos_algorithm_type ( RW):
        qos_algorithm_params (MRW):
    qos_supported_algorithms (SRO):
                other-config (MRW):
                 io_read_kbs ( RO): 0.000
                io_write_kbs ( RO): 0.000

I double check and can see iSCSI LUN attached on HV:

26537373734393364 dm-46 SCST_BIO,vdi
size=64G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 133:0:0:0 sds 65:32  active ready  running

[root@kst0-hv01 ~]# xe pbd-param-list uuid=ed62495f-f2db-5886-f688-7f9883dabcb1
uuid ( RO)                  : ed62495f-f2db-5886-f688-7f9883dabcb1
     host ( RO) [DEPRECATED]: 304ce08d-869e-41b9-9cfd-dd917f809b33
             host-uuid ( RO): 304ce08d-869e-41b9-9cfd-dd917f809b33
       host-name-label ( RO): kst0-hv01
               sr-uuid ( RO): b192fe5c-c32b-9ce9-9d4e-1d2222269120
         sr-name-label ( RO): san
         device-config (MRO): multihomelist: 172.18.3.20:3260; targetIQN: iqn.2015-07.cvetnaya:vdi; target: 172.18.3.20
    currently-attached ( RO): true
          other-config (MRW): storage_driver_domain: OpaqueRef:5c864a43-f80a-730d-e2c0-7e4e3a9caaa3; iscsi_sessions: 1

Is anybody have success story of such thing?

I can of course directly connect VM to iSCSI LUN in client side, but still prefer to do that on HV.

Thank you in advance.

P.S.

[root@kst0-hv01 ~]# xe sr-param-list uuid=aba7aac8-7b26-d9b8-c0a5-bd9fe025b6b5
uuid ( RO)                    : aba7aac8-7b26-d9b8-c0a5-bd9fe025b6b5
              name-label ( RW): san
        name-description ( RW):
                    host ( RO): kst0-hv01
      allowed-operations (SRO): VDI.introduce; VDI.create; PBD.create; PBD.destroy; plug; VDI.destroy; scan; VDI.clone; unplug
      current-operations (SRO):
                    VDIs (SRO): 17c4c57e-581a-1db3-c6cd-68c4cf9766d8; b0840d42-7bc1-66d7-a94d-e639cc9b3818
                    PBDs (SRO): b7a85f12-2a4e-0ec9-c290-ca3128676f03
      virtual-allocation ( RO): 206158430208
    physical-utilisation ( RO): 206158430208
           physical-size ( RO): 206158430208
                    type ( RO): iscsi
            content-type ( RO):
                  shared ( RW): false
           introduced-by ( RO): <not in database>
            other-config (MRW):
               sm-config (MRO): datatype: ISCSI; multipathable: true; targetIQN: iqn.2015-07.cvetnaya:vdi; target: 172.18.3.20; disktype: Raw
                   blobs ( RO):
     local-cache-enabled ( RO): false
                    tags (SRW):

Output of `xe sr-param-list uuid=...` would make sense here. Can you see the SR from XenCenter? — sam_pan_mariusz, Jul 17 '15 at 13:58
I just updated main post with 'xe sr-param-list' output. And yes, I can see SR and VDI in XenCenter. — Valentin, Jul 19 '15 at 08:52

score 1 · Answer 1 · answered Jul 18 '15 at 21:20

I have two solution which work stable. And i did nothing which is not supported by (citrix) xenserver.

NAS4FREE: Normal install on Dell T20 with 8 GB ECC-Ram; 4 * 3 TB HDs as raid-z1 (one spare). Service iSCSI activated. One lun with 2 TB as vm-storage, one lun for nightly cron backup of vms (snapshot) 2 TB and one lun for heartbeat 2 GB. Today installed update XS65ESP1005 with no problems.

Multipathing works stable.

Maybe you have (after shutdown and reboot of xenserver-machine) to repair sr.

ZFSGURU (Start testing 10 days ago): Asrock J1900, 8 GB Ram zfsguru installed from livecd as vm. For storage unused 4 * 1 TB hds during clean install Xenserver. 3 can be attached as removable storage ("SCSI"-drives for xenserver) to zfsguru-vm, installed iscsi target service in zfsguru-vm, created pool, created zvol for luns. one lun (500 GB for vm storage, 250 GB for second hd in windows 7 vm).

No issues. zfsguru works like a charm as it is simple with webgui. Transfer-rate better because "no lan traffic". Hint: no xenserver tools version 6.5 (but for 6.2) for BSD.

I can of course add a LUN as standart LVM over iSCSI storage for VM disks. But I want to use single LUN as single VM disk. Did you do that? — Valentin, Jul 19 '15 at 09:20
I tried it in the past - but never worked. I could not install any vm: CPU at 100 %, memory at 100% ... — newatrada, Jul 19 '15 at 17:32
Look at this - may be a solution for you: DRBD9 from linbit. And a short tutorial you can find: http://pve.proxmox.com/wiki/DRBD9 — newatrada, Jul 20 '15 at 11:49

score 1 · Answer 2 · answered Jul 24 '15 at 17:44

1

I ended up plugging iSCSI target directly from the VM, using iSCSI initiator & multipath.

answered Jul 24 '15 at 17:44

Valentin

21
1
1
6

XenServer, mapping SAN LUN to VDI

2 Answers2