How to use more than 3 virtual disks in Linux using CentOS and XenServer

Question

I've attached 5 virtual disks to a Virtual Machine in Citrix XenServer. The VM has the xs-tools installed. Initially it said that it couldn't add so many disks. After I installed the xs-tools, it let me add all the disks.

But /dev doesn't show all the disks. It shows these:

/dev/xvda
/dev/xvdb
/dev/xvdc
/dev/cdrom

Perhaps it is bound by the limits of an IDE bus? (3 disks + CD-ROM) If so, how does one change the VM to use SCSI?

Edit: According to the documentation:

2.6.3. VM Block Devices

In the PV Linux case, block devices are passed through as PV devices. XenServer does not attempt to emulate SCSI or IDE, but instead provides a more suitable interface in the virtual environment in the form of xvd* devices. It is also possible to get an sd* device using the same mechanism, where the PV driver inside the VM takes over the SCSI device namespace. This is not desirable so it is best to use xvd* where possible for PV guests (this is the default for Debian and RHEL).

For Windows or other fully virtualized guests, XenServer emulates an IDE bus in the form of an hd* device. When using Windows, installing the Citrix Tools for Virtual Machines installs a special PV driver that works in a similar way to Linux, except in the fully virtualized environment.

Still, with 5 virtual disks attached, I don't see the other xvd devices.

Edit #2: (attached requested info)

Host Machine:

XenServer 6.1
Linux version 2.6.32.43-0.4.1.xs1.6.10.777.170770xen (geeko@buildhost) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-51)) #1 SMP Wed Apr 17 05:52:03 EDT 2013

Guest Machine:

CentOS release 6.4 (Final)
Linux version 2.6.32-358.6.2.el6.x86_64 (mockbuild@c6b8.bsys.dev.centos.org) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Thu May 16 20:59:36 UTC 2013

Output of 'fdisk -l' on Guest Machine:

Note, the disk beyond the first 3 attached are not displaying -- there should be 4 100GB disks. (There are a total of 5 disks displayed in XenCenter -- 16GB, 100GB, 100GB, 100GB, 100GB)

Disk /dev/xvdb: 107.4 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xfb6c95b9

    Device Boot      Start         End      Blocks   Id  System
/dev/xvdb1               1       13054   104856223+  83  Linux

Disk /dev/xvda: 17.2 GB, 17179869184 bytes
255 heads, 63 sectors/track, 2088 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000e5f41

    Device Boot      Start         End      Blocks   Id  System
/dev/xvda1   *           1          64      512000   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/xvda2              64        2089    16264192   8e  Linux LVM

Disk /dev/xvdc: 107.4 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xed249ced

    Device Boot      Start         End      Blocks   Id  System
/dev/xvdc1               1       13054   104856223+  83  Linux

Disk /dev/mapper/vg_blue-lv_root: 14.6 GB, 14571012096 bytes
255 heads, 63 sectors/track, 1771 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/vg_blue-lv_swap: 2080 MB, 2080374784 bytes
255 heads, 63 sectors/track, 252 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

I see that the Linux versions say SMP. The Guest VM doesn't say "xen" in the name. However, I have already run yum install kernel-xen. Could be a clue?

Please provide CentOS, Linux Kernel & XenServer Version. Could you also add the output of "fdisk -l"? — Striker_84, Jun 06 '13 at 14:33
Have you rebooted your VM after citrix xs-tools installation? Normally xs-tools will install a xen-enabled linux kernel, therefore also check your bootloader configuration (xen-kernel entry). — Striker_84, Aug 14 '13 at 06:44
This is just a guess, but I run into this occasionally and the errors are similar. Verify you have increased the number of loop devices allowed. It may seem unrelated but the instantiation of a VM can be blocked by this. Verify you have increased the loop devices in /etc/modprobe.d/loop.conf using options loop max_loop=64 — Aaron, Jan 02 '15 at 01:43

score 1 · Answer 1 · edited Aug 20 '13 at 13:56

Same problem here. I think you will see error message in /var/log/messages

blkfront: your disk configuration is incorrect, please use an xvd device instead

Disks in GUI are /dev/hda, /dev/hdb, /dev/hdc, but on RHEL < 6 it is /dev/xvda/ etc... Disk can be added manually from cli, if you name it as /dev/xvd .

Found this: http://forums.citrix.com/message.jspa?messageID=1692261 Check Albert Czarnecki answeres.

"As like I said your machine is running in HVM mode. You must run command and change from HVM to PV: xe vm-param-set uuid=your_machine PV-bootloader=pygrub HVM-boot-policy="" and then run again machine."

Check also https://www.linuxnet.ch/xenserver-howto-convert-hvm-to-pv-rhelsuse/

Edit: I tried that, and now all disks are visible and system is PV .

Quick summary (may vary):

xe vm-list name-label=SERVERNAME params=uuid 
uuid ( RO)    : 2a596adf-ccf4-6469-cf89-702d723cfbb9

UUID=`xe vm-list name-label=SERVERNAME params=uuid | cut -d ":" -f 2`

Check if PV or HVM

xe vm-param-list uuid=2a596adf-ccf4-6469-cf89-702d723cfbb9 | egrep "PV-bootloader|HVM-boot-policy"
xe vm-param-list uuid=`echo ${UUID}` | egrep "PV-bootloader|HVM-boot-policy"
HVM-boot-policy ( RW): BIOS order
PV-bootloader ( RW): 
PV-bootloader-args ( RW):

Set PV:

xe vm-param-set uuid=2a596adf-ccf4-6469-cf89-702d723cfbb9 PV-bootloader=pygrub HVM-boot-policy=""
xe vm-param-set uuid=`echo ${UUID}` PV-bootloader=pygrub HVM-boot-policy=""

After reboot:

fdisk -l 2>/dev/null | grep xvd
Disk /dev/xvdb: 51.5 GB, 51539607552 bytes
Disk /dev/xvde: 1073 MB, 1073741824 bytes   <------ YEAAAAAH "test disk 1GB visible for os"
Disk /dev/xvda: 16.1 GB, 16106127360 bytes
/dev/xvda1   *           1          13      102400   83  Linux
/dev/xvda2              13         274     2097152   82  Linux swap / Solaris
/dev/xvda3             274        1959    13528064   8e  Linux LVM
Disk /dev/xvdc: 1073 MB, 1073741824 bytes

I just successfully finished migration of 7 RHEL 6 servers from HVM to PV. Does exists better template from Citrix ? — Amateur, Aug 21 '13 at 14:08
Interesting, mine has both the HVM-boot-policy and the PV-bootloader set. I cleared the HVM-boot-policy and rebooted. Now I just have the PV-bootloader=pygrub. But it's not working yet. I'll keep messing with this. — 101010, Aug 22 '13 at 19:14

How to use more than 3 virtual disks in Linux using CentOS and XenServer

1 Answers1