9

Does it make any sense to run a virtual machine with a "master" guest,
and in that master guest run many other guests?

Has anyone tested this? Is it even possible?
Are there better ways to accomplish my goals? (Read on.)
(I googled for "guest in guest" and found nothing)

What I want to do: I'd like to set up and test various virtual networks, with database and application servers, and test different firewall, DNS server, database server configs, etc.

I could do this by running guests directly on my host computer, but then I might need to modify the network config on my host (e.g. configure dnsmasq?), depending on what network setup I'm about to test. I'd also need to start each guest individually. Whereas if all guests run inside a master guest, then I can config the network on that master guest, and I need only start and stop that master guest, and take snapshots of it, to implicitly start & stop & snapshot all guests running inside it.

(I would have many master guests, probably only one running at a time.)

I'm using Ubuntu 11.4 and KVM, with hardware virtualization support (AMD-V).
Do you think the guests-in-the-guest would benefit from virtualization?

Update: I should have googled for "Nested Virtualization" not "guests in guest". Now I find lots of links :-) E.g. this example of actually migrating a running VM from the host machine to a nested VM host: VMotion from physical ESX 4 to virtual ESX 4
You who replied, thanks for mentioning "Nested Virtualization" :-)

Update: Performance info, for AMD processors, from 2008: [...] up until now, when kvm virtualizes a processor, the guest sees a cpu that is similar to the host processor, but does not have virtualization extensions. This means that you cannot run a hypervisor that needs these virtualization extensions within a guest (you can still run hypervisors that do not rely on these extensions, such as VMware, but with lower performance). With the new patches, the virtualized cpu does include the virtualization extensions; this means the guest can run a hypervisor, including kvm, and have its own guests. (thanks "wzzrd")

Update: Performance info on Intel processors, from 2011: "There ist no support for nested virtualization with intel vmx in the current version of kvm in the Ubuntu repositories. With the newest patches for kvm it is possible, but there are still in development"

KajMagnus
  • 375
  • 1
  • 4
  • 14
  • 3
    Keep a spinning top handy. – nedm May 27 '11 at 17:19
  • Nested virtualization is possible, but you double your CPU-related performance penalty, waste RAM because in the default configurations, both the host, master guest and "guest" guests will each other cache the same data multiple times, and IO performance will be disastrous for data that wasn't cached. – André Borie Oct 06 '15 at 00:42

8 Answers8

7

What you want to do is possible, it's just not really practical from a performance point of view.

user9517
  • 114,104
  • 20
  • 206
  • 289
6

ESXi can virtualize itself. So you can run ESXi as a guest on ESXi (or ESX for that matter).enter image description here

Directions found here

JamesBarnett
  • 1,129
  • 8
  • 12
3

  1. Its possible, read more here - http://blog.jasonruiz.com/2011/01/24/kvm-nested-virtualization-support/

  2. the current purpose of nested virt is testing, but I think it meant to test hypervisors not databases, dns, etc...

in order to test such services I think "normal" virtualization will do.

you can have a few different virtual networks, with different dhcp/dns/routing services.

John
  • 391
  • 1
  • 4
3

I don't know about other hypervisors, but it's definitely possible with ESX/i, as discussed here. Also, it's definitely not possible with Hyper-V, as discussed here.

Howewer, where this is possibile, it's totally unsupported and also quite useless, unless for testing purposes. I personally did it with ESX to test a VMware cluster, which required at least two ESX hosts, a Virtual Center machine and some shared storage; I didn't have more than one server nor any "real" storage available at that point, so I used virtual ESX hosts with a shared virtual disk (just like I would have done to test a Microsoft cluster). It worked, and it was also not terribly bad at performance. But I can't think of any reason in the world to use this kind of setup in a production environment.

Community
  • 1
Massimo
  • 68,714
  • 56
  • 196
  • 319
0

kvm within kvm will not work, though the inner guests can run on pure qemu, in emulation mode.

I'd leave the VMs on a single host, without overcomplicating things

dyasny
  • 18,482
  • 6
  • 48
  • 63
  • Untrue. It is very possible and has been since 2008 or so. Link: http://www.linux-kvm.com/content/kvm-nested-virtualization-works – wzzrd May 27 '11 at 11:15
  • 1
    yes, with very specific CPU instruction sets, and nowhere near production or serious testing environments. The T/S wants to test network configs and services, not additional hypervisors, so why complicate things this far? Especially since the standard management tools. You're right on the account of kvm within kvm _can_ work, under certain conditions, that was my mistake, but to answer the original question - this is not the best idea under any hypervisor – dyasny May 27 '11 at 11:31
  • "this is not the best idea under any hypervisor": now *that* is true :) here, have an upvote :) – wzzrd May 28 '11 at 20:01
  • LOL, not that an upvote is that important :) – dyasny May 28 '11 at 20:27
0

KVM probably is not the best option for your purpose, you should try Virtualbox.

With Virtualbox, you can make a virtual machine inside a virtual machine, and the network tools are very convenient.

erickzetta
  • 579
  • 2
  • 4
  • 1
    Rubbish. KVM will suffice perfectly. – wzzrd May 27 '11 at 11:16
  • @wzzrd I didn't say KVM is not suffice. I said Virtualbox may be a better option, in this case, since it's a complex configuration. – erickzetta May 27 '11 at 11:48
  • Actually, I like configuring KVM VMs with `virsh` :-) – KajMagnus May 27 '11 at 16:17
  • [VirtualBox does not support nested virtualization.](https://www.virtualbox.org/ticket/4032) – Michael Hampton Dec 04 '12 at 19:52
  • support for VMX, second level address translation (SLAT), para-ops, nested pages, machines within machines--this was a feature introduced way back in the 2.0 major release of VirtualBox. https://www.virtualbox.org/wiki/Changelog-2.0 – Bent Cardan Aug 11 '14 at 00:41
0

Have you checked out XenServer? The new beta version is called "XenServer boston" Read more about it here:

http://blogs.citrix.com/2011/05/18/xenserver-boston-beta-program/

In the release notes, you will see the following:

Virtual Appliance support. Within XenCenter you can create multi-VM virtual appliances (vApps), with relationships between the VMs for use with the boot sequence during Site Recovery. vApps can be easily imported and exported using the Open Virtualization Format (OVF) standard

Using this feature combined with Virtual Lans, you will be able to accomplish what you want. I'm not positive, but I believe you can also create a snapshot of all the Vm's in the Virtual Appliance at once.

Please let me know if you have any further questions about XenServer (I use it 2x every day)

Skyhawk
  • 14,149
  • 3
  • 52
  • 95
benathon
  • 472
  • 2
  • 12
-1

Whether it's possible or not is irrelevant. It just doesn't make sense. You gain absolutely nothing by nesting VMs, while losing a lot.

John Gardeniers
  • 27,262
  • 12
  • 53
  • 108
  • Actually, the/some KVM developers think it makes sense: on the KVM dev mailing list, there's [this thread](http://thread.gmane.org/gmane.comp.emulators.kvm.devel/21119) with a patch for nested virtualization support. I think the developers seems fairly happy, if you continue reading some replies. – KajMagnus May 27 '11 at 16:01
  • A non-KVM developer who likes nested virtualization: *"Another use is to have multiple virtualized networks inside one machine. Each network would have a different configuration of hosts. I've been waiting for this."* (From [here](http://avikivity.blogspot.com/2008/09/nested-svm-virtualization-for-kvm.html?showComment=1224091620000#c1350634817205522377).) – KajMagnus May 27 '11 at 16:02
  • @KajMagnus, just because people want it doesn't mean it makes sense. Some people simple can't see the wood for the forest. – John Gardeniers May 27 '11 at 23:25
  • Makes sense for testing scenarios. Want to deploy a test hypervisor deployment just to see how a particular feature behaves in a new version? Spin up a VM. You're correct that it's not much use for OP's particular usage, though. – Chris Thorpe Aug 15 '11 at 23:31
  • The IBM VM folks used to do this all the time when developing new versions of VM. There's a great [paper](http://www.leeandmelindavarian.com/Melinda/25paper.pdf) on it (eg p. 28), if you're interested. – fission Dec 05 '12 at 07:43