macbook pro windows boot and virtualize off usb drive

Question

I want to raid two usb3 flash drives install windows onto it this new volume and be able to virtualize it or boot off of it. Any thoughts? I've heard parallels can run a bootcamp install virtually. Macbook Pro 13 retina

Answer 1

I know it's been three years, but this is extremely possible now. (Actually it was just as possible then too.)

Not just possible - but fast, stable, reasonably straightforward, and - dare I say - elegant.

Non-essential prerequisites

First off, you'll want a fast thumbdrive, such as a ScanDisk Extreme USB 3.0. This is surprisingly fast for this very scenario. Mirroring two is quite possible, though not really necessary.

Personally, I'd also not plug it/them in directly, but via shorty right-angle USB extension cable[s]. A) Flexible cable protects the dongle against breaking [e.g. picture the laptop situation with large USB dongles sticking straight out of the side], B) Removes the "hey look a dongle - unplug me!" allure - especially with right-angles, and C) Fewer people are going to come up and give you their expert advice, warning you against running a virtual machine off of a USB dongle. Somehow, off of a USB cable seems more legit or maybe that's just me. Heck, I'd just velcro them to the laptop lid (as I used to do with full-blown 2.5" spinning USB HDDs in a very similar raw vmdk configuration).

Steps

For the general steps, I'll assume VirtualBox, since that's what I'm most familiar with and you asked about - but it also works with Vmware and should also work with Parallels (but with different and varying degrees of scriptability).

1. Create a "raw" VMDK file that points to your thumbdrive. (The whole thing, not just a partition.) For two mirrored thumbdrives, do this for both. This is the most important and unintuitive step which basically results in magic - more on that later.

2. Create a new virtual machine, and attach the new VMDK[s] to it. Ideally, configure the VM with ICH9, I/O APIC, hardware acceleration, and SATA storage controller.

3. Download a Windows installation CD (preferably in ISO file form). Attach that to the virtual machine and put it first in the boot order.

4. Start the virtual machine and wait for the Windows installer to load (will go very fast if ISO file).

5. I don't remember the steps for making the windows boot drive a mirror - if you install Windows first and then use Disk Management to mirror it, or create the mirror during the install process. Either way, it is exactly as easy as installing to two mirrored real HDDs, because that's what the installer thinks it is working with. But the only mirroring solution that is possible (or at least advisable), in this setup, is native NTFS mirroring. And besides, for straight-up mirroring, native NTFS mirroring is superior to NTFS on top of any other mirroring solution [except for more exotic configs like NTFS on ZFS-backed NAS], because it has control of more of the stack, and mirroring results in virtually zero additional CPU or memory resources (unlike parity-based RAID).

Scripting to automate the hassle of invalidated VMDKs

Your raw VMDK[s] will become invalid after every host reboot, and every time you unplug/replug one or both thumbdrives. Fortunately, you can script the steps to recreate the raw VMDK[s] on the fly, while starting the VM. All of the commands below are done via VBoxManage, except one where noted. Generally:

1. Delete the virtual SATA controller from the virtual machine configuration (to work around the VBoxManage limitation of not being able to remove the virtual drive[s] individually).

2. Delete the actual VMDK file[s] from the file system (using regular file system delete command, e.g. "rm"). Don't worry, they are only small metadata pointer files, containing no user data.

3. Create the VMDK[s] anew, pointing to the physical location of thumbdrive[s].

4. Create a new SATA controller in the virtual machine configuration.

5. Attach the VMDK[s] from step 3 to this new SATA controller (be sure to include the flag to enable host caching).

6. Start the virtual machine.

Presto, boom! You are running an OS that thinks it is on a HDD, from mirrored thumbdrives, and avoiding the read/write penalties typical of VMs. Even without mirroring, it will be surprisingly fast and useful (though not nearly as fast as on an SSD of course). For me, Linux in a VM running off of a fast SD card this way, feels as fast as natively from a HDD. (With a reasonably fast computer, enough RAM, and full hardware acceleration including memory mapping.)

Also bootable as a real "host" OS?

You didn't state explicitly that this needed to be able to also boot as a "real" Windows installation. But if so, there are two big obstacles:

1. Getting a native Windows installation to alternate between booting natively, and in a VM - without complaining about activation and/or drivers - is it's own huge challenge (e.g. Same Windows installation as both a host and guest). Parallels on Mac OS neatly solves it with Windows on Bootcamp (or at least did at one time), but scads software engineering hours went into making that possible. It is possible with other virtualization solutions, but seems to be somewhat of a black art with few references.

2. It seems that Windows wasn't meant to be installed on a thumbdrive, and apparently won't let you if it can help it (i.e. it's not being tricked by virtualization). So we're getting into uncharted territory when installing it to USB (easy with virtualization and raw VMDK mapping), and running it natively. It very well may divide by zero and complode the universe.

I can only tell you this for sure: Installing Linux on a real thumbdrive or SD card, natively, is at least possible but very challenging. Installing Linux on a thumbdrive or SD card, via a VMDK-mapped "hard drive" in a virtual machine, then booting it natively, is trivially easy! And works really well - in fact, large distribution upgrades don't bat an eye when doing so running natively, knowing full well it's running on a thumbdrive or SD card, without the comfort of being tricked by a VM.

So in short, go for it with Windows in a VM on thumbdrive[s]. Just be prepared to have it all blow up when trying to boot to thumbdrive Windows natively! But if it works, please share your results and you'd be hailed as a wizard.

What really makes this magic work, is running the VM on a raw VMDK. This is crucial:

1. It makes things a heck of alot easier, installation-wise.
2. It is very fast: it eliminates the big performance hit of double I/O, as only the underlying host handles the physical device I/O. (Rather than the guest doing "physical" I/O to the virtual HDD file, and then the host in turn doing physical I/O to the underlying device. Instead, the hypervisor intercepts the virtual writes, and sends them directly to underlying storage.)
3. VirtualBox at least has a big problem properly passing USB through to the VM (it's a tough technical challenge to crack), which makes booting to USB storage (including SD card readers) very problematic. With raw VMDK, it is very stable, with no awful USB kludges in the storage stack.

There are other possible solutions to mapping an OS to thumbdrive, raw VMDK isn't the only one:

1. USB passthrough. As mentioned before, very unstable, kludgey, and not advisable. If you go this route, at least use the raw VMDK solution to do the actual OS installation - then remove the VMDK and add it back in via USB passthrough.
2. Regular virtualized disk image file, stored on the thumbdrive. This solution will make installation even a little easier than the raw VMDK approach, and is easier to manage (you don't have to remap the raw VMDK every time). It's also far more portable - you can store the VM definion itself on the thumbdrive, along with its virtualized disk image file (and even a virtualbox installer executable), and arbitrarily move it around different host computers. (Though some extra work would be required to run a Windows VM on arbitrary hosts, as VirtualBox by default passes through host machine metadata.) But the big drawback is double I/O, which slows things to a crawl on a thumbdrive. But maybe that's worth it to you in exchange for being alot easier to manage (at least without scripting), and more portable. Also, this solution will not allow you to boot anything natively - it's just some regular data files on a thumbdrive.