Bare metal or virtualize?

Question

I am looking to upgrade servers and am trying to figure out a good plan.

We currently have 4 servers:

1. OpenBSD firewall/VPN server
2. FreeNAS backup servers (local) that receives ZFS snapshots
3. FreeNAS backup servers (remote) that receives ZFS snapshots
4. The workhorse FreeBSD server below.

FreeBSD Server

~2010 FreeBSD 8.4, 32gb ram, dual Xeon E5520 ZFS (8 disks, zraid of disks in mirrored pairs, 8TB)

Services:

• Samba
• Netatalk (Apple filesharing)
• Apache (mostly internal, some external facing sites)
• MySQL
• VirtualBox (Windows 2k3 instance)
• ZFS snapshots

My Plan (basic)

I am planning a server upgrade that would have us switch from one primary server to two servers that would each take some of the server duties from the list above (and would replicate to each other) so that if one goes down, I could rapidly activate all features on the second. Something like:

Server 1:

• Samba
• Netatalk (Apple filesharing)
• VirtualBox (Windows 2k3 instance)
• ZFS snapshots

Server 2:

• Apache (mostly internal, some external facing sites)
• MySQL
• ZFS snapshots

I've only ever run bare metal, and I have no experience with VMs other than running Windows 2k3 on VirtualBox. Should I look at running my server instances as VMs? I thought that might make restoring from a crash easier. In general, does this seem like a good plan?

I've been looking at ixSystems servers and Dell rack hardware, if that makes a difference. (I also have never used any rack mount equipment.)

Answer 1

No question, virtualize. The benefits and flexibility afforded by virtualization far outweigh the negligible performance hit.

Your plan, though is sub-optimal, primarily because Virtualbox is a desktop-grade virtualization solution and is not intended for server usage.

Here's what I'd suggest: install (free) VMware ESXi on both servers, then create VMs on them as needed. If you don't care for ESXi, then consider Hyper-V or KVM. Leave the host OS/hypervisor as "clean" as possible, responsible only for running your VMs, and create VMs as needed. Don't run any application processes on the host OS.

If you have some budget for this, pick up the VMware Essentials Plus bundle, which gets you vCenter, which will allow you to do things like live VM migrations between hosts, centralized management, backups using tools like Veeam, etc.

Once you move to a virtualized environment, you'll never go back.

Answer 2

Virtualizing will help with your needs tremndously.

We have a small business and virtualizing allows us to consolidate hardware, increase segregation of services to help with security, and helps with uptime because we can migrate VMs between hosts (hypervisors) very easily; something that is very difficult with baremetal.

We use dated enterprise hardware in pairs with backup parts kept on site (fans, drives, etc) but have both a primary and secondary host; Our host runs VMs serving DRBD, Apache, MYSQL, Samba, NFS, Reslio Sync, Dropbox, etc. We let our host manage RAID arrays using MDADM. Utilizing DRBD the VMs are kept in sync on a backup server so downtime is almost a non-issue even with a catastrophic hardware failure on the primary host.

But being a small business it simplifies hardware management, allows us to run less hardware which has far reaching implications on budget and IT resources, and consolidates the management of our services because it is natural to administer all the VMs from a single console on a workstation; for us through Xen Center as we use XenServer.

Further it allows us to segregate things so things like cloud services can be virtually segregated from internal services providing a high degree of security. For example we serve two separate cloud file services in two separate VMs; one for field personel accessible via mobile devices and one for office personel accessible via the internal network.

As a note, our backup server (not secondary host) is not virtualized purposefully so we have baremetal access to our files in case of a software or configuration failure with our host. That is if our host corrupts our VMs or data stores somehow we have baremetal access to the files and VMs still.

In the end we can provide our company with enterprise grade file, backup, web, cloud, and other services all in house for minimal cost and maximum uptime. It also allows us to expand as we can integrate other services; planned in the near future are VPN services for remote book keepers and Android form services for field personel that needs a windows software intermediary to interface with MySql. Without virutalizing we would need to buy, run, and administer more hardware; virtualizing has eliminated the problem of hardware all together when adding such services and we can simply focus on integration of the software/service which can be daunting enough.

Answer 3

If most of those machines are mostly idle most of the time, virtualizing them has a lot of cost and time saving benefits, as others have already pointed out. You should be aware that there is a significant performance impact from virtualization, as per my measurements in that article. Virtualbox in particular is pretty dire, so you would so well to make sure that you migrate everything to a proper hypervisor. I use KVM and these days performance is acceptable even for workloads as intensive as virtualized gaming.

Edit: part 2 of the article on virtualization performance overhead, on hardware of the following three generations from the original article, still showing a similar performance hit.

Edit 2: Virtualization performance impact on Zen 2 Epyc still amounts to 17% - 25%.

To reduce the performance impact of virtualization, you can keep a lot of the workloads you listed out of VMs, e.g. you can run your samba/netatalk servers on the host. If at all possible, run MySQL on the host's ZFS, with appropriate tuning of MySQL and ZFS. You can find slides from a lecture I gave on the subject of MySQL on ZFS here.

Another benefit you will get from virtualization is that you can back your VMs by zvols, which makes backups and restores for guests that don't natively support ZFS far less painful (it is truly refreshing being able to just zfs rollback a machine when it gets out of shape after a broken update).