Ganeti vs Proxmox

Question

I'm system administrator in small software house. I'm going to virtualise our servers. The main reason for doing this is providing highest possible uptime, but probably it will also increase resources utilization.

We have two servers. On one we have few development VMs and also its used as build server (Jenkins Master, but also Build Executor). On the 2nd one we got few some critical services (code repositories, issue tracker).

I would like to use those machines for creating two node clusters and create VMs for each service. I would like to use DRBD so moving machines between nodes was possible.

After some research my candidates are Proxmox and Ganeti. Which one would be better in my situation? I love Proxmox's simplicity (especially installation simplicity), but maybe there is valid reason for using Ganeti?

Answer 1

I'm not familiar with ProxMox but I am familiar with Ganeti as I use it in my job and work with the core developers.

If you have a small number of physical machines and VMs then doing things by hand is fine. You can memorize which VM is on which physical machine and run the right Xen command on the right node. If you only have one physical machine then you always know which machine your VMs are on.

To a new user, the most obvious benefit of Ganeti is that it knows where all the VMs are. From one master you can issue commands like "stop instance foo" and it looks up with foo is running and stops it. You can issue a command like 'create a VM of size X wherever there is free space' and it will find that free space and create the VM. If you have a lot of churn (creating and deleting VMs) Ganeti will save you a lot of time. It also helps you in a disaster: if the entire cluster loses power, Ganeti will re-start all the VMs when the cluster gets power and boots. (An example is here: Power Outage: A true test for Ganeti)

You can set up "templates" for VM configurations (what OS, disk storage configuration, etc.) Thus, maybe you are the only person expert enough to create a template but one they exist other more junior sysadmins on your team can easily create new instances.

The problem with doing things by hand when you have a small number of VMs is that if you are successful you will grow to the point that Ganeti would make sense. At that point you might have a crush of work and you won't have time to install Ganeti. This is a general problem with moving from doing things manually to installing automation: You don't have time to fix the leaking sink because you're too busy mopping the floor. To prevent this, I recommend installing Ganeti (or whatever automation you select) sooner rather than later.

Ganeti is 100% open source, requires no licenses. However, there is no commercial support available if you need it. I am not familiar with ProxMox but their web page says it requires a license; there are parts that are Open Source but it is not available without a license (from what I read on their web site). They have support plans listed on their website.

Ganeti has been available outside of Google since 2007. It has 5+ years of engineering behind it. It has been stable for quite some time. You can find Ganeti source code and the support community on code.google.com/p/ganeti I do not know how recent ProxMox was released.

Answer 2

I've been using Xen since 2003. I've been using Ganeti for about a year now, and would highly recommend it. I've never heard of ProxMox.

For years I manually managed LVM, DRBD and Xen configuration files in several clusters ranging in size from 2 to 5 nodes -- that was a nightmare. DRBD made things wonderfully reliable, and Xen made VMs easy to migrate between nodes, but whenever a node failed, I was in for a long, tedious process of creating new LVM volumes, getting DRBD configured and the mirrors rebuilt, then making sure Xen and the other configs were replicated... for a 5-node cluster, keeping things straight in my head could sometimes be painful. I kept wanting to script it all, but life kept getting in the way.

Then Ganeti came along. It was written for exactly my use case -- n-way clusters using Xen and DRBD. Those hours of painful, risk-filled operations are now compressed in some cases into a single ganeti subcommand. I love it. I've migrated most of my VMs into one big 10-node ganeti cluster now.

I've also migrated our Windows desktops into a Ganeti/KVM cluster -- Linux only on the desktops now, and they use VNC to get to the Windows VMs. The big advantage here is easier snapshots and backups, and the ability to sandbox each Windows app into its own VM.

By the way, even though you're talking about only a 2-node cluster right now, I'm pretty sure that, once you see the utility, you'll find yourself adding nodes and migrating everything you can into virtual machines. If your clustering platform can support more than 2 nodes, like Ganeti, then you'll be able to do that.

Answer 3

I don't use ganeti, but I use proxmox and I know it very well. Proxmox VE is a FULLY OPEN SOURCE project and all the code is available with the binaries. The base system is a Debian 64 minimal distribution with a custom installer, but you can also install your Debian system and add the proxmox repositories. One of the project main sponsor is the Autrian governament and this ensures that the develop of the product will be financed for a long time. Starting with the version 2.0 even the development process is open (before it was a closed list and you had to request the access to the company behind proxmox).

It features a user friendly interface, with almost all the feature found on vSphere, cluster support (based on corosync), HA for VM high availability, automated backup and restore support from interface. It supports also container virtualization with openvz, seamlessy integrated in the same infrastructure and interface of KVM, so you can choose the best virtualization strategy based on your needs. I have about 20 installations of Proxmox around, from single host, to 2-node cluster with drbd arriving to a 5 node cluster on FC SAN (>80VM).

For what I can see about ganeti, I can state that there is no competition. Proxmox wins for sure.

Answer 4

My complaint (possibly based on ignorance) about ganeti: I'm happy to do things at the CLI if necessary, but I don't believe wanting a GUI makes me a wimp. There seems to be a web-based ganeti management tool, but last I looked, it was VERY far from being plug&and play (e.g. a couple of pages of commands to type in on the host), and in a couple of attempts on a mock server, I never did get it to build and install correctly.