23

Standalone ESXi (4.1) host without any vCenter Server.

How to backup virtual machines as quickly and storage-friendly as possible?

I know I can access the ESXi console and use the standard Unix cp command, but this has the downfall of copying the whole VMDK files, not only their actually used space; so, for a 30-GB VMDK of which only 1 GB is used, the backup would take 30 full GBs of space, and time accordingly.

And yes, I know about thin-provisioned virtual disks, but they tend to behave very badly when physically copied, and/or to blow up to their full provisioned size; also, they are not recommended for actual VM performance.

It is ok for me to shut down the VMs before backing them up (i.e. I don't need "live" backups); but I need a way to copy them around efficiently; and yes, a way to automate shutdown/startup when taking a backup would also help.

I only have ESXi; no Service Console, no vCenter Server... what's the best way to handle this task? Also, what about restores?

Community
  • 1
Massimo
  • 68,714
  • 56
  • 196
  • 319

8 Answers8

20

My preferred solution for this is to simply export them to an ovf or ova file using either the vSphere client or the command line ovftool.

In the vSphere Client, make sure the VM is off, then highlight it and go to File->Export->Export OVF Template. Then just follow the prompts.

Restoring is a piece of cake, just do the reverse (the menu option is "Deploy OVF template", I think).

To create a thin backup using ovftool

ovftool -dm=thin  vi://<user>@<esxi-host>/<vm-name> <local-file>.ovf

You may also wish to check out some of the options at http://www.virtuallyghetto.com/, I know these are very popular and I think there are some good choices for backups, although I haven't looked at any of them too recently.

CPrescott
  • 15
  • 5
Jed Daniels
  • 7,172
  • 2
  • 33
  • 41
  • OVF backup and restore seems to work great as a poor man's VMWare backup option. Eventually I intend to automate using the cli ovftool, but shutting down the vm inside the script while the backup runs will be the tricky part. – steampowered Nov 24 '15 at 16:33
  • OVF is very slow. If you want to avoid downtime, you can use the web client to clone a running VM with paying for vMotion, then shutdown and backup the clone. No downtime. – steampowered May 12 '17 at 02:22
6

I ended up writing a script which copies the VM configuration files and uses vmkfstools -d to clone the VMDKs while preserving the thin provisioning.

For reference:

#!/bin/sh

if [ $# != 2 ]; then
        echo "Usage: $(basename $0) <SOURCE VM PATH> <DESTINATION PATH>"
        echo "Example: $(basename $0) /vmfs/volumes/datastore1/VM1 /vmfs/volumes/datastore2"
        exit
fi

vmx=$(basename $(/bin/ls $1/*.vmx))
name=$(grep displayName $1/$vmx | /bin/awk -F\" '{print $(NF-1)}')
vmxf=$(grep vmxf $1/$vmx | /bin/awk -F\" '{print $(NF-1)}')
nvram=$(grep nvram $1/$vmx | /bin/awk -F\" '{print $(NF-1)}')
vmdks=$(grep vmdk $1/$vmx | /bin/awk -F\" '{print $(NF-1)}')

echo "Started copying VM $name"

vmdir=$(basename $1)
destpath="$2/$vmdir"

echo "Source path: $1"
echo "Destination path: $destpath"

echo "Creating destination path $destpath"
/bin/mkdir -p $destpath

echo "Copying configuration files:"
echo $vmx
/bin/cp $1/$vmx $destpath
echo $vmxf
/bin/cp $1/$vmxf $destpath
echo $nvram
/bin/cp $1/$nvram $destpath

echo "Copying virtual disks:"
for vmdk in $vmdks;
do
        echo $vmdk
        /sbin/vmkfstools -d thin -i $1/$vmdk $destpath/$vmdk
done

echo "Completed copying VM $name"

This requires the VM to be powered off and have no active snapshots.

Massimo
  • 68,714
  • 56
  • 196
  • 319
  • Hi, what about thick lazy? – rovshango Sep 09 '15 at 19:22
  • Great script @Massimo, but I had to add double quotes to vmx assignment to avoid basename syntax error: vmx=$(basename "$(/bin/ls $1/*.vmx)") . But now, if $1 path contains spaces, it crashes because $1 is not quoted. I could not get rid of this problem. Any idea? – Blazeag Dec 14 '17 at 18:28
5

I don't know if this fits the bill for you, but VM Explorer does a nice job of performing hot or cold backups of virtual machines. I believe that with ESXi 4.1 VM Explorer allows you to perform VM guest backups from one host to another host as well.

THE JOATMON
  • 266
  • 8
  • 29
joeqwerty
  • 108,377
  • 6
  • 80
  • 171
4

I would try to use a proven solution of some sort rather than roll your own. The reduced hassle, time, and risk will readily pay for itself even if you opt for a solution that isn't free. All of these issues you are concerned about are addressed in any modern backup solution for ESXi.

The solution that a client uses in their ESXi environment with good luck is Veeam. There is even a free edition that may work for your needs: https://www.veeam.com/virtual-machine-backup-solution-free.html

Josh Richards
  • 161
  • 3
2

Ghetto VCB can do the backup while the machine is running. For the space you can use a deduplication+compression filesystem like lessfs on the backup server.

Mircea Vutcovici
  • 16,706
  • 4
  • 52
  • 80
2

XSIBackup is an ESXi service that runs in the ESXi hypervisor OS and has a minimal footprint. In fact your users will not notice there's a backup taking place in the background.

It offers two different tools: XSIBackup-Free and XSIBackup-Pro.

XSIBackup-Free offers all the features present in the Pro version, such as:

  • Hot backups
  • Cron scheduling
  • Instant differential backups (OneDiff)
  • Native block level deduplication plus compression to local datastores and to Linux servers (XSITools).

The Pro version offers all of the above plus extended tools that will speed up your data transfers (XSIDiff), backup certification via checksum, quick file comparison methods and an SSH GUI (Graphical User Interface) that allows to configure all options from a wizard, speeding up the setup of backup jobs.

https://33hops.com/xsibackup-vmware-esxi-backup.html

Daniel J.
  • 214
  • 1
  • 5
0

I'm exporting to OVF format, too. It's fine because it compress on the fly the image with gzip.

But the licenses are gone, because with importing an OVF file into an ESXI a machine with new hardware is created.

Solution:

I backup (download) the file folder of the virtual machine without the big image and compress them.

Both - ovf folder and compressed copy of virtual image files (without big image) - are in the same folder.

Restoring:

-> Importing OVF into ESXI and upload the uncompressed config (from backuped file folder) -> Virtual Machine is fine again

ismail

Ismail
  • 1
0

cpio can copy sparse files while preserving the "holes" in them.

MattBianco
  • 587
  • 1
  • 6
  • 23
  • AFAIK, there wasn't any tool that could handle sparse.vmdk files natively, apart from vmkfstools, which is limited to copy files within a local file system. That's why we created (c) XSIDiff as part of our suite of tools for ESXi. It copies only used blocks as vmkfstools does, but you can copy data over SSH and also as a TCP/IP client/server app. You can download a free working version here: https://33hops.com/download-xsidiff-trial.html – Daniel J. Aug 09 '17 at 17:38