12

Lets say we have one server with lxc installed, and a lxc container used for as a base img /var/lib/lxc/ubuntu_base. For simplicity let's forget the config changes after copying the base img.

some people suggest using subvolumes and snapshots for making new containers, but one could easily do cp --reflink with simmilar results.

So what is the propper way (or which is better) for managing multiple containers?

  • snapshots

This way seems best, but commands like lxc-destroy won't work since it won't be able to delete the directory.

btrfs subvolume snapshot /var/lib/lxc/ubuntu_base /var/lib/lxc/container_1
  • cp with reflink

I am not sure if there is any performance difference between this or snapshots

cp --reflink=always /var/lib/lxc/ubuntu_base /var/lib/lxc/container_1
  • or Is there any other better way of doing this that I am not aware of.

edit:

One thing I've seen with the reflink option is, that you can't delete the base container if others are running, because the /proc and /dev are mounted and never changed, se the reference is always the same. But shutting down all the coppied containers seems to help.

zidarsk8
  • 384
  • 1
  • 3
  • 12
  • I've used the btrfs snapshot feature to create new containers - and it works well (pretty quick provisioning etc). However, btrfs has a per-subvolume page cache - so although using snapshots is quick/disk space efficient, you're likely to end up having multiple copies of what's effectively the same binary in memory. – David Goodwin Mar 10 '17 at 20:21

3 Answers3

3

if you will use btrfs subvolumes for lxc, you need add the option user_subvol_rm_allowed in your /etc/fstab. Example extracted from one fstab file:

UUID=XXXXXXXXXXXXXXXXXXXXXX / btrfs subvol=@,user_subvol_rm_allowed,defaults 0 0

the option will permit that you can remove subvol without be root, only normal user. This capability is used by lxc when the snapshots go in btrfs subvolumes

Yonsy Solis
  • 284
  • 1
  • 9
3

Guess it depends on how big your base image it is. I'd probably lxc-create a new container and use Salt/Puppet etc to provision my containers and only lxc-clone for relatively bigger containers (e.g. dev containers with lots of tools installed and configured).

Note that lxc-clone will use the same backing store as the source. So to use subvolume you will need to create your containers with "-B btrfs". For example:

lxc-create -B btrfs -n mycontainer -t ubuntu

Then clone it with:

lxc-clone -s mycontainer mynewcontainer

In case you are using zfs to store you containers, there is an extra --zfsroot option to lxc-create so you can choose a zpool other than the default "tank". For example:

lxc-create -B zfs --zfsroot=data/lxc

Share and enjoy!

Lester Cheung
  • 659
  • 4
  • 11
  • I am on debian wheezy with BTRFS. It appears that the option `-B btrfs` is used by default - I assume because my debian is set-up with BTRFS. Actually, I created my first container (a 32 bit debian wheezy) without this option, and a subvolume was created for it. – lalebarde May 30 '15 at 20:21
  • @lalebarde that's probably `-B best` in action but good to know! – Lester Cheung Nov 12 '15 at 06:41
2

I am on Ubuntu LTS 14 and just ran the following (for first time even) and it worked like a charm:

lxc-stop -n ubuntu_base
lxc-clone -o ubuntu_base -n ubuntu_base_c1 -s
lxc-start -n ubuntu_base_c1 -d # make changes if needed
lxc-stop -n ubuntu_base_c1
lxc-snapshot -n ubuntu_base_c1

Using -s with lxc-clone will take a snapshot if backing store is btrfs (in your case).

Verify new clone/snapshots with

lxc-ls -f
btrfs subvolume list /var/lib/lxc

Hope that helps!

dapithor
  • 36
  • 2
  • 1
    Note that these days you need to specify _btrfs_ as the backing store when you run _lxc-create_ or _lxc-clone_ will give you a _overlayfs_ instead. – Lester Cheung Mar 31 '15 at 11:13
  • To just create a new "light" container, it seems that the two first lines are enough (lxc-stop and lxc-clone), thanks to the `-s` option, and that the three last ones (lxc_start, lxc_stop, lxc_snapshot) are just an helper process to manage future container evolutions. Is it correct ? – lalebarde May 30 '15 at 20:43