9

I'm using Docker Toolbox on OSX.

I've created a data volume container for storing persistent data: https://docs.docker.com/userguide/dockervolumes/#creating-and-mounting-a-data-volume-container.

I checked that this data is indeed stored on the boot2docker VM (created by docker-machine) and not on the container, so that it will persist. However "docker-machine restart " clears out this custom data on the vm.

I can't find documentation on what is happening. I found one forum post mentioning that data in /var/lib/docker will be preserved, but I could not find any official docs stating that and it also seems strange considering the persistent storage guide above doesn't use this path or even mention that your data will be deleted.

Is this expected and if so is there any official documentation on the correct path to store persistent data?

Edit: Adding example of failing senario

$ docker-machine ssh alt
docker@alt:~$ docker run -v /data:/var/lib/mysql --name mydata busybox sh -c "echo 'hello' > /var/lib/mysql/hello"
docker@alt:~$ docker run --rm --volumes-from mydata busybox sh -c "cat /var/lib/mysql/hello"
hello
docker@alt:~$ exit
$ docker-machine restart alt
Starting VM...
$ docker-machine ssh alt
docker@alt:~$ docker run --rm --volumes-from mydata busybox sh -c "cat /var/lib/mysql/hello"
cat: can't open '/var/lib/mysql/hello': No such file or directory
Gerry
  • 338
  • 2
  • 3
  • 12

3 Answers3

9

This definitely should work:

$ docker-machine ssh default
docker@default:~$ docker run -v /data --name mydata busybox true
docker@default:~$ docker run --volumes-from mydata busybox sh -c "echo hello >/data/hello"
docker@default:~$ docker run --volumes-from mydata busybox cat /data/hello
hello
docker@default:~$ exit
$ docker-machine restart default
Starting VM...
$ docker-machine ssh default
docker@default:~$ docker run --volumes-from mydata busybox cat /data/hello
hello

Can you elaborate more on the steps to reproduce your problem?

boot2docker has a read-only filesystem (will get wiped on reboot) with the exception of:

  1. Containers and their data (volumes) -- this is what you read about /var/lib/docker
  2. Docker images
  3. Docker configuration (e.g. /var/lib/boot2docker/profile where the daemon flags can be tweaked)
Gerry
  • 338
  • 2
  • 3
  • 12
nathanleclaire
  • 204
  • 1
  • 3
  • Hi, I've just suggested an edit to your response. Did you mean to specify the busybox image in the 2nd 3rd and 4th run commands? Without doing so docker tries to download images called "sh" and "cat" which I suspect is not what you intended. I will add a failing example above. – Gerry Sep 19 '15 at 12:01
  • Thank you for your help. Both you and @mc0e helped with this one so I've given you the tick and him the bounty (as well as a +1 for both). I hope you consider this fair. – Gerry Sep 19 '15 at 14:58
  • 1
    Ah yes, mis-typed. Sorry for the typo. – nathanleclaire Sep 22 '15 at 21:52
  • No apology necessary. I was just double checking to make sure I hadn't misunderstood. Cheers for all your help with this. – Gerry Sep 23 '15 at 04:52
6

I don't use boot2docker, but if /data gets wiped on reboot, that's where your volume is being stored (docker run -v /data:/var/lib/mysql), so it will be lost.

What you're doing is also combining two different patterns for handling volume persistence. In order to get persistence, containers can mount volumes from a specified location on the host system (which is presumed to be persistent), or they can be associated with a data container, and mounted with --volumes-from. It sounds like the host filesystem approach is not appropriate for boot2docker, and you should use the data volume pattern (only).

Presumably you should be creating your data container with -v /var/lib/mysql, rather than -v /data:/var/lib/mysql.

mc0e
  • 5,786
  • 17
  • 31
  • Creating a volume using the -v flag and then using volumes from that container in other containers using --volumes-from is exactly what the documentation shows though: https://docs.docker.com/userguide/dockervolumes/#creating-and-mounting-a-data-volume-container I would expect that if -v /location knows that it should be relative to /var/lib/docker/volumes then adding a colon shouldn't change that. At the very least this is unintuitive. -v /var/lib/mysql != -v /var/lib/mysql:/var/lib/mysql It also seems like the mysql-docker docs should be updated: https://hub.docker.com/_/mysql/#caveats – Gerry Sep 19 '15 at 14:53
  • Thank you for your help. Both you and @nathanleclaire helped with this one so I've given you the bounty (as it's worth more) and I gave the tick to nathanleclaire (as well as a +1 for both). I hope you think this is fair. – Gerry Sep 19 '15 at 14:56
  • @Gerry Certainly creating a volume and then using it via --volumes-from is advised. Mounting the ephemeral /data on your container is not. – mc0e Sep 19 '15 at 19:15
  • Yeah trouble is I didn't know /data was ephemeral until I restarted. – Gerry Sep 19 '15 at 19:42
0

Note that I am using docker for mac beta which is using the xhyve vm.

The contents of /var/lib/boot2docker will be persisted between machine restarts. So if you want certain files to be available on your vm put them in this directory.

If you want them to be accessible in a different location for running docker containers then you can add the following to /var/lib/boot2docker/profile:

mkdir -p /desired/path
ln -s /var/lib/boot2docker/your.file /desire/path/your.file

If you want to add other configuration to system files that will persist between vm restarts such as extra values in your hosts file then you can add a command such as the one below to /var/lib/boot2docker/profile:

echo '127.0.0.1 your.domain.com' >> /etc/hosts

Hope it helps

baoluofu
  • 1