0

Question

Setup

On a Proxmox server I create an LXC container with:

pct create 101 local:vztmpl/centos-8-default_20201210_amd64.tar.xz --features mknod=1,nesting=1,fuse=1 --ostype centos --storage=CTVM --description "centos glusterfs" --hostname thorgluster --password "password" --onboot 1 --net0 name=eth0,bridge=vmbr0 --unprivileged=0

Then can device mount /dev/sdd into it with:

lxc-device add -n 101 /dev/sdd

and adding the following lines to /etc/pve/lxc/101.conf so it mounts on reboot (8:48 is from lsblk):

lxc.cgroup2.devices.allow = b 8:48 rwm
lxc.mount.entry = /dev/sdd /dev/sdd none bind,optional,create=file

Note, I need to device mount not bind mount because I need to be able to run LVM commands on the drives from within the container.

Actual Question

However in the future, /dev/sdd may change names to /dev/sd[a-z] according to this post. So how can I deal with this name changing?

Attempted Solutions

  • I tried using /dev/disk/by-id/ata-ST2000DM008-2FR102_ZFL1FYFF instead of /dev/sdd but it didn't work.
  • I considered creating a permanent device node before mounting with mknod following this post but I haven't seen this done anywhere else and don't know if there are any other considerations when using mknod like this.
  • This post says the device name is tied to the port it's plugged into. Which would mean all I have to do is put physical labels on the disks and ports themselves.
  • I saw somewhere I can mount a virtual hard drive into the container which would avoid this problem but I haven't seen any documentation on how to do this. (edit: I might be able to use a loop device. Looking into this.)
  • I tried adding lvm.external=true to /etc/pve/lxc/101.conf but got an error.
mikeLundquist
  • 101
  • 2
  • > the device name is tied to the port it's plugged into. Which would mean all I have to do is put physical labels on the disks and ports themselves. --- no, this is wrong. There could be some correlation because the scan essentially is a loop which tries all ports in a sequence, but if 1st port has slower device that 2nd and 2nd port responds faster, devices may end up having "swapped names". – Nikita Kipriyanov May 11 '22 at 13:17
  • Back to the question: why aren't you using the proxmox storage framework properly, as described in the docs? Or, if you want to avoid LVM, just mount the drive to the host and make a directory bind mount, again, as described in the docs. – Nikita Kipriyanov May 11 '22 at 13:20
  • I want to use [glusterfs snapshots](https://rajeshjoseph.gitbooks.io/test-guide/content/features/chap-Managing_Snapshots.html) from within the LXC container. GlusterFS snapshots use LVM which manages devices directly. I'm going to try these steps: `storage mount to LXC -> create a loop device within LXC -> run pvcreate on the loop device -> install glusterfs on XFS on the loop device` hopefully loop devices work within LXC containers – mikeLundquist May 11 '22 at 13:31

2 Answers2

0

My advice is to use LVM on the host too. Create a volume group with the disk you want to use for your VMs:

vgcreate VMdisks /dev/sdd /dev/sde

Then create a logical volume that uses all of /dev/sdd:

lvcreate -n Centos01SystemDisk -l 100%PVS VMdisks /dev/sdd

Then use /dev/VMdisks/Centos01SystemDisk as your VM disk.

You'll have the benefit of not having to care about which disk is which, plus you'll have LVM capabilities (ability to extend an LV, create snapshots, etc).

wazoox
  • 6,782
  • 4
  • 30
  • 62
  • 1
    This is the same thing as creating storage of type `lvm` and assigning a new mountpoint to the container using PVE GUI properly, albeit the latter is better because it's cleaner and completely in line with the documentation. For instance, it'll enable backups of that volume. – Nikita Kipriyanov May 11 '22 at 13:23
  • @NikitaKipriyanov yes but apparently OP wants to use block devices directly; he may have a good reason for that (using Gluster apparently) – wazoox May 11 '22 at 16:06
0

The problem I was having was using /dev/sdd as the second argument to fstab. It should have been dev/sdd which proxmox will then append to. The line now looks like this

lxc.mount.entry = /dev/disk/by-id/ata-ST2000DM008-2FR102_ZFL1FYFF dev/sdd none bind,optional,create=file
mikeLundquist
  • 101
  • 2