40

We're updating our servers from a very out-of-date distro to a modern Debian Jessie based system, including lightdm / xfce, and of course systemd (and udisks2). One sticking point is automounting USB drives. We used to accomplish this with some udev rules. The old rules almost still work - the mount point gets created and the drive is mounted fine, but after a few seconds systemd is doing something that breaks the mount, so subsequent access attempts result in "Transport endpoint is not connected" errors.

Manually mounting the drive via the command line works fine. So does letting a file manager (thunar and thunar-volman, which in turn uses udisks2). But those are not viable options - these systems mostly run headless, so thunar isn't normally running. We need to be able to plug in disk drives for unattended cron-based backups.

I thought that modifying the udev script to spawn a detached job which waits a few seconds before performing the mount might do the trick, but systemd seems to go out of its way to prevent this - it somehow still waits for the detached job to finish before continuing.

Perhaps having the udev script tickle udisks2 somehow is the right approach? I'm at a lose, so any advice greatly appreciated.

Mike Blackwell
  • 995
  • 1
  • 7
  • 12
  • 1
    Only tangentially related, but... You're putting xfce on a server? – Parthian Shot Mar 28 '16 at 16:37
  • Ah, I used the term "server" rather loosely... All user interaction with the system is through a web app, typically accessed via a browser over the network. But some customers prefer a non-network solution, so we run Chrome on the console in a sort of kiosk mode. (This is also handy for debugging network config issues, you can plug in a monitor/mouse/keyboard and access basic diagnostic tools in the web app without needing Linux login credentials). There's probably a lighter weight solution than lightdm/xfce, but this was simplest to set up... – Mike Blackwell Mar 30 '16 at 14:50
  • For anyone wanting systemd-udevd rules directly running a script: I had this; it worked for a while, but at some point ceased running the script if udevd had been started automatically. Stop and restart from command line, and it'd be fine. On top of that, it never worked well with NTFS+FUSE because udev detected it had a long-running child process (ntfs-3g) and killed it after 60s. Bottom line: udev rules directly running a script is a waste of time. Go with udev rules and a systemd service instead, as noted in the answers. Then you don't have to deal with namespaces (MountFlags=slave) either. – Mark Aug 01 '17 at 14:40
  • I had a similar issue of a script started by udev not being able to make network connections. The solution below of using systemd worked for this too - thanks! – Quentin Stafford-Fraser Jan 28 '19 at 12:43
  • @ParthianShot what's the issue there? – user2305193 Jan 16 '21 at 17:29
  • 1
    @user2305193 Mostly security, but also best practices around server lifecycle management, and stability / package management concerns. Any X11-based window manager out there is going to pull in a lot of dependencies, and run some daemonized processes available over localhost. You usually want to minimize both of the above for security reasons, and manage a server remotely using config / orchestration tools (e.g. ansible, salt, Dockerfile, etc.) rather than by actually logging into it manually. And then there's dependency hell. – Parthian Shot Mar 27 '21 at 06:08

5 Answers5

42

After several false starts I figured this out. The key is to add a systemd unit service between udev and a mounting script.

(For the record, I was not able to get this working using udisks2 (via something like udisksctl mount -b /dev/sdb1) called either directly from a udev rule or from a systemd unit file. There seems to be a race condition and the device node isn't quite ready, resulting in Error looking up object for device /dev/sdb1. Unfortunate, since udisks2 could take care of all the mount point messyness...)

The heavy lifting is done by a shell script, which takes care of creating and removing mount points, and mounting and unmounting the drives.

/usr/local/bin/usb-mount.sh

#!/bin/bash

# This script is called from our systemd unit file to mount or unmount
# a USB drive.

usage()
{
    echo "Usage: $0 {add|remove} device_name (e.g. sdb1)"
    exit 1
}

if [[ $# -ne 2 ]]; then
    usage
fi

ACTION=$1
DEVBASE=$2
DEVICE="/dev/${DEVBASE}"

# See if this drive is already mounted, and if so where
MOUNT_POINT=$(/bin/mount | /bin/grep ${DEVICE} | /usr/bin/awk '{ print $3 }')

do_mount()
{
    if [[ -n ${MOUNT_POINT} ]]; then
        echo "Warning: ${DEVICE} is already mounted at ${MOUNT_POINT}"
        exit 1
    fi

    # Get info for this drive: $ID_FS_LABEL, $ID_FS_UUID, and $ID_FS_TYPE
    eval $(/sbin/blkid -o udev ${DEVICE})

    # Figure out a mount point to use
    LABEL=${ID_FS_LABEL}
    if [[ -z "${LABEL}" ]]; then
        LABEL=${DEVBASE}
    elif /bin/grep -q " /media/${LABEL} " /etc/mtab; then
        # Already in use, make a unique one
        LABEL+="-${DEVBASE}"
    fi
    MOUNT_POINT="/media/${LABEL}"

    echo "Mount point: ${MOUNT_POINT}"

    /bin/mkdir -p ${MOUNT_POINT}

    # Global mount options
    OPTS="rw,relatime"

    # File system type specific mount options
    if [[ ${ID_FS_TYPE} == "vfat" ]]; then
        OPTS+=",users,gid=100,umask=000,shortname=mixed,utf8=1,flush"
    fi

    if ! /bin/mount -o ${OPTS} ${DEVICE} ${MOUNT_POINT}; then
        echo "Error mounting ${DEVICE} (status = $?)"
        /bin/rmdir ${MOUNT_POINT}
        exit 1
    fi

    echo "**** Mounted ${DEVICE} at ${MOUNT_POINT} ****"
}

do_unmount()
{
    if [[ -z ${MOUNT_POINT} ]]; then
        echo "Warning: ${DEVICE} is not mounted"
    else
        /bin/umount -l ${DEVICE}
        echo "**** Unmounted ${DEVICE}"
    fi

    # Delete all empty dirs in /media that aren't being used as mount
    # points. This is kind of overkill, but if the drive was unmounted
    # prior to removal we no longer know its mount point, and we don't
    # want to leave it orphaned...
    for f in /media/* ; do
        if [[ -n $(/usr/bin/find "$f" -maxdepth 0 -type d -empty) ]]; then
            if ! /bin/grep -q " $f " /etc/mtab; then
                echo "**** Removing mount point $f"
                /bin/rmdir "$f"
            fi
        fi
    done
}

case "${ACTION}" in
    add)
        do_mount
        ;;
    remove)
        do_unmount
        ;;
    *)
        usage
        ;;
esac

The script, in turn, is called by a systemd unit file. We use the "@" filename syntax so we can pass the device name as an argument.

/etc/systemd/system/usb-mount@.service

[Unit]
Description=Mount USB Drive on %i

[Service]
Type=oneshot
RemainAfterExit=true
ExecStart=/usr/local/bin/usb-mount.sh add %i
ExecStop=/usr/local/bin/usb-mount.sh remove %i

Finally, some udev rules start and stop the systemd unit service on hotplug/unplug:

/etc/udev/rules.d/99-local.rules

KERNEL=="sd[a-z][0-9]", SUBSYSTEMS=="usb", ACTION=="add", RUN+="/bin/systemctl start usb-mount@%k.service"

KERNEL=="sd[a-z][0-9]", SUBSYSTEMS=="usb", ACTION=="remove", RUN+="/bin/systemctl stop usb-mount@%k.service"

This seems to do the trick! A couple of useful commands for debugging stuff like this:

  • udevadm control -l debug turns on verbose logging to /var/log/syslog so you can see what's happening.
  • udevadm control --reload-rules after you modify files in the rules.d dir (may not be necessary, but can't hurt...).
  • systemctl daemon-reload after you modify systemd unit files.
Mike Blackwell
  • 995
  • 1
  • 7
  • 12
  • 5
    Wow. This is awesome. Wish I could give multiple upvotes! The only thing I had to modify was that on my system, `blkid` does not seem to extract an `ID_FS_LABEL`, so I just used the `DEVBASE` rather than `LABEL` to construction the `MOUNT_POINT` instead. – Travis Griggs Jul 08 '16 at 23:36
  • Can this setup be modified to work with ATA/SCSI devices? See: http://serverfault.com/q/825779/297059 – user339676 Jan 13 '17 at 02:06
  • @Travis - You can use `udevadm` instead of `blkid`. It gives much more detail as well as additional information. (e.g., `udevadm info --query=property --name=sda1`) – user339676 Jan 13 '17 at 04:27
  • 2
    this doesn't work well on boot, if a usb device is already connected. Any ideas? – Michal Artazov Dec 18 '18 at 16:20
  • When nullglobs are not set, on unmount, the cleanup can generate an error like `/usr/bin/find: '/media/*': No such file or directory`. Cleanup can use an additional check like `if [ "$f" != "/media/*" ]; then` before running `find`. – Pro Backup Dec 26 '18 at 23:13
  • @Mike Blackwell - Thanks.. this is really helpful but I am having a problem with the rules you provided. If a USB drive is assigned with `sd[a-z]` then rules do not match. I tried `sd[a-z][0-9]|sd[a-z]`but in that case `sd[a-z][0-9]` do not match. I want a rule which can match both e.g. `sd[a-z]` and `sd[a-z][0-9]`. I tried different combinations but didn't succeed. – Raxesh Oriya Sep 21 '20 at 10:15
  • 1
    According to https://superuser.com/a/1364595/150943: _"You might be tempted to use `RUN+="/bin/systemctl start foo"` via udev. Don't do that. But if you do, then at least use the `systemctl --no-block` option so that you won't create a possible deadlock between the two components."_ – nh2 Jan 27 '21 at 10:27
  • Is this still the best answer as of Jan 2022? In other words: Is it still true that `systemd` is incapable of **natively** automount-ing "pluggable" USB devs ? By *natively*, I mean using only the options documented for `/etc/fstab`. – Seamus Jan 15 '22 at 11:20
  • The unmount portion of this script is a bit like shutting the stable door after the horse has bolted. It gives the impression that it is safe to use a script to **auto-unmount** USB devices. But I don't see how that can ever be possible?? The un-mount script won't run until you have already unplugged the USB drive. Doing that can cause physical damage and data loss. – FlexMcMurphy May 19 '22 at 21:56
18

there is a new, succinct systemd auto-mount option which can be used with fstab which allows you to use all the standardized mount permission options, and it looks like this:

  x-systemd.automount

an example of it in an fstab line:

  /dev/sdd1   /mnt/hitachi-one     auto     noauto,x-systemd.automount     0 2

the noauto option will mean it will not attempt to be mounted at boot, as with older software autofs.

after adding a new x-systemd.automount line to fstab you then need to run:

  sudo systemctl daemon-reload

and then both, or one, of the following:

  sudo systemctl restart remote-fs.target
  sudo systemctl restart local-fs.target

for more infomation about it:

https://wiki.archlinux.org/index.php/Fstab#Automount_with_systemd

4

Using pmount, systemd and Mike Blackwell's approach, you can simplify the whole thing:

/etc/systemd/system/usb-mount@.service

[Unit]
Description=Mount USB Drive on %i
[Service]
Type=oneshot
RemainAfterExit=true
ExecStart=/usr/bin/pmount --umask 000 /dev/%i /media/%i
ExecStop=/usr/bin/pumount /dev/%i

/etc/udev/rules.d/99-usb-mount.rules

ACTION=="add",KERNEL=="sd[a-z][0-9]*",SUBSYSTEMS=="usb",RUN+="/bin/systemctl start usb-mount@%k.service"
ACTION=="remove",KERNEL=="sd[a-z][0-9]*",SUBSYSTEMS=="usb",RUN+="/bin/systemctl stop usb-mount@%k.service"

HTH and thank you Mike.

Eric V.
  • 66
  • 1
3

I'd go with Warren Young's answer I have a few changes I made to

I added some space protection as it was giving errors from the eval of the environment for the drive.

I added a section to chmod a usb disk so all users have full access to non ntfs or vfat disks.

/usr/local/bin/usb-mount.sh

#!/bin/bash

# This script is called from our systemd unit file to mount or unmount
# a USB drive.

usage()
{
    echo "Usage: $0 {add|remove} device_name (e.g. sdb1)"
    exit 1
}

if [[ $# -ne 2 ]]; then
    usage
fi

ACTION="$1"
DEVBASE="$2"
DEVICE="/dev/${DEVBASE}"

# See if this drive is already mounted, and if so where
MOUNT_POINT=$(/bin/mount | /bin/grep ${DEVICE} | /usr/bin/awk '{ print $3 }')

do_mount()
{
    if [[ -n "${MOUNT_POINT}" ]]; then
        echo "Warning: ${DEVICE} is already mounted at ${MOUNT_POINT}"
        exit 1
    fi

    # Get info for this drive: $ID_FS_LABEL, $ID_FS_UUID, and $ID_FS_TYPE
    # added some sed's to avoid space issues
    eval $(/sbin/blkid -o udev ${DEVICE}|sed 's/=/="/'|sed 's/$/"/')

    # Figure out a mount point to use
    LABEL="${ID_FS_LABEL}"
    if [[ -z "${LABEL}" ]]; then
        LABEL="${DEVBASE}"
    elif /bin/grep -q " /media/${LABEL} " /etc/mtab; then
        # Already in use, make a unique one
        LABEL+="-${DEVBASE}"
    fi
    MOUNT_POINT="/media/${LABEL}"

    echo "Mount point: ${MOUNT_POINT}"

    /bin/mkdir -p "${MOUNT_POINT}"

    # Global mount options
    OPTS="rw,relatime"
    #added a chmod checker for file systems that don't 
    #understand allow all to read write
    CHMOD=no
    # File system type specific mount options
    if [[ ${ID_FS_TYPE} == "vfat" ]]; then
        OPTS+=",users,gid=100,umask=000,shortname=mixed,utf8=1,flush"
    #added options I wanted on ntfs
    elif [[ ${ID_FS_TYPE} == "ntfs" ]]; then
        OPTS+=",user,users,umask=000,allow_other"
    else
       CHMOD=yes
    fi

    if ! /bin/mount -o "${OPTS}" ${DEVICE} "${MOUNT_POINT}"; then
        echo "Error mounting ${DEVICE} (status = $?)"
        /bin/rmdir "${MOUNT_POINT}"
        exit 1
    fi


    echo "**** Mounted ${DEVICE} at ${MOUNT_POINT} ****"
    if [ "${CHMOD}" = "yes" ];then
        /usr/bin/find "${MOUNT_POINT}" -type f -exec chmod 0666 {} \;
        /usr/bin/find "${MOUNT_POINT}" -type d -exec chmod 0777 {} \;
    fi
}

do_unmount()
{
    if [[ -z ${MOUNT_POINT} ]]; then
        echo "Warning: ${DEVICE} is not mounted"
    else
        /bin/umount -l ${DEVICE}
        echo "**** Unmounted ${DEVICE}"
    fi

    # Delete all empty dirs in /media that aren't being used as mount
    # points. This is kind of overkill, but if the drive was unmounted
    # prior to removal we no longer know its mount point, and we don't
    # want to leave it orphaned...
    for f in /media/* ; do
        if [[ -n $(/usr/bin/find "$f" -maxdepth 0 -type d -empty) ]]; then
            if ! /bin/grep -q " $f " /etc/mtab; then
                echo "**** Removing mount point $f"
                /bin/rmdir "$f"
            fi
        fi
    done
}

case "${ACTION}" in
    add)
        do_mount
        ;;
    remove)
        do_unmount
        ;;
    *)
        usage
        ;;
 esac
penguinjeff
  • 131
  • 3
  • 1
    You may want to describe what is different between the original answer and yours in a few words, to make it more useful. PS: there was no answer by Warren Young; perhaps you meant Mike Blackwell’s answer that was edited? – Amir Jan 03 '19 at 20:06
2

I have modified the script from @MikeBlackwell to:

  • recognize device names that span multiple characters, not just /dev/sd[a-z] but /dev/sd[a-z]*; often the case with servers that have larger number of spindles.
  • track the list of automounted drives at /var/log/usb-mount.track
  • log the actions to /var/log/messages with tag usb-mount.sh
  • prefix device name with the device label for the mount point to not run in to problems with drives that haven't been assigned a label(empty?):/media/sdd2_usbtest, /media/sdd2_
  • included wrapper scripts to place the files appropriately and undo if required

Since @MikeBlackwell has already done most of the heavy lifting, I chose not to rewrite it; just made the necessary changes. I have acknowledged his work sighting his name and URI of the original answer.

Find it at https://github.com/raamsri/automount-usb

six-k
  • 121
  • 3
  • 1
    beware that on the issues page, someone pointed out they needed to **change the following to prevent system from not booting when disk is inserted(!)** `RUN+="/bin/systemctl --no-block start usb-mount@%k.service"` – user2305193 Jan 16 '21 at 17:44