How to create ext2 image without superuser rights?

Question

I need to generate several ext2 images. The obvious way to do that is to create an image, mount it & copy the content. But it requires root permissions twice (to chown the files and to mount the image). Also I found two tools for generating images: e2fsimage & genext2fs.

• genext2fs places the image in RAM while generating but size of one of my images is ~30GiB.

• e2fsimage crashes with some values of image size.

So how can I generate my images? It would be nice if the tool will calculate the image size itself.

Answer 1

Figured out why e2fsimage crashes. It is caused by int32 overflow when image size is greater than 4GiB. So the solution is to count required blocks & inodes, create loop file (truncate & mke2fs) and then use e2fsimage with -n argument (so it will not create the loopfile but use already created one)

Answer 2

mke2fs -d minimal runnable example without sudo

mke2fs is part of the e2fsprogs package. It is written by the famous Linux kernel filesystem developer Theodore Ts'o who is at Google as of 2018, and the source upstream is under kernel.org at: https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs Therefore, that repository can be considered a reference userland implementation of ext file system operations:

#!/usr/bin/env bash
set -eu

root_dir=root
img_file=img.ext2

# Create a test directory to convert to ext2.
mkdir -p "$root_dir"
echo asdf > "${root_dir}/qwer"

# Create a 32M ext2 without sudo.
# If 32M is not enough for the contents of the directory,
# it will fail.
rm -f "$img_file"
mke2fs \
  -L '' \
  -N 0 \
  -O ^64bit \
  -d "$root_dir" \
  -m 5 \
  -r 1 \
  -t ext2 \
  "$img_file" \
  32M \
;

# Test the ext2 by mounting it with sudo.
# sudo is only used for testing.
mountpoint=mnt
mkdir -p "$mountpoint"
sudo mount "$img_file" "$mountpoint"
sudo ls -l "$mountpoint"
sudo cmp "${mountpoint}/qwer" "${root_dir}/qwer"
sudo umount "$mountpoint"

GitHub upstream.

The key option is -d, which selects which directory to use for the image , and it is a relatively new addition to v1.43 in commit 0d4deba22e2aa95ad958b44972dc933fd0ebbc59

Therefore it works on Ubuntu 18.04 out of the box, which has e2fsprogs 1.44.1-1, but not Ubuntu 16.04, which is at 1.42.13.

However, we can do just like Buildroot and compile it from source easily on Ubuntu 16.04 with:

git clone git://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git
cd e2fsprogs
git checkout v1.44.4
./configure
make -j`nproc`
./misc/mke2fs -h

If mke2fs fails with:

__populate_fs: Operation not supported while setting xattrs for "qwer"
mke2fs: Operation not supported while populating file system

when add the option:

-E no_copy_xattrs

This is required for example when the root directory is in NFS or tmpfs instead of extX as those file systems don't seem to have extended properties.

mke2fs is often symlinked to mkfs.extX, and man mke2fs says that if you use call if with such symlink then -t is implied.

How I found this out and how to solve any future problems: Buildroot generates ext2 images without sudo as shown here, so I just ran the build with V=1 and extracted the commands from the image generation part that comes right at the end. Good old copy paste has never failed me.

TODO: describe how to solve the following problems:

• create sudo owned files in the image. Buildroot does it.
• automatically calculate the minimal required size. Initial estimate with du for file size and find . | wc for directory structure, min that with 32Mb (smaller failes), then double until the command works, is likely a very decent approach. Buildroot used to do this, but stopped for some reason, but easy to implement ourselves.
• conveniently extract all files from the partition:
  • https://unix.stackexchange.com/questions/434472/how-can-i-extract-files-from-a-disk-image-without-mounting-it
  • Access data from a linux partition inside an image without root privileges

Multiple partitions in one image file

See this: https://stackoverflow.com/questions/10949169/how-to-create-a-multi-partition-sd-image-without-root-privileges/52850819#52850819

Answer 3

Creating image does not need root privilege. Here is an example to create an ext2 image:

dd if=/dev/zero of=./MyDisk.ext2 bs=512 count=20480
mkfs.ext2 ./MyDisk.ext2

but the root privilege is needed to mount the device:

mkdir MyDisk
sudo mount ./MyDisk.ext2 MyDisk