System that can read hard-drive and exclusively write into RAM?

0

I hope to be at the right place. I could not find any suggestions on the "web" about my question. How could an O.S, let us say a distribution of linux, can be loaded from the hard-drive and execute all the modifications (all the applications) into the RAM ? In other words, how could we use an O.S to read data it needs from a hard-drive and only write what it needs into the RAM.

The aim is to be sure that all the "instructions" (this may be the wrong word) made by the O.S (and all the apps it executes) are made exclusively into the RAM (no write on the hard-disk).

I know some linux distributions can be entirely loaded into the RAM but the inconvenience is the size of the system we get : we can not have much available space for apps in the RAM whereas a normal system will find the needed app from the drive and load it into the memory. This could be done by using an extra RAM for the system but it is not a great solution compared to the use of a hard drive to save the apps we need (because of the actual prices of RAM)

I also thought about a virtual machine which can read the drive to use app and only write into RAM, in most cases. But it requires to load the VM from an existing, and loaded, O.S. so is there a way we can skip the existing O.S part ?

If you have any suggestions or technical words I have to use to search by myself, thank you.

Ben W

Posted 2019-04-05T07:00:57.260

Reputation: 11

Answers

0

I'm going to expand a little on the subject to help a complete beginner as I started.

Boot process

First hard drive is typically split into 2 sections :

  1. one for boot process (the section read by the computer when you power it up) typycally called /dev/sda1 under most linux distribution
  2. one for all the other things (like system and user files) nammed /dev/sdb1. (system and user files can be on different sections)

On the boot sector there is a kernel image, which is actually an initramfs image. An intramfs is "the smallest" functionnal system that is loaded into the RAM. It can be seen as a small version of your linux system : you will find modules, some directories that you see on your habitual system but with a different content. That small linux system is used to look at your computer configuration (what type of storage device you use, what are your compenents, etc) to load correct modules that will be used by your real system AND figure out where on your storage device is your real system files to make it accessible for later. In other words, when you power up your computer it reads the /dev/sda1 partiion and try to find the partition /dev/sdb1 for a later use. But to understand how your files are accessible you need to understand mount points

Mountpoint

Under linux, all directories are accessible under a single one called the root, which is /. Do not mixe root user storage /root with the root of the filesystem/. When you plug an external storage device, like an USB drive, your system will create a new directory and use it as the root for that device. In other words, under that directory you can access all files that are store on it. That directory is called a mountpoint. To create a mountpoint you need to have a device (an USB for example), a directory and an filesystem (a program that is used as an interface between your userspace programs and the physical memory of your device) and then use the following command :

mount -t name_filesystem name_device name_directory

Technically when you plug something you need to manually mount the device. But linux do it automatically when you plug a common device (like an USB). All the internal devices of your computer are automatically mounted during the boot thanks to the file file /etc/fstab which contains all the needed informations to mount them (like the partitions of your harddrive, cd-ROM reader, ...) To see all the other mountpoints currently used by the system you can see /proc/mounts

Mounting root filesystem at boot

Back to the boot process, when the initramfs image is loaded into the memory, it check which parition of your hard drive is used to store your files (like /dev/sdb1) and mount it under the root of your filesystem commonly called /root. That /root is not the same as the /root under your userspace (when your system is completly loaded). Under that /root you will have access to all your files and the /root you know :

/root <--- initramfs
   /home
      /myName
        ... all my files
   /root <--- super user

The root filesystem is mounted as read-only during boot and then remount as read-write over a virutal filesystem (which is just a way of dealing with different devices, thus different filesystems related to each device, with same commands for programs in your userspace)

Initrd vs Initramfs

Initramfs is the sucessor of initrd which was loaded into the memory with all the needed tools to then mount the real filesystem and finally unload itself from the memory. But an initramfs stays in memory and all the system is based around the root filesystem that is mounted by the initramfs. That difference also implies different commmands (for example pivot_root became switch_root)

Edit initramfs

The kernel image that is used for the boot process can be edit whithin the system. That image is store on /boot. You can edit it by uncompressing it. The process you want to edit is the init of your image (which is the first script executed) and precisly the part when the real filesystem is mounted under /root which look like :

log_begin_msg "Mounting root file system"
# Always load local and nfs (since these might be needed for /etc or
# /usr, irrespective of the boot script used to mount the rootfs).
. /scripts/local
. /scripts/nfs
. /scripts/${BOOT}
parse_numeric ${ROOT}
maybe_break mountroot
mount_top
mount_premount
mountroot

Note that ${ROOT} contains the name of the device used for your system files (for example /dev/sdb1 but it can be different)

But that is a messy solution because when the image will be update by your system (for some reasons) your modifications will be lost. But linux has the solution to keep your modifications into all the kernel images, even after updates : initramfs-tools. All the modifications you will have to make will be done under /etc/initramfs-tools Then after you have edited your image use the following command to update the kernel image :

update-initramfs -u

That image will be used after the next reboot. Thus to avoid crashing your system use a virtual machine as virtualBox or

Once built, you'll want to install your new kernel alongside your old one, and configure the bootloader to make the kernel selectable at boot time. This way even if something goes horribly wrong, you can still boot your system using your existing kernel.

Overlay

To mount overlay over an existing directory use :

mount -t overlay -o lowerdir=/overlay/lower,upperdir=/overlay/upper,workdir=/overlay/work overlay /myDir

Note that there is absolutly no space between lowerdir=..,upperdir=...,workdir=... and the last overlay can be replace by none (because you mount nothing, no device over /myDir; you just mount overlay). And when you do something bad (if a directory does not exists or you put spaces in the options for mounting -o ... the result will be "there is no such device under /etc/fstab" (overlay manages terribly errors during mounting). You can look at that link and that one to find more help.

Under /myDir you will see all files that are in /overlay/lower and all modifications that you made (which are stored in /overlay/upper) :

/overlay/lower      /overlay/upper          /myDir (mountpoint)
   /file1  ---------->  /file1   --------->   /file1
      'abcdef'            'abOdef'               'abOdef'
   /file2 -------------------------------->   /file2

tmpfs

tmpfs is a filesystem that can manipulate the RAM as a standard memory. To create a ram-disk (a piece of the RAM that act like a piece of an hard drive) use :

mount -t tmpfs none /myDir
           or
mount -t tmpfs tmpfs /myDir

Then all operations made into /myDir will be saved into the RAM.

Solution

Modules

Add overlay module to the kernel image (so the module can be used during the boot process). Add overlay into the file /etc/initramfs-tools/modules

Hooks

Add the following script into /etc/initramfs-tools/hooks (as an executable)

#!/bin/sh

#set kernel functions file as sources (so these functions can be used)
. /usr/share/initramfs-tools/scripts/functions
. /usr/share/initramfs-tools/hook-functions

#If you want to use a custom package into the initramfs use 
#copy_exec path_to_package
#For example if you want to use whiptail add : copy_exec /usr/bin/whiptail

Script

Add the following script into /etc/initramfs-tools/scripts/init-bottom (as an executable)

#!/bin/sh

PREREQ=""
prereqs()
{
   echo "$PREREQ"
}

case $1 in
prereqs)
   prereqs
   exit 0
   ;;
esac

DIR=/overlay
UPPER_DIR=$DIR/upper
LOWER_DIR=$DIR/lower
WORK_DIR=$DIR/work

mkdir $DIR
#create temporary filesystem in RAM
mount -t tmpfs tmpfs $DIR
if [ $? -ne 0 ]; then
    fail_err "Fail to mount tmpfs on $DIR"
    exit 1
fi

#create lower, upper and workdir for overlay
mkdir $UPPER_DIR $LOWER_DIR $WORK_DIR

# move root mountpoint to the lower layer
mount -n -o move ${rootmnt} $LOWER_DIR
if [ $? -ne 0 ]; then
    fail_err "Cannot move ${rootmnt} to $LOWER_DIR"
    exit 1
fi

#mount overlay on to the root
mount -t overlay -o lowerdir=$LOWER_DIR,upperdir=$UPPER_DIR,workdir=$WORK_DIR overlay ${rootmnt}
if [ $? -ne 0 ]; then
    fail_err "Cannot mount overlay on ${rootmnt} [lowerdir=$LOWER_DIR, upperdir=$UPPER_DIR, workdir=$WORK_DIR]"
    exit 1
fi

#mount virtual filesystem over the lower dir (as it's donne for the root with /proc and /sysfs which are mounted on to the root at the end of the init)

#uncomment that section if you want to access the layers of overlay whithin userspace
#mkdir -p ${rootmnt}$DIR
#mount -n -o rbind $DIR ${rootmnt}$DIR
#if [ $? -ne 0 ]; then
#    fail_err "Cannot remount ${rootmnt} and its sub mountpoints on ${rootmnt}$DIR"
#    exit 1
#fi

# edit fstab to match the actual modifications

#get the device that is mounted on the rootfilesystem
root_device="$(awk '$2 == "/" {print $0}' ${rootmnt}/etc/fstab)"
#check if that device exists
blkid $root_device
if [ ! $? -gt 0 ]; then
    target=${rootmnt}/etc/fstab
    target_c=$target.orig
    #create a copy of the file (that will be accessible later)
    cp $target $target_c
    #remove the root device from the file
    grep -v "$root_device" $target_c > $target

    #add overlay on root as a device in the file
    awk '$2 == "'${rootmnt}'" { $2 = "/" ; print $0}' /etc/mtab >> $target
fi

Update

Update the kernel image with : update-initramfs -u

Links to mount overlay over root filesystem

solve raspbian SD card corruption issues with read-only mounted root partition

Custom Initramfs

Unable to umount after pivot_root

Mount root as overlayfs

Can not mount overlayfs inside initrd

How to use OverlayFS to protect the root filesystem ?

Ben W

Posted 2019-04-05T07:00:57.260

Reputation: 11

0

It's possible, but not trivial. I'll give you some pointers, but I can't really provide detailed steps, as I've never tried this myself.

  • OverlayFS - It lets you create a "layered" filesystem, where lower layer is read-only and upper layer is writable. That being said, lower layer is still completely mutable (you can edit, delete files etc., but they aren't really altered - changes are stored in the upper layer entirely) and works like a full-fledged filesystem.

  • tmpfs - It's another full-fledged filesystem which resides entirely in memory. You can set a capacity lower than amount of RAM to prevent accidentally choking the OS. Also, unused capacity is available as regular RAM, which is nice.

  • pivot_root - A system call that moves / to a different directory.

Typically early in the boot process pivot_root is called to switch from early userspace to on-disk filesystem. You could alter this: create OverlayFS with on-disk filesystem in the lower layer and tmpfs in the upper one, then pivot_root to OverlayFS.

Be warned that upper layer can grow rather quickly, consuming a lot of available RAM. zram can be used to reduce RAM usage at the cost of CPU usage, but it's not an ultimate solution. Updating on-disk OS permanently will also be tricky.

gronostaj

Posted 2019-04-05T07:00:57.260

Reputation: 33 047

Thank a lot, I'm going to look at it – Ben W – 2019-04-05T12:33:00.220