Performance of RAMFS

Question

I am profiling some computation (video transcoding) on large datasets. Since I did not want my results to be influenced by I/O time (the system I am using has old/slow spinning disks) I though that RAMFS would be the right approach.

I created a "cache" folder in my home as ramfs:

sudo mount ramfs -t ramfs ./cache/

I use this folder to save the input and output files for my video processing pipeline. However, it looks like there is some "caching" still going on between the disk and the RAM when reading files stored in the RAMFS which I think should not happen. First I copy the input video file into the cache, notice that the file is around 1.5 GB, hence the need for RAMFS: https://media.xiph.org/video/derf/y4m/crowd_run_1080p50.y4m

cp crowd_run_1080p50.y4m ./cache/

When performing a transcoding operation, reading from the file I just copied and writing to RAMFS

perf stat -e minor-faults:u,major-faults:u  ~/bin/ffmpeg -i ~/cache/crowd_run_1080p50.y4m -c:v h264_qsv  -y ~/cache/out_qsv.mp4

perf still reports some major-faults:

        11,933      minor-faults:u                                              
           159      major-faults:u                                              

   3.588215239 seconds time elapsed

Running the same operation again shows no major faults and significantly faster completion time:

        11,955      minor-faults:u                                              
             0      major-faults:u                                              

   2.129031238 seconds time elapsed

Even more troubling if I flush the page cache of the OS

sudo sh -c 'echo 3 >/proc/sys/vm/drop_caches'

I will see major faults again. Why is that? I though that the whole file would stay in RAMFS together with the inodes and no flushing is possible from ramfs (unlike tmpfs that can swap). Why do I still see major faults? If I have to perform the same operation twice in order to get reliable results I might as well not use RAMFS, but I would like to understand what I am missing.

Answer 1

The preload daemon can also helps in this situation. It runs in background and learn which programs and libraries are often in use. It then readahead them on next boots for a faster launch. As opposite to memlockd those files are not locked in RAM but read once after boot so their RAM space can be reused for something else if required, hence there is no guarantee of a faster launch.