only flush the write cache if there's no read activity

Question

Normally the Linux Kernel flushes the write cache if a specified time limit is exeeded or the cache is full. This leads to stalling with HDDs since the writing process makes reading processes much slower.

That's why I want to avoid a flush of the write cache if there's only a bit read activity. Since I have 40 GB of RAM this should not be a problem. My thoughts on that would be to set the time limit to five minutes, tune the write cache size to your needs and write a script that flushes the cache ONLY if there is 0 read activity. I don't know how to do that, would the shell command sync offer good enough performance?

My goal is to have a write cache with a size like 1, 2 GBs. Flushing this would take a long time. Is there anything that can be stopped as soon there's read activity again or a higher disk PSI (would that even react fast enough??)

I have two Seagate (non-smr?) drives: ST1000DM010-2EP102 - but I guess that information isn't needed.

With dirty_ratio I currently set a dynamic cache size.

echo 360000 > /proc/sys/vm/dirty_expire_centisecs
echo 360000 > /proc/sys/vm/dirty_writeback_centisecs

This would set the maximum time to start flushing the write cache to 1 hour.

Reference: https://unix.stackexchange.com/questions/30286/can-i-configure-my-linux-system-for-more-aggressive-file-system-caching

Useful commands:

With the following command you can see how much is in the write cache:

cat /proc/meminfo | grep Dirty

Output:

Dirty:               104 kB

But this is not very important..

Where am I stuck?

I want to be able to almost instantly cancel the sync command (is this even possible) and I want to know if there is a process reading. That's all what I need to write the script.

I found an old script that's able to monitor read activity: awk '{print $1}' /sys/block/sdb/stat

How to use this?

if [ $old-value -lt $new-value ];then COMMAND;fi

Answer 1

Sources used: https://askubuntu.com/questions/184204/how-do-i-fetch-only-numbers-in-grep https://www.kernel.org/doc/html/latest/block/stat.html https://askubuntu.com/questions/184204/how-do-i-fetch-only-numbers-in-grep

script that I coded:

#!/bin/bash                                                                                                                                                                                                

oldreads=0
killed=True
while (true)
do
    reads=`awk '{print $1}' /sys/block/sdb/stat`
    if [ "$oldreads" == "$reads" ]
    then
        if [ $(grep -o '[[:digit:]]*'<<<$(cat /proc/meminfo | grep Dirty)) -gt 10240 ]
        then
            echo "sync"
            sync&
            syncPid="$!"
            oldreads="$reads"
            killed=False
        else
            echo "There's not more than a 10 MB in the write buffer, not syncing."
        fi
    elif [ $killed == False ]
    then
         echo "kill"
         kill "$syncPid"
         killed=True
    else
        echo "$oldreads => $reads"
        oldreads="$reads"
    fi
    sleep 1
done