Disable linux read and write file cache on partition

9

3

How do i disable the linux file cache on a xfs partition (both read an write).

We have a xfs partition over a hardware RAID that stores our RAW HD Video. Most of the shoots are 50-300gb each so the linux cache has a hit-rate of 0.001%.

I have tryed the sync option but it still fills up the cache when copinging the files.
( about 30x over per shoot :P )

/etc/fstab:

/dev/sdb1 /video xfs sync,noatime,nodiratime,logbufs=8 0 1

Im running debian lenny if it helps.

complistic

Posted 2011-02-08T05:38:04.823

Reputation: 313

Answers

6

there is a solution which perfectly fits your use case: http://code.google.com/p/pagecache-mangagement/

After checking out the source code:

svn checkout http://pagecache-mangagement.googlecode.com/svn/trunk/ pagecache-mangagement-read-only

issue

make
sudo cp *.so /usr/local/lib/
sudo cp *.sh /usr/local/bin/

then, you can start whatever application you want with

pagecache-mangagement.sh <App>

and the App won't fill up your caches!

Robert Metzger

Posted 2011-02-08T05:38:04.823

Reputation: 238

1This was just what I needed for a different job, same problem. Now my other programs can use there cache where it's more useful. – complistic – 2013-06-05T04:47:07.520

6

The nocache tool is designed for a similar goal: preventing specific commands from spoiling the current buffer cache. Just prepend ./nocache to your command:

./nocache cp -a ~/ /mnt/backup/home-$(hostname)

Note that the similar tool pagecache-mangagement - A tool to permit the management of the pagecache usage of arbitrary applications, as noted by rmetzger, is still in the same state as it was in 2008. Of course that can be either good or bad, or a mix :) I don't have a direct comparison, but I've asked the nocache folks for one

nealmcb

Posted 2011-02-08T05:38:04.823

Reputation: 496

5

This cannot be done cleanly. If you require this for certain applications then they should be modified to pass O_DIRECT to their open(2) calls.

Ignacio Vazquez-Abrams

Posted 2011-02-08T05:38:04.823

Reputation: 100 516

The files are written/read with various 3rd party applications... mv, md5sum, samba and the like. I maybe able to replace some of these with my own C app in the future... but as far as i can tell samba dosnt have a O_DIRECT option. – complistic – 2011-02-08T06:48:48.943

This seems to be the closet i could get. I'm now using O_DIRECT to copy from P2 card to the HDD and then again from P2 card to md5sum for checking. This removes the useless half of the cache so the only things that are cached video files on the HDD. – complistic – 2011-04-12T21:14:33.423

2

man raw. You can bind a raw device that bypasses the cache with that. It has some restrictions. I've only used it with CD/DVD devices, so I don't know how well it will work for you.

Keith

Posted 2011-02-08T05:38:04.823

Reputation: 7 263

Does the raw command bind the divice in a way i can mount my xfs partition to /video for use with samba? The documentation and google results seem to be a bit vague. – complistic – 2011-02-08T06:37:09.920

You shouldn't mix them. I think you can mount the new raw device instead. – Keith – 2011-02-08T06:39:00.530

Okay I will test this on an old blank drive when I get a chance. – complistic – 2011-02-08T06:55:12.713