How to find what is written to filesystem under linux

Question

How can i find out what processes write to a specific disc over time?

In my particular case I got a little homeserver running 24/7 and I included a script in the crontab to shutdown all drives that are not used (no change in /proc/diskstats for 15 minutes).

But my system disc won't come down at all. I'm suspecting logs but it's probably not only logs writing to the filesystem on the system disk and I don't want to go all the way moving the logfiles to something else just to find out the disc still doesn't spin down and there's nothing i can do against it.

Answer 1

I don't know, but I am guessing that there is nothing you can do about this. Here's my reasoning:

When you say "system disk", I imagine that you mean the disk that holds the root file system.

The root file system holds several directories and files which are monitored by various utilities, including /etc/cron.* which is watched and acted upon by cron. You probably could put /etc on a file system different from /, but it almost certainly would be more trouble than it's worth, and it certainly isn't a common setup.

/proc/diskstats includes disk read activity, not only writes.

Every time cron executes your script to check if there has been any disk activity, that almost certainly results in multiple reads from (even if not writes to) various locations: crontabs, command interpreters, libraries used by these, etc. Unless you are running your system with noatime, these reads also result in writes to update the access time timestamps on the files/inodes in question.

Thus, every time your script executes to check whether there has been any disk activtiy according to /proc/diskstats, that in itself results in disk activity according to /proc/diskstats. Your criteria of "no changed values" thus will never be met, and the drive will never spin down.

If this is a major concern to you, I would suggest moving everything that does not absolutely have to be on the root file system to a different file system, and run the root file system off a SSD or flash device, perhaps even mounted read-only. That should allow the spinning-platter drive to shut down at times when there is no explicit activity. However, I imagine that doing so is likely more trouble than it's worth. If you just want the drives to spin down to get rid of the noise, moving what you now have on the root file system to a small solid-state device (SSD or flash) will likely accomplish the same thing with considerably less effort. Just take into account the fact that those have a limited number of write cycles before failure (at a minimum, consider running with noatime, or with a file system designed with SSDs in mind).

Answer 2

Use command iotop.

This will show you in detail all commands that are writing to the disk.

Example output:

Total DISK READ: 0.00 B/s | Total DISK WRITE: 0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
    1 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % init
    2 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kthreadd]
    3 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [ksoftirqd/0]
    4 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kworker/0:0]
    6 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/0]
    7 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [watchdog/0]
    8 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/1]
   10 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [ksoftirqd/1]
   11 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kworker/0:1]
   12 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [watchdog/1]
   13 be/0 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [cpuset]
   14 be/0 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [khelper]
   15 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kdevtmpfs]
   16 be/0 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [netns]
   17 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [sync_supers]
   18 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [bdi-default]
   19 be/0 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kintegrityd]
   20 be/0 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kblockd]
   21 be/0 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [ata_sff]
   22 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [khubd]
   23 be/0 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [md]
   24 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kworker/1:1]
   25 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [khungtaskd]
   26 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kswapd0]
   27 be/5 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [ksmd]

Additionaly use command iostat which example output look like this:

Linux 3.2.6 (z)         10/03/2012      _i686_  (2 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.14    0.01    0.26    0.01    0.00   99.59

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda               0.81        13.67         9.14    8968739    5996546

Please also try harder when searching for already answered questions.