3

Just trying to get to the bottom of this. I run a Web app on a LAMP stack on Amazon EC2.

Typically the server is writing about 2.5MB/min to the hard drives. It looks like the apache log file and the amount actually written rises and falls precisely in the same shape as my network in/out graphs. The database is read only and the only writes to hard drive should be logs.

But precisely every 15 minutes the server also does a 12-15MB write to the hard drive - I do not know what this could be. The only clue is that it too follows the shape of my network in/out graph, only more exaggerated. There is no cronjob that runs on that frequency.

How do I go about figuring out what this 12-15MB write is ?

Toby Allen
  • 747
  • 2
  • 10
  • 23
Matt Parkins
  • 145
  • 5
  • 1
    I had a [similar problem](http://serverfault.com/questions/328212/5-5gb-written-daily-to-1-2gb-root-volume-4-times-previous-levels) (perhaps unrelated) where APC was set to use a file backed cache instead of a memory backed cache. You may also want to look into disk caching as a possible cause. Some helpful tools for tracking down the process responsible for the writes include `iotop` and `ftop`. – cyberx86 Jan 19 '12 at 21:45

1 Answers1

2

You could use inotify (http://inotify-tools.sourceforge.net/, the distribution you're using has most likely a package for this) to watch the log directory and see if there is increased activity there. Similarly, you can watch the rest of the file system to see where the writes go. Use something like

inotifywatch -r /var/log -e modify -t <timeout>

Once you have identified the file that is written to, you can use something like lsof (again, most likely available as a package in your distribution) to see which programs access that file, like

lsof /var/log/<file>

If new files are being written or the file descriptor isn't kept open, you can combine inotify and lsof like

inotifywait -e modify /var/log/<file>; lsof -p /var/log/<file>

If none of that helps, have a look at something like SystemTap (http://sourceware.org/systemtap/), although that might require you to compile/install a special kernel.

Lars Kotthoff
  • 646
  • 4
  • 10