4

For a Linux NFSv3 server, is there any way to see statistics on actual file access? In other words, I would like to see a timestamped log of every file read or written.

Motivation: I have several NFS servers that are slated to be replaced by some kind of "big iron" storage system. In my particular case, the NFS client load is virtually all reads of large files (averaging around 700 MB). One vendor's solution uses a large RAM disk as a cache to provide the kind of massive random read throughput that I need.

What I want is a way to study what the NFS file access pattern looks like so that I can "right size" the cache.

Matt
  • 1,037
  • 2
  • 14
  • 20

3 Answers3

5

I've been banging my head against this problem all day, with no solution found.

  1. You can turn up the debugging of the NFS server, but that doesn't provide much detail (if that exmaple is accurate) and will probably dominate a busy NFS server's disk with useless baggage logged in addition to the bare filenames.

  2. Another solution is adding rules to auditd/auditctl to log all reads or writes to the NFS directories, but that doesn't work for our Centos 6.X machines, for reasons I can't quite figure out yet. In /etc/audit/audit.rules on a client machine:

    # First rule - delete all
-D

# Increase the buffers to survive stress events.
# Make this bigger for busy systems
-b 8192

# Feel free to add below this line. See auditctl man page
-w /auto/ -p r -k read -k home
-w /auto/ -p w -k write -k home
-w /auto/ -p xa -k other -k home

...where I've given separate keys to reading, writing, and executing/changing attributes. My clients are autofs'd to mount a few different NFS directories, including their home directory, to /auto/ with soft links pointing the client machine's /home/users/ back to /auto/. I get logging of lots of stuff, but none of the files the users themselves seem to be modifying.

Troll the audit logs with ausearch -k read | aureport -f, for instance. grepping for .ODT or .PDF comes up with nothing, the only results are for metacity's configs, Chrome's crap, etc., etc.

Naturally, enabling audit on the server pointing at the real /home/users/XYZ only shows accesses from things interfacing with the server directly (mail clients) or users SSH'd into the server.

If you can figure out the right recipe for audit, or a dedicated solution all together, please, please, please share it! You'd think this would have been solved in 1993.

Eric L.
  • 205
  • 4
  • 12
3

Take a look at nfstrace https://github.com/epam/nfstrace. It captures network traffic and perform deep packet inspection. It finds NFSv3/v4 procedures (READ, WRITE etc.) and tracks its statistics. Also, you can develop new pluggable modules for nfstrace.

Pavel Karneliuk
  • 31
  • 1
  • 3
    Nice tool, but you ought to disclose [your association with the project](https://github.com/epam/nfstrace#authors) whilst plugging it (see, eg, http://serverfault.com/help/behavior). – MadHatter May 08 '15 at 10:36
0

Take a look at nfsstat. It can produce statistics for both a NFS client and server. It's a part of nfs-common on Ubuntu Linux. You can also look at iotop for the server-side, but that reports all disk I/O, not just NFS.

penguin359
  • 452
  • 3
  • 8
  • Unless my version of nfsstat is too old, it doesn't have the granularity that I need. I'm looking for _per file_ statistics. – Matt Apr 08 '11 at 13:28