finding directories that consume a lot of the AFS volume quota

Question

The AFS volume quota is almost exceeded for one of our computer users. Running fs listquota or fs lq gives him a warning:

olifri@ubuntu:~$ fs listquota ~
Volume Name                    Quota       Used %Used   Partition
H.olifri                      500000     492787   99%<<       38% <<WARNING

I would like to share with him a shell script that would detect which of his directories consume most of his AFS volume quota. He would then have a better idea of where to start deleting files.

Note that in the AFS filesystem a user can mount other AFS volumes inside an AFS volume. This means that a recursion with /usr/bin/find might enter other AFS volumes that we are not interested in.

Our AFS client software is Openafs 1.6.1 and it runs on Ubuntu 12.04 computers. We have no admin rights on the AFS file servers as they are administrated by another department.

The command /usr/bin/du seems to be unaware of the concept of AFS volumes. The best idea I have right now is to write a script that for each subdirectory will test if the directory is a mount point for an AFS volume. For instance the command fs lsmount could be used for that. For a normal directory fs lsmount produces this result

esjolund@ubuntu:~$ mkdir ~/testdir
esjolund@ubuntu:~$ fs lsmount ~/testdir
'/afs/pdc.kth.se/home/e/esjolund/testdir' is not a mount point.

and for an AFS mount point fs lsmount produces this result

esjolund@ubuntu:~$ fs mkmount -dir ~/testmount -vol prj.sbc.esjolund00
esjolund@ubuntu:~$ fs lsmount ~/testmount
'/afs/pdc.kth.se/home/e/esjolund/testmount' is a mount point for volume '#prj.sbc.esjolund00'

Before starting to write the shell script, I would like to hear if you have any better ideas of how to solve the problem?

score 0 · Accepted Answer · answered Nov 12 '12 at 14:28

#!/usr/bin/python                                                                                                                  
import os
import sys
import subprocess

if len(sys.argv) != 2:
  print >> sys.stderr, "error: Wrong number of arguments. One argument expected (the directory name)"
  sys.exit(1)

for dirpath, dirnames, filenames in os.walk(sys.argv[1]):
  for dirname in dirnames:
    subdirpath = os.path.join(dirpath, dirname)
    p = subprocess.Popen(["fs", "lsmount", subdirpath], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    # It seems we can distinguish an AFS volume mount point from a normal directory by just                                        
    # looking at the return code.                                                                                                  
    # Maybe we should also look at the stdout and stderr?                                                                          
    # (In other words the variables out and err)                                                                                   
    out, err = p.communicate()
    if ( p.returncode == 0 ):
      dirnames.remove(dirname)
  total_size = 0
  for filename in filenames:
    filepath = os.path.join(dirpath, filename)
    statinfo = os.lstat(filepath)
    total_size += statinfo.st_size
  print "%i %s" % (total_size, dirpath)

Use the command like this

olifri@ubuntu:~$ python /tmp/script.py  ~ | sort -n

finding directories that consume a lot of the AFS volume quota

1 Answers1