I have noticed 15% CPU load on the Server which is currently offline. It has mounted GlusterFS volume over TCP. Looking through the top, it shown me it's glusterfs. After that I tried to figure what exactly is using it and I got this:
# lsof /storage/
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
find 16433 nobody cwd DIR 0,19 8192 9259265867489333824 /storage/200000/200000/200700/200704/08
Then:
# ps uax | grep find
root 16415 0.0 0.0 4400 724 ? SN 06:34 0:00 /bin/sh /usr/bin/updatedb.findutils
root 16423 0.0 0.0 4400 336 ? SN 06:34 0:00 /bin/sh /usr/bin/updatedb.findutils
nobody 16431 0.0 0.0 39524 1376 ? SN 06:34 0:00 su nobody -s /bin/sh -c /usr/bin/find / -ignore_readdir_race \( -fstype NFS -o -fstype nfs -o -fstype nfs4 -o -fstype afs -o -fstype binfmt_misc -o -fstype proc -o -fstype smbfs -o -fstype autofs -o -fstype iso9660 -o -fstype ncpfs -o -fstype coda -o -fstype devpts -o -fstype ftpfs -o -fstype devfs -o -fstype mfs -o -fstype shfs -o -fstype sysfs -o -fstype cifs -o -fstype lustre_lite -o -fstype tmpfs -o -fstype usbfs -o -fstype udf -o -fstype ocfs2 -o -type d -regex '\(^/tmp$\)\|\(^/usr/tmp$\)\|\(^/var/tmp$\)\|\(^/afs$\)\|\(^/amd$\)\|\(^/alex$\)\|\(^/var/spool$\)\|\(^/sfs$\)\|\(^/media$\)\|\(^/var/lib/schroot/mount$\)' \) -prune -o -print0
nobody 16432 0.0 0.0 4400 616 ? SN 06:34 0:00 sh -c /usr/bin/find / -ignore_readdir_race \( -fstype NFS -o -fstype nfs -o -fstype nfs4 -o -fstype afs -o -fstype binfmt_misc -o -fstype proc -o -fstype smbfs -o -fstype autofs -o -fstype iso9660 -o -fstype ncpfs -o -fstype coda -o -fstype devpts -o -fstype ftpfs -o -fstype devfs -o -fstype mfs -o -fstype shfs -o -fstype sysfs -o -fstype cifs -o -fstype lustre_lite -o -fstype tmpfs -o -fstype usbfs -o -fstype udf -o -fstype ocfs2 -o -type d -regex '\(^/tmp$\)\|\(^/usr/tmp$\)\|\(^/var/tmp$\)\|\(^/afs$\)\|\(^/amd$\)\|\(^/alex$\)\|\(^/var/spool$\)\|\(^/sfs$\)\|\(^/media$\)\|\(^/var/lib/schroot/mount$\)' \) -prune -o -print0
nobody 16433 0.3 0.0 13612 1532 ? SN 06:34 0:38 /usr/bin/find / -ignore_readdir_race ( -fstype NFS -o -fstype nfs -o -fstype nfs4 -o -fstype afs -o -fstype binfmt_misc -o -fstype proc -o -fstype smbfs -o -fstype autofs -o -fstype iso9660 -o -fstype ncpfs -o -fstype coda -o -fstype devpts -o -fstype ftpfs -o -fstype devfs -o -fstype mfs -o -fstype shfs -o -fstype sysfs -o -fstype cifs -o -fstype lustre_lite -o -fstype tmpfs -o -fstype usbfs -o -fstype udf -o -fstype ocfs2 -o -type d -regex \(^/tmp$\)\|\(^/usr/tmp$\)\|\(^/var/tmp$\)\|\(^/afs$\)\|\(^/amd$\)\|\(^/alex$\)\|\(^/var/spool$\)\|\(^/sfs$\)\|\(^/media$\)\|\(^/var/lib/schroot/mount$\) ) -prune -o -print0
I killed 16432 and 16433 and CPU now is %0 again.
Can someone tell me anything about these ugly find commands? Is it possible it's cause by the other Server which also have this /storage mounted?
According monitoring, it happens every day at the same time.