why does `du` not show results for all files?

I (for curiosity's sake) am running du -a inside /usr/lib/git-core and it does not show results for all the files in that directory. Why does it leave out an arbitrary set of files?

this is what cd /usr/lib/git-core; du -a returns:

4   ./git-merge-resolve
972 ./git-config
12  ./git-show-index
404 ./git-remote-ftp
4   ./git-difftool
16  ./git-rebase
20  ./git-submodule
40  ./git-issues
12  ./git-filter-branch
36  ./git-add--interactive
4   ./git-web--browse
12  ./git-bisect
8   ./git-relink
412 ./git-fast-import
4   ./git-merge-one-file
248 ./git-instaweb
20  ./git-am
4   ./git-lost-found
4   ./git-notes
384 ./git-daemon
8   ./t_gitshelve.py
376 ./git-http-backend
8   ./git-repack
20  ./gitshelve.py
4   ./git-parse-remote
24  ./git-rebase--interactive
4   ./git-quiltimport
380 ./git-imap-send
8   ./git-pull
4   ./git-sh-setup
4   ./git-difftool--helper
4   ./git-request-pull
372 ./git-shell
4   ./git-merge-octopus
392 ./git-http-fetch
8   ./git-mergetool
380 ./git-upload-pack
12  ./git-mergetool--lib
12  ./git-stash
404 ./git-http-push
5052    .

but when I run ls, many more files show as existing in the current directory:

git
git-add
git-add--interactive
git-am
git-annotate
git-apply
git-archive
git-bisect
git-bisect--helper
git-blame
git-branch
git-bundle
git-cat-file
git-check-attr
git-check-ref-format
git-checkout
git-checkout-index
git-cherry
git-cherry-pick
git-clean
git-clone
git-commit
git-commit-tree
git-config
git-count-objects
git-daemon
git-describe
git-diff
git-diff-files
git-diff-index
git-diff-tree
git-difftool
git-difftool--helper
git-fast-export
git-fast-import
git-fetch
git-fetch-pack
git-filter-branch
git-fmt-merge-msg
git-for-each-ref
git-format-patch
git-fsck
git-fsck-objects
git-gc
git-get-tar-commit-id
git-grep
git-hash-object
git-help
git-http-backend
git-http-fetch
git-http-push
git-imap-send
git-index-pack
git-init
git-init-db
git-instaweb
git-issues
git-log
git-lost-found
git-ls-files
git-ls-remote
git-ls-tree
git-mailinfo
git-mailsplit
git-merge
git-merge-base
git-merge-file
git-merge-index
git-merge-octopus
git-merge-one-file
git-merge-ours
git-merge-recursive
git-merge-resolve
git-merge-subtree
git-merge-tree
git-mergetool
git-mergetool--lib
git-mktag
git-mktree
git-mv
git-name-rev
git-notes
git-pack-objects
git-pack-redundant
git-pack-refs
git-parse-remote
git-patch-id
git-peek-remote
git-prune
git-prune-packed
git-pull
git-push
git-quiltimport
git-read-tree
git-rebase
git-rebase--interactive
git-receive-pack
git-reflog
git-relink
git-remote
git-remote-ftp
git-remote-ftps
git-remote-http
git-remote-https
git-repack
git-replace
git-repo-config
git-request-pull
git-rerere
git-reset
git-rev-list
git-rev-parse
git-revert
git-rm
git-send-pack
git-sh-setup
git-shell
git-shortlog
git-show
git-show-branch
git-show-index
git-show-ref
git-stage
git-stash
git-status
git-stripspace
git-submodule
git-symbolic-ref
git-tag
git-tar-tree
git-unpack-file
git-unpack-objects
git-update-index
git-update-ref
git-update-server-info
git-upload-archive
git-upload-pack
git-var
git-verify-pack
git-verify-tag
git-web--browse
git-whatchanged
git-write-tree
gitshelve.py
t_gitshelve.py

Can anyone explain why du is returning info for only some files? I see no pattern here.

Running file $(du -a) | grep "^\." does not show any pattern either.

Alexander Bird

Posted 2011-05-22T23:28:04.740

Reputation: 1 697

Does this happen for other directories as well? What about the output of ls -la? – Bandit – 2011-05-22T23:54:39.907

In every other folder I try somewhere in ~, I find that du -a returns data for every file that ls -aR returns. (I tested that by running the following command in ~: ls -aR1 | grep -E "^[^.]|^\.[^.].*[^:]$|^\.:$" | wc -l; du -a | wc -l I know that's complicated, but please trust me that it's a good test, or show me what's wrong with it). – Alexander Bird – 2011-05-23T00:46:30.927

i see the same thing too on ubuntu natty .. under /usr/lib/git-core – freethinker – 2011-05-23T00:51:16.383

ls -aR1 | grep -E "^[^.]|^\.[^.].*[^:]$" > ../tmpls ; du -a | gawk 'match($2, /\.\/(.*)/, ary) {print ary[1]}' > ../tmpdu; sort ../tmpdu > ../tmpdus; sort ../tmpls > ../tmplss; diff ../tmplss ../tmpdus | grep '^<' | gawk '{print $2}' shows which files are not captured by du -a (for anyone else who may be obsessed with finding an answer like I am =D ). Interesting to note: all files left out by du are ELF files. Also, there are many, many files within /usr/lib which du decides to leave out... – Alexander Bird – 2011-05-23T01:06:16.347

AHA!! I actually already thought of hard links being the problem, but unfortunately for me, I thought that ls -l would show a syntax similar to soft links for all hard links. That is why I missed that they are sharing the same inode. You are my hero right now :-) – Alexander Bird – 2011-05-23T01:08:27.283

Strictly speaking, a better answer would be ls -AiR1, (1) to include “.” files, (2) because links might be in different directories, and (3) because -i works fine without -l. But congratulations on finding the basic issue. – Scott – 2014-06-13T14:03:13.033

why does `du` not show results for all files?

Answers