Linux utility for finding the largest files/directories

Question

I'm looking for a program to show me which files/directories occupy the most space, something like:

74% music
 \- 60% music1
 \- 14% music2
12% code
13% other

I know that it's possible in KDE3, but I'd rather not do that - KDE4 or command line are preferred.

Answer 1

To find the largest 10 files (linux/bash):

find . -type f -print0 | xargs -0 du | sort -n | tail -10 | cut -f2 | xargs -I{} du -sh {}

To find the largest 10 directories:

find . -type d -print0 | xargs -0 du | sort -n | tail -10 | cut -f2 | xargs -I{} du -sh {}

Only difference is -type {d:f}.

Handles files with spaces in the names, and produces human readable file sizes in the output. Largest file listed last. The argument to tail is the number of results you see (here the 10 largest).

There are two techniques used to handle spaces in file names. The find -print0 | xargs -0 uses null delimiters instead of spaces, and the second xargs -I{} uses newlines instead of spaces to terminate input items.

example:

$ find . -type f -print0 | xargs -0 du | sort -n | tail -10 | cut -f2 | xargs -I{} du -sh {}

  76M    ./snapshots/projects/weekly.1/onthisday/onthisday.tar.gz
  76M    ./snapshots/projects/weekly.2/onthisday/onthisday.tar.gz
  76M    ./snapshots/projects/weekly.3/onthisday/onthisday.tar.gz
  76M    ./tmp/projects/onthisday/onthisday.tar.gz
  114M   ./Dropbox/snapshots/weekly.tgz
  114M   ./Dropbox/snapshots/daily.tgz
  114M   ./Dropbox/snapshots/monthly.tgz
  117M   ./Calibre Library/Robert Martin/cc.mobi
  159M   ./.local/share/Trash/files/funky chicken.mpg
  346M   ./Downloads/The Walking Dead S02E02 ... (dutch subs nl).avi

Answer 2

I always use ncdu. It's interactive and very fast.

Answer 3

For a quick view:

du | sort -n

lists all directories with the largest last.

du --max-depth=1 * | sort -n

or, again, avoiding the redundant * :

du --max-depth=1 | sort -n

lists all the directories in the current directory with the largest last.

(-n parameter to sort is required so that the first field is sorted as a number rather than as text but this precludes using -h parameter to du as we need a significant number for the sort)

Other parameters to du are available if you want to follow symbolic links (default is not to follow symbolic links) or just show size of directory contents excluding subdirectories, for example. du can even include in the list the date and time when any file in the directory was last changed.

Answer 4

For most things, I prefer CLI tools, but for drive usage, I really like filelight. The presentation is more intuitive to me than any other space management tool I've seen.

Answer 5

Filelight is better for KDE users, but for completeness (question title is general) I must mention Baobab is included in Ubuntu, aka Disk Usage Analyzer:

Answer 6

A GUI tool, KDirStat, shows the data both in table form and graphically. You can see really quickly where most of the space is used.

I'm not sure if this is exactly the KDE tool you didn't want, but I think it still should be mentioned in a question like this. It's good and many people probably don't know it - I only learned about it recently myself.

Answer 7

A Combination is always the best trick on Unix.

du -sk $(find . -type d) | sort -n -k 1

Will show directory sizes in KB and sort to give the largest at the end.
Tree-view will however needs some more fu... is it really required?

Note that this scan is nested across directories so it will count sub-directories again for the higher directories and the base directory . will show up at the end as the total utilization sum.

You can however use a depth control on the find to search at a specific depth.
And, get a lot more involved with your scanning actually... depending on what you want. Depth control of find with -maxdepth and -mindepth can restrict to a specific sub-directory depth.

---

Here is a refined variation for your arg-too-long problem

find . -type d -exec du -sk {} \; |  sort -n -k 1

Answer 8

I like gt5. You can navigate the tree and open subdirectories to drill down for more detail. It uses a text-mode web browser, such as lynx, to display the results. Install elinks for best results.

Answer 9

Although it does not give you a nested output like that, try du

du -h /path/to/dir/

Running that on my Documents folder spits out the following:

josh-hunts-macbook:Documents joshhunt$ du -h
  0B    ./Adobe Scripts
  0B    ./Colloquy Transcripts
 23M    ./Electronic Arts/The Sims 3/Custom Music
  0B    ./Electronic Arts/The Sims 3/InstalledWorlds
364K    ./Electronic Arts/The Sims 3/Library
 77M    ./Electronic Arts/The Sims 3/Recorded Videos
101M    ./Electronic Arts/The Sims 3/Saves
 40M    ./Electronic Arts/The Sims 3/Screenshots
1.6M    ./Electronic Arts/The Sims 3/Thumbnails
387M    ./Electronic Arts/The Sims 3
387M    ./Electronic Arts
984K    ./English Advanced/Documents
1.8M    ./English Advanced
  0B    ./English Extension/Documents
212K    ./English Extension
100K    ./English Tutoring
5.6M    ./IPT/Multimedia Assessment Task
720K    ./IPT/Transaction Processing Systems
8.6M    ./IPT
1.5M    ./Job
432K    ./Legal Studies/Crime
8.0K    ./Legal Studies/Documents
144K    ./Legal Studies/Family/PDFs
692K    ./Legal Studies/Family
1.1M    ./Legal Studies
380K    ./Maths/Assessment Task 1
388K    ./Maths
[...]

Then you can sort the output by piping it through to sort

du /path/to/dir | sort -n

Answer 10

Here is the script which does it for you automatically.

http://www.thegeekscope.com/linux-script-to-find-largest-files/

Following is the sample output of the script:

**# sh get_largest_files.sh / 5**

[SIZE (BYTES)]     [% OF DISK] [OWNER]         [LAST MODIFIED ON]        [FILE] 

56421808           0%           root           2012-08-02 14:58:51       /usr/lib/locale/locale-archive
32464076           0%           root           2008-09-18 18:06:28       /usr/lib/libgcj.so.7rh.0.0
29147136           0%           root           2012-08-02 15:17:40       /var/lib/rpm/Packages
20278904           0%           root           2008-12-09 13:57:01       /usr/lib/xulrunner-1.9/libxul.so
16001944           0%           root           2012-08-02 15:02:36       /etc/selinux/targeted/modules/active/base.linked

Total disk size: 23792652288 Bytes
Total size occupied by these files: 154313868 Bytes  [ 0% of Total Disc Space  ]

*** Note: 0% represents less than 1% ***

You may find this script very handy and useful !

Answer 11

Although finding the percentage disk usage of each file/directory is beneficial, most of the time knowing largest files/directories inside the disk is sufficient.

So my favorite is this:

# du -a | sort -n -r | head -n 20

And output is like this:

28626644        .
28052128        ./www
28044812        ./www/vhosts
28017860        ./www/vhosts/example.com
23317776        ./www/vhosts/example.com/httpdocs
23295012        ./www/vhosts/example.com/httpdocs/myfolder
23271868        ./www/vhosts/example.com/httpdocs/myfolder/temp
11619576        ./www/vhosts/example.com/httpdocs/myfolder/temp/main
11590700        ./www/vhosts/example.com/httpdocs/myfolder/temp/main/user
11564748        ./www/vhosts/example.com/httpdocs/myfolder/temp/others
4699852         ./www/vhosts/example.com/stats
4479728         ./www/vhosts/example.com/stats/logs
4437900         ./www/vhosts/example.com/stats/logs/access_log.processed
401848          ./lib
323432          ./lib/mysql
246828          ./lib/mysql/mydatabase
215680          ./www/vhosts/example.com/stats/webstat
182364          ./www/vhosts/example.com/httpdocs/tmp/aaa.sql
181304          ./www/vhosts/example.com/httpdocs/tmp/bbb.sql
181144          ./www/vhosts/example.com/httpdocs/tmp/ccc.sql

Answer 12

To find the top 25 files in the current directory and its subdirectories:

find . -type f -exec ls -al {} \; | sort -nr -k5 | head -n 25

The will output the top 25 files by sorting based on the size of the files via the "sort -nr -k5" piped command.

Answer 13

Another alternative is agedu which breaks down disk space by last-access time, which makes it easier to locate space wasting files.

It even works on a server without X Windows by serving temporary web pages so usage can be analysed remotely, with graphs. Assuming the IP address of the server is 192.168.1.101, you can type this on the command line of the server

agedu -s / -w --address 192.168.1.101:60870 --auth basic -R

This prints the username, password and URL with which you can access the "GUI" and browse the results. When done, terminate agedu with Ctrl+D on the server.

Answer 14

du -chs /*

Will show you a list of the root directory.

Answer 15

To complete the list a little bit more, I add my favorite disk usage analyzer, which is xdiskusage.

The GUI remembers me of some other good ol' X utilities, it's fast and not bloated, but you can nevertheless navigate easily in the hierarchy and have some display options:

$ xdiskusage /usr

Answer 16

Try the following one-liner (displays top-20 biggest files in the current directory):

ls -1Rs | sed -e "s/^ *//" | grep "^[0-9]" | sort -nr | head -n20

or with human readable sizes:

ls -1Rhs | sed -e "s/^ *//" | grep "^[0-9]" | sort -hr | head -n20

The second command to work on OSX/BSD properly (as sort doesn't have -h), you need to install sort from coreutils.

So these aliases are useful to have in your rc files (every time when you need it):

alias big='du -ah . | sort -rh | head -20'
alias big-files='ls -1Rhs | sed -e "s/^ *//" | grep "^[0-9]" | sort -hr | head -n20'