How can a linux disk be full, but files total to significanly less?

Question

df reports 16G partition at 100%. du on root reports about 1G in files. Where did all my disk space go?

Answer 1

Files can be hidden by a mount point.

(A scheduled program (backup) must have run while while I had the secondary volume offline for maintenance, and gleefully recreated it's directory structure rather than failing.)

Answer 2

In addition to the wasted space matter that Doug discusses, some file systems have a limited number of inodes (unique file IDs) in total, and once those have been created there is no way to make new files. Use df -i to see where you stand on inodes.

Further, it is possible for ext[23] filesystems (and others?) to reserve a fraction of the disk to the superuser. Normally this is only a few percent (5 is typical on older systems), but it can be set much higher than that. I believe you want to use tune2fs here (Mac OS and possibly other unixes calls this tunefs) with the -m option.

Answer 3

You might have programs running with old unlinked files open. Those files are still stored on disk until those programs close, but will not be linked from the filesystem tree.

Note that this is different from windows behavior, which will not let you delete such a file.

Answer 4

Do you have a large number of very very small files? Linux (and most operating systems) write files in block-sized chunks (say 4k), and every file uses a multiple of that size. For example, if you have a 121 byte file, 4096 bytes are still used on the disk. The rest of the space in that block are essentially "wasted".

Answer 5

I had this exact same problem. I had an issue where my primary boot disk was filling up after I performed a sync of backup drives.

>rsync -avxHAXW /media/backup1 /media/backup2

After this sync my primary disk was full.... which baffled me. It turns out I was accidentally copying files to my boot disk, not my backup2. I corrected the mistake, deleted the massive file on my primary disk and reran rsync. Even after unmounting and remounting my backup disks my boot disk was out of space. I even tried cleaning up large files on my boot disk....

>find . -type f -print0 | xargs -0 du -s | sort -n | tail -25 | cut -f2 | xargs -I{} du -sh {}

Deleted any large files that were no longer necessary. I also made sure to empty the trash.

>empty-trash

After all of that work... disk still full. I decided to reboot the machine. After the reboot, my boot disk partition went from 100% full to 10% full! It appears the rsync process must have continued to run in the background holding onto disk resources. I hate giving this advice, but after doing all of the disk cleanup you might just need to reboot your machine to kill off any background processes that are hanging onto data, preventing disk from truly being freed.