9

My system is CentOS 6 x86_64 with root partition formatted as ext4. df reports around 3Gb as used space:

Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/md1              20158260   3433724  15700540  18% /

but du -sm -x / claims less than a single Gb is used actually:

[root@xxxx ~]# du -sm -x /
948     /

I wonder what is going on here. Usage numbers changed right after reboot. The filesystem claims it's clean, no errors in logs. I found this, but it did not explain a root of the problem. Should I just reformat the partition? Is there any way to track down this extra usage?

I also did the following to check that I don't have any data hidden from du by mounts on top of non-empty mountpoints:

[root@xxxx ~]# mount -o bind / /mnt/root
[root@xxxx ~]# du -sm /mnt/root/
949     /mnt/root/
[root@xxxx ~]#

No, that's not my case.

Alex
  • 7,789
  • 4
  • 36
  • 51
  • 2
    possible duplicate of [du vs. df difference](http://serverfault.com/questions/57098/du-vs-df-difference) – user9517 Sep 13 '11 at 06:52
  • @Iain Checked the link, thank you. The only relevant thing seems to be a lost mount problem (and the accepted solution on this link). I do not have small files and I have a similar box with the same partition layout and same files on / but it does not report 3G of used space. And I do not have a lost mount problem, but this is a subject to double check. – Alex Sep 13 '11 at 07:37
  • @Iain Checked the lost mount problem, no, that's not it. I will update my question. – Alex Sep 13 '11 at 09:28

4 Answers4

11

The first thing I would think is that you have deleted files. Using lsof -n | grep deleted will help you. Does the output of that command shows you some files? (maybe you have a huge log file still being written). If you have files opened by a process (a syslogger or something like Apache web server) that are being written into they may be using a lot of disk space and the easies way will be restarting the process owning of those deleted files.

If there aren't any deleted files, could you paste the output of running tune2fs -l ?

Pablo Martinez
  • 2,326
  • 16
  • 13
  • Yeah, that was it, thank you! Several qemu `.save` files, pretty huge. – Alex Sep 13 '11 at 10:07
  • 1
    Before I even got half way through your question, I was guessing this was the most the most likely cause. It actually makes for a great Unix interview question: "du says I have free space, but I get no space left on device errors. What's going on?" lsof is your friend in many situations. :) – Larold Sep 14 '11 at 05:09
  • 5
    Oh - one other thing to keep in mind. I know ext3 does this, and ext4 probably does as well. There's usually a small percentage of "reserved" space left for the root user. This is in case the filesystem fills up with user data - it still leaves room for the root user to run commands safely, go in, and fix the problem. If the *reserved* space fills up, then you're SOL and just need to delete stuff - very few other commands will work reliably when you're cd'ed into the full filesystem. I recall that 'tune2fs -m' sets the percentage. Not directly related, but a useful tidbit. – Larold Sep 14 '11 at 05:12
5

Force a filesystem check on next reboot. You may have unrecovered inodes that are holding data.

# touch /forcefsck  # Run as root, then reboot.

source: http://ubuntuforums.org/showthread.php?t=1360204&p=9209650#post9209650

user206746
  • 186
  • 1
  • 3
0

Ext3/4 can use up to 400MB for the journal file, which you can not see. The size of the journal size is automatically scaled with the size of the filesystem created or can be manually specified at filesystem creation time.

Joshua Hoblitt
  • 665
  • 4
  • 11
  • According to dumpe2fs, journal is only 128M on my system, and I am trying to reclaim more than 2G of space, as you can see. – Alex Sep 13 '11 at 07:28
-1

Today I had a similar issue with a couple of hosts that are running an application which someone had deleted the .tmp files that it produces but the process still had a file handle open which I found via lsof, when i restarted the process the disk space was released. lsof was a good place to start with figuring this issue out -

Munkeh72
  • 319
  • 2
  • 5
  • 2
    This does not add anything new to the given answers. If the answer by Pablo helped you, upvote it, but please don't add any new answers if you don't have anything to suggest that hasn't been posted before. – Gerald Schneider Feb 06 '19 at 11:13
  • I already did up-vote the answer.. no need to be picky ;) – Munkeh72 Feb 13 '19 at 16:28