3

I'm running Ubuntu and have had a problem where the root partition has run out of disk space.

When I perform df -h I get the following:

Filesystem            Size  Used Avail Use% Mounted on
/dev/sda6             4.6G  4.5G     0 100% /

Yet there are only 2GB of files actually using up this partition. I then ran the following df -i and I get the following:

Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/sda6             305824  118885  186939   39% /

I have no idea what the -i flag does but it clearly shows that only 39% is used.

Can anyone explain where my disk space has gone?

Camsoft
  • 911
  • 4
  • 12
  • 21

4 Answers4

8

You can try:

lsof +L1

to show unlinked files including their sizes and what has them open.

Dennis Williamson
  • 60,515
  • 14
  • 113
  • 148
6

You probably have a process keeping a file open that's been deleted. Deleted but open files keep consuming disk space, but don't show up in du or ls. "df -i" is a red herring, it just shows you how many files you have, or more technically, how many inodes.

If you can't figure out what process is holding a big (deleted) file open, and chances are it's a zombie or something hung, it might be time for a reboot.

Paul Tomblin
  • 5,217
  • 1
  • 27
  • 39
  • I think reboot may be the best option. I'm just worried that it will fail to boot due to there being no disk space on / – Camsoft Mar 16 '10 at 11:58
3

df -i shows inodes, which is not what you are looking for. Do you have an unusually large block size, maybe 128/256k? If you do, even a 1K file will take up that amount of space.

MDMarra
  • 100,183
  • 32
  • 195
  • 326
2

df -i show how many of your i-nodes you have used, not how much space you have used.

Some Unix/Linux filesystems (you don't say what you use), do have a limit on how many files/i-nodes they can hold in addition to the disk space limit. That means if you have many very small files your filesystem might be full even with space remaining because all i-nodes are used up. To check this you can look at the i-nodes used with the df -i command.

markus_b
  • 351
  • 1
  • 5
  • My question clearly shows the output of the `df -i` command. – Camsoft Mar 16 '10 at 12:15
  • To see where your disk space has gone you can use the 'du' command. It will show you, directory by directory, how much space is used. Start with 'du -xk /', maybe piped to 'less', as the list will be long. – markus_b Mar 16 '10 at 12:17
  • 1
    Yes, but 'df -i' is irrelevant in your situation as MarkM already stated. Your disk is full and the normal 'df' (no options shows this). – markus_b Mar 16 '10 at 12:19
  • I've already used `du` and I can only account for 2GB out of the 4GB `df` says is used. – Camsoft Mar 16 '10 at 12:25
  • Re-reading your question I have some additional remarks: - Start with removing the apt cache: 'rm -r /var/cache/apt/archives/*', this gives you some space - Install 'baobab', a graphical tool to see what is using disk-space - I think your root partition is too small with 4.5G I'd use a bigger partition (10-20G) – markus_b Mar 16 '10 at 12:26
  • If you can not account for all the space with 'du' the it might be that a running process has a big, but unlinked (deleted) file open. Such files do not show up with 'du' but use space in 'df'. You need to reboot to eliminate that possibility. – markus_b Mar 16 '10 at 12:29
  • `/var` is mounted to a different partition and has plenty of space left. I checked all the mount points that are on `sda6` and can only find 2GB of files using `du`. I don't think the server is running X11 and I don't have VNC access only SSH. – Camsoft Mar 16 '10 at 12:31
  • I moved 1GB of Apache log files from /etc to a different partition but it did not fee up any space. – Camsoft Mar 16 '10 at 12:31
  • Did you restart apache ? (absolutely essential, apache keeps the files open and they are not removed before you restart it !) – markus_b Mar 16 '10 at 12:33
  • You should change your apache config to write the logs to /var, otherwise you'll have the problem again shortly ! – markus_b Mar 16 '10 at 12:34
  • 1
    @markus_b - gee, I wish I'd said that 22 minutes before you. – Paul Tomblin Mar 16 '10 at 12:34
  • Yes, about everything we are discussing now have been said before ! – markus_b Mar 16 '10 at 12:37
  • Wer lesen kann, ist im Vorteil ! He He :-) – markus_b Mar 16 '10 at 12:38
  • Will reboot the server tomorrow morning as sites on server get 100k+ traffic – Camsoft Mar 16 '10 at 12:48
  • 1
    You can do a graceful restart: 'apache2ctl graceful', this will not interrupt your website. – markus_b Mar 16 '10 at 12:58