7

FINAL RESULT

I fixed this problem by resizing my xvda partition from 700MB to 10GB. I used Linode’s control panel to do that, which involved shutting down the VPS, running the resize operation, and then booting it back up. Many thanks to everyone who provided insights and solutions; they were all instrumental and helpful.

ORIGINAL QUESTION

I’m running Postfix as my MTA, and it has randomly decided to stop accepting incoming email:

Sep 14 07:50:26 zulu postfix/smtpd[27946]: warning: not enough free space in mail queue: 6832128 bytes < 1.5*message size limit

In my postconf:

message_size_limit = 10240000
queue_minfree = 25000000

All mail to my system is being rejected with 452 4.3.1 Insufficient system storage.

I know little to nothing about partitions and related magic, I only know that Postfix is seemingly telling me that it has no space, when in fact my system has plenty of space:

Finally, Postfix is still delivering the failure notices to my postmaster@ account, so I’m really quite at a loss.

EDIT

root@zulu:/# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda             679M  644M  6.9M  99% /
tmpfs                 249M     0  249M   0% /lib/init/rw
udev                   10M  112K  9.9M   2% /dev
tmpfs                 249M     0  249M   0% /dev/shm

As far as I know, the mail queue is stored in /var/spool/postfix. I’m not entirely sure how to determine if that is correct, but it seems correct:

root@zulu:/var/spool/postfix# ls -al
total 84
drwxr-xr-x 21 root    root     4096 Sep  6 13:58 .
drwxr-xr-x  4 root    root     4096 Sep  6 06:55 ..
drwx------  2 postfix root     4096 Sep 14 08:37 active
drwx------  2 postfix root     4096 Sep 13 13:19 bounce
drwx------  2 postfix root     4096 Sep  6 06:55 corrupt
drwx------  3 postfix root     4096 Sep 12 06:13 defer
drwx------  3 postfix root     4096 Sep 12 06:13 deferred
drwxr-xr-x  2 root    root     4096 May  4 17:30 dev
drwxr-xr-x  3 root    root     4096 Sep 14 07:37 etc
drwx------  2 postfix root     4096 Sep  6 06:55 flush
drwx------  2 postfix root     4096 Sep  6 06:55 hold
drwx------  2 postfix root     4096 Sep 14 08:37 incoming
drwxr-xr-x  2 root    root     4096 Sep 14 07:37 lib
drwx-wx--T  2 postfix postdrop 4096 Sep  8 21:41 maildrop
drwxr-xr-x  2 root    root     4096 Sep 12 06:13 pid
drwx------  2 postfix root     4096 Sep 14 07:37 private
drwx--s---  2 postfix postdrop 4096 Sep 14 07:37 public
drwx------  2 postfix root     4096 Sep  6 06:55 saved
drwx------  2 postfix root     4096 Sep  6 06:55 trace
drwxr-xr-x  3 root    root     4096 Sep  6 06:55 usr
drwxr-xr-x  3 root    root     4096 Sep  6 13:58 var

fdisk:

root@zulu:/# fdisk -l /dev/xvda

Disk /dev/xvda: 734 MB, 734003200 bytes
255 heads, 63 sectors/track, 89 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/xvda doesn't contain a valid partition table

I should probably note that this is a Linode VPS.

Glorfindel
  • 1,213
  • 3
  • 15
  • 22
Abraham Vegh
  • 1,045
  • 5
  • 17
  • 26

3 Answers3

8

The reason your showing lots of space is because your linode image is 956mb in size, and you have 20gb in total on your account. Hence, your actual server is only 956mb in size (giving you 6.9mb currently avilable). To solve- shutdown your linode instance, resize the image to use more of your account (suggest using it all, unless you have other images you need to run).

AliGibbs
  • 2,303
  • 20
  • 34
  • 2
    Just a FYI- linode support is awesome, tickets are usually replied to within 10minutes I find, and they know their system their advice is top-notch. – AliGibbs Sep 14 '11 at 13:44
  • I’ve only filed trivial tickets with them so far, and I’ve found just the same. :) – Abraham Vegh Sep 14 '11 at 14:46
4

Your root directory is full which is the issue. There are a few things you can do right away. One, run an apt-get clean all. This will delete any downloaded packages sitting on the file system. Next, do the following:

cd /var/log
du -hs

This will show how much space is being used for logs. If it is significant, go ahead and issue du -hs *. This will show how much space each directory in /var/log is using. Look for one that is using the most space. Enter it (cd where is the name of the problem directory. Issue a ls -lh and look at the file sizes for each. If the files end with .gz then you can simply delete them (rm ) or if there is a file that is large but not ending in .gz and you don't care about the contents issue:

> <filename>

Again, where is the name of the problem file. This should get you back to running and I am fairly confident you'll find your issue in /var/log.

Dustin
  • 199
  • 3
  • `apt-get clean all` dropped my usage from 99% to 89%, and now I’m investigating resizing the partition using Linode’s tools. – Abraham Vegh Sep 14 '11 at 13:27
  • Not a bad idea but still look at your log file usage. Log rotating is different on each distro and if Ubuntu is like Debian the it'll hold on to rotated logs longer than it needs to. – Dustin Sep 14 '11 at 13:32
  • Yep, `/var/log` is only 7.9M, so that’s not an issue right now, and I know I’ve been abusing the logs. I’m constantly looking at them though, so I’ll notice if they go through the roof. (Oh, and I’m on Debian 6, not Ubuntu.) – Abraham Vegh Sep 14 '11 at 13:35
3

Ok, I can see the problem, though I'm not sure where the rest of that reported disk space is. This line:

/dev/xvda             679M  644M  6.9M  99% /

says that your root filesystem is only 679MB in size, of which you only have 6.9MB free, (the amount reported in the error message).

/var/spool/postfix is in that filesystem (as is everything else). So if you're supposed to have 20GB available, then your disk hasn't been partitioned properly. If it's your own server, run fdisk -l /dev/xvda. That should show you the partition table. If it's a hosted service, talk to your provider.

SmallClanger
  • 8,947
  • 1
  • 31
  • 45
  • I updated the question with that output. Would it be advisable to create a new partition for the mail queue and/or the Maildirs, which are currently stored in `/home/vmail/`? If so, how would I go about doing that? – Abraham Vegh Sep 14 '11 at 13:01
  • 1
    I'm not familiar with Xen (or with Linode's implementation thereof), but my guess is that ther remainder of your allocated disk space is unpartitioned (possibly as `/dev/xvdb`, but I'm not sure about the naming convention.) Linoide should have documentation for how to access it. As for spearate paritions, it really depends on how you want to run your system. You could allocate, say, 2GB for `/var/spool/postfix`, which would ensure that mail overrun wouldn't fill your disk, but also means you 'll have pain if you need more than that at a later date. – SmallClanger Sep 14 '11 at 13:16
  • Would it make sense/even possible to resize `xvda` and simply make it larger? – Abraham Vegh Sep 14 '11 at 13:19
  • Okay, I’ve got two partitions, `xvda` and `xvdb`. `xvda` is the root, which is listed as being 700MB in size. `xvdb` is a swap partition, 256MB in size. So now I’m going to try using Linode’s tools to resize `xvda`. – Abraham Vegh Sep 14 '11 at 13:29
  • 1
    After reading Ali's answer, that seems to be the best option. I didn't know the 20GB limit was across your whole account. A flat resize of xvda will be your easiest route. – SmallClanger Sep 14 '11 at 13:34