Zimbra 7.2 OSE Mail Queue Disappeared

Question

System Info:

Description: Ubuntu 10.04.4 LTS

Linux mail 2.6.32-42-server #95-Ubuntu SMP Wed Jul 25 16:10:49 UTC 2012 x86_64 GNU/Linux

H/W path          Device      Class      Description
====================================================
                              system     Bochs
/0                            bus        Motherboard
/0/0                          memory     96KiB BIOS
/0/401                        processor  QEMU Virtual CPU version 1.0.50
/0/402                        processor  CPU
/0/1000                       memory     2GiB System Memory
/0/1000/0                     memory     2GiB DIMM RAM
/0/100                        bridge     440FX - 82441FX PMC [Natoma]
/0/100/1                      bridge     82371SB PIIX3 ISA [Natoma/Triton II]
/0/100/1.1        scsi1       storage    82371SB PIIX3 IDE [Natoma/Triton II]
/0/100/1.1/0.0.0  /dev/cdrom  disk       DVD reader
/0/100/1.2                    bus        82371SB PIIX3 USB [Natoma/Triton II]
/0/100/1.3                    bridge     82371AB/EB/MB PIIX4 ACPI
/0/100/2                      display    GD 5446
/0/100/a                      storage    Virtio block device
/0/100/12                     network    Virtio network device
/1                eth0        network    Ethernet interface

I accidentally installed Postfix from APT onto our Zimbra email server virtual machine (KVM/Proxmox VE 2.1). I purged it instantly, however, it did not help. The following are the steps I took in a desperate attempt to fix it:

Reinstalled Zimbra
Stopped Zimbra service
Reset permissions

chown -R zimbra:zimbra /opt/zimbra
Fixed permissions

/opt/zimbra/libexec/zmfixperms -verbose -extended
Restarted Zimbra service
Renewed the Zimbra SSH keys

zmsshkeygen
Updated Zimbra SSH keys

zmupdateauthkeys

Following the steps above, Zimbra now works again, the problem is that all the emails that were in the mail queue from a whole day of broken Zimbra, are now GONE! And I don't just mean that you can't see them in the GUI, they're not even on the file system anymore.

There were at least 75 emails that were in the queue when I attempted to fix Zimbra, and this is the state of the spool now:

zimbra@mail:~$ postqueue -p
Mail queue is empty

Where the heck did all my emails go!?

You're sure Zimbra's postfix didn't just deliver them out after startup? 75 messages is not many. — 84104, Aug 19 '12 at 05:13
Also, while `tree` is cute and all, you'd be better severed with `su - zimbra -c "postqueue -p"` — 84104, Aug 19 '12 at 05:17
@84104 There were over 50 emails from my monitoring server that were directed to my email address, and they never showed up... — Soviero, Aug 19 '12 at 15:44
You'll have to process zimbra's mail log. Under RHEL, it's `/var/log/maillog`. — 84104, Aug 19 '12 at 15:51
@84104 I found them in the mail log, now how do I go about getting them to my inbox? — Soviero, Aug 19 '12 at 16:22
@84104 I even have a list of all the email messages in the data store that got "lost," here: http://pastebin.com/SR3z5acD How do I go about re-inserting them into Zimbra? — Soviero, Aug 19 '12 at 16:36

score 0 · Accepted Answer · answered Aug 19 '12 at 20:02

If anyone has this problem in the future, this is how you repair it. You may end up with more than a few duplicates, be that's better than losing an important email...

To find the messages that got queued up, run:

grep -iRE '<recipient_email_address>' /var/log/mail.log* | grep -i 'queued as' | awk '{print $6}' | cut -d ':' -f 1 | xargs | tr ' ' '|'

To find the actual email files on the file system, run:

grep -iRE '<output_from_previous_command>' /opt/zimbra/store | grep -iv 'Message-Id' | cut -d ':' -f 1 | xargs

"Inject" email back to the user account they came from:

su -l zimbra

/opt/zimbra/bin/zmlmtpinject <output_from_previous_command> -r <recipient_email>

Keep in mind that you will lose all date/time information. The only way around that is by adding an "X-Zimbra-Received" header to each message containing a copy of the "Date:" header. Use your imagination for how to do this, Perl comes to mind...

Zimbra 7.2 OSE Mail Queue Disappeared

1 Answers1