What is the most portable and cross-platform format to store my email backups?

I currently use gmail, but this question is valid for any client: What is the best way to backup and store my email so that in the future I will be able to access it on a variety of platforms and clients.

Update:

While using gmail is certainly cross-platform and they have their own backup system, I am still looking to backup my email myself so that I can definitely know I have control over the backup and storage. I'm fine with letting gmail do it for now, but there's no guarantees.

mindless.panda

Posted 2010-03-23T15:34:15.960

Reputation: 6 642

When backing up Gmail without any Gmail-aware software, one might get duplicate messages due to Gmail's usage of labels. (Like a single Gmail message with multiple labels, appears to be multiple messages in different folders when using IMAP.) – Arjan – 2010-03-23T18:12:34.743

As for backups: one might want to restore a single message one deleted by accident. (Especially when only noting the deletion after new messages have arrived.) Hence it would be nice if one can search for messages, and restore selected messages? – Arjan – 2010-03-23T19:40:21.290

Answers

my solution has been getmail using imap and backing up to maildir. i then backup the maildir with crashplan to a remote drive.

yanokwa

Posted 2010-03-23T15:34:15.960

Reputation: 2 156

It seems quite clear to me how you store email, but not so much how one can retreive it afterwards. Say I have a a cron-job running on a server to fetch all company email from our email hoster. The emails could then be deleted from the origin server and still stay in the backup server. But how would a client (say on a mac with normal end user software like Mail) connect to this and search for some specific email? – oligofren – 2014-10-29T14:56:07.627

Have you setup an IMAP server? – oligofren – 2014-10-29T15:15:47.830

The two most common open formats for storing email are mbox and Maildir.

Both essentially just put the whole email (which is a stream of [ASCII] bytes) into a file. The only real difference is that mbox puts all emails into one file (concatenated, with a separator line), while Maildir puts all emails into one directory (hence the name :-)), but with one file per email.

Both are well supported by many programs, so both will do.

Both have some advantages and disadvantages in day-to-day handling; see the Wikipedia article for details. For archiving this should not be relevant, so either will do. Mbox is not really standardized (hence possibly compatibility problems), so I'd lean towards Maildir, but both are probably fine.

sleske

Posted 2010-03-23T15:34:15.960

Reputation: 19 887

I too use GMail and my answer is to have my own local IMAP server and to use fetchmail to retrieve the email and store a local copy. By using an IMAP server I can then use any mail client to access it, and by using an IMAP server that stores the emails as separate files (Cyrus IMAP) it means that even if the software fails I can still access the emails "by hand" if required.

For the really paranoid, using fetchmail with IMAP's idle command means that emails are downloaded when they arrive at GMail, keeping the window that you don't have a backup to a minimum.

Cry Havok

Posted 2010-03-23T15:34:15.960

Reputation: 3 486

When using IMAP to synchronise without backing up, manually deleted messages will be gone from both locations as soon as IMAP synchronisation takes place (like: immediately)... When using POP to feed the local IMAP server, the local IMAP server will have no notion about Gmail's labels and Deleted Items whatsoever, making restoring a backup a nightmare as the backup will then basically hold all messages one ever received? – Arjan – 2010-03-23T18:16:10.100

@Arjan - my goal here is to have a solid, working, accessible backup of my email, not my email as gmail stores (i.e. labels, etc.). While that would be nice, you've already pointed out all the problems. Seems like the best bet would be to simply backup the All Mail folder/label. Am I correct in assuming this would have everyone email only once? – mindless.panda – 2010-03-24T21:42:09.407

However by using fetchmail it doesn't matter if the messages are deleted from GMail, there is no synchronisation of deleted messages, only new ones. – Cry Havok – 2010-03-24T23:20:04.603

@Cry, but even if only all new messages are fetched, then the backup won't know what messages have been deleted at a later time. That's a lot of messages when one restores the backup? – Arjan – 2010-03-25T06:16:28.683

@Arjan - make your mind up ;) Either you want to be able to restore deleted messages, or you don't. – Cry Havok – 2010-03-25T07:13:02.030

True, but I would also want to be able to restore the last state of all messages -- which does not included messages I deleted a long time ago. Indeed, fetching all using POP or whatever allows one to restore message that have been accidentally deleted. But there's more kind of disasters... (I got my disasters covered, but not in a generic way that the OP seeks.) – Arjan – 2010-03-25T07:33:10.427

The standard is called mbox format and it's as old as the internet. Many third party email clients will import this format directly (many also use it internally.) There are also utilities for converting this to various other formats.

Chris Nava

Posted 2010-03-23T15:34:15.960

Reputation: 7 009

I'm leaning towards using Thunderbird to download all email and then create yearly backups - so you would suggest using mbox files for those yearly backups. – mindless.panda – 2010-03-23T16:46:53.963

As all messages will be stored in a single file, this might not be the best choice for a backup. However, there's many tools available to fix corrupted mbox-files, so maybe it doesn't really matter. – Arjan – 2010-03-23T18:10:02.330