Dovecot: deliver reply into same folder as replied-to message

Question

UPDATE (2 sept 2014 13:01): Just to clarify, I'm interested in handling an incoming reply from someone else, so coming in via SMTP. If it is a reply to an email that is already in some mail folder, it should be stored in that folder too.

I have a Postfix (SMTP) + Dovecot (IMAP) setup, with my /home/sybren/Maildir directory storing my mail. I have a Sieve filter to ensure my work email ends up in the "work" IMAP folder. My IMAP folder structure is quite extensive, with several sub-folders, say work/research/subject1, work/research/subject2, work/education/subject3, etc.

As an example, let's say I send an email to my colleagues with the subject "AAAAAA". After sending it, I move it into the folder work/research/subject1. Now I would like their replies "Re: AAAAAA" to be stored in work/research/subject1 too, by Dovecot.

Email is handed over from Postfix to Dovecot using this setting in main.cf (all on one line)

mailbox_command = /usr/lib/dovecot/deliver -c 
                        /etc/dovecot/dovecot.conf -m "${EXTENSION}"

My main.sieve file now contains simple rules like:

require ["fileinto"];

if header :contains ["To", "Cc", "Bcc", "From"] "@work.nl" {
    fileinto "work";
}

UPDATE 2 (2 sept 2014 15:16): This is the "flow" that I would like to see for an incoming email:

1. Postfix receives the email via SMTP.
2. Postfix performs spam/virus scanning.
3. Postfix hands the mail over to Dovecot for delivery.
4. Dovecot inspects some database of message ID to mail folder mappings.
5. If the In-Reply-To message ID is found, deliver to the corresponding mail folder.
6. If not, use the Sieve script to determine the appropriate location.

Of course, the last two steps could theoretically be part of the same Sieve script.

I hope someone can help me out!

Cheers,

Sybren

Answer 1

There is special functionality IMAP-threads intended exactly for that purpose. https://www.rfc-editor.org/rfc/rfc5256 Modern IMAP servers including dovecot already have it. When mailbox is in maildir format a special index is built that count on message-id. When you reply on message, original ID is stored in the headers In-Reply-To: and References:. So IMAP-server can easily retrieve all the messages in the thread and display them sequentially despite of actual location.

Therefore you just have to ensure that your MUA is configured to use threading.

Some MUAs like RoundCube can turn threading on/off for each maildir folder separately. Some other simply turn it on/off globally.

Answer 2

I don't think you can server-side.

Typically storing a copy of sent messages is done by your e-mail client program. Some clients have an option "save replies with original message" or similar instead of storing all replies in a single "Sent Items" folder.

In most cases the default behaviour of the client will be to use the IMAP connection to place the your copy of the reply directly in the "Sent Items" or another IMAP folder you select for that purpose. That means that those replies don't pass through the Dovecot Local Delivery Agent nor any of filtering rules that emails normally pass through when they arrive over SMTP.

Sometimes your email client will have an option to always send a "Bcc:" to send a copy of the message to your own mailbox, instead of storing that copy over IMAP. In that case your response will come through the Dovecot LDA and the message filters you have set up, although that might require changing your rules because for instance instead of matching the sender you then would need to filter on the recipient.

Most of the more advanced clients simply don't care in which folder your sent items are actually stored and will simply try to generate a view with the complete conversation thread.

Answer 3

Your IMAP server (ie Dovecot) is not responsible for deciding where to store an email. Your mail client (which might be webmail, or desktop/mobile software) copies the email to a folder of its choice. Sieve is unlikely to be involved.

For mail from the MTA, the MTA typically passes the mail to sieve, which then puts it in the appropriate imap folder (probably via a postfix's delivery script though, not through the imap interface). When the mail comes in via IMAP, it's generally just stored, not passed off to Sieve.

Dovecot plugins can intercept IMAP events and react to them. In theory a dovecot plugin could track when you move messages to a different folder, and keep information about them that can be used during email delivery. I'm not aware of any such project.

I suspect the semantics of what should happen in such a system might get complex and ambiguous enough that it just caused lost email and user confusion. It would be nice to have a smart system suggest where I might want to file an email to such that it's easier to do, but I don't think I'd want it to just automatically happen.

The closest thing I can think that's actually implemented is that I have heard of a bayesian filter being trained to select a mailbox to file new mail in based on training examples from the user. Sounds cool, but again, I expect the confusion from even a small proportion of mis-placed email would not be worth it. I forget the name of the tool.

It is more plausible for MUA software to have a feature where your replies get filed in the same folder as the message you are replying to. E.g. this is an option in Thunderbird. The semantics are much clearer in this case.