Retrieving emails from a “catch-all”/wildcard email address

2

I migrated an entire email system for my own domain from GMail to another provider, still using my domain name. In the new system, not all email addresses from the original GMail system were recreated. I set up a “catch-all”/wildcard email (for emails sent to non-existent email addresses) and it is working well.

After about a week, I then recreated in the new system some more email addresses that were in the GMail system originally. These email addresses are missing the emails sent to them during the one week period and all these missing emails are sitting in the “catch-all”/wildcard mailbox to which I have web, IMAP and POP3 access.

How can I extract the missing emails from the “catch-all”/wildcard mailbox and copy or forward them to the original mailboxes they were intended for? I want an automated process using a program in which I feed the target email addresses and any other relevant parameters.

I do not have root access in the new system. I have the same kind of access as in the “Google For Your Domain” admin panel plus there is an API for performing various functions through external programs.

Here are headers from two test emails to non-existent email addresses from another email within the same domain.


EMAIL 1

Received: from mxback2j.mail.yandex.net ([127.0.0.1])
    by mxback2j.mail.yandex.net with LMTP id 5X48SyJk
    for <fakeemail1@mydomainname.com>; Mon, 10 Aug 2015 12:39:49 +0300
Received: from web19j.yandex.ru (web19j.yandex.ru [2a02:6b8:0:1619::319])
    by mxback2j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id nJ7ByEVW6U-dn5ewS0i;
    Mon, 10 Aug 2015 12:39:49 +0300
X-Yandex-Front: mxback2j.mail.yandex.net
X-Yandex-TimeMark: 1439199589
Authentication-Results: mxback2j.mail.yandex.net; dkim=pass header.i=@mydomainname.com
Received: from 127.0.0.1 (localhost [127.0.0.1])
    by web19j.yandex.ru (Yandex) with ESMTP id 51EE419404BF;
    Mon, 10 Aug 2015 12:39:49 +0300 (MSK)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mydomainname.com;
    s=mail; t=1439199589;
    bh=yGsBXgbrF2nr8eQFWxTXC6fTX8Rm0yym2ZrODvEqvT0=;
    h=From:To:Subject:Date;
    b=rV4mcTxUHD+DZZUfOPzlFQAtMJsXOPy5x2wL9p6IIjOCqJBIN62LIwrfGOwh70+MN
     vzZoiNNeQmHv9ifAGw50NQFYciO+QJyVRVL5THarz0sDaEIWndtDwc2lnFjQqqlKBS
     6JvPP2x9A6pXr/ek1JSQM0FtfEvKeSlM27mBAl8E=
X-Yandex-Spam: 1
X-Yandex-Front: web19j.yandex.ru
X-Yandex-TimeMark: 1439199589
Received: by web19j.yandex.ru with HTTP;
    Mon, 10 Aug 2015 12:39:49 +0300
From: Admin <admin@mydomainname.com>
To: fakeemail1@mydomainname.com
Subject: Test
MIME-Version: 1.0
Message-Id: <887321439199589@web19j.yandex.ru>
X-Mailer: Yamail [ http://yandex.ru ] 5.0
Date: Mon, 10 Aug 2015 11:39:49 +0200
Content-Transfer-Encoding: 7bit
Content-Type: text/html
Return-Path: admin@mydomainname.com
X-Yandex-Forward: 96c95557c105a7cf33717bc08474a8bc
Test.

EMAIL 2

Received: from mxback2j.mail.yandex.net ([127.0.0.1])
    by mxback2j.mail.yandex.net with LMTP id hZsI1hDZ;
    Mon, 10 Aug 2015 12:40:38 +0300
Received: from web19j.yandex.ru (web19j.yandex.ru [5.45.198.60])
    by mxback2j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id VCUXOJBY0S-ec5qR18F;
    Mon, 10 Aug 2015 12:40:38 +0300
X-Yandex-Front: mxback2j.mail.yandex.net
X-Yandex-TimeMark: 1439199638
Authentication-Results: mxback2j.mail.yandex.net; dkim=pass header.i=@mydomainname.com
Received: from 127.0.0.1 (localhost [127.0.0.1])
    by web19j.yandex.ru (Yandex) with ESMTP id 81F1F19404D7;
    Mon, 10 Aug 2015 12:40:37 +0300 (MSK)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mydomainname.com;
    s=mail; t=1439199637;
    bh=yGsBXgbrF2nr8eQFWxTXC6fTX8Rm0yym2ZrODvEqvT0=;
    h=From:To:Cc:Subject:Date;
    b=lNQ/5a+qhQt2Wwuv6mVzzuuQX6zd+g0BlXUj/sreLPoqwW2UI1jQuzurfaJuOaiAu
     RzoAS59RuaQzuvce2/SA6a8IeephZpbZO0wj0MoP8ZmPbGCD2OI5pLMr2PgjFHSDrZ
     nOmgUAUmmA71C5HMRHdtjCeVfgFOlMoTP2B/uPL8=
X-Yandex-Spam: 1
X-Yandex-Front: web19j.yandex.ru
X-Yandex-TimeMark: 1439199637
Received: by web19j.yandex.ru with HTTP;
    Mon, 10 Aug 2015 12:40:37 +0300
From: Admin <admin@mydomainname.com>
To: fakeemail2@mydomainname.com
Cc: fakeemail3@mydomainname.com,
    fakeemail4@mydomainname.com
Subject: Test
MIME-Version: 1.0
Message-Id: <891911439199637@web19j.yandex.ru>
X-Mailer: Yamail [ http://yandex.ru ] 5.0
Date: Mon, 10 Aug 2015 11:40:37 +0200
Content-Transfer-Encoding: 7bit
Content-Type: text/html
Return-Path: admin@mydomainname.com
X-Yandex-Forward: 96c95557c105a7cf33717bc08474a8bc
Test.

Lethalmiko

Posted 2015-08-08T17:32:57.633

Reputation: 21

Could you name new MTA/SMTP and IMAP/POP server software serving your domain? It may reduce guesswork required. Does this specific catch-all implementation stores ALL original recipient addresses? – AnFi – 2015-08-08T18:37:22.387

Yandex Mail (imap.yandex.com/pop.yandex.com). The Catch-all mailbox only stores "bounced" emails (ie emails going to email addresses that do not exist). – Lethalmiko – 2015-08-10T08:25:55.007

Could you send two test message to 1 and 2-3 non existing email addresses serve by catch-all and post headers of the received messages? Recipient detection based on To: and CC: headers (as suggested in Rick's answer) may be sometimes misleading. Usually "single recipient" messages contain valuable hints. – AnFi – 2015-08-10T09:12:21.373

Added mail headers to original question. – Lethalmiko – 2015-08-10T09:52:24.973

In your case: for X in the latest/topmost received: header is the best recipient indicator but it is available only for single recipient messages. – AnFi – 2015-08-10T10:19:29.547

Answers

1

When I've done this in the past here's the approach I took in my script:

  1. Make an IMAP connection to the server and log in.
  2. Get a list of the messages with an IMAP SEARCH (SINCE DD-MM-YYYY) (BEFORE DD-MM-YYYY) command.
  3. Read in the list of recipients who are missing messages
  4. For each msg get the To: and Cc: recipients and check against the list
  5. For each matching recipient forward a copy of the message as a MIME msg/rfc822 attachment.

If you like I can post a link to my forward_missing_messages script (it is written in Perl) for what it's worth.

-Rick

Rick Sanders

Posted 2015-08-08T17:32:57.633

Reputation: 146

Yes please post the link to your script Rick and thanks for the reply. Is it not easier for the script to run a search in the inbox for all emails sent to a specific email address after it makes an IMAP connection? Then it can simply forward these to the same specified email address. Unless I am misunderstanding. – Lethalmiko – 2015-08-10T08:35:45.823

You're right, Lethalmiko, that is simpler. I've modified the forward_missing_messages script to take that approach. You can download it from www.athensfbc.com/public/forward_missing_messages.tar. The tar file includes a README that describes how to pass the arguments to the script. – Rick Sanders – 2015-08-10T12:49:29.927

Thank you very much rick. Will try out your script and revert. – Lethalmiko – 2015-08-10T23:03:13.537