2

I've noticed that I'm not getting certain emails in my Gmail and Yandex.Mail that are forwarded via UNIX systems (without SRS — not too sure if Sender Rewriting Scheme is still the best practice? Because with DMARC I think it'll also have to apply to the actual From: header within the email itself.) from DMARC-enabled senders.

I can't quite figure out what's going on — emails that always go through include PayPal.com, whereas Microsoft.com and some others get rejected (only getting delivered locally to systems that don't implement DMARC on the receiving side).

% echo _dmarc.{microsoft.com,paypal.com} | xargs -n1 dig -t txt | fgrep v=D
_dmarc.microsoft.com.   3302    IN      TXT     "v=DMARC1\; p=reject\; pct=100\; rua=mailto:d@rua.agari.com\; ruf=mailto:d@ruf.agari.com\; fo=1"
_dmarc.paypal.com.      3311    IN      TXT     "v=DMARC1\; p=reject\; rua=mailto:d@rua.agari.com\; ruf=mailto:d@ruf.agari.com"
%

When both domains have the same reject policy — and Google even specifically mentions that PayPal does have a definitive reject policy — I'm not exactly sure if there's something wrong in my own setup, or if it's the sending party that's to blame. Any ideas?

Is it just because of SPF's fail vs. softfail, or is there more to it?

% echo {microsoft.com,paypal.com} | xargs -n1 dig -t txt | fgrep v= | sed 's#[^[:space:]]*:[^[:space:]]*#:#g'
microsoft.com.          3332    IN      TXT     "v=spf1 : : : : : : : : : : -all"
paypal.com.             300     IN      TXT     "v=spf1 : : : : : : ~all"
%

Here's what Gmail reports for PayPal emails that do get delivered through forwarding:

ARC-Authentication-Results: i=1; mx.google.com;
       dkim=pass header.i=@mail.paypal.com header.s=pp-epsilon1 header.b=K96c6GUZ;
       spf=fail (google.com: domain of bounce@mail.paypal.com does not designate 2001:470:7240:: as permitted sender) smtp.mailfrom=bounce@mail.paypal.com;
       dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=paypal.com
Return-Path: <bounce@mail.paypal.com>
cnst
  • 12,948
  • 7
  • 51
  • 75
  • Not doing this sort of forwarding at all is best practice, but if you must, then you have to rewrite the sender address, or this happens. – Michael Hampton Sep 15 '19 at 20:32
  • @MichaelHampton but it never happens with PayPal.com. What am I supposed to use to rewrite the addresses? I've never seen anyone do this, TBH, and https://en.wikipedia.org/wiki/Comparison_of_mail_servers doesn't even have an SRS column amongst a couple of dozen of features that are being compared. – cnst Sep 15 '19 at 21:34
  • @MichaelHampton per https://serverfault.com/a/678335/110020, DMARC is supposed to pass if DKIM is still valid. – cnst Sep 15 '19 at 22:30
  • You have only given an example of a mail getting through, while the answer is in the messages that doesn't make it. At least mail server logs should have some information on why the message was rejected, unless Gmail silently ignores it. – Esa Jokinen Sep 16 '19 at 04:07
  • @EsaJokinen I cannot identical results because, obviously, the other message doesn't make it. The rejection email is the standard DMARC rejection from Gmail without any details about what exactly is going on other than the standard links that they provide. – cnst Sep 16 '19 at 06:49
  • I wrote an answer for you, but I'm afraid it's bad news. – Esa Jokinen Sep 16 '19 at 08:34

3 Answers3

1

The DMARC policy is protecting the domain used in the From: header: to pass the DMARC check this must align either with DKIM or SPF. For SPF, this requires a matching envelope sender (i.e. Return-Path i.e. address used in SMTP MAIL FROM command) with a passed SPF test.

With DMARC, forwarding the mail (without the final server being aware of it) will cause problems either way:

  • If you don't change the envelope sender, you won't pass the SPF check, and
  • if you change it, it won't align with the From: address anymore.

The example message from PayPal manages to pass the DMARC test, because it has a valid DKIM signature that is also in align with the From header. Because the error for the other domains was a standard DMARC rejection, we can assume the messages are either missing a valid DKIM signature or it's not aligned with the From header.

The only way around would be trusting that the forwarding server has already checked for SPF, DKIM & DMARC and blindly accepting every message coming from that server. That's how it's done in a standard primary/secondary MX configuration for the messages coming in through the secondary server – and how it should be done in any forwarding scenario accepted on both sides.

Unfortunately, based on Gmail Help Community's discussion on "Can I please turn off DMARC", Gmail doesn't allow adding exceptions for DMARC tests. Conclusion: do not forward to Gmail.

Esa Jokinen
  • 43,252
  • 2
  • 75
  • 122
  • I think Microsoft emails do have DKIM, but it's somehow invalid. I'm trying to find out why is it invalid. – cnst Sep 16 '19 at 22:54
1

It seems to me that the -all "hard fail" for Microsoft SPF could be the cause. If the receiving system has a mechanism for enforcing BOTH SPF and DMARC, then the SPF "hard fail" would cause this message to fail, even though DMARC would pass it due to the DKIM pass. Remember, SPF is it's own thing and was around before DMARC. One of the problems with using it, though, was that there was no standard for receiving systems in how to interpret the "soft fail" and "hard fail." So, receivers were on their own to decide what, if anything, to do with both. Microsoft may still have SPF working for inbound mails and may drop messages that evaluate to a "hard fail." Paypal uses a "Soft fail" which may be interpreted as not really important enough to block with the SPF mechanism. Both may be evaluated by DMARC, but the SPF check may kill the messages before DMARC has a chance to pass them though.

JoeMullarkey
  • 11
  • 1
0

There are 2 policies actually at play here. One is the hard fail policy for the Microsoft SPF and since you're not changing the envelope.from address, Gmail might just drop it based on SPF.

To tackle this, it would be wise to rewrite the envelope.from a.k.a return-path. However, this breaks the alignment that is required for DMARC to pass based on SPF. So, that leaves you DKIM to get you that PASS.

In forwarding scenarios, this situation is sometimes referred to as "DKIM Survival", since DMARC compliance is based on how well the DKIM signature survives the forwarding. Surviving forwarders depends largely on which header fields the forwarder changes, or if it strips the original signature (and optionally replaces it with its own).

In your case, the paypal message DKIM signature survives the forwarding. Elsewhere in the headers for that email you can find information on the DKIM signing itself and specifically the fields that have been signed. Depending on which type of email you receive from Microsoft (Marketing, Survey etc.), these are likely the header fields that are signed: h=From:To:Subject:Date:MIME-Version:Reply-To:List-ID:Message-ID:Content-Type; It is important to understand which fields your system touches while forwarding messages. Here's a reference to the RFC for DKIM that recommends which headers to actually sign: https://www.rfc-editor.org/rfc/rfc4871#section-5.5

And then there is ARC (for Authenticated Received Chain). It is still in Draft stage, but typically large mailbox hosters such as Google implement these new guidance rather quickly. DMARC Analyzer has a good description of what it does: https://www.dmarcanalyzer.com/arc-is-here/

I hope this helps you.

Community
  • 1
Reinto
  • 649
  • 4
  • 9
  • Per https://serverfault.com/a/678335/110020, either one has to pass. It seems that Microsoft's signature may be invalid, but not having direct access to Microsoft.com, it's a little hard to debug. – cnst Sep 18 '19 at 02:56
  • Clearly you have knowledge on the subject. So I'll just focus on your question: Yes, it seems like there is something you can do in your setup to mitigate the issue. It could be the fact that Microsoft implements a FAIL in SPF: SRS would fix this. On the other hand, you tell us that only systems that don't implement DMARC receive these emails properly. That would hint at DKIM survival. And because PayPal.com does not fail, possibly the forwarding changes a signed header, breaking the MS signature, as described above. – Reinto Sep 24 '19 at 09:53
  • I've since looked more into it, and noticed that one of the forwarders itself checks DKIM, and reports it as invalid, which would lead me to believe that it should be done before any sort of modification of the headers, and that it's indeed that Microsoft.com that's broken (it'll still work without forwarding because SPF would pass, resulting in a DMARC pass, too). – cnst Sep 24 '19 at 18:25