11

We have a linux (Debian) VPS with domain (let's say example.com with MX mail.example.com) that has SPF set up. There is dovecot+exim running. There is also Direct Admin on top of that.

When I send a mail to foreign server then everything is fine. There is server IP in the message and SPF check goes fine.

Some data changed (domain etc.):

Received: from mail.example.com (mail.example.com. [188.40.153.39])
    by mx.google.com with ESMTPS id ***.7.2015.02.18.04.09.46
    for <*@gmail.com>
    (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
    Wed, 18 Feb 2015 04:09:47 -0800 (PST)
Received-SPF: pass (google.com: domain of test@example.com designates 188.40.153.39 as permitted sender) client-ip=188.40.153.39;

But when I send it from a local mailbox to another local mailbox and then get mails into gmail through POP3 then I have problem with SPF because a message contains original client IP address and SPF check fails.

Authentication-Results: mx.google.com;
   spf=fail (google.com: domain of test@example.com does not designate 82.160.100.10 as permitted sender) smtp.mail=test@example.com
Received-SPF: fail (google.com: domain of test@example.com does not designate 82.160.100.10 as permitted sender) client-ip=82.160.100.10;

82.160.100.10 is IP of original sender.

Because of that problem our internal corespondence tends to be marked as spam in gmail for people who check their boxes through it.

Any ideas how to fix that?

EDIT: headers of test mail (changed our IPs and domain)

1) Email sent from one box to another - headers from Thunderbird client:

Return-path: <ldev@example.com>
Envelope-to: zbyszek@example.com
Delivery-date: Thu, 19 Feb 2015 11:41:20 +0100
Received: from nat10.net08-g2.isko.net.pl ([82.160.100.10] helo=[11.0.0.22])
    by mail.example.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128)
    (Exim 4.83)
    (envelope-from <ldev@example.com>)
    id 1YOOPC-0005Ud-Qq
    for zbyszek@example.com; Thu, 19 Feb 2015 11:41:20 +0100
Message-ID: <54E5BDCE.5040207@example.com>
Date: Thu, 19 Feb 2015 11:41:18 +0100
From: Head Developer <ldev@example.com>
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0
MIME-Version: 1.0
To: Zbyszek <zbyszek@example.com>
Subject: This is test
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

2) Same mail after being received by gmail (by automatic regular import by POP3):

Delivered-To: *@gmail.com
Received: by 10.140.86.210 with SMTP id p76csp775880qgd;
    Thu, 19 Feb 2015 02:47:12 -0800 (PST)
X-Received: by 10.140.102.165 with SMTP id w34mr10762910qge.26.1424342832562;
    Thu, 19 Feb 2015 02:47:12 -0800 (PST)
Authentication-Results: mx.google.com;
   spf=fail (google.com: domain of ldev@example.com does not designate 82.160.100.10 as permitted sender) smtp.mail=ldev@example.com
Received-SPF: fail (google.com: domain of ldev@example.com does not designate 82.160.100.10 as permitted sender) client-ip=82.160.100.10;
Received: by 10.224.31.8 with POP3 id w8mf619596qac.5;
    Thu, 19 Feb 2015 02:47:12 -0800 (PST)
X-Gmail-Fetch-Info: zbyszek@example.com 2 mail.example.com 110 zbyszek@example.com
Return-path: <ldev@example.com>
Envelope-to: zbyszek@example.com
Delivery-date: Thu, 19 Feb 2015 11:41:20 +0100
Received: from nat10.net08-g2.isko.net.pl ([82.160.100.10] helo=[11.0.0.22])
    by mail.example.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128)
    (Exim 4.83)
    (envelope-from <ldev@example.com>)
    id 1YOOXn-0005j5-Tm
    for zbyszek@example.com; Thu, 19 Feb 2015 11:41:20 +0100
Message-ID: <54E5BDCE.5040207@example.com>
Date: Thu, 19 Feb 2015 11:41:18 +0100
From: Head Developer <ldev@example.com>
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0
MIME-Version: 1.0
To: Zbyszek <zbyszek@example.com>
Subject: This is test
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

Edit: some extra info

  • method of delivery is SMTP (I send email from Thunderbird in home using mailbox ldev@example.com to another mailbox on same server zbyszek@example.com)
  • 82.160.100.10 is my home IP, nat10.net08-g2.isko.net.pl is my home hostname which resolves to that IP
  • 10.140.102.165 is gmail server IP
  • 11.0.0.22 is local IP in my home network
  • Mail server IP wasn't in any of headers (If it would be there it would be 188.40.153.39).
  • Hostname mail.example.com points to mail server IP
  • Domain IP wasn't in any of headers (if it would be there I would change it to 85.17.23.59)
  • domain has proper MX entry (that points to subdomain mail.example.com)
  • SPF record: "v=spf1 a mx ip4:188.40.153.39 -all"

Edit: uncovered IPs as not so sensitive

masegaloeh
  • 17,978
  • 9
  • 56
  • 104
Zbyszek
  • 175
  • 1
  • 10
  • When you say *local mailbox* do you mean a mail address on the same machine using a local delivery method using only the local part of the address without the domain? Thus the domain is delivered locally instead over SMTP? If yes, that's expected, if no, please elaborate on the problem and your setup. – sebix Feb 18 '15 at 14:38
  • Yes, target mailbox is on the same machine. I wonder if there is a method either to configure it so message would contain IP of server instead of client's IP or turn off SPF check for such messages (which is probably on gmail side when importing mail, but really I'm not 100% sure, don't know how to check it). – Zbyszek Feb 19 '15 at 09:47
  • I've updated question with headers – Zbyszek Feb 19 '15 at 10:53
  • Well, you changed the IPs and hostnames. It's nearly impossible to understand the headers then. – sebix Feb 19 '15 at 12:48
  • I would like not to disclose our servers nor my home IP here. Added some more info to hopefully make it more understandable. – Zbyszek Feb 19 '15 at 13:14
  • You have a public service running, hiding IPs and hostnames is totally useless. Your domain ends with *arts.com* and is located in Poland. Found that out by using search engines and the informations you did not remove :) – sebix Feb 19 '15 at 13:32
  • Yeah, but we have enough bots that parse our emails / hosts from different places and try to brute-force or spam us etc. I wonder how did you get the domain though. ;) – Zbyszek Feb 19 '15 at 13:55
  • 2
    http://meta.serverfault.com/q/963/126632 – Michael Hampton Feb 19 '15 at 14:41
  • I've uncovered IPs as those are not so sensitive, we see attack attempts every day anyway. Left emails and domain obfuscated because of email harvesting bots. – Zbyszek Feb 19 '15 at 22:05
  • 1
    Just for correction, [GMail doesn't support fetch email via IMAP](http://webapps.stackexchange.com/q/66094). [It fetch your email via POP3](https://support.google.com/mail/answer/21289?hl=en). – masegaloeh Feb 20 '15 at 15:25

3 Answers3

15

Disclaimer: This answer was speculation one until GMail person confirmed it.

Looks like it's GMail mishandle your fetched email here. Some peoples also report similar case with yours in here, here or here

The problem is: GMail also deploying SPF measure when scanning email after fetching it via POP3.

Normally SPF-checking take place in SMTP transaction by checking the domain parts of sender address and client IP address. But in POP3, GMail has to parsing header and find the last Received header.

Received: from nat10.net08-g2.* ([*.160.100.10] helo=[11.0.0.22])
    by mail.example.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128)
    (Exim 4.83)
    (envelope-from <ldev@example.com>)
    id 1YOOXn-0005j5-Tm
    for zbyszek@example.com; Thu, 19 Feb 2015 11:41:20 +0100

This is original email that fetched from your server. It states that your email accepts email from *.160.100.10 with sender ldev@example.com. In this stage Gmail pretends as your server and checks domain parts of sender address (example.com) and client IP address (*.160.100.10). The result was expected:

SPF softfail because domain of ldev@example.com does not designate *.160.100.10 as permitted sender

For workaround, you can set Gmail filters to never mark your email as Spam.

Community
  • 1
masegaloeh
  • 17,978
  • 9
  • 56
  • 104
  • Thank you for info. I wonder if there is any other better way than this workaround. Will accept this answer on monday if no better resolution to the problem will be found. – Zbyszek Feb 19 '15 at 14:07
  • How do the client (*.160.100.10) send email to exim? By submission port 587? Or by smtp port 25? – masegaloeh Feb 19 '15 at 14:09
  • On smtp port 25 – Zbyszek Feb 19 '15 at 14:13
  • Yes, with STARTTLS – Zbyszek Feb 19 '15 at 14:17
  • 2
    At first, I think that you can try bypass gmail SPF checker by add something in the header indicated that the client was using **authenticated SMTP session**. But looks like you already have ESMTPA header in your email and Gmail still ignore it :(. Proper spam checker will consider to this kind of header when calculating spam score. For example: [SpamAssasin will reduce the score of authenticated email](http://svn.apache.org/repos/asf/spamassassin/branches/3.2/lib/Mail/SpamAssassin/Message/Metadata/Received.pm) when there is ESMTPA header in Received header. – masegaloeh Feb 19 '15 at 14:41
  • 1
    7 years later - Gmail still does it (just happened to us, another server) – Zbyszek May 24 '22 at 19:41
2

It seems to be a bug that Gmail does not respect the ESMTPA in the Received header to show the MUA is a trusted host. Some possible workarounds come to mind:

  1. use split MX, that is, one Exim instance to receive and forward from authenticated clients, and a second to receive into mailboxes. That simulates the ISP-to-ISP mail that Gmail may be expecting, and there will be an IP address in the headers that matches the SPF record.
  2. add a Received header at the top that simulates the above transfer, by adding add_header = Received: by 10.224.31.8 with ESMTP ....
  3. for the users checking their mail in Gmail via POP, forward a copy to their Gmail account instead.
masegaloeh
  • 17,978
  • 9
  • 56
  • 104
Cedric Knight
  • 1,098
  • 6
  • 20
  • (2) looks interesting, could you elaborate why 10.224.31.8 though?. (3) is not an option for us as there is another problem with having a plenty of forwarders from one server with one IP - Gmail tends to think that those are "unsolicited bulk mails" and starts to put a rate limits on us (hosting active newsletter of one of our clients probably isn't helping) – Zbyszek Feb 22 '15 at 17:56
  • Sorry, I saw 10.224.31.8 and assumed it was a private address of your mailserver, but it looks like Gmail. If you have a separate public address you could use, maybe fake that. The line would have to comply with RFC 5322 and be sufficiently realistic to be interpreted as a relay by eg SpamAssassin. – Cedric Knight Feb 22 '15 at 18:16
  • Ok, thanks, will need to try it out soon. Do you know a way how to config that only for problematic mails eg. from local boxes to other local boxes? Or do I need to do that for all incoming mail? – Zbyszek Feb 22 '15 at 20:57
  • @Zbyszek Looks like you need to do it in every email which its sender domain has published proper SPF record. Just be safe, you should added it in every email – masegaloeh Feb 27 '15 at 04:00
  • 1
    anyway instead using `received_header_text` like this [proposed solution](http://serverfault.com/questions/671883/server-ip-variable-in-received-header-text-in-exim-config?lq=1), why not using [add_header feature from exim](http://www.exim.org/exim-html-current/doc/html/spec_html/ch-access_control_lists.html#SECTaddheadacl)? – masegaloeh Feb 27 '15 at 15:25
0

I had the same issue and I fixed it using he following Google doc. This might help you as well and worth trying.

https://support.google.com/a/answer/33786?hl=en

You will have to include the following record added to your existing spf TXT record. This helps Google to identify the source of the email address.

include:_spf.google.com

So you have to update your spf record as follows.

SPF record: "v=spf1 a mx ip4:188.40.153.39 include:_spf.google.com -all"

Then try sending the email and it should land directly to your inbox if your IP has good reputation.

Digin Dominic
  • 53
  • 7