10

Gmail is failing SPF check based on the client IP. These are the relevant headers:

Received-SPF: fail (google.com: domain of johndoe@example.com does not designate 164.77.240.58 as permitted sender) client-ip=164.77.240.58;
Received: from johndoe (unknown [164.77.240.58])
    by mail.example.com (Postfix) with ESMTP id 993643FE2D

The client IP (164.77.240.58) is the IP of johndoe's computer. The sender IP, the IP of mail.example.com, is included in the SPF record.

Why is Gmail failing based on the client IP instead of the sender IP? Is this how SPF is supposed to work?

Max Toro
  • 201
  • 2
  • 5
  • I would have expected their reference to "client IP" to simply refer to the client that connected to their SMTP server. (Which would be what you expected, afaict?) – Håkan Lindqvist Dec 30 '14 at 01:02
  • 3
    Can you include the `Received` header added by Gmail as well, to clarify where they actually received the mail from? – Håkan Lindqvist Dec 30 '14 at 01:08
  • @HåkanLindqvist The other *Recieved* headers only say `by ` – Max Toro Dec 30 '14 at 03:27
  • We have similar problems with import of mails through IMAP to gmail. Problem one (less similar): when we send email between two local mailboxes like a@example.com to b@example.com (but using SMTP port 25) then Gmail imports it and makes SPF check on client IP not server. http://serverfault.com/q/669584/ Problem two (more similar): when someone sends us email with ESMTP headers then Gmail imports it and makes SPF check with their server domain and our server IP. http://serverfault.com/q/670113/ – Zbyszek Feb 20 '15 at 10:53

3 Answers3

4

First, pull the spf record of example.com:

$ dig -t spf mail.example.com

Verify that example.com is on the senders list. Your spf record should look something like this:

"v=spf1 a:mail.example.com a:cname.example.com -all"

Take any domain names listed and do a DNS lookup on them to get the IP addresses:

$ dig mail.example.com

Then do a PTR look up to get the reverse DNS name for the IP:

$ dig -x XX.XX.XX.XX

The reverse IP lookup should match one of the records listed in the spf record. It would be helpful to start with the spf record though so we can see what's going on.

Pete
  • 137
  • 3
  • 3
    itym `dig example.com TXT`. Also, I don't believe the reverse lookup makes any difference to SPF as long as the SPF record does not make use of the `ptr` method. – Håkan Lindqvist Dec 30 '14 at 01:15
  • I'd like to point out that the special `dig -t spf` is now deprecated AS OF 2014. As @HåkanLindqvist pointed out you should now use `dig -t txt` or `dig example.com txt` . Some providers blacklist generic PTR records which is why i believe that was mentioned. – linuxgeek Jan 19 '22 at 17:53
2

Yes, google is improperly flagging messages as spam (SPF Softfail) and citing the sender's client-IP-address as not an authorized sender. This is wrong on several levels.

The messages in question, were not sent to a google hosted mailbox. Instead, the gmail user has configured gmail to fetch mail (POP3) from an account hosted on an other service and make those messages available in the gmail users inbox.

As such, these messages are NEVER actually received by a google mail-server. The messages are being fetched by Google, not received. Google should not be rewriting the mail headers as if they received them. If they wish to apply filters, they should only be reading the headers as written by the (non-google) mail server that is hosting the account.

SPF Softfail based on Client-IP: Google is reporting: "spf=softfail (googlecom: domain of transitioning usr@example.com does not designate x.x.x.x (client-ip) as permitted sender)"

Client-IP is the internet address of the senders physical location (home, business, coffee-shop). It would be impossible to include all possible client-ips in an spf record. SPF check should only consider the IP of the mail-server that delivered the message to the recipients mail server, in this case, a non-google server.

In this example, Debi is the Gmail user. Debi has configured Gmail to fetch messages from an other mailbox 'editor' hosted at supercp.com. Shawn sends a message to Editor which is received at supercp.com. Debi's gmail connects using POP3 to download the message. Google flags it as spam based on the IP address (residential ISP WAN address) where Shawn happened to be when he had Thunderbird mail client send the message.

Example email headers

Shawn Hughes
  • 21
  • 1
  • This is what appears to be happening for me. The SPF fail shows in gmail above the "X-Gmail-Fetch-Info:" header. The "SPF fail" that I see when clicking "Show Original" is complaining about an ip address that _Google_ added in a "Received" header while acting as the outgoing mail server. Google Should not be checking SPF when fetching via POP. – JasonWoof Jan 28 '22 at 00:57
  • "Be careful with this message Gmail could not verify.." --- I exactly agree with you. It's Google **IF** your SPF/DKIM is correct. This error started happening this week out of nowhere with POP3/SSL users who have 'Check mail from other accounts' under 'Accounts and Import' under Settings. No changes have been made to our DNS (It still validates fine from mail server IP and others.) Why Gmail started doing this is odd. I am going to have to add the IP most users come in from to our SPF at least to correct this situation. Fix it Google! – B. Shea Feb 24 '22 at 16:42
  • Typical minimal header recvd by GMAIL POP from 3rd party server: *Received-SPF: fail (google.com: domain of user@example.com does not designate 1.2.3.4 as permitted sender) client-ip=1.2.3.4; Received: by 2002:a2e:910f:0:b0:246:3eec:bb60 with POP3 id m15-20020a2e910f000000b002463fecbb60mf5238703ljg.7; X-Gmail-Fetch-Info: popuser@example.com 1 pop.example.com 995 popuser@example.com Received: from LAPTOP ([1.2.3.4]) ...* **("LAPTOP" client IP is NOT in the SPF of course)** – B. Shea Feb 24 '22 at 17:19
  • Also, ANY email coming through Gmail POP-Fetch that has relay information in header (assuming SPF is valid for this relay) will PASS. However if the given email message never left the local server, there is no relay info on it - ONLY the client sending IP. When Gmail-POP receives it, it does an SPF check on client IP only. How can you base SPF on a changing client IP? You cannot. = Gmail bug. SPF should be checked for SMTP (relayed) email (Not locally sent/non-relayed) and never for a 3rd party pop-fetcher (Gmail) - at least if no relay info is in header. – B. Shea Feb 24 '22 at 18:34
1

Yes, Google would be correct in identifying the SPF failure. The IP address that should be checked is the address that is connecting to Google's mail server. As there is no received header for Google, I suspect your mail server is checking SPF on the connection. It should only check SPF for unauthenticated connections from the Internet. Local connections and authenticated connections should bypass SPF validation.

SPF is intended to ensure the sending computer is permitted by the sending domain. Normally a domain would have 1 or 2 mail servers that handle all emails sent to or received from the Internet. These addresses should be the ones listed in the SPF record for the domain.

In this case, johndoe appears to be connecting to the domain's mail server. If the server is not on the domain's network, it is common to use an authenticated connection on the submission port (587). The mail server should then forward the message to Gmail and SPF should pass. If SPF still fails, the SPF record needs to be corrected to include IP of the mail server. There are several mechanisms that can be used.

My Email Policy ensures that all legitimate mail sent from my domain will pass SPF. There are some services that forward messages on behalf of my users which will fail SPF. However, the recorded failures I have received from servers validating DMARC have all been spammers.

BillThor
  • 27,354
  • 3
  • 35
  • 69
  • 3
    The mail server IP is included in the SPF record. That's the sender IP. But Gmail is rejecting based on the client IP, that's johndoe's computer. – Max Toro Dec 30 '14 at 03:22
  • @MaxToro According to the Received header johndoe's computer is sending directly to Google. It has no PTR record, so is unknown. It would likley work if johndoe's computer authenticated to Google. – BillThor Dec 30 '14 at 03:29
  • 1
    No, that's mail.example.com recieving from johndoe's computer. – Max Toro Dec 30 '14 at 03:31
  • @MaxToro Are you sure it isn't your server that rejecting the SPF record? The last active policy would be google's. Your mail log should tell if you are attempting to sent to google. – BillThor Dec 30 '14 at 03:40
  • See Shawn Hughes answer. This isn't the problem w/ ours. It's the POP3 fetch service Gmail uses to grab 'raw' emails from your server. The header ONLY shows 'received by' client IP (and **client machine IP is not in SPF record**). When in fact Gmail shouldn't look at the client as the 'received by IP' as it already knows where the POP3 email is coming from (your validated email server). Problem is w/ Gmail and "un-relayed" emails. All you can do is try and add client IP(s) to your SPF record. Or, have users avoid adding their accounts to 'Check mail from other accounts' under Gmail till fixed. – B. Shea Feb 24 '22 at 16:57
  • IMO - Gmail pop3 fetcher should be a "dumb downloader". Nothing less. Nothing more. The server it is grabbing emails from should have already filtered before you download via POP. If it hasn't, then that's on your local email administrator. Again: my opinion. – B. Shea Feb 24 '22 at 18:37