7

Emails sent from all 3 email addresses I have set up in the Rackspace Cloudways Add-On are ending up in Spam in GMail.

When I "View Original Message" in GMail, I see...

SPF:    NEUTRAL with IP 173.203.187.81
DMARC:  'FAIL'

... where 173.203.187.81 is an IP address Rackspace.

My DNS provider is CloudFlare.

There is a DMARC policy set up, which is the following...

_dmarc.boldstatements.com.au    v=DMARC1; p=none; ruf=mailto:admin@mydomain.net; rua=mailto:admin@mydomain.net

Cloudways provided me with this DKIM TXT record...

20220817-maluhsjy._domainkey    v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC0PqtvPuYkElqS+b80iEj4aepAdf6n+CDXRFTG/1Q8RMdw/D6hNmQpv8FCTyIuplZt/qTxBbBFrPLJK5tp7bqkSEG2YpPSnHDCGihaOCsRkJP0aAbnuQRmjHq6H0yCwtJKjRhW7H4pbjx9/LA6dXIaw4N1emtSLWcGejVrhVZ+CwIDAQAB

When I use dnschecker.org and a few other DNS lookup tools, to look at my TXT records I get the following strange output...

\239\187\191v=spf1 a mx include:_spf.elasticemail.com include:emailsrvr.com -all

These characters \239\187\191 are definitely not in the TXT record in CloudFlare.

CloudWays' support claims that these characters are causing the DMARC to fail, but since they don't appear on other DNS checkers such as https://mxtoolbox.com/ and https://www.whatsmydns.net/, and since SPF is returning "neutral", I suspect they are actually a bug in https://dnschecker.org/ and the DNS checker that Cloudways support are using.

Any thoughts?

NOTES

Thanks to Patrick Mevzek's answer below I was able to find a solution.

Just a quick description of how I ended up in this mess in the first place: Basically I copy pasted the values of the DNS records from a Cloudways support chat window straight into Cloudflare.

And to remove the characters I needed to copy the value from Cloudflare into Notepad++, change the encoding to ANSII, which made the extraneous characters appear, delete them, then change back to UTF-8 (just in case), and paste back into CloudFlare.

clayRay
  • 181
  • 8
  • 4
    Since you don't disclose the exact name of the domain, I can't check myself. *Ending up in Spam in Gmail* — Google might mark mail as a spam for a couple of reasons. Also, what is this XX.XX.XX.XX then? Do you know it, can you explain why it is appeared, is it permitted by some item in the (correct) SPF record? Also, could you infer that does Google see this DNS problem too? – Nikita Kipriyanov Aug 25 '22 at 08:12
  • @NikitaKipriyanov - I have added the IP address as well as one of the domains causing the GMail rejection, as well as the DKIM record provided by Cloudflare – clayRay Aug 25 '22 at 09:38
  • The DKIM record can't be the full DNS name, since the gramma for DKIM for the domain `example.com` is `SELECTOR._domainkey.example.com`. In your case the selector is `20220817-maluhsjy`. – Lasse Michael Mølgaard Aug 25 '22 at 10:19
  • @LasseMichaelMølgaard sure, and that means if the zone itself called `example.com`, the *relative DNS record name* for the DKIM key with that selector would be `20220817-maluhsjy._domainkey`, exactly what we see. That is correct. – Nikita Kipriyanov Aug 25 '22 at 10:27
  • OK, this IP seems to be mentioned in the `emailsrvr.com` SPF record. DMARC fail is strange, especially when the policy itself is set to `none`. It *could* be because Google failed to read your SPF record, e.g. they receive corrupted DNS too. Or it could be something else. Is it possible to (at least, temporarily) delegate this domain to some other, knowingly working DNS servers, and see if the problem persists? – Nikita Kipriyanov Aug 25 '22 at 10:39
  • Give the real name involved if you want real help or ask your DNS provider or use online troubleshooting tool DNSviz. – Patrick Mevzek Aug 25 '22 at 13:46
  • "These characters \239\187\191 are definitely not in the TXT record in CloudFlare." Famous last words. Please next time do not trust what you think you inserted. Verify by actually looking up the TXT record via DNS. You'd have immediately discovered that they where actually there and so you should have tried to edit the value inserted. – GACy20 Aug 26 '22 at 12:32
  • @GACy20 thanks for the advice, but unfortunately the characters were only appearing in dnschecker.org, not in whatsmydns.net or mxtoolbox.com, so I thought it was some sort of bug in dnschecker.org. – clayRay Aug 26 '22 at 23:45

3 Answers3

29

FWIW,

\239\187\191 is DNS encoding of 3 bytes of decimal values 239, 187, 191 which maps in hexadecimal to EF BB BF, which is the UTF-8 encoding of Unicode codepoint U+FEFF, which is ZERO WIDTH NO-BREAK SPACE

I suspect this TXT record was created by copy and pasting from somewhere and some "smart" behavior and obviously the space is not visible on screen in some UI but did land up in the TXT record.

This has to be cleaned, aka removed.

As for:

These characters \239\187\191 are definitely not in the TXT record in CloudFlare.

and

but since they don't appear on other DNS checkers such as https://mxtoolbox.com/ and https://www.whatsmydns.net/,

I suspect that various systems (mistakenly but there is unfortunately no standard there at all, DNS was invented far before Unicode/UTF-8 and then lots of things like SPF just decided to abuse TXT records) just consider the TXT record content to be a string in UTF-8 so they decode it and display it, but obviously the "zero-width space" is not visible on any HTML page.

A better UI would take care of that and display that properly and/or warn about it. An even better UI would even more so just remove that character when the record is added, since it is obviously wrong here (but the obvious in TXT record is limited… you have to see it is v=spf1 or similar and then act accordingly). Which now gives me a good idea on what I should fix in my own UI, thanks for the idea :-)

Patrick Mevzek
  • 9,273
  • 7
  • 29
  • 42
  • Whoa! Nice catch! – Nikita Kipriyanov Aug 25 '22 at 15:48
  • 6
    @user253751 "Zero Width No-Break Space" **is** the "Byte-Order Mark" - it's a character that has no effect on the text around it, so was chosen as a safe indicator. U+FFFE is the reserved Non-Character that result from reading it the wrong way around in *UTF-16*. – IMSoP Aug 25 '22 at 21:23
  • @PatrickMevzek Thanks! Wow! I originally thought it might be some hidden characters so I copied the text into NotePad++ and clicked Show All Characters, and didn't see anything. Then I thought copying it back into a fresh Cloudflare DNS entry from Notepad++ would clean it, but that didn't work either. After your answer I found [this other SO answer](https://stackoverflow.com/a/35153154/1309582) that showed I needed to change encoding to ANSII so I could remove the unwanted chars and then paste back. Phew! All from pasting directly to Cloudflare from a Cloudways chat window. – clayRay Aug 26 '22 at 03:50
  • Not exactly for this need but should also be useful for this need, I heavily rely on https://r12a.github.io/uniview/ for all Unicode ; It may look like to be a complicated UI but because it does a lot of things. In your case, you could try to copy and paste the string you had in the right big box and then click the down pointing arrow. You should then see on the left side the list of characters found in your string, as unicode codepoints. U+FEFF should appear there as first one. You can then edit the string on the right box and click arrow again and see how the list of codepoints changes. – Patrick Mevzek Aug 26 '22 at 05:05
  • 2
    @clayRay + complain to your DNS provider that its UI did not detect this and did not help you view the problem, and hence you ended up having published a content different from the one you thought you were publishing. Maybe they would be interested to fix this case. You are most probably certainly not the only one impacted by situations like this, even if low volumes. – Patrick Mevzek Aug 26 '22 at 05:07
2

SPF ‘neutral’ is a result that is definitely not possible with the SPF record you gave.

There is no qualifier in that record that would yield a ‘neutral’ result, the final mechanism is -all, which would yield ‘fail’.

Also, according to the RFC 7208, SPF records are found by selecting only TXT records that begin with v=spf1 (case-insensitive), so if those additional bytes are really part of the supposed record, it would never be recognised as an SPF record.

glts
  • 681
  • 4
  • 14
2

These characters \239\187\191 are definitely not in the TXT record in CloudFlare.

Yes, they are:

$ dig -t txt +short $yourdomain.com.au 
"\239\187\191v=spf1 a mx include:emailsrvr.com include:_spf.elasticemail.com ~all"
Ángel
  • 627
  • 3
  • 5
  • 1
    The user probably meant: "are not in the **UI** at CloudFlare" (when editing content) – Patrick Mevzek Aug 25 '22 at 22:48
  • 2
    @PatrickMevzek the question is based on who is wrong: dnschecker.org or the others. Testing it manually with dig shows these characters (greatly explained by you), so they are definitely there. He will probably need to place itself at the right of the v and backspace a few times to remove those two characters, then retype the v. – Ángel Aug 25 '22 at 22:51
  • "He will probably need to place itself at the right of the v and backspace a few times to remove those two characters, then retype the v." Yes, that is exactly what I said: it might not show on the UI (**of course** it is published in the DNS if some tools see it, they don't appear out of nowhere, they are in the zonefile), but still needs to be cleaned (or the developers of the UI need to be pestered out to do a cleaning automatically, I am sure lots of UI fail that) – Patrick Mevzek Aug 25 '22 at 23:34
  • And there is only one character (one unicode code point), not two. The UTF-8 representation of that character takes 3 bytes (octets). – Patrick Mevzek Aug 26 '22 at 00:01
  • 1
    He removes two characters: the bad one and the 'v', then adds the 'v' again. Removing the v may not be strictly necessary, but it's a way to ensure that you have surely removed everything at the beginning. – Ángel Aug 26 '22 at 00:14
  • 1
    Actually, deleting the characters wasn't something I could do in Cloudflare. I certainly tried hitting backspace next to the V. I had to copy the text from CloudFlare into Notepad++ and then change the encoding to ANSII. After which 3 characters appeared which I could delete, before changing back to UTF8. – clayRay Aug 26 '22 at 06:21