16

As a hosting provider, we send email on behalf of our clients, so we help them set up DKIM and SPF email records in their DNS to get email deliverability just right. We've been advising them to use http://mail-tester.com to test that they didn't miss anything, and I like this tool a lot.

One problem we've run into a few times, and I'm not sure about, is the DNS "limit" on the SPF record based on domain name. So if you have this:

v=spf1 a include:aspmx.googlemail.com include:campaignmonitor.com include:authsmtp.com include:mail.zendesk.com include:salesforce.com include:_hostedspf.discourse.org ~all

You'll get

example.com ... campaignmonitor.com: Maximum DNS-interactive term limit (10) exceeded

Like so:

mail-tester results

I have some questions about this.

  1. I count six domain names here, not 10, so why is it hitting "ten" DNS requests here? Answered here

  2. Is this 10 DNS interactive term limit a warning or a real error? e.g. should we care? It is nagging our customers a bit and they email us for support. Answered here

  3. Is this 10 DNS interactive term limit a real problem on today's web? As you can see, this customer has a lot of services sending email for them and they are all legitimate. Perhaps this DNS limit was set in the year 2000 when delegating email services like this were not common?

Yes, we can have our customers change the include to IPs in the SPF record but that puts us in a bind if we ever change IPs, a bunch of customers' stuff will break. Really don't want to do that..

What workarounds are there for this?

Community
  • 1
Jeff Atwood
  • 12,994
  • 20
  • 74
  • 92
  • Probably a duplicate: [PermError SPF Too Many Lookups and Reduction](http://serverfault.com/q/603797/126632). See also [Is the 10-DNS-lookup limit in the SPF spec typically enforced?](http://serverfault.com/q/584708/126632) – Michael Hampton Aug 24 '15 at 21:00
  • Darn, I searched for the error message but got zero hits. – Jeff Atwood Aug 24 '15 at 21:06
  • 2
    I wouldn't expect you to find anything searching for that. It came from an online testing tool, rather than a real world problem (in which you'd see something like the PermError message in the linked question). – Michael Hampton Aug 24 '15 at 21:07
  • I like those others, but I am not seeing answers that provide a workaround? Is this 10 lookup limit actually enforced in practice? – Jeff Atwood Aug 24 '15 at 21:20
  • The second question Michael linked to seems to answer that fairly well. In summary, "yes, the 10 lookup limit is enforced in practice". – womble Sep 09 '15 at 21:00
  • 1
    Add https://dmarcian.com/spf-survey/ to your toolset, make sure if you are providing an SPF for your customers, it is not the same SPF you use directly (don't include 3rd parties in your included spf) – Jacob Evans Sep 23 '15 at 04:18

4 Answers4

8

  1. Assuming that redundancies (like multiple references to _spf.google.com and the records it refers to) are only counted once, I count 17 lookups from the point where you've already looked up the initial record. (See below.)

  2. It refuses to look up all the records necessary to evaluate your SPF record because it would be "too much work". Presumably this means it will treat your domain as if it had no SPF record (or possibly reject it). The spec says that this results in permerror, which leaves it fairly open for the recipient to decide what to do.

  3. I think abuse has been going up rather than down, generally. This limit appears to be meant to thwart abusive sender domains that may otherwise be able to overwhelm the recipient with enormous chains of SPF, potentially leading to DoS.

I think that while outsourcing email is common, it's not actually that common to outsource email to six different providers. You'll have to optimize the SPF record somehow.
(For one thing, the reference to aspmx.googlemail.com seems wasteful as that immediately just redirects to a different name.) 

<lookup of example.com A>                   #1
$ dig aspmx.googlemail.com TXT +short       #2
"v=spf1 redirect=_spf.google.com"
$ dig _spf.google.com TXT +short            #3
"v=spf1 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com ~all"
$ dig _netblocks.google.com TXT +short      #4
"v=spf1 ip4:64.18.0.0/20 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:74.125.0.0/16 ip4:173.194.0.0/16 ip4:207.126.144.0/20 ip4:209.85.128.0/17 ip4:216.58.192.0/19 ip4:216.239.32.0/19 ~all"
$ dig _netblocks2.google.com TXT +short     #5
"v=spf1 ip6:2001:4860:4000::/36 ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:4000::/36 ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ~all"
$ dig _netblocks3.google.com TXT +short     #6
"v=spf1 ~all"
$ dig campaignmonitor.com TXT +short        #7
"google-site-verification=HcHoB67Mph6vl5_x4gK5MN9YwN5gMgfZYdNmsP07tIg"
"v=spf1 mx ptr ip4:23.253.29.45/29 ip4:203.65.192.250 include:cmail1.com include:_spf.google.com include:stspg-customer.com ~all"
$ dig cmail1.com TXT +short                 #8
"google-site-verification=HSJ8sL4AxQo0YHHNk9RwDqs0p3lJPGmc1nCrSsmous8"
"mailru-verification: 95d4c6eb0645b43c"
"v=spf1 ip4:103.28.42.0/24 ip4:146.88.28.0/24 ip4:163.47.180.0/22 ip4:203.55.21.0/24 ip4:204.75.142.0/24 ~all"
$ dig stspg-customer.com TXT +short         #9
"v=spf1 ip4:166.78.68.221 ip4:166.78.69.146 ip4:23.253.182.103 ip4:192.237.159.42 ip4:192.237.159.43 ip4:167.89.46.159 ip4:167.89.64.9 ip4:167.89.65.0 ip4:167.89.65.100 ip4:167.89.65.53 -all"
$ dig authsmtp.com TXT +short               #10
"v=spf1 include:spf-a.authsmtp.com include:spf-b.authsmtp.com ~all"
"google-site-verification=skc1TleK4GylDiNZUayfvWWgqZIxmmiRj4KgXlCgB8E"
$ dig spf-a.authsmtp.com TXT +short         #11
"v=spf1 ip4:62.13.128.0/24 ip4:62.13.129.128/25 ip4:62.13.136.0/22 ip4:62.13.140.0/22 ip4:62.13.144.0/22 ip4:62.13.148.0/23 ip4:62.13.150.0/23 ip4:62.13.152.0/23 ~all"
$ dig spf-b.authsmtp.com TXT +short         #12
"v=spf1 ip4:72.52.72.32/28 ip4:64.49.192.16/29 ip4:209.61.188.242 ip4:64.49.192.24 ip4:64.49.192.25 ip4:64.49.210.64/29 ip4:64.49.210.72/30 ip4:64.49.210.76 ip4:64.49.210.77 ip4:64.49.210.78 ~all"
$ dig mail.zendesk.com TXT +short           #13
"v=spf1 ip4:192.161.144.0/20 ip4:185.12.80.0/22 ip4:96.46.150.192/27 ip4:174.137.46.0/24 ~all"
$ dig salesforce.com TXT +short             #14
"adobe-idp-site-verification=898b7dda-16a9-41b7-9b84-22350b35b562"
"MS=749862C9F42827A017A6EA2D147C7E96B3006061"
"MS=ms68630177"
"v=spf1 include:_spf.google.com include:_spfblock.salesforce.com include:_qa.salesforce.com ip4:136.146.208.16/28 ip4:136.146.210.16/28 ip4:136.146.208.240/28 ip4:136.146.210.240/28 ip4:85.222.130.224/28 ip4:136.147.62.224/28 ip4:136.147.46.224/28 mx ~all"
$ dig _spfblock.salesforce.com TXT +short   #15
"v=spf1 ip4:96.43.144.0/20 ip4:182.50.76.0/22 ip4:202.129.242.0/23 ip4:204.14.232.0/21 ip4:62.17.146.128/26 ip4:64.18.0.0/20 ip4:207.126.144.0/20 ip4:68.232.207.20 ip4:207.67.38.45 ip4:198.245.81.1 ip4:198.245.95.4/30 ip4:136.146.128.64/27  ~all"
$ dig _qa.salesforce.com TXT +short         #16
"v=spf1 ip4:199.122.122.176/28 ip4:199.122.121.112/28 ip4:199.122.122.240/28 ip4:66.231.95.0/29 ~all"
$ dig _hostedspf.discourse.org TXT +short   #17
"v=spf1 ip4:64.71.148.0/29 ip6:2001:470:1:3c2::/64 -all"
Community
  • 1
Håkan Lindqvist
  • 33,741
  • 5
  • 65
  • 90
8

  1. Mostly already answered, please do note including Google this way is wrong - you want to use _spf.google.com or incur a penalty for the redirect:

     ○ → host -t txt aspmx.googlemail.com
 aspmx.googlemail.com descriptive text "v=spf1 redirect=_spf.google.com"

 ○ → host -t txt _spf.google.com
 _spf.google.com descriptive text "v=spf1 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com ~all"

That lookup will consume 5/10 all on its own - 4/10 still sucks but 20% less.

  1. It will stop processing and return a permanent error - it's up to the engine using the SPF to decide how it wants to treat a permanent error.

  2. Yes - without the processing limits SPF mechanisms could be used as a DoS amplifier against a third party or second party.

As a workaround, emails can come from a subdomain of the main property - community.largecorporation.com for instance.

Community
  • 1
MikeyB
  • 38,725
  • 10
  • 102
  • 186
  • I believe using a subdomain breaks DKIM though? I know we've had trouble with this in the past. Seems like that is the only solution though. – Jeff Atwood Aug 24 '15 at 23:39
  • 1
    @JeffAtwood Normally DKIM is is signed by the sending domaim. If you use a subdomain, then sign with the sub-domain. However, it is legimiate to sign for a subdomain, but may not get the processing. DKIM records need to be created relative to the signing domain. It is also common for the originator to sign the document to allow origin verification. – BillThor Aug 25 '15 at 03:08
  • 1
    As long as the respective SPF and DKIM records are present for the [mail domain rather than the root domain](http://serverfault.com/q/580073/2101) and you're signing with `d=subdomain.example.com`, it'll be fine. In theory. Better test it! – MikeyB Aug 25 '15 at 13:37
5

As the accepted answer to one of the linked questions makes clear, many of the underlying tools for UNIX systems do indeed enforce this limit (though not all in precisely the same way) so any SPF implementation which uses them - which is nearly all on UNIX - will enforce those limits also. Windows systems are a law unto themselves, and I cannot shed any light on them.

The workaround is to have a cron job that evaluates your chain of outsourced SPF records, expresses them all as ipv4 and ipv6 netblocks, and makes that into your record. Don't forget the -all.

In your case, you want customers to be able to publish an SPF record which they then don't need to maintain. One possibility would be to have each customer publish a record containing redirect=spf.client1.jeffs-company.example, and you then do the legwork of maintaining the list of netblocks at jeffs-company.example.

Perhaps this DNS limit was set in the year 2000 when delegating email services like this were not common?

The limit does make it hard to outsource your email to six or seven large operations; but arguably if you are doing that you have for all practical purposes lost control of your email anyway.

Somewhere, someday, some sub-sub-contracted programmer of whose existence you were completely unaware and over whom you have no control is going to misplace a semicolon, and a ton of bogus email will get sent with your SPF imprimatur squarely upon it. Full control of your email requires full control of your email infrastructure, and that is in my opinion entirely inconsistent with that much outsourcing.

Community
  • 1
MadHatter
  • 78,442
  • 20
  • 178
  • 229
0

Another way of working around those problems is looking at which software exactly is used to check SPF-settings. In my case it's cluebringer/PolicyD, which uses Mail::SPF::Server in the end and that accepts arguments relaxing otherwise hard-coded limits. The problem is that cluebringer itself doesn't support relaxing those arguments currently, but that might get changed in future and one might be able to simply tell receiving service providers about those possibilities to relax their settings.

If they decide to do so is of course out of ones control, but it's at least a chance.

Thorsten Schöning
  • 643
  • 5
  • 16