Will mailing lists "break" if SPF is too restrictive?

Question

I have recently configured my own mail server (Linux-based postfix + dovecot scenario). This is just for personal use - I have no bulk mail going out, no automatically generated mail outbound from the host, nothing like that. I've gone through the trouble to configure all the additional fun-to-debug email DNS records:

@                 IN  TXT  v=spf1 +mx -all
_domainkey        IN  TXT  o=-; r=dkim@example.com
mail._domainkey   IN  TXT  v=DKIM1; h=sha256; k=rsa; s=email; p=deadbeef
_adsp._domainkey  IN  TXT  dkim=all
_dmarc            IN  TXT  adkim=s; aspf=s; fo=1; p=none; pct=100; rf=afrf; ri=86400; rua=mailto:aggrep@example.com; ruf=mailto:authfail@example.com; sp=none; v=DMARC1;

I've got an IP that isn't on any blacklists, a PTR correctly configured, DKIM signatures validate perfectly, I thought everything was set up correctly.

But now I can't contribute to mailing lists. When I send to the list address, sometimes the message goes into a black hole, sometimes I get an email to my authfail@ address, and in other cases I see entries I believe are related in reports sent to aggrep@.

My theory is that the SPF policy is too restrictive. The mailman (or other) list server is acting as an SMTP relay for my messages, right? So I changed

@                 IN  TXT  v=spf1 +mx -all

to

@                 IN  TXT  v=spf1 +mx ~all

making the default action a softfail instead of a hardfail. The problem is, I don't want to go around spamming lists for no good reason to test this change. Has anyone else been here before and can verify/refute my theory?

---

EDIT 1:

Thinking back, and thanks @Alex for setting me straight, I really haven't provided enough data to make an accurate assessment. Here's an example of a notice I received on my authfail@ address when attempting to post to a mailing list:

This is a spf/dkim authentication-failure report for an email message received from IP 66.211.214.132 on Thu, 10 Jul 2014 20:58:52 +0800.
Below is some detail information about this message:
 1. SPF-authenticated Identifiers: archlinux.org;
 2. DKIM-authenticated Identifiers: none;
 3. DMARC Mechanism Check Result: Identifier non-aligned, DMARC mechanism check failures;

For more information please check Aggregate Reports or mail to abuse@126.com.



Feedback-Type: auth-failure
User-Agent: NtesDmarcReporter/1.0
Version: 1
Original-Mail-From: <arch-general-bounces@archlinux.org>
Arrival-Date: Thu, 10 Jul 2014 20:58:52 +0800
Source-IP: 66.211.214.132
Reported-Domain: example.com
Original-Envelope-Id: w8mowEA5UUwMjr5TlWQfBA--.250S2
Authentication-Results: 126.com; dkim=fail (signature error: RSA verify failed) header.d=example.com; spf=pass smtp.mailfrom=arch-general-bounces@archlinux.org
DKIM-Domain: example.com
Delivery-Result: delivered

Looks to me like this is a DKIM signature failure, but I have no idea why. Is the receiving server trying to verify my DKIM signature against the mailing-list-server's key, or vice versa? For some reason, I wouldn't expect this to happen - I remember reading somewhere that in cases like this Relays and such will sometimes remove/munge headers like this to ensure these types of failures don't occur?

---

EDIT 2:

Thanks to @Christopher Karel for referencing a DMARC report parsing tool at dmarcian.com. The lions share of entries are listed as forwarders (which makes sense). There is one server (*.mailhop.org) listed as "preserv[ing] DKIM" - I've sucessfully sent mail over one of the Ruby language forums which has worked, and I know from my research they use mailhop.org.

Under the category "Servers that break DKIM signatures (or create spoofed signatures)" are listed *.archlinux.org, *.google.com, *.mailhop.org (dunno why this appears here, maybe another list I'm on uses them as well in a different configuration), among others and the lists I've been most active on are Arch and a few hosted by Google Groups so this makes sense. About 400 messages in total - I haven't sent nearly that many messages, so I guess maybe it's counting retries.

I'm getting depressed - at the moment it seems like my choices are:

1. Keep SPF, DKIM, DMARC, and ADSP and give up using mailing lists, or
2. Drop this DNS security/reporting layer and have my normal outgoing mail rejected by Google, Yahoo!, Live, etc.

Answer 1

E-Mail security sucks. So in the end, you're probably going to be faced with a decision where all your options are terrible, and break different things for different reasons.

As for SPF specifically, a mailing list will cause a failure if it forwards a message, without rewriting the headers. A list can configure itself to work however it pleases, so there isn't a good general answer. But if messages from a list appear to come from the list itself, it's rewriting headers. If it appears to come from the sender, probably not. In general, mailing lists should work well with SPF on its own. Regular mail forwarding, on the other hand, will not.

When it comes to DKIM, any modifications to the message will cause a failure. This almost always occurs with a mailing list. So DKIM will usually bomb with mailing lists. But mail forwarding should be OK.

On top of it all, you've implemented DMARC. This is essentially a reporting infrastructure wrapped around DKIM and SPF. It works best if you implement both both authentication measures, but will also work fine with just one. You can configure DMARC to communicate a drop request for your messages, but more importantly, you can specify an address to receive success/failure reports. These are supported by most of the major E-Mail receivers. (GMail, Hotmail, Yahoo) This can give you insight on what messages are failing SPF checks, and why. Use this to inform your -all vs ~all decision.

Unfortunately, the DMARC specification requires alignment between the Sender domain and the SPF record that gets checked. In your case, the mailing list's SPF is being checked, and passes, but doesn't align with your domain. So DMARC bombs. Here's a reference from a mailing list administrator griping about as much.

The conclusion is the same as my opening sentence: E-Mail security Sucks. And all of your options suck too. IMHO, mailing lists suck as well, and life would be better if we replaced them. ;-)

Answer 2

I've not looked at the DKIM part.

Concerning the SPF record I see the following used in most example:

v=spf1 mx -all

This is documented here: http://www.openspf.org/SPF_Record_Syntax

However "+mx" should also be correct according to RFC 7208 (Thanks Chris for pointing this out). Maybe it is stil worth a try...

I really don't know what to suggest otherwise... double check all your DNS record (A / PTR / MX). You probably already did so. Knowing the actually domain name might help people troubleshoot - at least if DNS related.

Answer 3

Turns out there doesn't seem to be anything wrong with my configuration. What's happening is that my messages are being processed by mailman correctly, and being relayed out to the list. There are a couple of receivers however which (for whatever reason is unique to them) reject the message. Because I have actually correctly configured SPF, I'm seeing the rejection message from those destination SMTP servers, not from the Mailing List relay itself.

Some awesome folks in the Arch community helped me chase this one down, as they had access to said ML server.