Why is it even possible to forge sender header in e-mail?

Question

With so many popular e-mail providers forcing users to log on using their SMTP servers, why is it still possible to forge "From: " header in e-mails? What prevents users from simply discarding the e-mails in which the source domain of the sender doesn't match the domain of SMTP server?

Answer 1

tl;dr

• It's very easy to spoof a domain even with SPF controls enabled.

• The solution is to use DKIM + DMARC, or SPF + DMARC

• The email client is responsible for telling you if the message passes DMARC Display From verification

• The email protocol allows for legitimate spoofing using Resent-* headers and Sender headers. The email client (MUA) should display this exception whenever it exists.

There are a few misconceptions about SPF, namely:

1. SPF does not prevent email spoofing.
2. SPF alone doesn't affect, influence or, control the RFC 2822 Display From.
3. By default, the usefulness of SPF is to prevent backscatter issues and very simple spoofing scenarios.

Microsoft attempted to solve this issue with SenderID, (making SPF apply to the Display From address) but it was too complicated and didn't really solve the whole problem.

---

Some background

First know that there are two "from" addresses and two "to" addresses in every SMTP message. One is known as the RFC2821 Envelope, the other is the RFC2822 Message. They serve different purposes

The Envelope: (RFC2821)

• The envelope is metadata that doesn't appear in the SMTP header. It disappears when the message goes to the next MTA.

• The RCPT From: is where the NDRs will go. If a message is coming from Postmaster or a remailer service this is usually <> or someSystem@place.com. It's interesting to see that salesforce uses this similar to constantContact as a key in a database like toUserContactID@salesforce.com to see if the message bounced.

• The RCPT TO: is who the message is actually being sent to. It is used for "to" and "bcc" users alike. This doesn't usually affect the "display of addresses" in the mail client, but there are occasions where MTAs will display this field (if the RFC2822 headers are corrupt).

The Message (RFC2822)

• The message portion begins when the data command is issued.

• This information includes the SMTP headers you're familiar with, the message, and its attachments. Imagine all this data being copied and pasted from each MTA to the next, in succession until the message reaches the inbox.

• It is customary for each MTA to prefix the above mentioned copy and paste with information about the MTA (source IP, destination IP, etc). It also pastes the SPF check details.

• This is the Display From is placed. This is important. Spoofers are able to modify this.

• The Mail From: in the envelope is discarded and usually placed here as the return-path: address for NDRs

So how do we prevent people from modifying the Display From? Well DMARC redefines a second meaning for the SPF record. It recognizes that there is a difference between the Envelope From and the Display From, and that there are legitimate reasons for them to not match. Since SPF was originally defined to only care about Envelope From, if the Display From is different, DMARC will require a second DNS check to see if the message is allowed from that IP address.

To allow for forwarding scenarios, DMARC also allows the Display From to be cryptographically signed by DKIM, and if any unauthorized spammer or phisher were to attempt to assume that identity, the encryption would fail.

What is DKIM? DKIM is lightweight cryptographic technology that signs the data residing in the message. If you ever received a message from Gmail, Yahoo, or AOL then your messages were DKIM signed. Point being is that no one will ever know youre using DKIM encryption and signing unless you look in the headers. It's transparent.

DKIM can usually survive being forwarded, and transfered to different MTAs. Something that SPF can't do. Email administrators can use this to our advantage to prevent spoofing.

---

The problem lies with the SPF only checking the RFC2821 envelope, and not the Display From. Since most people care about the Display From shown in an email message, and not the return path NDR, we need a solution to protect and secure this piece.

This is where DMARC comes in. DMARC allows you to use a combination of a modified SPF check or DKIM to verify the Display From. DKIM allows you to cryptographically sign the RFC2822 Display From whenever the SPF doesn't match the Display From (which happens frequently).

---

Your questions

Why is it still possible to forge "From: " header in e-mails?

Some server administrators haven't implemented the latest technologies to prevent this sort of thing from happening. One of the major things preventing adoption of these technologies is "email forwarding services" such as a mailing list software, auto-forwarders, or school alumni remailer (.forwarder). Namely:

2. Either SPF or DKIM isn't configured.

3. A DMARC policy isn't set up.

4. The email client isn't displaying the verification results of the Display From and the Resent-* or Sender field.

What prevents users from simply discarding the e-mails in which the source domain of the sender doesn't match the domain of SMTP server?

What doesn't match: the envelope or the body? Well according to email standards the envelope shouldn't match if it's going through a remailer. In that case we need to DKIM sign the Display From and make sure the MUA verifies this.

Finally, the MUA (email client) needs to show if the sender is DMARC verified, and if someone is trying to override that with a Sender or Resent-From header.

Answer 2

What prevents users from simply discarding the e-mails in which the source domain of the sender doesn't match the domain of SMTP server?

Nothing. They are allowed to, and many use this to filter out spam.

Like a lot of technologies with security shortcomings, e-mail wasn't designed with security and authentication in mind. The purpose of the "from" field was more-or-less the same as for a traditional snail-mail envelope. To make e-mail usable in the light of 21st century needs and abuses, we have had to added adjuncts. Specifically, SPF addresses the first part of your question:

With so many popular e-mail providers forcing users to log on using their SMTP servers, why is it still possible to forge "From: " header in e-mails?

The solution is to verify the header/IP at receiving time. While this doesn't technically prevent the actual forging of the header, it does make forged headers less useful.

Answer 3

There is already partial protection against that: https://en.wikipedia.org/wiki/Sender_Policy_Framework

All popular email providers use it. But they handle SPF failure differently, some mail providers will totally discard messages based on that check, and some providers will just put it in spam folder.

Anyone can create his own email server and try to send as anyone.

You can check your gmail address (if you have one) and open the header of any email, and you will see the result of the SPF check on the top:

spf=pass (google.com: domain of *****@*****.com designates 174.**.**.** as permitted sender) smtp.mail=*****@*****.com;

Answer 4

[...] why is it still possible to forge "From: " header in e-mails?

SMTP is a protocol designed back when you could likely name every machine on the Internet, and running a machine that provided mail was a relatively big deal, undertaken by large organisations and very dedicated hobbiests. It doesn't have any built-in protection or security against bad actors and trusts pretty much whatever you tell it.

There are also a whole host of legitimate reasons you may want your 'From:' header to be different to your domain - perhaps your mail server is shared with many other domains (such as a web/mail hosting company), in which case your reverse (PTR) record and your forward (A/MX) record will necessarily differ.

However, given that the majority of users, in a majority of cases will be using a mail service that matches their mail domain; we can apply some logic around SMTP. This leads on to your next question:

What prevents users from simply discarding the e-mails in which the source domain of the sender doesn't match the domain of SMTP server?

The answer here is nothing, and people do.

There are a whole host of methods for identifying spam such as:

• As you suggest - checking that the sender domain matches the sending domain (either via DNS, or by connecting to the sender and checking)
• Checking the SPF records for the sending domain for 'allowed' hosts
• Bayesian spam filtering / Markovian Discrimination
• Blacklists

Classification problems like identifying spam/non-spam are major areas of study. It is, unfortunately, not just as simple as dropping mail that doesn't come from where you'd expect (witness the huge number of spam messages coming from gmail accounts to gmail accounts), and applying that rule is likely to hit a huge number of snags (in the case where your mail server is in some way shared); you can however include that in your analysis of the message and trust it less, and once you get past some level of certainty, just drop it.

Email will likely always have these sorts of abuse issues as it's inehently a system for allowing people to send and receive messages to anyone with as little friction as possible. By it's nature, it allows people you don't know to send you messages you weren't expecting, and, just like with the regular mail system, this is both a good and bad thing.