Is greylisting still an efficient method for preventing spam?

Question

I've used greylisting on my servers for many years, but I don't know how effective it is nowadays.

Is it still good for fighting spam in 2012?

Or is the typical spammer MTA capable of resending greylisted emails now?

Answer 1

I last looked at this quantitatively in July of this year (2012). In July, my mailserver received about 46,000 attempts to deliver mail; of those, about 1,750 returned and were permitted through by the greylisting (and passed valid sender domain, SPF and some other non-content-based tests). Of those, about another 1,500 were filtered by my content-based filtering..

Assuming that those 44,250 emails were spam (since they couldn't pass greylisting, I think that's a fair assumption), if it were not for the greylisting my content-based filtering would have had to deal with 46,000 mails instead of 1,750.

A twenty-five-fold increase in load on my content-based filtering would require me to have much beefier CPUs and more memory. That would in turn increase my monthly hosting costs, because of the extra power consumption (and, probably, the size of the server).

So in short, the last time I counted, yes, greylisting still made very, very good sense as part of a complete spam-filtering system. I have activated it for clients in the past few weeks, and all are extremely happy with the decrease in load on their content-based filtering systems also.

Edit: I note that I haven't answered the question about whether it's becoming less effective over time. When I turned it on, in late 2006, my estimate at that time was that it was filtering out about 95% of the spam. 1,750 as a proportion of 46,000 is about 4%, so my data suggest that it's not become less effective over that time period.

Answer 2

Update 2019: Conditional Greylisting is the best trade-off

After having used greylisting on all mails for a long time on a busy mailserver, I stopped doing so. It delays ham mails unnecessarily. This is especially annoying for mails with account activation links. It creates severe problems with cloud mailers that you need to whitelist (details in the answer below). Another disadvantage of greylisting all mails before passing them to your spam filter is that a learning spam filter misses the opportunity to get to know all the easy spam that the greylisting filtered away.

Instead of greylisting all mail, I now use a spam filter (rspamd, I highly recommend it) that only greylists mails that have a spam score between clear ham and clear spam. This way, ham mails hardly get delayed. On the other side, spam mails that hit greylisting often get detected as spam when the server tries the second time, because by then, the spam is often known in blacklists (RBLs, URIBLs) and fuzzy filters and thus gets a higher score.

So I recommend to greylist mails that are unclear whether they are spam or ham. Time really helps the spam filter to figure out the unclear cases.

Original answer from 2018:

I was always a great fan of greylisting. For these reasons:

• It does not only mark spam, it blocks it.
• It is legal to use as a service provider in Germany (unlike deleting spam mails after reception)
• It is simple and effective.
• It adds load to the spammer and not to your receiving mailserver. So even though the spammers may make it through your greylisting, you forced their machine to work harder and thus they can send less spam in total.
• It blocks almost no legit mail, unlike IP-based RBLs etc.
• It introduces delays, but you can whitelist clients (sending servers) of frequent contacts and whitelist recipients that really need email with minimum delay. Remember that using a spam filter like Spamassasin directly on all your mail (without greylisting) can introduce delays to legit mail as well: Some spammer sends so many mails to your server that the spam filter gets overloaded. Thus, it will send a temporary failure (e.g. 451) to the sending server of further incoming mails. This causes the same effects as greylisting, i.e. mails get delayed, with the exception that whitelisting is not that easy. Of course, you can use a cloud spam filter that scales to whatever power the spammer has, but that may be more expensive.
• Limited or no maintenance required. No blacklist that need to be updated and change over the time. No pattern-based rules that need to be updated.

But unfortunately, in my stats I see that in this year, greylisting becomes less and less effective. The amount of delayed messages really approaches the amount of greylisted messages rather fast, which means the amount of blocked spam is reducing.

In the last year (365 days), 55% of the greylisted messages made it eventually through greylisting, i.e. 45% got blocked.

mailgraph stats year

Note that this chart included a timeframe in which greylisted messages were not counted due to a configuration error of mailgraph, only delayed ones. This means this calculation overestimates delayed messages a little, in fact a little more mails got blocked.

In the last month, 64% got delayed and only 36% got blocked.

mailgraph stats month

In the last week, 75% got delayed and only 25% blocked.

mailgraph stats week

Moreover, looking at the total amount of blocked messages: This month, greylisting blocked 4 411 messages, but Amavisd (spamassasin) blocked 22 763 messages. This means only 16% of the spam gets blocked by greylisting, all the rest by amavisd.

Moreover, more and more cloud sending providers send from a bunch of several hundred IP-addresses. They attempt each transmission attempt from another IP. Thus, greylisting may block these mails for even days. Therefore, you need to whitelist all the "good" mail providers. This introduces new maintenance effort.

I have always been a great fan of greylisting, but sadly, I see that it is becoming less and less effective, and I think that I will disable it soon, as it starts to only delay 14% of my mails unnecessarily without blocking much spam.

The missleading stats

The amount of blocked mails in my (and your) stats may also be largely misleading. Let's take one email that is coming from a big cloud mail provider (like Microsoft's *.outbound.protection.outlook.com) that is not yet whitelisted. The first attempt fails. The second and third transmission attempts come from two other servers (IPs), so it again it fails, as the triplet does not match. Now the fourth attempt comes from the first server again and succeeds. This will be counted as one delayed transmission and four greylisted messages. My calculations above would indicate that 1/4=25% greylisted messages was delayed and 3/4=75% were blocked. But in fact, not a single message was blocked. Now we whitelist the servers of these mail providers, so they will not be greylisted anymore. What will happen is that the amount of greylisted messages will go down more than the amount of delayed messages. This means that the amount of blocked messages we calculate will go down. But it is not true that less messages were blocked.

In fact, what I did since February 2017, is adding more and more cloud mail providers to the whitelist to fight the problem of long delays due to greylisting. This may explain (partly?), why the amount of blocked mails that I calculate is going down rapidly. So maybe, I just thought all the time that greylisting is blocking lots of spam, but the amount of spam blocked was a lot less all the time, it was just calculated incorrectly. So be careful when interpreting your stats.

Answer 3

spambots usually still don't do message queueing , but some of them just send the spam twice to every recipient with a few minutes delay to defeat greylisting. also, nowadays, spam from spambots isn't the real problem anymore, spam from compromised yahoo accounts etc is much harder to catch.

From that point of view, greylisting is not as effective as it used to be. In combination with other anti-spam techniques it can still help, for example if your domain is often in the "first batch" of spam campaigns, greylisting can help delay the message long enough for domain/ip blacklists to catch up, so if the spam would have slipped through your filters on the first connection attempt, it maybe gets detected on the second attempt.

Answer 4

As a tangential issue, i don't like being in the position of having deployed a technique like greylisting without being able to measure its effectiveness. On Debian, with postfix as the MTA and postgrey as the greylisting policy engine, you can just apt-get install mailgraph to get a simple graph of accepted vs. rejected mail. Mailgraph is a bit old school and completely standalone, but it works, and its data or techniques could easily be integrated into a more complex modern monitoring system.

Answer 5

Get a reputation-based mail filter. Greylisting is a bit old-school and isn't a comprehensive solution. There are workarounds (from the spammer's perspective), and unpredictable mail delivery times for your users...

Either outsource the filtering to a cloud service or buy an appliance that has access to such a list and has other methods of validating spam. My recommendation is usually Barracuda for their appliance or for their cloud filtering solution. Both options have economies of scale and mature heuristics that provide a cleaner overall solution.

Looking at one of my client's Barracuda Spam Filter's report for September 2012, out of 98,457 messages, 1,623 were cut-off before even hitting the mail server because of bad recipients... 34,488 were blocked as SPAM. Only 96 questionable messages made it through. Those rated as SPAM were a combination of reputation, score, intent, three RBL's, Bayesian filtering and custom rulesets. All in one unit... All processed before hitting the relatively small mail server.

Also see: Fighting Spam - What can I do as an: Email Administrator, Domain Owner, or User?