I'm setting up a LAMP server and need to prevent SSH/FTP/etc. brute-force logon attempts from succeeding. I've seen many recommendations for both denyhosts and fail2ban, but few comparisons of the two. I also read that an IPTables rule can fill the same function.
Why would I choose one of these methods over another? How do people on serverfault handle this problem?
IIRC, DenyHosts will only watch your SSH service. If you need it to protect other services as well, Fail2ban is definitely a better choice. It is configurable to watch nearly any service if you are willing to tweak its configuration, but that shouldn't be necessary as the newer versions of Fail2ban include rulesets which are suitable for many popular server daemons. Using fail2ban over a simple iptables rate limit has the advantage of completely blocking an attacker for a specified amount of time, instead of simply reducing how quickly he can hammer your server. I've used fail2ban with great results on a number of production servers and have never seen one of those servers breached by a brute force attack since I've started using it.
Don't let them get to your machine in the first place! There are plenty of ways to stop brute force attempts before they get to your host, or even at the SSH level.
Having said that, protecting your Operating System with something like fail2ban is a great idea. Fail2ban is slightly different to DenyHosts, though they do play in the same space. Fail2ban uses iptables.
http://en.wikipedia.org/wiki/Fail2ban
Fail2ban is similar to DenyHosts ... but unlike DenyHosts which focuses on SSH, fail2ban can be configured to monitor any service that writes login attempts to a log file, and instead of using /etc/hosts.deny only to block IP addresses/hosts, fail2ban can use Netfilter/iptables and TCP Wrappers /etc/hosts.deny.
There are a number of important security techniques you should consider to help prevent brute force logins:
SSH:
Application:
ANOTHER GREAT WAY TO PROTECT SSH (I have used this for a decade or better) is to use the recent libraries in iptables natively(depending on your distro).
Basically it can be used as port knocking thats built into iptables. This will save you plenty of headaches. As long as you can tcp connect(telnet is one way. I have also used ssh clients and pointed them at the port. Anything that will do a tcp connection to a specified port number. I'm looking at you Putty!) from the client initiating the ssh connection you can use this.
Below is an example that will have iptables open port 22 to your host when you telnet from your host to the server on port 4103. You can then use a telnet to port 4102 or 4104 to close sed opening. The reason for both 4102 and 4104 is to keep a simple tcp scan from opening 22. Only a tcp connect(telnet) to port 4103 will allow you in.
Enjoy!
Oh and I favor Fail2Ban. More flexibility and I like that the ban happens in iptables rather than tcpwrappers.
iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -m recent --rcheck --name SSH -j ACCEPT
iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 4102 -m recent --name SSH --remove -j DROP
iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 4103 -m recent --name SSH --set -j DROP
iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 4104 -m recent --name SSH --remove -j DROP
Another difference between Fail2ban and Denyhosts is that Denyhosts can share the block list with other Denyhosts users. With Fail2ban, you can only block IPs that your server has seen before - with Denyhosts, a brute-force attempt may never even get to your server, if somebody else has seen it, and the block list is downloaded to your server before the attacker gets to your computer.
Yet another difference is that Fail2ban uses iptables, while Denyhosts uses tcpwrappers. Others have mentioned this difference before, but there are a couple of side notes worth mentioning.
iptables is limited in how many IP addresses you can block efficiently. That's probably one of the reasons why Fail2ban doesn't have a mechanism to share block lists.
Another effect is that when iptables is replaced with nftables, Fail2ban will probably stop working or needs to be rewritten. Denyhosts will likely continue working.
So, both have advantages and drawbacks. I like both; for myself, I'm using Denyhosts because usually I only want to protect SSH, and I like sharing the block list.
I use iptables rules to rate-limit new connections from the same IP address (SSH mainly, but it'd work fine for FTP, too). The advantage, as I see it, over "fail2ban" and other such tools is that the iptables route occurs totally in kernel mode and doesn't rely on any user mode tools to tail / parse log files.
If you can do it, limiting the source addresses that can access the protcols in question will, obviously, help as well.
With SSH, you really should be using certificate authentication and not accepting passwords anyway.
One thing to note about Fail2Ban is that it seems to use about 10MB more memory than DenyHosts. So if you are on a 128MB VPS you might want to look into that. Also, out-of-the-box fail2ban is only setup on SSH which means that with no changes to the config - DenyHosts does the same thing in less memory.
denyhosts is for ssh. fail2ban is more comprehensive (HTTP, FTP, etc.). Both use iptables behind the scenes.
Denyhosts version 3.0: Every time an IP address appears in a log file, Denyhosts opens the hosts.deny file and reads the whole thing to match the address. Every time. Nothing is cached in memory. If you have a huge hosts.deny file and are subject to many probes (many log file entries), Denyhosts becomes a CPU hog reading and re-reading the hosts.deny file for every IP address that appears. Not good.
If you enable iptables support, Denyhosts will create huge, slow lists of blocked IP addresses. Denyhosts doesn't use ipset or nftables to create efficient IP maps.
Instead of messing with tedious iptables or fail2ban config, why not have the open community do all the work for you, and use CSF/LFD instead? I can highly recommend it above all other mentioned options. See http://configserver.com/cp/csf.html for what it can do for your servers. CSF does not require a control-panel, it offers a simple UI itself, for those who do not want to do it by shell. And it's a lot of stable reliable non-resident perl-scripting.
fail2ban doesn't appear to have a mechanism for recognising a successful ssh login and resetting its count of failures.
The standard filter for sshd (at least on my debian install), clocks up a failure count for each ssh key that the client presents which the server rejects. Some users present many keys on every login and regularly get locked out, despite their logins having been successful once a few keys had been gone through.
As a result of the above, I'm currently thinking about moving away from fail2ban. In this respect at least, denyhosts is better. However, it's apparently no longer a good option, and no longer supported in more recent versions of debian (some discussion at https://www.chrissearle.org/2015/06/16/replacing-denyhosts-with-fail2ban-for-debian/)
I don't have a good solution here.
Actually, I think denyHost is able to prevent many other services besides sshd service. In its configure file - /etc/denyhosts.conf, there are some lines of code said:
# BLOCK_SERVICE: the service name that should be blocked in HOSTS_DENY
#
# man 5 hosts_access for details
#
# eg. sshd: 127.0.0.1 # will block sshd logins from 127.0.0.1
#
# To block all services for the offending host:
BLOCK_SERVICE = ALL
# To block only sshd:
# BLOCK_SERVICE = sshd
so if we set BLOCK_SERVICE variable to be ALL as above we can watch our ssh service.
