Using fail2ban for detecting suspicious activity whitin a webserver

Question

I know that fail2ban is mostly used for blocking IPs trying to brute force an SSH endpoint and other stuff like that.

However, I am wondering if you could also use fail2ban to detect (not necessarily prevent) suspicious activity within a webserver.

For example: Let's say an attacker managed to break into a webserver as a non-privileged account (by whatever means: be it a Remote Code Execution flaw in the web application, brute-forcing a password or even a vulnerability in the Operating System). The attacker is now able to run commands on that machine. One of the things an attacker might now try to do is to somehow conduct a privilege escalation attack to gain root access. I guess that most attacks like that and other activities and intruder will normally do within a webserver would cause some traces within the log files - which could be monitored and admins could be warned via email or something like that.

If this is the case: Does it make sense to use fail2ban for this task?

If yes: How do you do that? Which log files should be watches for what kind of regex? Are there ready-to-use fail2ban jails for that somewhere on the web? (Couldn't find any.)

The webserver I'm talking about is of "midlevel importance" I'd say and runs with Ubuntu 18.04.

(I know there are many tools or Intrusion Detection Systems better suitable for that task. However, using fail2ban would have some advantages in my case.)

Answer 1

Sounds like you are trying to build a web application firewall (WAF) over Fail2Ban by creating a bunch of custom filters. The problems with this approach include:

• Fail2Ban works after something has already been achieved. Now the attacker only needs to wait for the ban time or simply use another IP address to collect the prize.
• Fail2Ban can only monitor the parts of the requests that are logged somewhere; e.g. POST data and cookies sent by the client aren't usually logged anywhere.
• Youd would need to create and maintain a huge variety of filters. That's awfully lot of work.
• Fail2Ban consumes lots of CPU time processing all the logs against all the filters.

Instead, you should be using an existing WAF that has a wide and maintained rule set, like ModSecurity with OWASP ModSecurity Core Rule Set (CRS). After the suspicious requests have been blocked with ModSecurity, you can use the logs produced by ModSecurity to block the IP addresses using Fail2Ban. E.g. with Apache you could use the updated filter from Kazimer Corp.

Answer 2

I know fail2ban and use it to secure my server. I am going to provide you with general guidance about the tool and the security practices.

fail2ban is a log analyser that continuously scans the system logs for known attempts to break in, and that will put firewall rules to prevent the offending IP to establish connections at network level.

The main purpose of fail2ban is to protect SSH and mail servers from brute force authentication attacks. The tool comes with a set of known regular expressions to detect failed login attepts. This is its main job.

Three points here: - Fail2ban works on a pattern recognition, so it cannot detect anything that is now known earlier - As soon as you can write log rules, fail2ban can use and ban people - Fail2ban cannot detect suspicious activity, only deterministic

When you talk about suspicious activity, that is something fail2ban is not made for. This is why you won't find jails. I suspect that you are not using the correct tool for your purpose.

Also, I would like to add a point about that. You mentioned "break into webserver". I assume you mean break into SSH, not a web application.

In my experience, I have two advices for you.

• Enhance SSH security to the strongest, detect root escalation attempts

Fail2ban might protect from bruteforced sudo authentication.

Here is what you get in /var/log/messages if you fail to authenticate as root from sudo

  sudo: pam_unix(sudo:auth): authentication failure; logname=user uid=1000 euid=0 tty=/dev/pts/0 ruser=user rhost=  user=root

Unfortunately, it doesn't carry the IP address of the SSH client. Anyways, I see the log has a rhost= field, so we could investigate how it gets populated.

Anyway, access to SSH in my opinion should be granted only to an extremely restricted audience, and file system shall be tightened to the best to prevent global files from being read by unauthorized audience. Public key authentication prevents people from choosing their own w3akP@sswo®d

• Detect attempts to hack your web server, not your application

Fail2ban can analyse log patterns. I have found this article as an example of Apache log analysis. In general, once you have patterns, you can add them to fail2ban as attempts to attack. DDoS is an example, but rather difficult to detect from failure logs.

Popular applications (e.g. Wordpress) might have had known SQLi or XSS vulnerabilities. Robot farms attempt daily to hack into such installations by polling random addresses, even thosw who don't run the target applications.

Attempts to hack (via known vulnerability) a popular application have patterns. I could find some in my own logs, e.g. Unicode characters in the URL indicate suspicious things, but the core is that as soon as somebody knows an attack pattern they can add it to fail2ban rules.

All these practices, anyway, do not replace advanced or tailored security practices in the web world, such as

• Proactive monitoring
• Subscribing to security newsletters for popular application
• Using web application security tools for custom applications

The third point means: if you run your own web application, written by yourself or your consultant, you must make sure you or they follow best practices, e.g. OWASP, in securing the resources.

As a final example, think what happens if you own application accepts https://host/orderApp/orders?orderId=4587935793 by only checking that the user is logged in with correct role but without checking the ownership of the order. (source: I have seen exactly that in financial applications)

Conclusion

fail2ban was made to prevent brute force attacks. It is the right tool to protect only from known threats once you master the rules. Ask on serverfault.com about how to properly write new rules and jails

Edit

fail2ban is definitely not the tool for such use case. You need to examine logs and the system to detect the effects of a break in your application (say, a remote code execution vulnerability).

Once you happen to find it, fail2ban can only help you stop repeating that attack at a fast rate, but it doesn't prevent the first strike (since there must be a log of the perpetrated attack for fail2ban to notice).

Really, if you find a vulnerability in your application the only thing you can do is to fix it!!!