-3

Is it possible to use Fail2Ban to block IPs that request the same URL more than 5 times in 10 seconds?

I'm not talking about a specific URL, but any random URL of the site that is being requested repeatedly.

For example:

I don't want to block in a situation like this:

111.222.333.444 - - [29/Feb/2016:06:53:30] "GET /aaa.html HTTP/1.1"...
111.222.333.444 - - [29/Feb/2016:06:53:30] "GET /bbbb.html HTTP/1.1"...
111.222.333.444 - - [29/Feb/2016:06:53:30] "GET /ccccc.html HTTP/1.1"...
111.222.333.444 - - [29/Feb/2016:06:53:30] "GET /dddddd.html HTTP/1.1"...
111.222.333.444 - - [29/Feb/2016:06:53:30] "GET /eeeeeee.html HTTP/1.1"...
111.222.333.444 - - [29/Feb/2016:06:53:30] "GET /ffffffff.html HTTP/1.1"...

But I want to block in a situation like this:

111.222.333.444 - - [29/Feb/2016:06:53:30] "GET /aaa.html HTTP/1.1"...
111.222.333.444 - - [29/Feb/2016:06:53:30] "GET /aaa.html HTTP/1.1"...
111.222.333.444 - - [29/Feb/2016:06:53:30] "GET /aaa.html HTTP/1.1"...
111.222.333.444 - - [29/Feb/2016:06:53:30] "GET /aaa.html HTTP/1.1"...
111.222.333.444 - - [29/Feb/2016:06:53:30] "GET /aaa.html HTTP/1.1"...
111.222.333.444 - - [29/Feb/2016:06:53:30] "GET /aaa.html HTTP/1.1"...

Important Note: I am not asking if I am under DDoS, nor what could I do if I were. What I am asking is if it is possible to use Fail2Ban to ban IPs that repeatedly request the same URL and how to do it.

viniciussss
  • 185
  • 3
  • 8
  • 7
    Please be aware that quite often a potentially large number of unique (mobile) customers might be behind a NAT gateway and those different users will all show with the same ip-address. With the exception of genuine denial of service attacks, if your service can't handle multiple requests for valid URL's you have a different problem IMHO and fail2ban is not the solution either. Rather than focusing on fail2ban you might want to consider what you can [do in nginx](https://www.nginx.com/blog/mitigating-ddos-attacks-with-nginx-and-nginx-plus/) instead. – HBruijn Mar 04 '16 at 06:36
  • Thanks for the advice, HBruijn. But I keep wondering if it is possible to use Fail2ban the way mentioned, and how. The example I used is just a simplified illustration. But the answer would help me a lot with more advanced filters that I want to create. Also, I would be very careful not to ban legitimate users. – viniciussss Mar 04 '16 at 19:57
  • 1
    It has emerged that [you asked the wrong question](http://meta.stackexchange.com/q/66377/189912). Since you are using nginx, please see [its documentation](http://nginx.org/en/docs/http/ngx_http_limit_req_module.html). – Michael Hampton Mar 05 '16 at 14:46

2 Answers2

2

I think you might be better served by using the Apache mod_evasive module, which is designed to stop IP addresses from accessing the same URL multiple times per second.

Digital Ocean has a pretty good How To.

jordantrc
  • 51
  • 5
  • The problem is that I use Nginx – viniciussss Mar 04 '16 at 03:14
  • 1
    @viniciussss nginx already has built-in tools for this! – Michael Hampton Mar 05 '16 at 14:44
  • Thanks @MichaelHampton and everyone! I've read all the links you suggested. I understand now that I've asked a XY problem question and that it would be better using nginx built-in tools like ngx_http_limit_req_module. I was trying to use Fail2Ban because I wrongly thought that it would be more efficient to block the attacking IPs in the firewall, instead of letting this job to nginx. – viniciussss Mar 05 '16 at 17:51
2

Fail2ban is the wrong tool or this job. The clue is in the name. It is designed to monitor for failures where you can be reasonably certain that IP addresses causing error messages matching the relevant regex are good to be banned.

You are trying to monitor and take a action on a access log, you don't know if the entries relate to a legitimate access or not.

Your first problem is going to be designing a regex that will match the activity you want to monitor.

If you manage that then you have to decide what level of activity over what timescale is acceptable and at what point it becomes unacceptable.

If you get this far will almost certainly get false positives and false negatives, tuning will be very difficult.

user9517
  • 114,104
  • 20
  • 206
  • 289
  • 1
    Imagine that I have the statistics that, during years, it never happened that an IP have accessed a same URL more than 50 times within 30 seconds. So, I could safely create a filter that would ban IPs that make more than 100 requests for an unique URL within 60 seconds. Also, I could let this filter disabled and enable it just in case of a DDoS attack. In this last situation I wouldn't even mind to have some false positives. – viniciussss Mar 04 '16 at 21:09
  • The first problem you have is to constrict a meaningful regex.You really are doing it wrong. Look else where. You also state that this is nothing to do with DDOS but again you go on about it. If You are defending a DDOS by the time you are doing so the DDOS has worked and used your resources. – user9517 Mar 04 '16 at 21:10
  • You're right. I'm not being able to constrict the regex. That's why I am asking for help here. To be honest, I think it is not possible. But I would like to make sure. I really think that it would be good to have a filter like that. Thanks for your opinion. You might be right in the end. – viniciussss Mar 04 '16 at 21:16
  • Ok I'll tell you it's not possible. – user9517 Mar 04 '16 at 21:17
  • Thanks! I have very limited knowledge about DOS and DDOS. It is not happening to me right now, but I have some competitors that could try to harm me. Maybe they have limited knowledge about DOS and DDOS too and the attack would not be so strong. That filter could help me in a simple DOS situation. I will try other alternatives, as you suggested. – viniciussss Mar 04 '16 at 21:25
  • 1
    If you have limited understanding of DOS/DDOS then first you should spend some time learning about them The duplicate above is a good start and contains references you should also read. Once you understand what a (D)DOS is you will understand the futility of attempting to fight it on your own server. Also, claiming this has nothing to do with (D)DOS and then going on about it casts doubt upon the veracity of your statements. – user9517 Mar 04 '16 at 21:31