1

I have random IPs constantly targetting my Apache server. A sample of what I get in my log:

80.108.96.31 - - [18/Aug/2017:16:16:08 +0000] "GET /machine.xml HTTP/1.1" 403 520 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.2; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729)"
200.163.163.189 - - [18/Aug/2017:16:16:08 +0000] "GET /machine.xml HTTP/1.1" 403 520 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; InfoPath.3)"
45.51.75.129 - - [18/Aug/2017:16:16:08 +0000] "GET /user.xml HTTP/1.1" 403 517 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.2; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729)"
83.130.137.207 - - [18/Aug/2017:16:16:08 +0000] "GET /user.xml HTTP/1.1" 403 517 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.2; WOW64; Trident/7.0; .NET4.0C; .NET4.0E)"
180.191.87.191 - - [18/Aug/2017:16:16:08 +0000] "GET /machine.xml HTTP/1.1" 403 520 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.2; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729; InfoPath.3)"

I have implemented fail2ban to check the access log and ban for a year but I ended up having these huge access logs and the server disk got full of just the access logs. Help please.

Jenny D
  • 27,358
  • 21
  • 74
  • 110
Kal
  • 143
  • 6

1 Answers1

2

You'll probably want to install LogRotate to help you keep the logs a bit tidier.

Fail2Ban could work for what you're trying to do but you may need to create some detailed filters in order to get the results you're looking for, simply pointing Fail2Ban at the access log and setting a long timer isn't going to help at all.

Also if you're getting that many requests it's likely a distributed attack, and blocking a single IP at a time with Fail2Ban may not be the best approach -- however, if that's how you want to go about it I'd create filters specifically for user.xml, machine.xml, and for anyone creating more than a few 403s in a given time frame.

You may want to create your own firewall rules and drop any traffic from most of the world - you can find lists of 'known bad actor' IP blocks and just ban all those blocks - you can also do a WHOIS lookup on the IP's coming at you and pull out whatever blocks they're coming from and start blocking them that way.

Or create a firewall rule or .htaccess rule to simply only allow traffic from known locations that will use your apache server.

But really.... if this is a webserver that needs to be accessible to everyone then you're not going to be able to do much about it getting hit a lot, get LogRotate installed, expand your Fail2Ban rules a bit if you want and forget about it, you're never going to stop what you're seeing entirely unless the scope of your project would allow you to specifically ALLOW certain IPs only to hit your server.

Sam K
  • 506
  • 5
  • 20
  • The issue is that Fail2Ban relies on the logs for it to ban. So the log file gets bigger by the minute, at the same time fail2ban is doing its thing. – Kal Aug 23 '17 at 06:33
  • Yep, that's the life of running a web server ;) LogRotate lets you take yesterday's log, move it to a new file, compress it, and start a new file for today's log. You can tweak it to save X amount of days, compress a weekly or monthly archive, whatever you like, but its a massive space-saver. And if you can't hold 1 days worth of logs on your server, well, then get a server with more space I guess? How big of a log file daily are we talking? – Sam K Aug 23 '17 at 14:59