2

I'm trying to block web requests based on if they contain any of a group of strings. As far as I can tell between my own knowledge and searching online, what I have is correct. However, I'm still seeing these types of requests in the access logs.

Would access log entries still be created if the request was blocked in the Apache config? Or is what I have in my VHOST definition wrong?

RewriteEngine On
Options +FollowSymLinks
RewriteCond %{HTTP_USER_AGENT} ^.*(icevikatam|AhrefsBot|spbot|MJ12|majestic12|Ahrefs).*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*(Semrush|SISTRIX|80legs|HTTrack|Xenu|dataprovider).*$ [NC]
RewriteRule .* - [F,L]

My web server is Apache 2.4 for IBM i.

d.lanza38
  • 327
  • 1
  • 5
  • 13

1 Answers1

3

Would access log entries still be created if the request was blocked in the Apache config?

Yes, however, they should be logged as a 403 (Forbidden) - given the RewriteRule posted above.

Your mod_rewrite rule looks OK, except for a few minor tweaks...

^.*(icevikatam|AhrefsBot|spbot|MJ12|majestic12|Ahrefs).*$

You don't need the ^.* and .*$ anchors if you just want to match those words anywhere in the string. The above is the same as simply:

(icevikatam|AhrefsBot|spbot|MJ12|majestic12|Ahrefs)

RewriteRule .* - [F,L]

The F flag implies the L flag as well, so an explicit L flag is not required here.

MrWhite
  • 11,643
  • 4
  • 25
  • 40
  • 1
    Ah, now that I look at the status code they are all 403's. I added the `^.*` and `.*$` when I thought it wasn't working, I'll take them out now along with the `L` flag. – d.lanza38 Sep 12 '16 at 15:46