Trying to block Google bot and others from accessing some of my sites. Thing is I have one box that has a ton of virtual host files that do nothing more than do a proxy pass to other servers. I would like to block googlebot and would like avoid having to go to each individual .htaccess
file.
I believe you can do this in apache2.conf (httpd.conf) but I believe there might be some differences in the syntax.
For example I know that this works if I stick it in my .htaccess
:
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} googlebot [NC]
RewriteRule .* - [F,L]
</IfModule>
but it doesn't seem to work in apache2. What would the exact syntax be?
robots.txt
and .htaccess
is not an option. Yes I know this setup sounds retarded because it is. I just inherited it.