Our server hosts over a thousand sites, and some of them seem to have been hijacked by malicious scripts. These scripts run actions normally performed by a legitimate user en masse, causing severe strees on our server and often requires us to restart to clear the load. We don't have a way to find out what they are. Recently these attacks have begun to affect our daily operations. Our error log file is 70mb in size with messages similar to the following:
[timstamp] [error] [client xx.xxx.xx.xxx] File does not exist: /path/favicon.ico (File exists. This is the majority of all log entries)
[timstamp] [error] [client xxx.xxx.xx.xxx] client denied by server configuration: /path/to/cron.php (This is my TOP concern)
[timstamp] [error] [client xxx.xx.xx.xx] Directory index forbidden by Options directive: /another/path
[timstamp] [error] [client xx.xx.xxx.xxx] ALERT - canary mismatch on efree() - heap overflow detected (attacker 'xxx.xx.xxx.xxx', file '/path/to/index.php')
[timstamp] [error] [client xx.xx.xxx.xx] Directory index forbidden by Options directive: /path/to/another/file_or_folder/
[timstamp] [error] [client xxx.xx.x.xx] Invalid URI in request GET /../../ HTTP/1.1
[timstamp] [error] [client xx.xxx.xx.xx] client denied by server configuration: /path/to/another/web/file/
Invalid URI in request GET mydomain.com HTTP/1.0
Our DB log file is above 5gbs in size.
My question is, what can we do to counter these threats? Is there a way to ban IPs based on certain behavior? We are still sifting through our logs and trying to determine a course of action. Any guides, references, or tutorials that could be provided would be greatly appreciated.