1

Following up on this question as the answers are now 5 years old: Can I detect web app attacks by viewing my Apache log file?

My boss has tasked me with analyzing our access.log and error.log files after an attempted mySQL injection attack last week. It's pretty obvious when viewing the logs by hand, but we'd like something automated (either a service or a task that can be run regularly through cron) that will detect attack patterns.

We're using nginx, but that shouldn't matter because the logs are in standard format. Any suggestions on programs that do this type of log analysis? I don't care about standard traffic analysis that programs such as Webalyzer do.

Further, for those of you who do this type of attack detection and analysis, what patterns are you looking for in your log files other than just a larger than normal amount of traffic?

Also, do you look at both the access.log and error.log or just one?

Community
  • 1
Rick Chatham
  • 234
  • 1
  • 13
  • 1
    Product recommendations tend to get closed. Is there a way to re-phrase this question so that you can get the help you need without a product recommendation? – schroeder Mar 11 '15 at 17:00
  • I guess I could remove the word 'programs' but I'm not sure how useful that is. I just want to know how other people do this, as I'm not the only one, I'm sure, who's being asked for this. – Rick Chatham Mar 11 '15 at 18:29
  • 1
    By "standard format" do you mean SysLog format? – armani Mar 11 '15 at 18:46
  • The accepted answer for the other question highlights a tool that will parse the logs for many different types of attacks, including SQLi. – schroeder Mar 11 '15 at 19:07
  • @schroeder Thanks for the recommendations. The reason I asked this question again was that the other one was already closed and is 5 years old. I have a sneaking suspicion that newer/better programs have been developed since then. I've been able to find lots of web traffic analyzers, but I don't care about the marketing data. I need to know how to parse my logs for common attack types. – Rick Chatham Mar 12 '15 at 18:36
  • @armani I think we're using the NCSA Combined Log Format. – Rick Chatham Mar 12 '15 at 18:48
  • Is anyone still using Scalp? It looks like that program hasn't been updated in years. – Rick Chatham Mar 13 '15 at 00:05

1 Answers1

2

It depends if you're going to get a budget or not to do this. Welcome to the world of IDS/IPS and SIEMs.

My favorite tool that I used as an intern, and still use is OSSEC HIDS (Open Source Security Host-Based Intrusion Detection System).

It's a really sweet open source tool if you only need a small cluster of hosts monitored and don't have the funds to hire an entire SOC team or the budget for a giant, fully-supported SIEM like FireEye or ArchSight. You can pretty write your own rules (in XML) and it allows you to generate alerts based on those rules. You also set the directory paths you want monitored (i.e., var/adm/log or /var/syslog etc).

For a UI, you can use OSSEC's Web UI and host it on a server with apache. Also, you will need to know what you are looking for in order to write the rules, but there are tricks around this. You can set rules to look for the word "failed" that occurs more than 3 times in 60 seconds to generate an alert (just one example).

Hope that helps.

dtb_pen
  • 102
  • 2
  • That helps significantly. I appreciate you giving an actual answer instead of the (imho) lazy response of "this has already been answered". I added an edit to my original question because your last statement gets to the heart of things. The 'rules'. Of course I could write my own script to search for certain patterns but I'm looking to other experienced admins for direction based on attacks they've seen. In our case it was a huge amount of traffic coming from tor network exits, so we just added tor's list of known exits to our iptables. – Rick Chatham Mar 12 '15 at 18:42
  • 1
    What to look for in an attack is based on the attack itself. Now that we are talking about specific attacks, your tool-kit will get much bigger :) For example, you may want to configure your DMZ/Firewall/Web-application firewall to deny tor packets. You can use sandbox url's to checksum md5 hashes if a file is malicious. If you want to investigate tcp packets, you may want wireshark for pcap analysis. etc. etc. Raw syslogs only tell you half the story. I would say solid layer 7 security is #1. Then security awareness for company employees is #2. Take care of Layer 3 and Layer 4 next. – dtb_pen Mar 12 '15 at 19:30
  • 1
    Whoops - comment got cut off by the limit. In my experience, DDoS attacks happen the most frequently or someone doing nmap scans/flooding with requests. I still think getting some sort of open source IDS is key here. You'll have full network and host visibility based on what rules you write (not only patterns, but you can do checksums, monitor changes to critical directories, be notified when root access is gained, etc). – dtb_pen Mar 12 '15 at 19:36
  • I was going to recommend a SIEM, but you really need to have significant infrastructure to make one worth it. OP might have such assets, but it wasn't indicated in the question. I work in a SIEM every day, if you go that route then best of luck! – armani Mar 12 '15 at 22:07
  • @dtb_pen I was specifically told to look at SIEM, not IDS. (Not that I disagree, but we all have to do what the boss says.) – Rick Chatham Mar 12 '15 at 22:37
  • 1
    If you deploy OSSEC HIDS w/ Splunk and a couple of other tools, you now have a SIEM :). You can get really creative with what you choose to deploy. But best of luck - glad we could all help! – dtb_pen Mar 13 '15 at 13:04