Recommend an intrusion detection system (IDS/IPS), and are they worth it?

Question

I have tried out various network-based IDS and IPS systems throughout the years and have never been happy with the results. Either the systems were too difficult to manage, only triggered on well-known exploits based on old signatures, or were simply too chatty with the output.

In any case, I don't feel they provided real protection for our network. In some instances, they were harmful due to dropping valid connections or just plain failing.

In the past few years, I am sure things have changed, so what are the recommended IDS systems these days? Do they have heuristics that work and don't alert on legitimate traffic?

Or, is it just better to rely on good firewalls and hardened hosts?

If you recommend a system, how do you know it's doing its job?

As some have mentioned in the answers below, let's also get some feedback on host intrusion detection systems as they are closely related to network-based IDS.

For our current setup, we would need to monitor two separate networks with a total bandwidth of 50mbps. I am looking for some real-world feedback here, not a list of devices or services capable doing IDS.

Answer 1

One thought; you ask "are they worth it". I hate to give a non technical answer, but if your organization needs to have an IDS to indicate to a regulatory body that you are in compliance with some regulation or other, even if you find that from a technology perspective the device doesn't give you what you want, they may be by definition "worth it" if they keep you in compliance.

I'm not suggesting that "it doesn't matter if its good or not", obviously something that does a good job is preferred to something that doesn't; but reaching regulatory compliance is a goal in itself.

Answer 2

Intrusion detection systems are invaluable tools, but they need to be used properly. If you treat your NIDS as an alert-based system, where the alert is the end, you will get frustrated (ok, alert X was generated, what do I do now?).

I recommend looking at the NSM (Network security monitoring) approach where you mix NIDS (alerting systems) with session and content data, so you can properly examine any alert and better tune your IDS system.

*I can't link, so just google for taosecurity or NSM

In addition to the network-based information, if you mix HIDS + LIDS (log-based intrusion detection) you will get a clear view of what is going on.

**Plus, don't forget that these tools are not meant the protect you from an attack, but to act as a security camera (physical comparison) so proper incident response can be taken.

Answer 3

To have a good IDS, you need multiple sources. If an IDS has multiple alerts from multiple sources for the same attack, it will be able to fire an alert that has a whole lot more meaning then just a standard alert.

This is why you need to correlate output from HIDS (Host IDS) such as OSSEC and NIDS (Network IDS) such as Snort. This can be done using Prelude for example. Prelude will agregate and correlate alerts to be able to generate real security warnings that have a lot more meaning. Say per example you have a network attack, if it stays a network attack, it's probably nothing too bad but if it becomes a host attack, that wil trigger appropriate alerts with a high level of importance.

Answer 4

Several years ago I reviewed several intrusion prevention systems .

I wanted to deploy something between a couple of locations and the corporate network.
The system was to provide an easy to manage and monitor (something that could be handed off to a second tier help desk person). Automated alarming and reporting were also needed.

The system that I ended up choosing was the IPS from Tipping Point. We still like it after being in place for several years. Our implementation includes the subscription to their Digital Vaccine, which pushes out vulnerability and exploit rules weekly.

The system has been very useful to watch what is going on (alert but take no action) as well as automatically block or quarantine systems.

This ended up being a very useful tool for locating and isolating malware infected computers as well as blocking bandwidth hogging or security policy related traffic without having to work with router access control lists.

http://www.tippingpoint.com/products_ips.html

Answer 5

In my opinion, off-the-shelf IDS/IPS is not worth it unless you know the exact nature of all the activity that should be seen on your network. You can drive yourself nuts creating exceptions for stupid user behavior and misbehaving (legitimate) applications. On networks that aren't highly locked down, I've found the noise to be overwhelming in any of the systems I've used. That's why we eventually piped the backbone into a single linux machine that ran a custom piece of C code. That one piece of code encapsulated all the weirdnesses we knew about, and anything else was suspect.

If you do have a highly locked down network, the best systems will have some sort of integration with your perimeter device, so that there's complete policy match.

As far as knowing whether it's doing its job, the best way is to execute some attacks yourself periodically.

Answer 6

I think any IDS/IPS system has to be custom tuned to your environment to see any real benefits. Otherwise you just get flooded with false positives. But IDS/IPS will never replace proper firewalls and server hardening.

We've been using a Fortigate unit where I work for the past year and have been really happy with it. It does a lot more than just IDS/IPS so it may not be exactly what you're looking for but it's worth a look.

The IDS/IPS rules are updated automatically (default) or can be updated manually. I find that it's IDS/IPS rules are pretty manageable as well via it's web interface. I think it's ease of management is due to breaking down the protection into protection profiles which you then assign to rules on the firewall. So rather than looking at all the rules on every packet on the network you get much more focused protection and alerts.

Answer 7

At our organization we have a number of IDSes currently in place, including a mix of commercial systems and open. This is due in part to the type of historical considerations that happen at a university, and performance reasons. That being said, I'm going to talk about Snort for a little bit.

I have been rolling out an enterprise wide snort sensor disbursal for some time now. This is a smallish sized array currently (think <10), scoped to reach a couple of dozen. What I have learned going through this process has been invaluable; principally with techniques to manage both the number of alerts coming through as well as managing this many highly distributed nodes. Using MRTG as a guide, we have sensors seeing an average of 5Mbps up to 96MBps. Keep in mind that for the purposes of this answer I'm talking about IDS, not IDP.

The major findings are:

1. Snort is a very fully featured IDS and easily holds its own w.r.t. feature set to much larger and unnamed network appliance vendors.
2. The most interesting alerts come from the Emerging Threats project.
3. WSUS results in a stupidly large number of false positives, largely from the sfPortscan preprocessor.
4. Any more than 2/3 sensors requires a good configuration and patch management system.
5. Expect to see a very large number of false positives until aggressive tuning is performed.
6. BASE does not scale very well with a large number of alerts, and snort has no built in alert management system.

To be fair to snort, I have noticed 5 in a large number of systems, including Juniper and Cisco. I have also been told stories of how Snort can be installed and configured easier than TippingPoint, though I have never used that product.

All in all, I have been very happy with Snort. I largely preferred to turn on most rules, and spend my time tuning rather than going through thousands of rules and deciding which ones to turn on. This made the time spent tuning a little higher, but I planned for it from the outset. Also, as this project was ramping up we also went through a SEIM purchase, which made it easy to coordinate the two. So I have managed to leverage good log correlation and aggregation during the tuning process. If you have no such product, your experience tuning may be different.

Answer 8

I know lots of people will throw out snort as a solution, and it is good -- snort and sguil are a good combination for monitoring different subnets or VLANs, too.

We currently use Strataguard from StillSecure, it's a snort implementation on a hardened GNU/Linux distro. It's very easy to get up and running (much easier than snort alone), has a free version for lower-bandwidth environments, and a very intuitive and useful web interface. It makes it reasonably easy to update, tune, modify, and research rules.

While it can be installed in IPS mode and automatically lock down the firewall for you, we use it in IDS mode only -- installed it on the monitor port on our central switch, popped a second NIC in for management, and it's worked great for scrutinizing traffic. The number of false positives (espeically pre-tuning) is the only downside, but this does let us know it's working, and the interface makes it very easy to examine the rule signature, inspect the captured packets, and follow links to research the vulnerability so one can decide if the alert is truly a problem or not and adjust the alert or rule as necessary.

Answer 9

Sourcefire has a good system and they have components that help discover when new unexpected traffic starts emanating from a system. We run it in IDS mode rather than IPS mode because there are issues where legitimate traffic might be blocked, so we monitor the reports and overall it seems to do a pretty decent job.

Answer 10

Well before you can answer what IDS/IPS you need I would want to better understand your security architecture. What do you use to route and switch your network, what other security measures do you have in your security architecture?

What are the risks you are trying to mitigate, ie what information assets are at risk and from what?

Your question is too generic to give you anything but, what people think of product X and its the best for X reasons.

Security is a risk mitigation process and the implementation of IT security solutions needs to be inline with the identified risks. Just throwing IDS/IPS into your network based on what people think is the best product, is unproductive and a waste of time and money.

Cheers Shane

Answer 11

Snort combined with ACID/BASE for reporting, is pretty slick for an OSS product. I'd try that, at least to get your feet wet.

Answer 12

Intrusion detection systems are more than just a NIDS (network-based one). I find that for my environment, a HIDS is much more useful. Currently I am using OSSEC, which monitors my logs, files, etc.

So, if you are not getting enough value of Snort, try a different approach. Maybe modsecurity for apache or ossec for log analysis.

Answer 13

I would recommend Snort. Snort is supported by almost all other security tools, tutorials are readily available, and so are many front-end applications. There's no secret sauce, that makes one IDS better than another. The public and local rule sets provide the power.

But any IDS (HIDS or NIDS) is a waste of money unless you are willing to check the logs and alerts, hourly or daily. You need the time and the personnel to remove false positives and create new rules for local anomalies. An IDS is best described as a video camera for your network. Someone needs to be watching it, and have the authority to act upon the information it sends. Otherwise it's worthless.

Bottom line. Save money on software, use an open source IDS. Spend money on training, and develop a great security team.

Answer 14

Frankly, IDS is usually a total waste of time as the operators spend all their time tuning out the false positives. It becomes such a burden that the system is left in a corner and ignored.

Most organisations place the probe on the outside of the network, and are astonished to see thousands of attacks. It's like putting a burglar alarm on the outside of the house and being suprised that it goes off everytime someone walks by.

IDS is loved by security consultants to show how dangerous it is out there, auditors as a tick box, and ignored by everyone else as it is a complete waste of their time and resources.

The time would be better spent accepting that there are thousands of attacks every day, designing external access, and most of all making sure that the external facing systems are properly hardened.

Dave

Answer 15

When people ask for intrusion detection, I think of server IDSs as it doesn't matter who penetrates your network if they don't do anything once in. A IDS like AIDE will make snapshot hashes of a server allowing you to see exactly what has changed on disk over a certain period.

Some people prefer to reimage all their servers after a security breach, but I think can be a little overkill for most issues.