If a desicion is to be be made on selecting a mode of fail for an inline IPS that is protecting servers, what are the criteria that should be considored for selecting one of the failure modes:
1- Fail close: if the IPS fails, it will disconnect the server it is protecting.
2- Fail open: if the IPS fails, it will pass through all traffic to the server including any possible attacks.
What are the risks and benefits involved in both scenarios?
The way to look at which is better for you is to work out what your risk appetite is.
If you must have service at all costs then you don't want to fail closed, as any problem with that IPS will cause a Denial of Service. That is a very rare scenario though - the majority of implementations are configured to protect the server and the data on it.
This is where defence in depth comes into its own. You filter some traffic (eg everything but ports 80/443) using a hardware device - this gives you speed and the ability to remove the vast majority of the traffic that may impact you.
Then use a firewall to limit connections to only those you need. If it can do stateful inspection, this can be used to limit the types traffic.
Then you harden your web server, and locate it in a DMZ which is segregated from your internal network and databases by further firewalls or access controls on routers.
etc etc etc
Layering gives you the ability to cope with the failure of one component, so in your case, if your IPS fails you can make a call based on how secure your other layers are as to whether you fail open and cope with a reduced level of protection until fix, or fail closed and suffer an outage.
