Ok, our new build is having 100% cpu spikes on each server at random intervals. For long durations it make the site totally unresponsive - this will be at peak times as people in different countries log on to the site etc.
We've looked at perfmom, memory profilers, CLR profiler, sql profilers, Red gate ants profiler, tried load testing in UAT - but cannot even reproduce the problem. This could mean only thousands of users hitting the live site causes it to happen.
One pattern we did notice was that the new code - the broken build - actually uses noticably less threads.
We are also using spring for IOC - does this have a bed reputation?
To make things worse, we cannot deploy to live due to the business impact - so cannot narrow the problem down to subset of the new features we've added.
We truly are destroyed - has anyone got any battle scars that may save us a few lives?