0

I have a Windows 7 VM (ESX) with Visual Studio 2012 that handles continuous integration for a large C++ project. The solution currently builds with the /MP flag, but bluescreens have occurred using IncrediBuild as well.

It usually takes a couple of hours for it to bluescreen, and I've spent weeks trying to track down the source of the issue by installing various sets of software. At one point I saw a "Intel C++ Compiler Driver" crash message, and when I hit end process it rebooted the system, but I can't be sure that was the same issue.

Some things I've tried:

  • Clean of Windows 7
  • Visual Studio with no updates
  • Visual Studio with Update 4
  • Building with Incredibuild
  • Building with MSBuild
  • Visual Studio Patch KB2781514

The crashes usually seem to involve file io and when the system is doing a second task, but it only writes a minidump file occasionally. Here is a crash from yesterday:

Bugcheck code 0000000A
Arguments fffff6fb`7dbedf68 00000000`00000002 00000000`00000000 fffff800`01f100dc

RetAddr           : Args to Child                                                           : Call Site
fffff800`01e93169 : 00000000`0000000a fffff6fb`7dbedf68 00000000`00000002 00000000`00000000 : nt!KeBugCheckEx
fffff800`01e91de0 : 00000000`00000000 00000000`00000000 00000000`00000000 fffff6fb`7dbedf68 : nt!KiBugCheckDispatch+0x69
fffff800`01f100dc : fffff880`078dbae0 fffff8a0`102a7a90 000008ed`000008ed fffff880`078da950 : nt!KiPageFault+0x260
fffff800`01e91cee : fffff880`078da6e0 fffff880`078d4001 fffff880`078da340 fffff6fb`7dbedf68 : nt! ?? ::FNODOBFM::`string'+0x432bc
fffff800`01f100dc : bb879432`bcccdb41 00000000`3f7feaec becaef2c`40a0fb2a 3f800000`42c740b2 : nt!KiPageFault+0x16e
fffff800`01e91cee : 00000000`00000000 ffff6da8`956c4a16 00000000`00000023 fffff6fb`7dbedf68 : nt! ?? ::FNODOBFM::`string'+0x432bc
fffff800`01f100dc : fffff880`078daa10 fffff800`02188610 00000000`00000001 fffff880`078daa18 : nt!KiPageFault+0x16e
fffff800`01e91cee : 00000000`00000001 fffffa81`28708b00 fffffa81`00100020 fffff6fb`7dbf5020 : nt! ?? ::FNODOBFM::`string'+0x432bc
fffff800`01f100dc : fffffa81`26cca4a0 fffff880`078db390 fffffa81`21ee87c0 fffffa81`2272f3f0 : nt!KiPageFault+0x16e
fffff800`01e91cee : 00000000`00000000 fffffa81`2272f3f0 00000000`00000000 fffff880`01e00100 : nt! ?? ::FNODOBFM::`string'+0x432bc
fffff800`01e98ff4 : fffffa81`2272f3f0 fffffa81`26cca190 fffff880`078db390 fffff880`012d9730 : nt!KiPageFault+0x16e
fffff800`01e737b7 : fffff880`01e00180 fffff880`01e0010a 00000000`00000001 00000000`00000000 : nt!KiDeferredReadyThread+0xe4
fffff800`01e8920f : fffff8a0`17af8a0a fffff880`01e00180 00000000`00000000 fffff880`012e9306 : nt!KeSetEventBoostPriority+0x137
fffff880`01250413 : fffffa81`2272f438 00000000`00000000 fffffa81`2272f3f0 fffffa81`2272f3f0 : nt!ExpReleaseResourceForThreadLite+0x2bf
fffff880`012527b4 : fffffa81`2272f3f0 00000000`00000001 00000000`00000000 00000000`00000000 : Ntfs!NtfsCleanupIrpContext+0x1d3
fffff880`012e8885 : 00000000`00000000 fffffa81`294c7360 fffff880`078db390 fffffa81`2272f3f0 : Ntfs!NtfsExtendedCompleteRequestInternal+0xd4
fffff880`012513ad : fffffa81`2272f3f0 fffffa81`26cca190 fffff880`078db390 fffffa81`00000000 : Ntfs!NtfsCommonCreate+0x215e
fffff800`01e9f878 : fffff880`078db300 fffff800`01e975d1 fffff8a0`1528af50 00000000`00000000 : Ntfs!NtfsCommonCreateCallout+0x1d
fffff880`01251b2f : fffff880`01251390 fffff880`012509c0 fffff880`078db300 fffff880`012f1100 : nt!KeExpandKernelStackAndCalloutEx+0xd8
fffff880`012e9b3c : 00000000`00000000 00000000`00000000 fffff880`078db520 fffffa81`26cca190 : Ntfs!NtfsCommonCreateOnNewStack+0x4f
fffff880`01163bcf : fffffa81`22d67030 fffffa81`26cca190 00000000`00000000 00000000`00000000 : Ntfs!NtfsFsdCreate+0x1ac
fffff880`011832b9 : fffffa81`26cca190 fffffa81`22d88010 fffffa81`26cca100 fffffa81`22b85880 : fltmgr!FltpLegacyProcessingAfterPreCallbacksCompleted+0x24f
fffff800`02193efc : 00000000`00000005 fffffa81`21ee8978 fffffa81`294c7360 00000000`00000000 : fltmgr!FltpCreate+0x2a9
fffff800`0218f878 : fffffa81`22b83cd0 fffff800`00000000 fffffa81`21ee87c0 fffffa81`00000001 : nt!IopParseDevice+0x14d3
fffff800`02190a96 : 00000000`00000000 fffffa81`21ee87c0 fffff8a0`039bf010 fffffa81`21940750 : nt!ObpLookupObjectName+0x588
fffff800`0219239c : 00000000`00000400 00000000`00000000 fffffa81`294c7301 fffff880`078db958 : nt!ObOpenObjectByName+0x306
fffff800`0217cdd8 : 00000000`0018e288 fffff8a0`00100020 00000000`0018eae0 00000000`0018e228 : nt!IopCreateFile+0x2bc
fffff800`01e92e53 : fffffa81`29a721f0 00000000`00000001 fffffa81`28708b50 fffff800`0218b7a4 : nt!NtOpenFile+0x58
00000000`778915ea : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13
00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x778915ea

Any tips on how to try and track down the cause are greatly appreciated.

Henning
  • 44
  • 6

1 Answers1

0

You will need to download the debugging symbols for Windows 7, if you haven't done so already.

Then use something like WinDbg to load the dump file and analyze the stack trace. That should give you a clue about what is the real problem.

Additionally, check the Event Logs for errors right before the system rebooted. In Event Viewer go to "Application and Services Logs > Microsoft > Windows > Diagnostics-Performance" and look for event ID 100 to determine when it started. Then go back to System/Application events and lock backwards from that time.

Giovanni Tirloni
  • 5,693
  • 3
  • 24
  • 49
  • I'm using "SRV*c:\symbols*http://msdl.microsoft.com/download/symbols" in my Symbol Search Path of WinDbg to produce the stack above. Is there anything more I need to do? I get a few "The call to LoadLibrary(X) failed" where X is ext, exts, kext, and kdexts, but it didn't seem to affect the stack unwind. I'm trying to get the system to crash with a kernel memory dump set in my System Failure settings. Edit: there are no events in the viewer that are near the crash other than services coming back online after it reboots. – Henning Aug 12 '14 at 14:51
  • Sorry, I didn't see the symbol were working just fine (forgot to scroll horizontally). It seems to be most likely a issue NTFS as you suggested. Try running a chkdsk /f on the volume to see if it finds any errors. If not, next suspect would be Vmware and so I'd try to update Vmware Tools inside the VM and possibly look for updates in ESX itself. – Giovanni Tirloni Aug 12 '14 at 15:22