He was able to identify the issue, even without a kernel dump.
NET team’s suspicions that this was not their problem, but ours.įortunately, one of our excellent SfB developers had spent some time developing/debugging core Windows applications. I stated the symptoms, analysis done so far by all parties, and the. So, I engaged the Skype for Business (SfB, formerly Lync) product group (PG) by opening a bug. NET team, there was still no clear cause for the crashes, and no resolution for the customer. Rats!Īfter extensive debugging by myself and the.
Lync Server code has not properly synchronized the access to the OVERLAP structure, perhaps double-freeing the As a matter of fact, they said that this was clearly a Lync Server issue. Net team found no issue with their product. Now I can just let them do the heavy lifting and fix their component! Wrong. NET bug! Microsoft Customer Service and Support (CSS) has specialists for just about every product we produce, so I engaged the. I could also see an error code associated with the first chance exception: E0434F4DSystem.NullReferenceExceptionĠx80131506 (2148734214): COR_E_EXECUTIONENGINE - corerror.h: An internal error happened in the Common Language Runtime's Execution Engine I mean, the event IDs above pointed to a. There isn’t even a Lync Server module present in this stack! The issue continued even after the customer updated to the latest.
I’m not going to go deeply into the analysis of the memory dumps, but here is the faulting call stack:Ġb clr!EEDbgInterfaceImpl::DisablePreemptiveGCĠc clr!GCHolderEEInterface::LeaveInternalĠe clr!Debugger::SendExceptionHelperAndBlockĠf clr!Debugger::SendExceptionEventsWorkerġ1 clr!Debugger::FirstChanceManagedExceptionġ2 clr!EEToDebuggerExceptionInterfaceWrapper::FirstChanceManagedExceptionġ3 clr!ExceptionTracker::ProcessManagedCallFrameġ4 clr!ExceptionTracker::ProcessOSExceptionNotificationġd mscorlib_ni!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, *)Ģ5 clr!ManagedThreadBase_FullTransitionWithADĢ9 clr!ThreadpoolMgr::CompletionPortThreadStartīased on dump analysis, the root cause was far from clear, but it seemed that the problem was with the This syntax allowed us to capture the dump at the time of the first-chance NullReferenceException, which correlates to the “Exception Info” from the event ID 1026.Īnalysis and First-Chance Exception (mine) Procdump -ma -e 1 -f NullReferenceException -accepteula DataMCUSvc.exe User-mode memory dumps were also sent in, from the crashing processes.
Report Id: c99ad6ad-a303-11e5-80f9-005056a309a8ĭescription: The process was terminated due to an unhandled exception.Įxception Info: System.NullReferenceExceptionĪt System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, *) 0, time stamp: 0x5050e3deįaulting module name: mscorlib.ni.dll, version: 9.36331, time stamp: 0x561e0e38įaulting application start time: 0x01d1367cd6e190fbįaulting application path: D:\Program Files\Microsoft Lync Server 2013\Web Conferencing\DataMCUSvc.exeįaulting module path: C:\WINDOWS\assembly\NativeImages_v9_64\mscorlib\322be87054b632752961a02ac84a27c7\mscorlib.ni.dll Message: Faulting application name: DataMCUSvc.exe, version. The customer saw an event IDs 10 in quick succession in the application log, for example:
These affected processes included the DataMCUSvc, IMMCUSvc, ASMCUSvc, and also (less frequently) RtcHost.exe. One of Microsoft’s Premier customers created a support case, because several of the application pools on multiple Lync 2013 Front End Servers would crash several times per week, thus impacting service to end users. But I hope it leads to a better understanding of how a bug actually gets fixed through our internal processes, and why at times, it may seem like progress is not as fast as we (or customers) would like. This is a high-level view of the issue, in that many technical details were left out. This is one of those issues that I just HAD to write about, because of the odd twists and turns encountered on the way to producing a fix. This means that I see some of the most interesting problems reported by customers. I am a Skype for Business Escalation Engineer. Reviewed by - Jason Epperly, Windows Escalation Engineer, Microsoft This post has been republished via RSS it originally appeared at: Skype for Business Blog articles.įirst published on TECHNET on Aug 30, 2016