Fix Thread.Start while debugging bug on OSX.

The OSX exception logic is running on a separate thread from the one that the exception happened. The CatchHardwareExceptionHolder::IsEnabled used to check for h/w exception holders assumed it was running on the thread the exception/holders happened not the exception notification thread. Moved the h/w exception holder count to the CPalThread object instead of a TLS thread variable so the OSX exception code can check it given a CPalThread instance.

The main problem was that the stubmgr when creating a thread (for the start notification) put a breakpoint in ThePreStubPatch which is in the coreclr text section and because the h/w exception holder check was broken, it thought the breakpoint wasn't the debugger's and aborted the coreclr process.  The other part of this fix is to put a h/w exception holder around the called to ThePreStubPatch in prestub.cpp.

The stubmgr's delegate invoke subclass used the wrong registers for Linux/OSX to read the "this" and parameter registers. Added the proper ifdefs and registers (ecx -> rdi, rdx -> rsi) for the Unix platforms.

On both Linux and OSX, the h/w exception holder check in the exception routing logic needed to check if the int3/trap is in DBG_DebugBreak so asserts always abort/core dump. Move DBG_DebugBreak to an assembly worker so the start/end address can be used for this check.

1 files changed, 3 insertions, 3 deletions

diff --git a/src/vm/stubmgr.cpp b/src/vm/stubmgr.cpp
index f1b07b2e0c..f217216c53 100644
--- a/src/vm/stubmgr.cpp
+++ b/src/vm/stubmgr.cpp
@@ -2269,7 +2269,7 @@ BOOL DelegateInvokeStubManager::TraceManager(Thread *thread, TraceDestination *t
     {
         LOG((LF_CORDB, LL_INFO10000, "DISM::TraceManager: isSingle\n"));
 
-        orDelegate = (DELEGATEREF)ObjectToOBJECTREF((Object*)(size_t)pContext->Rcx);  // The "this" pointer is rcx
+        orDelegate = (DELEGATEREF)ObjectToOBJECTREF(StubManagerHelpers::GetThisPtr(pContext));
 
         // _methodPtr is where we are going to next.  However, in ngen cases, we may have a shuffle thunk
         // burned into the ngen image, in which case the shuffle thunk is not added to the range list of
@@ -2293,11 +2293,11 @@ BOOL DelegateInvokeStubManager::TraceManager(Thread *thread, TraceDestination *t
         if (pStub->GetPatchOffset() != 0)
         {
             // This stub has a hidden return buffer argument.
-            orDelegate = (DELEGATEREF)ObjectToOBJECTREF((Object*)(size_t)(pContext->Rdx));
+            orDelegate = (DELEGATEREF)ObjectToOBJECTREF(StubManagerHelpers::GetSecondArg(pContext));
         }
         else
         {
-            orDelegate = (DELEGATEREF)ObjectToOBJECTREF((Object*)(size_t)(pContext->Rcx));
+            orDelegate = (DELEGATEREF)ObjectToOBJECTREF(StubManagerHelpers::GetThisPtr(pContext));
         }
     }