Only register signals and create alt exception stack in coreclr. (#19309)

There was a couple of places where the DAC (IsValidObject, GetAppDomainForObject)
assumed that a NULL target/debuggee address would throw an exception that would
be caught by try/catch. Any other invalid address is handled with a software
exception throwed by the read memory functions. In general it is a better overall
design not to have any of the DBI/DAC, etc. code depend on hardware exceptions
being caught. On Linux the C++ runtime sometimes can't handle it. There is a
slight risk that there are other places in the DAC that make the NULL address
assumption but testing so far has found any.

Added PAL_SetInitializeDLLFlags as a fallback to allow the PAL_InitializeDLL flags
to be set for a PAL instance for the DAC where we could still register h/w signals
but not the altstack switching to reduce this risk. The flags can't be build time
conditional because we only build one coreclrpal.a library that all the modules
used. Having a PAL_InitializeFlags function doesn't really help either because of
the PAL_RegisterModule call to PAL_IntializeDLL and the LoadLibrary dance/protocol
that uses it to call the loading module's DLLMain.

Add PAL_SetInitializeFlags; remove flags from PAL_INITIALIZE and PAL_INITIALIZE_DLL
default. Add PAL_InitializeFlags() to allowing the default to be overriden.

1 files changed, 1 insertions, 0 deletions

diff --git a/src/pal/tests/palsuite/exception_handling/pal_sxs/test1/dlltest2.cpp b/src/pal/tests/palsuite/exception_handling/pal_sxs/test1/dlltest2.cpp
index 27fc213e4f..5b1d2781d1 100644
--- a/src/pal/tests/palsuite/exception_handling/pal_sxs/test1/dlltest2.cpp
+++ b/src/pal/tests/palsuite/exception_handling/pal_sxs/test1/dlltest2.cpp
@@ -17,6 +17,7 @@
 extern "C"
 int InitializeDllTest2()
 {
+    PAL_SetInitializeDLLFlags(PAL_INITIALIZE_DLL | PAL_INITIALIZE_REGISTER_SIGNALS | PAL_INITIALIZE_ENSURE_ALT_SIGNAL_STACK);
     return PAL_InitializeDLL();
 }