Added support for running in a sandbox on Mac (#20735)

* Added support for running in a sandbox on Mac

When running in a sandbox, the Mac operating system will limit access to resources, esp. the file system. Right now both Mutex and SharedMemory in the PAL are accessing the /tmp folder for which Mac does not provide the application permissions to access.

Instead, the sandbox provides the ability to share information between applications by using a shared container folder. This is done by registering the application with an Application Group ID. Using this ID, we can access the shared folder and read/write from it.

Since the .Net runtime can be loaded in multiple ways, we decided that the easiest way to let the runtime know what the application group ID is via an environment variable. Thus, if the NETCOREAPP_SANDBOX_APPLICATION_GROUP_ID environment variable is set (on Mac), the runtime will assume we are sandboxed, and will use the value provided as the application group ID. Note that due to limitations on semaphore file lengths, we will not allow application group IDs longer than 13 characters. This gives us 10 characters for the developer ID, and 3 extra characters for the group name.

When sandbox is disabled (the environment variable is empty) then the folder for Mutex and SharedMemory will continue to be rooted in /tmp. However when the sandbox is enabled, these files will be created under /user/{loginname}/Library/Group Containers/{AppGroupId}/.

Fixes #20473

* Made  gApplicationContainerPath a pointer so it does not get automatically deleted by the c runtime

* Made s_runtimeTempDirectoryPath and s_sharedMemoryDirectoryPath pointers so they are not automatically deleted by the c runtime

* Renamed gApplicationContainerPath to gSharedFilesPath

* Renamed NETCOREAPP_SANDBOX_APPLICATION_GROUP_ID to DOTNET_SANDBOX_APPLICATION_GROUP_ID

* Fixed usage of VerifyStringOperation

* Replaced new with InternalNew

* Wrapped Apple specific code with #ifdef

* Added exception handling during close

* Moved VerifyStringOperation macro into SharedMemoryManager

* Moved PathCharString variable declarations before AutoCleanup is declared.

* Fixed initialization functions not to throw

* Renamed CopyPath to BuildSharedFilesPath

* Fixed misc nits

* Fixed implicit conversions from BOOL to bool

* Moved MAX_APPLICATION_GROUP_ID_LENGTH inside ifdef APPLE

* Removed PAL_IsApplicationSandboxed

1 files changed, 51 insertions, 1 deletions

diff --git a/src/pal/src/init/pal.cpp b/src/pal/src/init/pal.cpp
index dafe43df3a..294f458981 100644
--- a/src/pal/src/init/pal.cpp
+++ b/src/pal/src/init/pal.cpp
@@ -355,6 +355,52 @@ Initialize(
         gPID = getpid();
         gSID = getsid(gPID);
 
+        // The gSharedFilesPath is allocated dynamically so its destructor does not get 
+        // called unexpectedly during cleanup
+        gSharedFilesPath = InternalNew<PathCharString>();
+        if (gSharedFilesPath == nullptr)
+        {
+            SetLastError(ERROR_NOT_ENOUGH_MEMORY);
+            goto done;
+        }
+
+#ifdef __APPLE__
+        // Store application group Id. It will be null if not set
+        gApplicationGroupId = getenv("DOTNET_SANDBOX_APPLICATION_GROUP_ID");
+
+        if (nullptr != gApplicationGroupId)
+        {
+            // Verify the length of the application group ID
+            gApplicationGroupIdLength = strlen(gApplicationGroupId);
+            if (gApplicationGroupIdLength > MAX_APPLICATION_GROUP_ID_LENGTH)
+            {
+                SetLastError(ERROR_BAD_LENGTH);
+                goto done;
+            }
+
+            // In sandbox, all IPC files (locks, pipes) should be written to the application group
+            // container. There will be no write permissions to TEMP_DIRECTORY_PATH
+            if (!GetApplicationContainerFolder(*gSharedFilesPath, gApplicationGroupId, gApplicationGroupIdLength))
+            {
+                SetLastError(ERROR_NOT_ENOUGH_MEMORY);
+                goto done;
+            }
+
+            // Verify the size of the path won't exceed maximum allowed size
+            if (gSharedFilesPath->GetCount() + SHARED_MEMORY_MAX_FILE_PATH_CHAR_COUNT + 1 /* null terminator */ > MAX_LONGPATH)
+            {
+                SetLastError(ERROR_FILENAME_EXCED_RANGE);
+            }
+        }
+        else
+#endif // __APPLE__
+        {
+            gSharedFilesPath->Set(TEMP_DIRECTORY_PATH);
+
+            // We can verify statically the non sandboxed case, since the size is known during compile time
+            static_assert_no_msg(string_countof(TEMP_DIRECTORY_PATH) + SHARED_MEMORY_MAX_FILE_PATH_CHAR_COUNT + 1 /* null terminator */ <= MAX_LONGPATH);
+        }
+
         fFirstTimeInit = true;
 
         InitializeDefaultStackSize();
@@ -393,7 +439,11 @@ Initialize(
             // we use large numbers of threads or have many open files.
         }
 
-        SharedMemoryManager::StaticInitialize();
+        if (!SharedMemoryManager::StaticInitialize())
+        {
+            ERROR("Shared memory static initialization failed!\n");
+            goto CLEANUP0;
+        }
 
         /* initialize the shared memory infrastructure */
         if (!SHMInitialize())