Implement an allocator for executable (JIT) memory in PAL

This change improves performance of CoreCLR with Server GC enabled by
about 30% according to ASP.NET benchmarks on Linux. The table below shows
number of requests per second that an ASP.NET benchmark could handle on my
machine before and after the change.

Pipeline        Before          After           Improvement
16              230K Req/sec    305K Req/sec    33%
256             240K Req/sec    340K Req/sec    42%

The problem was that with Server GC enabled, the GC initialization code was
reserving a large chunck (about 18GB on my machine) of virtual address
space during runtime initialization. Unfortunately, due to implementation
details of MM on Linux, GC memory was located next to the location of
libcoreclr. As a result, the runtime could not allocate memory for JIT'ed
code close to the coreclr library. Because of that the JIT'ed code had to
use jump stubs to call functions from the runtime (which can become very
expensive, for example, for write barriers).

This change fixes this issue by implementing a simple allocator that tries
to reserve (during process startup) a chuck of virtual memory that is
located near the coreclr library (within 2GB range) that can be later used
for JIT'ed code.

1 files changed, 7 insertions, 0 deletions

diff --git a/src/utilcode/util.cpp b/src/utilcode/util.cpp
index 062cf61d95..d7d3a9f4cb 100644
--- a/src/utilcode/util.cpp
+++ b/src/utilcode/util.cpp
@@ -526,6 +526,13 @@ BYTE * ClrVirtualAllocExecutable(SIZE_T dwSize,
     // Fall through to 
 #endif // USE_UPPER_ADDRESS
 
+#ifdef FEATURE_PAL
+    // Tell PAL to use the executable memory allocator to satisfy this request for virtual memory.
+    // This will allow us to place JIT'ed code close to the coreclr library
+    // and thus improve performance by avoiding jump stubs in managed code.
+    flAllocationType |= MEM_RESERVE_EXECUTABLE;
+#endif // FEATURE_PAL
+
     return (BYTE *) ClrVirtualAlloc (NULL, dwSize, flAllocationType, flProtect);
 
 }