Open source SuperPMI

OVERVIEW
========

This directory contains the SuperPMI tool used for testing the .NET
just-in-time (JIT) compiler.

SuperPMI has two uses:
1. Verification that a JIT code change doesn't cause any asserts.
2. Finding test code where two JIT compilers generate different code, or verifying
that the two compilers generate the same code.

Case #1 is useful for doing quick regression checking when making a source
code change to the JIT compiler. The process is: (a) make a JIT source code
change, (b) run that newly built JIT through a SuperPMI run to verify no
asserts have been introduced.

Case #2 is useful for generating assembly language diffs, to help analyze the
impact of a JIT code change.

SuperPMI works in two phases: collection and playback. In the collection phase,
the system is configured to collect SuperPMI data. Then, run any set of .NET managed
programs. When these managed programs invoke the JIT compiler, SuperPMI gathers and
captures all information passed between the JIT and its .NET host. In the
playback phase, SuperPMI loads the JIT directly, and causes it to compile all
the functions that it previously compiled, but using the collected data to
provide answers to various questions that the JIT needs to ask. The .NET
execution engine (EE) is not invoked at all.

TOOLS
==========

There are two native executable tools: superpmi and mcs. There is a .NET Core
C# program that is built as part of the coreclr repo tests build called
superpmicollect.exe.

All will show a help screen if passed -?.

COLLECTION
==========

Set the following environment variables:

SuperPMIShimLogPath=<full path to an empty temporary directory>
SuperPMIShimPath=<full path to clrjit.dll, the "standalone" JIT>
COMPlus_AltJit=*
COMPlus_AltJitName=superpmi-shim-collector.dll

(On Linux, use libclrjit.so and libsuperpmi-shim-collector.so. On Mac,
use libclrjit.dylib and libsuperpmi-shim-collector.dylib.)

Then, run some managed programs. When done running programs, un-set these variables.

Now, you will have a large number of .mc files. Merge these using the mcs
tool:

mcs -merge base.mch *.mc

One benefit of SuperPMI is the ability to remove duplicated compilations, so
on replay only unique functions are compiled. Use the following to create a
"unique" set of functions:

mcs -removeDup -thin base.mch unique.mch

Note that -thin is not required. However, it will delete all the compilation
result collected during the collection phase, which makes the resulting MCH
file smaller. Those compilation results are not required for playback.

Use the superpmicollect.exe tool to automate and simplify this process.

PLAYBACK
========

Once you have a merged, de-duplicated MCH collection, you can play it back
using:

superpmi unique.mch clrjit.dll

You can do this much faster by utilizing all the processors on your machine,
and replaying in parallel, using:

superpmi -p unique.mch clrjit.dll

REMAINING WORK
=============

The basic of assembly diffing are there, using the "coredistools"
package. The open source build needs to be altered to use this package
to wire up the correct build steps.

[tfs-changeset: 1623347]

1 files changed, 158 insertions, 0 deletions

diff --git a/src/ToolBox/superpmi/superpmi-shim-simple/iexecutionengine.cpp b/src/ToolBox/superpmi/superpmi-shim-simple/iexecutionengine.cpp
new file mode 100644
index 0000000000..a137266751
--- /dev/null
+++ b/src/ToolBox/superpmi/superpmi-shim-simple/iexecutionengine.cpp
@@ -0,0 +1,158 @@
+//
+// Copyright (c) Microsoft. All rights reserved.
+// Licensed under the MIT license. See LICENSE file in the project root for full license information.
+//
+
+#include "standardpch.h"
+#include "iexecutionengine.h"
+#include "superpmi-shim-simple.h"
+
+//***************************************************************************
+// IUnknown methods
+//***************************************************************************
+HRESULT STDMETHODCALLTYPE interceptor_IEE::QueryInterface(REFIID id, void **pInterface)
+{
+    return original_IEE->QueryInterface(id, pInterface);
+}
+ULONG STDMETHODCALLTYPE interceptor_IEE::AddRef()
+{
+    return original_IEE->AddRef();
+}
+ULONG STDMETHODCALLTYPE interceptor_IEE::Release()
+{
+    return original_IEE->Release();
+}
+
+//***************************************************************************
+// IExecutionEngine methods for TLS
+//***************************************************************************
+// Associate a callback for cleanup with a TLS slot
+VOID  STDMETHODCALLTYPE interceptor_IEE::TLS_AssociateCallback(DWORD slot, PTLS_CALLBACK_FUNCTION callback)
+{
+    original_IEE->TLS_AssociateCallback(slot, callback);
+}
+// Get the TLS block for fast Get/Set operations
+LPVOID* STDMETHODCALLTYPE interceptor_IEE::TLS_GetDataBlock()
+{
+    return original_IEE->TLS_GetDataBlock();
+}
+
+// Get the value at a slot
+LPVOID STDMETHODCALLTYPE interceptor_IEE::TLS_GetValue(DWORD slot)
+{
+    return original_IEE->TLS_GetValue(slot);
+}
+
+// Get the value at a slot, return FALSE if TLS info block doesn't exist
+BOOL STDMETHODCALLTYPE interceptor_IEE::TLS_CheckValue(DWORD slot, LPVOID * pValue)
+{
+    return original_IEE->TLS_CheckValue(slot, pValue);
+}
+// Set the value at a slot
+VOID STDMETHODCALLTYPE interceptor_IEE::TLS_SetValue(DWORD slot, LPVOID pData)
+{
+    original_IEE->TLS_SetValue(slot, pData);
+}
+// Free TLS memory block and make callback
+VOID STDMETHODCALLTYPE interceptor_IEE::TLS_ThreadDetaching()
+{
+    original_IEE->TLS_ThreadDetaching();
+}
+
+//***************************************************************************
+// IExecutionEngine methods for locking
+//***************************************************************************
+CRITSEC_COOKIE STDMETHODCALLTYPE interceptor_IEE::CreateLock(LPCSTR szTag, LPCSTR level, CrstFlags flags)
+{
+    return original_IEE->CreateLock(szTag, level, flags);
+}
+void STDMETHODCALLTYPE interceptor_IEE::DestroyLock(CRITSEC_COOKIE lock)
+{
+    original_IEE->DestroyLock(lock);
+}
+void STDMETHODCALLTYPE interceptor_IEE::AcquireLock(CRITSEC_COOKIE lock)
+{
+    original_IEE->AcquireLock(lock);
+}
+void STDMETHODCALLTYPE interceptor_IEE::ReleaseLock(CRITSEC_COOKIE lock)
+{
+    original_IEE->ReleaseLock(lock);
+}
+
+EVENT_COOKIE STDMETHODCALLTYPE interceptor_IEE::CreateAutoEvent(BOOL bInitialState)
+{
+    return original_IEE->CreateAutoEvent(bInitialState);
+}
+EVENT_COOKIE STDMETHODCALLTYPE interceptor_IEE::CreateManualEvent(BOOL bInitialState)
+{
+    return original_IEE->CreateManualEvent(bInitialState);
+}
+void STDMETHODCALLTYPE interceptor_IEE::CloseEvent(EVENT_COOKIE event)
+{
+    original_IEE->CloseEvent(event);
+}
+BOOL STDMETHODCALLTYPE interceptor_IEE::ClrSetEvent(EVENT_COOKIE event)
+{
+    return original_IEE->ClrSetEvent(event);
+}
+BOOL STDMETHODCALLTYPE interceptor_IEE::ClrResetEvent(EVENT_COOKIE event)
+{
+    return original_IEE->ClrResetEvent(event);
+}
+DWORD STDMETHODCALLTYPE interceptor_IEE::WaitForEvent(EVENT_COOKIE event, DWORD dwMilliseconds, BOOL bAlertable)
+{
+    return original_IEE->WaitForEvent(event, dwMilliseconds, bAlertable);
+}
+DWORD STDMETHODCALLTYPE interceptor_IEE::WaitForSingleObject(HANDLE handle, DWORD dwMilliseconds)
+{
+    return original_IEE->WaitForSingleObject(handle, dwMilliseconds);
+}
+SEMAPHORE_COOKIE STDMETHODCALLTYPE interceptor_IEE::ClrCreateSemaphore(DWORD dwInitial, DWORD dwMax)
+{
+    return original_IEE->ClrCreateSemaphore(dwInitial, dwMax);
+}
+void STDMETHODCALLTYPE interceptor_IEE::ClrCloseSemaphore(SEMAPHORE_COOKIE semaphore)
+{
+    original_IEE->ClrCloseSemaphore(semaphore);
+}
+DWORD STDMETHODCALLTYPE interceptor_IEE::ClrWaitForSemaphore(SEMAPHORE_COOKIE semaphore, DWORD dwMilliseconds, BOOL bAlertable)
+{
+    return original_IEE->ClrWaitForSemaphore(semaphore, dwMilliseconds, bAlertable);
+}
+BOOL STDMETHODCALLTYPE interceptor_IEE::ClrReleaseSemaphore(SEMAPHORE_COOKIE semaphore, LONG lReleaseCount, LONG *lpPreviousCount)
+{
+    return original_IEE->ClrReleaseSemaphore(semaphore, lReleaseCount, lpPreviousCount);
+}
+MUTEX_COOKIE STDMETHODCALLTYPE interceptor_IEE::ClrCreateMutex(LPSECURITY_ATTRIBUTES lpMutexAttributes,
+    BOOL bInitialOwner,
+    LPCTSTR lpName)
+{
+    return original_IEE->ClrCreateMutex(lpMutexAttributes, bInitialOwner, lpName);
+}
+void STDMETHODCALLTYPE interceptor_IEE::ClrCloseMutex(MUTEX_COOKIE mutex)
+{
+    original_IEE->ClrCloseMutex(mutex);
+}
+BOOL STDMETHODCALLTYPE interceptor_IEE::ClrReleaseMutex(MUTEX_COOKIE mutex)
+{
+    return original_IEE->ClrReleaseMutex(mutex);
+}
+DWORD STDMETHODCALLTYPE interceptor_IEE::ClrWaitForMutex(MUTEX_COOKIE mutex,
+    DWORD dwMilliseconds,
+    BOOL bAlertable)
+{
+    return original_IEE->ClrWaitForMutex(mutex, dwMilliseconds, bAlertable);
+}
+
+DWORD STDMETHODCALLTYPE interceptor_IEE::ClrSleepEx(DWORD dwMilliseconds, BOOL bAlertable)
+{
+    return original_IEE->ClrSleepEx(dwMilliseconds, bAlertable);
+}
+BOOL STDMETHODCALLTYPE interceptor_IEE::ClrAllocationDisallowed()
+{
+    return original_IEE->ClrAllocationDisallowed();
+}
+void STDMETHODCALLTYPE interceptor_IEE::GetLastThrownObjectExceptionFromThread(void **ppvException)
+{
+    original_IEE->GetLastThrownObjectExceptionFromThread(ppvException);
+}