SuperPMI Collect/Replay/AsmDiff tool (#21252)

This change adds superpmi.py. The tool feature three modes, collection, replay, and asmdiffs.

Collection
The collection logic is very similar to the logic in our superpmi-collect test. Mostly it just allows running a script which will run managed code and it will produce a .mch which is clean to be run against. See superpmi.md for more information on specific usage and problems.

Replay
Replay will take an existing .mch file and run the current jit over the collection. If there is no .mch file on disk, the script will download the latest collection and run against that.

AsmDiffs
superpmi.md has the latest information on what platforms support asmdiffs. So far, I have an updated OSX and Windows collection that I have run against. If there are binary diffs, the tool will automatically generate base & diff folders with the asm under each one. Future work would include automatically running jit-analyze over those locations.

In addition, the tool has the option to automatically run and diff jit dumps, I have found this to be useful to looking into diffs created, as re-running superpmi with different jits to collect this same information is somewhat tedious.

Future work
This change is in no way the end of the work needed to leverage superpmi effectively. Instead, it is a good first step. Below are some suggestions for future superpmi work:

Automated collections
Add pmi collection support
Leverage some of the new corefx work to use superpmi shim for collections of corefx runs
To be added/changed
I will unset zapdisable being set by default, it creates too much data, although it is useful it should be opt in
Will include example usage in superpmi.md.

1 files changed, 20 insertions, 2 deletions

diff --git a/tests/runtest.py b/tests/runtest.py
index 9677853e41..b25e1c89b5 100755
--- a/tests/runtest.py
+++ b/tests/runtest.py
@@ -67,7 +67,6 @@ else:
     import urllib.request
 
 sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "scripts"))
-
 from coreclr_arguments import *
 
 ################################################################################
@@ -150,6 +149,25 @@ file_name_cache = defaultdict(lambda: None)
 # Classes
 ################################################################################
 
+class TempFile:
+    def __init__(self, extension):
+        self.file = None
+        self.file_name = None
+        self.extension = extension
+
+    def __enter__(self):
+        self.file = tempfile.NamedTemporaryFile(delete=False, suffix=self.extension)
+
+        self.file_name = self.file.name
+
+        return self.file_name
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        try:
+            os.remove(self.file_name)
+        except:
+            print("Error failed to delete: {}.".format(self.file_name))
+
 class DebugEnv:
     def __init__(self, 
                  host_os, 
@@ -441,7 +459,7 @@ def create_and_use_test_env(_os, env, func):
 
     for key in env:
         value = env[key]
-        if "complus" in key.lower():
+        if "complus" in key.lower() or "superpmi" in key.lower():
             complus_vars[key] = value
 
     if len(list(complus_vars.keys())) > 0: