CoreFX CI Unix (#18753)

* Full Unix test support

* Remove Test helper build from build-test

Add msg prefix

Change if condition

* Change Linux URL

* Syntax error

* Fix Merge Conflict

* Address PR Feedback

* Remove duplicate generate_testhost call

* Address PR comments

* Clean up exclusion list

* Disable Client_ReadWriteCancelledToken_Throws_OperationCanceledException

* Disable all flavors of CancelledToken_Throws_OperationCanceledException

* Disable X509StoreTests.Constructor_DefaultStoreName - flaky on OSX

* Update documentation with Linux instructions and workflow

* Disable EventWaitHandleTests.Ctor_InvalidMode - failing on OSX

* Disable Server_ReadWriteCancelledToken_Throws_OperationCanceledException

11 files changed, 387 insertions, 42 deletions

diff --git a/Documentation/building/testing-with-corefx.md b/Documentation/building/testing-with-corefx.md
index 0f7beefc0a..b9a98cf015 100644
--- a/Documentation/building/testing-with-corefx.md
+++ b/Documentation/building/testing-with-corefx.md
@@ -29,42 +29,76 @@ Use the following instructions to test a change to the dotnet/coreclr repo using
 [run-corefx-tests.py](https://github.com/dotnet/coreclr/blob/master/tests/scripts/run-corefx-tests.py) will clone dotnet/corefx and run steps 2-4 above automatically.  It is primarily intended to be run by the dotnet/coreclr CI system, but it might provide a useful reference or shortcut for individuals running the tests locally.
 
 ## Using the built CoreCLR testhost 
-**These instructions are currently Windows only.**
 
 Instead of copying CoreCLR binaries you can also test your changes with an existing CoreFX build or CoreCLR's CI assemblies
 
 ### Locally-built CoreFX 
+
 Once you have finished steps 1, 2. and 4. above execute the following instructions to test your local CLR changes with the built-CoreFX changes.
 
-1. From `<coreclr_root>` run `build-test.cmd <arch> <build_type> skipmanaged` to generate the test host.
+1. From `<coreclr_root>` run
+`build-test.cmd <arch> <build_type> buildtesthostonly` 
+
+-or-
+
+`build-test.sh <arch> <build_type> generatetesthostonly` 
+
+to generate the test host.
 2. Navigate to `<corefx_root>\bin\tests\` and then the test you would like to run
 3. Run
 
 ```cmd
 <coreclr_root>\bin\<os>.<arch>.<build_type>\testhost\dotnet.exe <corefx_root>\bin\tests\<testname>\xunit.console.netcore.exe <testname>.dll
 ```
+-or-
+
+```sh
+<coreclr_root>/bin/<os>.<arch>.<build_type>/testhost/dotnet <corefx_root>/bin/tests/<testname>/xunit.console.netcore.exe <testname>.dll
+```
+
 followed by any extra command-line arguments.
 
 For example to run .NET Core Windows tests from System.Collections.Tests with an x64 Release build of CoreCLR.
 ```
-pushd C:\corefx\bin\tests\System.Collections.Tests
+cd C:\corefx\bin\tests\System.Collections.Tests
 C:\coreclr\bin\tests\Windows_NT.x64.Release\testhost\dotnet.exe .\xunit.console.netcore.exe .\System.Collections.Tests.dll -notrait category=nonnetcoretests -notrait category=nonwindowstests
 ```
 
+-or-
+
+```
+cd ~/corefx/bin/tests/System.Collections.Tests
+~/coreclr/bin/tests/Linux.x64.Release/testhost/dotnet .\xunit.console.netcore.exe .\System.Collections.Tests.dll -notrait category=nonnetcoretests -notrait category=nonlinuxtests
+```
+
 ### CI Script
-CoreCLR has an alternative way to run CoreFX tests, built for PR CI jobs. To run tests against pre-built binaries you can execute the following from the CoreCLR repo root:
+CoreCLR has an alternative way to run CoreFX tests, built for PR CI jobs. 
 
-1. `.\build.cmd <arch> <build_type>`
-2. `.\build-test.cmd <arch> <build_type> skipmanaged` - generates the test host
+To run tests against pre-built binaries you can execute the following from the CoreCLR repo root:
+
+#### Windows
+1. `.\build.cmd <arch> <build_type> skiptests`
+2. `.\build-test.cmd <arch> <build_type> buildtesthostonly` - generates the test host
 3. `.\tests\runtest.cmd <arch> <build_type> corefxtests|corefxtestsall` - runs CoreFX tests
 
+#### Linux and OSX
+1. `./build.sh <arch> <build_type> skiptests`
+2. `./build-test.sh <arch> <build_type>  generatetesthostonly`
+3. `./tests/runtest.sh  --corefxtests|--corefxtestsall --testHostDir=<path_to_testhost> --coreclr-src<path_to_coreclr>`
+
 CoreFXTests - runs all tests defined in TopN.Windows.CoreFX.issues.json or the test list specified with the argument `CoreFXTestList`
 CoreFXTestsAll - runs all tests available in the test list found at the URL in `.\coreclr\tests\CoreFX\CoreFXTestListURL.txt`.
+#### Linux - specific
+&lt;path_to_testhost&gt; - path to the coreclr test host built in step 2.
+&lt;path_to_coreclr&gt; - path to coreclr source 
 
 ### Helix Testing
 To use Helix-built binaries, substitute the URL in `.\coreclr\tests\CoreFX\CoreFXTestListURL.txt` with one acquired from a Helix test run and run the commands above.
 
-### Test List Format
+#### Workflow
+The CoreFX tests CI jobs run against cached test binaries in blob storage. This means that tests might need to be disabled until the test binaries are refreshed as breaking changes are merged in both CoreCLR and CoreFX. If you suspect a test is not failing because of a functional regression, but rather because it's stale you can add it to either the [Windows](https://github.com/dotnet/coreclr/blob/master/tests/CoreFX/TopN.CoreFX.x64.Windows.issues.json) or [Unix](https://github.com/dotnet/coreclr/blob/master/tests/CoreFX/TopN.CoreFX.x64.Unix.issues.json) test exclusion lists.
+
+#### Test List Format
 The tests defined in TopN.Windows.CoreFX.issues.json or the test list specified with the argument `CoreFXTestList` should conform to the following format -
 ```json
     {
diff --git a/build-test.sh b/build-test.sh
index b5badd09ab..ab73f979f7 100755
--- a/build-test.sh
+++ b/build-test.sh
@@ -187,8 +187,24 @@ generate_layout()
     if [ ! -f "${__BuildToolsDir}/Microsoft.CSharp.targets" ]; then
         ln -s "${__BuildToolsDir}/Microsoft.CSharp.Targets" "${__BuildToolsDir}/Microsoft.CSharp.targets"
     fi
+
 }
 
+generate_testhost()
+{
+    export TEST_HOST=$xUnitTestBinBase/testhost
+
+    if [ -d "${TEST_HOST}" ]; then
+        rm -rf $TEST_HOST
+    fi
+
+    echo "${__MsgPrefix}Creating test overlay..."    
+    mkdir -p $TEST_HOST
+
+    build_Tests_internal "Tests_Generate_TestHost" "${__ProjectDir}/tests/runtest.proj" "-testHost" "Creating test host"
+}
+
+
 build_Tests()
 {
     __TestDir=$__ProjectDir/tests
@@ -510,6 +526,7 @@ __RunTests=0
 __RebuildTests=0
 __BuildTestWrappers=0
 __GenerateLayoutOnly=
+__GenerateTestHostOnly=
 __priority1=
 CORE_ROOT=
 
@@ -641,11 +658,9 @@ while :; do
         generatelayoutonly)
             __GenerateLayoutOnly=1
             ;;
-
         generatetesthostonly)
-            exit 0
+            __GenerateTestHostOnly=1
             ;;
-
         buildagainstpackages)
             __BuildAgainstPackagesArg=1
             ;;
@@ -745,15 +760,22 @@ fi
 # Specify path to be set for CMAKE_INSTALL_PREFIX.
 # This is where all built CoreClr libraries will copied to.
 export __CMakeBinDir="$__BinDir"
+if [ [ ! -d "$__BinDir" ] || [ ! -d "$__BinDir/bin" ] ]; then
+    if [ [ -z "$__GenerateLayoutOnly" ] && [ -z "$__GenerateTestHostOnly" ] ]; then
 
-if [ ! -d "$__BinDir" ] || [ ! -d "$__BinDir/bin" ]; then
-
-    echo "Cannot find build directory for the CoreCLR Product or native tests."
-    echo "Please make sure CoreCLR and native tests are built before building managed tests."
-    echo "Example use: './build.sh $__BuildArch $__BuildType' without -skiptests switch"
+        echo "Cannot find build directory for the CoreCLR native tests."
+        echo "Please make sure native tests are built before building managed tests."
+        echo "Example use: './build.sh $__BuildArch $__BuildType' without -skiptests switch"
+    else
+        echo "Cannot find build directory for the CoreCLR Product."
+        echo "Please make sure CoreCLR and native tests are built before building managed tests."
+        echo "Example use: './build.sh $__BuildArch $__BuildType' "
+        fi
     exit 1
 fi
 
+
+
 # Configure environment if we are doing a cross compile.
 if [ $__CrossBuild == 1 ]; then
     export CROSSCOMPILE=1
@@ -770,12 +792,17 @@ initTargetDistroRid
 __CoreClrVersion=1.1.0
 __sharedFxDir=$__BuildToolsDir/dotnetcli/shared/Microsoft.NETCore.App/$__CoreClrVersion/
 
-echo "Building Tests..."
 
-if [ -z "$__GenerateLayoutOnly" ]; then
+if [[ (-z "$__GenerateLayoutOnly") && (-z "$__GenerateTestHostOnly") ]]; then
+    echo "Building Tests..."
     build_Tests
 else
+    echo "Generating test layout..."
     generate_layout
+    if [ ! -z "$__GenerateTestHostOnly" ]; then
+        echo "Generating test host..."        
+        generate_testhost
+    fi
 fi
 
 if [ $? -ne 0 ]; then
diff --git a/tests/CoreFX/CoreFXTestListURL_Linux.txt b/tests/CoreFX/CoreFXTestListURL_Linux.txt
index e7b7c744e7..29b8c2615b 100644
--- a/tests/CoreFX/CoreFXTestListURL_Linux.txt
+++ b/tests/CoreFX/CoreFXTestListURL_Linux.txt
@@ -1 +1 @@
-https://cloudcijobs.blob.core.windows.net/corertci/TestList_Linux.json
\ No newline at end of file
+https://cloudcijobs.blob.core.windows.net/coreclrci/CoreFXArchives_Linux/TestList.json
\ No newline at end of file
diff --git a/tests/CoreFX/CoreFXTestListURL_OSX.txt b/tests/CoreFX/CoreFXTestListURL_OSX.txt
index e201a86154..bb4b850f68 100644
--- a/tests/CoreFX/CoreFXTestListURL_OSX.txt
+++ b/tests/CoreFX/CoreFXTestListURL_OSX.txt
@@ -1 +1 @@
-https://cloudcijobs.blob.core.windows.net/corertci/TestList_OSX.json
\ No newline at end of file
+https://cloudcijobs.blob.core.windows.net/coreclrci/CoreFXArchives_OSX/TestList.json
\ No newline at end of file
diff --git a/tests/CoreFX/TopN.CoreFX.x64.Unix.issues.json b/tests/CoreFX/TopN.CoreFX.x64.Unix.issues.json
index 7076613451..005326b887 100644
--- a/tests/CoreFX/TopN.CoreFX.x64.Unix.issues.json
+++ b/tests/CoreFX/TopN.CoreFX.x64.Unix.issues.json
@@ -1,48 +1,208 @@
-// The expected format of this file can be found at https://github.com/dotnet/coreclr/blob/master/Documentation/building/testing-with-corefx.md
 [
     {
-        "name": "System.Collections.Tests",
+        "name": "System.Diagnostics.DiagnosticSource.Tests",
+        "enabled": true,
         "exclusions": {
             "namespaces": null,
             "classes": null,
             "methods": [
                 {
-                    "name": "System.Collections.Tests.SortedList_Generic_Tests_string_string.ICollection_Generic_Remove_DefaultValueContainedInCollection",
-                    "reason": "Exclusion Sample"
+                    "name": "System.Diagnostics.Tests.DiagnosticSourceEventSourceBridgeTests.TestShortcutKeywords",
+                    "reason": "Xunit.Sdk.EqualException Assert.Equal() Failure\\n          ↓ (pos 0)\\nExpected: Activity1Start\\nActual:   Event\\n          ↑ (pos 0)"
                 }
             ]
         }
     },
     {
-        "name": "System.Runtime.Extensions.Tests",
+        "name": "System.Diagnostics.Tracing.Tests",
+        "enabled": true,
         "exclusions": {
             "namespaces": null,
             "classes": null,
-            "methods": []
+            "methods": [
+                {
+                    "name": "BasicEventSourceTests.TestsUserErrors.Test_BadEventSource_MismatchedIds",
+                    "reason": "Xunit.Sdk.EqualException Assert.Equal() Failure\\nExpected: 2\\nActual:   1"
+                }
+            ]
+        }
+    },
+    {
+        "name": "System.Drawing.Common.Tests",
+        "enabled": true,
+        "exclusions": {
+            "namespaces": null,
+            "classes": null,
+            "methods": [
+                {
+                    "name": "System.Drawing.Tests.GraphicsTests.InterpolationMode_SetInvalid_ThrowsInvalidEnumArgumentException",
+                    "reason": "Xunit.Sdk.ThrowsException Assert.Throws() Failure\\nExpected: typeof(System.ComponentModel.InvalidEnumArgumentException)\\nActual:   typeof(System.ComponentModel.InvalidEnumArgumentException): The value of argument 'value' (-2) is invalid for Enum type 'InterpolationMode'.\\nParameter name: value"
+                },
+                {
+                    "name": "System.Drawing.Tests.GraphicsTests.CompositingQuality_SetInvalid_ThrowsInvalidEnumArgumentException",
+                    "reason": "Xunit.Sdk.ThrowsException Assert.Throws() Failure\\nExpected: typeof(System.ComponentModel.InvalidEnumArgumentException)\\nActual:   typeof(System.ComponentModel.InvalidEnumArgumentException): The value of argument 'value' (-2) is invalid for Enum type 'CompositingQuality'.\\nParameter name: value"
+                },
+                {
+                    "name": "System.Drawing.Tests.GraphicsTests.PixelOffsetMode_SetInvalid_ThrowsInvalidEnumArgumentException",
+                    "reason": "Xunit.Sdk.ThrowsException Assert.Throws() Failure\\nExpected: typeof(System.ComponentModel.InvalidEnumArgumentException)\\nActual:   typeof(System.ComponentModel.InvalidEnumArgumentException): The value of argument 'value' (-2) is invalid for Enum type 'PixelOffsetMode'.\\nParameter name: value"
+                },
+                {
+                    "name": "System.Drawing.Tests.GraphicsTests.PageUnit_SetInvalid_ThrowsInvalidEnumArgumentException",
+                    "reason": "Xunit.Sdk.ThrowsException Assert.Throws() Failure\\nExpected: typeof(System.ComponentModel.InvalidEnumArgumentException)\\nActual:   typeof(System.ComponentModel.InvalidEnumArgumentException): The value of argument 'value' (-1) is invalid for Enum type 'GraphicsUnit'.\\nParameter name: value"
+                },
+                {
+                    "name": "System.Drawing.Tests.GraphicsTests.SmoothingMode_SetInvalid_ThrowsInvalidEnumArgumentException",
+                    "reason": "Xunit.Sdk.ThrowsException Assert.Throws() Failure\\nExpected: typeof(System.ComponentModel.InvalidEnumArgumentException)\\nActual:   typeof(System.ComponentModel.InvalidEnumArgumentException): The value of argument 'value' (-2) is invalid for Enum type 'SmoothingMode'.\\nParameter name: value"
+                },
+                {
+                    "name": "System.Drawing.Tests.GraphicsTests.CompositingMode_SetInvalid_ThrowsInvalidEnumArgumentException",
+                    "reason": "Xunit.Sdk.ThrowsException Assert.Throws() Failure\\nExpected: typeof(System.ComponentModel.InvalidEnumArgumentException)\\nActual:   typeof(System.ComponentModel.InvalidEnumArgumentException): The value of argument 'value' (-1) is invalid for Enum type 'CompositingMode'.\\nParameter name: value"
+                },
+                {
+                    "name": "System.Drawing.Tests.GraphicsTests.TextRenderingHint_SetInvalid_ThrowsInvalidEnumArgumentException",
+                    "reason": "Xunit.Sdk.ThrowsException Assert.Throws() Failure\\nExpected: typeof(System.ComponentModel.InvalidEnumArgumentException)\\nActual:   typeof(System.ComponentModel.InvalidEnumArgumentException): The value of argument 'value' (-1) is invalid for Enum type 'TextRenderingHint'.\\nParameter name: value"
+                }
+            ]
+        }
+    },
+    {
+        "name": "System.IO.Pipes.Tests",
+        "enabled": true,
+        "exclusions": {
+            "namespaces": null,
+            "classes": null,
+            "methods": [
+                {
+                    "name": "System.IO.Pipes.Tests.NamedPipeTest_Simple_ServerInOutRead_ClientInOutWrite.Server_ReadWriteCancelledToken_Throws_OperationCanceledException",
+                    "reason": "https://github.com/dotnet/corefx/issues/16934"
+                },
+                {
+                    "name": "System.IO.Pipes.Tests.NamedPipeTest_Simple_ServerInOutWrite_ClientInOutRead.Server_ReadWriteCancelledToken_Throws_OperationCanceledException",
+                    "reason": "https://github.com/dotnet/corefx/issues/16934"
+                },
+                {
+                    "name": "System.IO.Pipes.Tests.NamedPipeTest_Simple_ServerInOut_ClientIn.Server_ReadWriteCancelledToken_Throws_OperationCanceledException",
+                    "reason": "https://github.com/dotnet/corefx/issues/16934"
+                },
+                {
+                    "name": "System.IO.Pipes.Tests.NamedPipeTest_Simple_ServerInOut_ClientOut.Server_ReadWriteCancelledToken_Throws_OperationCanceledException",
+                    "reason": "https://github.com/dotnet/corefx/issues/16934"
+                },
+                {
+                    "name": "System.IO.Pipes.Tests.NamedPipeTest_Simple_ServerOut_ClientIn.Server_ReadWriteCancelledToken_Throws_OperationCanceledException",
+                    "reason": "https://github.com/dotnet/corefx/issues/16934"
+                },
+                {
+                    "name": "System.IO.Pipes.Tests.NamedPipeTest_Simple_ServerIn_ClientOut.Server_ReadWriteCancelledToken_Throws_OperationCanceledException",
+                    "reason": "https://github.com/dotnet/corefx/issues/16934"
+                },
+                {
+                    "name": "System.IO.Pipes.Tests.NamedPipeTest_Simple_ServerInOutRead_ClientInOutWrite.Client_ReadWriteCancelledToken_Throws_OperationCanceledException",
+                    "reason": "https://github.com/dotnet/corefx/issues/16934"
+                },
+                {
+                    "name": "System.IO.Pipes.Tests.NamedPipeTest_Simple_ServerInOutWrite_ClientInOutRead.Client_ReadWriteCancelledToken_Throws_OperationCanceledException",
+                    "reason": "https://github.com/dotnet/corefx/issues/16934"
+                },
+                {
+                    "name": "System.IO.Pipes.Tests.NamedPipeTest_Simple_ServerInOut_ClientIn.Client_ReadWriteCancelledToken_Throws_OperationCanceledException",
+                    "reason": "https://github.com/dotnet/corefx/issues/16934"
+                },
+                {
+                    "name": "System.IO.Pipes.Tests.NamedPipeTest_Simple_ServerInOut_ClientOut.Client_ReadWriteCancelledToken_Throws_OperationCanceledException",
+                    "reason": "https://github.com/dotnet/corefx/issues/16934"
+                },
+                {
+                    "name": "System.IO.Pipes.Tests.NamedPipeTest_Simple_ServerOut_ClientIn.Client_ReadWriteCancelledToken_Throws_OperationCanceledException",
+                    "reason": "https://github.com/dotnet/corefx/issues/16934"
+                },
+                {
+                    "name": "System.IO.Pipes.Tests.NamedPipeTest_Simple_ServerIn_ClientOut.Client_ReadWriteCancelledToken_Throws_OperationCanceledException",
+                    "reason": "https://github.com/dotnet/corefx/issues/16934"
+                }
+            ]
         }
     },
     {
-        "name": "System.Runtime.Tests",
+        "name": "System.Net.Http.Functional.Tests",
+        "enabled": true,
         "exclusions": {
             "namespaces": null,
             "classes": null,
-            "methods": []
+            "methods": [
+                {
+                    "name": "System.Net.Http.Functional.Tests.PlatformHandler_DiagnosticsTest.SendAsync_ExpectedDiagnosticSourceActivityLogging",
+                    "reason": "System.Diagnostics.RemoteExecutorTestBase+RemoteInvokeHandle+RemoteExecutionException System.Diagnostics.RemoteExecutorTestBase+RemoteInvokeHandle+RemoteExecutionException : Remote process failed with an unhandled exception."
+                },
+                {
+                    "name": "System.Net.Http.Functional.Tests.SocketsHttpHandler_DiagnosticsTest.SendAsync_ExpectedDiagnosticSourceActivityLogging",
+                    "reason": "System.Diagnostics.RemoteExecutorTestBase+RemoteInvokeHandle+RemoteExecutionException System.Diagnostics.RemoteExecutorTestBase+RemoteInvokeHandle+RemoteExecutionException : Remote process failed with an unhandled exception."
+                }
+            ]
+        }
+    },
+    {
+        "name": "System.Net.HttpListener.Tests",
+        "enabled": true,
+        "exclusions": {
+            "namespaces": null,
+            "classes": null,
+            "methods": [
+                {
+                    "name": "System.Net.Tests.HttpRequestStreamTests.Read_NullBuffer_ThrowsArgumentNullException",
+                    "reason": "Xunit.Sdk.ThrowsException Assert.Throws() Failure\\nExpected: typeof(System.ArgumentNullException)\\nActual:   typeof(System.NullReferenceException): Object reference not set to an instance of an object."
+                },
+                {
+                    "name": "System.Net.Tests.HttpResponseStreamTests.Write_NullBuffer_ThrowsArgumentNullException",
+                    "reason": "Xunit.Sdk.ThrowsException Assert.Throws() Failure\\nExpected: typeof(System.ArgumentNullException)\\nActual:   typeof(System.NullReferenceException): Object reference not set to an instance of an object."
+                }
+            ]
         }
     },
     {
-        "name": "System.Threading.Tasks.Tests",
+        "name": "System.Security.Cryptography.X509Certificates.Tests",
+        "enabled": true,
         "exclusions": {
             "namespaces": null,
             "classes": null,
-            "methods": []
+            "methods": [
+                {
+                    "name": "System.Security.Cryptography.X509Certificates.Tests.X509StoreTests.Constructor_DefaultStoreName",
+                    "reason": "Assert.Equal() Failure ↓ (pos 1) Expected: My Actual:   MY ↑ (pos 1)"
+                }
+            ]
         }
     },
     {
         "name": "System.Threading.Tests",
+        "enabled": true,
         "exclusions": {
             "namespaces": null,
             "classes": null,
-            "methods": []
+            "methods": [
+                {
+                    "name": "System.Threading.Tests.EventWaitHandleTests.Ctor_InvalidMode",
+                    "reason": "Assert.Equal() Failure Expected: (null) Actual:   mode"
+                }
+            ]
+        }
+    },
+    {
+        "name": "System.Xml.Xsl.XslTransformApi.Tests",
+        "enabled": true,
+        "exclusions": {
+            "namespaces": null,
+            "classes": null,
+            "methods": [
+                {
+                    "name": "System.Xml.Tests.CXmlResolverTest.TC_AbsolutePath_Transform",
+                    "reason": "Xunit.Sdk.TrueException Assert.True() Failure\\nExpected: True\\nActual:   False"
+                },
+                {
+                    "name": "System.Xml.Tests.CTransformResolverTest.TC_AbsolutePath_Transform",
+                    "reason": "Xunit.Sdk.TrueException Assert.True() Failure\\nExpected: True\\nActual:   False"
+                }
+            ]
         }
     }
 ]
\ No newline at end of file
diff --git a/tests/runtest.proj b/tests/runtest.proj
index aaff5b1bdc..e7e3bc76a2 100644
--- a/tests/runtest.proj
+++ b/tests/runtest.proj
@@ -411,14 +411,18 @@ namespace $([System.String]::Copy($(Category)).Replace(".","_").Replace("\","").
 
     <PropertyGroup Condition="'$(OSGroup)'=='Windows_NT'">
       <HostFxrFileName>hostfxr</HostFxrFileName>
-      <DotnetExecutableName>dotnet.exe</DotnetExecutableName>
       <HostFxrFileExtension>dll</HostFxrFileExtension>
+      <DotnetExecutableName>dotnet.exe</DotnetExecutableName>
+      <HostPolicyFileName>hostpolicy</HostPolicyFileName>
+      <HostPolicyExtension>dll</HostPolicyExtension>
     </PropertyGroup>
     
     <PropertyGroup Condition="'$(OSGroup)'!='Windows_NT'">
       <HostFxrFileName>libhostfxr</HostFxrFileName>
       <HostFxrFileExtension Condition="'$(OSGroup)' == 'Linux' Or '$(OSGroup)' == 'FreeBSD'">so</HostFxrFileExtension>
       <HostFxrFileExtension Condition="$(OSGroup) =='OSX'">dylib</HostFxrFileExtension>
+      <HostPolicyFileName>libhostpolicy</HostPolicyFileName>
+      <HostPolicyExtension>$(HostFxrFileExtension)</HostPolicyExtension>
       <DotnetExecutableName>dotnet</DotnetExecutableName>
     </PropertyGroup>
     
@@ -432,7 +436,7 @@ namespace $([System.String]::Copy($(Category)).Replace(".","_").Replace("\","").
       <CoreCLRBinaries Include="$(CORE_ROOT)\**\*.*" Exclude="$(CORE_ROOT)\**\@(NetCoreAppPackagedAssemblies -> '%(Identity)' )" />
       <HostFxFile Include="$(ToolsDir)\dotnetcli\**\$(HostFxrFileName).$(HostFxrFileExtension)" />
       <DotnetExe Include="$(ToolsDir)\dotnetcli\$(DotnetExecutableName)" />
-      <HostPolicyFile Include="$(ToolsDir)\dotnetcli\**\hostpolicy.dll" />
+      <HostPolicyFile Include="$(ToolsDir)\dotnetcli\**\$(HostPolicyFileName).$(HostPolicyExtension)" />
     </ItemGroup>
 
     <Copy SourceFiles="@(HostFxFile)"
diff --git a/tests/runtest.sh b/tests/runtest.sh
index 58038f35f4..474b32cbc4 100755
--- a/tests/runtest.sh
+++ b/tests/runtest.sh
@@ -70,6 +70,9 @@ function print_usage {
     echo 'CoreFX Test Options '
     echo '  --corefxtests                    : Runs CoreFX tests'
     echo '  --corefxtestsall                 : Runs all available CoreFX tests'
+    echo '  --corefxtestlist=<path>          : Runs the CoreFX tests specified in the passed list'   
+    echo '  --testHostDir=<path>             : Directory containing a built test host including core binaries, test dependencies' 
+    echo '                                     and a dotnet executable'
     echo ''
     echo 'Runtime Code Coverage options:'
     echo '  --coreclr-coverage               : Optional argument to get coreclr code coverage reports'
@@ -411,9 +414,6 @@ function create_core_overlay {
     if [ ! -d "$coreClrBinDir" ]; then
         exit_with_error "$errorSource" "Directory specified by --coreClrBinDir does not exist: $coreClrBinDir"
     fi
-    if [ -z "$coreFxBinDir" ]; then
-        exit_with_error "$errorSource" "One of --coreOverlayDir or --coreFxBinDir must be specified." "$printUsage"
-    fi
 
     # Create the overlay
     coreOverlayDir=$testRootDir/Tests/coreoverlay
@@ -441,6 +441,81 @@ function create_core_overlay {
     copy_test_native_bin_to_test_root $coreOverlayDir
 }
 
+function create_testhost
+{
+    if [ ! -d "$testHostDir" ]; then
+        exit_with_error "$errorSource" "Did not find the test host directory: $testHostDir"
+    fi
+
+    # Initialize test variables
+    local buildToolsDir=$coreClrSrc/Tools
+    local dotnetExe=$buildToolsDir/dotnetcli/dotnet
+    local coreClrSrcTestDir=$coreClrSrc/tests
+    
+    if [ -z $coreClrBinDir ]; then
+        local coreClrBinDir=${coreClrSrc}/bin
+        export __CoreFXTestDir=${coreClrSrc}/bin/tests/CoreFX
+    else
+        export __CoreFXTestDir=${coreClrBinDir}/tests/CoreFX    
+    fi
+
+    local coreFXTestSetupUtilityName=CoreFX.TestUtils.TestFileSetup
+    local coreFXTestSetupUtility="${coreClrSrcTestDir}/src/Common/CoreFX/TestFileSetup/${coreFXTestSetupUtilityName}.csproj"
+    local coreFXTestSetupUtilityOutputPath=${__CoreFXTestDir}/TestUtilities
+    local coreFXTestBinariesOutputPath=${__CoreFXTestDir}/tests_downloaded
+    
+    if [ -z $CoreFXTestList]; then
+        local CoreFXTestList="${coreClrSrcTestDir}/CoreFX/TopN.CoreFX.x64.Unix.issues.json"
+    fi
+
+    case "${OSName}" in
+        # Check if we're running under OSX        
+        Darwin)
+            local coreFXTestRemoteURL=$(<${coreClrSrcTestDir}/CoreFX/CoreFXTestListURL_OSX.txt)
+            local coreFXTestExclusionDef=nonosxtests
+        ;;        
+        # Default to Linux        
+        *)
+            local coreFXTestRemoteURL=$(<${coreClrSrcTestDir}/CoreFX/CoreFXTestListURL_Linux.txt)
+            local coreFXTestExclusionDef=nonlinuxtests
+        ;;
+    esac
+
+    local coreFXTestExecutable=xunit.console.netcore.exe
+    local coreFXLogDir=${coreClrBinDir}/Logs/CoreFX/
+    local coreFXTestExecutableArgs="--notrait category=nonnetcoreapptests --notrait category=${coreFXTestExclusionDef} --notrait category=failing --notrait category=IgnoreForCI --notrait category=OuterLoop --notrait Benchmark=true"
+
+    chmod +x ${dotnetExe}
+    resetCommandArgs=("msbuild /t:Restore ${coreFXTestSetupUtility}")
+    echo "${dotnetExe} $resetCommandArgs"
+    "${dotnetExe}" $resetCommandArgs
+
+    buildCommandArgs=("msbuild ${coreFXTestSetupUtility} /p:OutputPath=${coreFXTestSetupUtilityOutputPath} /p:Platform=${_arch} /p:Configuration=Release")
+    echo "${dotnetExe} $buildCommandArgs"
+    "${dotnetExe}" $buildCommandArgs
+    
+    if [ "${RunCoreFXTestsAll}" == "1" ]; then
+        local coreFXRunCommand=--runAllTests
+    else
+        local coreFXRunCommand=--runSpecifiedTests
+    fi
+
+    local buildTestSetupUtilArgs=("${coreFXTestSetupUtilityOutputPath}/${coreFXTestSetupUtilityName}.dll --clean --outputDirectory ${coreFXTestBinariesOutputPath} --testListJsonPath ${CoreFXTestList} ${coreFXRunCommand} --dotnetPath ${testHostDir}/dotnet --testUrl ${coreFXTestRemoteURL} --executable ${coreFXTestExecutable} --log ${coreFXLogDir} ${coreFXTestExecutableArgs}")
+    echo "${dotnetExe} $buildTestSetupUtilArgs"
+    "${dotnetExe}" $buildTestSetupUtilArgs
+
+    local exitCode=$?
+    if [ $exitCode != 0 ]; then
+        echo Running CoreFX tests finished with failures
+    else
+        echo Running CoreFX tests finished successfully
+    fi    
+    
+    echo Check ${coreFXLogDir} for test run logs
+
+    exit ${exitCode}
+}
+
 declare -a skipCrossGenFiles
 
 function is_skip_crossgen_test {
@@ -1173,10 +1248,18 @@ do
             export RunCrossGen=1
             ;;
         --corefxtests)
-            exit 0
+            export RunCoreFXTests=1
             ;;
         --corefxtestsall)
-            exit 0
+            export RunCoreFXTests=1
+            export RunCoreFXTestsAll=1
+            ;;
+        --corefxtestlist)
+            export RunCoreFXTests=1
+            export CoreFXTestList=${i#*=} 
+            ;;
+        --testHostDir=*)
+            export testHostDir=${i#*=}
             ;;
         --sequential)
             ((maxProcesses = 1))
@@ -1248,6 +1331,36 @@ fi
 
 export COMPlus_gcServer="$serverGC"
 
+if [ "$RunCoreFXTests" == 1 ];
+then 
+    if [ -z "$coreClrSrc" ]
+    then
+        echo "Coreclr src files are required to run CoreFX tests"
+        echo "Coreclr src files root path can be passed using '--coreclr-src' argument"
+        print_usage
+        exit $EXIT_CODE_EXCEPTION
+    fi
+
+    if [ -z "$testHostDir" ]; then
+        echo "--testHostDir is required to run CoreFX tests"
+        print_usage
+        exit $EXIT_CODE_EXCEPTION
+    fi
+    
+    if [ ! -f "$testHostDir/dotnet" ]; then
+        echo "Executable dotnet not found in $testHostDir"
+        exit $EXIT_CODE_EXCEPTION
+    fi
+
+    if [ ! -d "$testHostDir" ]; then
+        echo "Directory specified by --testHostDir does not exist: $testRootDir"
+        exit $EXIT_CODE_EXCEPTION
+    fi
+
+    create_testhost
+    exit 0
+fi
+
 if [ -z "$testRootDir" ]; then
     echo "--testRootDir is required."
     print_usage
diff --git a/tests/src/Common/CoreFX/CoreFX.depproj b/tests/src/Common/CoreFX/CoreFX.depproj
index d919efc22c..e64aeb7705 100644
--- a/tests/src/Common/CoreFX/CoreFX.depproj
+++ b/tests/src/Common/CoreFX/CoreFX.depproj
@@ -16,6 +16,7 @@
       <XUnitAbstractionsVersion>2.0.1</XUnitAbstractionsVersion>
       <CoreFXXUnitPackageVersion>2.2.0-beta2-build3300</CoreFXXUnitPackageVersion>
       <MicrosoftDiagnosticsTracingTraceVentVersion>2.0.19</MicrosoftDiagnosticsTracingTraceVentVersion>
+      <MicrosoftDotnetPlatformAbstractionsVersion>2.1.0</MicrosoftDotnetPlatformAbstractionsVersion>
   </PropertyGroup>
 
   <!-- Switch RuntimeIdentifier according to currently running OSGroup -->
@@ -167,6 +168,13 @@
       <Version>$(MicrosoftDiagnosticsTracingTraceVentVersion)</Version>
     </PackageReference>
   </ItemGroup>
+  
+  <ItemGroup Condition="'$(OSGroup)' != 'Windows_NT'">
+    <PackageReference Include="Microsoft.DotNet.PlatformAbstractions">
+      <Version>$(MicrosoftDotnetPlatformAbstractionsVersion)</Version>
+    </PackageReference>
+  </ItemGroup>
+  
   <ItemGroup>
     <PackageToInclude Include="@(PackageReference -> '%(Identity)' )"/>
   </ItemGroup>
diff --git a/tests/src/Common/CoreFX/TestFileSetup/CoreFX.TestUtils.TestFileSetup.csproj b/tests/src/Common/CoreFX/TestFileSetup/CoreFX.TestUtils.TestFileSetup.csproj
index 90a5cf148f..751b3ab8d3 100644
--- a/tests/src/Common/CoreFX/TestFileSetup/CoreFX.TestUtils.TestFileSetup.csproj
+++ b/tests/src/Common/CoreFX/TestFileSetup/CoreFX.TestUtils.TestFileSetup.csproj
@@ -1,6 +1,7 @@
 <Project Sdk="Microsoft.NET.Sdk">
   <Import Project="..\..\..\..\..\dependencies.props" />
   <PropertyGroup>
+    <GenerateAssemblyInfo>false</GenerateAssemblyInfo>
     <SystemCommandLineVersion>0.1.0-e160909-1</SystemCommandLineVersion>
     <NewtonsoftJsonVersion>11.0.2</NewtonsoftJsonVersion>
     <NewtonsoftJsonSchemaVersion>3.0.10</NewtonsoftJsonSchemaVersion>
diff --git a/tests/src/Common/CoreFX/TestFileSetup/Helpers/TestFileHelper.cs b/tests/src/Common/CoreFX/TestFileSetup/Helpers/TestFileHelper.cs
index c30b97dbdb..e6e2c4ed6c 100644
--- a/tests/src/Common/CoreFX/TestFileSetup/Helpers/TestFileHelper.cs
+++ b/tests/src/Common/CoreFX/TestFileSetup/Helpers/TestFileHelper.cs
@@ -248,7 +248,7 @@ namespace CoreFX.TestUtils.TestFileSetup.Helpers
                 // Check URL for validity
                 if (!Uri.IsWellFormedUriString(payloadUri, UriKind.Absolute))
                     continue;
-
+                Console.WriteLine("Downloading " + testName + " from " + payloadUri);
                 // Download tests from specified URL
                 using (var response = await HttpClient.GetStreamAsync(payloadUri))
                 {
diff --git a/tests/src/Common/CoreFX/TestFileSetup/Program.cs b/tests/src/Common/CoreFX/TestFileSetup/Program.cs
index 107f242918..599dd6d717 100644
--- a/tests/src/Common/CoreFX/TestFileSetup/Program.cs
+++ b/tests/src/Common/CoreFX/TestFileSetup/Program.cs
@@ -17,7 +17,7 @@ namespace CoreFX.TestUtils.TestFileSetup
     /// <summary>
     /// This is a driver class, which downloads archived CoreFX test assemblies from a specified URL, lays out their contents
     /// and subsequently runs them in parallel. 
-    /// This is invoked from .tests\runtests.cmd and depends on a test host (CoreCLR components with a dotnet executable) being present
+    /// This is invoked from .tests/runtests.[cmd|sh] and depends on a test host (CoreCLR components with a dotnet executable) being present
     /// </summary>
     public class Program
     {
@@ -60,10 +60,7 @@ namespace CoreFX.TestUtils.TestFileSetup
                 
                 // Only run tests if the relevant commandline switch is passed
                 if (runSpecifiedTests || runAllTests)
-                {
                     exitCode = RunTests();
-
-                }
             }
             catch (AggregateException e)
             {
@@ -95,6 +92,7 @@ namespace CoreFX.TestUtils.TestFileSetup
                     {
                         exitCode = ExitCode.UnknownError;
                     }
+                    
                     return false;
                 });
             }