summaryrefslogtreecommitdiff
path: root/build-test.sh
diff options
context:
space:
mode:
authorSven Boemer <sbomer@gmail.com>2018-07-19 20:44:16 (GMT)
committerGitHub <noreply@github.com>2018-07-19 20:44:16 (GMT)
commitf5f9a3456abcfdea5d886fededaa0265c80281ed (patch)
treeb15c269f2606cf0c13066d443cd13ef58d35b09a /build-test.sh
parentf26b8eea691d51332b2bd46ff32643f235c8ab14 (diff)
downloadcoreclr-f5f9a3456abcfdea5d886fededaa0265c80281ed.zip
coreclr-f5f9a3456abcfdea5d886fededaa0265c80281ed.tar.gz
coreclr-f5f9a3456abcfdea5d886fededaa0265c80281ed.tar.bz2
Build xunit wrappers the same way on windows and unix (#18695)
* Initial change to allow build wrappers and runtest.py * Build xunit wrappers on unix The generated wrapper needs to target netcoreapp on unix. I had to exclude assets from the xunit package and introduce a dependency on the private corefx bits, to resolve a dependency conflict in which the generated wrapper was depending on an older System.Runtime.dll than the helper library. I also disabled binclash logging, because the wrapper build binplaces the helper library to the same location multiple times. I couldn't find a simple way to disable binclash logging for the wrapper build only, since that requires passing an empty switch to run.exe, and bash word splitting makes this nontrivial from build-test. * Correctly generate TestEnv xplat Note that this will still require changes to the test wrapper to actually source the TestEnv on unix * Build xunit wrappers using SDK * Target netcoreapp2.0 in xunit wrappers This way, the wrappers can build even if the 2.1 SDK isn't installed on the machine. * Restore to packages directory for xunit wrappers * Move common properties out to dir.common.props When building wrappers using the SDK, we need some basic properties (like the build os/arch/config, and the output directories) to be set. I factored out properties used by both the old test build and the new SDK-project test build. At first I tried using Directory.Build.props (which is automatically imported by the SDK), but our test build already imports SDK targets in various places, so this was resulting in duplicate imports. Instead, I used dir.common.props, and made the imports explicit. * Remove desktop-specific test wrapper csproj * Pass build os/arch/type and logsdir to msbuild from runtest.py * Remove xunit wrapper helper library from traversal build * Fix parameter passing in build-test.sh Use bash arrays to pass parameters for the build command. This makes it possible to pass arguments with spaces to build_Tests_internal. We use this to disable binclashlogging selectively (for the xunit wrapper build only). * Clean up factored .props files * Undo runtest.sh changes * Use latest xunit console runner everywhere * Remove extra StaticDependency on xunit.runner.console * Eliminate tests/src/dir.common.props, and rename dir.sdkbuild.props tests/src/dir.common.props was only used for the desktop-specific xunit wrapper helper library. There's no need for it any more, so its properties have been moved into tests/src/dir.props. dir.sdkbuild.props has been renamed to dir.common.props, since it contains properties used by SDK projects and buildtools projects. This change also re-enables the test build. * Reintroduce dir.sdkbuild.props as a place for SDK-only props With this, some properties shared by SDK projects can go in a global location. The TargetFramework is shared by all SDK projects in the test tree. This change also uses a property for the xunit package directory that contains the xunit.console.dll we copy to core_root. * Add xml namespace to dir.common.props This fixes a failure in the windows build. * Satisfy xunit analyzer * Satisfy xunit analyzer again * Use SDK msbuild to build wrappers On windows, the use of run.exe, config.json, and msbuild.cmd uses msbuild.exe on the path. This change will build wrappers using the local SDK via "dotnet msbuild", bypassing run.exe. Run.exe will go away entirely with the move from buildtools to arcade, so other build invocatios should follow suit. * Remove Microsoft.CSharp.Core.targets workaround UseBuildTools used to be true all the time. Now that we are building wrappers on core, UseBuildTools becomes false. However, the rest of the runtest.proj expects to build using buildtools, so we keep UseBuildTools true until we switch to arcade. The CSharpCoreTargetsPath was imported when running on core only. This used to happen only on unix, but now it also happens when building runtest.proj for the xunit wrappers on windows. On unix, this targets file was a symlink to itself to work around some buildtools logic that expected the file to exist. This workaround no longer appears necessary, and on windows, this was never used in the first place, so this change removes it. * Remove UseRoslynCompilers prop and unify roslyn import UseRoslynCompilers was introduced in buildtools by https://github.com/dotnet/buildtools/pull/947, with different behaviors on windows/unix. It was removed by https://github.com/dotnet/buildtools/pull/1974, so we can unify our roslyn imports now. * Don't copy xunit dlls to corefx test host The corefx tests run on specific versions of xunit dlls, defined in CoreFX.depproj. We want to use these versions in the test host, not those in CORE_ROOT, so exclude these from being copied to the test host directory. This fixes the failing corefx tests. * Don't pass run.exe arguments through build-test.cmd in test pipeline These arguments get passed along to the xunit wrapper build as unprocessed build args. They need to work for "dotnet msbuild" (used for the wrapper build) as well as for run.exe. * Fix parameter passing of priority arg in build-test.cmd UnprocessedBuildArgs should contain arguments in the format expected by msbuild, not by run.exe. * Fix parameter passing of unprocessed args in build-test.cmd The "--" syntax is used by run.exe to pass everything following to msbuild directly. It should not be a part of unprocessed args. * Pass TargetsWindowsArg to wrapper build in build-test.cmd Helix builds tests on windows and runs them on unix using the xunit wrappers. When cross-building the wrappers like this, TargetsWindows is set to false by the test build pipeline. This variable ensures that the wrapper uses correct directory separators when invoking the test .sh file. * Pass BuildTestsAgainstPackages arg to exclude unix tests Helix builds xunit wrappers on windows, and runs them on unix. The BuildTestsAgainstPackages should currently be set to true in the windows wrapper build to properly filter the .cmd files based on exclusions in issues.targets.
Diffstat (limited to 'build-test.sh')
-rwxr-xr-xbuild-test.sh71
1 files changed, 39 insertions, 32 deletions
diff --git a/build-test.sh b/build-test.sh
index ab73f97..fd2d8af 100755
--- a/build-test.sh
+++ b/build-test.sh
@@ -152,7 +152,7 @@ generate_layout()
# ===
# =========================================================================================
- build_Tests_internal "Restore_Packages" "${__ProjectDir}/tests/build.proj" " -BatchRestorePackages" "Restore product binaries (build tests)"
+ build_Tests_internal "Restore_Packages" "${__ProjectDir}/tests/build.proj" "Restore product binaries (build tests)" "-BatchRestorePackages"
if [ -n "$__UpdateInvalidPackagesArg" ]; then
__up=-updateinvalidpackageversion
@@ -172,7 +172,7 @@ generate_layout()
mkdir -p $CORE_ROOT
- build_Tests_internal "Tests_Overlay_Managed" "${__ProjectDir}/tests/runtest.proj" "-testOverlay" "Creating test overlay"
+ build_Tests_internal "Tests_Overlay_Managed" "${__ProjectDir}/tests/runtest.proj" "Creating test overlay" "-testOverlay"
chmod +x $__BinDir/corerun
chmod +x $__BinDir/crossgen
@@ -180,14 +180,6 @@ generate_layout()
# Make sure to copy over the pulled down packages
cp -r $__BinDir/* $CORE_ROOT/ > /dev/null
- # Work hardcoded path around
- if [ ! -f "${__BuildToolsDir}/Microsoft.CSharp.Core.Targets" ]; then
- ln -s "${__BuildToolsDir}/Microsoft.CSharp.Core.targets" "${__BuildToolsDir}/Microsoft.CSharp.Core.Targets"
- fi
- if [ ! -f "${__BuildToolsDir}/Microsoft.CSharp.targets" ]; then
- ln -s "${__BuildToolsDir}/Microsoft.CSharp.Targets" "${__BuildToolsDir}/Microsoft.CSharp.targets"
- fi
-
}
generate_testhost()
@@ -201,7 +193,7 @@ generate_testhost()
echo "${__MsgPrefix}Creating test overlay..."
mkdir -p $TEST_HOST
- build_Tests_internal "Tests_Generate_TestHost" "${__ProjectDir}/tests/runtest.proj" "-testHost" "Creating test host"
+ build_Tests_internal "Tests_Generate_TestHost" "${__ProjectDir}/tests/runtest.proj" "Creating test host" "-testHost"
}
@@ -246,15 +238,15 @@ build_Tests()
# ===
# =========================================================================================
- build_Tests_internal "Restore_Product" "${__ProjectDir}/tests/build.proj" " -BatchRestorePackages" "Restore product binaries (build tests)"
+ build_Tests_internal "Restore_Product" "${__ProjectDir}/tests/build.proj" "Restore product binaries (build tests)" "-BatchRestorePackages"
if [ -n "$__BuildAgainstPackagesArg" ]; then
- build_Tests_internal "Tests_GenerateRuntimeLayout" "${__ProjectDir}/tests/runtest.proj" "-BinPlaceRef -BinPlaceProduct -CopyCrossgenToProduct" "Restore product binaries (run tests)"
+ build_Tests_internal "Tests_GenerateRuntimeLayout" "${__ProjectDir}/tests/runtest.proj" "Restore product binaries (run tests)" "-BinPlaceRef" "-BinPlaceProduct" "-CopyCrossgenToProduct"
fi
echo "Starting the Managed Tests Build..."
- build_Tests_internal "Tests_Managed" "$__ProjectDir/tests/build.proj" "$__up" "Managed tests build (build tests)"
+ build_Tests_internal "Tests_Managed" "$__ProjectDir/tests/build.proj" "Managed tests build (build tests)" "$__up"
if [ $? -ne 0 ]; then
echo "${__MsgPrefix}Error: build failed. Refer to the build log files for details (above)"
@@ -270,7 +262,7 @@ build_Tests()
if [ ! -f $__XUnitWrapperBuiltMarker ]; then
- build_Tests_internal "Tests_XunitWrapper" "$__ProjectDir/tests/runtest.proj" "-BuildWrappers -MsBuildEventLogging=\" \" " "Test Xunit Wrapper"
+ build_Tests_internal "Tests_XunitWrapper" "$__ProjectDir/tests/runtest.proj" "Test Xunit Wrapper" "-BuildWrappers" "-MsBuildEventLogging= " "-TargetsWindows=false"
if [ $? -ne 0 ]; then
echo "${__MsgPrefix}Error: build failed. Refer to the build log files for details (above)"
@@ -295,16 +287,19 @@ build_Tests()
if [ $__ZipTests -ne 0 ]; then
echo "${__MsgPrefix}ZIP tests packages..."
- build_Tests_internal "Helix_Prep" "$__ProjectDir/tests/helixprep.proj" " " "Prep test binaries for Helix publishing"
+ build_Tests_internal "Helix_Prep" "$__ProjectDir/tests/helixprep.proj" "Prep test binaries for Helix publishing" " "
fi
}
build_Tests_internal()
{
subDirectoryName=$1
- projectName=$2
- extraBuildParameters=$3
- stepName="$4"
+ shift
+ projectName=$1
+ shift
+ stepName="$1"
+ shift
+ extraBuildParameters=("$@")
# Set up directories and file names
__BuildLogRootName=$subDirectoryName
@@ -312,6 +307,13 @@ build_Tests_internal()
__BuildWrn="$__LogsDir/${__BuildLogRootName}.${__BuildOS}.${__BuildArch}.${__BuildType}.wrn"
__BuildErr="$__LogsDir/${__BuildLogRootName}.${__BuildOS}.${__BuildArch}.${__BuildType}.err"
+ # Use binclashlogger by default if no other logger is specified
+ if [[ "${extraBuildParameters[*]}" == *"-MsBuildEventLogging"* ]]; then
+ msbuildEventLogging=""
+ else
+ msbuildEventLogging="-MsBuildEventLogging=\"/l:BinClashLogger,Tools/Microsoft.DotNet.Build.Tasks.dll;LogFile=binclash.log\""
+ fi
+
if [[ "$subDirectoryName" == "Tests_Managed" ]]; then
# Execute msbuild managed test build in stages - workaround for excessive data retention in MSBuild ConfigCache
# See https://github.com/Microsoft/msbuild/issues/2993
@@ -337,12 +339,16 @@ build_Tests_internal()
export TestBuildSlice=$slice
# Generate build command
- buildCommand="$__ProjectRoot/run.sh build -Project=$projectName -MsBuildLog=${__msbuildLog} -MsBuildWrn=${__msbuildWrn} -MsBuildErr=${__msbuildErr} -MsBuildEventLogging=\"/l:BinClashLogger,Tools/Microsoft.DotNet.Build.Tasks.dll;LogFile=binclash.log\" $extraBuildParameters $__RunArgs $__UnprocessedBuildArgs"
+ buildArgs=("-Project=$projectName" "-MsBuildLog=${__msbuildLog}" "-MsBuildWrn=${__msbuildWrn}" "-MsBuildErr=${__msbuildErr}")
+ buildArgs+=("$msbuildEventLogging")
+ buildArgs+=("${extraBuildParameters[@]}")
+ buildArgs+=("${__RunArgs[@]}")
+ buildArgs+=("${__UnprocessedBuildArgs[@]}")
echo "Building step '$stepName' slice=$slice via $buildCommand"
# Invoke MSBuild
- eval $buildCommand
+ "$__ProjectRoot/run.sh" build "${buildArgs[@]}"
# Make sure everything is OK
if [ $? -ne 0 ]; then
@@ -362,15 +368,16 @@ build_Tests_internal()
__msbuildErr="\"/flp2:ErrorsOnly;LogFile=${__BuildErr}\""
# Generate build command
- buildCommand="$__ProjectRoot/run.sh build -Project=$projectName -MsBuildLog=${__msbuildLog} -MsBuildWrn=${__msbuildWrn} -MsBuildErr=${__msbuildErr} -MsBuildEventLogging=\"/l:BinClashLogger,Tools/Microsoft.DotNet.Build.Tasks.dll;LogFile=binclash.log\" $extraBuildParameters $__RunArgs $__UnprocessedBuildArgs"
+ buildArgs=("-Project=$projectName" "-MsBuildLog=${__msbuildLog}" "-MsBuildWrn=${__msbuildWrn}" "-MsBuildErr=${__msbuildErr}")
+ buildArgs+=("$msbuildEventLogging")
+ buildArgs+=("${extraBuildParameters[@]}")
+ buildArgs+=("${__RunArgs[@]}")
+ buildArgs+=("${__UnprocessedBuildArgs[@]}")
echo "Building step '$stepName' via $buildCommand"
# Invoke MSBuild
- eval $buildCommand
-
- # Invoke MSBuild
- # $__ProjectRoot/run.sh build -Project=$projectName -MsBuildLog="$__msbuildLog" -MsBuildWrn="$__msbuildWrn" -MsBuildErr="$__msbuildErr" $extraBuildParameters $__RunArgs $__UnprocessedBuildArgs
+ "$__ProjectRoot/run.sh" build "${buildArgs[@]}"
# Make sure everything is OK
if [ $? -ne 0 ]; then
@@ -524,7 +531,7 @@ __ZipTests=0
__NativeTestIntermediatesDir=
__RunTests=0
__RebuildTests=0
-__BuildTestWrappers=0
+__BuildTestWrappers=1
__GenerateLayoutOnly=
__GenerateTestHostOnly=
__priority1=
@@ -686,10 +693,10 @@ while :; do
;;
priority1)
__priority1=1
- __UnprocessedBuildArgs="$__UnprocessedBuildArgs -priority=1"
+ __UnprocessedBuildArgs+=("-priority=1")
;;
*)
- __UnprocessedBuildArgs="$__UnprocessedBuildArgs $1"
+ __UnprocessedBuildArgs+=("$1")
;;
esac
@@ -697,12 +704,12 @@ while :; do
done
-__RunArgs="-BuildArch=$__BuildArch -BuildType=$__BuildType -BuildOS=$__BuildOS"
+__RunArgs=("-BuildArch=$__BuildArch" "-BuildType=$__BuildType" "-BuildOS=$__BuildOS")
# Configure environment if we are doing a verbose build
if [ $__VerboseBuild == 1 ]; then
export VERBOSE=1
- __RunArgs="$__RunArgs -verbose"
+ __RunArgs+=("-verbose")
fi
# Set default clang version
@@ -747,7 +754,7 @@ __CrossgenExe="$__CrossComponentBinDir/crossgen"
isMSBuildOnNETCoreSupported
-# CI_SPECIFIC - On CI machines, $HOME may not be set. In such a case, create a subfolder and set the variable to set.
+# CI_SPECIFIC - On CI machines, $HOME may not be set. In such a case, create a subfolder and set the variable to it.
# This is needed by CLI to function.
if [ -z "$HOME" ]; then
if [ ! -d "$__ProjectDir/temp_home" ]; then