summaryrefslogtreecommitdiff
path: root/build-test.sh
AgeCommit message (Collapse)AuthorFilesLines
2019-06-15abort build-tests if failed to restore stress dependencies (#25140)Sergey Andreenko1-1/+5
* abort build-tests if failed to restore stress dependencies * Fix arm condition. * Add arm64 condition. * Skip runtime dependencies for arm32/arm64.
2019-06-11use nodeReuse:false in build.sh and build-test.sh.jashook1-1/+1
2019-06-03Fix armel tests build (#24678)Gleb Balykov1-0/+4
* Fix build-tests script for cross build and fix tests for armel * Handle case of -skipcrossgen for cross builds * Fix build error in case of cross builds when ROOTFS_DIR is not passed explicitly * Fix portable build parameter setup for armel tests build * Move force of non-portable build for armel from build.sh/build-test.sh to init-distro-rid.sh * Fix priority1 tests build for armel
2019-05-31Fix #24867 (#24870)Jarret Shook1-1/+1
Pre SDK 3.0, a project type that is not .csproj would not include Directory.Build.targets. Change empty.proj to empty.csproj to allow source build, which uses a 2.1 sdk to build, correctly import Directory.Build.targets.
2019-05-30Remove BuildTools from product build (#24841)Elinor Fung1-1/+2
* Remove BuildTools imports from product build * Split scripts for installing dotnet vs BuildTools
2019-05-29Use Arcade for native versioning (#24785)Elinor Fung1-7/+8
2019-05-10Properly set the nuget cache for the repo (#24505)Aaron Robinson1-1/+1
* Properly set the nuget cache for the repo * Change coreclr NuGet package cache from "./packages" to "./.packages"
2019-04-16Downgrade xUnit package version (#23944)Egor Chesakov1-2/+1
* Use the same version of xUnit that arcade uses (2.4.1-pre.build.4059)
2019-04-12Enable precompiling CoreRoot overlay with crossgen for Linux arm/arm64Sergiy Kuryata1-5/+2
2019-04-12Enable default Ready2Run testing in pull requests. Workaround an issue with ↵Egor Chesakov1-1/+2
custom xunit.console.dll runner. (#23848)
2019-04-06Add crossgen option to build-test.shSergiy Kuryata1-0/+75
This option will cause the script to call crossgen to precompile the managed assemblies in CORE_ROOT after generating layout. Most of this code has been transferred from tests/bringup_runtest.sh.
2019-03-27Add call to setup-stress-dependencies.sh to build-test.sh. (#23435)Sergey Andreenko1-0/+6
2019-03-23Revert "Add call to setup-stress-dependencies.sh to build-test.sh. (#23354)" ↵Sergey Andreenko1-4/+0
(#23419) This reverts commit 22c4ff5c1d339119c12547ac2d1b35211162df49.
2019-03-23Add call to setup-stress-dependencies.sh to build-test.sh. (#23354)Sergey Andreenko1-0/+4
* Add call to setup-stress-dependencies.sh to build-test.sh. * Fix args. * hack __DistroRid * Do not use bash source. * Do not use bash source 2. * Fix init-distro-rid permissions. * Fix tabs. * return deleted source where it was needed
2019-03-18Support gcc for build-test.shSinan Kaya1-1/+39
2019-03-17Correct current script dir detectionSinan Kaya1-1/+2
2019-03-02Use arcade's version of dotnet to build (#22755)Steve MacLean1-5/+1
* Use arcade dotnet * Add cmake_msbuild.cmd Move msbuild.cmd to cmake_msbuild.bat Document intent that this file is only used to resolve Windows cmake dependency on desktop msbuild.exe Remove one instance of msbuild.cmd * Fix inittools.cmd * Remove spurious setup_vs_tools.cmd calls
2019-02-28Re-partition MSBuild test groups (#22885)Egor Chesakov1-7/+6
This changes a tests partitioning scheme of coreclr tests in several different ways: 1. Make the test groups are of equal (to some extent) sizes keeping the number of tests in one group less than 1000; 2. As a resulf of 1) this increases a number of groups needed for Pri0 tests to 3 and decreases a number of groups for Pri1 tests to 10; 3. This also changes a way of tests partitioning scheme is defined - instead of explicitly specifying MSBuild Include-patterns this defines a boundaries between test groups in a form of _StartsWith ItemGroup. Then I use StringComparer.Ordinal to pick tests that belong to a particular group.
2019-02-26Remove buildpipelines (#22835)Steve MacLean1-11/+0
* Remove old buildpipeline * Remove obsolete publish-packages * Remove obsolete helix scripts
2019-02-21[Unix|*] Rework and share the logic for determining __DistroRid (#22628)Jarret Shook1-81/+9
* [Unix|*] Rework and share the logic for determining __DistroRid This will consolidate all of our different functions which determine the __DistroRid into one file. In addition, it will remove the unused __HostDistroRid file. Lastly, it corrects the rid choice when crossbuilding musl-arm64 as the cross-rootfs non-portable build will correctly be picked up. * Fix typo and use __BuildArch in place of __Arch * Correct non-portable build decision * Add error for missing redhat-release * Fix redhat comparison * Correct previous rhel or * Fix cross scenerio * Fix missed $ * Set __RuntimeId for runtest.proj * Correct naming convention of file * Address feedback and support -portablebuild=0 With this change: 1. build.sh and build-test.sh should build portable by default. 2. init-distro-rid.sh checks __PortableBuild=1. If so, then it must force rhel 6 and freebsd to __PortableBuild=0 and use the non-portable build - For __PortableBuild=0 init-distro-rid.sh sets the non-portable rid correctly to "${ID}.${VERSION_ID}-${buildArch}" * Correct arguments * Fix if to include then * A few more changes to correct linux-musl-arm64 * Add error for non-cross build and correctly pass ROOTFS_DIR * Correctly exit instead of echo * Correct isCrossBuild and use rootfsDir only * Add documentation * Address feedback * Rework rhel if
2019-02-05Configure MSBuild debug path and save any logsBruce Forstall1-0/+15
We have had many problems in the CI with MSBuild failing, and not capturing the failure logs. See https://github.com/dotnet/coreclr/issues/20236. So, for all msbuild invocations, set MSBUILDDEBUGPATH to a new, distinguished, subdirectory of the bin\Logs directory, named "MsbuildDebugLogs". Change the CI to upload any log files found there.
2019-02-03Update one more place to use version.c instead of version.cppFilip Navara1-3/+3
2019-01-31Remove run.exe and config.json (#21608)Sven Boemer1-23/+37
This moves us one step away from buildtools and towards arcade. This replaces run.sh and run.cmd (and all invocations) with "dotnet.sh msbuild" and "msbuild.cmd". - I'm using these wrapper scripts for now instead of those in eng/common in order to retain the very helpful "Running <command>" output in the build logs. - I'm using msbuild.cmd instead of dotnet.sh to match the current behavior that uses desktop msbuild on windows, instead of dotnet. All of the arguments that used to be implicitly generated by run.exe and config.json are now explicit, resulting in longer (but easier to copy+paste) commands. Some of these arguments are likely unnecessary, but in this change my goal is just to match the run.exe behavior. Later, I would like to go through and clean up parameters that don't need to be passed in every invocation. I might also consider moving more of the common arguments out into variables in a later change. Some of the wrapper scripts now have limited support for parsing "-Argument=Value" style parameters, to support our existing buildpipeline infrastructure, since I thought this was easier to test than changing our buildpipeline definitions. We can remove that parsing logic once we stop using buildpipeline (which has happened at this point). Some subtle parts of the change: * Add msbuild.cmd This simulates the behavior of Tools\msbuild.cmd, which calls desktop msbuild. * Fix BuildOS processing in package build and publish Previously, config.json had its own processing that would set OSName. Instead, we now pass it in explicitly where it's needed (building packages), or not at all (publishing them). * Handle "=" in publish-packages.cmd and other scripts This seems necessary to properly handle the azure access token * Set __BuildOS in PublishPackages Required for cases where the build OS isn't detected during the build such as freebsd. * Use dotnet msbuild in runtest.py This prevents us from having to deal with different quote escape behavior on windows and linux. Previously, arguments like fileloggerparameters and the logger were given quotes to escape semicolons in the argument. On unix, this prevented the argument from being split up by bash. On windows, it seems that the run.cmd/run.exe tools would prevent the extra quotes from being passed to msbuild.exe (desktop msbuild would choke on the quotes if they were passed along). Unlike desktop msbuild, dotnet msbuild is able to parse the quoted strings, so we simply psas the quoted arguments directly to it on windows. We may be able to do the same on unix. * Fix build-test.sh problem with BuildOS When copying native files during the unix test build, we rely on __BuildOS being set. Fixing the import order and always setting __BuildOS fixes this. We should eventually fix the inconsistent use of BuildOS vs __BuildOS.
2019-01-04Specify value of __RuntimeId for non-portable Linux test builds in ↵Egor Chesakov1-0/+1
build-test.sh (#21777)
2018-12-03Correct platorm reach for the Official build (#21310)Jarret Shook1-6/+1
* Correct platorm reach for the Official build Does the following: 1. Sets up a ci rule for master and a pr rule for master 2. Corrects the OSX queues 3. Corrects ubuntu internal queue 4. Converts all linux jobs to build using containers 5. Only runs official builds on: Pri1, release, bringing us to parity with old process 6. Fixes centos build-test 7. Adds a super-annoying groupname tag that avoids name mangling. * make sure we only add triggers for internal builds * Fix syntax * Missing : * Remove pr/ci * Fix internal queue name * Correct Public/internal choices. * Add crossgen as well * Correct indentation * Correct checked pri1 tests * Address pr feedback * Add todo comments
2018-11-13Simplify build.sh command line arguments for Linux/arm cross build (Part 1) ↵Egor Chesakov1-2/+0
(#20836) * Don't use crosscomponent|-crosscomponent command line args in build.sh * Don't use CROSSCOMPONENT environment variable in build.sh src/pal/tools/gen-buildsys-clang.sh * Remove mentioning -crosscomponent from usage() in build-test.sh * Don't use -crosscomponent in buildpipeline * Don't use -crosscomponent in Groovy files * Remove mentioning crosscomponent in Documentation/building/linux-instructions.md * Stop building Hostx86/armel crossgen in build.sh * Disable CrossGen-ing of System.Private.CoreLib on Linux/armel
2018-10-09Simplify next command output for build-test.sh (#19903)Bruce Forstall1-34/+41
Simplify next command output for build-test.sh
2018-10-08Runtest.py on Windows Arm(64) (#20227)Jarret Shook1-31/+44
* Initial infrastructure work to get arm(64) runtest.cmd working * Add excludes and improve runtest.py 1) Adds a build_test_wrappers only to build-test.sh. 2) Adds arm64 windows excludes 3) Adds printlastresults to runtest.cmd 4) corrects runsequential in runtest.sh 5) Corrects SequentialRun in runtest.py 6) Minor improvements to printing test results and copying native test binaries * Address pr feedback * Add to issues targets for arm64 * Working Pri 0 testing. * Remove unecessary common msbuild arguments * add pri1 excludes * Remove common msbuild args * Fix silly python3 issue
2018-09-19Enable arm64 Linux testing in CIBruce Forstall1-2/+3
Testing is enabled on a set of Qualcomm Centriq arm64 servers running Ubuntu 16.04. The set of jobs enabled almost matches the set run for arm32 Linux testing, including innerloop, JIT and GC Stress, corefx, and R2R. Temporarily, the innerloop jobs are commit jobs (invoked when a PR is merged) instead of "default trigger" jobs (invoked when a PR is submitted), until we get more experience with the robustness of the machines and jobs. The machines are fast enough that they are not marked as "limited hardware" (like arm32 Linux machines). That means that many jobs are run daily, not weekly, as periodic jobs. Notes about the changes: 1. The Linux arm64 machines are managed by Helix, which allocates them to Jenkins. 2. The arm64 OS used has been renamed from "small_page_size" to "Ubuntu16.04". If we add large page size machines, we'll need to add a differentiator. 3. The Jenkins "copy artifacts" plug-in runs ridiculously slowly on this hardware, for unknown reasons, so we copy artifacts directly using "wget". 4. Tests are built using "build-test.sh" on the (cross) build machine; we don't use Windows-built tests. 5. Added Jenkins archiving of build .log/.wrn/.err files. 6. Various tests were disabled in issues.targets, and with a new arm64/corefx_linux_test_exclusions.txt file, to get jobs to run clean. (Several issues have been opened to track these and other known failures.)
2018-09-15Layout native test components correctly on unix (#19918)Jarret Shook1-30/+25
* Enable native test build and add skipmanaged * Fix adding native components to test dir * Clean pri1/pri0 builds with native lib copy This change disables the build of several tests which have native dependencies; however, would not build the native dependencies on unix. At copy time this would result in a build-test failure. In addition, it fixes, adding executables and dynamic/static libraries to the test's location. It has been tested with build-test.sh priority1. * Fix comment wording * Remove buildagainstpackage arg * Fix default clang version * Clean pri1 run OSX * Correct test count on unix * Address typo * Merge runtest.proj
2018-08-31Fix build-test.sh wrapper build (#19779)Jarret Shook1-7/+24
* Fix build-test.sh wrapper build In addition this change creates a json file with the build info so that the wrapper build may be skipped if it was built on the same os/arch/buildtype. * Address pr feedback
2018-08-30fix merge conflict in build-test.sh (#19744)Sergey Andreenko1-1/+1
2018-08-25Use runtest.py to run tests for all platforms (#19213)Jarret Shook1-14/+10
Change build-test.sh to always build the xunit wrappers. Before it would drop a token and check the existence of the token. Unify x64 linux/OSX/Windows excludes into one file, issues.targets. Includes different locations in the file which show where to put excludes. Remove all target specific aspects of issues.targets, all tests are excluded now via wildcard, this allows expanding to .cmd and .sh based on the built platform. Unify path separators to forward slash(/) in issues.targets to support both platforms Clean up issues.targets by removing long standing exclude tests, specifically tests that have been excluded due to missing features like rva_statics. Add DisableProjectBuild to tests which have been removed from issues.targets Conditionally add DisableProjectBuild to tests which have been marked as unsupported on unix. This is mostly a port of the unsupportedOnUnix.txt list. Instead of excluding the tests, unix will simply not build them. If tests are built on windows, they will be run but they will return pass, the test wrapper will check return instantly. All exclusions ported to issues.targets for linux targets. Expand runtest.py, this includes simple issues that made it past the original CR. In addition it adds more optional features to help with inner loop dev work such as: creating a repro folder under bin/repro/.. which sets up the env and calls the failing test. In addition a launch.json will now be created under bin/repro/.. which can be used to easily debug using vscode. More logging, such as printing failures, longest running tests ect. Initial excludes ported for arm64 windows Arm64 linux, armhf unix excludes and enables running runtest.sh for these targets. arm64 windows and arm32 windows excludes and enables running runtest.cmd on arm64 targets init-tools.sh changes to pull armhf and aarch64 dotnetcli init-tools.cmd changes to pull x86 packages for dotnetcli for arm64 windows runtest.cmd for almost all scenarios will call runtest.py runtest.sh for almsot all scenarios will call runtest.py Removes all logic for running tests using runtest.sh
2018-08-23fix syntax error in build-test.shJohn Salem1-1/+1
2018-08-16Build-test.sh handles native test assets (#19430)Aaron Robinson1-43/+142
* Respect Windows script argument to skip package building. * Bring build.sh logic closer to build.cmd with respect to passing CMAKE flags for building tests * Make cmake gen script find override file without using script arguments * build-test.sh can now build native test projects Remove compileoptions.cmake and push contents into configurecompiler.cmake which contains compiler configuration from repo root CMakeLists.txt
2018-08-07Add basic check that the correct number of tests is built (#19290)Bruce Forstall1-0/+14
Fixes #19286
2018-07-19Build xunit wrappers the same way on windows and unix (#18695)Sven Boemer1-32/+39
* Initial change to allow build wrappers and runtest.py * Build xunit wrappers on unix The generated wrapper needs to target netcoreapp on unix. I had to exclude assets from the xunit package and introduce a dependency on the private corefx bits, to resolve a dependency conflict in which the generated wrapper was depending on an older System.Runtime.dll than the helper library. I also disabled binclash logging, because the wrapper build binplaces the helper library to the same location multiple times. I couldn't find a simple way to disable binclash logging for the wrapper build only, since that requires passing an empty switch to run.exe, and bash word splitting makes this nontrivial from build-test. * Correctly generate TestEnv xplat Note that this will still require changes to the test wrapper to actually source the TestEnv on unix * Build xunit wrappers using SDK * Target netcoreapp2.0 in xunit wrappers This way, the wrappers can build even if the 2.1 SDK isn't installed on the machine. * Restore to packages directory for xunit wrappers * Move common properties out to dir.common.props When building wrappers using the SDK, we need some basic properties (like the build os/arch/config, and the output directories) to be set. I factored out properties used by both the old test build and the new SDK-project test build. At first I tried using Directory.Build.props (which is automatically imported by the SDK), but our test build already imports SDK targets in various places, so this was resulting in duplicate imports. Instead, I used dir.common.props, and made the imports explicit. * Remove desktop-specific test wrapper csproj * Pass build os/arch/type and logsdir to msbuild from runtest.py * Remove xunit wrapper helper library from traversal build * Fix parameter passing in build-test.sh Use bash arrays to pass parameters for the build command. This makes it possible to pass arguments with spaces to build_Tests_internal. We use this to disable binclashlogging selectively (for the xunit wrapper build only). * Clean up factored .props files * Undo runtest.sh changes * Use latest xunit console runner everywhere * Remove extra StaticDependency on xunit.runner.console * Eliminate tests/src/dir.common.props, and rename dir.sdkbuild.props tests/src/dir.common.props was only used for the desktop-specific xunit wrapper helper library. There's no need for it any more, so its properties have been moved into tests/src/dir.props. dir.sdkbuild.props has been renamed to dir.common.props, since it contains properties used by SDK projects and buildtools projects. This change also re-enables the test build. * Reintroduce dir.sdkbuild.props as a place for SDK-only props With this, some properties shared by SDK projects can go in a global location. The TargetFramework is shared by all SDK projects in the test tree. This change also uses a property for the xunit package directory that contains the xunit.console.dll we copy to core_root. * Add xml namespace to dir.common.props This fixes a failure in the windows build. * Satisfy xunit analyzer * Satisfy xunit analyzer again * Use SDK msbuild to build wrappers On windows, the use of run.exe, config.json, and msbuild.cmd uses msbuild.exe on the path. This change will build wrappers using the local SDK via "dotnet msbuild", bypassing run.exe. Run.exe will go away entirely with the move from buildtools to arcade, so other build invocatios should follow suit. * Remove Microsoft.CSharp.Core.targets workaround UseBuildTools used to be true all the time. Now that we are building wrappers on core, UseBuildTools becomes false. However, the rest of the runtest.proj expects to build using buildtools, so we keep UseBuildTools true until we switch to arcade. The CSharpCoreTargetsPath was imported when running on core only. This used to happen only on unix, but now it also happens when building runtest.proj for the xunit wrappers on windows. On unix, this targets file was a symlink to itself to work around some buildtools logic that expected the file to exist. This workaround no longer appears necessary, and on windows, this was never used in the first place, so this change removes it. * Remove UseRoslynCompilers prop and unify roslyn import UseRoslynCompilers was introduced in buildtools by https://github.com/dotnet/buildtools/pull/947, with different behaviors on windows/unix. It was removed by https://github.com/dotnet/buildtools/pull/1974, so we can unify our roslyn imports now. * Don't copy xunit dlls to corefx test host The corefx tests run on specific versions of xunit dlls, defined in CoreFX.depproj. We want to use these versions in the test host, not those in CORE_ROOT, so exclude these from being copied to the test host directory. This fixes the failing corefx tests. * Don't pass run.exe arguments through build-test.cmd in test pipeline These arguments get passed along to the xunit wrapper build as unprocessed build args. They need to work for "dotnet msbuild" (used for the wrapper build) as well as for run.exe. * Fix parameter passing of priority arg in build-test.cmd UnprocessedBuildArgs should contain arguments in the format expected by msbuild, not by run.exe. * Fix parameter passing of unprocessed args in build-test.cmd The "--" syntax is used by run.exe to pass everything following to msbuild directly. It should not be a part of unprocessed args. * Pass TargetsWindowsArg to wrapper build in build-test.cmd Helix builds tests on windows and runs them on unix using the xunit wrappers. When cross-building the wrappers like this, TargetsWindows is set to false by the test build pipeline. This variable ensures that the wrapper uses correct directory separators when invoking the test .sh file. * Pass BuildTestsAgainstPackages arg to exclude unix tests Helix builds xunit wrappers on windows, and runs them on unix. The BuildTestsAgainstPackages should currently be set to true in the windows wrapper build to properly filter the .cmd files based on exclusions in issues.targets.
2018-07-12CoreFX CI Unix (#18753)Andon Andonov1-10/+37
* Full Unix test support * Remove Test helper build from build-test Add msg prefix Change if condition * Change Linux URL * Syntax error * Fix Merge Conflict * Address PR Feedback * Remove duplicate generate_testhost call * Address PR comments * Clean up exclusion list * Disable Client_ReadWriteCancelledToken_Throws_OperationCanceledException * Disable all flavors of CancelledToken_Throws_OperationCanceledException * Disable X509StoreTests.Constructor_DefaultStoreName - flaky on OSX * Update documentation with Linux instructions and workflow * Disable EventWaitHandleTests.Ctor_InvalidMode - failing on OSX * Disable Server_ReadWriteCancelledToken_Throws_OperationCanceledException
2018-07-09CoreFX CI Unix Staging commit (#18750)Andon Andonov1-0/+5
* Modify netci.groovy * Add script exit codes * Ad PR Triggers for Ubuntu and OSX10.12 * Remove release PR triggers * Rename arch Remove IsJitStressTestScenario assert * Add correct asserts * Reformat conditions and add triggers for Release/Checked/Debug * Change %WORKSPACE% to \${WORKSPACE}
2018-05-04Split unix test builds in slices (#17785)Steve MacLean1-17/+70
* Split unix test builds in slices Ports #17161 to linux * Address review feedback
2018-04-25Merge pull request #17747 from weshaggard/RemoveAlpineBuildWes Haggard1-5/+3
Remove Alpine 3.6 builds
2018-04-24Remove Alpine 3.6 buildsWes Haggard1-5/+3
The alpine 3.6 builds have been replaced with the more generic linux-musl builds so removing them.
2018-04-21Fix unix test build by removing unnecessary 'managed_test_build' semafore fileJacek Blaszczynski1-14/+5
Fixes #17503 The error is caused by both: 1. Unnecessary usage of 'managed_test_build' semaphore file which is incorrectly set after /t:BatchRestorePackages build target and prevents managed test build which is invoked after semaphore alredy exists 2. Masked by the above error is a wrong condition in dirs.proj non-windows test build which was introduced by PR #17161 and which prevented unix build due to missing #17161 group build port to unix
2018-03-26build-test.sh: support Clang v4.0 to v6.0, fix always false logic condition ↵Jacek Blaszczynski1-9/+24
preventing tests build (#17025)
2018-01-30Building non-Windows builds using the CLI's MSBuild (#16101)Jose Perez Rodriguez1-1/+1
* Building non-Windows builds using the CLI's MSBuild
2017-12-14Remove CoreFX runtest dependencyjashook1-52/+159
This change will start using build-test.sh generatelayoutonly to build the coreoverlay directory for use with runtest.sh.
2017-10-23Fix RID extraction in packages and test build for Alpine (#14656)Jan Vorlicek1-4/+19
* Fix RID extraction in packages and test build for Alpine The host RID extraction in build-packages.sh and build-test.sh was not matching the one in build.sh * Reflect feedback
2017-10-19Add build-test osx supportjashook1-0/+6
2017-09-26Fixed Linux test buildRoman Art1-0/+4
2017-06-07Fix FreeBSD buildTom Deseyn1-8/+14