summaryrefslogtreecommitdiff
path: root/tests/runtest.cmd
AgeCommit message (Collapse)AuthorFilesLines
2019-06-11Enable lab support for coreclr tests in unloadable context (#25064)Jan Vorlicek1-3/+3
2019-03-01Use arcade's version of dotnet to build (#22755)Steve MacLean1-42/+1
* Use arcade dotnet * Add cmake_msbuild.cmd Move msbuild.cmd to cmake_msbuild.bat Document intent that this file is only used to resolve Windows cmake dependency on desktop msbuild.exe Remove one instance of msbuild.cmd * Fix inittools.cmd * Remove spurious setup_vs_tools.cmd calls
2019-02-27Remove BuildTestsAgainstPackages logicRuss Keldorph1-8/+1
No longer necessary without PipeBuild logic
2019-02-25Remove buildpipelines (#22835)Steve MacLean1-28/+0
* Remove old buildpipeline * Remove obsolete publish-packages * Remove obsolete helix scripts
2019-02-15Manually update the dependencies.props and move S.P.Corelib to use ↵Tanner Gooding1-18/+7
LangVersion=8.0 (#22452) * Update BuildTools to preview1-03713-01 (master) * Updating CoreCLR to use LangVersion=8.0 * Moving the Windows scripts to default to `dotnet msbuild` for managed components * Setting UseSharedCompilation=true * Changing some additional callsites that were using msbuild to use dotnet msbuild * Revert packages.builds to use Desktop msbuild on Windows * Fixing runtest.cmd to always set DotNetCli
2019-02-11Support building with VS2019 Preview (#22525)Tanner Gooding1-13/+12
* Support building with VS2019 Preview * Fixing gen-buildsys-win to only set the architecture for the VS generator * Refactoring Dev11/147911/fpcw.cpp so that it compiles under VS2019 * Removing the remaining traces of VS2015 build support
2019-02-06Add option to run tests in unloadable context (#22332)Jan Vorlicek1-0/+8
This change adds new "runincontext" option to the tests/runtest.cmd that allows running tests inside of an unloadable AssemblyLoadContext. It also adds new property that allows tests to be marked as incompatible with running this way. All known tests that have such issue are marked in this PR too.
2019-02-04Configure MSBuild debug path and save any logsBruce Forstall1-1/+6
We have had many problems in the CI with MSBuild failing, and not capturing the failure logs. See https://github.com/dotnet/coreclr/issues/20236. So, for all msbuild invocations, set MSBUILDDEBUGPATH to a new, distinguished, subdirectory of the bin\Logs directory, named "MsbuildDebugLogs". Change the CI to upload any log files found there.
2019-01-14Large version bubble Support (#21727)Andon Andonov1-41/+47
* Preliminary Changes * Module Index Resolution * Change infoModule encoding * Change referencing module in R2R * Pre-condition Check * Virtual Method Module Resolution * Remove Workarounds and add conditional import loading * Add signature kind module override * Add ELEMENT_TYPE_MODULE_ZAPSIG * Add switch to enable large version bubble * Cleanup * Change Native header check * Add large version bubble test * Add Large Version Bubble Checks * Cleanup * Revert unnecessary check * Change EncodeMethod Version Bubble Condition * Add Large Version Bubble asserts * Cleanup * Add default argument to runtests.py * Change test PreCommands * Revert whitespace changes * Change breaking conditional check * Streamline Version Bubble test * Address PR Feedback * Address PR Feedback #2 * Remove dead code * Add crossgen-time ifdef
2019-01-04Fix Windows arm32/arm64 testing (#21809)Bruce Forstall1-12/+15
* Fix Windows arm32/arm64 testing After the conversion to runtest.py, we were not getting any JIT or GC stress mode testing: 1. The generated test env batch file didn't have a ".bat" extension, so Windows wouldn't execute it. 2. The Python generated temp file was locked, so couldn't be read. Added more logging.
2018-11-16Merge pull request #21004 from BruceForstall/UpdateRuntestHelpBruce Forstall1-1/+5
Update help output of runtest.cmd/sh
2018-11-15Combine ExcludeList in AltJit scenarios for both host and altjit archs (#21018)Egor Chesakov1-0/+5
* Add -altjitarch command line argument in tests/runtest.cmd and -altjit_arch in tests/runtest.py * Specify altjitarch in for x86_arm_altjit and x64_arm64_altjit jobs in netci.groovy * Use AltJitArch MSBuild variable in tests/issues.targets to exclude tests based on either BuildArch or AltJitArch since both two JITs are running in altjit scenarios
2018-11-14Update help output of runtest.cmd/shBruce Forstall1-1/+5
2018-10-08Runtest.py on Windows Arm(64) (#20227)Jarret Shook1-8/+20
* Initial infrastructure work to get arm(64) runtest.cmd working * Add excludes and improve runtest.py 1) Adds a build_test_wrappers only to build-test.sh. 2) Adds arm64 windows excludes 3) Adds printlastresults to runtest.cmd 4) corrects runsequential in runtest.sh 5) Corrects SequentialRun in runtest.py 6) Minor improvements to printing test results and copying native test binaries * Address pr feedback * Add to issues targets for arm64 * Working Pri 0 testing. * Remove unecessary common msbuild arguments * add pri1 excludes * Remove common msbuild args * Fix silly python3 issue
2018-08-24Use runtest.py to run tests for all platforms (#19213)Jarret Shook1-1/+76
Change build-test.sh to always build the xunit wrappers. Before it would drop a token and check the existence of the token. Unify x64 linux/OSX/Windows excludes into one file, issues.targets. Includes different locations in the file which show where to put excludes. Remove all target specific aspects of issues.targets, all tests are excluded now via wildcard, this allows expanding to .cmd and .sh based on the built platform. Unify path separators to forward slash(/) in issues.targets to support both platforms Clean up issues.targets by removing long standing exclude tests, specifically tests that have been excluded due to missing features like rva_statics. Add DisableProjectBuild to tests which have been removed from issues.targets Conditionally add DisableProjectBuild to tests which have been marked as unsupported on unix. This is mostly a port of the unsupportedOnUnix.txt list. Instead of excluding the tests, unix will simply not build them. If tests are built on windows, they will be run but they will return pass, the test wrapper will check return instantly. All exclusions ported to issues.targets for linux targets. Expand runtest.py, this includes simple issues that made it past the original CR. In addition it adds more optional features to help with inner loop dev work such as: creating a repro folder under bin/repro/.. which sets up the env and calls the failing test. In addition a launch.json will now be created under bin/repro/.. which can be used to easily debug using vscode. More logging, such as printing failures, longest running tests ect. Initial excludes ported for arm64 windows Arm64 linux, armhf unix excludes and enables running runtest.sh for these targets. arm64 windows and arm32 windows excludes and enables running runtest.cmd on arm64 targets init-tools.sh changes to pull armhf and aarch64 dotnetcli init-tools.cmd changes to pull x86 packages for dotnetcli for arm64 windows runtest.cmd for almost all scenarios will call runtest.py runtest.sh for almsot all scenarios will call runtest.py Removes all logic for running tests using runtest.sh
2018-08-17Enable Tiered Compilation by default (#19525)Koundinya Veluri1-0/+1
Enable Tiered Compilation by default 1) Changes the default state of the tiered compilation feature check to be ON BY DEFAULT 2) Removed comments about the source about this being a work in progress. Although it will surely continue to evolve and improve, remaining issues would be better tracked in our issue tracking system with the same default presumption as other runtime features - assume it works unless noted otherwise. 3) Adjusts a number of tests and automated scripts that made assumptions that the default setting of this feature is off. 4) Stop accepting the deprecated env var COMPLUS_EXPERIMENTAL_TieredCompilation. I'm not aware it has any remaining usage but if so we're going to find out. 5) Adjust config names for JitBench
2018-08-05Clean up build.cmd/build-test.cmd/runtest.cmdBruce Forstall1-45/+60
Various minor cleanups: 1. Use the same naming/style for all three. 2. Make the style consistent throughout each. E.g., consistent indents, capitalization. 3. Add more consistent output, including the prefix tag used everywhere. 4. build.cmd: only build cross components if native build is enabled. 5. Use new __Logging variable to consolidate all logging arguments, to reduce large (unexpanded) command line lengths.
2018-07-16CoreFX test fixes (#18929)Jan Kotas1-1/+1
- Unify Windows and Unix exclusion files to avoid duplication - Disable outdated tests against live CoreFX
2018-06-27Windows CoreFX CI (#18365)Andon Andonov1-2/+104
* Add test list CL switch * End-To-End Test Run on Windows * Cleanup * MAX_PATH Workaround * Set Execution directory for CoreFX tests * Add All CoreFX PR Tests * Add test dependencies * Add extra dependencies * Add parallel test execution * Disable OuterLoop tests and System.Data.SqlClient.* tests * Initialize maximum degree of parallelization to Environment.ProcessCount * Remove unnecessary cli option * Update Dependencies * Add "enabled" property to tests * Remove exclusions due to TestUtilities mismatch * Add capability to run all tests for running Helix test lists directly * Refactor build script to build testhost when skipping managed tests * Disable failing System.Threading.Tests.EventWaitHandleTests.Ctor_InvalidMode * Add switch to skip native test build * Add testing documentation * Don't run tests marked as "disabled" when running all available tests * Add switch to build only testhost and remove Core_Root_Stage * Clean up TopN.CoreFX.Windows.issues.json * Refactor build-test.cmd * PR feedback - build pipeline and documentation * PR Feedback - Test Helper headers and comments * Fix buildtesthost option for only building CoreFX test dependencies * Disable intermittently failing test DrawBezier_PointFs
2018-06-21Windows CoreFX CI Staging Commit (#18388)Andon Andonov1-0/+2
* Netci * Add fake test switch * Add branch PR trigger * Add Checked corefx_innerloop jobs
2018-03-28Removing 'EXPERIMENTAL' from tiered compilation env var (#17283)Noah Falk1-2/+2
Things have progressed far enough that its time to use a friendlier name. The feature still still has performance aspects that need to be investigated and improved, but I don't want to scare people off simply because it isn't as fast as it could be. This also updates to use a newer CoreFX version for JitBench since that appeared to be broken, and updated some comments and usage of the tieredcompilation variable.
2018-03-06Remove vs2015 build dependenciesjashook1-34/+5
2018-02-14Clean up runtest.cmdBruce Forstall1-53/+98
Don't do NI and LOCK deletion (which is slow) if doing GenerateLayoutOnly. Plus, other smaller changes to make the file easier to read, such as adding more comments.
2017-12-20Fix per-test timeoutBruce Forstall1-1/+4
1. Add "timeout" option to runtest.cmd to set the per-test timeout, in ms. 2. Change NETCI to use this new option. This should fix various random test failures that frequently occur in Debug test runs due to timeout (e.g., chaos and type generator tests).
2017-12-07Add progress output to runtest.cmd so you don't think it's hungBruce Forstall1-0/+1
2017-11-26Fix Powershell error (#15217)Bruce Forstall1-2/+2
2017-11-22Fix all powershell invocations in scripts, CI code and tests to include ↵Jacek Blaszczynski1-2/+2
-NoProfile (#15164) Code execution in several scripts and tests can be affected by powershell profile execution at startup. Adding -NoProfile option to all powershell invocations which may be affected by execution of profile startup code.
2017-11-16Clean up netci.groovy (#15030)Bruce Forstall1-4/+20
* Clean up netci.groovy 1. Don't repeat entire list of JIT stress and R2R scenarios over and over; use functions to check for scenario groups. 2. Merge handling of R2R baseline and stress scenarios. 3. Add function to determine if a scenario is a JIT stress scenario. 4. Fix JIT stress mode display name generator to handle CoreFx tests as well, preserving the way they are currently displayed. 5. Fix R2R display name generator to handle baseline R2R scenario. 6. Fix bug where x86 R2R baseline job wasn't created. 7. Fix bugs where triggering some jobs, such as R2R jobs, would trigger lots of other jobs with insufficient trigger strings. Thus, more trigger strings now have architecture + scenario in them. E.g., "Innerloop" jobs now much be triggered using a phrase containing "Innerloop". 8. Simplify code that creates builds, and test run commands. 9. Fix mis-casing of COMPlus_JITMinOpts in R2R stress list. 10. Expand list of valid ARM Windows scenarios to match total list of scenarios, but with some scenarios commented out. Add the jitstress2 + jitstressregs to the acceptable list. Add note about the many scenarios in this list that are not in the full scenario list. Add function to determine if a scenario is in this list. 11. Support crossgen of frameworks using altjit 12. Don't support 'r2r' scenario on arm/armlb/arm64 Windows.
2017-11-15Re-enable Standalone GC job, running daily (#15015)Sean Gillespie1-0/+2
* Re-enable Standalone GC job, running daily * osGroup is already defined * Fix failed assertion
2017-11-14Fix RunTest.cmd argument parsingBruce Forstall1-2/+2
The "jitminopts" and "jitforcerelocs" options were eating whatever option followed them.
2017-10-27Address code review feedbackJacek Blaszczynski1-7/+3
2017-10-24Cleanup VsDevCmd.bat usage and improve build system messages to ease diagnosticsJacek Blaszczynski1-18/+21
2017-10-03Update Buildtools and Roslyn (#14228)Jose Perez Rodriguez1-0/+17
* Update Buildtools and Roslyn * Fixing importing of the roslyn files on msbuild 14.0 * Correctly looking for the latest VS Installation
2017-09-25Add CI PR triggers for x86_arm and x64_arm64 testing (#14158)Bruce Forstall1-2/+2
* Add CI PR triggers for x86_arm and x64_arm64 testing * Fix bug * Don't do GC stress with altjit
2017-08-15Make unit tests run clean with tiered compilationnoahfalk1-0/+2
A handful of tests are optimization sensitive and needed to be disabled because tiered jitting doesn't optimize right away. There was also a shutdown timing issue where the tiered jitting background compilation thread would continue calling into the JIT after the JIT was shutdown. This manifested as an error writing to the jit log after the stream had been closed.
2017-06-13Do not copy bin subdirectories to test core_rootMike Danes1-1/+1
2017-05-23CR feedback, fix BuildArch/BuildType, add ExamplesBruce Forstall1-6/+16
2017-05-22Update runtest.cmd help to reflect realityBruce Forstall1-21/+28
2017-05-11Windows Arm64 - Build/Run tests in CIjashook1-0/+4
Build tests with arm64 runs and upload/run them to the arm64 test machines.
2017-04-26Refactor ildasm round trip logicSven Boemer1-0/+8
This change factors the Ildasm round trip logic so that it should not require a separate test build. The ildasmroundtrip flag is no longer passed to build-test.cmd; instead the wrapper scripts are always generated with a check for an environment variable that can be set from runtests. Also remove separate test build for ilrt in the ci scripts
2017-03-22Exclude some tests from Linker testing.Swaroop Sridhar1-0/+4
Exclude some tests while testing via ILLINK. The reasons are mentioned in issues.targets.
2017-03-16Fix test failure detection when dumps are enabled.Pat Gavlin1-1/+2
2017-03-16Merge pull request #10214 from pgavlin/TimeoutCrashDumpsPat Gavlin1-0/+23
Take crash dumps for timed-out tests.
2017-03-16Take crash dumps for timed-out tests.Pat Gavlin1-0/+23
This change updates the test infrastructure to collect crash dumps for tests that time out. We've been seeing a number of tests that only time out in CI (e.g. #10076); hopefully this will help us root-cause the timeouts.
2017-03-16Delete mscorlib facade (#10157)Jan Kotas1-0/+2
Fixes #7607
2017-03-15Add support for testing via ILLINKSwaroop Sridhar1-0/+3
Add support for testing via ILLINK This commit has the following changes: 1) Setup the testing infrastructure to run the CoreCLR tests after linking through ILLinker `runtest.cmd /link <path-to-illink.exe>` 2) The linker is taken as an argument for runtest.cmd, typically the corebuild of https://github.com/mono/linker 3) Tests are run by linking the test binaries and all libraries except `System.Private.Corelib.dll` 4) For tests that use reflection, setup Reflection roots in source directories as `<test>.reflect.xml`
2017-03-10Fix delayed expansion scripting error in runtest.cmdBruce Forstall1-1/+1
2017-03-07Updating the various windows build scripts to have a comment explaining the ↵Tanner Gooding1-0/+14
VS version selection logic.
2017-03-06Updating the build scripts to support VS2017.Tanner Gooding1-22/+18
2017-01-24Reintroduce building Core_Root to runtest.cmdwtgodbe1-0/+19