diff options
author | Sean Gillespie <sean@swgillespie.me> | 2017-05-05 15:57:25 -0700 |
---|---|---|
committer | GitHub <noreply@github.com> | 2017-05-05 15:57:25 -0700 |
commit | 30a09322644a5512315d34bbb4fccfddf5795f0f (patch) | |
tree | d381eaed17b4327f3f1f5fc2db3a28175bfb2894 /Documentation/project-docs | |
parent | 624378eb5a90386777d05acda5d374af925f2756 (diff) | |
download | coreclr-30a09322644a5512315d34bbb4fccfddf5795f0f.tar.gz coreclr-30a09322644a5512315d34bbb4fccfddf5795f0f.tar.bz2 coreclr-30a09322644a5512315d34bbb4fccfddf5795f0f.zip |
Add a GC Reliability Framework job to the CI (#11280)
* Add GC Reliability Framework CI job
* Three changes:
1) To address offline feedback, update the CI trigger phase document
and the GC testing instructions
2) Set the timeout for GC RF CI jobs to 24 hours, so they don't time
out (they will run for 15 hours by default)
3) Refactor "scenario == 'gc_reliability_framework' to its own method
so that new GC RF scenarios can easily and cleanly be added if
desired in the future
Diffstat (limited to 'Documentation/project-docs')
-rw-r--r-- | Documentation/project-docs/ci-trigger-phrases.md | 17 | ||||
-rw-r--r-- | Documentation/project-docs/garbage-collector-guidelines.md | 33 |
2 files changed, 48 insertions, 2 deletions
diff --git a/Documentation/project-docs/ci-trigger-phrases.md b/Documentation/project-docs/ci-trigger-phrases.md index dd0e981ead..f57be03262 100644 --- a/Documentation/project-docs/ci-trigger-phrases.md +++ b/Documentation/project-docs/ci-trigger-phrases.md @@ -11,7 +11,10 @@ To trigger a job, post a comment on your PR with "@dotnet-bot {trigger-phrase}". - **Windows_NT x64 Release Priority 1 Build & Test:** "test Windows_NT pri1" - **Windows_NT x64 Release IL RoundTrip Build & Test:** "test Windows_NT ilrt" -- **Windows_NT x64 Release Long-Running GC Build & Test:**: "test Windows_NT x64 Release longgc" +- **Windows_NT x64 Release Long-Running GC Build & Test:**: "test Windows_NT Release longgc" +- **Windows_NT x64 Release GC Simulator:**: "test Windows_NT Release gcsimulator" +- **Windows_NT x64 Release Standalone GC:**: "test Windows_NT Release standalone_gc" +- **Windows_NT x64 Release GC Reliability Framework:**: "test Windows_NT Release gc_reliability_framework" - **Windows_NT x64 Release Ready-To-Run Priority 0 Build & Test:** "test Windows_NT Release r2r" - **Windows_NT x64 Checked Ready-To-Run Priority 0 Build & Test:** "test Windows_NT Checked r2r" - **Windows_NT x64 Release Ready-To-Run Priority 1 Build & Test:** "test Windows_NT Release pri1r2r" @@ -69,6 +72,8 @@ To trigger a job, post a comment on your PR with "@dotnet-bot {trigger-phrase}". - **Windows_NT x64 Checked GCStress=0xc JitStress=2 Build & Test:** "test Windows_NT gcstress0xc_jitstress2" - **Windows_NT x64 Checked GCStress=0xc MinOpts Heap Verify 1 Build & Test:** "test Windows_NT gcstress0xc_minopts_heapverify1" - **Windows_NT x64 Checked Long-Running GC Build & Test:**: "test Windows_NT x64 Checked longgc" +- **Windows_NT x64 Checked Standalone GC:**: "test Windows_NT Checked standalone_gc" +- **Windows_NT x64 Checked GC Reliability Framework:**: "test Windows_NT Checked gc_reliability_framework" - **Windows_NT x64 Formatting:**: "test Windows_NT formatting" - **Windows_NT x64 Checked CoreFX Baseline Build & Test:** "test Windows_NT corefx_baseline" - **Windows_NT x64 Checked CoreFX MinOpts Build & Test:** "test Windows_NT corefx_minopts" @@ -166,6 +171,11 @@ To trigger a job, post a comment on your PR with "@dotnet-bot {trigger-phrase}". - **Ubuntu x64 Release Priority 1 Build & Test:** "test Ubuntu pri1" - **Ubuntu x64 Release IL RoundTrip Build & Test:** "test Ubuntu ilrt" - **Ubuntu x64 Release Long-Running GC Build & Test:**: "test Ubuntu Release longgc" +- **Ubuntu x64 Release GC Simulator:**: "test Ubuntu Release gcsimulator" +- **Ubuntu x64 Release Standalone GC:**: "test Ubuntu Release standalone_gc" +- **Ubuntu x64 Checked Standalone GC:**: "test Ubuntu Checked standalone_gc" +- **Ubuntu x64 Release GC Reliability Framework:**: "test Ubuntu Release gc_reliability_framework" +- **Ubuntu x64 Checked GC Reliability Framework:**: "test Ubuntu Checked gc_reliability_framework" - **Ubuntu x64 Release Ready-To-Run Priority 0 Build & Test:** "test Ubuntu Release r2r" - **Ubuntu x64 Checked Ready-To-Run Priority 0 Build & Test:** "test Ubuntu Checked r2r" - **Ubuntu x64 Release Ready-To-Run Priority 1 Build & Test:** "test Ubuntu Release pri1r2r" @@ -248,6 +258,11 @@ To trigger a job, post a comment on your PR with "@dotnet-bot {trigger-phrase}". - **OSX x64 Release Priority 1 Build & Test:** "test OSX pri1" - **OSX x64 Release IL RoundTrip Build & Test:** "test OSX ilrt" - **OSX x64 Release Long-Running GC Build & Test:**: "test OSX Release longgc" +- **OSX x64 Release GC Simulator:**: "test OSX10.12 Release gcsimulator" +- **OSX x64 Release Standalone GC:**: "test OSX10.12 Release standalone_gc" +- **OSX x64 Checked Standalone GC:**: "test OSX10.12 Checked standalone_gc" +- **OSX x64 Release GC Reliability Framework:**: "test OSX10.12 Release gc_reliability_framework" +- **OSX x64 Checked GC Reliability Framework:**: "test OSX10.12 Checked gc_reliability_framework" - **OSX x64 Release Ready-To-Run Priority 0 Build & Test:** "test OSX Release r2r" - **OSX x64 Checked Ready-To-Run Priority 0 Build & Test:** "test OSX Checked r2r" - **OSX x64 Release Ready-To-Run Priority 1 Build & Test:** "test OSX Release pri1r2r" diff --git a/Documentation/project-docs/garbage-collector-guidelines.md b/Documentation/project-docs/garbage-collector-guidelines.md index 66d4c69b9f..1981c8ab6f 100644 --- a/Documentation/project-docs/garbage-collector-guidelines.md +++ b/Documentation/project-docs/garbage-collector-guidelines.md @@ -26,12 +26,43 @@ Required Testing: Validation of the behavior of the affected APIs. ## Stress Testing ## Stress testing must run for at least **48 hours** against a debug build. -Instructions for running stress are located in the repo at tests\src\GC\Stress\stress_run_readme.txt. +Stress testing for checked and release builds can be done on pull requests with The .NET CI infrastructure. +A stress run can be requested using the trigger phrase: + +``` +@dotnet_bot test <platform> <flavor> gc_reliability_framework +``` + +This will run the stress framework for the default amount of time (15 hours) on the given platform and build flavor. ## Functional Testing ## A functional test run executes the same code as a stress run, but only runs for 30 minutes. Instructions for running stress are located in the repo at tests\src\GC\Stress\stress_run_readme.txt. +It is recommended that you run at least some of the below PR-triggered CI jobs: + +``` +@dotnet_bot test Windows_NT Checked longgc +@dotnet_bot test OSX10.12 Checked longgc +@dotnet_bot test Ubuntu Checked longgc +@dotnet_bot test Windows_NT Checked standalone_gc +@dotnet_bot test OSX10.12 Checked standalone_gc +@dotnet_bot test Ubuntu Checked standalone_gc +``` + +The "Long GC" tests are a series of GC tests whose running time is too long or memory usage is too high to run with +the rest of the Priority 0 unit tests. The "Standalone GC" build mode builds and runs the GC in a semi-standalone manner +(see https://github.com/dotnet/coreclr/projects/3). + +You may also wish to run the GC Simulator tests. They may take up to 24 hours to complete and are known to sometimes fail on Ubuntu +due to poor interactions with the Linux OOM killer. However, they have proven to be quite useful in finding bugs in the past: + +``` +@dotnet_bot test Windows_NT Release gcsimulator +@dotnet_bot test Ubuntu Release gcsimulator +@dotnet_bot test OSX10.12 Release gcsimulator +``` + ## Performance Testing ## Coming soon. |