diff options
author | Bryan P. Arant <bryanar@microsoft.com> | 2015-10-22 18:15:39 -0700 |
---|---|---|
committer | Bryan P. Arant <bryanar@microsoft.com> | 2015-10-22 18:15:39 -0700 |
commit | 76f7dfbcbe226985deb4ff25a0c9dc7ffe3ffed5 (patch) | |
tree | 938da5a17575e44ce6df03501a1f5ec17ac114c4 /Documentation | |
parent | a7c205d15936a409b250970b3830be0f3561d2c3 (diff) | |
parent | 3ff774096c0a6634911db24a8406d3ca1466a289 (diff) | |
download | coreclr-76f7dfbcbe226985deb4ff25a0c9dc7ffe3ffed5.tar.gz coreclr-76f7dfbcbe226985deb4ff25a0c9dc7ffe3ffed5.tar.bz2 coreclr-76f7dfbcbe226985deb4ff25a0c9dc7ffe3ffed5.zip |
Merge pull request #1832 from bryanar/enable_crossgen_tests
- Changes required to enable crossgen on the tests.
- Added priority level parameter to buildtest.cmd
- Added crossgen parameter to buildtest.cmd
- Updated testing documentation to reflect prioritylevel testing, and crossgen testing.
Diffstat (limited to 'Documentation')
-rw-r--r-- | Documentation/building/test-configuration.md | 41 | ||||
-rw-r--r-- | Documentation/building/windows-test-instructions.md | 47 | ||||
-rw-r--r-- | Documentation/project-docs/tests.md | 8 |
3 files changed, 77 insertions, 19 deletions
diff --git a/Documentation/building/test-configuration.md b/Documentation/building/test-configuration.md new file mode 100644 index 0000000000..a1e67a6fef --- /dev/null +++ b/Documentation/building/test-configuration.md @@ -0,0 +1,41 @@ +## General Test Infrastructure Notes ## + +### Kinds of Build Properties ### +* Build Only +> `<CLRTestKind>BuildOnly</CLRTestKind>` + + * Builds an executable. + * Will not execute it. + +* Run Only +> `<CLRTestKind>RunOnly</CLRTestKind>` + + * Will only execute another assembly. +* Build and Run +> `<CLRTestKind>BuildAndRun</CLRTestKind>` + + * Builds an executable. + * Will execute said executable. +* Shared Libraries +> `<CLRTestKind>SharedLibrary</CLRTestKind>` + + * For building libraries common to zero or more tests. + + +By default (i.e. if not specified explicitly) a project file is BuildAndRun. + +### Priority ### +Testcases are categorized by their priority levels. The most important subset should be and is the smallest subset. This subset is called priority 0. + * By default, a testcase is priority 0. You must elect to de-prioritize a test. + * To de-prioritize a test, add a property _CLRTestPriority_ to the test's project file. +> `<CLRTestPriority>2</CLRTestPriority>` + * Lower priority values are always run in conjunction when running higher priority value tests. I.e. if a developer elects to do a priority 2 test run, then all priority 0, 1 and 2 tests are run. + +### Adding Tests ### +#### Converting an existing C# project #### + * Remove AssemblyName + * Swap in dir.props + * Swap in dir.targets + * Assign a CLRTestKind + * (optional) Assign a priority value + diff --git a/Documentation/building/windows-test-instructions.md b/Documentation/building/windows-test-instructions.md index 1935ce550c..6b74834048 100644 --- a/Documentation/building/windows-test-instructions.md +++ b/Documentation/building/windows-test-instructions.md @@ -3,15 +3,37 @@ Building and running tests on Windows **Building Tests** -In a clean command prompt, issue the following command: +To build the tests simply navigate to the tests directory above the repo and run, + + C:\git\coreclr>tests\buildtest.cmd + +*Cleaning Tests* + +**Note:** Cleaning should be done before all tests to be sure that the test assets are initialized correctly. To do a clean build of the tests, in a clean command prompt, issue the following command: C:\git\coreclr>tests\buildtest.cmd clean -**buildtest /?** will list supported parameters. +*Building tests that will CrossGen* + + C:\git\coreclr>tests\buildtest.cmd crossgen + +This will enable crossgen.exe to be run against test executables before they are executed. + +*Building Other Priority Tests* -**Note:** The above command (or building from the repo_root) must be done once, at the least, to ensure that all test dependencies are initialized correctly. + C:\git\coreclr>tests\buildtest.cmd priority 2 -In Visual Studio, open `<repo_root>\tests\src\AllTestProjects.sln`, build all the test projects or the one required. +The number '2' is just an example. The default value (if no priority is specified) is 0. To clarify, if '2' is specified, all tests with CLRTestPriorty 0, 1 AND 2 will be built and consequently run. + +**Example** + +To run a clean, priority 1, crossgen test pass: + + C:\git\coreclr>tests\buildtest.cmd clean crossgen priority 1 + +**buildtest /?** will list additional supported parameters. + +Additionally, there is a Visual Studio solution, `<repo_root>\tests\src\AllTestProjects.sln`, where users can build a particular testcase, or all priority 0 testcases that are within it. **Running Tests** @@ -50,11 +72,14 @@ If test changes are needed, make the change and build the test project. This wil **Authoring Tests (in VS)** + 1. Use an existing test such as `<repo_root>\tests\src\Exceptions\Finalization\Finalizer.csproj` as a template and copy it to a new folder under `<repo_root>\tests\src`. -2. Add the project of the new test to `<repo_root>\tests\src\AllTestProjects.sln` in VS -3. Add source files to this newly added project. -4. Indicate the success of the test by returning `100`. -5. Add the .NET CoreFX contract references, as required, via the Nuget Package Manager in Visual Studio. *Make sure this does not change the csproj. If it does, then undo the change in the csproj.* -6. Add any other projects as a dependency, if needed. -7. Build the test. -8. Follow the steps to re-run a failed test to validate the new test. +2. Be sure that the AssemblyName has been removed (this causes confusion with the way tests are generally handled behind the scenes by the build system). +3. [Assign a CLRTestKind/CLRTestPriority.](test-configuration.md) +4. Add the project of the new test to `<repo_root>\tests\src\AllTestProjects.sln` in VS +5. Add source files to this newly added project. +6. Indicate the success of the test by returning `100`. +7. Add the .NET CoreFX contract references, as required, via the Nuget Package Manager in Visual Studio. *Make sure this does not change the csproj. If it does, then undo the change in the csproj.* +8. Add any other projects as a dependency, if needed. +9. Build the test. +10. Follow the steps to re-run a failed test to validate the new test. diff --git a/Documentation/project-docs/tests.md b/Documentation/project-docs/tests.md index a1e67a6fef..ca6a5ef740 100644 --- a/Documentation/project-docs/tests.md +++ b/Documentation/project-docs/tests.md @@ -1,4 +1,3 @@ -## General Test Infrastructure Notes ## ### Kinds of Build Properties ### * Build Only @@ -31,11 +30,4 @@ Testcases are categorized by their priority levels. The most important subset sh > `<CLRTestPriority>2</CLRTestPriority>` * Lower priority values are always run in conjunction when running higher priority value tests. I.e. if a developer elects to do a priority 2 test run, then all priority 0, 1 and 2 tests are run. -### Adding Tests ### -#### Converting an existing C# project #### - * Remove AssemblyName - * Swap in dir.props - * Swap in dir.targets - * Assign a CLRTestKind - * (optional) Assign a priority value |