summaryrefslogtreecommitdiff
path: root/perf.groovy
AgeCommit message (Collapse)AuthorFilesLines
2018-03-14Turn sizeondisk tests back on (#16937)Michelle McDaniel1-1/+1
2018-03-14Remove call to GenerateLayoutOnly for throughput runs (#16919)Michelle McDaniel1-1/+0
2018-03-08Turn perf jobs back on (#16824)Michelle McDaniel1-34/+34
2018-03-06Disable Perf Runs on Commit (#16781)Michelle McDaniel1-35/+35
We need to disable these jobs on commit until dotnet/core-eng#2808 is fixed.
2018-02-07Archive CI perf results (#16230)Noah Falk1-0/+7
* Archive CI perf results I need to diagnose a failure that only reproes in CI but the CI scripts don't archive the logs on failure. This change should fix that. * Adding pipeline build per feedback
2018-01-30Add OS flag to calls to run-xunit-perf.py (#16100)Michelle McDaniel1-4/+4
We were uploading windows results as win32 instead of Windows_NT.
2018-01-24Fix syntax error in illink groovy (#15999)Michelle McDaniel1-1/+1
2018-01-23Remove hardcoded paths in linkbench scripts (#15939)Michelle McDaniel1-1/+2
* Remove hardcoded paths in linkbench scripts Linkbench has hardcoded paths based off of VS140COMNTOOLS, which not all machines will have (ie, machines with only VS2017 installed). This change removes the hardcoded paths, and replaces them with checks to make sure the tool is on the path (which they will be if we are in a VS environment of any kind - which we would have been in if VS140COMNTOOLS was already set).
2018-01-19Convert perf.groovy to use run-xunit-perf.py (#15881)Michelle McDaniel1-25/+17
* Convert perf.groovy to use run-xunit-perf.py Also removes run-xunit-perf.cmd and run-xunit-perf.sh. This change moves us to using build-tests.sh to generate the layout (ie pull down corefx), so this change fixes #12143.
2017-12-06Move all non-full opt, non-pgo perf legs to periodic (#15398)Michelle McDaniel1-3/+15
We don't need the perf results for min-opt/no-pgo runs as frequently as we need full-opt pgo runs (which we want to get per commit, if possible), so move them to only running once daily.
2017-12-01Change name to perf jobs to reflect the actual OS they run on (#15314)Victor "Nate" Graf1-1/+1
* Change name fo perf jobs to reflect the actual OS they run on * readd Ubuntu 14.04
2017-11-22Fix all powershell invocations in scripts, CI code and tests to include ↵Jacek Blaszczynski1-4/+4
-NoProfile (#15164) Code execution in several scripts and tests can be affected by powershell profile execution at startup. Adding -NoProfile option to all powershell invocations which may be affected by execution of profile startup code.
2017-11-20Remove x86 legacy_backend build and CI testingBruce Forstall1-18/+1
2017-11-15change labels to use new Ubuntu machine pool (#15043)Victor "Nate" Graf1-2/+2
2017-11-08Small number of bug fixes. (#14900)José Rivero1-15/+6
- Added stability prefix to the scenario benchmark (JitBench) - Specify output directory to the `run-xunit-perf.cmd` script and avoid the extra step to xcopy files to the archive folder. - Added a command line parser class to the illink scenario, and changed its behavior where it used to fail when a new command line option passed to xUnit was not recognized. - Save the output log of the tests into the sandbox-logs folder. - Updating the label of the machine pool used by the illink scenario
2017-11-02Add perf pipeline job full groovy (#14397)Michelle McDaniel1-115/+93
This change converts our perf testing to use pipeline jobs. Pipeline jobs allow us to do the following: 1) Test on the same commit for each of the test legs 2) Parallelize the build and test steps. 3) Separate the build and test steps from one another. This gives us the ability to use the same build assets for all of the test legs of the same configuration. It also allows us to build on virtual machines and test on perf machines, so we only use the perf resources for testing. 4) Have different test scenarios for PRs and rolling. This isn't strictly a benefit of pipeline jobs, but certainly is made easier by them. 5) Allows us to have one trigger for PR jobs which will get us all the perf testing scenarios. This change also cleans up the groovy scripting for perf testing.
2017-11-01Updating xUnit Benchmark Scenarios to capture Etw with Pmc support. (#14734)José Rivero1-12/+10
- Adding 'tiered' jitting - Enable JitBench to capture Pmc data (Profile=On) - Fixed some xUnit performance warnings - Enforce DOTNET_MULTILEVEL_LOOKUP environment to be turned off - Specify `dotnet publish` output directory.
2017-10-13Disabling legacy banckend perf CQ and scenario runs (#14468)José Rivero1-2/+2
- Recent changes to enforce data validation uncovered bug on the legacy backend runs. Now, we do not allow to upload tests with no performance data. CQ run bug tracked by: #14463 JitBench run bug tracked by: #14477
2017-10-09Fixing JitBench (#14364)José Rivero1-1/+1
- In the lab, the path where the JitBench sources are downloaded is already too long, which eventially causes `dotnet-install.ps1` to fail. Part of the changes are to reduce the PATH where the tests are placed. - Updated the JitBench version/commit being used. This one is already on 2.1 - Fixing typo on `perf.groovy` - Increasing the timeout value as JitBench keeps failing on the lab.
2017-10-02Small bug fixes on "perf.groovy" (#14259)José Rivero1-12/+12
- `submission-metadata` option is `--user-email`, not `--user` - We need to escape user input. On batch, " -> ""
2017-09-22Create dotnet deployment and acquisition size on disk test (#13956)Victor "Nate" Graf1-0/+82
* [WIP] First version of SoDBench. Contains bugs * [WIP] Move SoDBench files and increase error checking * [WIP] Add NugetConfig to enable pulling packages from myget * [WIP] Remove unhelpful templates and add back oses * [WIP] Add ability to specify channel * [WIP] Improve CSV writing * [WIP] Improve options parsing * Fix syntax error * [WIP] Add test leg to perf.groovy * [WIP] Adjust label to target an existing machine pool * Change label to run on virtual machine * Use setMachineAffinity * Add ASP.NET-Core feed and deafult to master as the channel * Change channel to master in perf.groovy * Move NuGet.Config up a directory so it only needs to be written once * Add CommandLine to external dependencies * Remove CommandLine as it is now in external.depproj * Adjust wget command to work more consistantly * Change calculation of relative paths for clarity * Set job to run daily instead of on push/PR * Build sodbench on job run * Remove MSBuild from job * Fix quote placement * Change metadata to be more accurate * Add rollup totals for each measured category * Refactor to use a tree rather than a dictionary * Limit report size * Publish in release configuration
2017-09-18Correct typos in reporoot/perf.groovyJacek Blaszczynski1-4/+4
2017-09-12Fixing archiving of Linux perf artifacts. (#13911)José Rivero1-1/+2
- Previous change did not fix archiving artifacts
2017-09-11Ubuntu runs are not archiving perf results because rsync does not create ↵José Rivero1-2/+2
destination folder (#13858)
2017-09-05Add nopgo option for throughput runsMichelle McDaniel1-62/+83
2017-09-05Moving the Windows Performance runs from Server 2012 to Server 2016. (#13725)José Rivero1-3/+3
2017-08-30Revert #13647 and #13638 (#13666)Daniel Podder1-18/+6
#13638 and #13647 attempted to replace the generic 'dotnet-bot' e-mail address with PR authors' e-mail addresses when uploading perf data to BenchView. Unfortunately, these changes are still broken for some users (specifically, if a user's e-mail address is not published/visible on their GitHub profile). There is no clean way to implement the proper fix, and the right fix will change once pipeline support is available. Rather than putting in something hacky now, I'm reverting these changes to unblock PRs. We should revisit these changes after pipeline jobs are available.
2017-08-29IlLink perf stopped running because the VM pool name does not exist. (#13672)José Rivero1-3/+2
2017-08-29Fixing IlLink job leg name. (#13646)José Rivero1-2/+2
- CoreClr Scenario and CoreClr IlLink jobs were aliased on CI.
2017-08-28Replace ghprbTriggerAuthorEmail -> ghprbPullAuthorEmail (#13647)Daniel Podder1-7/+7
Perf smoke tests have been broken since #13638. The issue appears to be that `ghprbTriggerAuthorEmail` is an empty string in most cases, so its value is not expanded. The attempted fix is to use `ghprbPullAuthorEmail` instead, which is always defined (since there's always an owner to a PR).
2017-08-28Merge pull request #13638 from adiaaida/updatePerfUserMichelle McDaniel1-6/+18
User the PR trigger email for benchview submit
2017-08-28Use the PR trigger email for benchview submitMichelle McDaniel1-6/+18
When users trigger perf runs for PRs, the alias used to submit in submission-metadata.json should be that user's email. We should only use dotnet-bot for official runs.
2017-08-25Minor performance infrastructure changes. (#13557)José Rivero1-24/+118
- IlLink scenario will run on a VM (currently it is only measuring MSIL size) - Updated the patterns of files to search for on scenario runs - Updated the JitBench commit hash being used () - Disable min_opt from the PR correctness mode. - String was not being interpolated.
2017-08-18`perf.groovy` updates. (#13465)José Rivero1-44/+54
- Increase how long performance artifacts are kept. - Added an extra step to move files to be archived into a different folder.
2017-08-17Add minopts scenario for Legacy backendRoman Artemev1-230/+294
2017-08-16`archiveSettings.addFiles` uses glob instead of path. (#13403)José Rivero1-7/+3
- Files were not being archived after last changes.
2017-08-15Small perf infrastructure changes (#13325)José Rivero1-45/+46
- Move xUnit Perf Api output files to a folder `Logs` under sandbox - Made necessary changes to `perf.groovy` to archive artifacts - Added a switch -outputdir to the run-xunit-perf* scripts - Enable running all CoreClr scenarios (JitBench & IlLink) despite failure - Move sandbox to bin folder - Use the default output dir: `.\bin\sandbox`
2017-08-07Swaping the order of how scenarios are run. (#13255)José Rivero1-1/+4
- Currently ILLink scenario is broken and the x64 scenario leg does not run JitBench By swaping the scenario it will let JitBench run while ILLink is being fixed.
2017-08-03Drop the first iteration value of JitBench. (#13211)José Rivero1-1/+1
2017-08-02Add JitMinOpts throughput jobsMichelle McDaniel1-105/+125
This change adds an opt_level throughput option so that we can run throughput jobs in minopts mode. This also adds jobs to the lab for min opts runs
2017-08-02Add JitBench scenario testing. (#11400)José Rivero1-24/+14
2017-06-21Align Linux TP benchmarks with Windows TP benchmarksMichelle McDaniel1-1/+1
On Windows, we download a set of assemblies that we have uploaded to MyGet. On Linux, we were building corefx each run to get the benchmarks. This change updates Linux to pull down the same benchmarks Windows uses, using curl to grab the nuget package from MyGet. Note: after this commit, we will have data for fewer assemblies on Linux, but we will now have the same set for both Windows and Linux. Some data in Benchview may be affected.
2017-06-19Remove sudo from throughput runMichelle McDaniel1-1/+1
Sudo is not necessary for running throughput testing, so remove sudo so we don't run into requests for passwords if the ubuntu machines are not set up correctly.
2017-06-16Fix perf.groovySwaroop Sridhar1-2/+2
Fix perf.groovy so that ILLink perf tests can be triggered.
2017-06-13Pull CoreCLR build from RHEL7.2 in perf runs (#12238)Daniel Podder1-35/+141
Our Ubuntu perf testing has been creating Linux CoreCLR builds by creating the build locally. This fits the local developer scenario, but doesn't actually test the bits that ship; the 'official' product configuration is generated by building on RHEL 7.2 and packaging those bits for various distros, including Ubuntu. In order to more accurately measure end-user-perceived performance, we should do the same thing official builds do: build on RHEL7.2, then copy those bits onto other OSes to test perf on those OSes. This PR makes that happen by structuring Linux perf and throughput jobs as flow jobs, with builds up front on a RHEL VM, before the bits are dropped onto Ubuntu perf test machines.
2017-06-12Add help statement for perf.groovyDrew Scoggins1-0/+4
When you use the help from dotnet bot you do not get any of the performance runs in the help. This should add those in.
2017-06-09Move ILLink Perf to a separate jobSwaroop Sridhar1-1/+89
This changes ILLink performance testing to its own leg. With this change, ILLink perf test will not be run for every PR by default. It can be requested explicitly on a PR, and will run on the rolling builds. JitBench build breaks have broken the PR builds a few times, so this change moves ILLink perf tests out of the PR job, so that MusicStore testing can be enabled again.
2017-06-08perf.groovy fails to sanitize <PR> name properly (#12167)José Rivero1-6/+6
Fix #12166 * Using "double quotes" instead of [square brackets] in the if condition.
2017-06-01Updating version of xUnit-Performance-Api (#11987)José Rivero1-2/+4
- This update contains a fix for this issue: https://github.com/Microsoft/xunit-performance/issues/230 - Consolidate the Api version in two places: `dependencies.prop` and `PerfHarness.csproj` - Move performance log files to root directory where Jenkins can archive them - Fixed the output to console, so it is easier to see on the Jenkins job the sccript steps live and get an exact repro step if needed
2017-05-26Enable linux perf runs (#11905)José Rivero1-39/+50
- Removing `[assembly: MeasureInstructionsRetired]` from performance tests. - Due to this issue: https://github.com/Microsoft/xunit-performance/issues/231, we are blocked from running on performance tests on Linux. The tag is not used by infrastructure and it is passed during runtime when profiling on Windows. - Adding the option to execute CoreRun with the stability prefix. - Moving file to be archived by Jenkins and updating groovy file. - Setting the maximum number of iterations.
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-18 07:04:10 +0000