diff options
author | driazati <driazati@users.noreply.github.com> | 2021-05-21 18:21:57 -0700 |
---|---|---|
committer | Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com> | 2021-05-21 18:23:40 -0700 |
commit | 84b6c629d3ed0aff3dea7d3e0c2fdb50fc46d8ab (patch) | |
tree | c65e736c1181db132083f38cf2bcd24ad54c264e /Makefile | |
parent | b842351b4fcce7eb088b58dfbcb86f54119f57f3 (diff) | |
download | pytorch-84b6c629d3ed0aff3dea7d3e0c2fdb50fc46d8ab.tar.gz pytorch-84b6c629d3ed0aff3dea7d3e0c2fdb50fc46d8ab.tar.bz2 pytorch-84b6c629d3ed0aff3dea7d3e0c2fdb50fc46d8ab.zip |
[lint] Move shellcheck to its own step (#58623)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/58623
This splits out everything shellcheck related into its own job that generates and checks GHA workflows, then shellchecks those + jenkins scripts. This PR also integrates shellcheck into the changed-only stuff in `actions_local_runner.py` so that shellcheck won't do anything unless someone edits a shell script in their local checkout. This is the final piece to clean up the output of `make quicklint` and speeds it up by a good bit (before it was shellchecking everything which took a few seconds):
```
$ make quicklint -j $(nproc)
✓ quick-checks: Ensure no unqualified noqa
✓ quick-checks: Ensure canonical include
✓ quick-checks: Ensure no unqualified type ignore
✓ quick-checks: Ensure no direct cub include
✓ quick-checks: Ensure no tabs
✓ quick-checks: Ensure no non-breaking spaces
✓ shellcheck: Regenerate workflows
✓ quick-checks: Ensure no versionless Python shebangs
✓ quick-checks: Ensure correct trailing newlines
✓ shellcheck: Assert that regenerating the workflows didn't change them
✓ mypy (skipped typestub generation)
✓ cmakelint: Run cmakelint
✓ quick-checks: Ensure no trailing spaces
✓ flake8
✓ shellcheck: Extract scripts from GitHub Actions workflows
✓ shellcheck: Run Shellcheck
real 0.92
user 6.12
sys 2.45
```
Test Plan: Imported from OSS
Reviewed By: nikithamalgifb
Differential Revision: D28617293
Pulled By: driazati
fbshipit-source-id: af960ed441db797d07697bfb8292aff5010ca45b
Diffstat (limited to 'Makefile')
-rw-r--r-- | Makefile | 38 |
1 files changed, 26 insertions, 12 deletions
@@ -32,13 +32,33 @@ generate-gha-workflows: .github/scripts/generate_ci_workflows.py $(MAKE) shellcheck-gha +shellcheck: + @$(PYTHON) tools/actions_local_runner.py \ + --file .github/workflows/lint.yml \ + --job 'shellcheck' \ + --step "Regenerate workflows" + @$(PYTHON) tools/actions_local_runner.py \ + --file .github/workflows/lint.yml \ + --job 'shellcheck' \ + --step "Assert that regenerating the workflows didn't change them" + @$(PYTHON) tools/actions_local_runner.py \ + --file .github/workflows/lint.yml \ + --job 'shellcheck' \ + --step 'Extract scripts from GitHub Actions workflows' + @$(PYTHON) tools/actions_local_runner.py \ + --file-filter '.sh' \ + $(CHANGED_ONLY) \ + --job 'shellcheck' + setup_lint: $(PYTHON) tools/actions_local_runner.py --file .github/workflows/lint.yml \ - --job 'flake8-py3' --step 'Install dependencies' --no-quiet + --job 'flake8-py3' --step 'Install dependencies' --no-quiet $(PYTHON) tools/actions_local_runner.py --file .github/workflows/lint.yml \ - --job 'cmakelint' --step 'Install dependencies' --no-quiet + --job 'cmakelint' --step 'Install dependencies' --no-quiet $(PYTHON) tools/actions_local_runner.py --file .github/workflows/lint.yml \ - --job 'mypy' --step 'Install dependencies' --no-quiet + --job 'mypy' --step 'Install dependencies' --no-quiet + $(PYTHON) tools/actions_local_runner.py --file .github/workflows/lint.yml \ + --job 'shellcheck' --step 'Install Jinja2' --no-quiet @if [ "$$(uname)" = "Darwin" ]; then \ if [ -z "$$(which brew)" ]; then \ @@ -48,16 +68,11 @@ setup_lint: brew install shellcheck; \ else \ $(PYTHON) tools/actions_local_runner.py --file .github/workflows/lint.yml \ - --job 'quick-checks' --step 'Install ShellCheck' --no-quiet; \ + --job 'shellcheck' --step 'Install ShellCheck' --no-quiet; \ fi pip install jinja2 quick_checks: - @$(PYTHON) tools/actions_local_runner.py \ - --file .github/workflows/lint.yml \ - --job 'quick-checks' \ - --step 'Extract scripts from GitHub Actions workflows' - # TODO: This is broken when 'git config submodule.recurse' is 'true' since the # lints will descend into third_party submodules @$(PYTHON) tools/actions_local_runner.py \ @@ -71,7 +86,6 @@ quick_checks: --step 'Ensure no unqualified noqa' \ --step 'Ensure no unqualified type ignore' \ --step 'Ensure no direct cub include' \ - --step 'Run ShellCheck' \ --step 'Ensure correct trailing newlines' flake8: @@ -102,7 +116,7 @@ toc: --job 'toc' \ --step "Regenerate ToCs and check that they didn't change" -lint: flake8 mypy quick_checks cmakelint generate-gha-workflows +lint: flake8 mypy quick_checks cmakelint shellcheck quicklint: CHANGED_ONLY=--changed-only -quicklint: mypy flake8 mypy quick_checks cmakelint generate-gha-workflows +quicklint: mypy flake8 mypy quick_checks cmakelint shellcheck |