Refactor native build to remove the compiler-override files (#23897)

* On systems that have both cmake 2 and cmake 3, use cmake 3.

Move CMAKE_EXPORT_COMPILE_COMMANDS to top level instead of in gen-buildsys-*. Define the CMake install prefix in gen-buildsys-* instead of pulling from an environment variable.

Define C++ standard as CMake property instead of as flag.

Move CLR_DEFINES_*_INIT out of overrides and into configurecompiler.cmake

Move flags that generate debug info into configurecompiler.cmake

Remove the CMAKE_USER_RULES_OVERRIDE files.

Add cmake version output for determining what cmake versions each CI system has.

Fix syntax in gen-buildsys-clang.

Change add_compile_definitions back to add_definitions

Add -D prefix for adding definitions via add_definitions

Remove extraneous double-quote

Change default config definition adding to the syntax in master

Switch back to old CMAKE_<LANG>_FLAGS way of setting the language standards and try to go back to 2.8.12 minimum

Switch back setting compile definitions for non-Windows branch too.

Use SET with CMAKE_<LANG>_FLAGS. Convert some usages of appending to CMAKE_<LANG>_FLAGS to add_compile_options where possible.

Set CMAKE_<LANG>_FLAGS_INIT instead of CMAKE_<LANG>_FLAGS

Make sure configureopimitzation.cmake is included correctly in test build.

Try to add brackets to get the Linux ARM compilation working correctly.

Define standard language version in configurecompiler.cmake instead of root CMakeLists (so tests get it)

Try to move langauge standard check to configure.cmake

define language standard in each root CMakeLists.txt

Fix off-Windows test build.

Set CMAKE_EXPORT_COMPILE_COMMANDS after the project() call

* Set CMAKE_USER_MAKE_RULES_OVERRIDE to "" to not break incremental builds after building on a branch that had it set to a path.

* Remove CMake version output.

* Move comment outside of multiline command.

* Retry setting CMAKE_USER_MAKE_RULES_OVERRIDE

* Remove unnecessary variable wrappers.

* Simplify cmake 3 resolution.

* Explicitly use CMAKE_CXX_FLAGS for C++-only flags that GCC complains about on C files.

* Set -Wall via CMAKE_<LANG>_FLAGS until we can move all flag settings to add_compile_options

* Fix typos

* Another temporary precedence issue.

* include configureoptimization.cmake in configurecompiler.cmake

* Move setting CMAKE_EXPORT_COMPILE_COMMANDS to configurecompiler.cmake.

* Rename configure.cmake -> verify_lto.cmake.

* Fix path to verify-lto

* Try using CMAKE_<LANG>_FLAGS instead of CMAKE_<LANG>_FLAGS_INIT.

* Revert name change to configure.camek

7 files changed, 12 insertions, 95 deletions

diff --git a/src/pal/tools/clang-compiler-override-arm.txt b/src/pal/tools/clang-compiler-override-arm.txt
deleted file mode 100644
index da25715ade..0000000000
--- a/src/pal/tools/clang-compiler-override-arm.txt
+++ /dev/null
@@ -1,20 +0,0 @@
-SET (CMAKE_C_FLAGS_INIT                "-Wall -std=c11")
-SET (CMAKE_C_FLAGS_DEBUG_INIT          "-g -O0")
-SET (CLR_C_FLAGS_CHECKED_INIT          "-g -O1")
-# Refer to the below instruction to support __thread with -O2/-O3 on Linux/ARM
-# https://github.com/dotnet/coreclr/blob/master/Documentation/building/linux-instructions.md
-SET (CMAKE_C_FLAGS_RELEASE_INIT        "-g -O1")
-SET (CMAKE_C_FLAGS_RELWITHDEBINFO_INIT "-g -O1")
-
-SET (CMAKE_CXX_FLAGS_INIT                "-Wall -Wno-null-conversion -std=c++11")
-SET (CMAKE_CXX_FLAGS_DEBUG_INIT          "-g -O0")
-SET (CLR_CXX_FLAGS_CHECKED_INIT          "-g -O1")
-SET (CMAKE_CXX_FLAGS_RELEASE_INIT        "-g -O1")
-SET (CMAKE_CXX_FLAGS_RELWITHDEBINFO_INIT "-g -O1")
-
-SET (CLR_DEFINES_DEBUG_INIT              DEBUG _DEBUG _DBG URTBLDENV_FRIENDLY=Checked BUILDENV_CHECKED=1)
-SET (CLR_DEFINES_CHECKED_INIT            DEBUG _DEBUG _DBG URTBLDENV_FRIENDLY=Checked BUILDENV_CHECKED=1)
-SET (CLR_DEFINES_RELEASE_INIT            NDEBUG URTBLDENV_FRIENDLY=Retail)
-SET (CLR_DEFINES_RELWITHDEBINFO_INIT     NDEBUG URTBLDENV_FRIENDLY=Retail)
-
-SET (CMAKE_INSTALL_PREFIX                $ENV{__CMakeBinDir})
diff --git a/src/pal/tools/clang-compiler-override.txt b/src/pal/tools/clang-compiler-override.txt
deleted file mode 100644
index ee1e63b63b..0000000000
--- a/src/pal/tools/clang-compiler-override.txt
+++ /dev/null
@@ -1,20 +0,0 @@
-SET (CMAKE_C_FLAGS_INIT                "-Wall -std=c11")
-SET (CMAKE_C_FLAGS_DEBUG_INIT          "-g -O0")
-SET (CLR_C_FLAGS_CHECKED_INIT          "-g -O2")
-# Refer to the below instruction to support __thread with -O2/-O3 on Linux/ARM
-# https://github.com/dotnet/coreclr/blob/master/Documentation/building/linux-instructions.md
-SET (CMAKE_C_FLAGS_RELEASE_INIT        "-g -O3")
-SET (CMAKE_C_FLAGS_RELWITHDEBINFO_INIT "-g -O2")
-
-SET (CMAKE_CXX_FLAGS_INIT                "-Wall -Wno-null-conversion -std=c++11")
-SET (CMAKE_CXX_FLAGS_DEBUG_INIT          "-g -O0")
-SET (CLR_CXX_FLAGS_CHECKED_INIT          "-g -O2")
-SET (CMAKE_CXX_FLAGS_RELEASE_INIT        "-g -O3")
-SET (CMAKE_CXX_FLAGS_RELWITHDEBINFO_INIT "-g -O2")
-
-SET (CLR_DEFINES_DEBUG_INIT              DEBUG _DEBUG _DBG URTBLDENV_FRIENDLY=Checked BUILDENV_CHECKED=1)
-SET (CLR_DEFINES_CHECKED_INIT            DEBUG _DEBUG _DBG URTBLDENV_FRIENDLY=Checked BUILDENV_CHECKED=1)
-SET (CLR_DEFINES_RELEASE_INIT            NDEBUG URTBLDENV_FRIENDLY=Retail)
-SET (CLR_DEFINES_RELWITHDEBINFO_INIT     NDEBUG URTBLDENV_FRIENDLY=Retail)
-
-SET (CMAKE_INSTALL_PREFIX                $ENV{__CMakeBinDir})
diff --git a/src/pal/tools/gcc-compiler-override.txt b/src/pal/tools/gcc-compiler-override.txt
deleted file mode 100644
index 337d41af16..0000000000
--- a/src/pal/tools/gcc-compiler-override.txt
+++ /dev/null
@@ -1,20 +0,0 @@
-SET (CMAKE_C_FLAGS_INIT                "-Wall -std=c11")
-SET (CMAKE_C_FLAGS_DEBUG_INIT          "-g -O0")
-SET (CLR_C_FLAGS_CHECKED_INIT          "-g -O2")
-# Refer to the below instruction to support __thread with -O2/-O3 on Linux/ARM
-# https://github.com/dotnet/coreclr/blob/master/Documentation/building/linux-instructions.md
-SET (CMAKE_C_FLAGS_RELEASE_INIT        "-g -O3")
-SET (CMAKE_C_FLAGS_RELWITHDEBINFO_INIT "-g -O2")
-
-SET (CMAKE_CXX_FLAGS_INIT                "-Wall  -Werror=conversion-null -std=c++11")
-SET (CMAKE_CXX_FLAGS_DEBUG_INIT          "-g -O0")
-SET (CLR_CXX_FLAGS_CHECKED_INIT          "-g -O2")
-SET (CMAKE_CXX_FLAGS_RELEASE_INIT        "-g -O3")
-SET (CMAKE_CXX_FLAGS_RELWITHDEBINFO_INIT "-g -O2")
-
-SET (CLR_DEFINES_DEBUG_INIT              DEBUG _DEBUG _DBG URTBLDENV_FRIENDLY=Checked BUILDENV_CHECKED=1)
-SET (CLR_DEFINES_CHECKED_INIT            DEBUG _DEBUG _DBG URTBLDENV_FRIENDLY=Checked BUILDENV_CHECKED=1)
-SET (CLR_DEFINES_RELEASE_INIT            NDEBUG URTBLDENV_FRIENDLY=Retail)
-SET (CLR_DEFINES_RELWITHDEBINFO_INIT     NDEBUG URTBLDENV_FRIENDLY=Retail)
-
-SET (CMAKE_INSTALL_PREFIX                $ENV{__CMakeBinDir})
diff --git a/src/pal/tools/gen-buildsys-clang.sh b/src/pal/tools/gen-buildsys-clang.sh
index 8f9881ba0c..125a1580d7 100755
--- a/src/pal/tools/gen-buildsys-clang.sh
+++ b/src/pal/tools/gen-buildsys-clang.sh
@@ -147,19 +147,9 @@ if [ "$build_arch" == "armel" ]; then
     cmake_extra_defines="$cmake_extra_defines -DARM_SOFTFP=1"
 fi
 
-clang_version=$( $CC --version | head -1 | sed 's/[^0-9]*\([0-9]*\.[0-9]*\).*/\1/' )
-# Use O1 option when the clang version is smaller than 3.9
-# Otherwise use O3 option in release build
-if [[ ( ${clang_version%.*} -eq 3  &&  ${clang_version#*.} -lt 9 ) &&
-      (  "$build_arch" == "arm" || "$build_arch" == "armel" ) ]]; then
-    overridefile=clang-compiler-override-arm.txt
-else
-    overridefile=clang-compiler-override.txt
-fi
-
 __currentScriptDir="$script_dir"
 
-cmake_command=cmake
+cmake_command=$(command -v cmake3 || command -v cmake)
 
 if [[ "$scan_build" == "ON" ]]; then
     export CCC_CC=$CC
@@ -168,17 +158,18 @@ if [[ "$scan_build" == "ON" ]]; then
     cmake_command="$SCAN_BUILD_COMMAND $cmake_command"
 fi
 
+# Include CMAKE_USER_MAKE_RULES_OVERRIDE as uninitialized since it will hold its value in the CMake cache otherwise can cause issues when branch switching
 $cmake_command \
   -G "$generator" \
-  "-DCMAKE_USER_MAKE_RULES_OVERRIDE=${__currentScriptDir}/$overridefile" \
   "-DCMAKE_AR=$llvm_ar" \
   "-DCMAKE_LINKER=$llvm_link" \
   "-DCMAKE_NM=$llvm_nm" \
   "-DCMAKE_OBJDUMP=$llvm_objdump" \
   "-DCMAKE_BUILD_TYPE=$buildtype" \
-  "-DCMAKE_EXPORT_COMPILE_COMMANDS=1 " \
   "-DCLR_CMAKE_ENABLE_CODE_COVERAGE=$code_coverage" \
+  "-DCMAKE_INSTALL_PREFIX=$__CMakeBinDir" \
   "-DCLR_CMAKE_COMPILER=Clang" \
+  "-DCMAKE_USER_MAKE_RULES_OVERRIDE=" \
   $cmake_extra_defines \
   $__UnprocessedCMakeArgs \
   "$1"
diff --git a/src/pal/tools/gen-buildsys-gcc.sh b/src/pal/tools/gen-buildsys-gcc.sh
index 2faf29a887..911324415e 100755
--- a/src/pal/tools/gen-buildsys-gcc.sh
+++ b/src/pal/tools/gen-buildsys-gcc.sh
@@ -158,13 +158,13 @@ if [ "$build_arch" = "armel" ]; then
     cmake_extra_defines="$cmake_extra_defines -DARM_SOFTFP=1"
 fi
 
-overridefile=gcc-compiler-override.txt
-
 __currentScriptDir="$script_dir"
 
-cmake \
+cmake_command=$(command -v cmake3 || command -v cmake)
+
+# Include CMAKE_USER_MAKE_RULES_OVERRIDE as uninitialized since it will hold its value in the CMake cache otherwise can cause issues when branch switching
+$cmake_command \
   -G "$generator" \
-  "-DCMAKE_USER_MAKE_RULES_OVERRIDE=${__currentScriptDir}/$overridefile" \
   "-DCMAKE_AR=$gcc_ar" \
   "-DCMAKE_LINKER=$gcc_link" \
   "-DCMAKE_NM=$gcc_nm" \
@@ -172,9 +172,10 @@ cmake \
   "-DCMAKE_OBJCOPY=$gcc_objcopy" \
   "-DCMAKE_OBJDUMP=$gcc_objdump" \
   "-DCMAKE_BUILD_TYPE=$buildtype" \
-  "-DCMAKE_EXPORT_COMPILE_COMMANDS=1 " \
   "-DCLR_CMAKE_ENABLE_CODE_COVERAGE=$code_coverage" \
   "-DCLR_CMAKE_COMPILER=GNU" \
+  "-DCMAKE_USER_MAKE_RULES_OVERRIDE=" \
+  "-DCMAKE_INSTALL_PREFIX=$__CMakeBinDir" \
   $cmake_extra_defines \
   "$__UnprocessedCMakeArgs" \
   "$1"
diff --git a/src/pal/tools/gen-buildsys-win.bat b/src/pal/tools/gen-buildsys-win.bat
index a3df018d06..093cf34e7e 100644
--- a/src/pal/tools/gen-buildsys-win.bat
+++ b/src/pal/tools/gen-buildsys-win.bat
@@ -44,8 +44,9 @@ if defined CMakePath goto DoGen
 :: Eval the output from set-cmake-path.ps1
 for /f "delims=" %%a in ('powershell -NoProfile -ExecutionPolicy ByPass "& "%basePath%\set-cmake-path.ps1""') do %%a
 
+
 :DoGen
-"%CMakePath%" "-DCMAKE_USER_MAKE_RULES_OVERRIDE=%basePath%\windows-compiler-override.txt" "-DCMAKE_INSTALL_PREFIX=%__CMakeBinDir%" "-DCLR_CMAKE_HOST_ARCH=%__Arch%" %__ExtraCmakeParams% -G "%__CmakeGenerator%" %__SourceDir%
+"%CMakePath%" -DCMAKE_USER_MAKE_RULES_OVERRIDE= "-DCMAKE_INSTALL_PREFIX=%__CMakeBinDir%" "-DCLR_CMAKE_HOST_ARCH=%__Arch%" %__ExtraCmakeParams% -G "%__CmakeGenerator%" %__SourceDir%
 endlocal
 GOTO :DONE
 
diff --git a/src/pal/tools/windows-compiler-override.txt b/src/pal/tools/windows-compiler-override.txt
deleted file mode 100644
index 8ed5c5b125..0000000000
--- a/src/pal/tools/windows-compiler-override.txt
+++ /dev/null
@@ -1,16 +0,0 @@
-SET (CMAKE_C_FLAGS_INIT                "/Wall /FC /Zc:strictStrings")
-SET (CMAKE_C_FLAGS_DEBUG_INIT          "/Od /Zi")
-SET (CLR_C_FLAGS_CHECKED_INIT          "/O1 /Zi")
-SET (CMAKE_C_FLAGS_RELEASE_INIT        "/Ox /Zi")
-SET (CMAKE_C_FLAGS_RELWITHDEBINFO_INIT "/O2 /Zi")
-
-SET (CMAKE_CXX_FLAGS_INIT                "/Wall /FC /Zc:strictStrings")
-SET (CMAKE_CXX_FLAGS_DEBUG_INIT          "/Od /Zi")
-SET (CLR_CXX_FLAGS_CHECKED_INIT          "/O1 /Zi")
-SET (CMAKE_CXX_FLAGS_RELEASE_INIT        "/Ox /Zi")
-SET (CMAKE_CXX_FLAGS_RELWITHDEBINFO_INIT "/O2 /Zi")
-
-SET (CLR_DEFINES_DEBUG_INIT              DEBUG _DEBUG _DBG URTBLDENV_FRIENDLY=Checked BUILDENV_CHECKED=1)
-SET (CLR_DEFINES_CHECKED_INIT            DEBUG _DEBUG _DBG URTBLDENV_FRIENDLY=Checked BUILDENV_CHECKED=1)
-SET (CLR_DEFINES_RELEASE_INIT            NDEBUG URTBLDENV_FRIENDLY=Retail)
-SET (CLR_DEFINES_RELWITHDEBINFO_INIT     NDEBUG URTBLDENV_FRIENDLY=Retail)