[OpenMP] Run an extra 'OpenMPOpt' pass in LTO-mode

The `OpenMPOpt` pass is pivotal to the performance of many OpenMP
offloading programs. When we perform non-LTO builds with OpenMP we used
to link the OpenMP deviceRTL individually for each TU. This lead to us
getting an additional attributor run on the combined runtime and user
code. When we used LTO we lost a run and suffered a large performance
degradation. This patch simply adds in the extra `OpenMPOpt` pass that
we miss into the LTO pipeline. This patch fixes the performance
regression shown in applications that used OpenMP offloading in LTO
mode.

Previously, this wasn't legal to do as we could emit new runtime calls
into the module. That was fixed by D142646.

Depends on D142646

Fixes https://github.com/llvm/llvm-project/issues/60300

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D142650

clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

@@ -513,8 +513,6 @@ std::unique_ptr<lto::LTO> createLTO(
   assert(CGOptLevelOrNone && "Invalid optimization level");
   Conf.CGOptLevel = *CGOptLevelOrNone;
   Conf.OptLevel = OptLevel[1] - '0';
-  if (Conf.OptLevel > 0)
-    Conf.UseDefaultPipeline = true;
   Conf.DefaultTriple = Triple.getTriple();
 
   LTOError = false;

llvm/lib/Passes/PassBuilderPipelines.cpp

@@ -1712,6 +1712,9 @@ PassBuilder::buildLTODefaultPipeline(OptimizationLevel Level,
   // Optimize globals again after we ran the inliner.
   MPM.addPass(GlobalOptPass());
 
+  // Run the OpenMPOpt pass again after global optimizations.
+  MPM.addPass(OpenMPOptPass(ThinOrFullLTOPhase::FullLTOPostLink));
+
   // Garbage collect dead functions.
   MPM.addPass(GlobalDCEPass());
 

llvm/test/Other/new-pm-lto-defaults.ll

@@ -78,6 +78,7 @@
 ; CHECK-O23SZ-NEXT: Running pass: InlinerPass
 ; CHECK-O23SZ-NEXT: Invalidating analysis: InlineAdvisorAnalysis
 ; CHECK-O23SZ-NEXT: Running pass: GlobalOptPass
+; CHECK-O23SZ-NEXT: Running pass: OpenMPOptPass
 ; CHECK-O23SZ-NEXT: Running pass: GlobalDCEPass
 ; CHECK-O23SZ-NEXT: Running pass: ArgumentPromotionPass
 ; CHECK-O23SZ-NEXT: Running pass: DeadArgumentEliminationPass