Florian Hahn 2bdc1a1337 [LV] Use frozen start value for FindLastIV if needed. (#132691) ...

FindLastIV introduces multiple uses of the start value, where in the
original source there was only a single use, when the epilogue is
vectorized.

Each use of undef may produce a different result, so introducing
multiple uses can produce incorrect results when the input is
undef/poison.

If the start value may be undef or poison, freeze it and use the frozen
value, which will be the same at all uses.

See the following scenarios in Alive2:
* Both main and epilogue vector loops execute, go to exit block: https://alive2.llvm.org/ce/z/_TSvRr
* Both main and epilogue vector loops execute, go to scalar loop: https://alive2.llvm.org/ce/z/CsPj5v
* Only epilogue vector loop executes, go to exit block: https://alive2.llvm.org/ce/z/5XqkNV
* Only epilogue vector loop executes, go to scalar loop: https://alive2.llvm.org/ce/z/JUpqRN

The latter 2 show requiring freezing the resume phi. That means we cannot freeze 
in the preheader. We could move the freeze to the main iteration count check, but
that would be a bit fragile to find and other transforms can sink the freeze if needed.


Depends on https://github.com/llvm/llvm-project/pull/132689
and https://github.com/llvm/llvm-project/pull/132690.

Fixes https://github.com/llvm/llvm-project/issues/126836

PR: https://github.com/llvm/llvm-project/pull/132691

2025-04-04 11:48:01 +01:00

..

SandboxVectorizer

[SandboxVec][NFC] Replace std::regex with llvm::Regex (#134110)

2025-04-02 13:46:56 -07:00

CMakeLists.txt

[VPlan] Refactor VPlan creation, add transform introducing region (NFC). (#128419)

2025-03-09 15:05:35 +00:00

LoadStoreVectorizer.cpp

[Vectorize] Avoid repeated hash lookups (NFC) (#126345)

2025-02-08 00:48:51 -08:00

LoopIdiomVectorize.cpp

[AArch64] Add MATCH loops to LoopIdiomVectorizePass (#101976)

2025-02-10 08:23:34 +00:00

LoopVectorizationLegality.cpp

[NFC]Make file-local cl::opt global variables static (#126486)

2025-03-03 13:46:33 +07:00

LoopVectorizationPlanner.h

[VPlan] Add VF as operand to VPScalarIVStepsRecipe.

2025-03-28 21:48:59 +00:00

LoopVectorize.cpp

[LV] Use frozen start value for FindLastIV if needed. (#132691)

2025-04-04 11:48:01 +01:00

SLPVectorizer.cpp

[SLP]Initial support for (masked)loads + compress and (masked)interleaved

2025-04-03 13:17:40 -07:00

VectorCombine.cpp

[Intrinsics] Add Intrinsic::getFnAttributes() (NFC) (#132029)

2025-03-20 09:20:39 +01:00

Vectorize.cpp

…

VPlan.cpp

[VPlan] Remove dead UF argument from VPTransformState ctor (NFC).

2025-03-30 17:31:00 +01:00

VPlan.h

[VPlan] Update VPInterleaveRecipe to take debug loc directly as arg (NFC)

2025-04-02 22:46:38 +01:00

VPlanAnalysis.cpp

[VPlan] Manage FindLastIV start value in ComputeFindLastIVResult (NFC) (#132690)

2025-03-27 18:34:13 +00:00

VPlanAnalysis.h

…

VPlanCFG.h

[VPlan] Construct immutable VPIRBBs for exit blocks at construction(NFC) (#128374)

2025-02-25 14:23:27 +00:00

VPlanConstruction.cpp

[VPlan] Refactor VPlan creation, add transform introducing region (NFC). (#128419)

2025-03-09 15:05:35 +00:00

VPlanDominatorTree.h

[VPlan] Restrict hoisting of broadcast operations using VPDominatorTree (#117138)

2025-03-13 07:16:04 -07:00

VPlanHCFGBuilder.cpp

[VPlan] Refactor VPlan creation, add transform introducing region (NFC). (#128419)

2025-03-09 15:05:35 +00:00

VPlanHCFGBuilder.h

[VPlan] Don't print VPlan DT after VPlan construction. (NFC)

2025-03-01 21:15:56 +00:00

VPlanHelpers.h

[VPlan] Remove dead UF argument from VPTransformState ctor (NFC).

2025-03-30 17:31:00 +01:00

VPlanPatternMatch.h

[VPlan] Add m_Deferred. NFC (#133736)

2025-03-31 21:01:28 +01:00

VPlanRecipes.cpp

[LV] Use frozen start value for FindLastIV if needed. (#132691)

2025-04-04 11:48:01 +01:00

VPlanSLP.cpp

[VPlan] Move auxiliary declarations out of VPlan.h (NFC). (#124104)

2025-02-02 13:44:07 +00:00

VPlanSLP.h

[llvm] Use *Set::insert_range (NFC) (#133353)

2025-03-27 20:44:20 -07:00

VPlanTransforms.cpp

[VPlan] Update VPInterleaveRecipe to take debug loc directly as arg (NFC)

2025-04-02 22:46:38 +01:00

VPlanTransforms.h

[VPlan] Add transformation to narrow interleave groups.

2025-03-20 19:41:37 +00:00

VPlanUnroll.cpp

[VPlan] Manage FindLastIV start value in ComputeFindLastIVResult (NFC) (#132690)

2025-03-27 18:34:13 +00:00

VPlanUtils.cpp

[VPlan] Pass some functions directly to all_of (NFC).

2025-03-13 18:50:11 +00:00

VPlanUtils.h

[VPlan] Remove duplicated VPDerivedIVRecipe handling (NFC).

2025-03-30 22:27:44 +01:00

VPlanValue.h

[VPlan] Remove remaining references to VPScalarPHIRecipe (NFC).

2025-03-24 19:37:00 +00:00

VPlanVerifier.cpp

[VPlan] Add new VPIRPhi overlay for VPIRInsts wrapping phi nodes (NFC). (#129387)

2025-03-28 08:43:46 +00:00

VPlanVerifier.h

…

VPRecipeBuilder.h

[VPlan] Remove fixHeaderPhis (NFC).

2025-02-23 10:51:20 +00:00