Florian Hahn 8df64ed777 [LV] Don't consider IV increments uniform if exit value is used outside. ...

In some cases, there might be a chain of uniform instructions producing
the exit value. To generate correct code in all cases, consider the IV
increment not uniform, if there are users outside the loop.

Instead, let VPlan narrow the IV, if possible using the logic from
3ff1d01985.

Test case from #122602 verified with Alive2:
    https://alive2.llvm.org/ce/z/bA4EGj

Fixes https://github.com/llvm/llvm-project/issues/122496.
Fixes https://github.com/llvm/llvm-project/issues/122602.

2025-01-12 22:03:21 +00:00

..

CostModel

[LoopVectorize] Teach some X86 cost model tests to use new vplan costs (#120738)

2024-12-20 15:08:08 +00:00

already-vectorized.ll

[LV] Disable runtime unrolling for vectorized loops.

2023-01-06 10:56:17 +00:00

amdlibm-calls-finite.ll

Adding more vector calls for -fveclib=AMDLIBM (#109662)

2024-10-29 10:09:55 +00:00

amdlibm-calls.ll

Adding more vector calls for -fveclib=AMDLIBM (#109662)

2024-10-29 10:09:55 +00:00

avx1.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

avx512.ll

BlockFrequencyInfoImpl: Avoid big numbers, increase precision for small spreads

2023-10-24 20:27:39 -07:00

consecutive-ptr-cg-bug.ll

[llvm] Remove br i1 undef from some regression tests [NFC] (#117112)

2024-11-21 08:06:56 +00:00

consecutive-ptr-uniforms.ll

[LV] Use IVUpdateMayOverflow to set HasNUW. (#111758)

2024-11-28 10:12:41 +00:00

constant-fold.ll

[VPlan] Remove loop region in optimizeForVFAndUF. (#108378)

2025-01-05 15:50:42 +00:00

constant-vector-operand.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

conversion-cost.ll

[VPlan] Compute induction end values in VPlan. (#112145)

2024-12-29 19:05:08 +00:00

cost-constant-known-via-scev.ll

[LLVM][IR] Use splat syntax when printing Constant[Data]Vector. (#112548)

2024-11-06 11:53:33 +00:00

cost-model-assert.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:27:31 +01:00

cost-model.ll

[VPlan] Compute induction end values in VPlan. (#112145)

2024-12-29 19:05:08 +00:00

divs-with-tail-folding.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

drop-inbounds-flags-for-reverse-vector-pointer.ll

[LV] Drop incorrect inbounds for reverse vector pointer when folding tail (#120730)

2025-01-07 06:14:01 -08:00

drop-poison-generating-flags.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

ephemeral-recipes.ll

…

epilog-vectorization-inductions.ll

[VPlan] Compute induction end values in VPlan. (#112145)

2024-12-29 19:05:08 +00:00

fixed-order-recurrence.ll

[VPlan] Compute induction end values in VPlan. (#112145)

2024-12-29 19:05:08 +00:00

float-induction-x86.ll

[VPlan] Compute induction end values in VPlan. (#112145)

2024-12-29 19:05:08 +00:00

fneg-cost.ll

[LoopVectorize][NFC] Rewrite tests to check output of vplan cost model (#113697)

2024-11-19 08:55:39 +00:00

fp32_to_uint32-cost-model.ll

…

fp64_to_uint32-cost-model.ll

…

fp80-widest-type.ll

…

fp_to_sint8-cost-model.ll

…

funclet.ll

[LV] Check for vector instruction in main vector loop.

2023-07-03 14:16:47 +01:00

gather_scatter.ll

[VPlan] Compute induction end values in VPlan. (#112145)

2024-12-29 19:05:08 +00:00

gather-cost.ll

…

gather-vs-interleave.ll

…

gcc-examples.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

gep-use-outside-loop.ll

Recommit "[VPlan] Try to narrow wide and replicating recipes to uniform recipes."

2025-01-12 20:10:28 +00:00

i1-reg-usage.ll

[NFC] Port all runlines for LoopVectorize pass tests to -passes syntax

2022-12-05 22:17:30 +03:00

illegal-parallel-loop-uniform-write.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

imprecise-through-phis.ll

[LLVM][IR] Use splat syntax when printing Constant[Data]Vector. (#112548)

2024-11-06 11:53:33 +00:00

induction-costs.ll

[VPlan] Compute induction end values in VPlan. (#112145)

2024-12-29 19:05:08 +00:00

induction-step.ll

[VPlan] Convert induction increment check to be VPlan-based.

2025-01-05 11:16:01 +00:00

int128_no_gather.ll

…

interleave_short_tc.ll

…

interleave-cost.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

interleave-opaque-pointers.ll

[VPlan] Don't created GEP x, 0 for interleave group pointers.

2024-10-08 12:08:13 +01:00

interleave-ptradd-with-replicated-operand.ll

[VPlan] Request lane 0 for pointer arg in PtrAdd.

2024-10-09 13:18:54 +01:00

interleaved-accesses-hoist-load-across-store.ll

[VPlan] Don't created GEP x, 0 for interleave group pointers.

2024-10-08 12:08:13 +01:00

interleaved-accesses-large-gap.ll

…

interleaved-accesses-sink-store-across-load.ll

[LLVM][IR] Use splat syntax when printing Constant[Data]Vector. (#112548)

2024-11-06 11:53:33 +00:00

interleaved-accesses-use-after-free.ll

…

interleaved-accesses-waw-dependency.ll

…

interleaving.ll

[InstCombine] Infer nusw + nneg -> nuw for getelementptr (#111144)

2024-12-05 14:36:40 +01:00

intrinsiccost.ll

[VPlan] Compute induction end values in VPlan. (#112145)

2024-12-29 19:05:08 +00:00

invariant-load-gather.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

invariant-store-vectorization.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

iv-live-outs.ll

[VPlan] Unify mayWriteToMemory and mayHaveSideEffects logic for VPInst.

2024-09-26 19:16:43 +01:00

libm-vector-calls-finite.ll

…

libm-vector-calls-VF2-VF8.ll

…

libm-vector-calls.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

limit-vf-by-tripcount.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

lit.local.cfg

[NFC][Py Reformat] Reformat lit.local.cfg python files in llvm

2023-05-17 17:03:15 +02:00

load-deref-pred.ll

[LLVM][IR] Use splat syntax when printing Constant[Data]Vector. (#112548)

2024-11-06 11:53:33 +00:00

masked_load_store.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

masked-store-cost.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

max-mstore.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

metadata-enable.ll

[VPlan] Propagate all GEP flags (#119899)

2024-12-17 13:48:50 +01:00

min-trip-count-switch.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

mul_slm_16bit.ll

[LoopVectorize][NFC] Rewrite tests to check output of vplan cost model (#113697)

2024-11-19 08:55:39 +00:00

multi-exit-cost.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

no_fpmath_with_hotness.ll

…

no_fpmath.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

no-vector.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

nontemporal.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

optsize.ll

[VPlan] Compute induction end values in VPlan. (#112145)

2024-12-29 19:05:08 +00:00

outer_loop_test1_no_explicit_vect_width.ll

[LLVM][IR] Use splat syntax when printing Constant[Data]Vector. (#112548)

2024-11-06 11:53:33 +00:00

parallel-loops-after-reg2mem.ll

…

parallel-loops.ll

[InstCombine] Set disjoint flag when turning Add into Or. (#72702)

2023-11-27 12:54:11 -08:00

pointer-runtime-checks-unprofitable.ll

[LV] Add uses of result to pointer-runtime-checks-unprofitable.ll test.

2024-07-15 09:59:46 +01:00

powof2div.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

pr23997.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

pr34438.ll

[VPlan] Remove loop region in optimizeForVFAndUF. (#108378)

2025-01-05 15:50:42 +00:00

pr35432.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

pr36524.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

pr39160.ll

[llvm] Remove br i1 undef from some regression tests [NFC] (#117112)

2024-11-21 08:06:56 +00:00

pr42674.ll

[ConstantFolding] Infer getelementptr nuw flag (#119214)

2024-12-09 16:44:05 +01:00

pr47437.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

pr48340.ll

[LLVM][IR] Use splat syntax when printing Constant[Data]Vector. (#112548)

2024-11-06 11:53:33 +00:00

pr51366-sunk-instruction-used-outside-of-loop.ll

[LLVM][IR] Use splat syntax when printing Constant[Data]Vector. (#112548)

2024-11-06 11:53:33 +00:00

pr54413-select-interleave-count-loop-with-cost-zero.ll

…

pr54634.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

pr55096-scalarize-add.ll

Reapply "[VPlan] Update scalar induction resume values in VPlan. (#110577)"

2024-12-06 19:41:51 +00:00

pr56319-vector-exit-cond-optimization-epilogue-vectorization.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

pr72969.ll

[VPlan] Compute induction end values in VPlan. (#112145)

2024-12-29 19:05:08 +00:00

pr81872.ll

[LV] Use IVUpdateMayOverflow to set HasNUW. (#111758)

2024-11-28 10:12:41 +00:00

pr109581-unused-blend.ll

[VPlan] Handle some VPInstructions in may{Read,Write}FromMemory (#120058)

2025-01-08 15:17:26 +08:00

predicate-switch.ll

[LLVM][IR] Use splat syntax when printing Constant[Data]Vector. (#112548)

2024-11-06 11:53:33 +00:00

predicated-instruction-cost.ll

[LV] Generalize predication checks from 2c8836c899 for operands.

2024-10-02 20:16:41 +01:00

propagate-metadata.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2023-01-04 17:25:42 +01:00

ptr-indvar-crash.ll

…

rauw-bug.ll

[llvm] Remove br i1 undef from some regression tests [NFC] (#117112)

2024-11-21 08:06:56 +00:00

reduction-crash.ll

[llvm] Remove br i1 undef from some regression tests [NFC] (#117112)

2024-11-21 08:06:56 +00:00

reduction-fastmath.ll

[LLVM][IR] Use splat syntax when printing Constant[Data]Vector. (#112548)

2024-11-06 11:53:33 +00:00

reduction-small-size.ll

[VPlan] Remove stray space when printing VPWidenCastRecipe.

2024-12-24 20:23:48 +00:00

redundant-vf2-cost.ll

Fix test failures introduced by PR #113697 (#116941)

2024-11-20 09:10:51 +00:00

reg-usage-debug.ll

[LV] Reland "Update logic for calculating register usage due to invariants"

2023-02-28 17:32:39 +05:30

reg-usage.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

register-assumption.ll

…

replicate-recipe-with-only-first-lane-used.ll

[VPlan] Only generate first lane for VPPredInstPHI if no others used.

2024-10-05 19:15:05 +01:00

replicate-uniform-call.ll

[VPlan] Handle some VPInstructions in may{Read,Write}FromMemory (#120058)

2025-01-08 15:17:26 +08:00

runtime-limit.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

scatter_crash.ll

[VPlan] Handle some VPInstructions in may{Read,Write}FromMemory (#120058)

2025-01-08 15:17:26 +08:00

slm-no-vectorize.ll

[LoopVectorize][X86] Regenerate slm-no-vectorize.ll

2023-06-13 14:15:37 +01:00

small-size.ll

[VPlan] Compute induction end values in VPlan. (#112145)

2024-12-29 19:05:08 +00:00

strided_load_cost.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

struct-store.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

svml-calls-finite.ll

…

svml-calls.ll

…

tail_folding_and_assume_safety.ll

…

tail_loop_folding.ll

[LV] Use IVUpdateMayOverflow to set HasNUW. (#111758)

2024-11-28 10:12:41 +00:00

transform-narrow-interleave-to-widen-memory.ll

[LV] Precommit tests for #106441.

2025-01-10 18:49:44 +00:00

tripcount.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

uint64_to_fp64-cost-model.ll

[VPlan] Remove stray space when printing VPWidenCastRecipe.

2024-12-24 20:23:48 +00:00

uniform_load.ll

…

uniform_mem_op.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

uniform-phi.ll

[LV] Don't consider IV increments uniform if exit value is used outside.

2025-01-12 22:03:21 +00:00

uniformshift.ll

[LoopVectorize][NFC] Rewrite tests to check output of vplan cost model (#113697)

2024-11-19 08:55:39 +00:00

unroll_selection.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

unroll-pm.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

unroll-small-loops.ll

…

veclib-calls.ll

…

vect.omp.force.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

vect.omp.force.small-tc.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

vector_max_bandwidth.ll

…

vector_ptr_load_store.ll

…

vector-scalar-select-cost.ll

[LoopVectorize][NFC] Rewrite tests to check output of vplan cost model (#113697)

2024-11-19 08:55:39 +00:00

vectorization-remarks-loopid-dbg.ll

[LV] Remove loop trip count threshold for deciding whether to interleave a loop (#67725)

2024-02-05 17:23:58 -08:00

vectorization-remarks-missed.ll

[LV] Only check isVectorizableEarlyExitLoop with multiple exits. (#121994)

2025-01-09 12:05:19 +00:00

vectorization-remarks-profitable.ll

[LoopVectorize] Convert some tests to opaque pointers (NFC)

2022-12-14 15:16:59 +01:00

vectorization-remarks.ll

[LV] Remove loop trip count threshold for deciding whether to interleave a loop (#67725)

2024-02-05 17:23:58 -08:00

vectorize-force-tail-with-evl.ll

[VPlan] Remove reverse() of predecessors from VPInstruction::generate.

2024-12-17 20:44:32 +00:00

vectorize-interleaved-accesses-gap.ll

[LV] Use IVUpdateMayOverflow to set HasNUW. (#111758)

2024-11-28 10:12:41 +00:00

vectorize-only-for-real.ll

…

vplan-native-inner-loop-only.ll

[LLVM][IR] Use splat syntax when printing Constant[Data]Vector. (#112548)

2024-11-06 11:53:33 +00:00

vplan-vp-intrinsics.ll

[VPlan] Use VPInstruction::Name when assigning names (NFCI).

2024-10-18 05:52:35 +01:00

widened-value-used-as-scalar-and-first-lane.ll

[LLVM][IR] Use splat syntax when printing Constant[Data]Vector. (#112548)

2024-11-06 11:53:33 +00:00

x86_fp80-interleaved-access.ll

…

x86_fp80-vector-store.ll

Revert "[InstCombine] Infer nuw for gep inbounds from base of object" (#120460)

2024-12-18 19:06:34 +01:00

x86-interleaved-accesses-masked-group.ll

[InstCombine] Infer nusw + nneg -> nuw for getelementptr (#111144)

2024-12-05 14:36:40 +01:00

x86-interleaved-store-accesses-with-gaps.ll

[InstCombine] Infer nusw + nneg -> nuw for getelementptr (#111144)

2024-12-05 14:36:40 +01:00

x86-pr39099.ll

[LLVM][IR] Use splat syntax when printing Constant[Data]Vector. (#112548)

2024-11-06 11:53:33 +00:00

x86-predication.ll

[LLVM][IR] Use splat syntax when printing Constant[Data]Vector. (#112548)

2024-11-06 11:53:33 +00:00