Florian Hahn 3bd161e98d [LV] Honor forced scalars in setVectorizedCallDecision. ...

Similarly to dd94537b4, setVectorizedCallDecision also did not consider
ForcedScalars. This lead to VPlans not reflecting the decision by the
legacy cost model (cost computation would use scalar cost, VPlan would
have VPWidenCallRecipe).

To fix this, check if the call has been forced to scalar in
setVectorizedCallDecision.

Note that this requires moving setVectorizedCallDecision after
collectLoopUniforms (which sets ForcedScalars). collectLoopUniforms does
not depend on call decisions and can safely be moved.

Fixes https://github.com/llvm/llvm-project/issues/107051.

2024-09-03 21:06:32 +01:00

..

aarch64-predication.ll

…

aarch64-unroll.ll

…

arbitrary-induction-step.ll

…

arm64-unroll.ll

…

backedge-overflow.ll

…

call-costs.ll

[LV] Honor forced scalars in setVectorizedCallDecision.

2024-09-03 21:06:32 +01:00

clamped-trip-count.ll

Move stepvector intrinsic out of experimental namespace (#98043)

2024-08-28 12:48:20 +01:00

conditional-branches-cost.ll

[VPlan] Skip branches marked as dead in cost precomputation.

2024-08-23 15:58:29 +01:00

cost-no-valid-vplans-built.ll

[VPlan] Return std::nullopt early if plans are empty.

2024-06-27 12:25:59 +01:00

deterministic-type-shrinkage.ll

[InstCombine] Remove some of the complexity-based canonicalization (#91185)

2024-08-21 12:02:54 +02:00

divs-with-scalable-vfs.ll

Move stepvector intrinsic out of experimental namespace (#98043)

2024-08-28 12:48:20 +01:00

eliminate-tail-predication.ll

…

epilog-vectorization-widen-inductions.ll

[VPlan] Explicitly handle scalar pointer inductions. (#83068)

2024-03-26 16:01:57 +01:00

extend-vectorization-factor-for-unprofitable-memops.ll

…

extractvalue-no-scalarization-required.ll

…

first-order-recurrence-fold-tail.ll

[VPlan] Introduce ResumePhi VPInstruction, use to create phi for FOR. (#94760)

2024-07-11 16:08:04 +01:00

first-order-recurrence.ll

Move several vector intrinsics out of experimental namespace (#88748)

2024-04-29 10:16:45 +01:00

fixed-order-recurrence.ll

[VPlan] Introduce ResumePhi VPInstruction, use to create phi for FOR. (#94760)

2024-07-11 16:08:04 +01:00

gather-cost.ll

…

gather-do-not-vectorize-addressing.ll

…

i1-reg-usage.ll

…

induction-costs-sve.ll

[LV] Remove unnecessary getRuntimeVF call when computing vector TC.

2024-06-12 14:35:37 +01:00

induction-costs.ll

[VPlan] Mark VPVectorPointer as only using the first part of the ptr.

2024-08-12 08:46:55 +01:00

induction-trunc.ll

…

interleave_count_for_estimated_tc.ll

…

interleave_count_for_known_tc.ll

…

interleave-allocsize-not-equal-typesize.ll

…

interleaved_cost.ll

[LV] Update tests to not have dead interleave groups.

2024-07-21 14:03:40 +01:00

interleaved-store-of-first-order-recurrence.ll

…

interleaved-vs-scalar.ll

…

interleaving-load-store.ll

…

interleaving-reduction.ll

[VPlan] Model branch cond to enter scalar epilogue in VPlan. (#92651)

2024-07-05 10:08:42 +01:00

intrinsiccost.ll

TTI: Fix special casing vectorization costs of saturating add/sub (#97463)

2024-08-06 17:33:52 +04:00

lit.local.cfg

…

loop-vectorization-factors.ll

[VPlan] Introduce ResumePhi VPInstruction, use to create phi for FOR. (#94760)

2024-07-11 16:08:04 +01:00

loopvectorize_pr33804_double.ll

…

low_trip_memcheck_cost.ll

…

masked-call.ll

[LLVM][VPlan] Pick more optimal initial value for VPBlend. (#104019)

2024-08-30 13:30:23 +01:00

masked-op-cost.ll

[TTI] getCommonMaskedMemoryOpCost - use the target getMemoryOpCost/getCFInstrCost implementations.

2024-05-11 12:50:26 +01:00

max-vf-for-interleaved.ll

…

maximize-bandwidth-invalidate.ll

…

mul-simplification.ll

[VPlan] Simplify MUL operands at recipe construction.

2024-09-02 21:25:31 +01:00

neon-gather-cost.ll

…

no_vector_instructions.ll

…

nontemporal-load-store.ll

…

outer_loop_prefer_scalable.ll

Move stepvector intrinsic out of experimental namespace (#98043)

2024-08-28 12:48:20 +01:00

outer_loop_test1_no_explicit_vect_width.ll

…

Oz-and-forced-vectorize.ll

…

pr31900.ll

…

pr33053.ll

…

pr36032.ll

…

pr46950-load-cast-context-crash.ll

…

pr56627.ll

…

pr60831-sve-inv-store-crash.ll

Move stepvector intrinsic out of experimental namespace (#98043)

2024-08-28 12:48:20 +01:00

pr73894.ll

[VPlan] Remove VPPredInstPHIRecipes without users after region merging.

2024-07-20 13:21:32 +01:00

predication_costs.ll

…

prefer-fixed-if-equal-to-scalable.ll

[LV][AArch64] Prefer Fixed over Scalable if cost-model is equal (Neoverse V2) (#95819)

2024-07-17 10:46:28 +01:00

reduction-recurrence-costs-sve.ll

[LV] Don't predicate divs with invariant divisor when folding tail (#98904)

2024-07-25 12:21:09 +01:00

reduction-small-size.ll

…

reg-usage.ll

…

runtime-check-size-based-threshold.ll

…

runtime-check-trip-count-decisions.ll

…

scalable-alloca.ll

[VPlan] Port invalid cost remarks to VPlan. (#99322)

2024-07-27 12:52:12 +01:00

scalable-avoid-scalarization.ll

Move stepvector intrinsic out of experimental namespace (#98043)

2024-08-28 12:48:20 +01:00

scalable-call.ll

[VPlan] Port invalid cost remarks to VPlan. (#99322)

2024-07-27 12:52:12 +01:00

scalable-predicate-instruction.ll

…

scalable-reduction-inloop-cond.ll

[LV] Prefer FLT_MIN/MAX for fmin/fmax reductions with ninf (#107141)

2024-09-03 12:21:54 -07:00

scalable-reductions-tf.ll

…

scalable-reductions.ll

Prefer use of 0.0 over -0.0 for fadd reductions w/nsz (in IR) (#106770)

2024-09-03 09:16:37 -07:00

scalable-strict-fadd.ll

[LLVM][VPlan] Pick more optimal initial value for VPBlend. (#104019)

2024-08-30 13:30:23 +01:00

scalable-vectorization-cost-tuning.ll

…

scalable-vectorization.ll

…

scalable-vf-hint.ll

…

scalar_interleave.ll

…

scalarize-store-with-predication.ll

…

scatter-cost.ll

[AArch64] Increase scatter overhead on Neoverse-V2 (#101296)

2024-08-14 10:12:40 +05:30

sdiv-pow2.ll

…

select-costs.ll

[LoopVectorize] LLVM fails to vectorise loops with multi-bool varables (#89226)

2024-07-15 20:21:50 +01:00

smallest-and-widest-types.ll

…

sme-vectorize.ll

[AArch64] Consider streaming mode in TTI interfaces for vectorization. (#96305)

2024-06-24 11:06:16 +01:00

store-costs-sve.ll

[LV] Remove unnecessary getRuntimeVF call when computing vector TC.

2024-06-12 14:35:37 +01:00

streaming-compatible-sve-no-maximize-bandwidth.ll

[VPlan] Model branch cond to enter scalar epilogue in VPlan. (#92651)

2024-07-05 10:08:42 +01:00

streaming-vectorization.ll

[LLVM][TTI][SME] Allow optional auto-vectorisation for streaming functions. (#101679)

2024-08-05 11:25:44 +01:00

strict-fadd-cost.ll

…

strict-fadd-vf1.ll

…

strict-fadd.ll

[VPlan] Model branch cond to enter scalar epilogue in VPlan. (#92651)

2024-07-05 10:08:42 +01:00

sve-basic-vec.ll

…

sve-cond-inv-loads.ll

[InstCombine] Remove some of the complexity-based canonicalization (#91185)

2024-08-21 12:02:54 +02:00

sve-epilog-vect-inloop-reductions.ll

[VPlan] Model branch cond to enter scalar epilogue in VPlan. (#92651)

2024-07-05 10:08:42 +01:00

sve-epilog-vect-reductions.ll

[VPlan] Model branch cond to enter scalar epilogue in VPlan. (#92651)

2024-07-05 10:08:42 +01:00

sve-epilog-vect-strict-reductions.ll

[VPlan] Model branch cond to enter scalar epilogue in VPlan. (#92651)

2024-07-05 10:08:42 +01:00

sve-epilog-vect-vscale-tune.ll

…

sve-epilog-vect.ll

[VPlan] Mark VPVectorPointer as only using the first part of the ptr.

2024-08-12 08:46:55 +01:00

sve-extract-last-veclane.ll

…

sve-fneg.ll

…

sve-gather-scatter-cost.ll

…

sve-gather-scatter.ll

Move stepvector intrinsic out of experimental namespace (#98043)

2024-08-28 12:48:20 +01:00

sve-illegal-type.ll

[LoopVectorize][AArch64] Add limited support for scalable vectorisation of i1 types (#95920)

2024-06-25 15:04:24 +01:00

sve-inductions-unusual-types.ll

Move stepvector intrinsic out of experimental namespace (#98043)

2024-08-28 12:48:20 +01:00

sve-inductions.ll

Move stepvector intrinsic out of experimental namespace (#98043)

2024-08-28 12:48:20 +01:00

sve-interleaved-accesses.ll

Prefer use of 0.0 over -0.0 for fadd reductions w/nsz (in IR) (#106770)

2024-09-03 09:16:37 -07:00

sve-interleaved-masked-accesses.ll

Move stepvector intrinsic out of experimental namespace (#98043)

2024-08-28 12:48:20 +01:00

sve-inv-loads.ll

…

sve-inv-store.ll

…

sve-invalid-costs.ll

[LoopVectorize] Fix build issue caused by #95920 (#96647)

2024-06-25 15:51:32 +01:00

sve-large-strides.ll

…

sve-live-out-pointer-induction.ll

[VPlan] Mark VPVectorPointer as only using the first part of the ptr.

2024-08-12 08:46:55 +01:00

sve-low-trip-count.ll

[LV] Remove unnecessary getRuntimeVF call when computing vector TC.

2024-06-12 14:35:37 +01:00

sve-masked-loadstore.ll

…

sve-multiexit.ll

…

sve-runtime-check-size-based-threshold.ll

…

sve-scalable-load-in-loop.ll

…

sve-select-cmp.ll

[VPlan] Model branch cond to enter scalar epilogue in VPlan. (#92651)

2024-07-05 10:08:42 +01:00

sve-strict-fadd-cost.ll

…

sve-tail-folding-cost.ll

…

sve-tail-folding-forced.ll

[LV] Remove unnecessary getRuntimeVF call when computing vector TC.

2024-06-12 14:35:37 +01:00

sve-tail-folding-option.ll

Prefer use of 0.0 over -0.0 for fadd reductions w/nsz (in IR) (#106770)

2024-09-03 09:16:37 -07:00

sve-tail-folding-optsize.ll

[LV] Remove unnecessary getRuntimeVF call when computing vector TC.

2024-06-12 14:35:37 +01:00

sve-tail-folding-overflow-checks.ll

…

sve-tail-folding-reductions.ll

[VPlan] Model branch cond to enter scalar epilogue in VPlan. (#92651)

2024-07-05 10:08:42 +01:00

sve-tail-folding-unroll.ll

[LV] Remove unnecessary getRuntimeVF call when computing vector TC.

2024-06-12 14:35:37 +01:00

sve-tail-folding.ll

[LLVM][VPlan] Pick more optimal initial value for VPBlend. (#104019)

2024-08-30 13:30:23 +01:00

sve-type-conv.ll

…

sve-vector-reverse-mask4.ll

Move several vector intrinsics out of experimental namespace (#88748)

2024-04-29 10:16:45 +01:00

sve-vector-reverse.ll

[InstCombine] Remove some of the complexity-based canonicalization (#91185)

2024-08-21 12:02:54 +02:00

sve-vfabi.ll

…

sve-widen-extractvalue.ll

…

sve-widen-gep.ll

Move stepvector intrinsic out of experimental namespace (#98043)

2024-08-28 12:48:20 +01:00

sve-widen-phi.ll

Move stepvector intrinsic out of experimental namespace (#98043)

2024-08-28 12:48:20 +01:00

synthesize-mask-for-call.ll

[VPlan] Model branch cond to enter scalar epilogue in VPlan. (#92651)

2024-07-05 10:08:42 +01:00

tail-fold-uniform-memops.ll

…

tail-folding-styles.ll

Move stepvector intrinsic out of experimental namespace (#98043)

2024-08-28 12:48:20 +01:00

type-shrinkage-insertelt.ll

…

type-shrinkage-zext-costs.ll

…

uniform-args-call-variants.ll

[InstCombine] Replace all dominated uses of condition with constants (#105510)

2024-09-01 09:49:23 +08:00

unsafe-vf-hint-remark.ll

…

veclib-calls-libsystem-darwin.ll

[aarch64] Add hyperbolic and arc trig intrinsic lowering (#98937)

2024-07-19 10:18:23 -04:00

veclib-function-calls.ll

[AArch64][TLI] Add TLI mappings for ArmPL modf, sincos, sincospi (#83143)

2024-04-10 09:34:46 +01:00

veclib-intrinsic-calls.ll

[aarch64] Add hyperbolic and arc trig intrinsic lowering (#98937)

2024-07-19 10:18:23 -04:00

vector-call-linear-args.ll

…

vector-reverse-mask4.ll

[InstCombine] Remove some of the complexity-based canonicalization (#91185)

2024-08-21 12:02:54 +02:00

vector-reverse.ll

…

widen-call-with-intrinsic-or-libfunc.ll

[VPlan] Model branch cond to enter scalar epilogue in VPlan. (#92651)

2024-07-05 10:08:42 +01:00

wider-VF-for-callinst.ll

…