Commit Graph

237 Commits

Author SHA1 Message Date
Florian Hahn
fbcf8a8cbb [ConstraintElim] Add (UGE, var, 0) to unsigned system for new vars. (#76262)
The constraint system used for ConstraintElimination assumes all
varibles to be signed. This can cause missed optimization in the
unsigned system, due to missing the information that all variables are
unsigned (non-negative).

Variables can be marked as non-negative by adding Var >= 0 for all
variables. This is done for arguments on ConstraintInfo construction and
after adding new variables. This handles cases like the ones outlined in
https://discourse.llvm.org/t/why-does-llvm-not-perform-range-analysis-on-integer-values/74341

The original example shared above is now handled without this change,
but adding another variable means that instcombine won't be able to
simplify examples like https://godbolt.org/z/hTnra7zdY

Adding the extra variables comes with a slight compile-time increase
https://llvm-compile-time-tracker.com/compare.php?from=7568b36a2bc1a1e496ec29246966ffdfc3a8b87f&to=641a47f0acce7755e340447386013a2e086f03d9&stat=instructions:u

stage1-O3    stage1-ReleaseThinLTO    stage1-ReleaseLTO-g  stage1-O0-g
 +0.04%           +0.07%                   +0.05%           +0.02%
stage2-O3    stage2-O0-g    stage2-clang
  +0.05%         +0.05%        +0.05%

https://github.com/llvm/llvm-project/pull/76262
2023-12-23 15:53:48 +01:00
Florian Hahn
68fb3d596e [ConstraintElim] Add test with select where the second op cant be poison.
Extra test for TODO from #75750.
2023-12-21 09:13:33 +00:00
Florian Hahn
18170d0f28 [ConstraintElim] Extend AND implication logic to support OR as well. (#76044)
Extend the logic check if an operand of an AND is implied by the other
to also support OR. This is done by checking if !op1 implies op2 or vice
versa.
2023-12-20 18:13:41 +01:00
Florian Hahn
7cf499c63b [ConstraintElim] Check if second op implies first for And. (#75750)
Generalize checkAndSecondOpImpliedByFirst to also check if the second
operand implies the first.
2023-12-20 11:58:35 +01:00
Florian Hahn
c014454f43 [ConstraintElim] Add extra tests with AND and OR of conditions.
Add additional tests where one of the operands of the AND/OR implies the
other.
2023-12-17 21:08:26 +00:00
Yingwei Zheng
26fbdff458 [ConstraintElim] Refactor checkCondition. NFC. (#75319)
This patch refactors `checkCondition` to handle min/max intrinsic calls
in #75306.
2023-12-13 23:20:01 +08:00
Nikita Popov
ea4ce16da2 [ConstraintElim] Use disjoint flag for decomposition (#74478)
Use the or disjoint flag for decomposing or into add, which will handle
cases that haveNoCommonBitsSet() may not be able to reinfer (e.g.
because they require context-sensitive facts, which the call here does
not use.)
2023-12-06 10:36:55 +01:00
Florian Hahn
d045e23c2d [ConstraintElim] Refactor GEP offset collection.
Move GEP offset collection to separate helper function and collect
variable and constant offsets in OffsetResult. For now, this only
supports 1 VariableOffset, but the new code structure can be more easily
extended to handle more offsets in the future.

The refactoring drops the check that the VariableOffset >= -1 * constant
offset. This is not needed to check whether the constraint is
monotonically increasing. The constant factors can be ignored, the
constraint will be monotonically increasing if all variables are
positive.

See https://alive2.llvm.org/ce/z/ah2uSQ,
    https://alive2.llvm.org/ce/z/NCADNZ
2023-11-27 09:05:58 +00:00
Florian Hahn
17a1104d81 [ConstraintElim] Add tests with GEPs with variable and negative offsets
Extra test coverage for precondition generation.
2023-11-23 20:53:02 +00:00
Florian Hahn
a9ea02d679 Recommit "[ConstraintElim] Treat ConstantPointerNull as constant offset 0."
This reverts commit 58286f9c66.

Now with the test correctly updated.

Original message:
    Treat ConstantPointerNull (null) as constant offset 0 in the constraint
    instead of a variable. This slightly reduces the number of variables. I
    was not able to find a test case where this actually caused any changes.
2023-11-23 15:56:37 +00:00
Florian Hahn
4ccdab3636 [ConstraintElim] Use isKnownNonNegative for condition transfer. (#72879)
Use isKnownNonNegative for information transfer. This can improve
results, in cases where ValueTracking can infer additional non-negative
info, e.g. for phi nodes.

This allows simplifying the check from
https://github.com/llvm/llvm-project/issues/63126 by
ConstraintElimination. It is also simplified by IndVarSimplify now; note
the changes in llvm/test/Transforms/PhaseOrdering/loop-access-checks.ll,
due to this now being simplified earlier.
2023-11-21 10:09:35 +00:00
Florian Hahn
97a6c1534f [ConstraintElim] Add extra info transfer tests for #72879.
Extend test coverage to include all predicates we support transfers for.
2023-11-20 22:42:30 +00:00
Florian Hahn
26ab444e88 [ConstraintElim] Make sure add-rec is for the current loop.
Update addInfoForInductions to also check if the add-rec is for the
current loop. Otherwise we might add incorrect facts or crash.

Fixes a miscompile & crash introduced by 00396e6a1a.
2023-11-08 14:07:28 +00:00
Florian Hahn
0d48a4645d [ConstraintElim] Add test for mis-compile with adjacent loops.
In the added test case, the AddRec from the first loop is used
incorrectly in the second loop.
2023-11-08 14:01:25 +00:00
Florian Hahn
4ae4167930 [ConstraintElim] Add tests that require (UGE, Variable, 0) info.
Inspired by
https://discourse.llvm.org/t/why-does-llvm-not-perform-range-analysis-on-integer-values/74341
and https://github.com/llvm/llvm-project/issues/63490.
2023-11-03 14:54:48 +00:00
Florian Hahn
ab6bd9436a [ConstraintElim] Add tests for additional SGT->UGT transfer.
Test cases inspired by
https://github.com/llvm/llvm-project/issues/63126.
2023-11-03 13:38:39 +00:00
Yingwei Zheng
4acc357a37 [ConstraintElim] Add missing check to make sure the bound is loop-invariant (#70555)
This PR adds a missing check for
e6a1657fa3
to make sure the bound is loop-invariant.

Example:
```
define i32 @main() {
entry:
  br label %for.header

for.header:
  %ind1 = phi i64 [ %ind1.i, %for.latch ], [ 0, %entry ]
  %ind2 = phi i64 [ %ind2.i, %for.latch ], [ 5, %entry ]
  %rem = srem i64 %ind2, 4
  %cmp6 = icmp eq i64 %ind1, %rem
  br i1 %cmp6, label %exit1, label %for.latch

for.latch:
  %ind2.i = add i64 %ind2, 1
  %ind1.i = add i64 %ind1, 1
  %cond = icmp eq i64 %ind1.i, 8
  br i1 %cond, label %exit2, label %for.header

exit2:
  %cmp = icmp eq i64 %rem, 0
  %ret = zext i1 %cmp to i32
  ret i32 %ret

exit1:
  ret i32 0
}
```
We cannot infer `%ind1 <u %rem` from `icmp eq i64 %ind1, %rem` in the
loop since `%rem` is not a loop-invariant.

Fixes #70510.
2023-11-02 23:38:05 +08:00
Nikita Popov
1d43096e16 [ConstraintElim] Don't decompose values wider than 64 bits (#68803)
Our coefficients are 64-bits, so adding/multiplying them can wrap in
64-bits even if there would be no wrapping the full bit width.

The alternative would be to check for overflows during all adds/muls in
decomposition. I assume that we don't particularly care about handling
wide integers here, so I've opted to bail out.

Fixes https://github.com/llvm/llvm-project/issues/68751.
2023-10-16 12:03:49 +02:00
Nikita Popov
2b74db6c9b [ConstraintElim] Add test for #68751 (NFC) 2023-10-11 15:52:46 +02:00
Florian Hahn
56a3e49a00 [ConstraintElim] Support decrementing inductions with step -1. (#68644)
Extend the logic in addInfoForInductions to support decrementing
inductions with a step of -1.

Fixes #64881.
2023-10-10 09:37:27 -07:00
Florian Hahn
00396e6a1a [ConstraintElim] Support arbitrary incoming values for inductions (#68032)
Support arbitray incoming values for AddRecs by getting the loop
predecessor and checking if its SCEV matches the AddRec start.

This is done after the existing check, which can help to catch cases
where the expression gets simplified by SCEV to either an IR constant or
existing value which can be used instead.
2023-10-03 12:27:56 +01:00
Florian Hahn
e9b33d085d [ConstraintElim] Add extra tests with nested loops and iv decrements.
Add extra test coverage for induction logic to cover nested loops and
loops with induction decrements. This adds coverage for upcoming
patches.
2023-10-02 12:51:51 +01:00
Florian Hahn
98e016d997 [ConstraintElim] Handle trivial (ICMP_ULE, 0, B) in doesHold.
D152730 may add trivial pre-conditions of the form (ICMP_ULE, 0, B),
which won't be handled automatically by the constraint system, because
we don't add Var >= 0 for all variables in the unsigned system.

Handling the trivial condition explicitly here avoids having the
increase the number of rows in the system per variable.

https://alive2.llvm.org/ce/z/QC92ur

Depends on D152730.

Fixes #63125.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D158776
2023-09-27 12:11:28 +01:00
Florian Hahn
e6a1657fa3 [ConstraintElim] Add A < B if A is an increasing phi for A != B.
This patch adds additional logic to add additional facts for A != B, if
A is a monotonically increasing induction phi. The motivating use case
for this is removing checks when using iterators with hardened libc++,
e.g. https://godbolt.org/z/zhKEP37vG.

The patch pulls in SCEV to detect AddRecs. If possible, the patch adds
the following facts for a AddRec phi PN with StartValue as incoming
value from the loo preheader and B being an upper bound for PN from a
condition in the loop header.

 * (ICMP_UGE, PN, StartValue)
 * (ICMP_ULT, PN, B) [if (ICMP_ULE, StartValue, B)]

The patch also adds an optional precondition to FactOrCheck (the new
DoesHold field) , which can be used to only add a fact if the
precondition holds at the point the fact is added to the constraint
system.

Depends on D151799.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D152730
2023-09-27 11:00:28 +01:00
Antonio Frighetto
241848b9b1 [ConstraintElimination] Extend unsigned-to-signed fact transfer
Minor addition of transferring unsigned facts to signed comparisons.

Proofs: https://alive2.llvm.org/ce/z/nqqzHb
2023-09-23 08:54:19 +00:00
Antonio Frighetto
681315d113 [ConstraintElimination] Introduce tests for PR66173
Introduce test cases for extending
unsigned-to-signed transfer logic.
2023-09-23 08:54:08 +00:00
Florian Hahn
39d7f700ce [ConstraintElim] Support adding facts from switch terminators. (#67061)
After 4a5bcbd560, switch instructions can now be handled in a
straight-forward manner by adding (ICMP_EQ, ConditionVal, CaseVal) for
te successor blocks per case.
2023-09-22 11:18:42 +01:00
Florian Hahn
0cb3530680 [ConstraintElim] Add additional switch case and use i8 instead of i32.
Shorten the types used to i8 for cheaper verification and add test case
where 2 cases have the same destination, as suggested in #67061.
2023-09-22 11:11:46 +01:00
Florian Hahn
2d14317a9d [ConstraintElim] Add extra switch test coverage. 2023-09-21 21:32:50 +01:00
Yingwei Zheng
821c332b53 [ConstraintElim] Add missing checks in test_overflow_in_negate_constraint (#65841)
This patch adds missing checks in the function
`test_overflow_in_negate_constraint`.
Related commit: 0a0181dc20
2023-09-11 02:52:26 +08:00
Florian Hahn
4a5bcbd560 [ConstraintElim] Store conditional facts as (Predicate, Op0, Op1).
This allows to add facts even if no corresponding ICmp instruction
exists in the IR.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D158837
2023-08-30 10:54:28 +01:00
Erik Desjardins
df112cba03 [Tests][ConstraintElim] autogen newly-added case in large-constant-ints.ll (NFC)
I forgot to do this in 66ec5df3a7 / https://reviews.llvm.org/D158810.

Since this is testing for an assertion failure, the test checks don't matter, but we might as well avoid unnecessary churn the next time someone modifies this test.
2023-08-25 17:29:40 -04:00
Erik Desjardins
66ec5df3a7 [ConstraintElim] fix crash with large constants in mul nsw
Another case of https://github.com/llvm/llvm-project/issues/55085.

The added test would trip an assertion due to calling `getSExtValue()` on a value that doesn't fit in int64_t.

Differential Revision: https://reviews.llvm.org/D158810
2023-08-25 13:20:02 -04:00
Florian Hahn
be5013a0a2 [ConstraintElim] Add some tests with inductions that may wrap.
Extra tests for D152730, D158777.
2023-08-25 11:49:52 +01:00
Florian Hahn
13ffde316a [ConstraintElim] Remove dead compares after simplification.
Remove compares after replacing all uses. Cleaning dead compares can
enable additional simplifications when adjusting the position of the
pass slightly. In particular, it seems like the additional dead
instructions may prevent SimplifyCFG performing some folds.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D158760
2023-08-24 22:12:57 +01:00
Florian Hahn
9851edfc6c [ConstraintElim] Add ptr phi tests without inbounds.
Extra test coverage for D152730.
2023-08-17 11:09:56 +01:00
Florian Hahn
ca0d3c99e1 [ConstraintElim] Add bottom tested loops with EQ predicates.
Extra test coverage for D152730.
2023-08-17 11:09:19 +01:00
Yingwei Zheng
92a11eb32c [ConstraintElim] Add facts implied by MinMaxIntrinsic
Fixes https://github.com/llvm/llvm-project/issues/63896 and https://github.com/rust-lang/rust/issues/113757.
This patch adds facts implied by llvm.smin/smax/umin/umax intrinsics.

Reviewed By: fhahn

Differential Revision: https://reviews.llvm.org/D155412
2023-07-24 15:03:34 +08:00
Yingwei Zheng
de0d27c1a4 [ConstraintElim] Add test cases from PR63896. NFC.
This patch adds some test cases from https://github.com/llvm/llvm-project/issues/63896.

Reviewed By: fhahn

Differential Revision: https://reviews.llvm.org/D155853
2023-07-24 13:57:00 +08:00
Yingwei Zheng
cce5324994 [ConstraintElim] Store the triple Pred + LHS + RHS in ReproducerEntry instead of CmpInst + Not
This patch represents a condition with `Pred + LHS + RHS` in ReproducerEntry instead of `CmpInst + Not`.
It avoids creating temporary ICmpInsts in D155412.

Reviewed By: nikic, fhahn

Differential Revision: https://reviews.llvm.org/D155782
2023-07-20 23:21:36 +08:00
Florian Hahn
e1e7b01935 [ConstraintElim] Add phi tests with multiple GEP indices.
Extra tests for D152730.
2023-06-30 16:28:37 +01:00
Florian Hahn
cbacab26e7 [ConstraintElim] Add extra tests and split up test file.
Extra tests for D152730, including more tests where the step GEP doesn't
use the phi and inbounds are missing.

Also split up test file to make it easier to verify for Alive2.
2023-06-30 15:16:23 +01:00
Antonio Frighetto
a2ba4e8075 [ConstraintElimination] Handle solving-only ICMP_NE predicates
Simplification of non-equality predicates for solving constraint
systems is now supported by checking the validity of related
inequalities and equalities.

Differential Revision: https://reviews.llvm.org/D152684
2023-06-29 21:22:48 +02:00
Florian Hahn
d34bc66ef9 [ConstraintElim] Add ptr phi tests with upper bounds with const offsets.
Extra tests for D152730.
2023-06-29 11:18:44 +01:00
Florian Hahn
c92c9462a0 [ConstraintElim] Add pointer induction tests with struct types.
Extra tests for D152730.
2023-06-29 09:35:36 +01:00
Florian Hahn
36999318f0 [ConstraintElim] Add tests with phis and different alloc sizes/end ptrs.
Extra tests for D152730
2023-06-28 23:12:06 +01:00
Florian Hahn
82aeec6570 [ConstraintElim] Add additional induction phi tests with end argument.
Extra tests for D152730 with different GEP step sizes and the end pointer
being an argument.
2023-06-28 21:10:44 +01:00
Florian Hahn
0ad6879a62 [ConstraintElim] Try to use first cmp to prove second cmp for ANDs.
This patch extends the existing logic to handle cases where we have
branch conditions of the form (AND icmp, icmp) where the first icmp
implies the second. This can improve results in some cases, e.g. if
SimplifyCFG folded conditions from multiple branches to an AND.

The implementation handles this by adding a new type of check
(AndImpliedCheck), which are queued before conditional facts for the same
block.

When encountering AndImpliedChecks during solving, the first condition
is optimistically added to the constraint system, then we check if the
second icmp can be simplified, and finally the newly added entries are
removed.

The reason for doing things this way is to avoid clashes with signed
<-> unsigned condition transfer, which require us to re-order facts to
increase effectiveness.

Reviewed By: nikic, antoniofrighetto

Differential Revision: https://reviews.llvm.org/D151799
2023-06-28 13:19:41 +01:00
Florian Hahn
6c25c58a4d [ConstraintElim] Track and simplify conditions at use.
This patch updates ConstraintElimination to track uses of conditions in
the worklist. This allows simplifying conditions using the context that
holds directly at the condition, instead of where the condition is
defined.

This allows us to catch more cases in practice: there are multiple
code-size changes for CTMark while compile-time remains unchanged:
https://llvm-compile-time-tracker.com/compare.php?from=4b020cca9363bebab4643f89cfa92ab2fcc7976c&to=7a6e84b8f029713c137814cd46eec775d24a54a1&stat=instructions:u

This should help to simplify D151799.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D153660
2023-06-27 17:27:49 +01:00
Florian Hahn
72c826cf6c [ConstraintElim] Add additional induction phi tests.
Extra tests with different GEP step sizes for D152730.
2023-06-27 12:22:11 +01:00