This is a patch to fix duplicated dbg.values in the JumpThreading pass not
pointing towards their local value, and instead towards the variable in the
original block.
JumpThreadingPass::cloneInstructions is the changed function to target metadata
as well as normal cloned values.
Reviewed By: jmorse, StephenTozer
Differential Revision: https://reviews.llvm.org/D140006
Jump threading can replace select and unconditional branch with
conditional branch, but when doing so loses profile information.
This destructive transform can eventually lead to a performance
degradation due to folding of branches in
shouldFoldCondBranchesToCommonDestination as branch probabilities
are no longer known.
Patch by Roman Paukner!
Differential Revision: https://reviews.llvm.org/D138132
Reviewed By: mkazantsev
This is a recurring pattern: We want to find the nearest common
dominator (instruction) for two instructions, but currently only
provide an API for the nearest common dominator of two basic blocks.
Add an overload that accepts and return instructions.
Similar to D138404, we were not guarding against extra uses of the Mul.
In most cases other checks would catch the issue due to unsupported
instructions in the outer loop, but certain non-canonical loop forms
could still get through.
Fixes#59339
Differential Revision: https://reviews.llvm.org/D141114
When fetching allocation sizes, we almost always want to have the
size in bytes, but we were only providing an InBits API. Also add
the corresponding byte-based conjugate to save some *8 and /8
juggling everywhere.
Use deduction guides instead of helper functions.
The only non-automatic changes have been:
1. ArrayRef(some_uint8_pointer, 0) needs to be changed into ArrayRef(some_uint8_pointer, (size_t)0) to avoid an ambiguous call with ArrayRef((uint8_t*), (uint8_t*))
2. CVSymbol sym(makeArrayRef(symStorage)); needed to be rewritten as CVSymbol sym{ArrayRef(symStorage)}; otherwise the compiler is confused and thinks we have a (bad) function prototype. There was a few similar situation across the codebase.
3. ADL doesn't seem to work the same for deduction-guides and functions, so at some point the llvm namespace must be explicitly stated.
4. The "reference mode" of makeArrayRef(ArrayRef<T> &) that acts as no-op is not supported (a constructor cannot achieve that).
Per reviewers' comment, some useless makeArrayRef have been removed in the process.
This is a follow-up to https://reviews.llvm.org/D140896 that introduced
the deduction guides.
Differential Revision: https://reviews.llvm.org/D140955
Once the constraint system grows too large in terms of number of rows,
queries can become very slow. This patch adds a new option to limit the
number of rows tracked.
The python script below can be used to generate worst-case IR with a
chain of conditional branches with N branches.
With this limit, we get the following runtimes:
* python3 generate.py 100: 0.1s
* python3 generate.py 1000: 2s
* python3 generate.py 10000: 4s
Without the limit, the case with 1000 chained conditions takes 20+
seconds.
generate.py:
import sys
N = int(sys.argv[1])
args = []
checks = []
for i in range(0, N):
args.append('i32 %l{}'.format(i))
checks.append("""
bb{0}:
%c{0} = icmp uge i32 %l{0}, 100
br i1 %c{0}, label %bb{1}, label %exit
""".format(i, i+1))
print("""
define i1 @foo({0}) {{
{1}
bb{2}:
%c{2} = icmp uge i32 %l0, 100
ret i1 %c{2}
exit:
ret i1 false
}}
""".format(' ,'.join(args), '\n'.join(checks), N))
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D140926
For `srem x, y`, if abs(constant range of x) less than abs(constant
range of y), we can simplify it as:
`srem x, y => x` if y is guaranteed to be positive.
'srem x, y => -x' if y is guaranteed to be negative.
Differential Revision: https://reviews.llvm.org/D140405
I think this doesn't make any difference right now, but once
we take into account that branch on undef is UB in
programUndefinedIfUndefOrPoison() the new position of the branch
would imply that the condition can't be poison, which would
defeat the purpose of the freeze insertion here. We need to
perform the check before the branch is moved.
As per the post-commit feedback - that was not the correct precondition
to avoid it here. I think we should generally start changing mentality
about `freeze`, the fact that we have been conditioned to be afraid of it
(or of anything in LLVM in general) is the key problem here.
This kind of thing happens really frequently in LLVM's very own
shuffle combining methods, and it is even considered bad practice
to use `%` there, instead of using this expansion directly.
Though, many of the cases there have variable divisors,
so this won't help everything.
Simple case: https://alive2.llvm.org/ce/z/PjvYf-
There's alternative expansion via `umin`:
https://alive2.llvm.org/ce/z/hWCVPb
BUT while we can transform the first expansion
into the `umin` one (e.g. for SCEV):
https://alive2.llvm.org/ce/z/iNxKmJ
... we can't go in the opposite direction.
Also, the non-`umin` expansion seems somewhat more codegen-friendly:
https://godbolt.org/z/qzjx5bqWKhttps://godbolt.org/z/a7bj1axbx
There's second variant of precondition:
https://alive2.llvm.org/ce/z/zE6cbM
but there the numerator must be non-undef / must be frozen.
Previous we only handled triangle CFGs. This patch expands that
to support diamonds, where the div and rem appear in the then/else
sides of a condition. In that case, we can hoist the div into the
shared predecessor.
This could be generalized further to use nearest common ancestors,
but some of the conditions for hoisting would then require
post-dominator information.
Reviewed By: nikic, lebedev.ri
Differential Revision: https://reviews.llvm.org/D140647
Introduce an option to rematerialize derived pointers immediately
before their uses instead of after every statepoint. This can be
beneficial when pointer is live across many statepoints but has
few uses.
Initial implementation is simple and rematerializes derived pointer
before every use, even if there are several uses in the same block
or rematerialization instructions can be hoisted etc.
Transformation is considered profitable if we would insert less
instructions than we would insert after every live statepoint.
Depends on D138910, D138911
Reviewed By: anna, skatkov
Differential Revision: https://reviews.llvm.org/D138912
Reapply with a fix for a failing debuginfo assignment tracking test.
-----
Allow sinking stores where both operands are the same, don't require
them to have an identical GEP in each block.
This came up when migrating tests to opaque pointers, where
zero-index GEPs are omitted.
Allow sinking stores where both operands are the same, don't require
them to have an identical GEP in each block.
This came up when migrating tests to opaque pointers, where
zero-index GEPs are omitted.
Inspecting the downstream use of the cpyAlign, it is clear that
`performCallSlotOptzn` is expecting it to represent the alignment
of the copy destination, not the minimum of the src and dest
alignments. This patch renames the parameter to make this more
obvious.
I believe this change is NFC, because the downstream code has
alignment checks such that it all works out in the end. I have not
been able to construct a test case that actually triggers a change
in output.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D140603
Update handlePhiDef to consider the probability argument in an expect.with.probability intrinsic when annotating BranchInsts.
In addition, we also disallow non-constant probability arguments in this intrinsic.
Differential Revsion: https://reviews.llvm.org/D140337
This reverts commit bf88ba0f87,
relands 9f27f4536e, but without a bug:
we *REALLY* should not be defaulting to address space 0
when address space is not specified...
Target-extension types represent types that need to be preserved through
optimization, but otherwise are not introspectable by target-independent
optimizations. This patch doesn't add any uses of these types by an existing
backend, it only provides basic infrastructure such that these types would work
correctly.
Reviewed By: nikic, barannikov88
Differential Revision: https://reviews.llvm.org/D135202
This facilitates replacing llvm::Any with std::any.
- Deprecate any_isa in favor of using any_cast(Any*) and checking for
nullptr because C++17 has no any_isa.
- Remove the assert from any_cast(Any*), so it returns nullptr if the
type is not correct. This aligns it with std::any_cast(any*).
Use any_cast(Any*) throughout LLVM instead of checks with any_isa.
This is the first part outlined in
https://discourse.llvm.org/t/rfc-switching-from-llvm-any-to-std-any/67176
Differential Revision: https://reviews.llvm.org/D139973
During scalar promotion, if there are additional potentially-aliasing
loads outside the promoted set, we can still perform a load-only
promotion. As the stores are retained, any potentially-aliasing
loads will still read the correct value.
This increases the number of load promotions in llvm-test-suite by
a factor of two:
| Old | New
licm.NumPromotionCandidates | 4448 | 6038
licm.NumLoadPromoted | 479 | 1069
licm.NumLoadStorePromoted | 1459 | 1459
Unfortunately, this does have some impact on compile-time:
http://llvm-compile-time-tracker.com/compare.php?from=57f7f0d6cf0706a88e1ecb74f3d3e8891cceabfa&to=72b811738148aab399966a0435f13b695da1c1c8&stat=instructions
In part this is because we now have less early bailouts from
promotion, but also due to second order effects (e.g. for one case
I looked at we spend more time in SLP now).
Differential Revision: https://reviews.llvm.org/D133192
The value map of last peeled iteration is computed within peelLoop API.
This patch exposes it for callers of peelLoop.
While this is not currently used by upstream passes, we have a usecase
downstream which benefits from this API update. Future users of peelLoop
can also use the ValueMap if needed.
Similar value maps are exposed by other loop utilities such as loop
cloning.
Differential Revision: https://reviews.llvm.org/D138228
This patch updates propgatesPoison to take a Use as argument and
propagatesPoison now returns true if the passed in operand causes the
user to yield poison if the operand is poison
This allows propagating poison if the condition of a select is poison.
This helps improve results for programUndefinedIfUndefOrPoison.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D111643
value() has undesired exception checking semantics and calls
__throw_bad_optional_access in libc++. Moreover, the API is unavailable without
_LIBCPP_NO_EXCEPTIONS on older Mach-O platforms (see
_LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS).
* This is a recommit of 3c4d2a0396,
* which was reverted in 25f01d593c,
because it exposed a miscompile in PPC backend, which was resolved
in https://reviews.llvm.org/D140089 / cb3f415cd2.
* which was a recommit of cf624b23bc,
* which was reverted in 5cfc22cafe,
because the cut-off on the number of vector elements was not low enough,
and it triggered both SDAG SDNode operand number assertions,
5and caused compile time explosions in some cases.
Let's try with something really *REALLY* conservative first,
just to get somewhere, and try to bump it later.
FIXME: should this respect TTI reg width * num vec regs?
Original commit message:
Now, there's a big caveat here - these bytes
are abstract bytes, not the i8 we have in LLVM,
so strictly speaking this is not exactly legal,
see e.g. https://github.com/AliveToolkit/alive2/issues/860
^ the "bytes" "could" have been a pointer,
and loading it as an integer inserts an implicit ptrtoint.
But at the same time,
InstCombine's `InstCombinerImpl::SimplifyAnyMemTransfer()`
would expand a memtransfer of 1/2/4/8 bytes
into integer-typed load+store,
so this isn't exactly a new problem.
Note that in memory, poison is byte-wise,
so we really can't widen elements,
but SROA seems to be inconsistent here.
Fixes#59116.
Use a consistent type for the operands() methods of different SCEV
types. Also make the API consistent by only providing operands(),
rather than also providin op_begin() and op_end() for some of them.
Fixes https://github.com/llvm/llvm-project/issues/59023
PHI nodes that are in the second loop only have the first loop as its
predecessor. These PHI nodes should be sunk to the end of the fused
loop. If the second loop uses the PHI, then the loops cannot be fused.
I don't think this should happen in typical compilation workflows.
The PHI will be in a dedicated exit block of the first loop following
LCSSA transformations.
Reviewed By: aeubanks
Differential Revision: https://reviews.llvm.org/D139812
The Assignment Tracking debug-info feature is outlined in this RFC:
https://discourse.llvm.org/t/
rfc-assignment-tracking-a-better-way-of-specifying-variable-locations-in-ir
Split dbg.assign intrinsics into fragments similarly to what SROA already does
for dbg.declares, except that there's many more intrinsics to split. The
function migrateDebugInfo generates new dbg.assigns intrinsic for each part of
a split store.
Reviewed By: jmorse
Differential Revision: https://reviews.llvm.org/D133296
