Credits: this change is based on analysis and a proof of concept by
gerbens@google.com.
Before, the compiler loses track of end as 'this' and other references
possibly escape beyond the compiler's scope. This can be see in the
generated assembly:
16.28 │200c80: mov %r15d,(%rax)
60.87 │200c83: add $0x4,%rax
│200c87: mov %rax,-0x38(%rbp)
0.03 │200c8b: → jmpq 200d4e
...
...
1.69 │200d4e: cmp %r15d,%r12d
│200d51: → je 200c40
16.34 │200d57: inc %r15d
0.05 │200d5a: mov -0x38(%rbp),%rax
3.27 │200d5e: mov -0x30(%rbp),%r13
1.47 │200d62: cmp %r13,%rax
│200d65: → jne 200c80
We fix this by always explicitly storing the loaded local and pointer
back at the end of push back. This generates some slight source 'noise',
but creates nice and compact fast path code, i.e.:
32.64 │200760: mov %r14d,(%r12)
9.97 │200764: add $0x4,%r12
6.97 │200768: mov %r12,-0x38(%rbp)
32.17 │20076c: add $0x1,%r14d
2.36 │200770: cmp %r14d,%ebx
│200773: → je 200730
8.98 │200775: mov -0x30(%rbp),%r13
6.75 │200779: cmp %r13,%r12
│20077c: → jne 200760
Now there is a single store for the push_back value (as before), and a
single store for the end without a reload (dependency).
For fully local vectors, (i.e., not referenced elsewhere), the capacity
load and store inside the loop could also be removed, but this requires
more substantial refactoring inside vector.
Differential Revision: https://reviews.llvm.org/D80588
This is a conservative workaround for broken liveness tracking of
SUBREG_TO_REG to speculatively fix all targets. The current reported
failures are on X86 only, but this issue should appear for all targets
that use SUBREG_TO_REG. The next minimally correct refinement would be
to disallow only implicit defs.
The coalescer now introduces implicit-defs of the super register to
track the dependency on other subregisters. If we see such an implicit
operand, we cannot simply treat the subregister def as the result
operand in case downstream users depend on the implicitly defined
parts. Really target implementations should be considering the
implicit defs and trying to interpret them appropriately (maybe with
some generic helpers). The full implicit def could possibly be
reported as the move result, rather than the subregister def but that
requires additional work.
Hopefully fixes#64060 as well.
This needs to be applied to the release branch.
https://reviews.llvm.org/D156346
Support for vector scale range arguments, for AArch64 scalable vector
extension (SVE) support.
Adds -msve-vector-bits to the flang frontend, and for flang fc1 the
options are -mvscale-min and -mvscale-max (optional). These match the
clang and clang cc1 options for the same purposes.
A further patch will actually USE these arguments.
Currently coalescing with SUBREG_TO_REG introduces an invisible load
bearing undef. There is liveness for the super register not
represented in the MIR.
This is part 1 of a fix for regressions that appeared after
b7836d8562. The allocator started
recognizing undef-def subregister MOVs as copies. Since there was no
representation for the dependency on the high bits, different undef
segments of the super register ended up disconnected and downstream
users ended up observing different undefs than they did previously.
This does not yet fix the regression. The isCopyInstr handling needs
to start handling implicit-defs on any instruction.
I wanted to include an end to end IR test since the actual failure
only appeared with an interaction between the coalescer and the
allocator. It's a bit bigger than I'd like but I'm having a bit of
trouble reducing it to something which definitely shows a diff that's
meaningful.
The same problem likely exists everywhere trying to do anything with
SUBREG_TO_REG. I don't understand how this managed to be broken for so
long.
This needs to be applied to the release branch.
https://reviews.llvm.org/D156345
Before af744f0b84, the first entry
among the search paths was the empty string, indicating searching
in (or starting from) the current directory. After
af744f0b84, the toolchain/clang
specific lib directories were added at the head of the search path.
This would cause lookups of literal file names or relative paths
to match paths in the toolchain, if there are coincidental files
with similar names there, even if they would be find in the current
directory as well.
Change addClangLibSearchPaths to append to the list like all other
operations on searchPaths - but move the invocation of the
function to the right place in the sequence.
This fixes#67779.
Introduce isDesirableCastOp() which determines whether IR builder
and constant folding should produce constant expressions for a
given cast type. This mirrors what we do for binary operators.
Mark zext/sext as undesirable, which prevents most creations of such
constant expressions. This is still somewhat incomplete and there
are a few more places that can create zext/sext expressions.
This is part of the work for
https://discourse.llvm.org/t/rfc-remove-most-constant-expressions/63179.
The reason for the odd result in the constantexpr-fneg.c test is
that initially the "a[]" global is created with an [0 x i32] type,
at which point the icmp expression cannot be folded. Later it is
replaced with an [1 x i32] global and the icmp gets folded away.
But at that point we no longer fold the zext.
If this was coalescing a def of a subregister with a def of the super
register, it was introducing a redundant super-register def and
marking the subregister def as dead.
Resulting in something like:
dead $eax = MOVr0, implicit-def $rax, implicit-def $rax
Avoid this by checking if the new instruction already has the super
def, so we end up with this instead:
dead $eax = MOVr0, implicit-def $rax
The dead flag looks suspicious to me, seems like it's easy to buggily
interpret dead def of subreg and a non-dead def of an aliasing
register. It seems to be intentional though.
https://reviews.llvm.org/D156343
Close https://github.com/llvm/llvm-project/issues/67893
The root cause of the crash is an oversight that we missed the point
that the same module can be imported multiple times. And we should use
`SmallSetVector` instead of `SmallVector` to filter the case.
This reverts one part of commit
9f4dfcb795, with a modified comment added
about the code.
Ideally, this would only be reinstated temporarily - but given the
situation in vcpkg, it looks likely that they would keep passing the
duplicate options for quite some time. The conflicting CRT choice
usually are benign but only would cause warnings about one option
overriding the other, if passing e.g. "/MDd /MT".
However when vcpkg currently sets these options in CMAKE_*_FLAGS_DEBUG,
it passes the redundant option /D_DEBUG; thus the compiler finally ends
up with e.g. "/D_DEBUG /MDd /MT", which has the effect of defining
_DEBUG while using a release mode CRT, which allegedly breaks the build.
There's a PR up for removing this redundant /D_DEBUG option in vcpkg in
https://github.com/microsoft/vcpkg/pull/34123. With that in place, this
change wouldn't be strictly needed.
Permit an implicit-def of a virtual register when rematerializing if
it defines a super register of a subregister def. The
rematerialization pre-legality check should really have been checking
the implicit operands, but that should be fixed separately.
https://reviews.llvm.org/D156331
When performing a masked load of an unpacked SVE vector type, i.e.
nxv8i8, followed by a zero- or sign-extend to an illegal wide type
such as nxv8i32 we typically end up with a combination of an
extending masked load and pair(s) of uunpklo/hi or sunpklo/hi
instructions. For example, see test @masked_sload_8i8_8i32 in file
CodeGen/AArch64/sve-masked-ldst-sext.ll
where
%aval = call <vscale x 8 x i8> @llvm.masked.load.nxv8i8(...
%aext = sext <vscale x 8 x i8> %aval to <vscale x 8 x i32>
gets lowered to
ld1sb { z1.h }, ...
sunpklo z0.s, z1.h
sunpkhi z1.s, z1.h
Currently the cost for the 'sext' operation in the example above is
1, whereas this patch changes it to 2 to reflect the pair of
instructions required. Similarly, when doing a masked load of a
nxv8i8 and extending to nxv8i64 the cost is changed to 6 to reflect
the 6 unpacks required.
Not sure how to produce a test that demonstrates the problem
today. The coalescer would have to introduce a verifier caught SSA
violation, like multiple defs of a virtual register. I'm not sure what
would do that now, but an upcoming patch will.
https://reviews.llvm.org/D156271
This reverts commit 6403287eff.
This is failing on all but 1 of Linaro's flang builders.
CMake Error at /home/tcwg-buildbot/worker/clang-aarch64-full-2stage/llvm/flang-rt/unittests/CMakeLists.txt:37 (message):
Target llvm_gtest not found.
Values that are the result of buffer allocation ops are guaranteed to
*not* be the same allocation as block arguments of containing blocks.
This fact can be used to allow for more aggressive simplification of
`bufferization.dealloc` ops.
The addition of the type kind to the profile ID of IntegerLiterals
results in e.g. size_t and unsigned long literals mismatch even on
platforms where they are canonically the same type. This patch checks
the Canonical field to determine whether to canonicalize the type first.
rdar://116063468
Follow up up of https://github.com/llvm/llvm-project/pull/67693
- Zero initialize uninitialized components of saved derived type entity
with a default initial value.
- Zero initialize uninitialized storage of common blocks with a member
with an initial value.
- Zero initialized uninitialized saved equivalence
This removes all the cases where fir.global are created with an initial
value that results in an undef in LLVM for part of the global, leading
in surprising LLVM optimizations at -O2 for Fortran folks that expects
there saved variables to be zero initialized if there is no explicit or
default initial value.
Don't remove the outermost parentheses surrounding a return statement
expression when inside a function/lambda that has the decltype(auto)
return type.
Fixed#67892.
In the process of vectorization of the affine loop, the 0 vector size
causes the crash with building the invalid AffineForOp. We can catch the
case beforehand propagating to the assertion.
See: https://github.com/llvm/llvm-project/issues/64262
This change adds two related DAG combines which together will take a
left-reduce scalar add tree of an explode_vector, and will incrementally
form a vector reduction of the vector prefix. If the entire vector is
reduced, the result will be a reduction over the entire vector.
Profitability wise, this relies on vredsum being cheaper than a pair of
extracts and scalar add. Given vredsum is linear in LMUL, and the
vslidedown required for the extract is *also* linear in LMUL, this is
clearly true at higher index values. At N=2, it's a bit questionable,
but I think the vredsum form is probably a better canonical form
anyways.
Note that this only matches left reduces. This happens to be the
motivating example I have (from spec2017 x264). This approach could be
generalized to handle right reduces without much effort, and could be
generalized to handle any reduce whose tree starts with adjacent
elements if desired. The approach fails for a reduce such as (A+C)+(B+D)
because we can't find a root to start the reduce with without scanning
the entire associative add expression. We could maybe explore using
masked reduces for the root node, but that seems of questionable
profitability. (As in, worth questioning - I haven't explored in any
detail.)
This is covering up a deficiency in SLP. If SLP encounters the scalar
form of reduce_or(A) + reduce_sum(a) where a is some common
vectorizeable tree, SLP will sometimes fail to revisit one of the
reductions after vectorizing the other. Fixing this in SLP is hard, and
there's no good reason not to handle the easy cases in the backend.
Another option here would be to do this in VectorCombine or generic DAG.
I chose not to as the profitability of the non-legal typed prefix cases
is very target dependent. I think this makes sense as a starting point,
even if we move it elsewhere later.
This is currently restructed only to add reduces, but obviously makes
sense for any associative reduction operator. Once this is approved, I
plan to extend it in this manner. I'm simply staging work in case we
decide to go in another direction.
Without this patch, we pass Endian as one of the parameters to the
constructor of DWARFDataExtractor. The problem is that Endian is of:
enum endianness {big, little, native};
whereas the constructor is expecting "bool IsLittleEndian". That is,
we are relying on an implicit conversion to convert big and little to
false and true, respectively.
When we migrate llvm::support::endianness to std::endian in future, we
can no longer rely on an implicit conversion because std::endian is
declared with "enum class". Even if we could, the conversion would
not be guaranteed to work because, for example, libcxx defines:
enum class endian {
little = 0xDEAD,
big = 0xFACE,
:
where big and little are not boolean values.
This patch fixes the problem by properly converting Endian to a
boolean value.
Without this patch, we pass G.getEndianness() as one of the parameters
to DWARFContext::create. The problem is that G.getEndianness() is of:
enum endianness {big, little, native};
whereas DWARFContext::create is expecting "bool isLittleEndian". That
is, we are relying on an implicit conversion to convert big and little
to false and true, respectively.
When we migrate llvm::support::endianness to std::endian in future, we
can no longer rely on an implicit conversion because std::endian is
declared with "enum class". Even if we could, the conversion would
not be guaranteed to work because, for example, libcxx defines:
enum class endian {
little = 0xDEAD,
big = 0xFACE,
:
where big and little are not boolean values.
This patch fixes the problem by properly converting G.getEndianness()
to a boolean value.
This patch updates AffineParallelOp::verify() to check each result type matches
its corresponding reduction op (i.e, the result type must be a `FloatType` if
the reduction attribute is `addf`)
affine.parallel will crash on --lower-affine if the corresponding result type
cannot match the reduction attribute.
```
%128 = affine.parallel (%arg2, %arg3) = (0, 0) to (8, 7) reduce ("maxf") -> (memref<8x7xf32>) {
%alloc_33 = memref.alloc() : memref<8x7xf32>
affine.yield %alloc_33 : memref<8x7xf32>
}
```
This will crash and report a type conversion issue when we run `mlir-opt --lower-affine`
```
Assertion failed: (isa<To>(Val) && "cast<Ty>() argument of incompatible type!"), function cast, file Casting.h, line 572.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0. Program arguments: mlir-opt --lower-affine temp.mlir
#0 0x0000000102a18f18 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/workspacebin/mlir-opt+0x1002f8f18)
#1 0x0000000102a171b4 llvm::sys::RunSignalHandlers() (/workspacebin/mlir-opt+0x1002f71b4)
#2 0x0000000102a195c4 SignalHandler(int) (/workspacebin/mlir-opt+0x1002f95c4)
#3 0x00000001be7894c4 (/usr/lib/system/libsystem_platform.dylib+0x1803414c4)
#4 0x00000001be771ee0 (/usr/lib/system/libsystem_pthread.dylib+0x180329ee0)
#5 0x00000001be6ac340 (/usr/lib/system/libsystem_c.dylib+0x180264340)
#6 0x00000001be6ab754 (/usr/lib/system/libsystem_c.dylib+0x180263754)
#7 0x0000000106864790 mlir::arith::getIdentityValueAttr(mlir::arith::AtomicRMWKind, mlir::Type, mlir::OpBuilder&, mlir::Location) (.cold.4) (/workspacebin/mlir-opt+0x104144790)
#8 0x0000000102ba66ac mlir::arith::getIdentityValueAttr(mlir::arith::AtomicRMWKind, mlir::Type, mlir::OpBuilder&, mlir::Location) (/workspacebin/mlir-opt+0x1004866ac)
#9 0x0000000102ba6910 mlir::arith::getIdentityValue(mlir::arith::AtomicRMWKind, mlir::Type, mlir::OpBuilder&, mlir::Location) (/workspacebin/mlir-opt+0x100486910)
...
```
Fixes#64068
Reviewed By: mehdi_amini
Differential Revision: https://reviews.llvm.org/D157985
Prior to 591c4b64b3, the mingw specific
linker options -mthreads, -mconsole, -mwindows and -mdll would be
tolerated also at compile time, but generating a warning about being
unused.
After that commit, they were marked as target specific, which means that
it's an error if they're unused (which would consider them used for the
wrong target). These specific options are only relevant when linking,
but we want to tolerate them at compile time too, like before.
This was fixed for -mthreads in
a79995ca60, while the other options didn't
seem to be commonly used during compilation.
After the 17.x release, we've got more reports about this actually being
an issue, in #64464. Therefore, apply the same fix for them; marking
them as tolerated for mingw targets during compilation, even if they're
unused. Also add a testcase for -mthreads which was already handled.
Thus, this fixes#64464.
Without this patch, we pass Endian as one of the parameters to the
constructor of DataExtractor. The problem is that Endian is of:
enum endianness {big, little, native};
whereas the constructor is expecting "bool IsLittleEndian". That is,
we are relying on an implicit conversion to convert big and little to
false and true, respectively.
When we migrate llvm::support::endianness to std::endian in future, we
can no longer rely on an implicit conversion because std::endian is
declared with "enum class". Even if we could, the conversion would
not be guaranteed to work because, for example, libcxx defines:
enum class endian {
little = 0xDEAD,
big = 0xFACE,
:
where big and little are not boolean values.
This patch fixes the problem by properly converting Endian to a
boolean value.
- Add tests for computeOverflowFor*Sub functions
- extend the computeOverflowForSignedSub/computeOverflowForUnsignedSub
implementations with ConstantRange (#37109)
We could maybe extend this by allowing the lowest subop to have multiple uses and extract the lowest subvector result of the concatenated op, but let's just get the fix in first.
Fixes#67333
When custom install names and rpaths setups are used they may not
work in the build tree as-is (namely when using absolute paths for
install names in order to avoid rpath juggling in downstream projects).
Add a flag for opting out of this behaviour.
See: https://reviews.llvm.org/D42463
