Commit Graph

430 Commits

Author SHA1 Message Date
Roman Lebedev
d579885e32 [NFC] Port all GlobalOpt tests to -passes= syntax 2022-12-08 02:38:44 +03:00
Bjorn Pettersson
0676acb6fd [test] Switch to use -passes syntax in a bunch of test cases
Should cover most of the tests for GVN, GVNHoist, GVNSink, GlobalOpt,
GlobalSplit, InstCombine, Reassociate, SROA and TailCallElim that
had not been updated earlier.
2022-11-29 13:29:02 +01:00
Nikita Popov
304f1d59ca [IR] Switch everything to use memory attribute
This switches everything to use the memory attribute proposed in
https://discourse.llvm.org/t/rfc-unify-memory-effect-attributes/65579.
The old argmemonly, inaccessiblememonly and inaccessiblemem_or_argmemonly
attributes are dropped. The readnone, readonly and writeonly attributes
are restricted to parameters only.

The old attributes are auto-upgraded both in bitcode and IR.
The bitcode upgrade is a policy requirement that has to be retained
indefinitely. The IR upgrade is mainly there so it's not necessary
to update all tests using memory attributes in this patch, which
is already large enough. We could drop that part after migrating
tests, or retain it longer term, to make it easier to import IR
from older LLVM versions.

High-level Function/CallBase APIs like doesNotAccessMemory() or
setDoesNotAccessMemory() are mapped transparently to the memory
attribute. Code that directly manipulates attributes (e.g. via
AttributeList) on the other hand needs to switch to working with
the memory attribute instead.

Differential Revision: https://reviews.llvm.org/D135780
2022-11-04 10:21:38 +01:00
Arthur Eubanks
8c49b01a1e [GlobalOpt] Don't remove inalloca from varargs functions
Varargs and inalloca have a weird interaction where varargs are actually
passed via the inalloca alloca. Removing inalloca breaks the varargs
because they're still not passed as separate arguments.

Fixes #58718

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D137182
2022-11-01 13:04:05 -07:00
Ben Dunbobbin
7eee2a2d44 [IR] Don't allow DLL storage-class and local linkage
Disallow this meaningless combination. Doing so simplifies analysis
of LLVM code w.r.t t DLL storage-class, and prevents mistakes with
DLL storage class.

- Change the assembler to reject DLL storage class on symbols with
  local linkage.
- Change the bitcode reader to clear the DLL Storage class when the
  linkage is local for auto-upgrading
- Update LangRef.

There is an existing restriction on non-default visibility and local
linkage which this is modelled on.

Differential Review: https://reviews.llvm.org/D134784
2022-09-30 00:26:01 +01:00
Nuno Lopes
bc698c77af add test for -enable-global-analyses=0 [NFC] 2022-09-19 19:25:14 +01:00
Florian Hahn
91e67c0749 [GlobalOpt] Add test case for #56762.
Add test case where GlobalOpt fails to remove loads to global fields
with struct types.
2022-09-02 11:33:07 +01:00
Mikael Holmen
51d4c7ceea [GlobalOpt] Fix debug variance problem in hasOnlyColdCalls
hasOnlyColdCalls skipped over calls to intrinsics, but it did so after
checking the linkage of the called function. This meant that the presence
of a call to a debug intrinsic could affect the outcome of the
optimization.

In my original reproducer (for an out of tree target) it was particularly
interesting, because the actual IR after GlobalOpt was not different with
debug instrinsics present, so -print-after-all printouts didn't show
anything there.

However, without debuginfo, GlobalOpt went further and ran
BlockFrequencyAnalysis and (more importanly) LoopAnalysis, and later on in
the pipeline, instcombine behaved in different ways when LoopInfo was
present.

So a call to a dbg.declare prevented running LoopAnalysis in
GlobalOpt, which later prevented InstCombine from doing an optimization.

The dbg-intrinsic-loopanalysis.ll testcase tries to expose this.

Then I also noted that adding a dbg.declare actually made the existing
testcase colccc_coldsites.ll generate different code, so I modified that
to now test it behaves the same way with and without the dbg.declare.

Reviewed By: nikic, fhahn

Differential Revision: https://reviews.llvm.org/D133193
2022-09-02 12:29:44 +02:00
Cameron McInally
38d58c1b37 [GlobalOpt] Bail out of GlobalOpt SROA if a Scalable Vector is seen
The SROA algorithm won't work for Scalable Vectors, since we don't
know how many bytes are loaded/stored. Bail out if a Scalable
Vector is seen.

Differential Revision: https://reviews.llvm.org/D132417
2022-08-24 13:17:59 -07:00
Augie Fackler
12c0bf8ba9 tests: add attributes that would normally come from inferattrs
As my goal is to remove at least _some_ functions from the static list
in MemoryBuiltins.cpp, these tests either need to run inferattrs or
statically declare these attributes to keep passing. A couple of tests
had alternate cases which are no longer meaningful, e.g.
`malloc-load-removal.ll`.

Differential Revision: https://reviews.llvm.org/D123087
2022-07-25 17:29:00 -04:00
Johannes Doerfert
dfac030271 [Intrinsics] Add nocallback to the memset/cpy/move intrinsics
These were forgotten when D118680 was applied. Similar to D125937.

Differential Revision: https://reviews.llvm.org/D129516
2022-07-21 22:52:46 -05:00
Alexander Shaposhnikov
e9afdf838e [GlobalOpt] Enable evaluation of atomic loads
Relax the check to allow evaluation of atomic loads
(but still skip volatile loads).

Test plan:
1/ ninja check-llvm check-clang
2/ Bootstrapped LLVM/Clang pass tests

Differential revision: https://reviews.llvm.org/D130211
2022-07-21 21:36:11 +00:00
Alexander Shaposhnikov
67f1fe8597 [GlobalOpt] Enable evaluation of atomic stores
Relax the check to allow evaluation of atomic stores
(but still skip volatile stores).

Test plan:
1/ ninja check-llvm check-clang
2/ Bootstrapped LLVM/Clang pass tests

Differential revision: https://reviews.llvm.org/D129841
2022-07-20 22:33:58 +00:00
Florian Hahn
2d5d6c343b [GlobalOpt] Add more tests with large number of stores to globals.
Add a few more test cases for D129525.
2022-07-13 11:13:21 -07:00
Florian Hahn
139378ef8e [GlobalOpt] Add test that requires splitting up global into many.
Add test that hits the limit introduced in 4796b4ae7b.
2022-07-11 16:34:07 -07:00
Nikita Popov
11950efe06 [ConstExpr] Remove div/rem constant expressions
D128820 stopped creating div/rem constant expressions by default;
this patch removes support for them entirely.

The getUDiv(), getExactUDiv(), getSDiv(), getExactSDiv(), getURem()
and getSRem() on ConstantExpr are removed, and ConstantExpr::get()
now only accepts binary operators for which
ConstantExpr::isSupportedBinOp() returns true. Uses of these methods
may be replaced either by corresponding IRBuilder methods, or
ConstantFoldBinaryOpOperands (if a constant result is required).

On the C API side, LLVMConstUDiv, LLVMConstExactUDiv, LLVMConstSDiv,
LLVMConstExactSDiv, LLVMConstURem and LLVMConstSRem are removed and
corresponding LLVMBuild methods should be used.

Importantly, this also means that constant expressions can no longer
trap! This patch still keeps the canTrap() method to minimize diff --
I plan to drop it in a separate NFC patch.

Differential Revision: https://reviews.llvm.org/D129148
2022-07-06 10:11:34 +02:00
Nikita Popov
f65c88c42f [GlobalOpt] Fix memset handling in global ctor evaluation (PR55859)
The global ctor evaluator currently handles by checking whether the
memset memory is already zero, and skips it in that case. However,
it only actually checks the first byte of the memory being set.

This patch extends the code to check all bytes being set. This is
done byte-by-byte to avoid converting undef values to zeros in
larger reads. However, the handling is still not completely correct,
because there might still be padding bytes (though probably this
doesn't matter much in practice, as I'd expect global variable
padding to be zero-initialized in practice).

Mostly fixes https://github.com/llvm/llvm-project/issues/55859.

Differential Revision: https://reviews.llvm.org/D128532
2022-06-27 16:50:49 +02:00
Nikita Popov
771c46ad29 [GlobalOpt] Add tests for memset with non-zero value (NFC) 2022-06-27 16:36:25 +02:00
Arthur Eubanks
e422c0d3b2 [GlobalOpt] Perform store->dominated load forwarding for stored once globals
The initial land incorrectly optimized forwarding non-Constants in non-nosync/norecurse functions. Bail on non-Constants since norecurse should cause global -> alloca promotion anyway.

The initial land also incorrectly assumed that StoredOnceStore was the only store to the global, but it actually means that only one value other than the global initializer is stored. Add a check that there's only one store.

Compile time tracker:
https://llvm-compile-time-tracker.com/compare.php?from=c80b88ee29f34078d2149de94e27600093e6c7c0&to=ef2c2b7772424b6861a75e794f3c31b45167304a&stat=instructions

Reviewed By: nikic, asbirlea, jdoerfert

Differential Revision: https://reviews.llvm.org/D128128
2022-06-24 09:09:26 -07:00
Nikita Popov
999aa6bf7f [GlobalOpt] Add tests for PR55859 (NFC) 2022-06-24 16:06:37 +02:00
Arthur Eubanks
b257acd266 [test][GlobalOpt] Update precommitted test 2022-06-23 09:56:31 -07:00
Arthur Eubanks
b5db65e0da Reland [GlobalOpt] Preserve CFG analyses
The only place we modify the CFG is when calling
removeUnreachableBlocks(), so insert a callback there which invalidates
analyses for that function (or recomputes DT in the legacy PM).

We may delete functions, make sure to clear analyses for those
functions. (this was missed in the original revision)

Small compile time wins across the board:
https://llvm-compile-time-tracker.com/compare.php?from=f444ea8ce0aaaa5ec1a4129809389da15cc41396&to=698f41f4fc26cbf1006ed5d88e9d658edfc5b749&stat=instructions

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D128145
2022-06-21 09:19:59 -07:00
Arthur Eubanks
13ff7d6f39 Revert "[GlobalOpt] Perform store->dominated load forwarding for stored once globals"
This reverts commit 6f348b146b.

Am seeing internal test failures plus a linux kernel breakage reported due to this.
2022-06-20 10:26:47 -07:00
Arthur Eubanks
6f348b146b [GlobalOpt] Perform store->dominated load forwarding for stored once globals
Compile time tracker:
https://llvm-compile-time-tracker.com/compare.php?from=1e556f459b44dd0ca4073e932f66ecb6f40fe31a&to=6d7bed4e1e72c6a8592748626091274209740a40&stat=instructions

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D128128
2022-06-19 10:27:20 -07:00
Arthur Eubanks
1e556f459b [test][GlobalOpt] Update precommitted test 2022-06-18 21:58:16 -07:00
Arthur Eubanks
9fac606af2 [test][GlobalOpt] Regenerate some tests 2022-06-18 21:34:38 -07:00
Arthur Eubanks
213d489707 [test][GlobalOpt] Precommit more tests
And fix up existing tests to not have so much UB.
2022-06-18 19:36:48 -07:00
Arthur Eubanks
07b9937d0c [test][GlobalOpt] Precommit test 2022-06-18 15:44:27 -07:00
Fangrui Song
3d6872b09f [GlobalOpt][test] Remove br i1 undef 2022-06-10 20:36:22 -07:00
Alexander Shaposhnikov
badd088c57 [GlobalOpt] Enable optimization of constructors with different priorities
Adjust `optimizeGlobalCtorsList` to handle the case of different priorities.
This addresses the issue https://github.com/llvm/llvm-project/issues/55083.

Test plan: ninja check-all

Differential revision: https://reviews.llvm.org/D125278
2022-05-13 22:19:29 +00:00
Arthur Eubanks
b07aab8fc1 [GlobalOpt] Iterate over replaced values deterministically to constprop
If there are pre-existing dead instructions, the order we visit replaced
values can cause us sometimes to not delete dead instructions.

The added test non-deterministically failed without the change.
2022-05-02 09:43:20 -07:00
Arthur Eubanks
4e65291837 [OpaquePtr][GlobalOpt] Don't attempt to evaluate global constructors with arguments
Previously all entries in global_ctors had to have the void()* type and
we'd skip evaluating bitcasted functions. With opaque pointers we may
see the function directly.

Fixes #55147.

Reviewed By: #opaque-pointers, nikic

Differential Revision: https://reviews.llvm.org/D124553
2022-04-27 19:00:44 -07:00
Nikita Popov
db561064f6 [GlobalOpt] Handle non-instruction MTI source (PR54572)
This was reusing a cast to GlobalVariable to check for an
Instruction, which means we'll try to dereference a null pointer
if it's not actually a GlobalVariable. We should be casting
MTI->getSource() instead.

I don't think this problem is really specific to opaque pointers,
but it certainly makes it a lot easier to reproduce.

Fixes https://github.com/llvm/llvm-project/issues/54572.
2022-03-28 14:28:47 +02:00
Fangrui Song
c6692f819e [GlobalOpt] Don't replace alias with aliasee if either alias/aliasee may be preemptible
Generalize D99629 for ELF. A default visibility non-local symbol is preemptible
in a -shared link. `isInterposable` is an insufficient condition.

Moreover, a non-preemptible alias may be referenced in a sub constant expression
which intends to lower to a PC-relative relocation. Replacing the alias with a
preemptible aliasee may introduce a linker error.

Respect dso_preemptable and suppress optimization to fix the abose issues. With
the change, `alias = 345` will not be rewritten to use aliasee in a `-fpic`
compile.
```
int aliasee;
extern int alias __attribute__((alias("aliasee"), visibility("hidden")));
void foo() { alias = 345; } // intended to access the local copy
```

While here, refine the condition for the alias as well.

For some binary formats like COFF, `isInterposable` is a sufficient condition.
But I think canonicalization for the changed case has little advantage, so I
don't bother to add the `Triple(M.getTargetTriple()).isOSBinFormatELF()` or
`getPICLevel/getPIELevel` complexity.

For instrumentations, it's recommended not to create aliases that refer to
globals that have a weak linkage or is preemptible. However, the following is
supported and the IR needs to handle such cases.
```
int aliasee __attribute__((weak));
extern int alias __attribute__((alias("aliasee")));
```

There are other places where GlobalAlias isInterposable usage may need to be
fixed.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D107249
2022-03-18 14:17:05 -07:00
Nikita Popov
067c035012 [GlobalOpt] Handle undef global_ctors gracefully
If there are no ctors, then this can have an arbirary zero-sized
value. The current code checks for null, but it could also be
undef or poison.

Replacing the specific null check with a check for
non-ConstantArray.
2022-03-10 16:02:12 +01:00
Arthur Eubanks
f0b61f7957 Revert "[GlobalOpt] Don't replace alias with aliasee if either alias/aliasee may be preemptible"
This reverts commit 30e8f83c84.

Causes huge compile time regressions on certain large files. Will followup offline with author.
2022-03-03 11:04:14 -08:00
Fangrui Song
30e8f83c84 [GlobalOpt] Don't replace alias with aliasee if either alias/aliasee may be preemptible
Generalize D99629 for ELF. A default visibility non-local symbol is preemptible
in a -shared link. `isInterposable` is an insufficient condition.

Moreover, a non-preemptible alias may be referenced in a sub constant expression
which intends to lower to a PC-relative relocation. Replacing the alias with a
preemptible aliasee may introduce a linker error.

Respect dso_preemptable and suppress optimization to fix the abose issues. With
the change, `alias = 345` will not be rewritten to use aliasee in a `-fpic`
compile.
```
int aliasee;
extern int alias __attribute__((alias("aliasee"), visibility("hidden")));
void foo() { alias = 345; } // intended to access the local copy
```

While here, refine the condition for the alias as well.

For some binary formats like COFF, `isInterposable` is a sufficient condition.
But I think canonicalization for the changed case has little advantage, so I
don't bother to add the `Triple(M.getTargetTriple()).isOSBinFormatELF()` or
`getPICLevel/getPIELevel` complexity.

For instrumentations, it's recommended not to create aliases that refer to
globals that have a weak linkage or is preemptible. However, the following is
supported and the IR needs to handle such cases.
```
int aliasee __attribute__((weak));
extern int alias __attribute__((alias("aliasee")));
```

There are other places where GlobalAlias isInterposable usage may need to be
fixed.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D107249
2022-02-01 10:41:16 -08:00
Nikita Popov
236fbf571d [GlobalStatus] Skip non-pointer dead constant users
Constant expressions with a non-pointer result type used an early
exit that bypassed the later dead constant user check, and resulted
in different optimization outcomes depending on whether dead users
were present or not.

This fixes the issue reported in https://reviews.llvm.org/D117223#3287039.
2022-02-01 15:51:32 +01:00
Philip Reames
26049b8ce3 [GlobalOpt] Generalize malloc-to-global for any allocation function
We can generalize the malloc-to-global transform for other allocation functions which are both a) removable, and b) have a known initialization value.

One subtlety that I want to point out - mostly because I hadn't realized it was true until I took a closer look - is that the existing code doesn't prove that initialization/malloc happens only once. The initialization function can be called multiple times. This is correct without special handling for malloc as undef can map to any value previously written, but a non-undef initializing allocation it means we may end up memseting the new global repeatedly. In particular, this means it's not legal to fold the memset into the initializer of the global.

Differential Revision: https://reviews.llvm.org/D117503
2022-01-17 15:06:23 -08:00
Philip Reames
30715365d4 [test] precommit new test for D117503 2022-01-17 15:00:18 -08:00
Nikita Popov
499f1ca79f [GlobalOpt] Use generic type when converting malloc to global
The malloc to global transform currently determines the type of the
global by looking at bitcasts of the malloc. This is limited (the
transform fails if there are multiple different types) and
incompatible with opaque pointers.

My initial approach was to construct an appropriate struct type
based on usage in loads/stores. What this patch does instead is
to always create an [i8 x AllocSize] global, without trying to
guess types at all.

This does mean that other transforms that require a certain global
type may break. I fixed two of these in D117034 and D117223, which
I believe should be sufficient to avoid regressions. In particular,
the global SRA change should end up splitting the global into
naturally-typed sub-globals, at which point all other optimizations
should work.

Differential Revision: https://reviews.llvm.org/D117092
2022-01-17 09:55:33 +01:00
Nikita Popov
4796b4ae7b [GlobalOpt] Make global SRA offset based
Currently global SRA uses the GEP structure to determine how to
split the global. This patch instead analyses the loads and stores
that are performed on the global, and collects which types are used
at which offset, and then splits the global according to those.

This is both more general, and works fine with opaque pointers.
This is also closer to how ordinary SROA is performed.

Differential Revision: https://reviews.llvm.org/D117223
2022-01-17 09:28:36 +01:00
Nikita Popov
be219323a2 [GlobalOpt] Add test for SRA with i8 array type (NFC) 2022-01-14 10:18:02 +01:00
Philip Reames
213193c184 [test] precommit coverage for D117249 2022-01-13 13:42:39 -08:00
Nikita Popov
aba7c3c033 [ConstantFold] Check uniform value in ConstantFoldLoadFromConst()
This case is automatically handled if ConstantFoldLoadFromConstPtr()
is used. Make sure that ConstantFoldLoadFromConst() also handles it.
2022-01-13 14:40:19 +01:00
Nikita Popov
1cbb456123 [GlobalOpt] Fix global to select transform under opaque pointers
We need to check that the load/store type is also the same, as this
is no longer implicitly checked through the pointer type.
2022-01-13 11:13:06 +01:00
Nikita Popov
f3e87176e1 [GlobalOpt] Support "stored once" optimization for different types
GlobalOpt can optimize a global with undef initializer and a single
store to put the stored value into the initializer instead. Currently,
this requires the type of the global and the store to match.

This patch extends support to cases with different types (but same
size), in which case we create a new global to replace the old one.

Differential Revision: https://reviews.llvm.org/D117034
2022-01-12 09:39:31 +01:00
Nikita Popov
94d6263391 [GlobalStatus] Look through non-constexpr casts
analyzeGlobal() looks through non-constexpr cast instructions when
looking for users. However, this particular place only strips the
casts again if they are constexprs. We should be looking through all
casts here.
2022-01-11 16:02:35 +01:00
Nikita Popov
3404127b4e [GlobalOpt] Regenerate test checks (NFC) 2022-01-11 15:34:34 +01:00
Nikita Popov
6e474d3308 [GlobalOpt][Evaluator] Fix off by one error in bounds check (PR53002)
We should bail out if the index is >= the size, not > the size.

Fixes https://github.com/llvm/llvm-project/issues/53002.
2022-01-05 14:06:02 +01:00