There are two motivations for this change:
1. It considerably simplifies adding support for the realloc operation to the
new buffer deallocation pass by lowering the realloc such that no
deallocation operation is inserted and the deallocation pass itself can
insert that dealloc
2. The lowering is expressed on a higher level and thus easier to understand,
and the lowerings of the memref operations it is composed of don't have to
be duplicated in the MemRefToLLVM lowering (also see discussion in
https://reviews.llvm.org/D133424)
Reviewed By: springerm
Differential Revision: https://reviews.llvm.org/D159430
sparse_tensor ops cannot be bufferized with One-Shot Bufferize. (They can only be analyzed.) The sparse compiler does the actual lowering to memref. Produce a proper error message instead of crashing.
This fixes#61311.
Differential Revision: https://reviews.llvm.org/D158728
These methods are needed for use with `Diagnostic::operator<<` etc.
The definitions follow the pattern of `Diagnostic::str` by simply wrapping the underlying `print(raw_ostream)` method. Although there is some overhead for constructing the `std::string`, this seems like the overall most-efficient option: since this overhead only occurs on the error path (under the current intended usage). An alternative approach would be to have one method construct a `Twine` directly, and then have the print method pass the twine to the stream; however, that would mean introducing the overhead of twine construction on the common/happy path of simply printing things out.
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D157643
These new methods help clean up some code for doing LvlExpr-analysis during DimExpr-inference.
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D157647
These checks have been fired when comparing same elements like comp(a, a). Let's fix it. I don't have commit rights.
Danila Kutenin
kutdanila@yandex.ru
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D152152
Replaced the "NEW_SYNTAX" with the more readable "map"
(which we may, or may not keep). Minor improvement in
keyword parsing, migrated a few more examples over.
Reviewed By: Peiming, yinying-lisa-li
Differential Revision: https://reviews.llvm.org/D158325
Registering the SparsificationAndBufferization into a proper TD pass
has the advantage that it can be invoked and tested in isolation. This
change also moves some bufferization specific set up from the pipeline
file into the pass file, keeping the logic more locally.
Reviewed By: Peiming
Differential Revision: https://reviews.llvm.org/D158219
Alphabetical order, less whitespace
removed some verbose comments for readability
(which are preserved in the history if ever needed)
Reviewed By: Peiming
Differential Revision: https://reviews.llvm.org/D158114
This revision is needed to support bufferization of `cf.br`/`cf.cond_br`. It will also be useful for better analysis of loop ops.
This revision generalizes `getAliasingOpResults` to `getAliasingValues`. An OpOperand can now not only alias with OpResults but also with BlockArguments. In the case of `cf.br` (will be added in a later revision): a `cf.br` operand will alias with the corresponding argument of the destination block.
If an op does not implement the `BufferizableOpInterface`, the analysis in conservative. It previously assumed that an OpOperand may alias with each OpResult. It now assumes that an OpOperand may alias with each OpResult and each BlockArgument of the entry block.
Differential Revision: https://reviews.llvm.org/D157957
Consistent order of ops and related methods.
Also, renamed SpGEMMGetSizeOp to SpMatGetSizeOp
since this is a general utility for sparse matrices,
not specific to GEMM ops only.
Reviewed By: Peiming
Differential Revision: https://reviews.llvm.org/D157922
Renames the old `DimLvlExpr::getContext` to `tryGetContext` to make it clear that the result could be a nullptr. Also adds new delegation methods `DimSpec::tryGetContext` and `LvlSpec::getContext`.
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D157163
Improves the conversion from `DimLvlMap` to STEA, in order to correct rank-mismatch issues in the roundtrip tests.
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D157162
This commit makes `DimLvlMap::print` consistent with `DimLvlMapParser` with respect to both the forward-decls per se, as well as the all-or-none constraint on LvlVar-bindings in LvlSpecs.
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D156768
Previously, the commented out code in the `DimLvlMap` ctor would result in `VarSet::add` raising an OOB error; which should be impossible because the ctor asserted `DimLvlMap::isWF` which ensures that all variables occuring in the map are within bounds for the ranks.
The root cause of that bug was the `VarSet` ctor using `SmallBitVector::reserve` which does not actually change the size of the bitvectors (hence the subsequent OOB). This is corrected by using any of `SmallBitVector::resize`, the move-ctor, or the copy-ctor. Since the default-initialized bitvectors being modified/overwritten have size zero, there shouldn't be any significant performance difference between these three implementations.
Reviewed By: Peiming
Differential Revision: https://reviews.llvm.org/D155999
This restores D132311, which was reverted in
29c841ce93 (Sep 2022) due to certain files
not buildable with GCC 7.3.0. The previous attempt was reverted by
6cd9608fb3 (Dec 2020).
This time, GCC 7.3.0 has existing build errors for a long time due to
structured bindings for many files, e.g.
```
llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:9098:13: error: cannot decompose class type ‘std::pair<llvm::Value*, const llvm::SCEV*>’: both it and it
s base class ‘std::pair<llvm::Value*, const llvm::SCEV*>’ have non-static data members
for (auto [_, Stride] : Legal->getLAI()->getSymbolicStrides()) {
^~~~~~~~~~~
```
... and also some `error: duplicate initialization of` instances due to llvm/Transforms/IPO/Attributor.h.
---
GCC 7.5.0 has a bug that, without this change, certain `SmallVector` with a `std::pair` element type like `SmallVector<std::pair<Instruction * const, Info>, 0> X;` lead to spurious
```
/tmp/opt/gcc-7.5.0/include/c++/7.5.0/type_traits:878:48: error: constructor required before non-static data member for ‘...’ has been parsed
```
Switching to std::is_trivially_{copy/move}_constructible fixes the error.
This reverts commit 2e0e00ed84
and reverts commit a6eb40692c
and reverts commit 585cbe3f63.
15 tests are broken on the mlir-nvidia buildbot:
'cuModuleLoadData(&module, data)' failed with 'CUDA_ERROR_INVALID_SOURCE'
'cuModuleGetFunction(&function, module, name)' failed with 'CUDA_ERROR_INVALID_HANDLE'
'cuLaunchKernel(function, gridX, gridY, gridZ, blockX, blockY, blockZ, smem, stream, params, extra)' failed with 'CUDA_ERROR_INVALID_HANDLE'
'cuModuleUnload(module)' failed with 'CUDA_ERROR_INVALID_HANDLE'
This work improves how we compile the generated PTX code using the `ptxas` compiler. Currently, we rely on the driver's jit API to compile the PTX code. However, this approach has some limitations. It doesn't always produce the same binary output as the ptxas compiler, leading to potential inconsistencies in the generated Cubin files.
This work introduces a significant improvement by directly utilizing the ptxas compiler for PTX compilation. By doing so, we can achieve more consistent and reliable results in generating cubin files. Key Benefits:
- Using the Ptxas compiler directly ensures that the cubin files generated during the build process remain consistent with CUDA compilation using `nvcc` or `clang`.
- Another advantage of this work is that it allows developers to experiment with different ptxas compilers without the need to change the compiler. Performance among ptxas compiler versions are vary, therefore, one can easily try different ptxas compilers.
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D155563
This commit comprises a number of related changes:
(1) Reintroduces the semantic distinction between `parseVarUsage` vs `parseVarBinding`, adds documentation explaining the distinction, and adds commentary to the one place that violates the desired/intended semantics.
(2) Improves documentation/commentary about the forward-declaration of level-vars, and about the meaning of the `bool` parameter to `parseLvlSpec`.
(2) Removes the `VarEnv::addVars` method, and instead has `DimLvlMapParser` handle the conversion issues directly. In particular, the parser now stores and maintains the `{dims,lvls}AndSymbols` arrays, thereby avoiding the O(n^2) behavior of scanning through the entire `VarEnv` for each `parse{Dim,Lvl}Spec` call. Unfortunately there still remains another source of O(n^2) behavior, namely: the `AsmParser::parseAffineExpr` method will copy the `DimLvlMapParser::{dims,lvls}AndSymbols` arrays into `AffineParser::dimsAndSymbols` on each `parse{Dim,Lvl}Spec` call; but fixing that would require extensive changes to `AffineParser` itself.
Depends On D155532
Reviewed By: Peiming
Differential Revision: https://reviews.llvm.org/D155533
This change is mostly for brevity's sake; but it also paves the way for the `Policy` enum to be reuseable for other situations that require the same three-way semantics.
Reviewed By: Peiming
Differential Revision: https://reviews.llvm.org/D155532
These two headers both contained a strange mix of definitions related to
both patterns and non-pattern transforms. Put patterns and "populate"
functions into Patterns.h and standalone transforms into Transforms.h.
Depends On: D155223
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D155454
Since `StringRef::empty` can be used in constexpr context, it seems reasonable that `SMLoc::isValid` should be too. The default-ctor and `operator==` are made constexpr for consistency.
In particular, the `constexpr` keyword is needed for silencing warnings on Windows (whereas Linux allows constexpr usage without the keyword).
Reviewed By: jpienaar
Differential Revision: https://reviews.llvm.org/D154741
