Use an array of structures to represent the indices for the tailing COO region
of a sparse tensor.
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D140870
This commit updates how the `SparseTensorConversion` pass handles `NewOp`. It breaks up the underlying `openSparseTensor` function into two parts (`SparseTensorReader::create` and `SparseTensorReader::readSparseTensor`) so that the pass can inject code for constructing `lvlSizes` between those two parts. Migrating the construction of `lvlSizes` out of the runtime and into the pass is a necessary first step toward fully supporting non-permutations. (The alternative would be for the pass to generate a `FuncOp` for performing the construction and then passing that to the runtime; which doesn't seem to have any benefits over the design of this commit.) And since the pass now generates the code to call these two functions, this change also removes the `Action::kFromFile` value from the enum used by `_mlir_ciface_newSparseTensor`.
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D138363
This is generated by running
```
sed --in-place 's/[[:space:]]\+$//' mlir/**/*.td
sed --in-place 's/[[:space:]]\+$//' mlir/**/*.mlir
```
Reviewed By: rriddle, dcaballe
Differential Revision: https://reviews.llvm.org/D138866
The attribute tells the operator to handle symmetric structures for 2D tensors.
By default, the operator assumes the input tensor is not symmetric.
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D138230
Fix a problem in convert op rewriting where it used the original index for
ToIndicesOp.
Extend the concatenate op rewriting to handle dense destination and dynamic
shape destination.
Make the concatenate op integration test run on the codegen path.
Reviewed By: Peiming
Differential Revision: https://reviews.llvm.org/D138057
Modify the integration test to check number_of_entries and use it to limit for
outputing sparse tensor values.
Reviewed By: aartbik, Peiming
Differential Revision: https://reviews.llvm.org/D138046
Refactor the rewriting of sparse_tensor.sort to support the implementation of
sparse_tensor.sort_coo.
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D137522
Permutation wasn't handled correctly. Add a test for the rewriting.
Extend an integration test to run with enable_runtime_library=false to
also test the rewriting.
Reviewed By: Peiming
Differential Revision: https://reviews.llvm.org/D137845
(1) also fixes memory leak in sparse2dense rewriting
(2) still needs fix in dense2sparse by skipping zeros
Reviewed By: wrengr
Differential Revision: https://reviews.llvm.org/D137736
This revision generalizes lowering the sparse_tensor.insert op into actual code that directly operates on the memrefs of a sparse storage scheme. The current insertion strategy does *not* rely on a cursor anymore, with introduces some testing overhead for each insertion (but still proportional to the rank, as before). Over time, we can optimize the code generation, but this version enables us to finish the effort to migrate from library to actual codegen.
Things to do:
(1) carefully deal with (un)ordered and (not)unique
(2) omit overhead when not needed
(3) optimize and specialize
(4) try to avoid the pointer "cleanup" (at HasInserts), and make sure the storage scheme is consistent at every insertion point (so that it can "escape" without concerns).
Reviewed By: Peiming
Differential Revision: https://reviews.llvm.org/D137457
This patch fix the re-revert D135927 (which caused a windows build failure) to re-enable parallel for/reduction. It also fix a warning caused by D137442.
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D137565
Replace the quick sort partition method with one that is more similar to the
method used by C++ std quick sort. This improves the runtime for sorting
sk_2005.mtx by more than 10x.
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D137290
The old "dumpAndRelease" names are no longer valid since the "release" part is handled separately now.
Reviewed By: Peiming
Differential Revision: https://reviews.llvm.org/D136899
As pointed out by Matthias: "DenseBufferizationPass should be run right after TensorCopyInsertionPass. (Running it after bufferizing the sparse IR is also OK.) The reason for this is that whether copies are needed for not depends on the structure of the program (SSA use-def chains). In particular, running the canonicalizer in-between is problematic because it could introduce new RaW conflicts"
Reviewed By: aartbik
Differential Revision: https://reviews.llvm.org/D136980
This patch disables 2 bf16 tests that are currently not supported on
AArch64. I've triaged these failures and opened [1] to track this. I
don't have a simple reproducer for dense_output_bf16.mlir, but it's
rather clear that both tests fail due to missing support for `bfloat`
operations in the AArch64 backend.
I'm not sure what the path forward to enable these tests on AArch64
should be. I think that there are two options:
* AArch64 backened gains capability to legalize these nodes containing
`bfloat` operands, or
* MLIR (similarly to Clang) is taught not to emit such nodes in the
first place.
[1] https://github.com/llvm/llvm-project/issues/58465
Differential Revision: https://reviews.llvm.org/D136273
