Commit Graph

493 Commits

Author SHA1 Message Date
Peiming Liu
dd0d5acfa6 [mlir][sparse] remove redundant template parameter (NFC)
The template parameter is no longer needed after MutSparseTensorDescriptor
is implemented as a subclass of SparseTensorDescriptorImpl. The only purpose
for it was to enable SFINAE.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D141303
2023-01-09 19:01:37 +00:00
Mehdi Amini
5e92f7edd2 Apply clang-tidy fixes for performance-unnecessary-value-param in SparseTensorCodegen.cpp (NFC) 2023-01-06 10:38:54 +00:00
Jie Fu
a021db346e [mlir] Fix build error due to -Wsign-compare after revision D140871
This patch fixes build failure due to -Wsign-compare in sparse2SparseRewrite(...) after https://reviews.llvm.org/D140871.

```
llvm-project/mlir/lib/Dialect/SparseTensor/Transforms/SparseTensorRewriting.cpp:842:32: error: comparison of integers of different signs: 'uint64_t' (aka 'unsigned long') and 'int64_t' (aka 'long') [-Werror,-Wsign-compare]
        for (uint64_t i = 0; i < rank; i++) {
                             ~ ^ ~~~~
1 error generated.
```

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D141104
2023-01-05 19:36:13 -08:00
bixia1
81e3079d0f [mlir][sparse] Replace sparse_tensor.sort with sparse_tensor.sort_coo for sorting COO tensors.
Add codegen pattern for sparse_tensor.indices_buffer.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140871
2023-01-05 15:42:57 -08:00
bixia1
9bde3d0cc5 [mlir][sparse] Add operator sparse_tensor.indices_buffer.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140762
2023-01-05 09:35:55 -08:00
bixia1
3fdd85da06 [mlir][sparse] Add AOS optimization.
Use an array of structures to represent the indices for the tailing COO region
of a sparse tensor.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140870
2023-01-04 18:16:04 -08:00
bixia1
63d31a4d15 [mlir][sparse] Move some member functions from SparseTensorDescriptorImpl to MutSparseTensorDescriptor.
This is to prepare for implementing AOS optimization.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D141002
2023-01-04 13:05:43 -08:00
Aart Bik
e568d0016e [mlir][sparse] minor code layout edits
Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D140934
2023-01-04 09:56:16 -08:00
bixia1
90aa436291 [mlir][sparse] Add layout to the memref for the indices buffers to prepare for the AOS storage optimization for COO regions.
Fix relevant FileCheck tests.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140742
2023-01-04 07:36:11 -08:00
Aart Bik
3ab0067278 [mlir][sparse] layout fixes (NFC)
Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D140761
2022-12-29 12:30:02 -08:00
bixia1
840e2ba336 [mlir][sparse] Use DLT in the mangled function names for insertion.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140484
2022-12-28 08:21:22 -08:00
Aart Bik
36fd28752f [mlir][sparse] move emitter ownership into environment
last bits and pieces of the environment refactoring

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D140709
2022-12-27 16:55:07 -08:00
Peiming Liu
781eabeb40 [mlir][sparse] refactoring loop emitter into its own files.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140701
2022-12-27 19:12:05 +00:00
Aart Bik
431f6a543e [sparse][mlir][vectorization] add support for shift-by-invariant
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D140596
2022-12-27 11:07:13 -08:00
Peiming Liu
006340baf7 [mlir][sparse] add missing dependent dialect.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140595
2022-12-23 01:44:12 +00:00
Peiming Liu
988733c600 [mlir][sparse] use sparse_tensor::StorageSpecifier to store dim/memSizes
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140130
2022-12-23 00:47:36 +00:00
Peiming Liu
083ddffe47 [mlir][sparse] introduce sparse_tensor::StorageSpecifierToLLVM pass
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140122
2022-12-22 22:45:15 +00:00
Peiming Liu
49be68b8aa [mlir][sparse] make loop emitter API more concise.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140583
2022-12-22 21:17:29 +00:00
Aart Bik
8109d5e911 [mlir][sparse] move loop boundary method to codegenenv
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D140578
2022-12-22 12:40:45 -08:00
Aart Bik
384049a755 [mlir][sparse] completed codegen environment privatization
All members are now private and access is through delegate
or convenience methods only (except the loop emitter, which
is still under refactoring).

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D140519
2022-12-22 10:34:43 -08:00
Mehdi Amini
9f7de08ba6 Apply clang-tidy fixes for llvm-else-after-return in SparseVectorization.cpp (NFC) 2022-12-22 15:33:01 +00:00
Peiming Liu
7e1eac5116 [mlir][sparse] add initialize() API to LoopEmitter to support post-constructor initialization
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140444
2022-12-21 01:18:42 +00:00
Aart Bik
fbe611309e [mlir][sparse] refactored codegen environment into its own file
Also, as a proof of concept, all functionality related to reductions
has been refactored into private fields and a clean public API. As a
result, some dead code was found as well. This approach also simplifies
asserting on a proper environment state for each call.

NOTE: making all other fields private and migrating more methods into
      this new class is still TBD in yes another next revision!

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D140443
2022-12-20 16:58:59 -08:00
Aart Bik
98f93e3b72 [mlir][sparse] factorized merger/emitter/codegen into single environment
This cleans up a lot of parameter passing. It also prepares adding
proper "delegate" functions to the new environment and moving this
out into its own class with a better OO design.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D140257
2022-12-20 11:34:12 -08:00
Aart Bik
2e1caa470b [mlir][sparse] minor merger code cleanup
Moved larger constructor from header to CPP file.
Used toMLIRString() for DimLvlType debug.
Minor layout changes.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D140342
2022-12-19 13:31:25 -08:00
Ramkumar Ramachandra
0de16fafa5 mlir/DialectConversion: use std::optional (NFC)
This is part of an effort to migrate from llvm::Optional to
std::optional. This patch touches DialectConversion, and modifies
existing conversions and tests appropriately.

See also: https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

Signed-off-by: Ramkumar Ramachandra <r@artagnon.com>

Differential Revision: https://reviews.llvm.org/D140303
2022-12-19 18:48:59 +01:00
Fangrui Song
76b1100372 [mlir] llvm::Optional::value() && => operator*/operator->
std::optional::value() has undesired exception checking semantics and is
unavailable in older Xcode (see _LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS). The
call sites block std::optional migration.
2022-12-17 19:23:50 +00:00
Fangrui Song
cbb0981388 [mlir] llvm::Optional::value => operator*/operator->
std::optional::value() has undesired exception checking semantics and is
unavailable in older Xcode (see _LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS). The
call sites block std::optional migration.
2022-12-17 19:07:38 +00:00
Ramkumar Ramachandra
22426110c5 mlir/tblgen: use std::optional in generation
This is part of an effort to migrate from llvm::Optional to
std::optional. This patch changes the way mlir-tblgen generates .inc
files, and modifies tests and documentation appropriately. It is a "no
compromises" patch, and doesn't leave the user with an unpleasant mix of
llvm::Optional and std::optional.

A non-trivial change has been made to ControlFlowInterfaces to split one
constructor into two, relating to a build failure on Windows.

See also: https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

Signed-off-by: Ramkumar Ramachandra <r@artagnon.com>

Differential Revision: https://reviews.llvm.org/D138934
2022-12-17 11:13:26 +01:00
Peiming Liu
a3672add76 [mlir][sparse] avoid unnecessary tmp COO buffer and convert when lowering ConcatentateOp.
When concat along dim 0, and all inputs/outputs are ordered with identity dimension ordering,
the concatenated coordinates will be yield in lexOrder, thus no need to sort.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140228
2022-12-16 18:26:39 +00:00
wren romano
96fef4dc3c [mlir][sparse] Added new SparseTensorEncodingAttr::withoutOrdering factory
Reviewed By: aartbik, Peiming

Differential Revision: https://reviews.llvm.org/D140171
2022-12-15 18:14:54 -08:00
Aart Bik
9c9b47c976 [mlir][sparse] add dim level type toString convenience method
Reviewed By: wrengr, bixia

Differential Revision: https://reviews.llvm.org/D140165
2022-12-15 16:11:30 -08:00
Peiming Liu
509974af02 [mlir][sparse] add folder to sparse_tensor.storage.get operation.
Reviewed By: aartbik, wrengr

Differential Revision: https://reviews.llvm.org/D140172
2022-12-15 23:51:38 +00:00
bixia1
089e120060 [mlir][sparse] Make the remaining integration tests run with vectorization.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140057
2022-12-14 15:13:12 -08:00
bixia1
bfad07268b [mlir][sparse] Add another call to ConvertVectorToLLVMPass, to lower the vector operations added by ConvertMathToLLVMPass.
Run sparse_tanh with vectorization.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D139958
2022-12-14 15:06:34 -08:00
wren romano
62896428a7 [mlir][sparse] Moving/renaming genBuffer to allocaBuffer
This allows allocaBuffer to be used outside of SparseTensorConversion.cpp, which will be helpful for a some future commits.

Reviewed By: aartbik, Peiming

Differential Revision: https://reviews.llvm.org/D140047
2022-12-14 14:29:36 -08:00
Peiming Liu
71cc0f1c04 [mlir][sparse] introduce sparse_tensor::StorageSpecifierType and related operations on it
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D139961
2022-12-14 22:27:22 +00:00
Aart Bik
70ac598197 [mlir][sparse][simd] only accept proper unit stride subscripts
Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D139983
2022-12-14 13:07:14 -08:00
wren romano
387755a35d [mlir][sparse] Simplifying SparseTensorEncodingAttr function arguments
Since STEA isa Attribute, and that's just (a wrapper around) a pointer, the extra `const` and `&` aren't necessary for function arguments.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D139886
2022-12-12 17:05:56 -08:00
Matthias Springer
be630f07de [mlir][bufferize] Implement BufferizableOpInterface for tensor.empty
The op is not bufferizable but should be analyzable (for `EliminateEmptyTensors`, which uses the bufferization infrastructure).

Also improve debugging functionality and error messages.

Also adds a missing pass to the sparse pipeline. (tensor.empty should be replaced with bufferization.alloc_tensor, but it sometimes used to work without depending on how the tensor.empty is used. Now we always fail explicitly.)
2022-12-12 14:19:38 +01:00
Peiming Liu
b4e2b7f90c [mlir][sparse] avoid sorting when unnecessary when convert sparse tensors.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D139744
2022-12-10 00:24:42 +00:00
Peiming Liu
faa75f94f1 [mlir][sparse] reject kernels with non-sparsfiable reduction expression.
To address https://github.com/llvm/llvm-project/issues/59394.

Reduction on negation of the output tensor is a non-sparsifiable kernel, it creates cyclic dependency.

This patch reject those cases instead of crashing.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D139659
2022-12-08 23:36:30 +00:00
bixia1
ea4be70cea [mlir][sparse] Fix problems in creating complex zero for initialization.
Reviewed By: aartbik, wrengr

Differential Revision: https://reviews.llvm.org/D139591
2022-12-08 07:49:27 -08:00
Aart Bik
16aa4e4bd1 [mlir][sparse] introduce sparse vectorization to the sparse compiler pipeline
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D139581
2022-12-07 16:06:53 -08:00
bixia1
a0568eabaf [mlir][sparse] Add dependence on bufferization.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D139571
2022-12-07 15:18:36 -08:00
bixia1
19cde2df95 [mlir][sparse] Improve concatenate operation conversion for the case with annotated all dense result.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D139345
2022-12-07 12:06:50 -08:00
Aart Bik
65074179f2 [mlir][sparse] make fusion for SDDMM more robust
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D139456
2022-12-06 14:32:19 -08:00
Peiming Liu
191c43f60e Revert "Revert "[mlir][sparse] Refactoring: abstract sparse tensor memory scheme into a SparseTensorDescriptor class.""
This reverts commit 10033a179f. Plus, it fixed windows warnings and gcc errors

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D139384
2022-12-06 17:12:06 +00:00
Adrian Kuegel
f083c9bdef [mlir][SparseTensor] Apply ClangTidyLegacy finding (NFC).
Converting integer literal to bool, use bool literal instead.
2022-12-06 13:29:47 +01:00
Stella Stamenova
10033a179f Revert "[mlir][sparse] Refactoring: abstract sparse tensor memory scheme into a SparseTensorDescriptor class."
This reverts commit 8a7e69d145.

This broke the windows mlir buildbot: https://lab.llvm.org/buildbot/#/builders/13/builds/29257
2022-12-05 17:20:01 -08:00