Commit Graph

80 Commits

Author SHA1 Message Date
Nicolas Vasilache
c9fb3c6ea6 [mlir][Tensor] Update ParallelInsertSlicOp semantics to match that of InsertSliceOp
This revision updates the op semantics to also allow rank-reducing behavior as well
as updates the implementation to reuse code between the sequential and the parallel
version of the op.

Depends on D128920

Differential Revision: https://reviews.llvm.org/D128985
2022-07-04 02:37:46 -07:00
Nicolas Vasilache
7fbf55c927 [mlir][Tensor] Move ParallelInsertSlice to the tensor dialect
This is moslty NFC and will allow tensor.parallel_insert_slice to gain
rank-reducing semantics by reusing the vast majority of the tensor.insert_slice impl.

Depends on D128857

Differential Revision: https://reviews.llvm.org/D128920
2022-07-04 01:53:12 -07:00
Nicolas Vasilache
741f8f2bed [mlir][Tensor][NFC] Better document rank-reducing behavior of ExtractSliceOp and cleanup 2022-06-29 07:37:58 -07:00
Jacques Pienaar
2d70eff802 [mlir] Flip more uses to prefixed accessor form (NFC).
Try to keep the final flip small. Need to flip MemRef as there are many
templated cases with it and Tensor.
2022-06-26 19:12:38 -07:00
Aart Bik
e7d3ba1066 [mlir][sparse] accept sparse reshape (expand/collapse)
This revision makes sure we accept sparse tensors as arguments
of the expand/collapse reshaping operations in the tensor dialect.
Note that the actual lowering to runnable IR is still TBD.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D128311
2022-06-22 09:40:38 -07:00
Kazu Hirata
6d5fc1e3d5 [mlir] Don't use Optional::getValue (NFC) 2022-06-20 23:20:25 -07:00
Kazu Hirata
037f09959a [mlir] Don't use Optional::hasValue (NFC) 2022-06-20 11:22:37 -07:00
Jacques Pienaar
eca86cb2ed [mlir] Start migrating more dialects to prefixed form
Marked all dialects that could be (reasonably) easily flipped to _Both
prefix. Updating the accessors to prefixed form will happen in follow
up, this was to flush out conflicts and to mark all dialects explicitly
as I plan to flip OpBase default to _Prefixed to avoid needing to
migrate new dialects.

Except for Standalone example which got flipped to _Prefixed.

Differential Revision: https://reviews.llvm.org/D128027
2022-06-18 10:10:31 -07:00
Thomas Raoux
f2676b151d [mlir][tensor] Add canonicalization for tensor.cast from extract_slice
Propagate static size information into extract_slice producer if
possible.

Differential Revision: https://reviews.llvm.org/D125972
2022-05-19 17:34:59 +00:00
Chris Lattner
f21896f2c6 [DenseElementAttr] Simplify the public API for creating these.
Instead of requiring the client to compute the "isSplat" bit,
compute it internally.  This makes the logic more consistent
and defines away a lot of "elements.size()==1" in the clients.

This addresses Issue #55185

Differential Revision: https://reviews.llvm.org/D125447
2022-05-12 16:18:23 +01:00
River Riddle
eda6f907d2 [mlir][NFC] Shift a bunch of dialect includes from the .h to the .cpp
Now that dialect constructors are generated in the .cpp file, we can
drop all of the dependent dialect includes from the .h file.

Differential Revision: https://reviews.llvm.org/D124298
2022-04-23 01:09:29 -07:00
Matthias Springer
8544523dcb [mlir][tensor] Promote extract(from_elements(...)) to folding pattern
Differential Revision: https://reviews.llvm.org/D123617
2022-04-20 23:47:42 +09:00
gysit
973dbe20f6 [mlir][tensor] Add pattern to fold ExtractSliceOp, PadOp chains.
The pattern folds chains of tensor::ExtractSliceOp, tensor::PadOp pairs if they pad different dimensions. Repeated tiling and padding of the tiled dimensions may introduce such chains. This canonicalization pattern folds these chains to a single tensor::ExtractSliceOp, tensor::PadOp pair that pads all dimensions at once, which simplifies vectorization and bufferization.

Example:
```mlir
   %0 = tensor.extract_slice %input[16, 0] [%sz0, 64] [1, 1]
       : tensor<64x64xf32> to tensor<?x64xf32>
   %1 = tensor.pad %0 low[0, 0] high[%pw0, 0] { ...
     } : tensor<?x64xf32> to tensor<8x64xf32>
   %2 = tensor.extract_slice %1[0, 4] [8, %sz1] [1, 1]
        : tensor<8x64xf32> to tensor<8x?xf32>
   %res = tensor.pad %2 nofold low[0, 0] high[0, %pw1] { ...
     } : tensor<8x?xf32> to tensor<8x4xf32>
```
folds into:
 ```mlir
   %0 = tensor.extract_slice %input[16, 4] [%sz0, %sz1] [1, 1]
        : tensor<64x64xf32> to tensor<?x?xf32>
   %res = tensor.pad %0 nofold low[0, 0] high[%pw0, %pw1] { ...
     } : tensor<?x?xf32> to tensor<8x4xf32>
 ```

Reviewed By: nicolasvasilache, hanchung

Differential Revision: https://reviews.llvm.org/D122722
2022-04-11 14:28:59 +00:00
Alexander Belyaev
747b10be95 Revert "Revert "[mlir] Rewrite canonicalization of collapse(expand) and expand(collapse).""
This reverts commit 96e9b6c9dc.
2022-04-06 12:18:30 +02:00
Hanhan Wang
96e9b6c9dc Revert "[mlir] Rewrite canonicalization of collapse(expand) and expand(collapse)."
This reverts commit 64f659bee6.

An invalid tensor.expand_shape op is generated with the commit. To repro:

$ mlir-opt -canonicalize a.mlir

```
func @foo(%0: tensor<1x1xf32>, %1: tensor<1x1xf32>, %2: tensor<1x1xf32>) -> tensor<1x1xf32> {
  %cst = arith.constant 0.000000e+00 : f32
  %3 = linalg.init_tensor [8, 1] : tensor<8x1xf32>
  %4 = linalg.fill ins(%cst : f32) outs(%3 : tensor<8x1xf32>) -> tensor<8x1xf32>
  %5 = tensor.collapse_shape %0 [] : tensor<1x1xf32> into tensor<f32>
  %6 = tensor.insert_slice %5 into %4[0, 0] [1, 1] [1, 1] : tensor<f32> into tensor<8x1xf32>
  %7 = linalg.init_tensor [8, 1] : tensor<8x1xf32>
  %8 = linalg.fill ins(%cst : f32) outs(%7 : tensor<8x1xf32>) -> tensor<8x1xf32>
  %9 = tensor.collapse_shape %2 [] : tensor<1x1xf32> into tensor<f32>
  %10 = tensor.insert_slice %9 into %8[0, 0] [1, 1] [1, 1] : tensor<f32> into tensor<8x1xf32>
  %11 = tensor.collapse_shape %6 [[0, 1]] : tensor<8x1xf32> into tensor<8xf32>
  %12 = linalg.init_tensor [8] : tensor<8xf32>
  %13 = linalg.generic {indexing_maps = [affine_map<(d0) -> (d0)>, affine_map<(d0) -> (d0)>], iterator_types = ["parallel"]} ins(%11 : tensor<8xf32>) outs(%12 : tensor<8xf32>) {
  ^bb0(%arg3: f32, %arg4: f32):
    linalg.yield %arg3 : f32
  } -> tensor<8xf32>
  %14 = tensor.expand_shape %13 [[0, 1, 2, 3]] : tensor<8xf32> into tensor<1x1x8x1xf32>
  %15 = tensor.collapse_shape %1 [] : tensor<1x1xf32> into tensor<f32>
  %16 = linalg.init_tensor [] : tensor<f32>
  %17 = linalg.generic {indexing_maps = [affine_map<() -> ()>, affine_map<() -> ()>], iterator_types = []} ins(%15 : tensor<f32>) outs(%16 : tensor<f32>) {
  ^bb0(%arg3: f32, %arg4: f32):
    linalg.yield %arg3 : f32
  } -> tensor<f32>
  %18 = tensor.expand_shape %17 [] : tensor<f32> into tensor<1x1x1x1xf32>
  %19 = tensor.collapse_shape %10 [[0, 1]] : tensor<8x1xf32> into tensor<8xf32>
  %20 = linalg.init_tensor [8] : tensor<8xf32>
  %21 = linalg.generic {indexing_maps = [affine_map<(d0) -> (d0)>, affine_map<(d0) -> (d0)>], iterator_types = ["parallel"]} ins(%19 : tensor<8xf32>) outs(%20 : tensor<8xf32>) {
  ^bb0(%arg3: f32, %arg4: f32):
    linalg.yield %arg3 : f32
  } -> tensor<8xf32>
  %22 = tensor.expand_shape %21 [[0, 1, 2, 3]] : tensor<8xf32> into tensor<1x1x8x1xf32>
  %23 = linalg.mmt4d {comment = "f32*f32->f32, aarch64, matrix*vector"} ins(%14, %18 : tensor<1x1x8x1xf32>, tensor<1x1x1x1xf32>) outs(%22 : tensor<1x1x8x1xf32>) -> tensor<1x1x8x1xf32>
  %24 = tensor.collapse_shape %23 [[0, 1, 2, 3]] : tensor<1x1x8x1xf32> into tensor<8xf32>
  %25 = linalg.init_tensor [8] : tensor<8xf32>
  %26 = linalg.generic {indexing_maps = [affine_map<(d0) -> (d0)>, affine_map<(d0) -> (d0)>], iterator_types = ["parallel"]} ins(%24 : tensor<8xf32>) outs(%25 : tensor<8xf32>) {
  ^bb0(%arg3: f32, %arg4: f32):
    linalg.yield %arg3 : f32
  } -> tensor<8xf32>
  %27 = tensor.expand_shape %26 [[0, 1]] : tensor<8xf32> into tensor<8x1xf32>
  %28 = tensor.extract_slice %27[0, 0] [1, 1] [1, 1] : tensor<8x1xf32> to tensor<f32>
  %29 = tensor.expand_shape %28 [] : tensor<f32> into tensor<1x1xf32>
  return %29 : tensor<1x1xf32>
}
```

Differential Revision: https://reviews.llvm.org/D123161
2022-04-05 15:05:41 -07:00
Alexander Belyaev
64f659bee6 [mlir] Rewrite canonicalization of collapse(expand) and expand(collapse).
Differential Revision: https://reviews.llvm.org/D122666
2022-04-05 10:03:07 +02:00
Markus Böck
e13d23bc6c [mlir] Rename OpAsmParser::OperandType to OpAsmParser::UnresolvedOperand
I am not sure about the meaning of Type in the name (was it meant be interpreted as Kind?), and given the importance and meaning of Type in the context of MLIR, its probably better to rename it. Given the comment in the source code, the suggestion in the GitHub issue and the final discussions in the review, this patch renames the OperandType to UnresolvedOperand.

Fixes https://github.com/llvm/llvm-project/issues/54446

Differential Revision: https://reviews.llvm.org/D122142
2022-03-21 21:42:13 +01:00
Thomas Raoux
b4d08dfd9d [mlir] Remove incorrect builders for ExpandShapeOp
ExpandShapeOp builder cannot infer the result type since it doesn't know
how the dimension needs to be split. Remove this builder so that it
doesn't get used accidently. Also remove one potential path using it in
generic fusion.

Differential Revision: https://reviews.llvm.org/D122019
2022-03-18 22:31:17 +00:00
Matthias Springer
fdb41a22ac [mlir][tensor] Implement ReifyRankedShapedTypeOpInterface on GenerateOp
Differential Revision: https://reviews.llvm.org/D121520
2022-03-16 18:59:27 +09:00
Chia-hung Duan
ed645f6336 [mlir] Support verification order (3/3)
In this CL, update the function name of verifier according to the
behavior. If a verifier needs to access the region then it'll be updated
to `verifyRegions`.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D120373
2022-03-11 01:16:28 +00:00
Mahesh Ravishankar
589eac6524 [mlir] Add canonicalizations for op -> tensor.cast folding.
A `tensor.cast` consumer can be folded with its producer. This is
beneficial only if the result of the tensor cast is more static than
the source. This patch adds a utility function to check that this is
the case, and adds a couple of canonicalizations patterns that fold an
operation with `tensor.cast` conusmers.

Reviewed By: gysit

Differential Revision: https://reviews.llvm.org/D120950
2022-03-08 18:26:55 +00:00
Okwan Kwon
4c901bf447 [mlir] Match Arithmetic::ConstantOp and Tensor::ExtractSliceOp.
Add a pattern matcher for ExtractSliceOp when its source is a constant.

The matching heuristics can be governed by the control function since
generating a new constant is not always beneficial.

Differential Revision: https://reviews.llvm.org/D119605
2022-02-28 23:09:03 +00:00
Okwan Kwon
4f5eb53e68 Revert "[mlir] Fold Arithmetic::ConstantOp and Tensor::ExtractSliceOp."
This reverts commit 3104994104.
2022-02-28 19:14:05 +00:00
Okwan Kwon
3104994104 [mlir] Fold Arithmetic::ConstantOp and Tensor::ExtractSliceOp.
Fold ExtractSliceOp when the source is a constant.
2022-02-28 17:47:29 +00:00
Okwan Kwon
f79f430d4b Fold Tensor.extract_slice into a constant splat.
Fold arith.extract_slice into arith.constant when the source is a constant
splat and the result type is statically shaped.
2022-02-22 21:39:57 +00:00
Benjamin Kramer
1af15de6b7 [mlir] Switch {collapse,expand}_shape ops to the declarative assembly format
Same functionality, a lot less code.
2022-02-17 20:00:05 +01:00
Ivan Butygin
cd0d095c07 [mlir][tensor] Check ops generated by InsertSliceOpCastFolder are valid
Fixes https://github.com/llvm/llvm-project/issues/53099

Differential Revision: https://reviews.llvm.org/D119663
2022-02-13 21:37:31 +03:00
River Riddle
2418cd92c0 [mlir] Update uses of parser/printer ODS op field to hasCustomAssemblyFormat
The parser/printer fields are deprecated and in the process of being removed.
2022-02-07 19:03:58 -08:00
Benjamin Kramer
6635c12ada [mlir] Use SmallBitVector instead of SmallDenseSet for AffineMap::compressSymbols
This is both more efficient and more ergonomic to use, as inverting a
bit vector is trivial while inverting a set is annoying.

Sadly this leaks into a bunch of APIs downstream, so adapt them as well.

This would be NFC, but there is an ordering dependency in MemRefOps's
computeMemRefRankReductionMask. This is now deterministic, previously it
was dependent on SmallDenseSet's unspecified iteration order.

Differential Revision: https://reviews.llvm.org/D119076
2022-02-07 00:21:44 +01:00
River Riddle
ead1107257 [mlir] Move StandardOps/Utils to Arithmetic and sever a bunch of dependencies on Standard
The Utils.cpp file in StandardOps essentially just contains utilities for interacting with arithmetic
operations, and at this point makes more sense as a utility file for the arithemtic dialect.

Differential Revision: https://reviews.llvm.org/D118280
2022-02-02 14:45:12 -08:00
River Riddle
6a8ba3186e [mlir] Split std.splat into tensor.splat and vector.splat
This is part of the larger effort to split the standard dialect. This will also allow for pruning some
additional dependencies on Standard (done in a followup).

Differential Revision: https://reviews.llvm.org/D118202
2022-02-02 14:45:12 -08:00
River Riddle
b98dc0351a [mlir][NFC] Update MemRef/Tensor operations to use hasVerifier instead of verifier
The verifier field is deprecated, and slated for removal.

Differential Revision: https://reviews.llvm.org/D118821
2022-02-02 13:34:30 -08:00
Frederik Gossen
2c7b0685e1 Fix tensor.extract for complex elements 2022-01-28 04:33:15 +01:00
Rob Suderman
7c984be21a [mlir] Propagate arith.index_cast past tensor.extract
If we are extracting it is more useful to push the index_cast past the
extraction. This increases the chance the tensor.extract can evaluated at
compile time.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D118204
2022-01-25 22:16:07 -08:00
Rob Suderman
d81a3c51e7 [mlir] Fold tensor.reshape operations into tensor.from_elements.
There is not much of a benefit to reshape a from element vs reloading it.
Updated to progagate shape manipulations into the output type of
tensor.from_elements.

Reviewed By: NatashaKnk

Differential Revision: https://reviews.llvm.org/D118201
2022-01-25 15:54:57 -08:00
Alexander Belyaev
fd0c6f5391 [mlir] Move linalg::PadTensorOp to tensor::PadOp.
RFC: https://llvm.discourse.group/t/rfc-move-linalg-padtensorop-to-tensor-padop/5785

Differential Revision: https://reviews.llvm.org/D117892
2022-01-21 20:02:39 +01:00
River Riddle
e084679f96 [mlir] Make locations required when adding/creating block arguments
BlockArguments gained the ability to have locations attached a while ago, but they
have always been optional. This goes against the core tenant of MLIR where location
information is a requirement, so this commit updates the API to require locations.

Fixes #53279

Differential Revision: https://reviews.llvm.org/D117633
2022-01-19 17:35:35 -08:00
River Riddle
d75c3e8396 [mlir] Don't print // no predecessors on entry blocks
Entry blocks can never have predecessors, so this is unnecessary.

Fixes #53287

Differential Revision: https://reviews.llvm.org/D117713
2022-01-19 15:57:58 -08:00
River Riddle
676bfb2a22 [mlir] Refactor ShapedType into an interface
ShapedType was created in a time before interfaces, and is one of the earliest
type base classes in the ecosystem. This commit refactors ShapedType into
an interface, which is what it would have been if interfaces had existed at that
time. The API of ShapedType and it's derived classes are essentially untouched
by this refactor, with the exception being the API surrounding kDynamicIndex
(which requires a sole home).

For now, the API of ShapedType and its name have been kept as consistent to
the current state of the world as possible (to help with potential migration churn,
among other reasons). Moving forward though, we should look into potentially
restructuring its API and possible its name as well (it should really have "Interface"
at the end like other interfaces at the very least).

One other potentially interesting note is that I've attached the ShapedType::Trait
to TensorType/BaseMemRefType to act as mixins for the ShapedType API. This
is kind of weird, but allows for sharing the same API (i.e. preventing API loss from
the transition from base class -> Interface). This inheritance doesn't affect any
of the derived classes, it is just for API mixin.

Differential Revision: https://reviews.llvm.org/D116962
2022-01-12 14:12:09 -08:00
Mehdi Amini
e4853be2f1 Apply clang-tidy fixes for performance-for-range-copy to MLIR (NFC) 2022-01-02 22:19:56 +00:00
MaheshRavishankar
7df7586a0b [mlir][MemRef] Deprecate unspecified trailing offset, size, and strides semantics of OffsetSizeAndStrideOpInterface.
The semantics of the ops that implement the
`OffsetSizeAndStrideOpInterface` is that if the number of offsets,
sizes or strides are less than the rank of the source, then some
default values are filled along the trailing dimensions (0 for offset,
source dimension of sizes, and 1 for strides). This is confusing,
especially with rank-reducing semantics. Immediate issue here is that
the methods of `OffsetSizeAndStridesOpInterface` assumes that the
number of values is same as the source rank. This cause out-of-bounds
errors.

So simplifying the specification of `OffsetSizeAndStridesOpInterface`
to make it invalid to specify number of offsets/sizes/strides not
equal to the source rank.

Differential Revision: https://reviews.llvm.org/D115677
2021-12-29 11:18:29 -08:00
Mehdi Amini
02b6fb218e Fix clang-tidy issues in mlir/ (NFC)
Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D115956
2021-12-20 20:25:01 +00:00
Alexander Belyaev
88df30c8d8 [mlir] Add canonicalization for extract(tensor.from_elements) in 0d case.
Differential Revision: https://reviews.llvm.org/D115875
2021-12-16 15:46:57 +01:00
Alexander Belyaev
f77e9f8768 [mlir] Extend tensor.from_elements to support N-D case.
RFC: https://llvm.discourse.group/t/rfc-extend-tensor-fromelementsop-to-n-d/4715

Differential Revision: https://reviews.llvm.org/D115821
2021-12-16 14:52:41 +01:00
Alexander Belyaev
15f8f3e20a [mlir] Split std.rank into tensor.rank and memref.rank.
Move `std.rank` similarly to how `std.dim` was moved to TensorOps and MemRefOps.

Differential Revision: https://reviews.llvm.org/D115665
2021-12-14 10:15:55 +01:00
Nicolas Vasilache
f2e945a393 Revert "[mlir][tensor] Fix insert_slice + tensor cast overflow"
This reverts commit 5601821dae.

The prefix + canonical complete behavior is actually obsolete and should not be reintroduced.
Reverting.
2021-12-10 22:53:52 +00:00
Nicolas Vasilache
5601821dae [mlir][tensor] Fix insert_slice + tensor cast overflow
InsertSliceOp may have subprefix semantics where missing trailing dimensions
are automatically inferred directly from the operand shape.
This revision fixes an overflow that occurs in such cases when the impl is based on the op rank.

Differential Revision: https://reviews.llvm.org/D115549
2021-12-10 21:41:26 +00:00
Alexander Belyaev
b618880e7b [mlir] Move linalg.tensor_expand/collapse_shape to TensorDialect.
RFC: https://llvm.discourse.group/t/rfc-reshape-ops-restructuring/3310

linalg.fill gets a canonicalizer, because `FoldFillWithTensorReshape` cannot be moved to tensorops (it uses linalg::FillOp inside). Before it was listed as a canonicalization pattern for the reshape operations, now it became a canonicalization for FillOp.

Differential Revision: https://reviews.llvm.org/D115502
2021-12-10 12:11:48 +01:00
Mehdi Amini
be0a7e9f27 Adjust "end namespace" comment in MLIR to match new agree'd coding style
See D115115 and this mailing list discussion:
https://lists.llvm.org/pipermail/llvm-dev/2021-December/154199.html

Differential Revision: https://reviews.llvm.org/D115309
2021-12-08 06:05:26 +00:00
Nicolas Vasilache
a08b750ce9 [mlir][tensor] InsertSliceOp verification.
This revision reintroduces tensor.insert_slice verification which seems
to have vanished over time: a verifier was initially introduced in cf9503c1b7
but for some reason the invalid.mlir was not properly updated; as time passed the verifier was not called anymore and later the code was deleted.

As a consequence, a non-negligible portion of tests has run astray using invalid
tensor.insert_slice semantics and needed to be fixed.

Also, extract isRankReducedType from TensorOps for better reuse
Originally, this facility was used by both tensor and memref forms but
it got copied around as dialects were split.

Differential Revision: https://reviews.llvm.org/D114715
2021-11-30 20:37:06 +00:00