The new function is a wrapper around the regular `getStridesAndOffset`
that offers a more compact way (as in writing less code) of getting the
relevant information.
This method is intended to be used only when it is known that the
LogicalResult of the regular `getStridesAndOffset` must be "succeeded".
This warpper will assert on that.
Differential Revision: https://reviews.llvm.org/D139529
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
Do not generate CollapseShapeOps/ExpandShapeOps that have the same source and result shape. Generate casts instead. Such reshapes became invalid with D138498.
Differential Revision: https://reviews.llvm.org/D138557
This pass has outgrown its original goal and is now going to be used to
expand certain memref operations before lowering.
Reflect that in the name.
The pass is now called expand-strided-metadata.
NFC
Differential Revision: https://reviews.llvm.org/D138448
The methods in `SideEffectUtils.h` (and their implementations in
`SideEffectUtils.cpp`) seem to have similar intent to methods already
existing in `SideEffectInterfaces.h`. Move the decleration (and
implementation) from `SideEffectUtils.h` (and `SideEffectUtils.cpp`)
into `SideEffectInterfaces.h` (and `SideEffectInterface.cpp`).
Also drop the `SideEffectInterface::hasNoEffect` method in favor of
`mlir::isMemoryEffectFree` which actually recurses into the operation
instead of just relying on the `hasRecursiveMemoryEffectTrait`
exclusively.
Differential Revision: https://reviews.llvm.org/D137857
Essentially, this patches changes the anchor point of the
`extract_strided_metadata(reshapeLikeOp)` pattern from
`extract_strided_metadata` to `reshapeLikeOp`.
In details, this means that instead of replacing:
```
base, offset, sizes, strides =
extract_strided_metadata(reshapeLikeOp(src))
```
With
```
base, offset = extract_strided_metadata(src)
sizes = <some math>
strides = <some math>
```
We replace only the reshapeLikeOp part and connect it back with a
reinterpret_cast:
```
val = reshapeLikeOp(src)
```
=>
```
base, offset, ... = extract_strided_metadata(src)
sizes = <some math>
strides = <some math>
val = reinterpret_cast base, offset, sizes, strides
Differential Revision: https://reviews.llvm.org/D136386
Essentially, this patches changes the anchor point of the
`extract_strided_metadata(subview)` pattern from
`extract_strided_metadata` to `subview`.
In details, this means that instead of replacing:
```
base, offset, sizes, strides = extract_strided_metadata(subview(src))
```
With
```
base, ... = extract_strided_metadata(src)
offset = <some math>
sizes = subSizes
strides = <some math>
```
We replace only the subview part and connect it back with a
reinterpret_cast:
```
val = subview(src)
```
=>
```
base, ... = extract_strided_metadata(src)
offset = <some math>
sizes = subSizes
strides = <some math>
val = reinterpret_cast base, offset, sizes, strides
```
Differential Revision: https://reviews.llvm.org/D135839
Prior to this patch the canonicalization pattern that turns
`reinterpret_cast(extract_strided_metadata)` into cast was only applied
when all the input operands of the `reinterpret_cast` are exactly all the
output results of the `extract_strided_metadata`.
This missed simplification opportunities when the values would have hold
the same constant values, but yet, come from different actual values.
E.g., prior to this patch, a pattern of the form:
```
%base, %offset = extract_strided_metadata %source : memref<i16>
reinterpret_cast %base to offset:[0]
```
Wouldn't have been simplified into a simple cast, because %offset is not
directly the same value object as 0.
This patch teaches this pattern how to check if the constant values
match what the results of the `extract_strided_metadata` operation would
have hold.
Differential Revision: https://reviews.llvm.org/D135736
The `SimplifyExtractStridedMetadata` pass features a pattern that forward
statically known information (offset, sizes, strides) to their respective
users.
This patch moves this pattern from this pass to the
`extract_strided_metadata` folding patterns.
Differential Revision: https://reviews.llvm.org/D135797
Prior to this patch we were wrongly applying the sub-strides to the
computation of the final offset of the subview.
Put differently, we were computing the offset as:
```
offset = baseOffset + sum(subOffset#i * baseStrides#i * subSizes#i)
```
Whereas we should be doing:
```
offset = baseOffset + sum(subOffset#i * baseStrides#i)
```
I.e., drop the subSizes#i term from the sum.
Differential Revision: https://reviews.llvm.org/D136107
The `InferTypeOpInterface` generates builders for things it can infer
the types.
Thanks to that interface we can:
- Eliminate a builder for DimOp, and
- Describe how to infer the result types of `extract_strided_metadata`
from its source, and get a simpler builder as a result
NFC
Differential Revision: https://reviews.llvm.org/D135734
Add a new pass and conversions to emulate wide integer operations over memrefs.
The emulation is implemented on top of the existing pass to emulate wide integer arith ops.
Improve naming in the arith pass to avoid potential name clashes.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D135722
These operations have undefined behavior if the index is not less than the rank of the source tensor / memref, so they cannot be freely speculated like they were before this patch. After this patch we speculate them only if we can prove that they don't have UB.
Depends on D135505.
Reviewed By: mravishankar
Differential Revision: https://reviews.llvm.org/D135748
This patch takes the first step towards a more principled modeling of undefined behavior in MLIR as discussed in the following discourse threads:
1. https://discourse.llvm.org/t/semantics-modeling-undefined-behavior-and-side-effects/4812
2. https://discourse.llvm.org/t/rfc-mark-tensor-dim-and-memref-dim-as-side-effecting/65729
This patch in particular does the following:
1. Introduces a ConditionallySpeculatable OpInterface that dynamically determines whether an Operation can be speculated.
2. Re-defines `NoSideEffect` to allow undefined behavior, making it necessary but not sufficient for speculation. Also renames it to `NoMemoryEffect`.
3. Makes LICM respect the above semantics.
4. Changes all ops tagged with `NoSideEffect` today to additionally implement ConditionallySpeculatable and mark themselves as always speculatable. This combined trait is named `Pure`. This makes this change NFC.
For out of tree dialects:
1. Replace `NoSideEffect` with `Pure` if the operation does not have any memory effects, undefined behavior or infinite loops.
2. Replace `NoSideEffect` with `NoSideEffect` otherwise.
The next steps in this process are (I'm proposing to do these in upcoming patches):
1. Update operations like `tensor.dim`, `memref.dim`, `scf.for`, `affine.for` to implement a correct hook for `ConditionallySpeculatable`. I'm also happy to update ops in other dialects if the respective dialect owners would like to and can give me some pointers.
2. Update other passes that speculate operations to consult `ConditionallySpeculatable` in addition to `NoMemoryEffect`. I could not find any other than LICM on a quick skim, but I could have missed some.
3. Add some documentation / FAQs detailing the differences between side effects, undefined behavior, speculatabilty.
Reviewed By: rriddle, mehdi_amini
Differential Revision: https://reviews.llvm.org/D135505
NFC. Remove unnecessary builder argument in an Affine Utils helper
function: normalizeMemRefType. A builder was never needed. While on
this, fix a clang-tidy warning from the same file.
Reviewed By: dcaballe
Differential Revision: https://reviews.llvm.org/D135423
Extend multi-buffering to simplify the affine map created if any of its operands are constants. This avoids downstream problems where more complex affine.apply operations cannot be expanded.
Transfer attributes from the old allocation to the new allocation.
Reviewed By: ThomasRaoux
Differential Revision: https://reviews.llvm.org/D134894
This interface is implemented by memref.dim and tensor.dim. This change makes it possible to remove a build dependency of the Affine dialect on the Tensor dialect (and maybe also the MemRef dialect in the future).
Differential Revision: https://reviews.llvm.org/D133595
This flips all of the remaining dialects to prefixed except for linalg, which
will be done in a followup.
Differential Revision: https://reviews.llvm.org/D134995
The new pattern gets rid of the collapse_shape operation while
materializing its effects on the sizes, and the strides of the base
object.
In other words, this simplification replaces:
```
baseBuffer, offset, sizes, strides =
extract_strided_metadata(collapse_shape(memref))
```
With
```
baseBuffer, offset, baseSizes, baseStrides =
extract_strided_metadata(memref)
for reassDim in {0 .. collapseRank - 1}
sizes#reassDim = product(baseSizes#i for i in group[reassDim])
strides#reassDim = baseStrides[group[reassDim].back()]
```
Note: baseBuffer and offset are unaffected by the collapse_shape
operation.
Differential Revision: https://reviews.llvm.org/D134826
Add the plumbing necessary to call the memref dialect's multiBuffer
function. This will allow separation between choosing which buffers
to multi-buffer and the actual transform.
Alter the multibuffer function to return the newly created
allocation if multi-buffering succeeds. This is necessary to
communicate with the transform dialect hooks what allocation
multi-buffering created.
Reviewed By: ftynse, nicolasvasilache
Differential Revision: https://reviews.llvm.org/D133985
Teach the pass that simplifies extract_strided_metadata(other_op(memref))
how to get rid of extract_strided_metadata when they are fed by
allocLikeOp.
For the simplification to happen the allocLikeOp needs to have been
normalized. I.e., no weird offset and strides.
When this is the case, we replace:
```
base, offset, sizes, strides =
extract_strided_metadata(allocLikeOp(allocSizes))
```
With
```
base = reinterpret_cast allocLikeOp(allocSizes) to a flat memref<eltTy>
offset = 0
sizes = allocSizes
strides#i = prod(allocSizes#j, for j in {i+1..rank-1})
```
The computation involving dynamic sizes are expanded in affine.apply.
Differential Revision: https://reviews.llvm.org/D134577
`memref.extract_strided_metadata` can forward constants independently of the
exsistence of other operations such as subview or reshape.
Differential Revision: https://reviews.llvm.org/D134603
Add a pattern to the pass that simplifies
extract_strided_metadata(other_op(memref)).
The new pattern gets rid of the expand_shape operation while
materializing its effects on the sizes, and the strides of
the base object.
In other words, this simplification replaces:
```
baseBuffer, offset, sizes, strides =
extract_strided_metadata(expand_shape(memref))
```
With
```
baseBuffer, offset, baseSizes, baseStrides =
extract_strided_metadata(memref)
sizes#reassIdx =
baseSizes#reassDim / product(expandShapeSizes#j,
for j in group excluding
reassIdx)
strides#reassIdx =
baseStrides#reassDim * product(expandShapeSizes#j,
for j in
reassIdx+1..
reassIdx+group.size-1)
```
Where `reassIdx` is a reassociation index for the group at
`reassDim` and `expandShapeSizes#j` is either:
- The constant size at dimension j, derived directly from the
result type of the expand_shape op, or
- An affine expression: baseSizes#reassDim / product of all
constant sizes in expandShapeSizes.
Note: baseBuffer and offset are unaffected by the expand_shape
operation.
Differential Revision: https://reviews.llvm.org/D133625
Add memref.realloc and canonicalization of the op. Add conversion patterns for
lowering the op to LLVM using unaligned alloc or aligned alloc based on the
conversion option.
Add filecheck tests for parsing and converting the op. Add an integration test.
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D133424
This function must be implemented for all ops, where the result memref type is different from the input memref type.
Differential Revision: https://reviews.llvm.org/D134331
Currently, there's an optimization that claims dimensions of size 1 are always
contiguous. This is not necessarily the case for subviews.
```
Input:
[
[
[0, 1],
[2, 3]
],
[
[4, 5]
[6, 7]
]
]
Subview:
[
[
[0, 1],
],
[
[4, 5]
]
]
```
The old logic treats this subview as contiguous, when it is not.
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D134026
The three following ops in the memref dialect: transpose, expand_shape,
collapse_shape, have been originally designed to operate on memrefs with
strided layouts but had to go through the affine map representation as the type
did not support anything else. Make these ops produce memref values with
StridedLayoutAttr instead now that it is available.
Depends On D133938
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D133947
Memref subview operation has been initially designed to work on memrefs with
strided layouts only and has never supported anything else. Port it to use the
recently added StridedLayoutAttr instead of extracting the strided from
implicitly from affine maps.
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D133938
Add a dedicated pass to simplify
extract_strided_metadata(other_op(memref)).
Currently the pass features only one pattern:
extract_strided_metadata(subview).
The goal is to get rid of the subview while materializing its effects on
the offset, sizes, and strides with respect to the base object.
In other words, this simplification replaces:
```
baseBuffer, offset, sizes, strides =
extract_strided_metadata(
subview(memref, subOffset, subSizes, subStrides))
```
With
```
baseBuffer, baseOffset, baseSizes, baseStrides =
extract_strided_metadata(memref)
strides#i = baseStrides#i * subSizes#i
offset = baseOffset + sum(subOffset#i * strides#i)
sizes = subSizes
```
Differential Revision: https://reviews.llvm.org/D133166
The patch introduces the required changes to update the pass declarations and definitions to use the new autogenerated files and allow dropping the old infrastructure.
Reviewed By: mehdi_amini, rriddle
Differential Review: https://reviews.llvm.org/D132838
The patch introduces the required changes to update the pass declarations and definitions to use the new autogenerated files and allow dropping the old infrastructure.
Reviewed By: mehdi_amini, rriddle
Differential Review: https://reviews.llvm.org/D132838
Introduce a new attribute to represent the strided memref layout. Strided
layouts are omnipresent in code generation flows and are the only kind of
layouts produced and supported by a half of operation in the memref dialect
(view-related, shape-related). However, they are internally represented as
affine maps that require a somewhat fragile extraction of the strides from the
linear form that also comes with an overhead. Furthermore, textual
representation of strided layouts as affine maps is difficult to read: compare
`affine_map<(d0, d1, d2)[s0, s1] -> (d0*32 + d1*s0 + s1 + d2)>` with
`strides: [32, ?, 1], offset: ?`. While a rudimentary support for parsing a
syntactically sugared version of the strided layout has existed in the codebase
for a long time, it does not go as far as this commit to make the strided
layout a first-class attribute in the IR.
This introduces the attribute and updates the tests that using the pre-existing
sugared form to use the new attribute instead. Most memref created
programmatically, e.g., in passes, still use the affine form with further
extraction of strides and will be updated separately.
Update and clean-up the memref type documentation that has gotten stale and has
been referring to the details of affine map composition that are long gone.
See https://discourse.llvm.org/t/rfc-materialize-strided-memref-layout-as-an-attribute/64211.
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D132864
Add a canonicalizetion step for
reinterpret_cast(extract_strided_metadata).
This step replaces this sequence of operations by either:
- A noop, i.e., the original memref is directly used, or
- A plain cast of the original memref
The choice is ultimately made based on whether the original memref type
is equal to what the reinterpret_cast iss producing. For instance, the
reinterpret_cast could be changing some dimensions from static to
dynamic and in such case, we need to keep a cast.
The transformation is currently only performed when the reinterpret_cast
uses exactly the same arguments as what the extract_strided_metadata
produces. It may be possible to be more aggressive here but I wanted to
start with a relatively simple MLIR patch for my first one!
Differential Revision: https://reviews.llvm.org/D132776
