This revision modifies the mem2reg interfaces and algorithm to be more
omfortable to use as a pattern. The motivation behind this is that
currently the pattern needs to be applied to the scope op of the region
in which allocators should be promoted. However, a more natural way to
apply the pattern would be to apply it on the allocator directly. This
is not only clearer but easier to parallelize.
This revision changes the mem2reg pattern to operate this way. This
required restraining the interfaces to only mutate IR using
RewriterBase, as the previously used escape hatch is not granular enough
to match on the region that is modified only. This has the unfortunate
cost of preventing batching allocator promotion and making the block
argument adding logic more complex. Because batching no longer made any
sense, I made the internal analyzer/promoter decoupling private again.
This also adds statistics to the mem2reg infrastructure.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D150432
Types have been introduced a while ago and provide for better
readability and transform-time verification. Use them in the ops from
the structured transform dialect extension.
In most cases, the types are appended as trailing functional types or a
derived format of the functional type that allows for an empty right
hand size without the annoying `-> ()` syntax (similarly to `func.func`
declaration that may omit the arrow). When handles are used inside mixed
static/dynamic lists, such as tile sizes, types of those handles follow
them immediately as in `sizes [%0 : !transform.any_value, 42]`. This
allows for better readability than matching the trailing type.
Update code to remove hardcoded PDL dependencies and expunge PDL from
structured transform op code.
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D144515
This adds QuantizationDialect to the dependent dialects of TOSA
This fixes the intermittent bug when creating uniform quantized type when none was parsed in.
LLVM ERROR: can't create type 'mlir::quant::UniformQuantizedType' because storage uniquer isn't initialized: the dialect was likely not loaded, or the type wasn't added with addTypes<...>() in the Dialect::initialize() method.
This happens, for example, in convert-tfl-uint8 pass when trying to create uniform quantized type i8 with zero-point=-128 to convert from ui8 type.
Signed-off-by: Tai Ly <tai.ly@arm.com>
Change-Id: I204248a45fd728d0cec8dc20214cb0b74de81e7b
Reviewed By: eric-k256
Differential Revision: https://reviews.llvm.org/D149354
The sparse compiler now has two prototype strategies for GPU acceleration:
* CUDA codegen: this converts sparsified code to CUDA threads
* CUDA libgen: this converts pre-sparsified code to cuSPARSE library calls
This revision introduces the first steps required for the second approach.
Reviewed By: ThomasRaoux
Differential Revision: https://reviews.llvm.org/D150170
Structured fusion proceeds by iteratively finding the next suitable
producer to be fused into the loop. Therefore, it shouldn't matter if
the same producer is listed multiple times (e.g., it is used as multiple
operands). Adjust the implementation of the transform op to support this
case.
Also fix the checking code in the interpreter to actually respect the
TransformOpInterface indication that repeated payload is allowed, it
seems to have been accidentally dropped in one of the refactorings.
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D150561
Instead of passing traversal options as a long list of arguments, store them in a TraversalConfig object and pass that object.
Differential Revision: https://reviews.llvm.org/D143927
Instead of returning an `ArrayRef<Operation *>`, return at iterator that skips ops that were erased/replaced while iterating over the payload ops.
This fixes an issue in conjuction with TrackingListener, where a tracked op was erased during the iteration. Elements may not be removed from an array while iterating over it; this invalidates the iterator.
When ops are erased/removed via `replacePayloadOp`, they are not immediately removed from the mappings data structure. Instead, they are set to `nullptr`. `nullptr`s are not enumerated by `getPayloadOps`. At the end of each transformation, `nullptr`s are removed from the mapping data structure.
Differential Revision: https://reviews.llvm.org/D149847
This commit removes constness from DILocation::getMergedLocation and
fixes all its users accordingly.
Having constness on the parameters forced the return type to be const
as well, which does force usage of `const_cast` when the location needs
to be used in metadata nodes.
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D149942
Add if condition removal pattern for acc.host_data in a same way as
acc.enter_data, acc.exit_data and acc.update.
The condition is removed from the op if it is a true constant. If
it is a false constant then the region is inlined before the op
and the op is removed.
Reviewed By: vzakhari
Differential Revision: https://reviews.llvm.org/D150480
Tosa supports fp16 and fp32 accumulator type for fp16 input, but
no way to tell for computational operators whether accumulator
should be fp16 or fp32 from input type. Add this new attribute
to specify the type.
Set to fp32 by default for now. When the time fp16 is supported,
the accumulator type can be selected based on trade-off between
performance and accuracy.
Reviewed By: jpienaar
Differential Revision: https://reviews.llvm.org/D146317
This revision extends the GPU dialect with ops that can be lowered to
host-oriented sparse matrix library calls (in this case cuSparse focused
although the ops could be generalized to support more GPUs in principle).
This will allow the "sparse compiler pipeline" to accelerate sparse operations
(see follow up revisions with examples of this).
For some background;
https://discourse.llvm.org/t/sparse-compiler-and-gpu-code-generation/69786/2
Reviewed By: ThomasRaoux
Differential Revision: https://reviews.llvm.org/D150152
This patch adds verification on registered IRDL operations, types,
and attributes.
This is done through an interface implemented by operations from the
`irdl` dialect, which translate the operations into `Constraint`.
This interface is then use in the `registerDialect` function to
generate verifiers for the entire operation/type/attribute.
Depends on D145733
Reviewed By: Mogball
Differential Revision: https://reviews.llvm.org/D145734
This change adds the following three operations and unit tests for them:
- conv_3d_ncdhw_fcdhw
- depthwise_conv_1d_ncw_cw
- depthwise_conv_3d_ncdhw_cdhw
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D150054
The TrackingListener was unnecessarily strict. Existing ops are now allowed when updating payload ops mappings due to `replaceOp` in the TrackingListener.
Differential Revision: https://reviews.llvm.org/D150429
The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionality in addition to defining methods with the same name.
This change begins the migration of uses of the method to the
corresponding function call as has been decided as more consistent.
Note that there still exist classes that only define methods directly,
such as AffineExpr, and this does not include work currently to support
a functional cast/isa call.
Context:
* https://mlir.llvm.org/deprecation/ at "Use the free function variants for dyn_cast/cast/isa/…"
* Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443
Implementation:
This follows a previous patch that updated calls
`op.cast<T>()-> cast<T>(op)`. However some cases could not handle an
unprefixed `cast` call due to occurrences of variables named cast, or
occurring inside of class definitions which would resolve to the method.
All C++ files that did not work automatically with `cast<T>()` are
updated here to `llvm::cast` and similar with the intention that they
can be easily updated after the methods are removed through a
find-replace.
See https://github.com/llvm/llvm-project/compare/main...tpopp:llvm-project:tidy-cast-check
for the clang-tidy check that is used and then update printed
occurrences of the function to include `llvm::` before.
One can then run the following:
```
ninja -C $BUILD_DIR clang-tidy
run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\
-export-fixes /tmp/cast/casts.yaml mlir/*\
-header-filter=mlir/ -fix
rm -rf $BUILD_DIR/tools/mlir/**/*.inc
```
Differential Revision: https://reviews.llvm.org/D150348
The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionality in addition to defining methods with the same name.
This change begins the migration of uses of the method to the
corresponding function call as has been decided as more consistent.
Note that there still exist classes that only define methods directly,
such as AffineExpr, and this does not include work currently to support
a functional cast/isa call.
Caveats include:
- This clang-tidy script probably has more problems.
- This only touches C++ code, so nothing that is being generated.
Context:
- https://mlir.llvm.org/deprecation/ at "Use the free function variants
for dyn_cast/cast/isa/…"
- Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443
Implementation:
This first patch was created with the following steps. The intention is
to only do automated changes at first, so I waste less time if it's
reverted, and so the first mass change is more clear as an example to
other teams that will need to follow similar steps.
Steps are described per line, as comments are removed by git:
0. Retrieve the change from the following to build clang-tidy with an
additional check:
https://github.com/llvm/llvm-project/compare/main...tpopp:llvm-project:tidy-cast-check
1. Build clang-tidy
2. Run clang-tidy over your entire codebase while disabling all checks
and enabling the one relevant one. Run on all header files also.
3. Delete .inc files that were also modified, so the next build rebuilds
them to a pure state.
4. Some changes have been deleted for the following reasons:
- Some files had a variable also named cast
- Some files had not included a header file that defines the cast
functions
- Some files are definitions of the classes that have the casting
methods, so the code still refers to the method instead of the
function without adding a prefix or removing the method declaration
at the same time.
```
ninja -C $BUILD_DIR clang-tidy
run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\
-header-filter=mlir/ mlir/* -fix
rm -rf $BUILD_DIR/tools/mlir/**/*.inc
git restore mlir/lib/IR mlir/lib/Dialect/DLTI/DLTI.cpp\
mlir/lib/Dialect/Complex/IR/ComplexDialect.cpp\
mlir/lib/**/IR/\
mlir/lib/Dialect/SparseTensor/Transforms/SparseVectorization.cpp\
mlir/lib/Dialect/Vector/Transforms/LowerVectorMultiReduction.cpp\
mlir/test/lib/Dialect/Test/TestTypes.cpp\
mlir/test/lib/Dialect/Transform/TestTransformDialectExtension.cpp\
mlir/test/lib/Dialect/Test/TestAttributes.cpp\
mlir/unittests/TableGen/EnumsGenTest.cpp\
mlir/test/python/lib/PythonTestCAPI.cpp\
mlir/include/mlir/IR/
```
Differential Revision: https://reviews.llvm.org/D150123
The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionality in addition to defining methods with the same name.
This change continues the migration of uses of the method to the
corresponding function call as has been decided as more consistent.
This commit attempts to update all occurrences of the casts in .td
files, although it is likely that a couple were missed.
Context:
- https://mlir.llvm.org/deprecation/ at "Use the free function variants for dyn_cast/cast/isa/…"
- Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443
Implementation:
Unfortunatley, this was not automated, but was handled by mindlessly
going to next occurrences of patterns, selecting the piece of code to
be moved into the function call, and running a vim macro over the span
of around 4 hours.
Differential Revision: https://reviews.llvm.org/D150199
Typically GPUs cannot access memory in sub-byte manner. So for
sub-byte integer type values, we need to either expand them to
full bytes or tightly pack them. This commit adds support for
tightly packed power-of-two sub-byte types.
Sub-byte types aren't allowed in SPIR-V spec, so there are no
compute/storage capability for them like other supported integer
types. So we don't recognize sub-byte types in `spirv::ScalarType`.
We just special case them in type converter and always convert
to use i32 under the hood.
Reviewed By: kuhar
Differential Revision: https://reviews.llvm.org/D150395
The acc.host_data operation models the OpenACC
host_data construct (2.8). The host_data construct
defines a region where the address of data in device memory
available on the host. The operation is modeled in a similar way
than acc.data operation.
Reviewed By: razvanlupusoru, jeanPerier
Differential Revision: https://reviews.llvm.org/D150289
Add C and python bindings for InferShapedTypeOpInterface
and ShapedTypeComponents. This allows users to invoke
InferShapedTypeOpInterface for ops that implement it.
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D149494
The information needed for translation is now encoded in the dialect
operations and does not require a dedicated pass to be extracted.
Remove the obsolete passes that were performing operand legalization.
Reviewed By: jeanPerier
Differential Revision: https://reviews.llvm.org/D150248
Improve the constant import to handle zeroinitializer as well as
additional float types such as quad floats. The logic got restructured
to avoid creating intermediate dense element attributes when
constructing multi-dimensional arrays. Additionally, we also leverage
the fact that we do not need to iterate all elements of splat constants.
Reviewed By: Dinistro
Differential Revision: https://reviews.llvm.org/D150274
Remove old clause operands from acc.kernels operation since
the new dataOperands is now in place.
private and firstprivate will receive some redesign but are
not part of the new dataOperands.
Depends on D150224
Reviewed By: vzakhari
Differential Revision: https://reviews.llvm.org/D150225
Remove old clause operands from acc.serial operation since
the new dataOperands is now in place.
private and firstprivate will receive some redesign but are
not part of the new dataOperands.
Depends on D150207
Reviewed By: vzakhari
Differential Revision: https://reviews.llvm.org/D150224
