Support extra concrete class declarations and definitions under NativeTrait that get injected into the class that specifies the trait. Extra declarations and definitions can be passed in as template arguments for NativeOpTraitNativeAttrTrait and NativeTypeTrait.
Usage examples of this feature include:
- Creating a wrapper Trait for authoring inferReturnTypes with the OpAdaptor by specifying necessary Op specific declarations and definitions directly in the trait
- Refactoring the InferTensorType trait
Reviewed By: jpienaar
Differential Revision: https://reviews.llvm.org/D154731
This class has been causing me no end of grief for a long time, and the way it is used by mlir-tblgen is technically an ODR violation in certain situations.
Due to the way that the build is layered, it is important that the MLIR tablegen libraries only depend on the LLVM tablegen libraries, not on anything else (like MLIRSupport). It has to be this way because these libraries/binaries are special and must pre-exist the full shared libraries. Therefore, the dependency chain must be clean (and static).
At some point, someone pulled out a separate build target for just IndendedOstream in an attempt to satisfy the constraint. But because it is weird in different ways, this target was never installed properly as part of distributions, etc -- this causes problems for downstreams seeking to build a tblggen binary that doesn't itself have ODR/shared library problems.
I was attempting to fix the distribution stuff but just opted to collapse this into a header-only library and not try to solve this with build layering. I think this is the safest and the least bad thing for such a dep. This also makes for a clean comment that actually explains the constraint (which I was having trouble verbalizing with the weird subset dependency).
Differential Revision: https://reviews.llvm.org/D153393
This is adding a new interface (`BytecodeOpInterface`) to allow operations to
opt-in skipping conversion to attribute and serializing properties to native
bytecode.
The scheme relies on a new section where properties are stored in sequence
{ size, serialize_properties }, ...
The operations are storing the index of a properties, a table of offset is
built when loading the properties section the first time.
This is a re-commit of 837d1ce0dc which conflicted with another patch upgrading
the bytecode and the collision wasn't properly resolved before.
Differential Revision: https://reviews.llvm.org/D151065
The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionality in addition to defining methods with the same name.
This change begins the migration of uses of the method to the
corresponding function call as has been decided as more consistent.
Note that there still exist classes that only define methods directly,
such as AffineExpr, and this does not include work currently to support
a functional cast/isa call.
Context:
- https://mlir.llvm.org/deprecation/ at "Use the free function variants
for dyn_cast/cast/isa/…"
- Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443
Implementation:
This patch updates all remaining uses of the deprecated functionality in
mlir/. This was done with clang-tidy as described below and further
modifications to GPUBase.td and OpenMPOpsInterfaces.td.
Steps are described per line, as comments are removed by git:
0. Retrieve the change from the following to build clang-tidy with an
additional check:
main...tpopp:llvm-project:tidy-cast-check
1. Build clang-tidy
2. Run clang-tidy over your entire codebase while disabling all checks
and enabling the one relevant one. Run on all header files also.
3. Delete .inc files that were also modified, so the next build rebuilds
them to a pure state.
```
ninja -C $BUILD_DIR clang-tidy
run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\
-header-filter=mlir/ mlir/* -fix
rm -rf $BUILD_DIR/tools/mlir/**/*.inc
```
Differential Revision: https://reviews.llvm.org/D151542
This reverts commit ca5a12fd69
and follow-up fixes:
df34c288c407dc906883ab80ad0095837d1ce0dc
The first commit was incomplete and broken, I'll prepare a new version
later, in the meantime pull this work out of tree.
This is adding a new interface (`BytecodeOpInterface`) to allow operations to
opt-in skipping conversion to attribute and serializing properties to native
bytecode.
The scheme relies on a new section where properties are stored in sequence
{ size, serialize_properties }, ...
The operations are storing the index of a properties, a table of offset is
built when loading the properties section the first time.
Back-deployment to version prior to 4 are relying on getAttrDictionnary() which
we intend to deprecate and remove: that is putting a de-factor end-of-support
horizon for supporting deployments to version older than 4.
Differential Revision: https://reviews.llvm.org/D151065
This new features enabled to dedicate custom storage inline within operations.
This storage can be used as an alternative to attributes to store data that is
specific to an operation. Attribute can also be stored inside the properties
storage if desired, but any kind of data can be present as well. This offers
a way to store and mutate data without uniquing in the Context like Attribute.
See the OpPropertiesTest.cpp for an example where a struct with a
std::vector<> is attached to an operation and mutated in-place:
struct TestProperties {
int a = -1;
float b = -1.;
std::vector<int64_t> array = {-33};
};
More complex scheme (including reference-counting) are also possible.
The only constraint to enable storing a C++ object as "properties" on an
operation is to implement three functions:
- convert from the candidate object to an Attribute
- convert from the Attribute to the candidate object
- hash the object
Optional the parsing and printing can also be customized with 2 extra
functions.
A new options is introduced to ODS to allow dialects to specify:
let usePropertiesForAttributes = 1;
When set to true, the inherent attributes for all the ops in this dialect
will be using properties instead of being stored alongside discardable
attributes.
The TestDialect showcases this feature.
Another change is that we introduce new APIs on the Operation class
to access separately the inherent attributes from the discardable ones.
We envision deprecating and removing the `getAttr()`, `getAttrsDictionary()`,
and other similar method which don't make the distinction explicit, leading
to an entirely separate namespace for discardable attributes.
Recommit d572cd1b06 after fixing python bindings build.
Differential Revision: https://reviews.llvm.org/D141742
This new features enabled to dedicate custom storage inline within operations.
This storage can be used as an alternative to attributes to store data that is
specific to an operation. Attribute can also be stored inside the properties
storage if desired, but any kind of data can be present as well. This offers
a way to store and mutate data without uniquing in the Context like Attribute.
See the OpPropertiesTest.cpp for an example where a struct with a
std::vector<> is attached to an operation and mutated in-place:
struct TestProperties {
int a = -1;
float b = -1.;
std::vector<int64_t> array = {-33};
};
More complex scheme (including reference-counting) are also possible.
The only constraint to enable storing a C++ object as "properties" on an
operation is to implement three functions:
- convert from the candidate object to an Attribute
- convert from the Attribute to the candidate object
- hash the object
Optional the parsing and printing can also be customized with 2 extra
functions.
A new options is introduced to ODS to allow dialects to specify:
let usePropertiesForAttributes = 1;
When set to true, the inherent attributes for all the ops in this dialect
will be using properties instead of being stored alongside discardable
attributes.
The TestDialect showcases this feature.
Another change is that we introduce new APIs on the Operation class
to access separately the inherent attributes from the discardable ones.
We envision deprecating and removing the `getAttr()`, `getAttrsDictionary()`,
and other similar method which don't make the distinction explicit, leading
to an entirely separate namespace for discardable attributes.
Differential Revision: https://reviews.llvm.org/D141742
Replace references to enumerate results with either result_pairs
(reference wrapper type) or structured bindings. I did not use
structured bindings everywhere as it wasn't clear to me it would
improve readability.
This is in preparation to the switch to zip semantics which won't
support non-const lvalue reference to elements:
https://reviews.llvm.org/D144503.
I chose to use values instead of const lvalue-refs because MLIR is
biased towards avoiding `const` local variables. This won't degrade
performance because currently `result_pair` is cheap to copy (size_t
+ iterator), and in the future, the enumerator iterator dereference
will return temporaries anyway.
Reviewed By: dblaikie
Differential Revision: https://reviews.llvm.org/D146006
This allows discouraging the use of specific build methods of an op by having TableGen generate C++ code, instructing the C++ compiler to warn on use of the `build` method.
The implementation uses the C++14 `[[deprecated(...)]]`` for this purpose. I considered using `LLVM_DEPRECATED`, but thought requiring a fix-it was not necassery, nor would the syntax in ODS have been very nice.
My motivation for this change is that in the future I'd like to deprecate the use `build` methods in the LLVM Dialect, not using explicit pointer types for ops such as `llvm.load` or `llvm.alloca`, which makes the code not future proof for opaque pointers. In-tree has to be clean first before I could commit such a change of course, but I thought the initial infrastructure change could already be submitted.
Differential Revision: https://reviews.llvm.org/D143190
This allows for interfaces to define a set of "base classes",
which are interfaces whose methods/extra class decls/etc.
should be inherited by the derived interface. This more
easily enables combining interfaces and their dependencies,
without lots of awkard casting. Additional implicit conversion
operators also greatly simplify the conversion process.
One other aspect of this "inheritance" is that we also implicitly
add the base interfaces to the attr/op/type. The user can still
add them manually if desired, but this should help remove some
of the boiler plate when an interface has dependencies.
See https://discourse.llvm.org/t/interface-inheritance-and-dependencies-interface-method-visibility-interface-composition
Differential Revision: https://reviews.llvm.org/D140198
SymbolOpInterface overrides the base classof to provide support
for optionally implementing the interface. This is currently placed
in the extraClassDeclarations, but that is kind of awkard given that
it requires underlying knowledge of how the base classof is implemented.
This commit adds a proper "extraClassOf" field to allow interfaces to
implement this, which abstracts away the default classof logic.
Differential Revision: https://reviews.llvm.org/D140197
As discussed in https://reviews.llvm.org/D140886, emitting a warning if the old API is used may be beneficial to encourage migration to the new fold API.
This reuse the existing `Deprecated` infrastructure within TableGen, and simply marks the `def` for `kEmitRawAttributesFolder` causing a use of it in a record (even if set within a base class) to emit a warning.
Error message as printed in the terminal:
```
Included from C:/llvm-project/mlir/python/mlir/dialects/TensorOps.td:13:
Included from C:/llvm-project/mlir/include\mlir/Dialect/Tensor/IR/TensorOps.td:12:
C:/llvm-project/mlir/include\mlir/Dialect/Tensor/IR/TensorBase.td:14:5: warning: Using deprecated def `kEmitRawAttributesFolder`
def Tensor_Dialect : Dialect {
^
note: 'useFoldAPI' of 'kEmitRawAttributesFolder' (default) has been deprecated and is pending removal. Please switch to 'kEmitFoldAdaptorFolder'. See https://discourse.llvm.org/t/psa-new-improved-fold-method-signature-has-landed-please-update-your-downstream-projects/67618
```
Differential Revision: https://reviews.llvm.org/D141604
Incrementing past the end iterator of any container in C++ is immediate undefined behaviour.
This is guaranteed to occur in the loop condition due to the expression cur = earlyIncIt++, which when earlyIncIt is the end iterator (aka we just did the last iteration of the loop), will do an increment on the end iterator.
To fix this, the patch refactors the loop to a more conventional loop using iterators, with the only difference being that the increment happens through the erase operation, which conveniently returns an iterator to the element after the erased element. Thanks to that guarantee there is also no need to use std::list over std::vector.
I also opted to reduce the inner loop find_if, because I felt splitting the "search" and the effects of if it was successful made the code (subjectively) nicer, and also avoided having to add an extra "bool erased" to the outer loop body.
Differential Revision: https://reviews.llvm.org/D141758
Ops that use TypesMatchWith to constrain result types for verification
and to infer result types during parser generation should also be able
to have the `inferReturnTypes` method auto generated. This patch
upgrades the logic for generating `inferReturnTypes` to handle the
TypesMatchWith trait by building a type inference graph where each edge
corresponds to "type of A can be inferred from type of B", supporting
transformers other than `"$_self"`.
Reviewed By: lattner, rriddle
Differential Revision: https://reviews.llvm.org/D141231
This is part of the RFC for a better fold API: https://discourse.llvm.org/t/rfc-a-better-fold-api-using-more-generic-adaptors/67374
This patch implements the required foldHook changes and the TableGen machinery for generating `fold` method signatures using `FoldAdaptor` for ops, based on the value of `useFoldAPI` of the dialect. It may be one of 2 values, with convenient named constants to create a quasi enum. The new `fold` method will then be generated if `kEmitFoldAdaptorFolder` is used.
Since the new `FoldAdaptor` approach is strictly better than the old signature, part of this patch updates the documentation and all example to encourage use of the new `fold` signature.
Included are also tests exercising the new API, ensuring proper construction of the `FoldAdaptor` and proper generation by TableGen.
Differential Revision: https://reviews.llvm.org/D140886
This is part of the RFC for a better fold API: https://discourse.llvm.org/t/rfc-a-better-fold-api-using-more-generic-adaptors/67374
This patch implements the generation of generic adaptors through TableGen. These are essentially a generalization of Adaptors, as implemented previously, but instead of indexing into a `mlir::ValueRange`, they may index into any container, regardless of the element type. This allows the use of the convenient getter methods of Adaptors to be reused on ranges that are the result of some kind of mapping functions of an ops operands.
In the case of the fold API in the RFC, this would be `ArrayRef<Attribute>`, which is a mapping of the operands to their possibly-constant values.
Implementation wise, some special care was taken to not cause a compile time regression, nor to break any kind of source compatibility.
For that purpose, the current adaptor class was split into three:
* A generic adaptor base class, within the detail namespace as it is an implementation detail, which implements all APIs independent of the range type used for the operands. This is all the attribute and region related code. Since it is not templated, its implementation does not have to be inline and can be put into the cpp source file
* The actual generic adaptor, which has a template parameter for the range that should be indexed into for retrieving operands. It implements all the getters for operands, as they are dependent on the range type. It publicly inherits from the generic adaptor base class
* A class named as adaptors have been named so far, inheriting from the generic adaptor class with `mlir::ValueRange` as range to index into. It implements the rest of the API, specific to `mlir::ValueRange` adaptors, which have previously been part of the adaptor. This boils down to a constructor from the Op type as well as the verify function.
The last class having the exact same API surface and name as Adaptors did previously leads to full source compatibility.
Differential Revision: https://reviews.llvm.org/D140660
value() has undesired exception checking semantics and calls
__throw_bad_optional_access in libc++. Moreover, the API is unavailable without
_LIBCPP_NO_EXCEPTIONS on older Mach-O platforms (see
_LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS).
The remove*Attr methods were not being generated with the correct
camelcase method.
Depends on D139470
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D139471
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
This patch consolidates the notions of an optional parameter and a
default parameter. An optional parameter is a parameter equal to its
default value, which for a "purely optional" parameter is its "null"
value.
This allows the existing `comparator` and `defaultValue` fields to be
used enabled more complex "optional" parameters, such as empty arrays.
Depends on D133812
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D133816
Most dialects have already flipped to prefixed, and the intention to switch
has been telegraphed for a while.
Differential Revision: https://reviews.llvm.org/D133179
Its header was already part of the TableGen library, but unusable as uses of its functions or classes would lead to undefined references when linking. This fixes that.
The pass tablegen backend has been reworked to remove the monolithic nature of the autogenerated declarations.
The pass public header can be generated with the -gen-pass-decls option. It contains options structs and registrations: the inclusion of options structs can be controlled individually for each pass by defining the GEN_PASS_DECL_PASSNAME macro; the declaration of the registrations have been kept together and can still be included by defining the GEN_PASS_REGISTRATION macro.
The private code used for the pass implementation (i.e. the pass base class and the constructors definitions, if missing from tablegen) can be generated with the -gen-pass-defs option. Similarly to the declarations file, the definitions of each pass can be enabled by defining the GEN_PASS_DEF_PASNAME variable.
While doing so, the pass base class has been enriched to also accept a the aformentioned struct of options and copy them to the actual pass options, thus allowing each pass to also be configurable within C++ and not only through command line.
Reviewed By: rriddle, mehdi_amini, Mogball, jpienaar
Differential Revision: https://reviews.llvm.org/D131839
Previously we are using IntegerAttr to back all SPIR-V enum
attributes. Therefore we all such attributes are showed like
IntegerAttr in IRs, which is barely readable and breaks
roundtripability of the IR. This commit changes to use
`EnumAttr` as the base directly so that we can have separate
attribute definitions and better IR printing.
Reviewed By: kuhar
Differential Revision: https://reviews.llvm.org/D131311
This has previously been done for `mlir-opt` and `mlir-reduce` and roughly the same approach has been done here.
The use case for having a separate library is that it is easier for downstream to make custom TableGen backends/executable that work on top of the utilities that are defined in `mlir/TableGen`.
The customization point here is the same one as for any upstream TableGen backends: One can add a new generator by simply creating a global instance of `mlir::GenRegistration`.
Differential Revision: https://reviews.llvm.org/D131112
This patch removes the `type` field from `Attribute` along with the
`Attribute::getType` accessor.
Going forward, this means that attributes in MLIR will no longer have
types as a first-class concept. This patch lays the groundwork to
incrementally remove or refactor code that relies on generic attributes
being typed. The immediate impact will be on attributes that rely on
`Attribute` containing a type, such as `IntegerAttr`,
`DenseElementsAttr`, and `ml_program::ExternAttr`, which will now need
to define a type parameter on their storage classes. This will save
memory as all other attribute kinds will no longer contain a type.
Moreover, it will not be possible to generically query the type of an
attribute directly. This patch provides an attribute interface
`TypedAttr` that implements only one method, `getType`, which can be
used to generically query the types of attributes that implement the
interface. This interface can be used to retain the concept of a "typed
attribute". The ODS-generated accessor for a `type` parameter
automatically implements this method.
Next steps will be to refactor the assembly formats of certain operations
that rely on `parseAttribute(type)` and `printAttributeWithoutType` to
remove special handling of type elision until `type` can be removed from
the dialect parsing hook entirely; and incrementally remove uses of
`TypedAttr`.
Reviewed By: lattner, rriddle, jpienaar
Differential Revision: https://reviews.llvm.org/D130092
This warning was added because using attribute or type assembly formats
with `skipDefaultBuilders` set could cause compilation errors, since the
required builder prototype may not necessarily be generated and would
need to be checked by hand. This patch removes the warning because a
warning that the generated C++ "might" not compile is not particularly
useful. Attempting to address the TODO (i.e. detect whether a builder of
the correct prototype is provided) would be fragile since it would not
be possible to account for implicit conversions, etc.
In general, ODS should not be emitting warnings in cases like these.
Reviewed By: rriddle, wrengr
Differential Revision: https://reviews.llvm.org/D130102
This warning was added because using attribute or type assembly formats
with `skipDefaultBuilders` set could cause compilation errors, since the
required builder prototype may not necessarily be generated and would
need to be checked by hand. This patch removes the warning because a
warning that the generated C++ "might" not compile is not particularly
useful. Attempting to address the TODO (i.e. detect whether a builder of
the correct prototype is provided) would be fragile since it would not
be possible to account for implicit conversions, etc.
In general, ODS should not be emitting warnings in cases like these.
For AttrDef declarations, place specified code in extraClassDefinition into the generated *.cpp.inc file.
Reviewed By: Mogball, rriddle
Differential Revision: https://reviews.llvm.org/D129574
This patch allows custom attribute and type builders to return
something other than the C++ type of the attribute or type.
This is useful for attributes or types that may perform extra work during
construction (e.g. canonicalization) that could result in a different
kind of attribute or type being returned.
Reviewed By: rriddle, lattner
Differential Revision: https://reviews.llvm.org/D129792
By making TypeInterfaces and AttrInterfaces, Types and Attrs respectively it'd then be possible to use them anywhere where a Type or Attr may go. That is within the arguments and results of an Op definition, in a RewritePattern etc.
Prior to this change users had to separately define a Type or Attr, with a predicate to check whether a type or attribute implements a given interface. Such code will be redundant now.
Removing such occurrences in upstream dialects will be part of a separate patch.
As part of implementing this patch, slight refactoring had to be done. In particular, Interfaces cppClassName field was renamed to cppInterfaceName as it "clashed" with TypeConstraints cppClassName. In particular Interfaces cppClassName expected just the class name, without any namespaces, while TypeConstraints cppClassName expected a fully qualified class name.
Differential Revision: https://reviews.llvm.org/D129209
Per @rriddle, we do not want to require `skipDefaultBuilders=0` per se; that is, even though the `assemblyFormat`-generated parser requires a builder with the same prototype as the default-builder, that prototype could instead be implemented via custom `builders`. This differential reduces the FatalError introduced in D128555 to a non-fatal Warning instead, so that users can still be informed of the error condition (rather than waiting for the C++ compiler to fail).
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D129234
This differential improves two error conditions, by detecting them earlier and by providing better messages to help users understand what went wrong.
Reviewed By: jpienaar
Differential Revision: https://reviews.llvm.org/D128555
