MSVC fails to parse this construct, leading to
MlirTranslateMain.cpp(70): error C2065: 'inputSplitMarker': undeclared identifier
Just switching to brace init works around the issue
This allows to define custom splitters, which is interesting for
non-MLIR inputs and outputs to `mlir-translate`. For example, one may
use `; -----` as a splitter of `.ll` files. The splitters are now passed
as arguments into `splitAndProcessBuffer`, the input splitter defaulting
to the previous default (`// -----`) and the output splitter defaulting
to the empty string, which also corresponds to the previous default. The
behavior of the input split marker should not change at all; however,
outputs now have one new line *more* than before if there is no splitter
(old: `insertMarkerInOutput = false`, new: `outputSplitMarker = ""`) and
one new line *less* if there is one. The value of the input splitter is
exposed as a command line options of `mlir-translate` and other tools as
an optional value to the previously existing flag `-split-input-file`,
which defaults to the default splitter if not specified; the value of
the output splitter is exposed with the new `-output-split-marker`,
which default to the empty string in `mlir-translate` and the default
splitter in the other tools. In short, the previous usage or omission of
the flags should result in previous behavior (modulo the new lines
mentioned before).
Fixes that
```
Pattern {
let tuple = (attr<"3 : i34">);
not tuple.0;
erase _;
}
```
would crash the PDLL parser because it expected a native constraint
after `not`.
From https://reviews.llvm.org/D153245
This adds support for native PDL (and PDLL) C++ constraints to return
results.
This is useful for situations where a pattern checks for certain
constraints of multiple interdependent attributes and computes a new
attribute value based on them. Currently, for such an example it is
required to escape to C++ during matching to perform the check and after
a successful match again escape to native C++ to perform the computation
during the rewriting part of the pattern. With this work we can do the
computation in C++ during matching and use the result in the rewriting
part of the pattern. Effectively this enables a choice in the trade-off
of memory consumption during matching vs recomputation of values.
This is an example of a situation where this is useful: We have two
operations with certain attributes that have interdependent constraints.
For instance `attr_foo: one_of [0, 2, 4, 8], attr_bar: one_of [0, 2, 4,
8]` and `attr_foo == attr_bar`. The pattern should only match if all
conditions are true. The new operation should be created with a new
attribute which is computed from the two matched attributes e.g.
`attr_baz = attr_foo * attr_bar`. For the check we already escape to
native C++ and have all values at hand so it makes sense to directly
compute the new attribute value as well:
```
Constraint checkAndCompute(attr0: Attr, attr1: Attr) -> Attr;
Pattern example with benefit(1) {
let foo = op<test.foo>() {attr = attr_foo : Attr};
let bar = op<test.bar>(foo) {attr = attr_bar : Attr};
let attr_baz = checkAndCompute(attr_foo, attr_bar);
rewrite bar with {
let baz = op<test.baz> {attr=attr_baz};
replace bar with baz;
};
}
```
To achieve this the following notable changes were necessary:
PDLL:
- Remove check in PDLL parser that prevented native constraints from
returning results
PDL:
- Change PDL definition of pdl.apply_native_constraint to allow variadic
results
PDL_interp:
- Change PDL_interp definition of pdl_interp.apply_constraint to allow
variadic results
PDLToPDLInterp Pass:
The input to the pass is an arbitrary number of PDL patterns. The pass
collects the predicates that are required to match all of the pdl
patterns and establishes an ordering that allows creation of a single
efficient matcher function to match all of them. Values that are matched
and possibly used in the rewriting part of a pattern are represented as
positions. This allows fusion and thus reusing a single position for
multiple matching patterns. Accordingly, we introduce
ConstraintPosition, which records the type and index of the result of
the constraint. The problem is for the corresponding value to be used in
the rewriting part of a pattern it has to be an input to the
pdl_interp.record_match operation, which is generated early during the
pass such that its surrounding block can be referred to by branching
operations. In consequence the value has to be materialized after the
original pdl.apply_native_constraint has been deleted but before we get
the chance to generate the corresponding pdl_interp.apply_constraint
operation. We solve this by emitting a placeholder value when a
ConstraintPosition is evaluated. These placeholder values (due to fusion
there may be multiple for one constraint result) are replaced later when
the actual pdl_interp.apply_constraint operation is created.
Changes since the phabricator review:
- Addressed all comments
- In particular, removed registerConstraintFunctionWithResults and
instead changed registerConstraintFunction so that contraint functions
always have results (empty by default)
- Thus we don't need to reuse `rewriteFunctions` to store constraint
functions with results anymore, and can instead use
`constraintFunctions`
- Perform a stable sort of ConstraintQuestion, so that
ConstraintQuestion appear before other ConstraintQuestion that use their
results.
- Don't create placeholders for pdl_interp::ApplyConstraintOp. Instead
generate the `pdl_interp::ApplyConstraintOp` before generating the
successor block.
- Fixed a test failure in the pdl python bindings
Original code by @martin-luecke
Co-authored-by: martin-luecke <martinpaul.luecke@amd.com>
The patch enables roundtrip to textual file when running
`--verifyRoundtrip`. The verification is successful if both textual and
bytecode formats can roundtrip successfully.
This PR adds API `makeReproducer` and cl::opt flag
`--mlir-generate-reproducer=<filename>` in order to allow for mlir
reproducer dumps even when the pipeline doesn't crash.
This PR also decouples the code that handles generation of an MLIR
reproducer from the crash recovery portion. The purpose is to allow for
generating reproducers outside of the context of a compiler crash.
This will be useful for frameworks and runtimes that use MLIR where it
is needed to reproduce the pipeline behavior for reasons outside of
diagnosing crashes. An example is for diagnosing performance issues
using offline tools, where being able to dump the reproducer from a
runtime compiler would be helpful.
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
When serializing to bytecode, users can select the option to elide
resources from the bytecode file. This will instruct the bytecode writer
to serialize only the key and resource kind, while skipping
serialization of the data buffer. At parsing, the IR is built in memory
with valid (but empty) resource handlers.
Seprate registeration and CLI parsing from `MlirOptMain` to
`mlir::registrationAndParseCLIOptions` and a new `MlirOptMain` so that
external tools to drive `mlir-opt` can call these two new APIs to get CLI
option values before running `MlirOptMain`.
This commit adds the initial version of the mlir-query tool, which leverages the pre-existing matchers defined in mlir/include/mlir/IR/Matchers.h
The tool provides the following set of basic queries:
hasOpAttrName(string)
hasOpName(string)
isConstantOp()
isNegInfFloat()
isNegZeroFloat()
isNonZero()
isOne()
isOneFloat()
isPosInfFloat()
isPosZeroFloat()
isZero()
isZeroFloat()
Reviewed By: jpienaar
Differential Revision: https://reviews.llvm.org/D155127
This commit enables the expression of negated native constraints in PDLL:
If a constraint is prefixed with "not" it is parsed as a negated constraint and hence the attribute `isNegated` of the emitted `pdl.apply_native_constraint` operation is set to `true`.
In first instance this is only supported for the calling of external native C++ constraints and generation of PDL patterns.
Previously, negating a native constraint would have been handled by creating an additional native call, e.g.
```PDLL
Constraint checkA(input: Attr);
Constarint checkNotA(input: Attr);
```
or by including an explicit additional operand for negation, e.g.
`Constraint checkA(input: Attr, negated: Attr);`
With this a constraint can simply be negated by prefixing it with `not`. e.g.
```PDLL
Constraint simpleConstraint(op: Op);
Pattern example {
let inputOp = op<test.bar>() ->(type: Type);
let root = op<test.foo>(inputOp.0) -> ();
not simpleConstraint(inputOp);
simpleConstraint(root);
erase root;
}
```
Depends on [[ https://reviews.llvm.org/D153871 | D153871 ]]
Reviewed By: Mogball
Differential Revision: https://reviews.llvm.org/D153959
Functions are always callable operations and thus every operation
implementing the `FunctionOpInterface` also implements the
`CallableOpInterface`. The only exception was the FuncOp in the toy
example. To make implementation of the `FunctionOpInterface` easier,
this commit lets `FunctionOpInterface` inherit from
`CallableOpInterface` and merges some of their methods. More precisely,
the `CallableOpInterface` has methods to get the argument and result
attributes and a method to get the result types of the callable region.
These methods are always implemented the same way as their analogues in
`FunctionOpInterface` and thus this commit moves all the argument and
result attribute handling methods to the callable interface as well as
the methods to get the argument and result types. The
`FuntionOpInterface` then does not have to declare them as well, but
just inherits them from the `CallableOpInterface`.
Adding the inheritance relation also required to move the
`FunctionOpInterface` from the IR directory to the Interfaces directory
since IR should not depend on Interfaces.
Reviewed By: jpienaar, springerm
Differential Revision: https://reviews.llvm.org/D157988
This wires in attribute and type aliases into the MLIR LSP server. This
will allow goto definition and find references on attribute and type
references, which should make debugging locations and other metadata
easier.
Depends on D158781
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D158782
This allows for users of the lsp transport libraries to process replies
in parallel, without overlapping/clobbering the output.
Differential Revision: https://reviews.llvm.org/D156295
This is specified in the spec, but we just never really needed it. This
allows for users of the LSP libraries to inspect information about the
client that is connected to the server.
Differential Revision: https://reviews.llvm.org/D155566
In preparation for removing the `#include "llvm/ADT/StringExtras.h"`
from the header to source file of `llvm/Support/Error.h`, first add in
all the missing includes that were previously included transitively
through this header.
This class has been causing me no end of grief for a long time, and the way it is used by mlir-tblgen is technically an ODR violation in certain situations.
Due to the way that the build is layered, it is important that the MLIR tablegen libraries only depend on the LLVM tablegen libraries, not on anything else (like MLIRSupport). It has to be this way because these libraries/binaries are special and must pre-exist the full shared libraries. Therefore, the dependency chain must be clean (and static).
At some point, someone pulled out a separate build target for just IndendedOstream in an attempt to satisfy the constraint. But because it is weird in different ways, this target was never installed properly as part of distributions, etc -- this causes problems for downstreams seeking to build a tblggen binary that doesn't itself have ODR/shared library problems.
I was attempting to fix the distribution stuff but just opted to collapse this into a header-only library and not try to solve this with build layering. I think this is the safest and the least bad thing for such a dep. This also makes for a clean comment that actually explains the constraint (which I was having trouble verbalizing with the weird subset dependency).
Differential Revision: https://reviews.llvm.org/D153393
The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionality in addition to defining methods with the same name.
This change begins the migration of uses of the method to the
corresponding function call as has been decided as more consistent.
Note that there still exist classes that only define methods directly,
such as AffineExpr, and this does not include work currently to support
a functional cast/isa call.
Context:
- https://mlir.llvm.org/deprecation/ at "Use the free function variants
for dyn_cast/cast/isa/…"
- Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443
Implementation:
This patch updates all remaining uses of the deprecated functionality in
mlir/. This was done with clang-tidy as described below and further
modifications to GPUBase.td and OpenMPOpsInterfaces.td.
Steps are described per line, as comments are removed by git:
0. Retrieve the change from the following to build clang-tidy with an
additional check:
main...tpopp:llvm-project:tidy-cast-check
1. Build clang-tidy
2. Run clang-tidy over your entire codebase while disabling all checks
and enabling the one relevant one. Run on all header files also.
3. Delete .inc files that were also modified, so the next build rebuilds
them to a pure state.
```
ninja -C $BUILD_DIR clang-tidy
run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\
-header-filter=mlir/ mlir/* -fix
rm -rf $BUILD_DIR/tools/mlir/**/*.inc
```
Differential Revision: https://reviews.llvm.org/D151542
Running:
MLIR_OPT_CHECK_IR_ROUNDTRIP=1 ninja check-mlir
will now exercises all of our test with a round-trip to bytecode and a comparison for equality.
Reviewed By: rriddle, ftynse, jpienaar
Differential Revision: https://reviews.llvm.org/D90088
The revision adds a flag to mlir translate that suppresses
any non-error diagnostics. The flag is useful when importing
LLVM IR to LLVM dialect, which produces a lot of
warnings due to dropped metadata and debug intrinsics.
Reviewed By: Dinistro
Differential Revision: https://reviews.llvm.org/D150547
The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionality in addition to defining methods with the same name.
This change begins the migration of uses of the method to the
corresponding function call as has been decided as more consistent.
Note that there still exist classes that only define methods directly,
such as AffineExpr, and this does not include work currently to support
a functional cast/isa call.
Caveats include:
- This clang-tidy script probably has more problems.
- This only touches C++ code, so nothing that is being generated.
Context:
- https://mlir.llvm.org/deprecation/ at "Use the free function variants
for dyn_cast/cast/isa/…"
- Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443
Implementation:
This first patch was created with the following steps. The intention is
to only do automated changes at first, so I waste less time if it's
reverted, and so the first mass change is more clear as an example to
other teams that will need to follow similar steps.
Steps are described per line, as comments are removed by git:
0. Retrieve the change from the following to build clang-tidy with an
additional check:
https://github.com/llvm/llvm-project/compare/main...tpopp:llvm-project:tidy-cast-check
1. Build clang-tidy
2. Run clang-tidy over your entire codebase while disabling all checks
and enabling the one relevant one. Run on all header files also.
3. Delete .inc files that were also modified, so the next build rebuilds
them to a pure state.
4. Some changes have been deleted for the following reasons:
- Some files had a variable also named cast
- Some files had not included a header file that defines the cast
functions
- Some files are definitions of the classes that have the casting
methods, so the code still refers to the method instead of the
function without adding a prefix or removing the method declaration
at the same time.
```
ninja -C $BUILD_DIR clang-tidy
run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\
-header-filter=mlir/ mlir/* -fix
rm -rf $BUILD_DIR/tools/mlir/**/*.inc
git restore mlir/lib/IR mlir/lib/Dialect/DLTI/DLTI.cpp\
mlir/lib/Dialect/Complex/IR/ComplexDialect.cpp\
mlir/lib/**/IR/\
mlir/lib/Dialect/SparseTensor/Transforms/SparseVectorization.cpp\
mlir/lib/Dialect/Vector/Transforms/LowerVectorMultiReduction.cpp\
mlir/test/lib/Dialect/Test/TestTypes.cpp\
mlir/test/lib/Dialect/Transform/TestTransformDialectExtension.cpp\
mlir/test/lib/Dialect/Test/TestAttributes.cpp\
mlir/unittests/TableGen/EnumsGenTest.cpp\
mlir/test/python/lib/PythonTestCAPI.cpp\
mlir/include/mlir/IR/
```
Differential Revision: https://reviews.llvm.org/D150123
When tooling out there produces a reproducer that is archived, the first thing
a user is likely to expect is to process this as they do with any MLIR file.
However https://reviews.llvm.org/D126447 changed the behavior of mlir-opt to
eliminate the `--run-reproducer` option and instead automatically run it when
present in the input file. This creates a discrepancy in how mlir-opt behaves
when fed with an input file, and is surprising for users.
The explicit passing of `--run-reproducer` to express user-intent seems more
in line with what is expected from `mlir-opt`.
Reviewed By: rriddle, jpienaar
Differential Revision: https://reviews.llvm.org/D149820
Can't return a well-formed IR output while enabling version to be bumped
up during emission. Previously it would return min version but
potentially invalid IR which was confusing, instead make it return
error and abort immediately instead.
Differential Revision: https://reviews.llvm.org/D149569
Add method to set a desired bytecode file format to generate. Change
write method to be able to return status including the minimum bytecode
version needed by reader. This enables generating an older version of
the bytecode (not dialect ops, attributes or types). But this does not
guarantee that an older version can always be generated, e.g., if a
dialect uses a new encoding only available at later bytecode version.
This clamps setting to at most current version.
Differential Revision: https://reviews.llvm.org/D146555
This includes a small runtime acting as callback for the ExecutionEngine
and a C API that makes it possible to control from the debugger.
A python script for LLDB is included that hook a new `mlir` subcommand
and allows to set breakpoints and inspect the current action, the context
and the stack.
Differential Revision: https://reviews.llvm.org/D144817
This patch add support for loading IRDL dialects at runtime
with `mlir-opt`.
Given the following `dialect.irdl` file:
```mlir
module {
irdl.dialect @cmath {
irdl.type @complex {
%0 = irdl.is f32
%1 = irdl.is f64
%2 = irdl.any_of(%0, %1)
irdl.parameters(%2)
}
irdl.operation @norm {
%0 = irdl.any
%1 = irdl.parametric @complex<%0>
irdl.operands(%1)
irdl.results(%0)
}
}
```
the IRDL file can be loaded with the `mlir-opt --irdl-file=dialect.irdl`
command, and the following file can then be parsed:
```mlir
func.func @conorm(%p: !cmath.complex<f32>, %q: !cmath.complex<f32>) -> f32 {
%norm_p = "cmath.norm"(%p) : (!cmath.complex<f32>) -> f32
%norm_q = "cmath.norm"(%q) : (!cmath.complex<f32>) -> f32
%pq = arith.mulf %norm_p, %norm_q : f32
return %pq : f32
}
```
To minimize the size of this patch, the operation, attribute, and type verifier are all always returning `success()`.
Depends on D144692
Reviewed By: rriddle, Mogball, mehdi_amini
Differential Revision: https://reviews.llvm.org/D144693
This will match the locations attached to the IRunits passed in as context
with an action.
This is a recommit of d09c80515d after fixing the test on Windows.
Differential Revision: https://reviews.llvm.org/D144815
This patch add support for loading IRDL dialects at runtime
with `mlir-opt`.
Given the following `dialect.irdl` file:
```mlir
module {
irdl.dialect @cmath {
irdl.type @complex {
%0 = irdl.is f32
%1 = irdl.is f64
%2 = irdl.any_of(%0, %1)
irdl.parameters(%2)
}
irdl.operation @norm {
%0 = irdl.any
%1 = irdl.parametric @complex<%0>
irdl.operands(%1)
irdl.results(%0)
}
}
```
the IRDL file can be loaded with the `mlir-opt --irdl-file=dialect.irdl`
command, and the following file can then be parsed:
```mlir
func.func @conorm(%p: !cmath.complex<f32>, %q: !cmath.complex<f32>) -> f32 {
%norm_p = "cmath.norm"(%p) : (!cmath.complex<f32>) -> f32
%norm_q = "cmath.norm"(%q) : (!cmath.complex<f32>) -> f32
%pq = arith.mulf %norm_p, %norm_q : f32
return %pq : f32
}
```
To minimize the size of this patch, the operation, attribute, and type verifier are all always returning `success()`.
Depends on D144692
Reviewed By: rriddle, Mogball, mehdi_amini
Differential Revision: https://reviews.llvm.org/D144693
Implementation of Pass and Dialect Plugins that mirrors LLVM Pass Plugin
implementation from the new pass manager.
Currently the implementation only supports using the pass-pipeline option
for adding passes. This restriction is imposed by the `PassPipelineCLParser`
variable in mlir/lib/Tools/mlir-opt/MlirOptMain.cpp:114 that loads the
parse options statically before parsing the cmd line args.
```
mlir-opt stanalone-plugin.mlir --load-dialect-plugin=lib/libStandalonePlugin.so --pass-pipeline="builtin.module(standalone-switch-bar-foo)"
```
Reviewed By: rriddle, mehdi_amini
Differential Revision: https://reviews.llvm.org/D147053
Implementation of Pass and Dialect Plugins that mirrors LLVM Pass Plugin implementation from the new pass manager.
Currently the implementation only supports using the pass-pipeline option for adding passes. This restriction is imposed by the `PassPipelineCLParser` variable in mlir/lib/Tools/mlir-opt/MlirOptMain.cpp:114 that loads the parse options statically before parsing the cmd line args.
```
mlir-opt stanalone-plugin.mlir --load-dialect-plugin=lib/libStandalonePlugin.so --pass-pipeline="builtin.module(standalone-switch-bar-foo)"
```
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D147053
