This patch fixes:
mlir/lib/Pass/PassRegistry.cpp:425:37: error: ISO C++ requires the
name after '::~' to be found in the same scope as the name before
'::~' [-Werror,-Wdtor-name]
Today, when the pass infra schedules a pass/nested-pipeline on a set of
ops, it exits early as soon as it fails on one of the ops. This leads to
non-exhaustive, and more importantly, non-deterministic error reporting
(under async).
This PR removes the early termination behavior so that all ops have a
chance to run through the current pass/nested-pipeline, and all errors
are reported (async diagnostics are already ordered). This guarantees
deterministic & full error reporting. As a result, it's also no longer
necessary to -split-input-file with one error per split when testing
with -verify-diagnostics.
As specified in the docs,
1) raw_string_ostream is always unbuffered and
2) the underlying buffer may be used directly
( 65b13610a5 for further reference )
* Don't call raw_string_ostream::flush(), which is essentially a no-op.
* Avoid unneeded calls to raw_string_ostream::str(), to avoid excess indirection.
- Added a default parsing implementation to `PassOptions` to allow
`Option`/`ListOption` to wrap PassOption objects. This is helpful when
creating meta-pipelines (pass pipelines composed of pass pipelines).
- Updated `ListOption` printing to enable round-tripping the output of
`dump-pass-pipeline` back into `mlir-opt` for more complex structures.
Currently, the only way to see the passes that were registered is by
calling “mlir-opt --help”. However, for compilers with 500+ passes, the
help message becomes too long and sometimes hard to understand. In this
PR I add a new "--list-passes" option to mlir-opt, which can be used for
printing only the registered passes, a feature that would be extremely
useful.
This tutorial gives an introduction to the `mlir-opt` tool, focusing on
how to run basic passes with and without options, run pass pipelines
from the CLI, and point out particularly useful flags.
---------
Co-authored-by: Jeremy Kun <j2kun@users.noreply.github.com>
Co-authored-by: Mehdi Amini <joker.eph@gmail.com>
The PassRegistry parser properly handles escape tokens (', ", {}) when
parsing pass options from string but then does not strip the escape
tokens when providing the values back to the caller.
This change updates the parser such that escape tokens are properly
removed and whitespace is trimmed when extracting option values.
This change expands the existing instrumentation that prints the IR
before/after each pass to an output stream (usually stderr). It adds
a new configuration that will print the output of each pass to a
separate file. The files will be organized into a directory tree
rooted at a specified directory. For existing tools, a CL option
`-mlir-print-ir-tree-dir` is added to specify this directory and
activate the new printing config.
The created directory tree mirrors the nesting structure of the IR. For
example,
if the IR is congruent to the pass-pipeline
"builtin.module(pass1,pass2,func.func(pass3,pass4),pass5)", and
`-mlir-print-ir-tree-dir=/tmp/pipeline_output`, then then the tree file
tree
created will look like:
```
/tmp/pass_output
├── builtin_module_the_symbol_name
│ ├── 0_pass1.mlir
│ ├── 1_pass2.mlir
│ ├── 2_pass5.mlir
│ ├── func_func_my_func_name
│ │ ├── 1_0_pass3.mlir
│ │ ├── 1_1_pass4.mlir
│ ├── func_func_my_other_func_name
│ │ ├── 1_0_pass3.mlir
│ │ ├── 1_1_pass4.mlir
```
The subdirectories are named by concatenating the relevant parent
operation names and symbol name (if present). The printer keeps a
counter associated with ops that are targeted by passes and their
isolated-from-above parents. Each filename is given a numeric prefix
using the counter value for the op that the pass is targeting and then
prepending the counter values for each parent. This gives a naming
where it is easy to distinguish which passes may have run concurrently
vs. which have a clear ordering. In the above example, for both
`1_1_pass4.mlir` files, the first `1` refers to the counter for the
parent op, and the second refers to the counter for the respective
function.
There is no good way to report detailed errors from inside
`Pass::initializeOptions` function as context may not be available at
this point and writing directly to `llvm::errs()` is not composable.
See
https://github.com/llvm/llvm-project/pull/87166#discussion_r1546426763
* Add error handler callback to `Pass::initializeOptions`
* Update `PassOptions::parseFromString` to support custom error stream
instead of using `llvm::errs()` directly.
* Update default `Pass::initializeOptions` implementation to propagate
error string from `parseFromString` to new error handler.
* Update `MapMemRefStorageClassPass` to report error details using new
API.
This PR adds API `makeReproducer` and cl::opt flag
`--mlir-generate-reproducer=<filename>` in order to allow for mlir
reproducer dumps even when the pipeline doesn't crash.
This PR also decouples the code that handles generation of an MLIR
reproducer from the crash recovery portion. The purpose is to allow for
generating reproducers outside of the context of a compiler crash.
This will be useful for frameworks and runtimes that use MLIR where it
is needed to reproduce the pipeline behavior for reasons outside of
diagnosing crashes. An example is for diagnosing performance issues
using offline tools, where being able to dump the reproducer from a
runtime compiler would be helpful.
This patch moves PassExecutionAction to Pass.h so that it can be used by
the action framework to introspect and intercede in pass managers that
might be set up opaquely. This provides for a very particular use case,
which essentially involves being able to intercede in a PassManager and
skip or apply individual passes. Because of this, this patch also adds a
test for this use case to verify that it could in fact work.
This patch fixes:
mlir/lib/Pass/PassRegistry.cpp:376:37: error: ISO C++ requires the
name after '::~' to be found in the same scope as the name before
'::~' [-Werror,-Wdtor-name]
The following pattern fails on recent GCC versions with -std=c++20 flag
passed and succeeds with -std=c++17. Such behavior is not observed on
Clang 16.0.
```
template <typename T>
struct Foo {
Foo<T>(int a) {}
};
```
This patch removes template parameter from constructor in two occurences
to make the following command complete successfully:
bazel build -c fastbuild --cxxopt=-std=c++20 --host_cxxopt=-std=c++20
@llvm-project//mlir/...
This patch is similar to https://reviews.llvm.org/D154782
Co-authored-by: Alexander Batashev <a.batashev@partner.samsung.com>
The same transform op can now be used to apply registered pass pipelines.
This revision also adds a helper function for querying `PassPipelineInfo` objects and moves the corresponding `lookup` function for `PassInfo` objects to the `PassInfo` class.
Differential Revision: https://reviews.llvm.org/D159211
The current logic hashes the context to detect registration changes and re-run
the pass initialization. However it wasn't checking for changes to the
pipeline, so a pass that would get added after a first run would not be
initialized during subsequent runs.
Reviewed By: Mogball
Differential Revision: https://reviews.llvm.org/D158377
Add extra error checking to prevent passes from being run on unsupported ops through the pass manager infrastructure.
Differential Revision: https://reviews.llvm.org/D153144
The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionality in addition to defining methods with the same name.
This change begins the migration of uses of the method to the
corresponding function call as has been decided as more consistent.
Note that there still exist classes that only define methods directly,
such as AffineExpr, and this does not include work currently to support
a functional cast/isa call.
Context:
- https://mlir.llvm.org/deprecation/ at "Use the free function variants
for dyn_cast/cast/isa/…"
- Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443
Implementation:
This patch updates all remaining uses of the deprecated functionality in
mlir/. This was done with clang-tidy as described below and further
modifications to GPUBase.td and OpenMPOpsInterfaces.td.
Steps are described per line, as comments are removed by git:
0. Retrieve the change from the following to build clang-tidy with an
additional check:
main...tpopp:llvm-project:tidy-cast-check
1. Build clang-tidy
2. Run clang-tidy over your entire codebase while disabling all checks
and enabling the one relevant one. Run on all header files also.
3. Delete .inc files that were also modified, so the next build rebuilds
them to a pure state.
```
ninja -C $BUILD_DIR clang-tidy
run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\
-header-filter=mlir/ mlir/* -fix
rm -rf $BUILD_DIR/tools/mlir/**/*.inc
```
Differential Revision: https://reviews.llvm.org/D151542
Implementation of Pass and Dialect Plugins that mirrors LLVM Pass Plugin implementation from the new pass manager.
Currently the implementation only supports using the pass-pipeline option for adding passes. This restriction is imposed by the `PassPipelineCLParser` variable in mlir/lib/Tools/mlir-opt/MlirOptMain.cpp:114 that loads the parse options statically before parsing the cmd line args.
```
mlir-opt stanalone-plugin.mlir --load-dialect-plugin=lib/libStandalonePlugin.so --pass-pipeline="builtin.module(standalone-switch-bar-foo)"
```
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D147053
-mlir-print-ir-module-scope option cannot be used without disabling multithread for pass manager. For the usability, we can throw a validation error in mlir-opt instead of assertion failure.
Issue: https://github.com/llvm/llvm-project/issues/61578
Reviewed By: mehdi_amini
Differential Revision: https://reviews.llvm.org/D146785
IRUnit is defined as:
using IRUnit = PointerUnion<Operation *, Region *, Block *, Value>;
The tracing::Action is extended to take an ArrayRef<IRUnit> as context to
describe an Action. It is demonstrated in the "ActionLogging" observer.
Reviewed By: rriddle, Mogball
Differential Revision: https://reviews.llvm.org/D144814
Integrate the `tracing::ExecutionContext()` into mlir-opt with a new
--log-action-to=<file> option to demonstrate the feature.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D144813
Replace references to enumerate results with either result_pairs
(reference wrapper type) or structured bindings. I did not use
structured bindings everywhere as it wasn't clear to me it would
improve readability.
This is in preparation to the switch to zip semantics which won't
support non-const lvalue reference to elements:
https://reviews.llvm.org/D144503.
I chose to use values instead of const lvalue-refs because MLIR is
biased towards avoiding `const` local variables. This won't degrade
performance because currently `result_pair` is cheap to copy (size_t
+ iterator), and in the future, the enumerator iterator dereference
will return temporaries anyway.
Reviewed By: dblaikie
Differential Revision: https://reviews.llvm.org/D146006
User was confused by previous wording when pass was reported as failing
due to signal in unrelated thread being caught.
Differential Revision: https://reviews.llvm.org/D145213
An user might want to add extra spaces for better readability, e.g:
```
mypm = pm.PassManager.parse(f"""builtin.module(
mypass1,
func.func(mypass2,mypass3)
)""")
```
GitHub issue #59151
The parser was not taking into account the possibility of spaces after
`)`or `}`
Differential Revision: https://reviews.llvm.org/D142821
Currently `PassManager` defaults to being anchored on `builtin.module`.
Switching the default makes `PassManager` consistent with
`OpPassManager` and avoids the implicit dependency on `builtin.module`.
Specifying the anchor op type isn't strictly necessary when using
explicit nesting (existing pipelines will continue to work), but I've
updated most call sites to specify the anchor since it allows for better
error-checking during pipeline construction.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D137731
Clear active contexts and running passes whenever finalizing crash
report message. Ran into segfault where a failure in dynamic pipeline
resulted in querying a pass whose passmanager had already been destroyed
come time for creating summary of running passes. Conservatively clear
both running states as I don't think there is recovery intended from
pass pipeline failure.
Additionally restrict to one reproducer per report - else we end up
clobbering the same reproducer file over and over again. So instead of
ending with last reproducer we now end up with the first reproducer
while not creating and clobbering reproducers over and over again.
Differential Revision: https://reviews.llvm.org/D140488
When running in parallel, nesting more than once caused
statistics to be dropped.
Fix by also preparing "async" pass managers before merging,
as they may also have "async" pass managers within.
Add test checking reported statistics have expected values
with and without threading enabled.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D139459
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
The pipeline strings have been getting more verbose over time, adding an
alias for the option should help improve the ergonomics a bit.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D137347
Including the anchor op ensures that all pass manager settings are fully
specified, and makes the string consistent with the printed form.
Depends on D134622
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D134623
In D134622 the printed form of a pass manager is changed to include the
name of the op that the pass manager is anchored on. This updates the
`-pass-pipeline` argument format to include the anchor op as well, so
that the printed form of a pipeline can be directly passed to
`-pass-pipeline`. In most cases this requires updating
`-pass-pipeline='pipeline'` to
`-pass-pipeline='builtin.module(pipeline)'`.
This also fixes an outdated assert that prevented running a
`PassManager` anchored on `'any'`.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D134900
Currently `-pass-pipeline` can be specified multiple times and mixed
with the individual `-pass-name` options. Removing this feature will
allow for including the pipeline anchor as part of the option
argument (see D134900).
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D135745
It can be useful to external users as well for detecting if there were any changes in IR between passes.
Differential Revision: https://reviews.llvm.org/D137078
The error stream isn't being propagated properly, causing some output to
be sent to stderr even when a different stream is given.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D136414
