The current implementation of `convertPowfOp` requires a calculation of
`a * a` but, max\<fp16\> ~= 65,504, and if `a` is about 16, it will
overflow so get INF in fp8 or fp16 easily.
Remove support when `a < 0`. Overhead of handling negative value of `a`
is large and easy to overflow;
- related issue in iree:
https://github.com/iree-org/iree/issues/15936
The convertFloorOp pattern incurs precision loss when floating-point
numbers exceed the representable range of int64. This pattern should be
removed.
Fixes https://github.com/llvm/llvm-project/issues/119836
The greedy rewriter is used in many different flows and it has a lot of
convenience (work list management, debugging actions, tracing, etc). But
it combines two kinds of greedy behavior 1) how ops are matched, 2)
folding wherever it can.
These are independent forms of greedy and leads to inefficiency. E.g.,
cases where one need to create different phases in lowering and is
required to applying patterns in specific order split across different
passes. Using the driver one ends up needlessly retrying folding/having
multiple rounds of folding attempts, where one final run would have
sufficed.
Of course folks can locally avoid this behavior by just building their
own, but this is also a common requested feature that folks keep on
working around locally in suboptimal ways.
For downstream users, there should be no behavioral change. Updating
from the deprecated should just be a find and replace (e.g., `find ./
-type f -exec sed -i
's|applyPatternsAndFoldGreedily|applyPatternsGreedily|g' {} \;` variety)
as the API arguments hasn't changed between the two.
Lowering `math.powf` to `llvm.intr.powf` will result in `pow(x, 0) =
1`, even for `x=0`. When using the Math dialect expansion patterns,
`pow(0, 0)` will result in `-nan`, however, This change adds two
additional instructions to the lowering to ensure the `pow(x, 0)` case
lowers to to `1` regardless of the value of `x`.
Resolves https://github.com/llvm/llvm-project/issues/118945.
This patch disambiguates 0-rank vectors and scalars in
PolynomialApproximation. This fixes a bug in PolynomialApproximation
where 0-rank vectors would be treated as scalars and arguments would not
be broadcasted properly.
This commit marks the type converter in `populate...` functions as
`const`. This is useful for debugging.
Patterns already take a `const` type converter. However, some
`populate...` functions do not only add new patterns, but also add
additional type conversion rules. That makes it difficult to find the
place where a type conversion was added in the code base. With this
change, all `populate...` functions that only populate pattern now have
a `const` type converter. Programmers can then conclude from the
function signature that these functions do not register any new type
conversion rules.
Also some minor cleanups around the 1:N dialect conversion
infrastructure, which did not always pass the type converter as a
`const` object internally.
Instead of hardcoding all fp smaller than 32 bits are unsupported we
provide a way to pass supported floating point types as well as the
target type. fp64 and fp32 are implicitly supported.
CC: @krzysz00 @manupak
The polynomial approximation for asin is only good between [-9/16,
9/16]. Values beyond that range must be remapped to achieve good numeric
results. This is done by the equation below:
`arcsin(x) = PI/2 - arcsin(sqrt(1.0 - x*x))`
Add an `fastMathAttr` on `arith::extf` and `arith::truncf`. If these two
ops are inserted by some promotion passes (like legalize-to-f32 /
emulate-unsupported-floats), they will be labeled as
`FastMathFlags::contract`, denoting that they can be then `eliminated by
canonicalizer`.
The `elimination` can help improve performance, while may introduce some
numerical differences.
Add an `fastMathAttr` on `arith::extf` and `arith::truncf`. If these two
ops are inserted by some promotion passes (like legalize-to-f32 /
emulate-unsupported-floats), they will be labeled as
`FastMathFlags::contract`, denoting that they can be then `eliminated by
canonicalizer`.
The `elimination` can help improve performance, while may introduce some
numerical differences.
The previous implementation decomposes tanh(x) into
`(exp(2x) - 1)/(exp(2x)+1), x < 0`
`(1 - exp(-2x))/(1 + exp(-2x)), x >= 0`
This is fine as it avoids overflow with the exponential, but the whole
decomposition is computed for both cases unconditionally, then the
result is chosen based off the sign of the input. This results in doing
two expensive exp computations.
The proposed change avoids doing the whole computation twice by
exploiting the reflection symmetry `tanh(-x) = -tanh(x)`. We can
"normalize" the input to be positive by setting `y = sign(x) * x`, where
the sign of `x` is computed as `sign(x) = (float)(x > 0) * (-2) + 1`.
Then compute `z = tanh(y) `with the decomposition above for `x >=0` and
"denormalize" the result `z * sign(x)` to retain the sign. The reason it
is done this way is that it is very amenable to vectorization.
This method trades the duplicate decomposition computations (which takes
5 instructions including an extra expensive exp and div) for 4 cheap
instructions to compute the signs value
`arith.cmpf `(which is a pre-existing instruction in the previous impl)
`arith.sitofp`
`arith.mulf`
`arith.addf`
and 1 more instruction to get the right sign in the result
5. `arith.mulf`.
Moreover, numerically, this implementation will yield the exact same
results as the previous implementation.
As part of the relanding, a casting issue from the original commit has
been fixed, i.e. casting bool to float with `uitofp`. Additionally a
correctness test with `mlir-cpu-runner` has been added.
This simply updates the rewrites to propagate the scalable flags (which
as they do not alter the vector shape, is pretty simple).
The added tests are simply scalable versions of the existing vector
tests.
The previous implementation decomposes `tanh(x)` into
`(exp(2x) - 1)/(exp(2x)+1), x < 0`
`(1 - exp(-2x))/(1 + exp(-2x)), x >= 0`
This is fine as it avoids overflow with the exponential, but the whole
decomposition is computed for both cases unconditionally, then the
result is chosen based off the sign of the input. This results in doing
two expensive `exp` computations.
The proposed change avoids doing the whole computation twice by
exploiting the reflection symmetry `tanh(-x) = -tanh(x)`. We can
"normalize" the input to be positive by setting `y = sign(x) * x`, where
the sign of `x` is computed as `sign(x) = (float)(x > 0) * (-2) + 1`.
Then compute `z = tanh(y)` with the decomposition above for `x >=0` and
"denormalize" the result `z * sign(x)` to retain the sign. The reason it
is done this way is that it is very amenable to vectorization.
This method trades the duplicate decomposition computations (which takes
5 instructions including an extra expensive `exp` and `div`) for 4 cheap
instructions to compute the signs value
1. `arith.cmpf` (which is a pre-existing instruction in the previous
impl)
2. `arith.sitofp`
3. `arith.mulf`
4. `arith.addf`
and 1 more instruction to get the right sign in the result
5. `arith.mulf`. Moreover, numerically, this implementation will yield
the exact same results as the previous implementation.
These patterns can already be used via
populateMathPolynomialApproximationPatterns, but that includes a number
of other patterns that may not be needed.
There are already similar functions for expansion.
For now only adding tanh and erf since I have a concrete use case for
these two.
Common backends (LLVM, SPIR-V) only supports 1D vectors, LLVM conversion
handles ND vectors (N >= 2) as `array<array<... vector>>` and SPIR-V
conversion doesn't handle them at all at the moment. Sometimes it's
preferable to treat multidim vectors as linearized 1D. Add pass to do
this. Only constants and simple elementwise ops are supported for now.
@krzysz00 I've extracted yours result type conversion code from
LegalizeToF32 and moved it to common place.
Also, add ConversionPattern class operating on traits.
Since most of the operations in the `math` dialect don't have
low-precision implementations, add the -math-legalize-to-f32 pass that
goes through and brackets low-precision math funcitons (like `math.sin
%0 : f16`) with `arith.extf` and `arith.truncf`. This preserves the
original semantics of the math operation but allows lowering to proceed.
Versions of this lowering are already implicitly present in some passes,
like ConvertGPUToROCDL. However, because those are implicit rewrites,
they hide the floating-point extension and truncation, preventing anyone
from writing passes that operate on those implitic extf/truncf pairs.
Exposing this legalization explicitly is needed to allow lowening 8-bit
floats on AMD GPUs, as the implementation of extf and truncf on that
platform requires the complex logic found in ArithToAMDGPU, which runs
before the GPU to ROCDL lowering.
This PR adds promised interface declarations for
`ConvertToLLVMPatternInterface` in all the dialects that support the
`ConvertToLLVM` dialect extension.
Promised interfaces allow a dialect to declare that it will have an
implementation of a particular interface, crashing the program if one
isn't provided when the interface is used.
Return poison from foldBinary/unary if argument(s) is poison. Add ub dialect as dependency to affected dialects (arith, math, spirv, shape).
Add poison materialization to dialects. Add tests for some ops from each dialect.
Not all affected ops are covered as it will involve a huge copypaste.
Differential Revision: https://reviews.llvm.org/D159013
Powf expansion currently returns NaN when the base is negative.
This is because taking natural log of a negative number gives
NaN. This patch will square the base and half the exponent, thereby
getting around the negative base problem.
Reviewed By: rsuderman
Differential Revision: https://reviews.llvm.org/D158797
Used the cephes numerical approximation for `math.atan`. This is a
significant accuracy improvement over the previous taylor series
approximation.
Reviewed By: jpienaar
Differential Revision: https://reviews.llvm.org/D153656
The existing lowering has lower precision for certain use cases, e.g.
tanh. Improved version should demonstrate an overall higher level of precision.
Reviewed By: cota, jpienaar
Differential Revision: https://reviews.llvm.org/D153592
The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionality in addition to defining methods with the same name.
This change begins the migration of uses of the method to the
corresponding function call as has been decided as more consistent.
Note that there still exist classes that only define methods directly,
such as AffineExpr, and this does not include work currently to support
a functional cast/isa call.
Caveats include:
- This clang-tidy script probably has more problems.
- This only touches C++ code, so nothing that is being generated.
Context:
- https://mlir.llvm.org/deprecation/ at "Use the free function variants
for dyn_cast/cast/isa/…"
- Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443
Implementation:
This first patch was created with the following steps. The intention is
to only do automated changes at first, so I waste less time if it's
reverted, and so the first mass change is more clear as an example to
other teams that will need to follow similar steps.
Steps are described per line, as comments are removed by git:
0. Retrieve the change from the following to build clang-tidy with an
additional check:
https://github.com/llvm/llvm-project/compare/main...tpopp:llvm-project:tidy-cast-check
1. Build clang-tidy
2. Run clang-tidy over your entire codebase while disabling all checks
and enabling the one relevant one. Run on all header files also.
3. Delete .inc files that were also modified, so the next build rebuilds
them to a pure state.
4. Some changes have been deleted for the following reasons:
- Some files had a variable also named cast
- Some files had not included a header file that defines the cast
functions
- Some files are definitions of the classes that have the casting
methods, so the code still refers to the method instead of the
function without adding a prefix or removing the method declaration
at the same time.
```
ninja -C $BUILD_DIR clang-tidy
run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\
-header-filter=mlir/ mlir/* -fix
rm -rf $BUILD_DIR/tools/mlir/**/*.inc
git restore mlir/lib/IR mlir/lib/Dialect/DLTI/DLTI.cpp\
mlir/lib/Dialect/Complex/IR/ComplexDialect.cpp\
mlir/lib/**/IR/\
mlir/lib/Dialect/SparseTensor/Transforms/SparseVectorization.cpp\
mlir/lib/Dialect/Vector/Transforms/LowerVectorMultiReduction.cpp\
mlir/test/lib/Dialect/Test/TestTypes.cpp\
mlir/test/lib/Dialect/Transform/TestTransformDialectExtension.cpp\
mlir/test/lib/Dialect/Test/TestAttributes.cpp\
mlir/unittests/TableGen/EnumsGenTest.cpp\
mlir/test/python/lib/PythonTestCAPI.cpp\
mlir/include/mlir/IR/
```
Differential Revision: https://reviews.llvm.org/D150123
The dependencies were set up improperly likely due to past code
locations. MathTransforms shouldn't depend on VectorUtils which add a
whole bunch of additional dependencies; it instead depends on the SCF
dialect.
Differential Revision: https://reviews.llvm.org/D149797
This reverts commit 87cef78fa1.
The issue in the original revert is that a lit test expecting a `-nan`
as an output was failing on M2. Since the IEEE 754-2008 standard does
not require the sign to be printed when displaying a `nan`, this
commit changes the `CHECK` for `-nan` to one that checks the result
value bitcasted to an `i32` to ensure that input is being left
unchanged. This check should now be independent of platform being used
to run test.
Reviewed By: jpienaar, mehdi_amini
Differential Revision: https://reviews.llvm.org/D148941
Currently conversions to interfaces may happen implicitly (e.g.
`Attribute -> TypedAttr`), failing a runtime assert if the interface
isn't actually implemented. This change marks the `Interface(ValueT)`
constructor as explicit so that a cast is required.
Where it was straightforward to I adjusted code to not require casts,
otherwise I just made them explicit.
Depends on D148491, D148492
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D148493
This adds `arith::ConstantOp::materialize`, which builds a constant from
an attribute and type only if it would result in a valid op. This is
useful for dialect `materializeConstant` hooks, and allows for removing
the previous `Attribute, Type` builder which was only used during
materialization.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D148491
This commit adds a pattern that expands `math.roundeven` into
`math.round` + some ops from `arith`. This is needed to be able to run
`math.roundeven` in a vectorized manner.
Reviewed By: jpienaar
Differential Revision: https://reviews.llvm.org/D148285
