The first test case added in the LIT test demonstrates the problem.
Even though we did not consider the inner loop as a candidate for
the transformation due to the array_coor with a slice, we decided to
version the outer loop for the same function argument.
During the cloning of the outer loop we dropped the slicing completely
producing invalid code.
I restructured the code so that we record all arg uses that cannot be
transformed (regardless of the reason), and then fixup the usage
information across the loop nests. I also noticed that we may generate
redundant contiguity checks for the inner loops, so I fixed it
since it was easy with the new way of keeping the usage data.
This is the last piece required for the loop versioning patch to work on
code lowered via HLFIR. With this patch, HLFIR performance on spec2017
roms is now similar to the FIR lowering.
Adding support for fir.array_coor means that many more loops will be
versioned, even in the FIR lowering. So far as I have seen, these do not
seem to have an impact on performance for the benchmarks I tried, but I
expect it would speed up some programs, if the loop being versioned
happened to be the hot code.
The main difference between fir.array_coor and fir.coordinate_of is
that fir.coordinate_of uses zero-based indices, whereas fir.array_coor
uses the indices as specified in the Fortran program (starting from 1 by
default, but also supporting non default lower bounds). I opted to
transform fir.array_coor operations into fir.coordinate_of operations
because this allows both to share the same offset calculation logic.
The tricky bit of this patch is getting the correct lower bounds for the
array operand to subtract from the fir.array_coor indices to get a
zero-based indices. So far as I can tell, the FIR lowering will always
provide lower bounds (shift) information in the shape operand to the
fir.array_coor when non-default lower bounds are used. If none is given,
I originally tried falling back to reading lower bounds from the box,
but this led to misscompilation in SPEC2017 cam4. Therefore the pass
instead assumes that if it can't already find an SSA value for the shift
information, the default lower bound (1) should be used.
A suspect the incorrect lower bounds in the box for the FIR lowering was
already a known issue (see https://reviews.llvm.org/D158119).
Differential Revision: https://reviews.llvm.org/D158597
This patch fixes multiple tests failing with segfault due to accessing
absent argument box before the loop versioning check.
The absent arguments might be treated as contiguous for the purpose
of loop versioning, but this is not done in this patch.
Reviewed By: PeteSteinfeld
Differential Revision: https://reviews.llvm.org/D158800
Since https://reviews.llvm.org/D158119, many boxes lowered via HLFIR are
reboxed with better lower bounds information after they are declared.
For the loop versioning pass to support FIR lowered via HLFIR, it needs
to dereference fir.rebox operations to figure out that the variable was
a function argument.
I decided to modify the existing dereferencing of fir.declare so that
the declared/reboxed value is used in the versioned loop instead of the
function argument. This makes it easier for the improved lower bounds
information to be accessed. In doing this, I changed ArgInfo to store
ArgInfo::arg by value instead of by pointer because mlir::Value has
value-type semantics.
Differential Revision: https://reviews.llvm.org/D158408
On x86, a simplified F128 maxval ends up calling fmaxl that does not
work properly for F128 arguments. It is probably an LLVM issue, but
we also should not use arith.maxf if NaN or -0.0 operands are possible.
The change is to use cmpf and select. Unfortunately, these arith ops
do not support FastMathFlags currently, so I will have to fix this
sooner or later (depending on how this affects performance).
Reviewed By: kiranchandramohan
Differential Revision: https://reviews.llvm.org/D158200
When FIR comes from HLFIR, there will be a fir.declare operation between
the source and the usage of each source variable (and some temporary
allocations). This pass needs to be able to follow these so that it can
still transform loops when HLFIR is used, otherwise it mistakenly
assumes these values are not function arguments.
More work is needed after this patch to fully support HLFIR, because the
generated code tends to use fir.array_coor instead of fir.coordinate_of.
Differential Revision: https://reviews.llvm.org/D157964
This patch improves the implementation of a recent function filtering
workaround to address problems uncovered by D154247.
In particular, the problem was related to the removal of functions called from
within target regions. Since target regions have to remain until LLVM IR is
generated, removing these functions from MLIR results in undefined references
any time there are calls to them in a target region. This patch modifies the
MLIR function filtering pass to make these functions "external" rather than
removing them. This way, the processing and lowering of MLIR functions that
will eventually be discarded is still prevented, but no calls to undefined
functions remain either.
Additionally, the approach of just filtering host-only functions during device
compilation, and not filtering device-only functions during host compilation,
is maintained. This is because code generation for device-only functions is
required for host fallback to work.
Depends on D156988
Differential Revision: https://reviews.llvm.org/D155827
This patch adds support for selecting which functions are lowered to LLVM IR
from MLIR depending on declare target information and whether host or device
code is being generated.
The approach proposed by this patch is to perform the filtering in two stages:
- An MLIR transformation pass, which is added to the Flang translation flow
after the `OMPEarlyOutliningPass`. The functions that are kept are those
that match the OpenMP processor (host or device) the compiler invocation
is targeting, according to the presence of the `-fopenmp-is-target-device`
compiler option and declare target information. All functions contaning an
`omp.target` are also kept, regardless of the declare target information of
the function, due to the need for keeping target regions visible for both
host and device compilation.
- A filtering step during translation to LLVM IR, which is peformed for those
functions that were kept because of the presence of a target region inside.
If the targeted OpenMP processor does not match the declare target
information of the function, then it is removed from the LLVM IR after its
contents have been processed and translated. Since they should only contain
an omp.target operation which, in turn, should have been outlined into
another LLVM IR function, the wrapper can be deleted at that point.
Depends on D150328 and D150329.
Differential Revision: https://reviews.llvm.org/D147641
The old fir.allocmem operation returned a !fir.heap<.> type. The new
fir.alloca operation returns a !fir.ref<.> type. This patch inserts a
fir.convert so that the old type is preserved. This prevents verifier
failures when types returned from fir.if statements don't match the
expected type.
Differential Revision: https://reviews.llvm.org/D151921
Despite me being convinced that the use of divide didn't produce any
divide instructions, it does in fact add more instructions than using
a plain shift operation.
This patch simply changes the divide to a shift right, with an
assert to check that the "divisor" is a power of two.
Reviewed By: kiranchandramohan, tblah
Differential Revision: https://reviews.llvm.org/D151880
In upstream mlir, the dialect conversion infrastructure is used for
lowering from one dialect to another: the passes are of the form
XToYPass. Whereas, transformations within the same dialect tend to use
applyPatternsAndFoldGreedily.
In this case, the full complexity of applyPatternsAndFoldGreedily isn't
needed so we can get away with the simpler applyOpPatternsAndFold.
This change was suggested by @jeanPerier
The old differential revision for this patch was
https://reviews.llvm.org/D150853
Re-applying here fixing the issue which led to the patch being reverted. The
issue was from erasing uses of the allocation operation while still iterating
over those uses (leading to a use-after-free). I have added a regression
test which catches this bug for -fsanitize=address builds, but it is
hard to reliably cause a crash from the use-after-free in normal builds.
Differential Revision: https://reviews.llvm.org/D151728
This patch makes more than 2D arrays work, with a fix for the way that
loop index is calculated. Removing the restriction of number of
dimensions.
This also changes the way that the actual index is calculated, such that
the stride is used rather than the extent of the previous dimension. Some
tests failed without fixing this - this was likely a latent bug in the
2D version too, but found in a test using 3D arrays, so wouldn't
have been found with 2D only. This introduces a division on the index
calculation - however it should be a nice and constant value allowing
a shift to be used to actually divide - or otherwise removed by using
other methods to calculate the result. In analysing code generated with
optimisation at -O3, there are no divides produced.
Some minor refactoring to avoid repeatedly asking for the "rank" of the
array being worked on.
This improves some of the SPEC-2017 ROMS code, in the same way as the
limited 2D array improvements - less overhead spent calculating array
indices in the inner-most loop and better use of vector-instructions.
Reviewed By: kiranchandramohan
Differential Revision: https://reviews.llvm.org/D151140
Remove old clause operands from acc.parallel operation since
the new dataOperands is now in place.
private, firstprivate and reductions will receive some redesign but are
not part of the new dataOperands.
Reviewed By: razvanlupusoru
Differential Revision: https://reviews.llvm.org/D150207
Since the new data operand operations have been added in D148389 and
adopted on acc.data in D149673, the old clause operands are no longer
needed.
The LegalizeDataOpForLLVMTranslation will become obsolete when all
operations will be cleaned. For the time being only the appropriate
part are being removed.
processOperands will also receive some updates once all the operands
will be coming from an acc data operand operation.
Reviewed By: razvanlupusoru
Differential Revision: https://reviews.llvm.org/D150155
Since the new data operand operations have been added in D148389 and
adopted on acc.exit_data in D149601, the old clause operands are no longer
needed.
The LegalizeDataOpForLLVMTranslation will become obsolete when all
operations will be cleaned. For the time being only the appropriate
part are being removed.
processOperands will also receive some updates once all the operands
will be coming from an acc data operand operation.
Reviewed By: jeanPerier
Differential Revision: https://reviews.llvm.org/D150145
Since the new data operand operations have been added in D148389 and
adopted on acc.enter_data in D148721, the old clause operands are no longer
needed.
The LegalizeDataOpForLLVMTranslation will become obsolete when all
operations will be cleaned. For the time being only the appropriate
part are being removed.
processOperands will also receive some updates once all the operands
will be coming from an acc data operand operation.
Reviewed By: jeanPerier
Differential Revision: https://reviews.llvm.org/D150132
Since the new data operand operations have been added in D148389 and
adopted on acc.update in D149909, the old clause operands are no longer
needed. This is a first patch to start cleaning the OpenACC operations
with data clause operands.
The `LegalizeDataOpForLLVMTranslation` will become obsolete when all
operations will be cleaned. For the time being only the appropriate
part are being removed.
`processOperands` will also receive some updates once all the operands
will be coming from an acc data operand operation.
Reviewed By: razvanlupusoru, jeanPerier
Differential Revision: https://reviews.llvm.org/D150053
Another test based on review comments added late in the review.
This one confirms that the multiplication and addition of the outer
index to the inner index and thus form the 2D index.
Reviewed By: tblah
Differential Revision: https://reviews.llvm.org/D149265
These two tests were created from little snippets added late
in the review of the loop versioning work. The code was fixed
to cope with the situation and correctly compile these samples.
This adds tests to avoid regressions in this area.
Reviewed By: tblah
Differential Revision: https://reviews.llvm.org/D148649
Introduce conditional code to identify stride of "one element", and simplify the array accesses for that case.
This allows better loop performance in various benchmarks.
Reviewed By: tblah, kiranchandramohan
Differential Revision: https://reviews.llvm.org/D141306
Remove the custoom parser and printer for the acc.parallel
operation and use the assembly format directly.
Reviewed By: PeteSteinfeld, razvanlupusoru
Differential Revision: https://reviews.llvm.org/D148183
Similar to D148039 but for the FIR to LLVM IR
conversion pass.
The inner part of the acc.loop has been removed since the rest of the
pipeline is not ready and would raise an error here. This was passing
until now because the acc.loop was discarded completely.
Reviewed By: PeteSteinfeld
Differential Revision: https://reviews.llvm.org/D148057
fir.if currently isn't treated as a 'proper' conditional, so passes are unable to determine which regions are executed at times.
This patch gives fir.if this interface, which shouldn't do too much on its own but should allow future changes to take advantage
for various purposes
Reviewed By: vzakhari
Differential Revision: https://reviews.llvm.org/D145165
Previously the mask would be loaded as the appropriate integer type and cast to I1 to pass to
fir.if, however this truncates the integer and so would cast 6 to 0. By loading values as logicals
and casting to I1 this problem is avoided.
Reviewed By: Leporacanthicus
Differential Revision: https://reviews.llvm.org/D144974
Previously COUNT would cast the mask input to logical<4> before passing it
to the runtime function, this has been changed to allow different types of logical.
Reviewed By: tblah
Differential Revision: https://reviews.llvm.org/D144867
This patch adds minloc to the simplify intrinsics pass, supporting calls with KIND or MASK arguments while calls which have BACK, DIM or have a CHARACTER input array are rejected. This patch is targeting exchange2, and in benchmarks provides a ~11% improvement in performance.
Also included are some minor style changes / cleanup in simplifyIntrinsics.cpp.
Reviewed By: vzakhari
Differential Revision: https://reviews.llvm.org/D144103
Some functions (e.g. the main function) end with a call to the STOP
statement instead of a func.return. This is lowered as a call to the
stop runtime function followed by a fir.unreachable. fir.unreachable is
a terminator and so this can cause functions to have no func.return.
The stack arrays pass looks to see which heap allocations have always
been freed by the time a function returns. Without any returns, the pass
does not detect any freed allocations. This patch changes this behaviour
so that fir.unreachable is checked as well as func.return.
This allows 15 heap allocations for array temporaries in spec2017
exchange2's main function to be moved to the stack.
Differential Revision: https://reviews.llvm.org/D143918
When rank > 1, the inital value would be lost on inner loops, leading to the wrong
value to be returned, e.g. This would return T. This patch fixes this to use the correct
inital value for all cases.
```
Integer :: m(0,10)
Any(m .eq 0)
```
Reviewed By: vdonaldson
Differential Revision: https://reviews.llvm.org/D143899
This patch provides a simplified version of the Any intrinsic as well as the All intrinsic
that can be used for inlining or simpiler use cases. These changes are targeting exchange2, and
provide a ~9% performance increase.
Reviewed By: Leporacanthicus, vzakhari
Differential Revision: https://reviews.llvm.org/D142977
The implementation of -fstack-arrays was added in
https://reviews.llvm.org/D140415
The new macro BoolOptionWithoutMarshalling in Options.td avoids
generating code to store the flags in clang data structures. For
example, writing something like
defm stack_arrays : BoolOption<"f", "stack-arrays",
CodeGenOpts<"StackArrays">, [...]
Would generate code referring to `clang::CodeGenOpts::StackArrays`, which
does not exist.
Differential Revision: https://reviews.llvm.org/D140972
This pass implements the `-fstack-arrays` flag. See the RFC in
`flang/docs/fstack-arrays.md` for more information.
Differential revision: https://reviews.llvm.org/D140415
Simple fix to check for rank in the same way as other intrinsics to allow
runtime count to take over when dealing with unknown dimension arrays.
Fixes#60356
Reviewed By: Leporacanthicus
Differential Revision: https://reviews.llvm.org/D142877
This patch adds a simplfiied version of count for the simplify intrinsics pass, allowing the function to be inlined.
This was done specifically to help improve performance for exchange2, and provides a ~12% performance increase.
Reviewed By: vzakhari, Leporacanthicus
Differential Revision: https://reviews.llvm.org/D142209
This ensures that functions in included files have the correct path
in their file metadata.
Note: This patch also sets all locations to have the full path names.
Reviewed By: vzakhari, PeteSteinfeld
Differential Revision: https://reviews.llvm.org/D142263
Previously, the DISubroutineType attribute used an optional result
parameter and an optional argument types array to model the subroutine
signature. LLVM IR debug metadata, on the other hand, has one types
list whose first entry maps to the result type. That entry may be
null to model a void result type. The type list may also be entirely
empty not specifying any type information. The latter is problematic
since the current DISubroutineType attribute cannot express it.
The revision changes DISubroutineTypeAttr to closely follow the
LLVM metadata design. In particular, it uses a single types parameter
array to model the subroutine signature and introduces an explicit
DIVoidResultTypeAttr to model the null entries.
Reviewed By: Dinistro
Differential Revision: https://reviews.llvm.org/D141261
This reverts commit 81f57b6
and relands commit a960547
Fixes flang build and drop_begin on an empty array ref.
Recent changes to MLIR meant that Flang does not generate any debug line
table information.
This patch adds a pass that provides some foundation work with which
basic line table debug info can be generated. A walk is performed on
all the `func` ops in the module and they are decorated with a fusedLoc
op that contains the debug metadata for the subroutine along with
location information.
Alternatives include populating this info during lowering or during FIR
to LLVM Dialect conversion.
Note: Patches in future will add
-> more realistic debug info for types and other fields.
-> driver flags to control generation of debug.
Fixes#58634.
Reviewed By: awarzynski, vzakhari
Differential Revision: https://reviews.llvm.org/D137956
In general, the meaning of fastmath flags on a call during inlining
is that the call's operation flags must be ignored. For user functions
that means that the fastmath flags used for the function definition
override any call site's fastmath flags. For intrinsic functions
we can use the call site's fastmath flags, but we have to make sure
that the call sites with different flags produce/use different
simplified versions of the same intrinsic function.
Differential Revision: https://reviews.llvm.org/D138048
Create simplified functions for each rank with "x<rank>" suffix
that implement multidimensional reductions. To enable this I had to fix
an issue with taking incorrect box shape in cases of sliced embox/rebox.
Differential Revision: https://reviews.llvm.org/D133820
The SUM function does appear to be safe to use, so remove the
experimental flag for the SUM operation.
Reviewed By: vzakhari, awarzynski
Differential Revision: https://reviews.llvm.org/D132567
Add simplifcation pass for MAXVAL intrinsic function
This refactors some of the code to allow variation on the
initialization value and operation performed within the loop,
reusing the majority of code for both SUM and MAXVAL.
Adding tests for the test-cases that produce different output
than the SUM function.
Reviewed By: vzakhari
Differential Revision: https://reviews.llvm.org/D132234
Under some conditions, the defining op may be NULL, so
accept that rahter than try to use it and crash!
Adds test to prevent regression
Fixes github issue #57201
Reviewed By: vzakhari
Differential Revision: https://reviews.llvm.org/D132238
