When the rhs expression has some constants and a device symbol, an
implicit data transfer needs to be generated for the device symbol and
the computation with the constant is done on the host.
This patch adds a flag to mark hlfir.declare of host variables that are
captured in some internal procedure.
It enables implementing a simple fir.call handling in
fir::AliasAnalysis::getModRef leveraging Fortran language specifications
and without a data flow analysis.
This will allow implementing an optimization for "array =
array_function()" where array storage is passed directly into the hidden
result argument to "array_function" when it can be proven that
arraY_function does not reference "array".
Captured host variables are very tricky because they may be accessed
indirectly in any calls if the internal procedure address was captured
via some global procedure pointer. Without flagging them, there is no
way around doing a complex inter procedural data flow analysis:
- checking that the call is not made to an internal procedure is not
enough because of the possibility of indirect calls made to internal
procedures inside the callee.
- checking that the current func.func has no internal procedure is not
enough because this would be invalid with inlining when an procedure
with internal procedures is inlined inside a procedure without internal
procedure.
We generate `cuf.free` and `func.return` twice if a return statement
exists at the end of program.
```f90
program test
integer, device :: a(10)
return
end
```
```
% flang -x cuda test.cuf -mmlir --mlir-print-ir-after-all
error: loc("/path/to/test.cuf":3:3): 'func.return' op must be the last operation in the parent block
// -----// IR Dump After Fortran::lower::VerifierPass Failed () //----- //
```
Dumped IR:
```mlir
"func.func"() <{function_type = () -> (), sym_name = "_QQmain"}> ({
...
"cuf.free"(%5#1) <{data_attr = #cuf.cuda<device>}> : (!fir.ref<!fir.array<10xi32>>) -> ()
"func.return"() : () -> ()
"cuf.free"(%5#1) <{data_attr = #cuf.cuda<device>}> : (!fir.ref<!fir.array<10xi32>>) -> ()
"func.return"() : () -> ()
}
...
```
The routine `genExitRoutine` in `Bridge.cpp` is guarded by
`blockIsUnterminated()` to make sure that `func.return` is generated
only at the end of a block. However, we redundantly run
`bridge.fctCtx().finalizeAndKeep()` before `genExitRoutine` in this
case, resulting in two pairs of `cuf.free` and `func.return`. This PR
fixes `Bridge.cpp` by using `blockIsUnterminated()` to guard
`finalizeAndKeep` as well.
If you have the following multi-range `do concurrent` loop:
```fortran
do concurrent(i=1:n, j=1:bar(n*m, n/m))
a(i) = n
end do
```
Currently, flang generates the following IR:
```mlir
fir.do_loop %arg1 = %42 to %44 step %c1 unordered {
...
%53:3 = hlfir.associate %49 {adapt.valuebyref} : (i32) -> (!fir.ref<i32>, !fir.ref<i32>, i1)
%54:3 = hlfir.associate %52 {adapt.valuebyref} : (i32) -> (!fir.ref<i32>, !fir.ref<i32>, i1)
%55 = fir.call @_QFPbar(%53#1, %54#1) fastmath<contract> : (!fir.ref<i32>, !fir.ref<i32>) -> i32
hlfir.end_associate %53#1, %53#2 : !fir.ref<i32>, i1
hlfir.end_associate %54#1, %54#2 : !fir.ref<i32>, i1
%56 = fir.convert %55 : (i32) -> index
...
fir.do_loop %arg2 = %46 to %56 step %c1_4 unordered {
...
}
}
```
However, if `bar` is impure, then we have a direct violation of the
standard:
```
C1143 A reference to an impure procedure shall not appear within a DO CONCURRENT construct.
```
Moreover, the standard describes the execution of `do concurrent`
construct in multiple stages:
```
11.1.7.4 Execution of a DO construct
...
11.1.7.4.2 DO CONCURRENT loop control
The concurrent-limit and concurrent-step expressions in the concurrent-control-list are evaluated. ...
11.1.7.4.3 The execution cycle
...
The block of a DO CONCURRENT construct is executed for every active combination of the index-name values.
Each execution of the block is an iteration. The executions may occur in any order.
```
From the above 2 points, it seems to me that execution is divided in
multiple consecutive stages: 11.1.7.4.2 is the stage where we evaluate
all control expressions including the step and then 11.1.7.4.3 is the
stage to execute the block of the concurrent loop itself using the
combination of possible iteration values.
nsw is now added to do-variable increment when -fno-wrapv is enabled as
GFortran seems to do.
That means the option introduced by #91579 isn't necessary any more.
Note that the feature of -flang-experimental-integer-overflow is enabled
by default.
The underlying issue was caused by a file included in two different
places which resulted in duplicate definition errors when linking
individual shared libraries. This was fixed in c3201ddaea
[#109874].
Add support for the -frecord-command-line option that will produce the
llvm.commandline metadata which will eventually be saved in the object
file. This behavior is also supported in clang. Some refactoring of the
code in flang to handle these command line options was carried out. The
corresponding -grecord-command-line option which saves the command line
in the debug information has not yet been enabled for flang.
The new LLVM stack save/restore intrinsic operations are more convenient
than function calls because they do not add function declarations to the
module and therefore do not block the parallelisation of passes.
Furthermore they could be much more easily marked with memory effects
than function calls if that ever proved useful.
This builds on top of #107879.
Resolves#108016
Mark the symbol with OmpShared, and then check that later in lowering to
avoid making a local loop index.
OpenMP 5.2 says: "Loop iteration variables of loops that are not associated
with any OpenMP directive maybe listed in data-sharing attribute clauses on
the surrounding teams, parallel or taskgenerating construct, and on enclosed
constructs, subject to other restrictions."
Tests updated to match the extra OmpShared attribute.
Add regression test for lowering to hlfir.
Closes#102961
---------
Co-authored-by: Tom Eccles <tom.eccles@arm.com>
This patch creates a simple RAII wrapper class for `SymMap` to make it
easier to use and prevent a missing matching `popScope()` for a
`pushScope()` call on simple use cases.
Some push-pop pairs are replaced with instances of the new class by this
patch.
Lowering was crashing when cuf kernels has an unstructured construct.
Blocks created by PFT need to be re-created inside of the operation like
it is done for OpenACC construct.
Code lowering always generates fir.if else blocks for source level if
statements, whether needed or not. Change this to only generate else
blocks that are needed.
ALLOCATE and DEALLOCATE statements can be inlined in device function.
This patch updates the condition that determined to inline these actions
in lowering.
This avoid runtime calls in device function code and can speed up the
execution.
Also move `isCudaDeviceContext` from `Bridge.cpp` so it can be used
elsewhere.
`cuf.data_transfer` will be converted to runtime calls to cuda runtime
api and these are not supported in device code. assignment in OpenACC
region will be handled by the OpenACC code gen so we avoid to generate
data transfer on them.
#106120 Simplify the data transfer when possible by using the reference
and a shape. This bypass the declare op. In order to keep the declare op
around, use the second results of the declare op which achieve the same.
When doing data transfer with dynamic sized array, we are currently
generating a data transfer between two descriptors. If the shape values
can be provided, we can keep the data transfer between two references.
This patch adds the shape operands to the operation.
This will be exploited in lowering in a follow up patch.
This brings the behavior of flang in line with clang which also adds
this metadata unconditionally.
Co-authored-by: Tarun Prabhu <tarun.prabhu@gmail.com>
Reland #96746 with the proper Support/CMakelist.txt change.
fir.type does not contain all Fortran level information about
components. For instance, component lower bounds and default initial
value are lost. For correctness purpose, this does not matter because
this information is "applied" in lowering (e.g., when addressing the
components, the lower bounds are reflected in the hlfir.designate).
However, this "loss" of information will prevent the generation of
correct debug info for the type (needs to know about lower bounds). The
initial value could help building some optimization pass to get rid of
initialization runtime calls.
This patch adds lower bound and initial value information into
fir.type_info via a new fir.dt_component operation. This operation is
generated only for component that needs it, which helps keeping the IR
small for "boring" types.
In general, adding Fortran level info in fir.type_info will allow
delaying the generation of "type descriptors" gobals that are very
verbose in FIR and make it hard to work with FIR dumps from applications
with many derived types.
In nested constructs where a given variable is privatized more than
once, using the default clause, the innermost host association symbol
will point to the previous host association symbol.
Such symbol lacks the allocatable attribute and can't be used to
generate the type of the symbol to be cloned. Use the ultimate
symbol instead.
Fixes#85594, #80398
fir.type does not contain all Fortran level information about
components. For instance, component lower bounds and default initial
value are lost. For correctness purpose, this does not matter because
this information is "applied" in lowering (e.g., when addressing the
components, the lower bounds are reflected in the hlfir.designate).
However, this "loss" of information will prevent the generation of
correct debug info for the type (needs to know about lower bounds). The
initial value could help building some optimization pass to get rid of
initialization runtime calls.
This patch adds lower bound and initial value information into
fir.type_info via a new fir.dt_component operation. This operation is
generated only for component that needs it, which helps keeping the IR
small for "boring" types.
In general, adding Fortran level info in fir.type_info will allow
delaying the generation of "type descriptors" gobals that are very
verbose in FIR and make it hard to work with FIR dumps from applications
with many derived types.
This PR adds -mtune as a valid flang flag and passes the information
through to LLVM IR as an attribute on all functions. No specific
architecture optimizations are added at this time.
The lowering of copyprivate clauses with allocatable or pointer
variables was incorrect. This happened because the values passed to
copyVar() are always wrapped in SymbolBox::Intrinsic, which
resulted in allocatable/pointer variables being handled as regular
ones.
This is fixed by providing to copyVar() the attributes of the
variables being copied, to make it possible to detect and handle
allocatable/pointer variables correctly.
Fixes#95801
PINNED is a CUDA data attribute meant for the host variables. Do not
consider it when computing the number of device variables in assignment
for the cuda data transfer.
This patch implements support for the VECTOR ALWAYS directive, which
forces
vectorization to occurr when possible regardless of a decision by the
cost
model. This is done by adding an attribute to the branch into the loop
in LLVM
to indicate that the loop should always be vectorized.
This patch only implements this directive on plan structured do loops
without labels. Support for unstructured loops and array
expressions is planned for future patches.
* Add reductionOperands and reductionAttrs to cuf's KernelOp.
* Parsing is already working and the tree has the info: here I make the
Bridge emit the updated KernelOp with reduction information added.
* Check |reductionAttrs| = |reductionOperands| in verifier
* Add a test
@clementval @vzakhari
---------
Co-authored-by: Iman Hosseini <imanh@nvidia.com>
Co-authored-by: Valentin Clement (バレンタイン クレメン) <clementval@gmail.com>
There is currently support for lowering directives that appear outside
of a module or procedure, or inside the body of a module or procedure.
Extend this to support directives at the CONTAINS level of a module or
procedure, such as directives 3, 5, 7 9, and 10 in:
!dir$ some directive 1
module m
!dir$ some directive 2
contains
!dir$ some directive 3
subroutine p
!dir$ some directive 4
contains
!dir$ some directive 5
subroutine s1
!dir$ some directive 6
end subroutine s1
!dir$ some directive 7
subroutine s2
!dir$ some directive 8
end subroutine s2
!dir$ some directive 9
end subroutine p
!dir$ some directive 10
end module m
!dir$ some directive 11
This is done by looking for CONTAINS statements at the module or
procedure level, while ignoring CONTAINS statements at the derived type
level.
Derived from #92480. This PR updates the lowering process of DO
CONCURRENT to support F'2023 REDUCE clause. The structure
`IncrementLoopInfo` is extended to have both reduction operations and
symbols in `reduceSymList`. The function `getConcurrentControl`
constructs `reduceSymList` for the innermost loop. Finally,
`genFIRIncrementLoopBegin` builds `fir.do_loop` with reduction operands.
We currently complain that the argument may not be a procedure, which is
confusing. Distinguish the NULL() case from other error cases (which are
indeed procedures). And clean up the utility predicates used for these
tests -- the current IsProcedure() is really just a test for a procedure
designator.
Lower select rank according to [assumed-rank lowering design
doc](https://github.com/llvm/llvm-project/blob/main/flang/docs/AssumedRank.md).
The construct is lowered using fir.box_rank and fir.select_case
operation and, for the non pointer/allocatable case, a
fir.is_assumed_size + conditional branch before the select_case to deal
with the assumed-size case.
The way the CFG logic is generated, apart from the extra conditional
branch for assumed-size, is similar to what is done for SELECT CASE
lowering, hence the sharing of the construct level visitor.
For the CFG parts. The main difference is that we need to keep track of
the selector to cook it and map it inside the cases (hence the new
members of the ConstructContext).
The only TODOs left are to deal with the RANK(*) case for polymorphic
entities and PDTs. I will do the polymorphic case in a distinct patch,
this patch has enough content.
Fortran::evaluate::IsSimplyContiguous change is needed to avoid generating
copy-in/copy-out runtime calls when passing the RANK(*) associating
entity to some implicit interface.
This PR contains 2 commits:
1. A commit to reapply changes introduced #91116 (was reverted earlier
due to test suite failures)
2. A commit containing a possible solution for the issue causing the
test suite failures. In particular, it introduces a simple symbol
visitor class to keep track of the current active OMP construct and
marking this active construct as the scope defining the symbol being
visisted.
