This patch speeds up the compilation time of the example in
https://github.com/llvm/llvm-project/issues/76478#issuecomment-2011023289
from 2 minutes with my builds to about 2 seconds.
MLIR timers showed more than 98% of the time was spend in BoxedProcedure
trying to figure out if a type needs to be converted.
This is because walking the fir.type members is very expansive for types
containing many components and/or components with many sub-components.
Increase the caching time of visited types from "the type being visited"
to "the whole pass". Use DenseMap since it is not ok anymore to assume
this container will only have a few elements.
This PR extracts `FIROpConversion` and `FIROpAndTypeConversion`
templated base patterns to a header file. All the functions from
FIROpConversion that do not require the template argument are moved to a
base class named `ConvertFIRToLLVMPattern`.
This move is done so the `FIROpConversion` pattern and all its utility
functions can be reused outside of the codegen pass.
For the most part the code is only moved to the new files and not
modified. The only update is that addition of the PatternBenefit
argument with a default value to the constructor so it can be forwarded
to the `ConversionPattern` ctor.
This split is done in a similar way for the `ConvertOpToLLVMPattern`
base pattern that is based on the `ConvertToLLVMPattern` base class in
`mlir/include/mlir/Conversion/LLVMCommon/Pattern.h`.
OpenMP reduction declare operations can contain FIR code which needs to
be lowered to LLVM. With array reductions, these regions can contain
more complicated operations which need PreCGRewriting. A similar extra
case was already needed for fir::GlobalOp.
OpenMP array reductions 3/6
Previous PR: https://github.com/llvm/llvm-project/pull/84953
Next PR: https://github.com/llvm/llvm-project/pull/84955
Advise to place the alloca at the start of the first block of whichever
region (init or combiner) we are currently inside.
It probably isn't safe to put an alloca inside of a combiner region
because this will be executed multiple times. But that would be a bug to
fix in Lower/OpenMP.cpp, not here.
OpenMP array reductions 1/6
Next PR: https://github.com/llvm/llvm-project/pull/84953
TBAA builder assumed that all loads/stores are inside of functions and
hit an assertion once it found loads and stores inside of an
omp::ReductionDeclareOp.
For now just don't add TBAA tags to those loads and stores. They would
end up in a different TBAA tree to the host function after
OpenMPIRBuilder inlines them anyway so there isn't an easy way of making
this work.
The FIR dialect has been initiated before many interfaces have been
introduced to MLIR. This patch expose the FIR to LLVM patterns in a
`populateFIRToLLVMConversionPatterns` function. The idea is to be able
to add the `ConvertToLLVMPatternInterface`. This is not directly
possible since the FIR dialect does not currently use the table
infrastructure for its definition.
Follow up patches will move the FIR dialect definition to table gen and
then implement the interface.
Four "issues" on GitHub report possible performance problems, likely
detected by static analysis. None of them would ever make a measureable
difference in compilation time, but I'm resolving them to clean up the
open issues list.
Fixes https://github.com/llvm/llvm-project/issues/79703, .../79705,
.../79706, & .../79707.
Descriptor addendum have a field to hold length parameters (currently
only one). This field is currently never used because flang does not
lowered derived types with length parameters.
However, leaving it uninitialized is causing bugs in code like gFTL
where the code is trying to sort POINTERs (see [1]). More precisely, it
is an issue when two pointers should compare equal (same base address),
because the uninitialized values in the addendum may differ depending on
the "stack history" and optimization level.
Always initialized the length parameters field in the addendum to zero.
[1]:
dc93a5fc2f/include/v1/templates/set_impl.inc (L312)
The type being transferred to an integer array may look like:
```
TYPE :: localwrapper
TYPE(T), POINTER :: item
END TYPE localwrapper
```
Which in flang case ends-up transferring a descriptor to an integer
array, the code in gFTL later compare the integer arrays. This logic is
used when building set data structures in gFTL.
This patch seeks to add an initial lowering for pointers and allocatable variables
captured by implicit and explicit map in Flang OpenMP for Target operations that
take map clauses e.g. Target, Target Update. Target Exit/Enter etc.
Currently this is done by treating the type that lowers to a descriptor
(allocatable/pointer/assumed shape) as a map of a record type (e.g. a structure) as that's
effectively what descriptor types lower to in LLVM-IR and what they're represented as
in the Fortran runtime (written in C/C++). The descriptor effectively lowers to a structure
containing scalar and array elements that represent various aspects of the underlying
data being mapped (lower bound, upper bound, extent being the main ones of interest
in most cases) and a pointer to the allocated data. In this current iteration of the mapping
we map the structure in it's entirety and then attach the underlying data pointer and map
the data to the device, this allows most of the required data to be resident on the device
for use. Currently we do not support the addendum (another block of pointer data), but
it shouldn't be too difficult to extend this to support it.
The MapInfoOp generation for descriptor types is primarily handled in an optimization
pass, where it expands BoxType (descriptor types) map captures into two maps, one for
the structure (scalar elements) and the other for the pointer data (base address) and
links them in a Parent <-> Child relationship. The later lowering processes will then treat
them as a conjoined structure with a pointer member map.
This patch forwards the target CPU and features information from the
Flang frontend to MLIR func.func operation attributes, which are later
used to populate the target_cpu and target_features llvm.func
attributes.
This is achieved in two stages:
1. Introduce the `fir.target_cpu` and `fir.target_features` module
attributes with information from the target machine immediately after
the initial creation of the MLIR module in the lowering bridge.
2. Update the target rewrite flang pass to get this information from the
module and pass it along to all func.func MLIR operations, respectively
as attributes named `target_cpu` and `target_features`. These attributes
will be automatically picked up during Func to LLVM dialect lowering and
used to initialize the corresponding llvm.func named attributes.
The target rewrite and FIR to LLVM lowering passes are updated with the
ability to override these module attributes, and the `CodeGenSpecifics`
optimizer class is augmented to make this information available to
target-specific MLIR transformations.
This completes a full flow by which target CPU and features make it all
the way from compiler options to LLVM IR function attributes.
The existing type size computation in LoopVersioning does not work
for REAL*10, because the compute element size is 10 bytes,
which violates the power-of-two assertion.
We'd better use the DataLayout for computing the storage size
of each element of an array of the given type.
The code in BoxedProcedureRewrite was erasing the mappings <old type,
new type> once "new type" was fully translated. This was done in an
invalid way because the map was an llvm::SmallMapVector that do not have
stable iterators, and new items were added to the map between the
creation of the iterator and its erasure.
It is anyway not needed and expensive to erase the types (see
llvm::MapVector note), the cache can be used for the whole pass, which
is very beneficial in the context of an apps using "big derived types"
(dozens of components/parents).
The pass was mistakenly identifying a fir.box_addr on a
fir.box/fir.class of a derived type with procedure pointer components as
being a fir.box_addr on a procedure.
Simply check if the input type is a fir.box_proc or function type (if
input already rewritten) and insert convert only in this case.
This caused "invalid fir.convert" internal error.
A user defining and using free/malloc via BIND(C) would previously cause
flang to crash when generating LLVM IR with error "redefinition of
symbol named 'free'". This was caused by flang codegen not expecting to
find a mlir::func::FuncOp definition of these function and emitting a
new mlir::LLVM::FuncOp that later conflicted when translating the
mlir::func::FuncOp.
This is a slightly more slimmed down and up-to-date version of the older
PR from here: https://reviews.llvm.org/D144203, written by @jsjodin,
which has already under gone some review.
This PR places allocas in the alloca address space specified by the
provided data layout (default is 0 for all address spaces, unless
explicitly specified by the layout), and then will cast these alloca's
to the program address space if this address space is different from the
allocation address space. For most architectures data layouts, this will
be a no-op, as they have a flat address space. But in the case of AMDGPU
it will result in allocas being placed in the correct address space (5,
private), and then casted into the correct program address space (0,
generic). This results in correct (partially, a follow up PR will be
forthcoming soon) generation of allocations inside of device code.
This PR is in addition to the work by @skatrak in this PR:
https://github.com/llvm/llvm-project/pull/69599 and adds seperate and
neccesary functionality of casting alloca's from their address space to
the program address space, both are independent PRs, although there is
some minor overlap e.g. this PR incorporates some of the useful helper
functions from 69599, so whichever lands first will need a minor rebase.
Co-author: jsjodin
This commit renames 4 pattern rewriter API functions:
* `updateRootInPlace` -> `modifyOpInPlace`
* `startRootUpdate` -> `startOpModification`
* `finalizeRootUpdate` -> `finalizeOpModification`
* `cancelRootUpdate` -> `cancelOpModification`
The term "root" is a misnomer. The root is the op that a rewrite pattern
matches against
(https://mlir.llvm.org/docs/PatternRewriter/#root-operation-name-optional).
A rewriter must be notified of all in-place op modifications, not just
in-place modifications of the root
(https://mlir.llvm.org/docs/PatternRewriter/#pattern-rewriter). The old
function names were confusing and have contributed to various broken
rewrite patterns.
Note: The new function names use the term "modify" instead of "update"
for consistency with the `RewriterBase::Listener` terminology
(`notifyOperationModified`).
The boxed-procedure pass is lowering the fir.boxproc type. Although this
is not common, this types may end-up as block arguments (or be part of
derived type that are block arguments).
Update the pass to update block argument types too.
Derived type passed with VALUE in BIND(C) context must be passed like C
struct and LLVM is not implementing the ABI for this (it is up to the
frontends like clang).
Previous patch #75802 implemented the simple cases where the derived
type have one field, this patch implements the general case. Note that
the generated LLVM IR is compliant from a X86-64 C ABI point of view and
compatible with clang generated assembly, but that it is not guaranteed
to match the LLVM IR signatures generated by clang for the C equivalent
functions because several LLVM IR signatures may lead to the same X86-64
signature.
Lower procedure pointer components, except in the context of structure
constructor (left TODO).
Procedure pointer components lowering share most of the lowering logic
of procedure poionters with the following particularities:
- They are components, so an hlfir.designate must be generated to
retrieve the procedure pointer address from its derived type base.
- They may have a PASS argument. While there is no dispatching as with
type bound procedure, special care must be taken to retrieve the derived
type component base in this case since semantics placed it in the
argument list and not in the evaluate::ProcedureDesignator.
These components also bring a new level of recursive MLIR types since a
fir.type may now contain a component with an MLIR function type where
one of the argument is the fir.type itself. This required moving the
"derived type in construction" stackto the converter so that the object
and function type lowering utilities share the same state (currently the
function type utilty would end-up creating a new stack when lowering its
arguments, leading to infinite loops). The BoxedProcedurePass also
needed an update to deal with this recursive aspect.
Implement the C struct passing ABI on X86-64 for the trivial case where
the structs have one element. This is required to cover some cases of
BIND(C) derived type pass with the VALUE attribute.
In the context of C/Fortran interoperability (BIND(C)), it is possible
to give the VALUE attribute to a BIND(C) derived type dummy, which
according to Fortran 2018 18.3.6 - 2. (4) implies that it must be passed
like the equivalent C structure value. The way C structure value are
passed is ABI dependent.
LLVM does not implement the C struct ABI passing for LLVM aggregate type
arguments. It is up to the front-end, like clang is doing, to split the
struct into registers or pass the struct on the stack (llvm "byval") as
required by the target ABI.
So the logic for C struct passing sits in clang. Using it from flang
requires setting up a lot of clang context and to bridge FIR/MLIR
representation to clang AST representation for function signatures (in
both directions). It is a non trivial task.
See
https://stackoverflow.com/questions/39438033/passing-structs-by-value-in-llvm-ir/75002581#75002581.
Since BIND(C) struct are rather limited as opposed to generic C struct
(e.g. no bit fields). It is easier to provide a limited implementation
of it for the case that matter to Fortran.
This patch:
- Updates the generic target rewrite pass to keep track of both the new
argument type and attributes. The motivation for this is to be able to
tell if a previously marshalled argument is passed in memory (it is a C
pointer), or if it is being passed on the stack (has the byval llvm
attributes).
- Adds an entry point in the target specific codegen to marshal struct
arguments, and use it in the generic target rewrite pass.
- Implements limited support for the X86-64 case. So far, the support
allows telling if a struct must be passed in register or on the stack,
and to deal with the stack case. The register case is left TODO in this
patch.
The X86-64 ABI implemented is the System V ABI for AMD64 version 1.0
`nsw` is a flag for LLVM arithmetic operations meaning "no signed wrap".
If this keyword is present, the result of the operation is a poison
value if overflow occurs. Adding this keyword permits LLVM to re-order
integer arithmetic more aggressively.
In
https://discourse.llvm.org/t/rfc-changes-to-fircg-xarray-coor-codegen-to-allow-better-hoisting/75257/16
@vzakhari observed that adding nsw is useful to enable hoisting of
address calculations after some loops (or is at least a step in that
direction).
Classic flang also adds nsw to address calculations.
`llvm.fcmp` does support fast math attributes therefore so should
`arith.cmpf`.
The heavy churn in flang tests are because flang sets
`fastmath<contract>` by default on all operations that support the fast
math interface. Downstream users of MLIR should not be so effected.
This was requested in https://github.com/llvm/llvm-project/issues/74263
COMPLEX(10) passing by value and returning follows C complex
passing/returning ABI.
Cover the COMPLEX(10) case (X87 / __Complex long double on X86-64).
Implements System V ABI for AMD64 version 1.0.
The LLVM signatures match the one generated by clang for the __Complex
long double case.
Note that a FIXME is added for the COMPLEX(8) case that is incorrect in
a corner case. This will be fixed when dealing with passing derived type
by value in BIND(C) context.
…ortran_binding.h
... and into the ISO_Fortran_binding_wrapper.h header, through which the
compiler and runtime access its contents. This change ensures that user
code that #includes ISO_Fortran_binding.h within 'extern "C" {' doesn't
encounter mysterious namespace errors.
This update makes the user visible messages relating to features that
are not yet implemented be more consistent. I also cleaned up some of
the code.
For NYI messages that refer to intrinsics, I made sure the the message
begins with "not yet implemented: intrinsic:" to make them easier to
recognize.
I created some utility functions for NYI reporting that I put into
.../include/Optimizer/Support/Utils.h. These mainly convert MLIR types
to their Fortran equivalents.
I converted the NYI code to use the newly created utility functions.
This operation allows computing the address of descriptor fields. It is
needed to help attaching descriptors in OpenMP/OpenACC target region.
The pointers inside the descriptor structure must be mapped too, but the
fir.box is abstract, so these fields cannot be computed with
fir.coordinate_of.
To preserve the abstraction of the descriptor layout in FIR, introduce
an operation specifically to !fir.ref<fir.box<>> address fields based on
field names (base_addr or derived_type).
Propagate fast math flags through complex number lowering (when lowering
fir.*c directly to llvm floating point operations).
The lowering path through the MLIR complex dialect is unchanged.
This leads to a small improvement in spec2017 fotonik3d_r.
!llvm.ptr<T> typed pointers are depreciated in MLIR LLVM dialects. Flang
codegen still generated them and relied on mlir.llvm codegen to LLVM to
turn them into opaque pointers.
This patch update FIR codegen to directly emit and work with LLVM opaque
pointers.
Addresses https://github.com/llvm/llvm-project/issues/69303
- All places generating GEPs need to add an extra type argument with the
base type (the T that was previously in the llvm.ptr<T> of the base).
- llvm.alloca must also be provided the object type. In the process, I
doscovered that we were shamelessly copying all the attribute from
fir.alloca to the llvm.alloca, which makes no sense for the operand
segments. The updated code that cannot take an attribute dictionnary in
the llvm.alloca builder with opaque pointers only propagate the "pinned"
and "bindc_name" attributes to help debugging the generated IR.
- Updating all the places that rely on getting the llvm object type from
lowered llvm.ptr<T> arguments to get it from a type conversion of the
original fir types.
- Updating all the places that were generating llvm.ptr<T> types to
generate the opaque llvm.ptr type.
- Updating all the codegen tests checking generated MLIR llvm dialect.
Many tests are testing directly LLVM IR, and this change is a no-op for
those (which is expected).
The ultimate intention is to have this pass enabled by default whenever
we are optimizing for speed. But for now, just add the arguments so this
can be more easily tested.
PR: https://github.com/llvm/llvm-project/pull/68595
This is important to ensure that tags end up in the same trees that were
created in the FIR TBAA pass. If they are in different trees then
everything in one tree will be assumed to MayAlias with everything in the
other tree. This leads to poor performance.
@vzakhari requested that the old (not-per-function) trees are
maintained so I left the old test intact.
PR: https://github.com/llvm/llvm-project/pull/68437
This interface allows (HL)FIR passes to add TBAA information to fir.load
and fir.store. If present, these TBAA tags take precedence over those
added during CodeGen.
We can't reuse mlir::LLVMIR::AliasAnalysisOpInterface because that uses
the mlir::LLVMIR namespace so it tries to define methods for fir
operations in the wrong namespace. But I did re-use the tbaa tag type to
minimise boilerplate code.
The new builders are to preserve the old interface without the tbaa tag.
The goal is to progressively propagate all the derived type info that is
currently in the runtime type info globals into a FIR operation that can
be easily queried and used by FIR/HLFIR passes.
When this will be complete, the last step will be to stop generating the
runtime info global in lowering, but to do that later in or just before
codegen to keep the FIR files readable (on the added type-info.f90
tests, the lowered runtime info globals takes a whooping 2.6 millions
characters on 1600 lines of the FIR textual output. The fir.type_info that
contains all the info required to generate those globals for such
"trivial" types takes 1721 characters on 9 lines).
So far this patch simply starts by replacing the fir.dispatch_table
operation by the fir.type_info operation and to add the noinit/
nofinal/nodestroy flags to it. These flags will soon be used in HLFIR to
better rewrite hlfir.assign with derived types.
Since HLFIR bufferization can introduce shallow copies of derived
type values we have to be careful not to treat these load/store
operations as data-only-accesses. If a derived type has descriptor
members, we attach any-access tag now.
This is not standard but is vastly expected by existing code.
This was implemented by https://reviews.llvm.org/D149877 for simple
scalars, but MLIR lacked a generic way to deal with aggregate types
(arrays and derived type).
Support was recently added in
https://github.com/llvm/llvm-project/pull/65508. Leverage it to zero
initialize all types.
This revision replaces the LLVM dialect NullOp by the recently
introduced ZeroOp. The ZeroOp is more generic in the sense that it
represents zero values of any LLVM type rather than null pointers only.
This is a follow to https://github.com/llvm/llvm-project/pull/65508
This fixes a bug where functions generated by the MLIR Math dialect, for
example ipowi, would fail to link with link.exe on Windows due to having
linkonce linkage but no associated comdat. Adding the comdat on ELF also
allows linkers to perform better garbage collection in the binary.
Simply adding comdats to all functions with this linkage type should
also cover future cases where linkonce or linkonce_odr functions might
be necessary.
While creating a temporary alloca for a box in OpenMp region, the
insertion point should be the OpenMP region block instead of the
function entry block.
