This patch adds support for selecting which functions are lowered to LLVM IR
from MLIR depending on declare target information and whether host or device
code is being generated.
The approach proposed by this patch is to perform the filtering in two stages:
- An MLIR transformation pass, which is added to the Flang translation flow
after the `OMPEarlyOutliningPass`. The functions that are kept are those
that match the OpenMP processor (host or device) the compiler invocation
is targeting, according to the presence of the `-fopenmp-is-target-device`
compiler option and declare target information. All functions contaning an
`omp.target` are also kept, regardless of the declare target information of
the function, due to the need for keeping target regions visible for both
host and device compilation.
- A filtering step during translation to LLVM IR, which is peformed for those
functions that were kept because of the presence of a target region inside.
If the targeted OpenMP processor does not match the declare target
information of the function, then it is removed from the LLVM IR after its
contents have been processed and translated. Since they should only contain
an omp.target operation which, in turn, should have been outlined into
another LLVM IR function, the wrapper can be deleted at that point.
Depends on D150328 and D150329.
Differential Revision: https://reviews.llvm.org/D147641
Using MLIR attributes instead of metadata has many advantages:
* No indirection: Attributes can simply refer to each other seemlessly without having to use the indirection of `SymbolRefAttr`. This also gives us correctness by construction in a lot of places as well
* Multithreading safe: The Attribute infrastructure gives us thread-safety for free. Creating operations and inserting them into a block is not thread-safe. This is a major use case for e.g. the inliner in MLIR which runs in parallel
* Easier to create: There is no need for a builder or a metadata region
This patch therefore does exactly that. It leverages the new distinct attributes to create distinct access groups in a deterministic and threadsafe manner.
Differential Revision: https://reviews.llvm.org/D155285
Using MLIR attributes instead of metadata has many advantages:
* No indirection: Attributes can simply refer to each other seemlessly without having to use the indirection of `SymbolRefAttr`. This also gives us correctness by construction in a lot of places as well
* Multithreading save: The Attribute infrastructure gives us thread-safety for free. Creating operations and inserting them into a block is not thread-safe. This is a major use case for e.g. the inliner in MLIR which runs in parallel
* Easier to create: There is no need for a builder or a metadata region
This patch therefore does exactly that. It leverages the new distinct attributes to create distinct alias domains and scopes in a deterministic and threadsafe manner.
Differential Revision: https://reviews.llvm.org/D155159
This patch implements an early outlining transform of omp.target operations in
flang. The pass is needed because optimizations may cross target op region
boundaries, but with the outlining the resulting functions only contain a
single omp.target op plus a func.return, so there should not be any opportunity
to optimize across region boundaries.
The patch also adds an interface to be able to store and retrieve the parent
function name of the original target operation. This is needed to be able to
create correct kernel function names when lowering to LLVM-IR.
Reviewed By: kiranchandramohan, domada
Differential Revision: https://reviews.llvm.org/D154879
This adds operations for binary multiplicative arithmetic operators to
EmitC. The input and output arguments for the remainder operator are
restricted to index (emitted as size_t), integers and the EmitC opaque
types (as the operator can be overloaded for a custom type). The
multiplication and division operator further support floating point
numbers.
Reviewed By: jpienaar
Differential Revision: https://reviews.llvm.org/D154846
Define `llvm.intr.var.annotation`, `llvm.intr.ptr.annotation` and
`llvm.intr.annotation` in the llvm dialect as `llvm.var.annotation`,
`llvm.ptr.annotation` and `llvm.annotation` counterparts.
Signed-off-by: Victor Perez <victor.perez@codeplay.com>
Differential Revision: https://reviews.llvm.org/D154842
`nvgpu.device_async_copy` is lowered into `cp.async` PTX instruction. However, NVPTX backend does not support its all mode especially when zero padding is needed. Therefore, current MLIR implementation genereates inline assembly for that.
This work simplifies PTX generation for `nvgpu.device_async_copy`, and implements it by `NVVMToLLVM` Pass.
Depends on D154060
Reviewed By: nicolasvasilache, manishucsd
Differential Revision: https://reviews.llvm.org/D154345
This is in preparation for adding the KHR version of the cooperative
matrix extension, `SPV_KHR_cooperative_matrix`, that comes with
equivalent ops and type. These are not cross-extension compatible,
so it's better to add prefixes/suffixes to the Nvidia one,
`SPV_NV_cooperative_matrix`, before adding the KHR counterparts.
In near future, I plan for these two extensions to co-exist in
the SPIR-V dialect, but we may want to remove the NV one at some point.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D154799
This patch renames the `OpenMPIRBuilderConfig` flags to reduce confusion over
their meaning. `IsTargetCodegen` becomes `IsGPU`, whereas `IsEmbedded` becomes
`IsTargetDevice`. The `-fopenmp-is-device` compiler option is also renamed to
`-fopenmp-is-target-device` and the `omp.is_device` MLIR attribute is renamed
to `omp.is_target_device`. Getters and setters of all these renamed properties
are also updated accordingly. Many unit tests have been updated to use the new
names, but an alias for the `-fopenmp-is-device` option is created so that
external programs do not stop working after the name change.
`IsGPU` is set when the target triple is AMDGCN or NVIDIA PTX, and it is only
valid if `IsTargetDevice` is specified as well. `IsTargetDevice` is set by the
`-fopenmp-is-target-device` compiler frontend option, which is only added to
the OpenMP device invocation for offloading-enabled programs.
Differential Revision: https://reviews.llvm.org/D154591
This commit ensures that DICompositeTypes are exported as distinct
metadata nodes for structures, classes, unions, and enumerations.
Not emitting these as distinct metadata can potentially cause linking
issues involving debug sections.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D154712
This patch changes the emitTargetDataCalls function in clang to make use of the OpenMPIRBuilder::createTargetData function for Target Data directive code gen.
Reapplying commit 0d8d718171 after fixing libomptarget test failure related to debug info.
Depends on D146557
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D150860
This commit changes the LLVM IR import to use UnkownLoc for missing
debug locations. This change ensures that we do not accidentially
introduce faulty locations that can influence debugging post export.
This behavior change is not applied to locations of global metadata
operations, as their location will not be exported.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D154416
This commit adds an optional section attribute to the `LLVMFuncOp` and
adds import and export functionality for it.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D154219
Debug intrinsics are allowed to violate SSA dominance and might thus
cause the LLVM import to produce invalid LLVM dialect. This commit
ensures that the debug intrinsics are emitted right after the definition
of their SSA operands.
As the position of debug intrinsics has no meaning, changing it has no
semantic implication.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D153984
This revision adds support for the llvm.dbg.label.intrinsic
and the corresponding DILabel metadata.
Reviewed By: Dinistro
Differential Revision: https://reviews.llvm.org/D153975
This revision adds comdat support to functions. Additionally,
it ensures only comdats that have uses are imported/exported and
only non-empty global comdat operations are created.
Reviewed By: Dinistro
Differential Revision: https://reviews.llvm.org/D153739
Whereas LLVM currently doesn't have any types for 8-bit floats, and
whereas existing 8-bit float APIs (for instance, the AMDGCN
intrinsics) take such floats as (packed) bytes, translate the MLIR
8-bit float types to i8 during LLVM lowering.
In order to not special-case arith.constant for bitcasting constants
to their integer form, amend the MLIR to LLVM translator to turn 8-bit
float constants into i8 constants with the same value (by use of
APFloat's bitcast method).
This change can be reverted once LLVM has 8-bit float types.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D153160
The revision adds a flag to the LLVM IR import
that avoids emitting expensive warnings about
unsupported debug intrinsics and unhandled
metadata.
Reviewed By: Dinistro
Differential Revision: https://reviews.llvm.org/D153625
This adds operations for binary additive operators to EmitC. The input
arguments to these ops can be EmitC pointers and thus the operations can
be used for pointer arithmetic.
Reviewed By: jpienaar
Differential Revision: https://reviews.llvm.org/D149963
This commit ensures that an empty list of result attributes is not
imported as an empty `ArrayAttr`. Instead, the attribute is just not
added to the `LLVMFuncOp`.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D153553
This revision ensures SwitchOps with case and condition
bitwidths other than 32-bit are imported properly. It adds an
APInt based builder to the SwitchOp and implements
a verifier that checks that the condition and the case
value types match.
Reviewed By: Dinistro
Differential Revision: https://reviews.llvm.org/D153438
The LLVM comdat operation specifies how to deduplicate globals with the
same key in two different object files. This is necessary on Windows
where e.g. two object files with linkonce globals will not link unless
a comdat for those globals is specified. It is also supported in the ELF
format.
Differential Revision: https://reviews.llvm.org/D150796
Key changes:
- Refactor the createTargetData function to make use of the emitOffloadingArrays and emitOffloadingArraysArgument functions to generate code.
- Added a new emitIfClause helper function to allow handling if clauses in a similar fashion to Clang.
- Updated the MLIR side of code to account for changes to createTargetData.
Depends on D149872
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D146557
This patch adds two LLVM intrinsics to the ArmSME dialect:
* llvm.aarch64.sme.za.enable
* llvm.aarch64.sme.za.disable
for enabling the ZA storage array [1], as well as patterns for inserting
them during legalization to LLVM at the start and end of functions if
the function has the 'arm_za' attribute (D152695).
In the future ZA should probably be automatically enabled/disabled when
lowering from vector to SME, but this should be sufficient for now at
least until we have patterns lowering to SME instructions that use ZA.
N.B. The backend function attribute 'aarch64_pstate_za_new' can be used
manage ZA state (as was originally tried in D152694), but it emits calls
to the following SME support routines [2] for the lazy-save mechanism
[3]:
* __arm_tpidr2_restore
* __arm_tpidr2_save
These will soon be added to compiler-rt but there's currently no public
implementation, and using this attribute would introduce an MLIR
dependency on compiler-rt. Furthermore, this mechanism is for routines
with ZA enabled calling other routines with it also enabled. We can
choose not to enable ZA in the compiler when this is case.
Depends on D152695
[1] https://developer.arm.com/documentation/ddi0616/aa
[2] https://github.com/ARM-software/abi-aa/blob/main/aapcs64/aapcs64.rst#sme-support-routines
[3] https://github.com/ARM-software/abi-aa/blob/main/aapcs64/aapcs64.rst#the-za-lazy-saving-scheme
Reviewed By: awarzynski, dcaballe
Differential Revision: https://reviews.llvm.org/D153050
This patch creates the ArmSME dialect, and provides the intrinsic op
definition necessary for lowering to LLVM IR.
This will cover most instructions interacting with the ZA tile register,
not covering SME2 instructions.
Source: https://developer.arm.com/documentation/ddi0616/latest
Reviewed By: awarzynski, c-rhodes
Differential Revision: https://reviews.llvm.org/D152878
This commit changes intrinsics that have immarg parameter attributes to
model these parameters as attributes, instead of operands. Using
operands only works if the operation is an `llvm.mlir.constant`,
otherwise the exported LLVMIR is invalid.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D151692
Integer constants with bit width less than a word (e.g., i8, i16)
should be bit extended based on its type to be SPIR-V spec-compliant.
Previously, the decision was based on the most significant bit of the
value which ignores the signless semantics and causes problems when
interfacing with SPIR-V tools.
Dealing with numeric literals: the SPIR-V spec says, "If a numeric
type’s bit width is less than 32-bits, the value appears in the
low-order bits of the word, and the high-order bits must be 0 for
a floating-point type or integer type with Signedness of 0, or sign
extended for an integer type with a Signedness of 1 (similarly for the
remaining bits of widths larger than 32 bits but not a multiple of 32
bits)."
Therefore, signless integers (e.g., i8, i16) and unsigned integers
should be 0-extended, and signed integers (e.g., si8, si16) should be
sign-extended.
Patch By: mshahneo
Reviewed By: kuhar
Differential Revision: https://reviews.llvm.org/D151767
This fixes a buildbot failure where the dependency on the omp dialect
in the LLVM dialect caused error. Instead of accessing the interface
defined in the omp dialect we directly access the attributes
instead. To make this work the IsDeviceAttr is removed and replaced
with a BoolAttr instead.
Reviewed By: kiranchandramohan
Differential Revision: https://reviews.llvm.org/D151745
The revision adds the alias analysis and access group
interfaces to the call operation.
Reviewed By: Dinistro
Differential Revision: https://reviews.llvm.org/D151693
Add support for the `llvm::TargetExtType` to the MLIR LLVM dialect.
Target extension types were introduced to represent target-specific types, which are opaque to the compiler and optimizations.
The patch also enforces some of the constraints defined for the target extension type in the LLVM language reference manual.
Signed-off-by: Lukas Sommer <lukas.sommer@codeplay.com>
Reviewed By: ftynse, gysit, Dinistro
Differential Revision: https://reviews.llvm.org/D151446
"LinkageAttributes" decoration allow a SPIR-V module to import
external functions and global variables, or export functions or
global variables for other SPIR-V modules to link against and use.
Import/export capability is extremely important when using outside
libraries (e.g., intrinsic libraries).
Added decorations:
- LinkageAttributes
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D148749
MLIR version attribute should be lowered to LLVM IR module metadata.
The lowering is done by OpenMPIRBuilder.
Differential Revision: https://reviews.llvm.org/D150574
Reviewed By: kiranchandramohan
This patch adds support in the `OpenMPIRBuilder` for generating working
device code for OpenMP target regions. It generates and handles the
result of a call to `__kmpc_target_init()` at the beginning of the
function resulting from outlining each target region, and it also
generates the matching `__kmpc_target_deinit()` call before returning.
It relies on the implementation of target region outlining for host
codegen to handle the production of the new function and the lowering of
its body based on the contents of the associated target region.
Depends on D147172
Differential Revision: https://reviews.llvm.org/D147940
