When we offload to the target, the pointers to data used by the kernel are passed in arrays created by `OMPIRBuilder`. These arrays of pointers are allocated on the stack on the host. This is fine for the most part because absent the `nowait` clause, the default behavior is that target tasks are included tasks. That is, the host is blocked until the offloaded target kernel is done. In turn, this means that the host's stack frame is intact and accessing the array of pointers when offloading is safe. However, when `nowait` is used on the `!$ omp target` instance, then the target task is a deferred task meaning, the generating task on the host does not have to wait for the target task to finish. In such cases, it is very likely that the stack frame of the function invoking the target call is wound up thereby leading to memory access errors as shown below. ``` AMDGPU error: Error in hsa_amd_memory_pool_allocate: HSA_STATUS_ERROR_INVALID_ALLOCATION: The requested allocation is not valid. AMDGPU error: Error in hsa_amd_memory_pool_allocate: HSA_STATUS_ERROR_INVALID_ALLOCATION: The requested allocation is not valid. "PluginInterface" error: Failure to allocate device memory: Failed to allocate from memory manager fort.cod.out: /llvm/llvm-project/offload/plugins-nextgen/common/src/PluginInterface.cpp:1434: Error llvm::omp::target::plugin::PinnedAllocationMapTy::lockMappedHostBuffer(void *, size_t): Assertion `HstPtr && "Invalid pointer"' failed. Aborted (core dumped) ``` This PR implements support in `OMPIRBuilder` to store these arrays of pointers in the task structure that is passed to the target task thereby ensuring it is available to the target task when the target task is eventually scheduled. --------- Co-authored-by: Sergio Afonso <safonsof@amd.com>
Multi-Level Intermediate Representation
See https://mlir.llvm.org/ for more information.