caio/clang-p2996
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[Offload] Improve liboffload documentation (#142403)

Browse Source 
- Update the main README to reflect the current project status
- Rework the main API generation documentation. General fixes/tidying,
but also spell out explicitly how to make API changes at the top of the
document since this is what most people will care about.

---------

Co-authored-by: Martin Grant <martingrant@outlook.com>
This commit is contained in:
Callum Fare
2025-07-02 13:52:27 +01:00
committed by GitHub
parent 4e213159af
commit acb52a8a98
3 changed files with 102 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files

105
offload/liboffload/API/README.md
View File

@@ -1,8 +1,5 @@
# Offload API definitions # Offload API definitions
**Note**: This is a work-in-progress. It is loosely based on equivalent
tooling in Unified Runtime.
The Tablegen files in this directory are used to define the Offload API. They The Tablegen files in this directory are used to define the Offload API. They
are used with the `offload-tblgen` tool to generate API headers, print headers, are used with the `offload-tblgen` tool to generate API headers, print headers,
and other implementation details. and other implementation details.
@@ -10,17 +7,48 @@ and other implementation details.
The root file is `OffloadAPI.td` - additional `.td` files can be included in The root file is `OffloadAPI.td` - additional `.td` files can be included in
this file to add them to the API. this file to add them to the API.
## Modifying the API
API modifications, including additions, can be made by modifying the existing
`.td` files. It is also possible to add a new tablegen file to the API by adding
it to the includes in `OffloadAPI.td`. When Offload is rebuilt the new
definition will be included in the generated files.
Most API changes and additions do not require any additional work beyond this,
other than new functions which are described below.
### Adding a new function to the API
When a new function is added (e.g. `offloadDeviceFoo`), the actual entry
point is automatically generated, which contains validation and tracing code.
It expects an implementation function (`offloadDeviceFoo_impl`) to be defined,
which it will call into. The definition of this implementation function should
be added to `liboffload/src/OffloadImpl.cpp`
In short, the steps to add a new function are:
* Add the new function definition to the `.td` files.
* Build the `LLVMOffload` target. The relevant files will be regenerated, but
the library will fail to link because it is missing the implementation
function.
* Add the new implementation function to `liboffload/src/OffloadImpl.cpp`. You
can copy the new function declaration from the generated
`OffloadImplFuncDecls.inc` file.
* Rebuild `LLVMOffload`
## API Objects ## API Objects
The API consists of a number of objects, which always have a *name* field and The API consists of a number of objects, which always have a *name* field and
*description* field, and are one of the following types: *description* field, and are one of the following types:
### Function ### Function
Represents an API entry point function. Has a list of returns and parameters. Represents an API entry point function. Has a list of returns and parameters.
Also has fields for details (representing a bullet-point list of Also has fields for details (representing a bullet-point list of information
information about the function that would otherwise be too detailed for the about the function that would otherwise be too detailed for the description),
description), and analogues (equivalent functions in other APIs). and analogues (equivalent functions in other APIs).
#### Parameter #### Parameter
Represents a parameter to a function, has *type*, *name*, and *desc* fields. Represents a parameter to a function, has *type*, *name*, and *desc* fields.
Also has a *flags* field containing flags representing whether the parameter is Also has a *flags* field containing flags representing whether the parameter is
in, out, or optional. in, out, or optional.
@@ -30,17 +58,24 @@ A *handle* type is a pointer to an opaque struct, used to abstract over
plugin-specific implementation details. plugin-specific implementation details.
There are two special variants of a *parameter*: There are two special variants of a *parameter*:
* **RangedParameter** - Represents a parameter that has a range described by other parameters. Generally these are pointers to an arbitrary number of objects. The range is used for generating validation and printing code. E.g, a range might be between `(0, NumDevices)` * **RangedParameter** - Represents a parameter that has a range described by
* **TypeTaggedParameter** - Represents a parameter (usually of `void*` type) that has the type and size of its pointee data described by other function parameters. The type is usually described by a type-tagged enum. This allows functions (e.g. `olGetDeviceInfo`) to return data of an arbitrary type. other parameters. Generally these are pointers to an arbitrary number of
objects. The range is used for generating validation and printing code. E.g,
a range might be between `(0, NumDevices)`
* **TypeTaggedParameter** - Represents a parameter (usually of `void*` type)
that has the type and size of its pointee data described by other function
parameters. The type is usually described by a type-tagged enum. This allows
functions (e.g. `olGetDeviceInfo`) to return data of an arbitrary type.
#### Return #### Return
A return represents a possible return code from the function, and optionally a A return represents a possible return code from the function, and optionally a
list of conditions in which this value may be returned. The conditions list is list of conditions in which this value may be returned. The conditions list is
not expected to be exhaustive. A condition is considered free-form text, but not expected to be exhaustive. A condition is considered free-form text, but if
if it is wrapped in \`backticks\` then it is treated as literal code it is wrapped in \`backticks\` then it is treated as literal code representing
representing an error condition (e.g. `someParam < 1`). These conditions are an error condition (e.g. `someParam < 1`). These conditions are used to
used to automatically create validation checks by the `offload-tblgen` automatically create validation checks by the `offload-tblgen` validation
validation generator. generator.
Returns are automatically generated for functions with pointer or handle Returns are automatically generated for functions with pointer or handle
parameters, so API authors do not need to exhaustively add null checks for parameters, so API authors do not need to exhaustively add null checks for
@@ -49,6 +84,7 @@ values automatically.
### Struct ### Struct
Represents a struct. Contains a list of members, which each have a *type*, Represents a struct. Contains a list of members, which each have a *type*,
*name*, and *desc*. *name*, and *desc*.
@@ -59,24 +95,28 @@ actual C++ inheritance, but instead explicitly has those members copied in,
which preserves ABI compatibility with C. which preserves ABI compatibility with C.
### Enum ### Enum
Represents a C-style enum. Contains a list of `etor` values, which have a name Represents a C-style enum. Contains a list of `etor` values, which have a name
and description. and description.
A `TaggedEtor` record type also exists which additionally takes a type. This type A `TaggedEtor` record type also exists which additionally takes a type. This
is used when the enum is used as a parameter to a function with a type-tagged type is used when the enum is used as a parameter to a function with a
function parameter (e.g. `olGetDeviceInfo`). type-tagged function parameter (e.g. `olGetDeviceInfo`).
All enums automatically get a `<enum_name>_FORCE_UINT32 = 0x7fffffff` value, All enums automatically get a `<enum_name>_FORCE_UINT32 = 0x7fffffff` value,
which forces the underlying type to be uint32. which forces the underlying type to be uint32.
### Handle ### Handle
Represents a pointer to an opaque struct, as described in the Parameter section. Represents a pointer to an opaque struct, as described in the Parameter section.
It does not take any extra fields. It does not take any extra fields.
### Typedef ### Typedef
Represents a typedef, contains only a *value* field. Represents a typedef, contains only a *value* field.
### Macro ### Macro
Represents a C preprocessor `#define`. Contains a *value* field. Optionally Represents a C preprocessor `#define`. Contains a *value* field. Optionally
takes a *condition* field, which allows the macro to be conditionally defined, takes a *condition* field, which allows the macro to be conditionally defined,
and an *alt_value* field, which represents the value if the condition is false. and an *alt_value* field, which represents the value if the condition is false.
@@ -90,15 +130,20 @@ files, rather than requiring a mix of C source and tablegen.
## Generation ## Generation
### API header ### API header
``` ```
./offload-tblgen -I <path-to-llvm>/offload/API <path-to-llvm>/offload/API/OffloadAPI.td --gen-api ./offload-tblgen -I <path-to-llvm>/offload/API <path-to-llvm>/offload/API/OffloadAPI.td --gen-api
``` ```
The comments in the generated header are in Doxygen format, although The comments in the generated header are in Doxygen format, although
generating documentation from them hasn't been implemented yet. generating documentation from them hasn't been implemented yet.
The entirety of this header is generated by Tablegen, rather than having a predefined header file that includes one or more `.inc` files. This is because this header is expected to be part of the installation and distributed to end-users, so should be self-contained. The entirety of this header is generated by Tablegen, rather than having a
predefined header file that includes one or more `.inc` files. This is because
this header is expected to be part of the installation and distributed to
end-users, so should be self-contained.
### Entry Points ### Entry Points
``` ```
./offload-tblgen -I <path-to-llvm>/offload/API <path-to-llvm>/offload/API/OffloadAPI.td --gen-entry-points ./offload-tblgen -I <path-to-llvm>/offload/API <path-to-llvm>/offload/API/OffloadAPI.td --gen-entry-points
``` ```
@@ -111,6 +156,7 @@ function calls with arguments and results. The tracing can be enabled with the
`OFFLOAD_TRACE` environment variable. `OFFLOAD_TRACE` environment variable.
### Implementation function declarations ### Implementation function declarations
``` ```
./offload-tblgen -I <path-to-llvm>/offload/API <path-to-llvm>/offload/API/OffloadAPI.td --gen-impl-func-decls ./offload-tblgen -I <path-to-llvm>/offload/API <path-to-llvm>/offload/API/OffloadAPI.td --gen-impl-func-decls
``` ```
@@ -118,11 +164,12 @@ Generates declarations of the implementation of functions of every entry point
in the API, e.g. `offloadDeviceFoo_impl` for `offloadDeviceFoo`. in the API, e.g. `offloadDeviceFoo_impl` for `offloadDeviceFoo`.
### Print header ### Print header
``` ```
./offload-tblgen -I <path-to-llvm>/offload/API <path-to-llvm>/offload/API/OffloadAPI.td --gen-print-header ./offload-tblgen -I <path-to-llvm>/offload/API <path-to-llvm>/offload/API/OffloadAPI.td --gen-print-header
``` ```
This header contains `std::ostream &operator<<(std::ostream&)` definitions for This header contains `llvm::raw_ostream &operator<<(llvm::raw_ostream &)`
various API objects, including function parameters. definitions for various API objects, including function parameters.
As with the API header, it is expected that this header is part of the installed As with the API header, it is expected that this header is part of the installed
package, so it is entirely generated by Tablegen. package, so it is entirely generated by Tablegen.
@@ -130,21 +177,7 @@ package, so it is entirely generated by Tablegen.
For ease of implementation, and since it is not strictly part of the API, this For ease of implementation, and since it is not strictly part of the API, this
is a C++ header file. If a C version is desirable it could be added. is a C++ header file. If a C version is desirable it could be added.
### Future Tablegen backends ### Additional Tablegen backends
`RecordTypes.hpp` contains wrappers for all of the API object types, which will
allow more backends to be easily added in future.
## Adding to the API `RecordTypes.hpp` contains wrappers for all of the API object types, which
allows new backends to be easily added if needed.
A new object can be added to the API by adding to one of the existing `.td`
files. It is also possible to add a new tablegen file to the API by adding it
to the includes in `OffloadAPI.td`. When the `OffloadGenerate` target is
rebuilt, the new definition will be included in the generated files.
### Adding a new entry point
When a new entry point is added (e.g. `offloadDeviceFoo`), the actual entry
point is automatically generated, which contains validation and tracing code.
It expects an implementation function (`offloadDeviceFoo_impl`) to be defined,
which it will call into. The definition of this implementation function should
be added to `src/OffloadImpl.cpp`

38
offload/liboffload/README.md
View File

@@ -1,8 +1,34 @@
# Offload New API # Liboffload
This directory contains the implementation of the experimental work-in-progress This directory contains the implementation of the work-in-progress new API for
new API for Offload. It builds on top of the existing plugin implementations but Offload. It builds on top of the existing plugin implementations but provides a
provides a single level of abstraction suitable for runtimes for languages other single level of abstraction suitable for implementation of many offloading
than OpenMP to be built on top of. language runtimes, rather than just OpenMP.
See the [API definition readme](API/README.md) for implementation details. ## Testing liboffload
The main test suite for liboffload can be run with the `check-offload-unit`
target, which runs the `offload.unittests` executable. The test suite will
automatically run on every available device, but can be restricted to a single
platform (CUDA, AMDGPU) with a command line argument:
```sh
$ ./offload.unittests --platform=CUDA
```
Tracing of Offload API calls can be enabled by setting the `OFFLOAD_TRACE`
environment variable. This works with any program that uses liboffload.
```sh
$ OFFLOAD_TRACE=1 ./offload.unittests
---> olInit()-> OL_SUCCESS
# etc
```
The host plugin is not currently supported.
## Modifying liboffload
The main header (`OffloadAPI.h`) and some implementation details are
autogenerated with tablegen. See the [API definition README](API/README.md)
for implementation details.

2
offload/tools/offload-tblgen/MiscGen.cpp
View File

@@ -82,7 +82,7 @@ void EmitOffloadErrcodes(const RecordKeeper &Records, raw_ostream &OS) {
#endif #endif
// Error codes are shared between PluginInterface and liboffload. // Error codes are shared between PluginInterface and liboffload.
// To add new error codes, add them to offload/liboffload/API/Common.td and run the GenerateOffload target. // To add new error codes, add them to offload/liboffload/API/Common.td.
)"; )";