3b4e79398d
This patch introduces support for Integrated Distributed ThinLTO (DTLTO) in ELF LLD. DTLTO enables the distribution of ThinLTO backend compilations via external distribution systems, such as Incredibuild, during the traditional link step: https://llvm.org/docs/DTLTO.html. It is expected that users will invoke DTLTO through the compiler driver (e.g., Clang) rather than calling LLD directly. A Clang-side interface for DTLTO will be added in a follow-up patch. Note: Bitcode members of archives (thin or non-thin) are not currently supported. This will be addressed in a future change. As a consequence of this lack of support, this patch is not sufficient to allow for self-hosting an LLVM build with DTLTO. Theoretically, --start-lib/--end-lib could be used instead of archives in a self-host build. However, it's unclear how --start-lib/--end-lib can be easily used with the LLVM build system. Testing: - ELF LLD `lit` test coverage has been added, using a mock distributor to avoid requiring Clang. - Cross-project `lit` tests cover integration with Clang. For the design discussion of the DTLTO feature, see: #126654.
43 lines
1.7 KiB
ReStructuredText
43 lines
1.7 KiB
ReStructuredText
Integrated Distributed ThinLTO (DTLTO)
|
|
======================================
|
|
|
|
Integrated Distributed ThinLTO (DTLTO) enables the distribution of backend
|
|
ThinLTO compilations via external distribution systems, such as Incredibuild,
|
|
during the traditional link step.
|
|
|
|
The implementation is documented here: https://llvm.org/docs/DTLTO.html.
|
|
|
|
Currently, DTLTO is only supported in ELF LLD. Support will be added to other
|
|
LLD flavours in the future.
|
|
|
|
ELF LLD
|
|
-------
|
|
|
|
The command-line interface is as follows:
|
|
|
|
- ``--thinlto-distributor=<path>``
|
|
Specifies the file to execute as the distributor process. If specified,
|
|
ThinLTO backend compilations will be distributed.
|
|
|
|
- ``--thinlto-remote-compiler=<path>``
|
|
Specifies the path to the compiler that the distributor process will use for
|
|
backend compilations. The compiler invoked must match the version of LLD.
|
|
|
|
- ``--thinlto-distributor-arg=<arg>``
|
|
Specifies ``<arg>`` on the command line when invoking the distributor.
|
|
Can be specified multiple times.
|
|
|
|
- ``--thinlto-remote-compiler-arg=<arg>``
|
|
Appends ``<arg>`` to the remote compiler's command line.
|
|
Can be specified multiple times.
|
|
|
|
Options that introduce extra input/output files may cause miscompilation if
|
|
the distribution system does not automatically handle pushing/fetching them to
|
|
remote nodes. In such cases, configure the distributor - possibly using
|
|
``--thinlto-distributor-arg=`` - to manage these dependencies. See the
|
|
distributor documentation for details.
|
|
|
|
Some LLD LTO options (e.g., ``--lto-sample-profile=<file>``) are supported.
|
|
Currently, other options are silently accepted but do not have the intended
|
|
effect. Support for such options will be expanded in the future.
|