Commit Graph

106 Commits

Author SHA1 Message Date
Daniil Kovalev
cca9115b1c [lld][AArch64][ELF][PAC] Support AUTH relocations and AUTH ELF marking (#72714)
This patch adds lld support for:

- Dynamic R_AARCH64_AUTH_* relocations (without including RELR compressed AUTH
relocations) as described here:
https://github.com/ARM-software/abi-aa/blob/main/pauthabielf64/pauthabielf64.rst#auth-variant-dynamic-relocations

- .note.AARCH64-PAUTH-ABI-tag section as defined here
https://github.com/ARM-software/abi-aa/blob/main/pauthabielf64/pauthabielf64.rst#elf-marking

Depends on #72713 and #85231

---------

Co-authored-by: Peter Collingbourne <peter@pcc.me.uk>
Co-authored-by: Fangrui Song <i@maskray.me>
2024-04-04 12:38:09 +03:00
Fangrui Song
e115c00565 [ELF] Reject certain unknown section types (#85173)
Unknown section sections may require special linking rules, and
rejecting such sections for older linkers may be desired. For example,
if we introduce a new section type to replace a control structure (e.g.
relocations), it would be nice for older linkers to reject the new
section type. GNU ld allows certain unknown section types:

* [SHT_LOUSER,SHT_HIUSER] and non-SHF_ALLOC
* [SHT_LOOS,SHT_HIOS] and non-SHF_OS_NONCONFORMING

but reports errors and stops linking for others (unless
--no-warn-mismatch is specified). Port its behavior. For convenience, we
additionally allow all [SHT_LOPROC,SHT_HIPROC] types so that we don't
have to hard code all known types for each processor.

Close https://github.com/llvm/llvm-project/issues/84812
2024-03-15 09:50:23 -07:00
Fangrui Song
f1ca2a0967 [ELF] Add --compress-section to compress matched non-SHF_ALLOC sections
--compress-sections <section-glib>=[none|zlib|zstd] is similar to
--compress-debug-sections but applies to broader sections without the
SHF_ALLOC flag. lld will report an error if a SHF_ALLOC section is
matched. An interesting use case is to compress `.strtab`/`.symtab`,
which consume a significant portion of the file size (15.1% for a
release build of Clang).

An older revision is available at https://reviews.llvm.org/D154641 .
This patch focuses on non-allocated sections for safety. Moving
`maybeCompress` as D154641 does not handle STT_SECTION symbols for
`-r --compress-debug-sections=zlib` (see `relocatable-section-symbol.s`
from #66804).

Since different output sections may use different compression
algorithms, we need CompressedData::type to generalize
config->compressDebugSections.

GNU ld feature request: https://sourceware.org/bugzilla/show_bug.cgi?id=27452

Link: https://discourse.llvm.org/t/rfc-compress-arbitrary-sections-with-ld-lld-compress-sections/71674

Pull Request: https://github.com/llvm/llvm-project/pull/84855
2024-03-12 10:56:14 -07:00
Fangrui Song
78762357d4 [ELF] Support placing .lbss/.lrodata/.ldata after .bss
https://reviews.llvm.org/D150510 places .lrodata before .rodata to
minimize the number of permission transitions in the memory image.
However, this layout is less ideal for -fno-pic code (which is still
important).

Small code model -fno-pic code has R_X86_64_32S relocations with a range
of `[0,2**31)` (if we ignore the negative area). Placing `.lrodata`
earlier exerts relocation pressure on such code. Non-x86 64-bit
architectures generally have a similar `[0,2**31)` limitation if they
don't use PC-relative relocations.

If we place .lrodata later, we will need one extra PT_LOAD. Two layouts
are appealing:

* .bss/.lbss/.lrodata/.ldata (GNU ld)
* .bss/.ldata/.lbss/.lrodata

The GNU ld layout has the nice property that there is only one BSS
(except .tbss/.relro_padding). Add -z lrodata-after-bss to support
this layout.

Since a read-only PT_LOAD segment (for large data sections) may appear
after RW PT_LOAD segments. The placement of `_etext` has to be adjusted.

Pull Request: https://github.com/llvm/llvm-project/pull/81224
2024-02-20 13:59:49 -08:00
spupyrev
b53c04a8da Reapply [ELF] Making cdsort default for function reordering (#68638)
Edited lld/ELF/Options.td to cdsort as well

CDSort function reordering outperforms the existing default heuristic (
hfsort/C^3) in terms of the performance of generated binaries while
being (almost) as fast. Thus, the suggestion is to change the default.
The speedup is up to 1.5% perf for large front-end binaries, and can be
moderate/neutral for "small" benchmarks.

High-level **perf impact** on two selected binaries:
clang-10 binary (built with LTO+AutoFDO/CSSPGO): wins on top of C^3 in
[0.3%..0.8%]
rocksDB-8 binary (built with LTO+CSSPGO): wins on top of C^3 in
[0.8%..1.5%]

More detailed measurements on the clang binary is at
[here](https://reviews.llvm.org/D152834#4445042)
2023-11-03 16:03:06 -07:00
Fangrui Song
60b3e05967 [ELF] Restore the --call-graph-profile-sort=hfsort default before #68638
The high time complexity of cache-directed sort is a real issue and is not
appropriate as the default, at least for now
(https://github.com/llvm/llvm-project/pull/68638#issuecomment-1760918891).
2023-10-12 22:58:42 -07:00
spupyrev
d5c1d735ad [ELF] Making cdsort default for function reordering (#68638)
CDSort function reordering outperforms the existing default heuristic (
hfsort/C^3) in terms of the performance of generated binaries while
being (almost) as fast. Thus, the suggestion is to change the default.
The speedup is up to 1.5% perf for large front-end binaries, and can be
moderate/neutral for "small" benchmarks.

High-level **perf impact** on two selected binaries:
clang-10 binary (built with LTO+AutoFDO/CSSPGO): wins on top of C^3 in
[0.3%..0.8%]
rocksDB-8 binary (built with LTO+CSSPGO): wins on top of C^3 in
[0.8%..1.5%]

More detailed measurements on the clang binary is at
[here](https://reviews.llvm.org/D152834#4445042)
2023-10-10 09:06:31 -07:00
spupyrev
904b3f66f5 [ELF] A new code layout algorithm for function reordering [3a/3]
We are brining a new algorithm for function layout (reordering) based on the
call graph (extracted from a profile data). The algorithm is an improvement of
top of a known heuristic, C^3. It tries to co-locate hot and frequently executed
together functions in the resulting ordering. Unlike C^3, it explores a larger
search space and have an objective closely tied to the performance of
instruction and i-TLB caches. Hence, the name CDS = Cache-Directed Sort.
The algorithm can be used at the linking or post-linking (e.g., BOLT) stage.
Refer to https://reviews.llvm.org/D152834 for the actual implementation of the
reordering algorithm.

This diff adds a linker option to replace the existing C^3 heuristic with CDS.
The new behavior can be turned on by passing "--use-cache-directed-sort".
(the plan is to make it default in a next diff)

**Perf-impact**
clang-10 binary (built with LTO+AutoFDO/CSSPGO): wins on top of C^3 in [0.3%..0.8%]
rocksDB-8 binary (built with LTO+CSSPGO): wins on top of C^3 in [0.8%..1.5%]

Note that function layout affects the perf the most on older machines (with
smaller instruction/iTLB caches) and when huge pages are not enabled. The impact
on newer processors with huge pages enabled is likely neutral/minor.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D152840
2023-09-26 06:24:34 -07:00
Fangrui Song
8c556b7e2b [ELF] Change --call-graph-profile-sort to accept an argument
Change the FF form --call-graph-profile-sort to --call-graph-profile-sort={none,hfsort}.
This will be extended to support llvm/lib/Transforms/Utils/CodeLayout.cpp.

--call-graph-profile-sort is not used in the wild but
--no-call-graph-profile-sort is (Chromium). Make --no-call-graph-profile-sort an
alias for --call-graph-profile-sort=none.

Reviewed By: rahmanl

Differential Revision: https://reviews.llvm.org/D159544
2023-09-25 09:49:40 -07:00
Shoaib Meenai
97e39f96c8 [ELF] Add -Bsymbolic-non-weak
This adds a new -Bsymbolic option that directly binds all non-weak
symbols. There's a couple of reasons motivating this:
* The new flag will match the default behavior on Mach-O, so you can get
  consistent behavior across platforms.
* We have use cases for which making weak data preemptible is useful,
  but we don't want to pessimize access to non-weak data. (For a large
  internal app, we measured 2000+ data symbols whose accesses would be
  unnecessarily pessimized by `-Bsymbolic-functions`.)

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D158322
2023-08-21 09:11:51 -07:00
Paul Kirth
14e3bec8fc Reland "[lld] Preliminary fat-lto-object support"
This patch adds support to lld for --fat-lto-objects. We add a new
--fat-lto-objects option to LLD, and slightly change how it chooses input
files in the driver when the option is set.

Fat LTO objects contain both LTO compatible IR, as well as generated object
code. This allows users to defer the choice of whether to use LTO or not to
link-time. This is a feature available in GCC for some time, and makes the
existing -ffat-lto-objects option functional in the same way as GCC's.

If the --fat-lto-objects option is passed to LLD and the input files are fat
object files, then the linker will chose the LTO compatible bitcode sections
embedded within the fat object and link them together using LTO. Otherwise,
standard object file linking is done using the assembly section in the object
files.

The previous version of this patch had a missing `REQUIRES: x86` line in
`fatlto.invalid.s`. Additionally, it was reported that this patch caused
a test failure in `export-dynamic-symbols.s`, however,
29112a9946 disabled the
`export-dynamic-symbols.s` test on Windows due to a quotation difference
between platforms, unrelated to this patch.

Original RFC: https://discourse.llvm.org/t/rfc-ffat-lto-objects-support/63977

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D146778
2023-08-18 22:51:25 +00:00
WANG Xuerui
6084ee7420 [lld][ELF] Support LoongArch
This adds support for the LoongArch ELF psABI v2.00 [1] relocation
model to LLD. The deprecated stack-machine-based psABI v1 relocs are not
supported.

The code is tested by successfully bootstrapping a Gentoo/LoongArch
stage3, complete with common GNU userland tools and both the LLVM and
GNU toolchains (GNU toolchain is present only for building glibc,
LLVM+Clang+LLD are used for the rest). Large programs like QEMU are
tested to work as well.

[1]: https://loongson.github.io/LoongArch-Documentation/LoongArch-ELF-ABI-EN.html

Reviewed By: MaskRay, SixWeining

Differential Revision: https://reviews.llvm.org/D138135
2023-07-25 17:06:07 +08:00
Paul Kirth
1733d94963 Revert "[lld] Preliminary fat-lto-object support"
This reverts commit c9953d9891 and a
forward fix in 3a45b843de.

D14677 causes some failure on windows bots that the forward fix did not
address. Thus I'm reverting until the underlying cause can me triaged.
2023-07-20 03:37:48 +00:00
Paul Kirth
3a45b843de [lld] Preliminary fat-lto-object support
This patch adds support to lld for --fat-lto-objects. We add a new
--fat-lto-objects flag to LLD, and slightly change how it chooses input
files in the driver when the flag is set.

Fat LTO objects contain both LTO compatible IR, as well as generated object
code. This allows users to defer the choice of whether to use LTO or not to
link-time. This is a feature available in GCC for some time, and makes the
existing -ffat-lto-objects flag functional in the same way as GCC's.

If the --fat-lto-objects option is passed to LLD and the input files are fat
object files, then the linker will chose the LTO compatible bitcode sections
embedded within the fat object and link them together using LTO. Otherwise,
standard object file linking is done using the assembly section in the object
files.

Original RFC: https://discourse.llvm.org/t/rfc-ffat-lto-objects-support/63977

Depends on D146777

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D146778
2023-07-19 23:07:42 +00:00
Simi Pallipurath
f146763e07 Revert "Revert "[lld][Arm] Big Endian - Byte invariant support.""
This reverts commit d8851384c6.

Reason: Applied the fix for the Asan buildbot failures.
2023-06-22 16:10:18 +01:00
Simi Pallipurath
d8851384c6 Revert "[lld][Arm] Big Endian - Byte invariant support."
This reverts commit 8cf8956897.
2023-06-20 17:27:44 +01:00
Simi Pallipurath
8cf8956897 [lld][Arm] Big Endian - Byte invariant support.
Arm has BE8 big endian configuration called a byte-invariant(every byte has the same address on little and big-endian systems).

When in BE8 mode:
  1. Instructions are big-endian in relocatable objects but
     little-endian in executables and shared objects.
  2. Data is big-endian.
  3. The data encoding of the ELF file is ELFDATA2MSB.

To support BE8 without an ABI break for relocatable objects,the linker takes on the responsibility of changing the endianness of instructions. At a high level the only difference between BE32 and BE8 in the linker is that for BE8:
  1. The linker sets the flag EF_ARM_BE8 in the ELF header.
  2. The linker endian reverses the instructions, but not data.

This patch adds BE8 big endian support for Arm. To endian reverse the instructions we'll need access to the mapping symbols. Code sections can contain a mix of Arm, Thumb and literal data. We need to endian reverse Arm instructions as words, Thumb instructions
as half-words and ignore literal data.The only way to find these transitions precisely is by using mapping symbols. The instruction reversal will need to take place after relocation. For Arm BE8 code sections (Section has SHF_EXECINSTR flag ) we inserted a step after relocation to endian reverse the instructions. The implementation strategy i have used here is to write all sections BE32  including SyntheticSections then endian reverse all code in InputSections via mapping symbols.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D150870
2023-06-20 14:08:21 +01:00
Fangrui Song
39c20a63b1 [ELF] Add --remap-inputs= and --remap-inputs-file=
--remap-inputs-file= can be specified multiple times, each naming a
remap file that contains `from-glob=to-file` lines or `#`-led comments.
('=' is used a separator a la -fdebug-prefix-map=)
--remap-inputs-file= can be used to:

* replace an input file. E.g. `"*/libz.so=exp/libz.so"` can replace a resolved
  `-lz` without updating the input file list or (if used) a response file.
  When debugging an application where a bug is isolated to one single
  input file, this option gives a convenient way to test fixes.
* remove an input file with `/dev/null` (changed to `NUL` on Windows), e.g.
  `"a.o=/dev/null"`. A build system may add unneeded dependencies.
  This option gives a convenient way to test the result removing some inputs.

`--remap-inputs=a.o=aa.o` can be specified to provide one pattern without using
an extra file.
(bash/zsh process substitution is handy for specifying a pattern without using
a remap file, e.g. `--remap-inputs-file=<(printf 'a.o=aa.o')`, but it may be
unavailable in some systems. An extra file can be inconvenient for a build
system.)

Exact patterns are tested before wildcard patterns. In case of a tie, the first
patterns wins. This is an implementation detail that users should not rely on.

Co-authored-by: Marco Elver <elver@google.com>
Link: https://discourse.llvm.org/t/rfc-support-exclude-inputs/70070

Reviewed By: melver, peter.smith

Differential Revision: https://reviews.llvm.org/D148859
2023-04-26 13:18:55 -07:00
Craig Topper
85444794cd [lld][RISCV] Implement GP relaxation for R_RISCV_HI20/R_RISCV_LO12_I/R_RISCV_LO12_S.
This implements support for relaxing these relocations to use the GP
register to compute addresses of globals in the .sdata and .sbss
sections.

This feature is off by default and must be enabled by passing
--relax-gp to the linker.

The GP register might not always be the "global pointer". It can
be used for other purposes. See discussion here
https://github.com/riscv-non-isa/riscv-elf-psabi-doc/pull/371

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D143673
2023-04-13 10:52:15 -07:00
Fangrui Song
3d570a5c5f [ELF][docs] Mention LLD_REPRODUCE and LLD_VERSION
Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D143600
2023-02-09 14:38:17 -08:00
Dan Albert
241dbd3105 [ELF] Enable --no-undefined-version by default
Allowing incorrect version scripts is not a helpful default. Flip that
to help users find their bugs at build time rather than at run time.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D135402
2022-12-08 01:41:18 +00:00
Jon Chesterfield
b1fdeeec1a Revert "[LLD] Enable --no-undefined-version by default."
This reverts commit eedbe44b87.
2022-11-09 16:17:33 +00:00
Dan Albert
eedbe44b87 [LLD] Enable --no-undefined-version by default.
Allowing incorrect version scripts is not a helpful default. Flip that
to help users find their bugs at build time rather than at run time.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D135402
2022-11-07 22:15:28 +00:00
Fangrui Song
26fcee601f [ELF] Add --no-warnings/-w
Mach-O ld64 supports -w to suppress warnings. GNU ld 2.40 will support the
option as well (https://sourceware.org/bugzilla/show_bug.cgi?id=29654).

This feature has some small value. E.g. when analyzing a large executable with
relocation overflow issues, we may use --noinhibit-exec --emit-relocs to get an
output file with static relocations despite relocation overflow issues. -w can
significantly improve the link time as printing the massive warnings is slow.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D136569
2022-10-24 10:54:09 -07:00
Slava Zakharin
88da0de14f Revert "[Libomp] Do not error on undefined version script symbols"
This reverts commit 096f93e73d.

Revert "[Libomptarget] Make the plugins ingore undefined exported symbols"

This reverts commit 3f62314c23.

Revert "[LLD] Enable --no-undefined-version by default."

This reverts commit 7ec8b0d162.

Three commits are reverted because of the current omp build fail
with GNU ld. See discussion here: https://reviews.llvm.org/rG096f93e73dc3
2022-10-13 14:12:07 -07:00
Dan Albert
7ec8b0d162 [LLD] Enable --no-undefined-version by default.
Allowing incorrect version scripts is not a helpful default. Flip that
to help users find their bugs at build time rather than at run time.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D135402
2022-10-12 12:29:38 -07:00
Fangrui Song
06010fd1be [ELF] Make -V an alias for -v
In GNU ld,

* --version skips linker input processing.
* -v and -V keep processing if there is any input file. -V has more
  information we don't support.

We currently make -V an alias for --version which skips input processing.
On many `*-freebsd` and `powerpc-*` targets, `gcc -v` passes `-V` to ld
and expects to process input. Make -V an alias for -v to provide
compatibility.

Fix https://github.com/llvm/llvm-project/issues/57859
2022-09-20 17:12:44 -07:00
Fangrui Song
449f2ca146 [ELF] Add --compress-debug-sections=zstd
`clang -gz=zstd a.o` passes this option to the linker. This option compresses output
debug sections with zstd and sets ch_type to ELFCOMPRESS_ZSTD. As of today, very
few DWARF consumers recognize ELFCOMPRESS_ZSTD.

Use the llvm::zstd::compress API with level llvm::zstd::DefaultCompression (5),
which we may tune after we have more experience with zstd output.
zstd has built-in parallel compression support (so we don't need to do D117853
for zlib), which is not leveraged yet.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D133548
2022-09-09 10:30:18 -07:00
Fangrui Song
4a8de2832a [ELF] Add -z pack-relative-relocs
GNU ld 2.38 added -z pack-relative-relocs which is similar to
--pack-dyn-relocs=relr but synthesizes the `GLIBC_ABI_DT_RELR` version
dependency if a shared object named `libc.so.*` has a `GLIBC_2.*` version
dependency.

This is used to implement the (as some glibc folks call) version lockout
mechanism. Add this option, because glibc does not want to support
--pack-dyn-relocs=relr which does not add `GLIBC_ABI_DT_RELR`.
See https://maskray.me/blog/2021-10-31-relative-relocations-and-relr for
detail.

Close https://github.com/llvm/llvm-project/issues/53775

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D120701
2022-03-10 19:54:21 -08:00
Fangrui Song
4631cba10b [ELF][docs] Remove ignore -dc from ld.lld.1 2022-02-09 10:38:36 -08:00
Fangrui Song
ce45c95694 [ELF] Remove obscure -dp and GNU ld incompatible --[no-]define-common, ignore -d/-dc
https://maskray.me/blog/2022-02-06-all-about-common-symbols#no-define-common

In GNU ld, -dc only affects -r links and causes COMMON symbols to be allocated.
--no-define-common is defined to make COMMON symbols undefined for -shared.
AIUI --no-define-common is a workaround around glibc 2.1 time and not really useful.

gold confuses --define-common with -d/FORCE_COMMON_ALLOCATION and implements
--define-common with -d semantics. Its --no-define-common is incompatible with
GNU ld.

In ld.lld, b2a23cf3c0 fixed the default -r
behavior for COMMON symbols but ported the incompatible gold
--[no-]define-common. To the best of my knowledge, no project uses -dp
--[no-]define-common. So just remove these options.

-d/-dc are used by the following projects:

* grub grub-core/genmod.sh.in uses -Wl,-r,-d (https://lists.gnu.org/archive/html/grub-devel/2022-02/msg00088.html)
* FreeBSD crunchgen uses -Wl,-dc (https://reviews.freebsd.org/D34215)

A no-op implementation works for them. Only when a program inspects relocatable
output by itself and does not recognize COMMON symbols, there may be a problem.
This is an extremely unlikely case.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D119108
2022-02-09 10:35:53 -08:00
Fangrui Song
bfc2f4b122 [ELF] Update help messages to prefer canonical name for some long options
And improve the help message for --pop-state.
2022-01-06 00:43:46 -08:00
Daniel Kiss
2b4e6052b3 [lld] Add cet-report and bti-report flags
Implement cet-report as supported in binutils.
bti-report has the same behaviour for AArch64-BTI.

Fixes https://github.com/llvm/llvm-project/issues/44828

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D113901
2021-12-16 16:26:26 +01:00
Fangrui Song
1ce51a5f35 [ELF] --cref: If -Map is specified, print to the map file
PR48282: This behavior matches GNU ld and gold.

Reviewed By: markj

Differential Revision: https://reviews.llvm.org/D114663
2021-11-29 14:14:53 -08:00
Fangrui Song
3b4dd68de5 [ELF][PPC64] Make --power10-stubs/--no-power10-stubs proper aliases for --power10-stubs={auto,no}
This allows --power10-stubs= and --[no-]power10-stubs to override each other
(they are position dependent in GNU ld).

Also improve --help messages and the manpage.

Note: GNU ld's default "auto" mode uses heuristics to decide whether Power10
instructions are used. Arguably it is a design mistake of R_PPC64_REL24_NOTOC
(acked by the relevant folks on a libc-alpha discussion). We don't implement
"auto", so the default --power10-stubs is the same as "yes".
2021-11-26 11:51:45 -08:00
Fangrui Song
a05384dc89 [ELF] Make --no-relax disable R_X86_64_GOTPCRELX and R_X86_64_REX_GOTPCRELX GOT optimization
This brings back the original version of D81359.
I have found several use cases now.

* Unlike GNU ld, LLD's relocation processing is one pass. If we decide to
  optimize(relax) R_X86_64_{,REX_}GOTPCRELX, we will suppress GOT generation and
  cannot undo the decision later. Optimizing R_X86_64_REX_GOTPCRELX can usually
  make it easy to hit `relocation R_X86_64_REX_GOTPCRELX out of range` because
  the distance to GOT is usually shorter. Without --no-relax, the user has to
  recompile with `-Wa,-mrelax-relocations=no`.
* The option would help during my investigationg of the root cause of https://git.kernel.org/linus/09e43968db40c33a73e9ddbfd937f46d5c334924
* There is need for relaxation for AArch64 & RISC-V. Implementing this for
  x86-64 improves consistency with little target-specific cost (two-line
  X86_64.cpp change).

Reviewed By: alexander-shaposhnikov

Differential Revision: https://reviews.llvm.org/D113615
2021-11-12 09:47:31 -08:00
Fangrui Song
a954bb18b1 [ELF] Add --why-extract= to query why archive members/lazy object files are extracted
Similar to D69607 but for archive member extraction unrelated to GC. This patch adds --why-extract=.

Prior art:

GNU ld -M prints
```
Archive member included to satisfy reference by file (symbol)

a.a(a.o)                      main.o (a)
b.a(b.o)                      (b())
```

-M is mainly for input section/symbol assignment <-> output section mapping
(often huge output) and the information may appear ad-hoc.

Apple ld64
```
__Z1bv forced load of b.a(b.o)
_a forced load of a.a(a.o)
```

It doesn't say the reference file.

Arm's proprietary linker
```
Selecting member vsnprintf.o(c_wfu.l) to define vsnprintf.
...
Loading member vsnprintf.o from c_wfu.l.
              definition:  vsnprintf
              reference :  _printf_a
```

---

--why-extract= gives the user the full data (which is much shorter than GNU ld
-Map). It is easy to track a chain of references to one archive member with a
one-liner, e.g.

```
% ld.lld main.o a_b.a b_c.a c.a -o /dev/null --why-extract=- | tee stdout
reference       extracted       symbol
main.o  a_b.a(a_b.o)    a
a_b.a(a_b.o)    b_c.a(b_c.o)    b()
b_c.a(b_c.o)    c.a(c.o)        c()

% ruby -ane 'BEGIN{p={}}; p[$F[1]]=[$F[0],$F[2]] if $.>1; END{x="c.a(c.o)"; while y=p[x]; puts "#{y[0]} extracts #{x} to resolve #{y[1]}"; x=y[0] end}' stdout
b_c.a(b_c.o) extracts c.a(c.o) to resolve c()
a_b.a(a_b.o) extracts b_c.a(b_c.o) to resolve b()
main.o extracts a_b.a(a_b.o) to resolve a
```

Archive member extraction happens before --gc-sections, so this may not be a live path
under --gc-sections, but I think it is a good approximation in practice.

* Specifying a file avoids output interleaving with --verbose.
* Required `=` prevents accidental overwrite of an input if the user forgets `=`. (Most of compiler drivers' long options accept `=` but not ` `)

Differential Revision: https://reviews.llvm.org/D109572
2021-09-20 09:52:30 -07:00
Fangrui Song
44361e5b90 [ELF] Add --export-dynamic-symbol-list
This is available in GNU ld 2.35 and can be seen as a shortcut for multiple
--export-dynamic-symbol, or a --dynamic-list variant without the symbolic intention.

In the long term, this option probably should be preferred over --dynamic-list.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D107317
2021-08-03 09:01:03 -07:00
Fangrui Song
b06426da76 [ELF] Add -Bsymbolic-non-weak-functions
This option is a subset of -Bsymbolic-functions. It applies to STB_GLOBAL
STT_FUNC definitions.

The address of a vague linkage function (STB_WEAK STT_FUNC, e.g. an inline
function, a template instantiation) seen by a -Bsymbolic-functions linked
shared object may be different from the address seen from outside the shared
object. Such cases are uncommon. (ELF/Mach-O programs may use
`-fvisibility-inlines-hidden` to break such pointer equality.  On Windows,
correct dllexport and dllimport are needed to make pointer equality work.
Windows link.exe enables /OPT:ICF by default so different inline functions may
have the same address.)

```
// a.cc -> a.o -> a.so (-Bsymbolic-functions)
inline void f() {}
void *g() { return (void *)&f; }

// b.cc -> b.o -> exe
// The address is different!
inline void f() {}
```

-Bsymbolic-non-weak-functions is a safer (C++ conforming) subset of
-Bsymbolic-functions, which can make such programs work.

Implementations usually emit a vague linkage definition in a COMDAT group.  We
could detect the group (with more code) but I feel that we should just check
STB_WEAK for simplicity. A weak definition will thus serve as an escape hatch
for rare cases when users want interposition on definitions.

GNU ld feature request: https://sourceware.org/bugzilla/show_bug.cgi?id=27871

Longer write-up: https://maskray.me/blog/2021-05-16-elf-interposition-and-bsymbolic

If Linux distributions migrate to protected non-vague-linkage external linkage
functions by default, the linker option can still be handy because it allows
rapid experiment without recompilation. Protected function addresses currently
have deep issues in GNU ld.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D102570
2021-07-29 14:46:53 -07:00
Fangrui Song
7f0acc4e4f [docs] ld.lld.1: Mention -z nostart-stop-gc 2021-05-21 19:57:51 -07:00
Fangrui Song
4adf7a7604 [ELF] Add -Bno-symbolic
This option will be available in GNU ld 2.27 (https://sourceware.org/bugzilla/show_bug.cgi?id=27834).
This option can cancel previously specified -Bsymbolic and
-Bsymbolic-functions.  This is useful for excluding some links when the
default uses -Bsymbolic-functions.

Reviewed By: jhenderson, peter.smith

Differential Revision: https://reviews.llvm.org/D102383
2021-05-14 09:40:32 -07:00
Ben Dunbobbin
5dd9f44c17 [LLD] Improve --strip-all help text
This is a slight improvement to the help text, as I was slightly
surprised when strip-all did more than remove the symbol table.

Currently, we match gold's help text for strip-all and strip-debug.
I think that the GNU documentation for these options is not particularly
clear. However, I have opted to make only a minor change here and keep
the help text similar to gold's as these are mature options that are
well understood.

ld.bfd (https://sourceware.org/binutils/docs/ld/Options.html) has a
similar implication although it defines strip-debug as a subset of
strip-all. However, felt that noting that strip-all implies strip-debug
is better; because, with the ld.bfd approach you have to read both the
--strip-debug and the --strip-all help text to understand the behaviour
of --strip-all (and the --strip-all help text doesn't indicate that he
--strip-debug help text is related).

Differential Revision: https://reviews.llvm.org/D101890
2021-05-06 12:34:06 +01:00
Fangrui Song
16c30c3c23 [ELF] Change --shuffle-sections=<seed> to --shuffle-sections=<section-glob>=<seed>
`--shuffle-sections=<seed>` applies to all sections.  The new
`--shuffle-sections=<section-glob>=<seed>` makes shuffling selective.  To the
best of my knowledge, the option is only used as debugging, so just drop the
original form.

`--shuffle-sections '.init_array*=-1'` `--shuffle-sections '.fini_array*=-1'`.
reverses static constructors/destructors of the same priority.
Useful to detect some static initialization order fiasco.

`--shuffle-sections '.data*=-1'`
reverses `.data*` sections. Useful to detect unfunded pointer comparison results
of two unrelated objects.

If certain sections have an intrinsic order, the old form cannot be used.

Differential Revision: https://reviews.llvm.org/D98679
2021-03-18 10:18:19 -07:00
Fangrui Song
423cb321df [ELF] Special case --shuffle-sections=-1 to reverse input sections
If the number of sections changes, which is common for re-links after
incremental updates, the section order may change drastically.

Special case -1 to reverse input sections. This is a stable transform.
The section order is more resilient to incremental updates.  Usually the
code issue (e.g. Static Initialization Order Fiasco, assuming pointer
comparison result of two unrelated objects) is due to the relative order
between two problematic input files A and B.  Checking the regular order
and the reversed order is sufficient.

Differential Revision: https://reviews.llvm.org/D98445
2021-03-17 09:32:44 -07:00
Fangrui Song
4bbcd63eea [ELF] Add -z start-stop-gc to let __start_/__stop_ not retain C identifier name sections
For one metadata section usage, each text section references a metadata section.
The metadata sections have a C identifier name to allow the runtime to collect them via `__start_/__stop_` symbols.

Since `__start_`/`__stop_` references are always present from live sections, the
C identifier name sections appear like GC roots, which means they cannot be
discarded by `ld --gc-sections`.

To make such sections GCable, either SHF_LINK_ORDER or a section group is needed.

SHF_LINK_ORDER is not suitable for the references can be inlined into other functions
(See D97430:
Function A (in the section .text.A) references its `__sancov_guard` section.
Function B inlines A (so now .text.B references `__sancov_guard` - this is invalid with the semantics of SHF_LINK_ORDER).

In the linking stage,
if `.text.A` gets discarded, and `__sancov_guard` is retained via the reference from `.text.B`,
the output will be invalid because `__sancov_guard` references the discarded `.text.A`.
LLD errors "sh_link points to discarded section".
)

A section group have size overhead, and is cumbersome when there is just one metadata section.

Add `-z start-stop-gc` to drop the "__start_/__stop_ references retain
non-SHF_LINK_ORDER non-SHF_GROUP C identifier name sections" rule.
We reserve the rights to switch the default in the future.

Reviewed By: phosek, jrtc27

Differential Revision: https://reviews.llvm.org/D96914
2021-02-25 15:46:37 -08:00
Fangrui Song
eea34aae2e [ELF] Inspect -EL & -EB for OUTPUT_FORMAT(default, big, little)
Choose big if -EB is specified, little if -EL is specified, or default if neither is specified.
The new behavior matches GNU ld.

Fixes: https://github.com/ClangBuiltLinux/linux/issues/1025

Differential Revision: https://reviews.llvm.org/D96214
2021-02-08 10:34:57 -08:00
Fangrui Song
d24b94f070 [ELF] --wrap: retain __wrap_foo if foo is defined in an object/bitcode file
If foo is referenced in any object file, bitcode file or shared object,
`__wrap_foo` should be retained as the redirection target of sym
(f96ff3c0f8).

If the object file defining foo has foo references, we cannot easily distinguish
the case from cases where foo is not referenced (we haven't scanned
relocations). Retain `__wrap_foo` because we choose to wrap sym references
regardless of whether sym is defined to keep non-LTO/LTO/relocatable links' behaviors similar
https://sourceware.org/bugzilla/show_bug.cgi?id=26358 .

If foo is defined in a shared object, `__wrap_foo` can still be omitted
(`wrap-dynamic-undef.s`).

Reviewed By: andrewng

Differential Revision: https://reviews.llvm.org/D95152
2021-01-22 09:20:29 -08:00
Sean Fertile
8f91f38148 [LLD] Search archives for symbol defs to override COMMON symbols.
This patch changes the archive handling to enable the semantics needed
for legacy FORTRAN common blocks and block data. When we have a COMMON
definition of a symbol and are including an archive, LLD will now
search the members for global/weak defintions to override the COMMON
symbol. The previous LLD behavior (where a member would only be included
if it satisifed some other needed symbol definition) can be re-enabled with the
option '-no-fortran-common'.

Differential Revision: https://reviews.llvm.org/D86142
2020-12-07 10:09:19 -05:00
serge-sans-paille
1e70ec10eb [lld] Provide a hook to customize undefined symbols error handling
This is a follow up to https://reviews.llvm.org/D87758, implementing the missing
symbol part, as done by binutils.

Differential Revision: https://reviews.llvm.org/D89687
2020-11-09 13:28:48 +01:00
serge-sans-paille
cfc32267e2 Provide a hook to customize missing library error handling
Make it possible for lld users to provide a custom script that would help to
find missing libraries. A possible scenario could be:

    % clang /tmp/a.c -fuse-ld=lld -loauth -Wl,--error-handling-script=/tmp/addLibrary.py
    unable to find library -loauth
    looking for relevant packages to provides that library

        liboauth-0.9.7-4.el7.i686
        liboauth-devel-0.9.7-4.el7.i686
        liboauth-0.9.7-4.el7.x86_64
        liboauth-devel-0.9.7-4.el7.x86_64
        pix-1.6.1-3.el7.x86_64

Where addLibrary would be called with the missing library name as first argument
(in that case addLibrary.py oauth)

Differential Revision: https://reviews.llvm.org/D87758
2020-11-03 11:01:29 +01:00