Commit Graph

1789 Commits

Author SHA1 Message Date
Fangrui Song
943baf3274 [ELF] Make compareByFilePosition a strict weak order
This fixes the new test linkerscript/enable-non-contiguous-regions.test
from #90007 in -stdlib=libc++ -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_DEBUG builds.

adjustOutputSections does not discard the output section .potential_a
because it contained .a (which would be spilled to .actual_a).
.potential_a and .bc have the same address and will cause an assertion
failure.
2024-05-13 15:47:35 -07:00
Daniel Thornburgh
66466ff151 Reland: [LLD] Implement --enable-non-contiguous-regions (#90007)
When enabled, input sections that would otherwise overflow a memory
region are instead spilled to the next matching output section.

This feature parallels the one in GNU LD, but there are some differences
from its documented behavior:

- /DISCARD/ only matches previously-unmatched sections (i.e., the flag
does not affect it).

- If a section fails to fit at any of its matches, the link fails
instead of discarding the section.

- The flag --enable-non-contiguous-regions-warnings is not implemented,
as it exists to warn about such occurrences.

The implementation places stubs at possible spill locations, and
replaces them with the original input section when effecting spills.
Spilling decisions occur after address assignment. Sections are spilled
in reverse order of assignment, with each spill naively decreasing the
size of the affected memory regions. This continues until the memory
regions are brought back under size. Spilling anything causes another
pass of address assignment, and this continues to fixed point.

Spilling after rather than during assignment allows the algorithm to
consider the size effects of unspillable input sections that appear
later in the assignment. Otherwise, such sections (e.g. thunks) may
force an overflow, even if spilling something earlier could have avoided
it.

A few notable feature interactions occur:

- Stubs affect alignment, ONLY_IF_RO, etc, broadly as if a copy of the
input section were actually placed there.

- SHF_MERGE synthetic sections use the spill list of their first
contained input section (the one that gives the section its name).

- ICF occurs oblivious to spill sections; spill lists for merged-away
sections become inert and are removed after assignment.

- SHF_LINK_ORDER and .ARM.exidx are ordered according to the final
section ordering, after all spilling has completed.

- INSERT BEFORE/AFTER and OVERWRITE_SECTIONS are explicitly disallowed.
2024-05-13 11:06:54 -07:00
Daniel Thornburgh
81f34afa5c Revert "[LLD] Implement --enable-non-contiguous-regions" (#92005)
Reverts llvm/llvm-project#90007

Broke in merging I think.
2024-05-13 10:38:40 -07:00
Daniel Thornburgh
673114447b [LLD] Implement --enable-non-contiguous-regions (#90007)
When enabled, input sections that would otherwise overflow a memory
region are instead spilled to the next matching output section.

This feature parallels the one in GNU LD, but there are some differences
from its documented behavior:

- /DISCARD/ only matches previously-unmatched sections (i.e., the flag
does not affect it).

- If a section fails to fit at any of its matches, the link fails
instead of discarding the section.

- The flag --enable-non-contiguous-regions-warnings is not implemented,
as it exists to warn about such occurrences.

The implementation places stubs at possible spill locations, and
replaces them with the original input section when effecting spills.
Spilling decisions occur after address assignment. Sections are spilled
in reverse order of assignment, with each spill naively decreasing the
size of the affected memory regions. This continues until the memory
regions are brought back under size. Spilling anything causes another
pass of address assignment, and this continues to fixed point.

Spilling after rather than during assignment allows the algorithm to
consider the size effects of unspillable input sections that appear
later in the assignment. Otherwise, such sections (e.g. thunks) may
force an overflow, even if spilling something earlier could have avoided
it.

A few notable feature interactions occur:

- Stubs affect alignment, ONLY_IF_RO, etc, broadly as if a copy of the
input section were actually placed there.

- SHF_MERGE synthetic sections use the spill list of their first
contained input section (the one that gives the section its name).

- ICF occurs oblivious to spill sections; spill lists for merged-away
sections become inert and are removed after assignment.

- SHF_LINK_ORDER and .ARM.exidx are ordered according to the final
section ordering, after all spilling has completed.

- INSERT BEFORE/AFTER and OVERWRITE_SECTIONS are explicitly disallowed.
2024-05-13 10:30:50 -07:00
Kazu Hirata
f841ca0c35 Use StringRef::operator== instead of StringRef::equals (NFC) (#91864)
I'm planning to remove StringRef::equals in favor of
StringRef::operator==.

- StringRef::operator==/!= outnumber StringRef::equals by a factor of
  276 under llvm-project/ in terms of their usage.

- The elimination of StringRef::equals brings StringRef closer to
  std::string_view, which has operator== but not equals.

- S == "foo" is more readable than S.equals("foo"), especially for
  !Long.Expression.equals("str") vs Long.Expression != "str".
2024-05-12 23:08:40 -07:00
cmtice
16711b431b [lld][ELF] Add --debug-names to create merged .debug_names. (#86508)
`clang -g -gpubnames` (with optional -gsplit-dwarf) creates the
`.debug_names` section ("per-CU" index). By default lld concatenates
input `.debug_names` sections into an output `.debug_names` section.
LLDB can consume the concatenated section but the lookup performance is
not good.

This patch adds --debug-names to create a per-module index by combining
the per-CU indexes into a single index that covers the entire load
module. The produced `.debug_names` is a replacement for `.gdb_index`.

Type units (-fdebug-types-section) are not handled yet.

Co-authored-by: Fangrui Song <i@maskray.me>

---------

Co-authored-by: Fangrui Song <i@maskray.me>
2024-04-18 14:41:14 -07:00
Fangrui Song
c258f57398 [ELF] Move createSyntheticSections from Writer.cpp to SyntheticSections.cpp. NFC
SyntheticSections.cpp is more appropriate. This change enables
elimination of many explicit template instantiations.

Due to `make<SymbolTableSection<ELFT>>(*strtab)` in Arch/ARM.cpp,
we do not remove explicit template instantiations for SymbolTableSection.
2024-04-10 13:42:51 -07:00
Fangrui Song
ee284d2da0 [ELF] Avoid make<GdbIndexSection>. NFC 2024-04-09 21:32:37 -07:00
Daniil Kovalev
cca9115b1c [lld][AArch64][ELF][PAC] Support AUTH relocations and AUTH ELF marking (#72714)
This patch adds lld support for:

- Dynamic R_AARCH64_AUTH_* relocations (without including RELR compressed AUTH
relocations) as described here:
https://github.com/ARM-software/abi-aa/blob/main/pauthabielf64/pauthabielf64.rst#auth-variant-dynamic-relocations

- .note.AARCH64-PAUTH-ABI-tag section as defined here
https://github.com/ARM-software/abi-aa/blob/main/pauthabielf64/pauthabielf64.rst#elf-marking

Depends on #72713 and #85231

---------

Co-authored-by: Peter Collingbourne <peter@pcc.me.uk>
Co-authored-by: Fangrui Song <i@maskray.me>
2024-04-04 12:38:09 +03:00
Fangrui Song
18a49f03aa [ELF] Merge relaIplt into relaDyn
`relaIplt` was added so that IRELATIVE relocations are placed at the end
of .rela.dyn (since https://reviews.llvm.org/D65651) or .rela.plt
(--pack-dyn-relocs=android[+relr]). Unfortunately, handling `relaIplt`
requires special cases all over the code base. We can extend
partitionRels/computeRels to partition both RELATIVE and IRELATIVE
relocations, rendering `relaIplt` unneeded.

The change allows IRELATIVE relocations in the DT_ANDROID_REL[A] table
(untested?!), which may be processed before other types of relocations.
This seems acceptable for Bionic's DEFINE_IFUNC_FOR use cases.

In addition, this change simplies changing .rel[a].dyn to a compact
relocation format (CREL).

SHF_INFO_LINK is removed from .rel[a].dyn with IRELATIVE relocations.
(See https://reviews.llvm.org/D89828).
2024-03-24 14:07:09 -07:00
Fangrui Song
0e47dfede4 [ELF] Add isStaticRelSecType to simplify SHT_REL/SHT_RELA testing. NFC
and make it easier to introduce a new relocation format.

https://discourse.llvm.org/t/rfc-relleb-a-compact-relocation-format-for-elf/77600

Pull Request: https://github.com/llvm/llvm-project/pull/85893
2024-03-20 09:58:56 -07:00
Fangrui Song
8fe3e70e81 [ELF] Eliminate symbols demoted due to /DISCARD/ discarded sections (#85167)
#69295 demoted Defined symbols relative to discarded sections.
If such a symbol is unreferenced, the desired behavior is to
eliminate it from .symtab just like --gc-sections discarded
definitions.
Linux kernel's CONFIG_DEBUG_FORCE_WEAK_PER_CPU=y configuration expects
that the unreferenced `unused` is not emitted to .symtab
(https://github.com/ClangBuiltLinux/linux/issues/2006).

For relocations referencing demoted symbols, the symbol index restores
to 0 like older lld (`R_X86_64_64 0` in `discard-section.s`).

Fix #85048
2024-03-14 09:51:27 -07:00
DeanSturtevant1
335ac4108d Improve readability of "undefined reference" message (#82671)
The current message implies a command line flag caused an undefined
reference. This of course is wrong and causes confusion. The message now
more accurately reflects the true state of affairs.
2024-02-27 13:01:25 -05:00
Fangrui Song
78762357d4 [ELF] Support placing .lbss/.lrodata/.ldata after .bss
https://reviews.llvm.org/D150510 places .lrodata before .rodata to
minimize the number of permission transitions in the memory image.
However, this layout is less ideal for -fno-pic code (which is still
important).

Small code model -fno-pic code has R_X86_64_32S relocations with a range
of `[0,2**31)` (if we ignore the negative area). Placing `.lrodata`
earlier exerts relocation pressure on such code. Non-x86 64-bit
architectures generally have a similar `[0,2**31)` limitation if they
don't use PC-relative relocations.

If we place .lrodata later, we will need one extra PT_LOAD. Two layouts
are appealing:

* .bss/.lbss/.lrodata/.ldata (GNU ld)
* .bss/.ldata/.lbss/.lrodata

The GNU ld layout has the nice property that there is only one BSS
(except .tbss/.relro_padding). Add -z lrodata-after-bss to support
this layout.

Since a read-only PT_LOAD segment (for large data sections) may appear
after RW PT_LOAD segments. The placement of `_etext` has to be adjusted.

Pull Request: https://github.com/llvm/llvm-project/pull/81224
2024-02-20 13:59:49 -08:00
Fangrui Song
25cec33521 [ELF] Place _edata before .bss in the presence of .ldata
This minor issue is identified while working on #81224.
2024-02-12 18:14:19 -08:00
Fangrui Song
5f26b902d5 [ELF] Apply forgotten change to #81223 2024-02-09 12:09:42 -08:00
Fangrui Song
0329c1b6d8 [ELF] --no-rosegment: don't mark read-only PT_LOAD segments executable (#81223)
Once we move `.lrodata` after .bss (#78521), or if we use `SECTIONS`
commands, certain read-only sections may be in their own PT_LOAD, not in
the traditional "text segment". Current --no-rosegment code may
unnecessarily mark read-only PT_LOAD executable. Fix it.
2024-02-09 10:38:03 -08:00
Jinyang He
06a728f3fe [lld][ELF] Support relax R_LARCH_ALIGN (#78692)
Refer to commit 6611d58f5b ("Relax R_RISCV_ALIGN"), we can relax
R_LARCH_ALIGN by same way. Reuse `SymbolAnchor`, `RISCVRelaxAux` and
`initSymbolAnchors` to simplify codes. As `riscvFinalizeRelax` is an
arch-specific function, put it override on `TargetInfo::finalizeRelax`,
so that LoongArch can override it, too.

The flow of relax R_LARCH_ALIGN is almost consistent with RISCV. The
difference is that LoongArch only has 4-bytes NOP and all executable
insn is 4-bytes aligned. So LoongArch not need rewrite NOP sequence.
Alignment maxBytesEmit parameter is supported in psABI v2.30.
2024-02-06 09:09:13 +08:00
Fangrui Song
dee8786f70 [ELF] Fix compareSections assertion failure when OutputDescs in sectionCommands are non-contiguous
In a `--defsym y0=0 -T a.lds` link where a.lds contains only INSERT
commands, the `script->sectionCommands` layout may be:
```
orphan sections
SymbolAssignment due to --defsym
sections created by INSERT commands
```

The `OutputDesc` objects are not contiguous in sortInputSections, and
`compareSections` will be called with a SymbolAssignment argument,
leading to an assertion failure.
2024-02-01 21:20:27 -08:00
Fangrui Song
e390bda978 [ELF] Suppress --no-allow-shlib-undefined diagnostic when a SharedSymbol is overridden by a hidden visibility Defined which is later discarded
Commit 1981b1b6b9 unexpectedly strengthened
--no-allow-shlib-undefined to catch a kind of ODR violation.
More precisely, when all three conditions are met, the new
`--no-allow-shlib-undefined` code reports an error.

* There is a DSO undef that has been satisfied by a definition from
  another DSO.
* The `SharedSymbol` is overridden by a non-exported (usually of hidden
  visibility) definition in a relocatable object file (`Defined`).
* The section containing the `Defined` is garbage-collected (it is not
  part of `.dynsym` and is not marked as live).

Technically, the hidden Defined in the executable can be intentional: it
can be meant to remain non-exported and not interact with any dynamic
symbols of the same name that might exist in other DSOs. To allow for
such use cases, allocate a new bit in
Symbol and relax the --no-allow-shlib-undefined check to before
commit 1981b1b6b9.
2024-01-22 10:09:35 -08:00
Fangrui Song
43b13341fb [ELF] Add internal InputFile (#78944)
Based on https://reviews.llvm.org/D45375 . Introduce a new InputFile
kind `InternalKind`, use it for

* `ctx.internalFile`: for linker-defined symbols and some synthesized
`Undefined`
* `createInternalFile`: for symbol assignments and --defsym

I picked "internal" instead of "synthetic" to avoid confusion with
SyntheticSection.

Currently a symbol's file is one of: nullptr, ObjKind, SharedKind,
BitcodeKind, BinaryKind. Now it's non-null (I plan to add an
`assert(file)` to Symbol::Symbol and change `toString(const InputFile
*)`
separately).

Debugging and error reporting gets improved. The immediate user-facing
difference is more descriptive "File" column in the --cref output. This
patch may unlock further simplification.

Currently each symbol assignment gets its own
`createInternalFile(cmd->location)`. Two symbol assignments in a linker
script do not share the same file. Making the file the same would be
nice, but would require non trivial code.
2024-01-22 09:09:46 -08:00
Mitch Phillips
b399c84073 [NFC] [lld] [MTE] Rename MemtagDescriptors to MemtagGlobalDescriptors (#77300)
Requested in https://github.com/llvm/llvm-project/pull/77078, I agree
that we may as well be unambiguous.
2024-01-09 10:06:21 +01:00
Mitch Phillips
a831a21e4d [lld] [MTE] Allow android note for static executables. (#77078)
Florian pointed out that we're accidentally eliding the Android note for
static executables, as it's guarded behind the "can have memtag globals"
conditional. Of course, memtag globals are unsupported for static
executables, but we should still allow static binaries to produce the
Android note (as that's the only way they get MTE).
2024-01-08 11:22:38 +01:00
Fangrui Song
49168b2512 [ELF] Enhance --no-allow-shlib-undefined to report non-exported definition (#70769)
For a DSO with all DT_NEEDED entries accounted for, if it contains an
undefined non-weak symbol that shares a name with a non-exported
definition (hidden visibility or localized by a version script), and
there is no DSO definition, we should also report an error. Because the
definition is not exported, it cannot resolve the DSO reference at
runtime.

GNU ld introduced this error-checking in [April
2003](https://sourceware.org/pipermail/binutils/2003-April/026568.html).
The feature is available for executable links but not for -shared, and
it is
orthogonal to --no-allow-shlib-undefined. We make the feature part of
--no-allow-shlib-undefined and work with -shared when
--no-allow-shlib-undefined is specified.

A subset of this error-checking is covered by commit
1981b1b6b9 for --gc-sections discarded
sections. This patch covers non-discarded sections as well.

Internally, I have identified 2 bugs (which would fail with
LD_BIND_NOW=1) covered by commit
1981b1b6b9
2023-11-03 11:05:09 -07:00
Fangrui Song
ec0e556e67 [ELF] Merge copyLocalSymbols and demoteLocalSymbolsInDiscardedSections (#69425)
Follow-up to #69295: In `Writer<ELFT>::run`, the symbol passes are
flexible:
they can be placed almost everywhere before `scanRelocations`, with a
constraint
that the `computeIsPreemptible` pass must be invoked for linker-defined
non-local symbols.

Merge copyLocalSymbols and demoteLocalSymbolsInDiscardedSections to
simplify
code:

* Demoting local symbols can be made unconditional, not constrainted to
/DISCARD/ uses due to performance concerns
* `includeInSymtab` can be made faster
* Make symbol passes close to each other
* Decrease data cache misses due to saving an iteration over local
symbols

There is no speedup, likely due to the unconditional `dr->section`
access in `demoteAndCopyLocalSymbols`.

`gc-sections-tls.s` no longer reports an error because the TLS symbol is
converted to an Undefined.
2023-10-18 08:56:17 -07:00
Fangrui Song
1981b1b6b9 [ELF] Demote symbols in /DISCARD/ discarded sections to Undefined (#69295)
When an input section is matched by /DISCARD/ in a linker script, GNU ld
reports errors for relocations referencing symbols defined in the section:

    `.aaa' referenced in section `.bbb' of a.o: defined in discarded section `.aaa' of a.o

Implement the error by demoting eligible symbols to `Undefined` and changing
STB_WEAK to STB_GLOBAL. As a side benefit, in relocatable links, relocations
referencing symbols defined relative to /DISCARD/ discarded sections no longer
set symbol/type to zeros.

It's arguable whether a weak reference to a discarded symbol should lead to
errors. GNU ld reports an error and our demoting approach reports an error as
well.

Close #58891

Co-authored-by: Bevin Hansson <bevin.hansson@ericsson.com>
2023-10-17 14:10:52 -07:00
Fangrui Song
fc5d815d54 [ELF] Merge demoteSymbols and isPreemptible computation. NFC
Remove one iteration of symtab and slightly improve the performance.
2023-10-17 13:52:08 -07:00
Fangrui Song
e9b9a1d320 [ELF] Move demoteSymbols to Writer.cpp. NFC
History of demoteSharedSymbols:

* https://reviews.llvm.org/D45536 demotes SharedSymbol
* https://reviews.llvm.org/D111365 demotes lazy symbols
* The pending #69295 will demote symbols defined in discarded sections

The pass is placed after markLive just to be clear that it needs `isNeeded`
information computed by markLive. The remaining passes in Driver.cpp do not use
symbol information. Move the pass to Writer.cpp to be closer to other
symbol-related passes.
2023-10-17 13:16:50 -07:00
Mitch Phillips
144d127bef [lld] [MTE] Drop MTE globals for fully static executables, not ban (#68217)
Integrating MTE globals on Android revealed a lot of cases where
libraries are built as both archives and DSOs, and they're linked into
fully static and dynamic executables respectively.

MTE globals doesn't work for fully static executables. They need a
dynamic loader to process the special R_AARCH64_RELATIVE relocation
semantics with the encoded offset. Fully static executables that had
out-of-bounds derived symbols (like 'int* foo_end = foo[16]') crash
under MTE globals w/ static executables. So, LLD in its current form
simply errors out when you try and compile a fully static executable
that has a single MTE global variable in it.

It seems like a much better idea to simply have LLD not do the special
work for MTE globals in fully static contexts, and to drop any
unnecessary metadata. This means that you can build archives with MTE
globals and link them into both fully-static and dynamic executables.
2023-10-10 17:32:10 +02:00
Arthur Eubanks
9d6ec280fc [lld/ELF] Don't relax R_X86_64_(REX_)GOTPCRELX when offset is too far
For each R_X86_64_(REX_)GOTPCRELX relocation, check that the offset to the symbol is representable with 2^32 signed offset. If not, add a GOT entry for it and set its expr to R_GOT_PC so that we emit the GOT load instead of the relaxed lea. Do this in finalizeAddressDependentContent() where we iteratively attempt this (e.g. RISCV uses this for relaxation, ARM uses this to insert thunks).

Decided not to do the opposite of inserting GOT entries initially and removing them when relaxable because removing GOT entries isn't simple.

One drawback of this approach is that if we see any GOTPCRELX relocation, we'll create an empty .got even if it's not required in the end.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D157020
2023-10-04 13:03:56 -07:00
Fangrui Song
0de0b6dded [ELF] Postpone "unable to move location counter backward" error (#66854)
The size of .ARM.exidx may shrink across `assignAddress` calls. It is
possible
that the initial iteration has a larger location counter, causing
`__code_size =
__code_end - .; osec : { . += __code_size; }` to report an error, while
the error would
have been suppressed for subsequent `assignAddress` iterations.

Other sections like .relr.dyn may change sizes across `assignAddress`
calls as
well. However, their initial size is zero, so it is difficiult to
trigger a
similar error.

Similar to https://reviews.llvm.org/D152170, postpone the error
reporting.
Fix #66836. While here, add more information to the error message.
2023-09-20 09:06:45 -07:00
Fangrui Song
5a58e98c20 [ELF] Align the end of PT_GNU_RELRO associated PT_LOAD to a common-page-size boundary (#66042)
Close #57618: currently we align the end of PT_GNU_RELRO to a
common-page-size
boundary, but do not align the end of the associated PT_LOAD. This is
benign
when runtime_page_size >= common-page-size.

However, when runtime_page_size < common-page-size, it is possible that
`alignUp(end(PT_LOAD), page_size) < alignDown(end(PT_GNU_RELRO),
page_size)`.
In this case, rtld's mprotect call for PT_GNU_RELRO will apply to
unmapped
regions and lead to an error, e.g.

```
error while loading shared libraries: cannot apply additional memory protection after relocation: Cannot allocate memory
```

To fix the issue, add a padding section .relro_padding like mold, which
is contained in the PT_GNU_RELRO segment and the associated PT_LOAD
segment. The section also prevents strip from corrupting PT_LOAD program
headers.

.relro_padding has the largest `sortRank` among RELRO sections.
Therefore, it is naturally placed at the end of `PT_GNU_RELRO` segment
in the absence of `PHDRS`/`SECTIONS` commands.

In the presence of `SECTIONS` commands, we place .relro_padding
immediately before a symbol assignment using DATA_SEGMENT_RELRO_END (see
also https://reviews.llvm.org/D124656), if present.
DATA_SEGMENT_RELRO_END is changed to align to max-page-size instead of
common-page-size.

Some edge cases worth mentioning:

* ppc64-toc-addis-nop.s: when PHDRS is present, do not append
.relro_padding
* avoid-empty-program-headers.s: when the only RELRO section is .tbss,
it is not part of PT_LOAD segment, therefore we do not append
.relro_padding.

---

Close #65002: GNU ld from 2.39 onwards aligns the end of PT_GNU_RELRO to
a
max-page-size boundary (https://sourceware.org/PR28824) so that the last
page is
protected even if runtime_page_size > common-page-size.

In my opinion, losing protection for the last page when the runtime page
size is
larger than common-page-size is not really an issue. Double mapping a
page of up
to max-common-page for the protection could cause undesired VM waste.
Internally
we had users complaining about 2MiB max-page-size applying to shared
objects.

Therefore, the end of .relro_padding is padded to a common-page-size
boundary. Users who are really anxious can set common-page-size to match
their runtime page size.

---

17 tests need updating as there are lots of change detectors.
2023-09-14 10:33:11 -07:00
Fangrui Song
e057d8973c [ELF][PPC64] Use the regular placement for .branch_lt
The currently rule places .branch_lt after .data, which does not make
sense. The original contributor probably wanted to place .branch_lt
before .got/.toc, but failed to notice that .got/.toc are RELRO and
placed earlier.

Remove the special case so that .branch_lt is actually closer to .toc,
alleviating the distance issue.
2023-09-13 19:15:42 -07:00
Mitch Phillips
ca35a19aca [lld] Synthesize metadata for MTE globals
As per the ABI at
https://github.com/ARM-software/abi-aa/blob/main/memtagabielf64/memtagabielf64.rst,
this patch interprets the SHT_AARCH64_MEMTAG_GLOBALS_STATIC section,
which contains R_NONE relocations to tagged globals, and emits a
SHT_AARCH64_MEMTAG_GLOBALS_DYNAMIC section, with the correct
DT_AARCH64_MEMTAG_GLOBALS and DT_AARCH64_MEMTAG_GLOBALSSZ dynamic
entries. This section describes, in a uleb-encoded stream, global memory
ranges that should be tagged with MTE.

We are also out of bits to spare in the LLD Symbol class. As a result,
I've reused the 'needsTocRestore' bit, which is a PPC64 only feature.
Now, it's also used for 'isTagged' on AArch64.

An entry in SHT_AARCH64_MEMTAG_GLOBALS_STATIC is practically a guarantee
from an objfile that all references to the linked symbol are through the
GOT, and meet correct alignment requirements. As a result, we go through
all symbols and make sure that, for all symbols $SYM, all object files
that reference $SYM also have a SHT_AARCH64_MEMTAG_GLOBALS_STATIC entry
for $SYM. If this isn't the case, we demote the symbol to being
untagged. Symbols that are imported from other DSOs should always be
fine, as they're GOT-referenced (and thus the GOT entry either has the
correct tag or not, depending on whether it's tagged in the defining DSO
or not).

Additionally hand-tested by building {libc, libm, libc++, libm, and libnetd}
on Android with some experimental MTE globals support in the
linker/libc.

Reviewed By: MaskRay, peter.smith

Differential Revision: https://reviews.llvm.org/D152921
2023-07-31 17:07:42 +02:00
Fangrui Song
a290db3af1 [ELF] --build-id=fast: switch to xxh3_64bits 2023-07-19 11:37:42 -07:00
Amilendra Kodithuwakku
9acbab60e5 [LLD][ELF] Cortex-M Security Extensions (CMSE) Support
This commit provides linker support for Cortex-M Security Extensions (CMSE).
The specification for this feature can be found in ARM v8-M Security Extensions:
Requirements on Development Tools.

The linker synthesizes a security gateway veneer in a special section;
`.gnu.sgstubs`, when it finds non-local symbols `__acle_se_<entry>` and `<entry>`,
defined relative to the same text section and having the same address. The
address of `<entry>` is retargeted to the starting address of the
linker-synthesized security gateway veneer in section `.gnu.sgstubs`.

In summary, the linker translates input:

```
    .text
  entry:
  __acle_se_entry:
    [entry_code]

```
into:

```
    .section .gnu.sgstubs
  entry:
    SG
    B.W __acle_se_entry

    .text
  __acle_se_entry:
    [entry_code]
```

If addresses of `__acle_se_<entry>` and `<entry>` are not equal, the linker
considers that `<entry>` already defines a secure gateway veneer so does not
synthesize one.

If `--out-implib=<out.lib>` is specified, the linker writes the list of secure
gateway veneers into a CMSE import library `<out.lib>`. The CMSE import library
will have 3 sections: `.symtab`, `.strtab`, `.shstrtab`. For every secure gateway
veneer <entry> at address `<addr>`, `.symtab` contains a `SHN_ABS` symbol `<entry>` with
value `<addr>`.

If `--in-implib=<in.lib>` is specified, the linker reads the existing CMSE import
library `<in.lib>` and preserves the entry function addresses in the resulting
executable and new import library.

Reviewed By: MaskRay, peter.smith

Differential Revision: https://reviews.llvm.org/D139092
2023-07-06 11:34:07 +01:00
Simi Pallipurath
f146763e07 Revert "Revert "[lld][Arm] Big Endian - Byte invariant support.""
This reverts commit d8851384c6.

Reason: Applied the fix for the Asan buildbot failures.
2023-06-22 16:10:18 +01:00
Mitch Phillips
cd116e0460 Revert "Revert "Revert "[LLD][ELF] Cortex-M Security Extensions (CMSE) Support"""
This reverts commit 9246df7049.

Reason: This patch broke the UBSan buildbots. See more information in
the original phabricator review: https://reviews.llvm.org/D139092
2023-06-22 14:33:57 +02:00
Amilendra Kodithuwakku
9246df7049 Revert "Revert "[LLD][ELF] Cortex-M Security Extensions (CMSE) Support""
This reverts commit a685ddf1d1.

This relands Arm CMSE support (D139092) and fixes the GCC build bot errors.
2023-06-21 22:27:13 +01:00
Amilendra Kodithuwakku
a685ddf1d1 Revert "[LLD][ELF] Cortex-M Security Extensions (CMSE) Support"
This reverts commit c4fea39056.

I am reverting this for now until I figure out how to fix
the build bot errors and warnings.

Errors:
llvm-project/lld/ELF/Arch/ARM.cpp:1300:29: error: expected primary-expression before ‘>’ token
 osec->writeHeaderTo<ELFT>(++sHdrs);

Warnings:
llvm-project/lld/ELF/Arch/ARM.cpp:1306:31: warning: left operand of comma operator has no effect [-Wunused-value]
2023-06-21 16:13:44 +01:00
Amilendra Kodithuwakku
c4fea39056 [LLD][ELF] Cortex-M Security Extensions (CMSE) Support
This commit provides linker support for Cortex-M Security Extensions (CMSE).
The specification for this feature can be found in ARM v8-M Security Extensions:
Requirements on Development Tools.

The linker synthesizes a security gateway veneer in a special section;
`.gnu.sgstubs`, when it finds non-local symbols `__acle_se_<entry>` and `<entry>`,
defined relative to the same text section and having the same address. The
address of `<entry>` is retargeted to the starting address of the
linker-synthesized security gateway veneer in section `.gnu.sgstubs`.

In summary, the linker translates input:

```
    .text
  entry:
  __acle_se_entry:
    [entry_code]

```
into:

```
    .section .gnu.sgstubs
  entry:
    SG
    B.W __acle_se_entry

    .text
  __acle_se_entry:
    [entry_code]
```

If addresses of `__acle_se_<entry>` and `<entry>` are not equal, the linker
considers that `<entry>` already defines a secure gateway veneer so does not
synthesize one.

If `--out-implib=<out.lib>` is specified, the linker writes the list of secure
gateway veneers into a CMSE import library `<out.lib>`. The CMSE import library
will have 3 sections: `.symtab`, `.strtab`, `.shstrtab`. For every secure gateway
veneer <entry> at address `<addr>`, `.symtab` contains a `SHN_ABS` symbol `<entry>` with
value `<addr>`.

If `--in-implib=<in.lib>` is specified, the linker reads the existing CMSE import
library `<in.lib>` and preserves the entry function addresses in the resulting
executable and new import library.

Reviewed By: MaskRay, peter.smith

Differential Revision: https://reviews.llvm.org/D139092
2023-06-21 14:47:34 +01:00
Simi Pallipurath
d8851384c6 Revert "[lld][Arm] Big Endian - Byte invariant support."
This reverts commit 8cf8956897.
2023-06-20 17:27:44 +01:00
Simi Pallipurath
8cf8956897 [lld][Arm] Big Endian - Byte invariant support.
Arm has BE8 big endian configuration called a byte-invariant(every byte has the same address on little and big-endian systems).

When in BE8 mode:
  1. Instructions are big-endian in relocatable objects but
     little-endian in executables and shared objects.
  2. Data is big-endian.
  3. The data encoding of the ELF file is ELFDATA2MSB.

To support BE8 without an ABI break for relocatable objects,the linker takes on the responsibility of changing the endianness of instructions. At a high level the only difference between BE32 and BE8 in the linker is that for BE8:
  1. The linker sets the flag EF_ARM_BE8 in the ELF header.
  2. The linker endian reverses the instructions, but not data.

This patch adds BE8 big endian support for Arm. To endian reverse the instructions we'll need access to the mapping symbols. Code sections can contain a mix of Arm, Thumb and literal data. We need to endian reverse Arm instructions as words, Thumb instructions
as half-words and ignore literal data.The only way to find these transitions precisely is by using mapping symbols. The instruction reversal will need to take place after relocation. For Arm BE8 code sections (Section has SHF_EXECINSTR flag ) we inserted a step after relocation to endian reverse the instructions. The implementation strategy i have used here is to write all sections BE32  including SyntheticSections then endian reverse all code in InputSections via mapping symbols.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D150870
2023-06-20 14:08:21 +01:00
Andreu Carminati
e4118a7ac0 [ELF] Fix early overflow check in finalizeAddressDependentContent
LLD terminates with errors when it detects overflows in the
finalizeAddressDependentContent calculation. Although, sometimes, those errors
are not really errors, but an intermediate result of an ongoing address
calculation.  If we continue the fixed-point algorithm we can converge to the
correct result.

This patch

* Removes the verification inside the fixed point algorithm.
* Calls checkMemoryRegions at the end.

Reviewed By: peter.smith, MaskRay

Differential Revision: https://reviews.llvm.org/D152170
2023-06-14 15:26:31 -07:00
Fangrui Song
698ac4aba5 [ELF] Add PT_RISCV_ATTRIBUTES program header
Close https://github.com/llvm/llvm-project/issues/63084

Unlike AArch32, RISC-V defines PT_RISCV_ATTRIBUTES to include the
SHT_RISCV_ATTRIBUTES section. There is no real-world use case yet.

We place PT_RISCV_ATTRIBUTES after PT_GNU_STACK, similar to PT_ARM_EXIDX. GNU ld
places PT_RISCV_ATTRIBUTES earlier, but the placement should not matter.

Link: https://github.com/riscv-non-isa/riscv-elf-psabi-doc/pull/71

Reviewed By: asb

Differential Revision: https://reviews.llvm.org/D152065
2023-06-06 13:06:21 -07:00
Fangrui Song
8d85c96e0e [lld] StringRef::{starts,ends}with => {starts,ends}_with. NFC
The latter form is now preferred to be similar to C++20 starts_with.
This replacement also removes one function call when startswith is not inlined.
2023-06-05 14:36:19 -07:00
Fangrui Song
8aea109504 [ELF] x86-64: place .lrodata, .lbss, and .ldata away from code sections
The x86-64 medium code model utilizes large data sections, namely .lrodata,
.lbss, and .ldata (along with some variants of .ldata). There is a proposal to
extend the use of large data sections to the large code model as well[1].

This patch aims to place large data sections away from code sections in order to
alleviate relocation overflow pressure caused by code sections referencing
regular data sections.

```
.lrodata
.rodata
.text     # if --ro-segment, MAXPAGESIZE alignment
RELRO     # MAXPAGESIZE alignment
.data     # MAXPAGESIZE alignment
.bss
.ldata    # MAXPAGESIZE alignment
.lbss
```

In comparison to GNU ld, which places .lbss, .lrodata, and .ldata after .bss, we
place .lrodata above .rodata to minimize the number of permission transitions in
the memory image.

While GNU ld places .lbss after .bss, the subsequent sections don't reuse the
file offset bytes of BSS.

Our approach is to place .ldata and .lbss after .bss and create a PT_LOAD
segment for .bss to large data section transition in the absence of SECTIONS
commands. assignFileOffsets ensures we insert an alignment instead of allocating
space for BSS, and therefore we don't waste more than MAXPAGESIZE bytes. We have
a missing optimization to prevent all waste, but implementing it would introduce
complexity and likely be error-prone.

GNU ld's layout introduces 2 more MAXPAGESIZE alignments while ours
introduces just one.

[1]: https://groups.google.com/g/x86-64-abi/c/jnQdJeabxiU "Large data sections for the large code model"

With help from Arthur Eubanks.

Co-authored-by: James Y Knight <jyknight@google.com>

Reviewed By: aeubanks, tkoeppe

Differential Revision: https://reviews.llvm.org/D150510
2023-05-25 07:35:38 -07:00
Petr Hosek
811cbfc262 [lld][ELF] Implement –print-memory-usage
This option was introduced in GNU ld in
https://sourceware.org/legacy-ml/binutils/2015-06/msg00086.html and is
often used in embedded development. This change implements this option
in LLD matching the GNU ld output verbatim.

Differential Revision: https://reviews.llvm.org/D150644
2023-05-25 07:14:18 +00:00
Fangrui Song
2473b1af08 [ELF] Simplify getSectionRank and rewrite comments
Replace some RF_ flags with integer literals.
Rewrite the isWrite/isExec block to make the code block order reflect
the section order.
Rewrite some imprecise comments.

This is NFC, if we don't count invalid cases such as non-writable TLS
and non-writable RELRO.
2023-05-12 23:58:39 -07:00
Fangrui Song
a2648bc4ea [ELF] Remove remnant ranks for PPC64 ELFv1 special sections 2023-05-12 23:21:14 -07:00