Commit Graph

147 Commits

Author SHA1 Message Date
Jez Ng
681cfeb591 [lld-macho][nfc] Have InputSection ctors take some parameters
This is motivated by an upcoming diff in which the
WordLiteralInputSection ctor sets itself up based on the value of its
section flags. As such, it needs to be passed the `flags` value as part
of its ctor parameters, instead of having them assigned after the fact
in `parseSection()`. While refactoring code to make that possible, I
figured it would make sense for the other InputSections to also take
their initial values as ctor parameters.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D103978
2021-06-11 19:50:09 -04:00
Jez Ng
5d88f2dd94 [lld-macho] Deduplicate fixed-width literals
Conceptually, the implementation is pretty straightforward: we put each
literal value into a hashtable, and then write out the keys of that
hashtable at the end.

In contrast with ELF, the Mach-O format does not support variable-length
literals that aren't strings. Its literals are either 4, 8, or 16 bytes
in length. LLD-ELF dedups its literals via sorting + uniq'ing, but since
we don't need to worry about overly-long values, we should be able to do
a faster job by just hashing.

That said, the implementation right now is far from optimal, because we
add to those hashtables serially. To parallelize this, we'll need a
basic concurrent hashtable (only needs to support concurrent writes w/o
interleave reads), which shouldn't be to hard to implement, but I'd like
to punt on it for now.

Numbers for linking chromium_framework on my 3.2 GHz 16-Core Intel Xeon W:

      N           Min           Max        Median           Avg        Stddev
  x  20          4.27          4.39         4.315        4.3225   0.033225703
  +  20          4.36          4.82          4.44        4.4845    0.13152846
  Difference at 95.0% confidence
          0.162 +/- 0.0613971
          3.74783% +/- 1.42041%
          (Student's t, pooled s = 0.0959262)

This corresponds to binary size savings of 2MB out of 335MB, or 0.6%.
It's not a great tradeoff as-is, but as mentioned our implementation can
be signficantly optimized, and literal dedup will unlock more
opportunities for ICF to identify identical structures that reference
the same literals.

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D103113
2021-06-11 19:50:08 -04:00
Nico Weber
0e399eb527 [lld/mac] When handling @loader_path, use realpath() of symlinks
This is important for Frameworks, which are usually symlinks.

ld64 gets this right for @rpath that's replaced with @loader_path, but not for
bare @loader_path -- ld64's code calls realpath() in that case too, but ignores
the result.

ld64 somehow manages to find libbar1.dylib in the test without the
explicit `-rpath` in Foo1. I don't understand why or how. But this
change is a step forward and fixes an immediate problem I'm having,
so let's start with this :)

Differential Revision: https://reviews.llvm.org/D103990
2021-06-09 20:36:07 -04:00
Jez Ng
04259cde15 [lld-macho] Implement cstring deduplication
Our implementation draws heavily from LLD-ELF's, which in turn delegates
its string deduplication to llvm-mc's StringTableBuilder. The messiness of
this diff is largely due to the fact that we've previously assumed that
all InputSections get concatenated together to form the output. This is
no longer true with CStringInputSections, which split their contents into
StringPieces. StringPieces are much more lightweight than InputSections,
which is important as we create a lot of them. They may also overlap in
the output, which makes it possible for strings to be tail-merged. In
fact, the initial version of this diff implemented tail merging, but
I've dropped it for reasons I'll explain later.

**Alignment Issues**

Mergeable cstring literals are found under the `__TEXT,__cstring`
section. In contrast to ELF, which puts strings that need different
alignments into different sections, clang's Mach-O backend puts them all
in one section. Strings that need to be aligned have the `.p2align`
directive emitted before them, which simply translates into zero padding
in the object file.

I *think* ld64 extracts the desired per-string alignment from this data
by preserving each string's offset from the last section-aligned
address. I'm not entirely certain since it doesn't seem consistent about
doing this; but perhaps this can be chalked up to cases where ld64 has
to deduplicate strings with different offset/alignment combos -- it
seems to pick one of their alignments to preserve. This doesn't seem
correct in general; we can in fact can induce ld64 to produce a crashing
binary just by linking in an additional object file that only contains
cstrings and no code. See PR50563 for details.

Moreover, this scheme seems rather inefficient: since unaligned and
aligned strings are all put in the same section, which has a single
alignment value, it doesn't seem possible to tell whether a given string
doesn't have any alignment requirements. Preserving offset+alignments
for strings that don't need it is wasteful.

In practice, the crashes seen so far seem to stem from x86_64 SIMD
operations on cstrings. X86_64 requires SIMD accesses to be
16-byte-aligned. So for now, I'm thinking of just aligning all strings
to 16 bytes on x86_64. This is indeed wasteful, but implementation-wise
it's simpler than preserving per-string alignment+offsets. It also
avoids the aforementioned crash after deduplication of
differently-aligned strings. Finally, the overhead is not huge: using
16-byte alignment (vs no alignment) is only a 0.5% size overhead when
linking chromium_framework.

With these alignment requirements, it doesn't make sense to attempt tail
merging -- most strings will not be eligible since their overlaps aren't
likely to start at a 16-byte boundary. Tail-merging (with alignment) for
chromium_framework only improves size by 0.3%.

It's worth noting that LLD-ELF only does tail merging at `-O2`. By
default (at `-O1`), it just deduplicates w/o tail merging. @thakis has
also mentioned that they saw it regress compressed size in some cases
and therefore turned it off. `ld64` does not seem to do tail merging at
all.

**Performance Numbers**

CString deduplication reduces chromium_framework from 250MB to 242MB, or
about a 3.2% reduction.

Numbers for linking chromium_framework on my 3.2 GHz 16-Core Intel Xeon W:

      N           Min           Max        Median           Avg        Stddev
  x  20          3.91          4.03         3.935          3.95   0.034641016
  +  20          3.99          4.14         4.015        4.0365     0.0492336
  Difference at 95.0% confidence
          0.0865 +/- 0.027245
          2.18987% +/- 0.689746%
          (Student's t, pooled s = 0.0425673)

As expected, cstring merging incurs some non-trivial overhead.

When passing `--no-literal-merge`, it seems that performance is the
same, i.e. the refactoring in this diff didn't cost us.

      N           Min           Max        Median           Avg        Stddev
  x  20          3.91          4.03         3.935          3.95   0.034641016
  +  20          3.89          4.02         3.935        3.9435   0.043197831
  No difference proven at 95.0% confidence

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D102964
2021-06-07 23:48:35 -04:00
Nico Weber
17c43c4045 [lld/mac] Add reexports after reexporter to inputFiles
When a library "host"'s reexports change their installName with
`$ld$os10.11$install_name$host`, we used to write a load command for "host" but
write the version numbers of the reexport instead of "host". This fixes that.

I first thought that the rule is to take the version numbers from the library
that originally had that install name (implemented in D103819), but that's not
what ld64 seems to be doing: It takes the version number from the first dylib
with that install name it loads, and it loads the reexporting library before
the reexports. We already did most of that, we just added reexports before the
reexporter. After this change, we add the reexporter before the reexports.

Addresses https://bugs.llvm.org/show_bug.cgi?id=49800#c11 part 1.

(ld64 seems to add reexports after processing _all_ files on the command line,
while we add them right after the reexporter. For the common case of reexport +
$ld$ symbol changing back to the exporter name, this doesn't make a difference,
but you can construct a case where it does. I expect this to not make a
difference in practice though.)

Differential Revision: https://reviews.llvm.org/D103821
2021-06-07 17:04:03 -04:00
Nico Weber
c5ffe97988 [lld/mac] Implement support for searching dylibs with @rpath/ in install name
Also adjust a few comments, and move the DylibFile comment talking about
umbrella next to the parameter again.

Differential Revision: https://reviews.llvm.org/D103783
2021-06-07 06:22:52 -04:00
Nico Weber
52489021cf [lld/mac] Implement support for searching dylibs with @loader_path/ in install name
Differential Revision: https://reviews.llvm.org/D103779
2021-06-06 20:19:50 -04:00
Nico Weber
a48bd587f7 [lld/mac] Implement support for searching dylibs with @executable_path/ in install name
Differential Revision: https://reviews.llvm.org/D103775
2021-06-06 20:01:50 -04:00
Nico Weber
7def700667 [lld/mac] Rename DylibFile::dylibName to DylibFile::installName
The flag to set it is called `-install_name`, and it's called `installName` in tbd files.

No behavior change.

Differential Revision: https://reviews.llvm.org/D103776
2021-06-06 20:00:35 -04:00
Nico Weber
e910437443 [lld/mac] Use fewer magic numbers in magic $ld$ handling code
Also simply a conditional and de-alias a variable.
Minor cleanups, no behavior change.

Differential Revision: https://reviews.llvm.org/D103774
2021-06-06 18:13:16 -04:00
Alexander Shaposhnikov
5e49ee8794 [lld][MachO] Add support for $ld$install_name symbols
This diff adds support for $ld$install_name symbols.

Test plan: make check-lld-macho

Differential revision: https://reviews.llvm.org/D103746
2021-06-05 12:58:59 -07:00
Alexander Shaposhnikov
1309c181a8 [lld][MachO] Add first bits to support special symbols
This diff adds first bits to support special symbols $ld$previous* in LLD.
$ld$* symbols modify properties/behavior of the library
(e.g. its install name, compatibility version or hide/add symbols)
for specific target versions.

Test plan: make check-lld-macho

Differential revision: https://reviews.llvm.org/D103505
2021-06-04 23:32:26 -07:00
Jez Ng
6881f29a36 [lld-macho] Parse re-exports of nested TAPI documents
D103423 neglected to call `parseReexports()` for nested TBD
documents, leading to symbol resolution failures when trying to look up
a symbol nested more than one level deep in a TBD file. This fixes the
regression and adds a test.

It also appears that `umbrella` wasn't being set properly when calling
`parseLoadCommands` -- it's supposed to resolve to `this` if `nullptr`
is passed. I didn't write a failing test case for this but I've made
`umbrella` a member so the previous behavior should be preserved.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D103586
2021-06-03 12:02:30 -04:00
Nico Weber
a5645513db [lld/mac] Implement -dead_strip
Also adds support for live_support sections, no_dead_strip sections,
.no_dead_strip symbols.

Chromium Framework 345MB unstripped -> 250MB stripped
(vs 290MB unstripped -> 236M stripped with ld64).

Doing dead stripping is a bit faster than not, because so much less
data needs to be processed:

    % ministat lld_*
    x lld_nostrip.txt
    + lld_strip.txt
        N           Min           Max        Median           Avg        Stddev
    x  10      3.929414       4.07692     4.0269079     4.0089678   0.044214794
    +  10     3.8129408     3.9025559     3.8670411     3.8642573   0.024779651
    Difference at 95.0% confidence
            -0.144711 +/- 0.0336749
            -3.60967% +/- 0.839989%
            (Student's t, pooled s = 0.0358398)

This interacts with many parts of the linker. I tried to add test coverage
for all added `isLive()` checks, so that some test will fail if any of them
is removed. I checked that the test expectations for the most part match
ld64's behavior (except for live-support-iterations.s, see the comment
in the test). Interacts with:
- debug info
- export tries
- import opcodes
- flags like -exported_symbol(s_list)
- -U / dynamic_lookup
- mod_init_funcs, mod_term_funcs
- weak symbol handling
- unwind info
- stubs
- map files
- -sectcreate
- undefined, dylib, common, defined (both absolute and normal) symbols

It's possible it interacts with more features I didn't think of,
of course.

I also did some manual testing:
- check-llvm check-clang check-lld work with lld with this patch
  as host linker and -dead_strip enabled
- Chromium still starts
- Chromium's base_unittests still pass, including unwind tests

Implemenation-wise, this is InputSection-based, so it'll work for
object files with .subsections_via_symbols (which includes all
object files generated by clang). I first based this on the COFF
implementation, but later realized that things are more similar to ELF.
I think it'd be good to refactor MarkLive.cpp to look more like the ELF
part at some point, but I'd like to get a working state checked in first.

Mechanical parts:
- Rename canOmitFromOutput to wasCoalesced (no behavior change)
  since it really is for weak coalesced symbols
- Add noDeadStrip to Defined, corresponding to N_NO_DEAD_STRIP
  (`.no_dead_strip` in asm)

Fixes PR49276.

Differential Revision: https://reviews.llvm.org/D103324
2021-06-02 11:09:26 -04:00
Nico Weber
476e4d65d4 [lld/mac] Address review feedback and improve a comment
I forgot to move the message() call around as requested in D103428
before committing that change. Move it now.

Also, improve the ordinal uniq'ing comment. I hadn't realized that the
distinct-but-identical files happen with --reproduce and not in general.

No behavior change.

Differential Revision: https://reviews.llvm.org/D103522
2021-06-02 10:54:53 -04:00
Vy Nguyen
8f89c054af [lld-macho][nfc] Remove unnecessary use of Optional<T*>
In all of these cases, the functions could simply return a nullptr instead of {}.
There is no case where Optional<nullptr> has a special meaning.

Differential Revision: https://reviews.llvm.org/D103489
2021-06-01 18:35:31 -04:00
Nico Weber
2c1903412b [lld/mac] Implement removal of unused dylibs
This omits load commands for unreferenced dylibs if:
- the dylib was loaded implicitly,
- it is marked MH_DEAD_STRIPPABLE_DYLIB
- or -dead_strip_dylibs is passed

This matches ld64.

Currently, the "is dylib referenced" state is computed before dead code
stripping and is not updated after dead code stripping. This too matches ld64.
We should do better here.

With this, clang-format linked with lld (like with ld64) no longer has
libobjc.A.dylib in `otool -L` output. (It was implicitly loaded as a reexport
of CoreFoundation.framework, but it's not needed.)

Differential Revision: https://reviews.llvm.org/D103430
2021-06-01 16:06:30 -04:00
Nico Weber
24979e1113 [lld/mac] Don't load DylibFiles from the DylibFile constructor
loadDylib() keeps a name->DylibFile cache, but it only writes
to the cache once the DylibFile constructor has completed.
So dylib loads done recursively from the DylibFile constructor
wouldn't use the cache.

Now, we load additional dylibs after writing to the cache,
which means the cache now gets used for dylibs loaded because
they're referenced from other dylibs.

Related to PR49514 and PR50101, but no dramatic behavior change in itself.
(Technically we no longer crash when a tbd file reexports itself,
but that doesn't happen in practice. We now accept it silently instead
of crashing; ld64 has a diag for the reexport cycle.)

Differential Revision: https://reviews.llvm.org/D103423
2021-06-01 15:31:02 -04:00
Jez Ng
8535834ef7 [lld-macho][nfc] Misc code cleanup
* Move `static_asserts` into cpp instead of header file. I noticed they
  had been separated from the main class definition in the header, so I
  set about to clean that up, then figured it made more sense as part of
  the cpp file so as not to incur unnecessary compile-time overhead.

* Remove unnecessary `virtual`s

* Remove unnecessary comment / reword another comment
2021-05-25 14:58:29 -04:00
Alexander Shaposhnikov
57501e512e [lld][MachO] Fix code formatting
Apply clang-format -style=llvm to InputFile.cpp. NFC.

Test plan: make check-all
2021-05-23 20:35:55 -07:00
Nico Weber
4a12248ee2 [lld/mac] Honor REFERENCED_DYAMICALLY, set it on __mh_execute_header
Has the effect that `__mh_execute_header` stays in the symbol table of
outputs even after running `strip` on the output. I don't know if that's
important for anything -- my motivation for the patch is just is to make
the output more similar to ld64.

(Corresponds to symbolTableInAndNeverStrip in ld64.)

Differential Revision: https://reviews.llvm.org/D102619
2021-05-17 14:22:12 -04:00
Jez Ng
b1c3c2e4fc [lld-macho] Fix order file arch filtering
We had a hardcoded check and a stale TODO, written back when we only had
support for one architecture.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D102154
2021-05-10 15:45:54 -04:00
Nico Weber
7f673fcaa9 [lld/mac] Fix alignment on subsections
On a section with alignment of 16, subsections aligned to 16-byte
boundaries should keep their 16-byte alignment.

Fixes PR50274. (The same bug could have happened with -order_file
previously.)

Differential Revision: https://reviews.llvm.org/D102139
2021-05-09 21:00:56 -04:00
Nico Weber
d5a70db193 [lld/mac] Write every weak symbol only once in the output
Before this, if an inline function was defined in several input files,
lld would write each copy of the inline function the output. With this
patch, it only writes one copy.

Reduces the size of Chromium Framework from 378MB to 345MB (compared
to 290MB linked with ld64, which also does dead-stripping, which we
don't do yet), and makes linking it faster:

        N           Min           Max        Median           Avg        Stddev
    x  10     3.9957051     4.3496981     4.1411121      4.156837    0.10092097
    +  10      3.908154      4.169318     3.9712729     3.9846753   0.075773012
    Difference at 95.0% confidence
            -0.172162 +/- 0.083847
            -4.14165% +/- 2.01709%
            (Student's t, pooled s = 0.0892373)

Implementation-wise, when merging two weak symbols, this sets a
"canOmitFromOutput" on the InputSection belonging to the weak symbol not put in
the symbol table. We then don't write InputSections that have this set, as long
as they are not referenced from other symbols. (This happens e.g. for object
files that don't set .subsections_via_symbols or that use .alt_entry.)

Some restrictions:
- not yet done for bitcode inputs
- no "comdat" handling (`kindNoneGroupSubordinate*` in ld64) --
  Frame Descriptor Entries (FDEs), Language Specific Data Areas (LSDAs)
  (that is, catch block unwind information) and Personality Routines
  associated with weak functions still not stripped. This is wasteful,
  but harmless.
- However, this does strip weaks from __unwind_info (which is needed for
  correctness and not just for size)
- This nopes out on InputSections that are referenced form more than
  one symbol (eg from .alt_entry) for now

Things that work based on symbols Just Work:
- map files (change in MapFile.cpp is no-op and not needed; I just
  found it a bit more explicit)
- exports

Things that work with inputSections need to explicitly check if
an inputSection is written (e.g. unwind info).

This patch is useful in itself, but it's also likely also a useful foundation
for dead_strip.

I used to have a "canoncialRepresentative" pointer on InputSection instead of
just the bool, which would be handy for ICF too. But I ended up not needing it
for this patch, so I removed that again for now.

Differential Revision: https://reviews.llvm.org/D102076
2021-05-07 17:11:40 -04:00
Jez Ng
9260760235 [lld-macho] Support loading of zippered dylibs
ld64 can emit dylibs that support more than one platform (typically macOS and
macCatalyst). This diff allows LLD to read in those dylibs. Note that this is a
super bare-bones implementation -- in particular, I haven't added support for
LLD to emit those multi-platform dylibs, nor have I added a variety of
validation checks that ld64 does. Until we have a use-case for emitting zippered
dylibs, I think this is good enough.

Fixes PR49597.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D101954
2021-05-06 11:19:40 -04:00
Vy Nguyen
23233ad139 [lld-macho] Check simulator platforms to avoid issuing false positive errors.
Currently the linker causes unnecessary errors when either the target or the config's platform is a simulator.

Differential Revision: https://reviews.llvm.org/D101855
2021-05-05 18:07:58 -04:00
Jez Ng
001ba65375 [lld-macho] De-templatize mach_header operations
@thakis pointed out that `mach_header` and `mach_header_64`
actually have the same set of (used) fields, with the 64-bit version
having extra padding. So we can access the fields we need using the
single `mach_header` type instead of using templates to switch between
the two.

I also spotted a potential issue where hasObjCSection tries to parse a
file w/o checking if it does indeed match the target arch... As such,
I've added a quick magic number check to ensure we don't access invalid
memory during `findCommand()`.

Addresses PR50180.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D101724
2021-05-03 18:31:23 -04:00
Jez Ng
05c5363b39 [lld-macho] Parse & emit the N_ARM_THUMB_DEF symbol flag
Eventually we'll use this flag to properly handle bl/blx
opcodes.

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D101558
2021-04-30 16:17:26 -04:00
Nico Weber
4b456038e4 [lld/mac] Tweak two comments and fix style on one variable name
Cosmetic, no behavior change.
2021-04-30 09:30:51 -04:00
Nico Weber
a38ebed258 [lld/mac] Implement support for .weak_def_can_be_hidden
I first had a more invasive patch for this (D101069), but while trying
to get that polished for review I realized that lld's current symbol
merging semantics mean that only a very small code change is needed.
So this goes with the smaller patch for now.

This has no effect on projects that build with -fvisibility=hidden
(e.g.  chromium), since these see .private_extern symbols instead.

It does have an effect on projects that build with -fvisibility-inlines-hidden
(e.g. llvm) in -O2 builds, where LLVM's GlobalOpt pass will promote most inline
functions from .weak_definition to .weak_def_can_be_hidden.

Before this patch:

    % ls -l out/gn/bin/clang out/gn/lib/libclang.dylib
    -rwxr-xr-x  1 thakis  staff  113059936 Apr 22 11:51 out/gn/bin/clang
    -rwxr-xr-x  1 thakis  staff   86370064 Apr 22 11:51 out/gn/lib/libclang.dylib
    % out/gn/bin/llvm-objdump --macho --weak-bind out/gn/bin/clang | wc -l
        8291
    % out/gn/bin/llvm-objdump --macho --weak-bind out/gn/lib/libclang.dylib | wc -l
        5698

With this patch:

    % ls -l out/gn/bin/clang out/gn/lib/libclang.dylib
    -rwxr-xr-x  1 thakis  staff  111721096 Apr 22 11:55 out/gn/bin/clang
    -rwxr-xr-x  1 thakis  staff   85291208 Apr 22 11:55 out/gn/lib/libclang.dylib
    thakis@MBP llvm-project % out/gn/bin/llvm-objdump --macho --weak-bind out/gn/bin/clang | wc -l
         725
    thakis@MBP llvm-project % out/gn/bin/llvm-objdump --macho --weak-bind out/gn/lib/libclang.dylib | wc -l
         542

Linking clang becomes a tiny bit faster with this patch:

    x 100    0.67263818    0.77847815    0.69430709    0.69877208   0.017715892
    + 100    0.67209601    0.73323393    0.68600798    0.68917346   0.012824377
    Difference at 95.0% confidence
            -0.00959861 +/- 0.00428661
            -1.37364% +/- 0.613449%
            (Student's t, pooled s = 0.0154648)

This only happens if lld with the patch and lld without the patch are both
linked with an lld with the patch or both linked with an lld without the patch
(...or with ld64). I accidentally linked the lld with the patch with an lld
without the patch and the other way round at first. In that setup, no
difference is found. That makese sense, since having fewer weak imports will
make the linked output a bit faster too. So not only does this make linking
binaries such as clang a bit faster (since fewer exports need to be written to
the export trie by lld), the linked output binary binary is also a bit faster
(since dyld needs to process fewer dynamic imports).

This also happens to fix the one `check-clang` failure when using lld as host
linker, but mostly for silly reasons: See crbug.com/1183336, mostly comment 26.
The real bug here is that c-index-test links all of LLVM both statically and
dynamically, which is an ODR violation. Things just happen to work with this
patch.

So after this patch, check-clang, check-lld, check-llvm all pass with lld as
host linker :)

Differential Revision: https://reviews.llvm.org/D101080
2021-04-22 22:51:34 -04:00
Jez Ng
8c17a87515 [re-land][lld-macho] Fix min version check
We had got it backwards... the minimum version of the target
should be higher than the min version of the object files, presumably
since new platforms are backwards-compatible with older formats.

Fixes PR50078.

The original commit (aa05439c9c) broke many tests that had inputs too
new for our target platform (10.0). This commit changes the inputs to
target 10.0, which was the simpler thing to do, but we should really
just have our lit.local.cfg default to targeting 10.15... we're not
likely to ever have proper support for the older versions anyway. I will
follow up with a change to that effect.

Differential Revision: https://reviews.llvm.org/D101114
2021-04-22 19:35:32 -04:00
Jez Ng
75ecb804b1 Revert "[lld-macho] Fix min version check"
This reverts commit aa05439c9c.
2021-04-22 19:07:41 -04:00
Jez Ng
aa05439c9c [lld-macho] Fix min version check
We had got it backwards... the minimum version of the target
should be higher than the min version of the object files, presumably
since new platforms are backwards-compatible with older formats.

Fixes PR50078.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D101114
2021-04-22 18:25:44 -04:00
Jez Ng
ed4a4e3312 [lld-macho][nfc] Add accessors for commonly-used PlatformInfo fields
As discussed here: https://reviews.llvm.org/D100523#inline-951543

Reviewed By: #lld-macho, thakis, alexshap

Differential Revision: https://reviews.llvm.org/D100978
2021-04-21 15:43:56 -04:00
Alexander Shaposhnikov
b5720354ef [lld][MachO] Refactor findCommand
Refactor findCommand to allow passing multiple types. NFC.

Test plan: make check-lld-macho

Differential revision: https://reviews.llvm.org/D100954
2021-04-21 08:38:17 -07:00
Alexander Shaposhnikov
5c835e1ae5 [lld][MachO] Add support for LC_VERSION_MIN_* load commands
This diff adds initial support for the legacy LC_VERSION_MIN_* load commands.

Test plan: make check-lld-macho

Differential revision: https://reviews.llvm.org/D100523
2021-04-21 05:41:14 -07:00
Jez Ng
7208bd4b32 [lld-macho] Skip platform checks for a few libSystem re-exports
XCode 12 ships with mismatched platforms for these libraries,
so this hack is necessary...

Fixes PR49799.

Reviewed By: #lld-macho, gkm, smeenai

Differential Revision: https://reviews.llvm.org/D100913
2021-04-20 19:54:53 -04:00
Jez Ng
1aa29dffce [lld-macho] Support subtractor relocations that reference sections
The minuend (but not the subtrahend) can reference a section.

Note that we do not yet properly validate that the subtrahend isn't
referencing a section; I've filed PR50034 to track that.

I've also extended the reloc-subtractor.s test to reorder symbols, to
make sure that the addends are being associated with the minuend (and not
the subtrahend) relocation.

Fixes PR49999.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D100804
2021-04-20 16:58:57 -04:00
Jez Ng
3142fc3b5b [lld-macho] Have toString() emit full path to archive files
It doesn't make sense to take just the base filename for archives when we emit
the full path for object files. (LLD-ELF emits the full path too.)

This will also make it easier to write a proper test for {D100147}.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D100357
2021-04-13 10:43:28 -04:00
Jez Ng
2461804b48 [lld-macho] Symbol::value should always be uint64_t
D98837 migrated a bunch of `value`s to uint64_t, but missed these.
2021-04-06 17:54:11 -04:00
Jez Ng
ceec610754 [lld-macho] Fix & refactor symbol size calculations
I noticed two problems with the previous implementation:

* N_ALT_ENTRY symbols weren't being handled correctly -- they should
  determine the size of the previous symbol, even though they don't
  cause a new section to be created
* The last symbol in a section had its size calculated wrongly;
  the first subsection's size was used instead of the last one

I decided to take the opportunity to refactor things as well, mainly to
realize my observation
[here](https://reviews.llvm.org/D98837#inline-931511) that we could
avoid doing a binary search to match symbols with subsections. I think
the resulting code is a bit simpler too.

      N           Min           Max        Median           Avg        Stddev
  x  20          4.31          4.43          4.37        4.3775   0.034162922
  +  20          4.32          4.43          4.38        4.3755    0.02799906
  No difference proven at 95.0% confidence

Reviewed By: #lld-macho, alexshap

Differential Revision: https://reviews.llvm.org/D99972
2021-04-06 15:10:01 -04:00
Jez Ng
e0df2b540a [lld-macho] Rename SubsectionMapping to SubsectionMap
We bikeshedded about it here: https://reviews.llvm.org/D98837#inline-931557

I initially suggested SubsectionMapping, but I thought the discussion
landed on doing `std::vector<SubsectionEntry>`. @alexshap went and did
both, but on hindsight I regret adding 3 more characters to an already
long name, and I think SubsectionEntry is descriptive enough...

This diff also renames `subsectionMap` to `subsecMap` for consistency
with other variable names in the codebase.
2021-04-06 14:26:13 -04:00
Cyndy Ishida
0116d04d04 [TextAPI] move source code files out of subdirectory, NFC
TextAPI/ELF has moved out into InterfaceStubs, so theres no longer a
need to seperate out TextAPI between formats.

Reviewed By: ributzka, int3, #lld-macho

Differential Revision: https://reviews.llvm.org/D99811
2021-04-05 10:24:42 -07:00
Jez Ng
817d98d841 [lld-macho][nfc] Refactor in preparation for 32-bit support
The main challenge was handling the different on-disk structures (e.g.
`mach_header` vs `mach_header_64`). I tried to strike a balance between
sprinkling `target->wordSize == 8` checks everywhere (branchy = slow, and ugly)
and templatizing everything (causes code bloat, also ugly). I think I struck a
decent balance by judicious use of type erasure.

Note that LLD-ELF has a similar architecture, though it seems to use more templating.

Linking chromium_framework takes about the same time before and after this
change:

      N           Min           Max        Median           Avg        Stddev
  x  20          4.52          4.67         4.595        4.5945   0.044423204
  +  20           4.5          4.71         4.575         4.582   0.056344803
  No difference proven at 95.0% confidence

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D99633
2021-04-02 18:46:39 -04:00
Yang Fan
d441dee5c2 [lld][MachO] Fix -Wsign-compare warning (NFC)
GCC warning:
```
/llvm-project/lld/MachO/InputFiles.cpp:484:24: warning: comparison of integer expressions of different signedness: ‘int64_t’ {aka ‘long int’} and ‘uint64_t’ {aka ‘long unsigned int’} [-Wsign-compare]
484 |           return value < subsectionEntry.offset;
    |                  ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~
```
2021-04-02 11:33:56 +08:00
Alexander Shaposhnikov
f6ad045366 [lld][MachO] Make emitEndFunStab independent from .subsections_via_symbols
This diff addresses FIXME in SyntheticSections.cpp and removes
the dependency of emitEndFunStab on .subsections_via_symbols.

Test plan: make check-lld-macho

Differential revision: https://reviews.llvm.org/D99054
2021-04-01 17:48:09 -07:00
Alexander Shaposhnikov
f1e4e2fb20 [lld][MachO] Refactor handling of subsections
This diff is a preparation for fixing FunStabs (incorrect size calculation).
std::map<uint32_t, InputSection*> (SubsectionMap) is replaced with
a sorted vector + binary search. If .subsections_via_symbols is set
this vector will contain the list of subsections, otherwise,
the offsets will be used for calculating the symbols sizes.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D98837
2021-03-31 16:52:53 -07:00
Jez Ng
dc8bee9265 [lld-macho] Check address ranges when applying relocations
This diff required fixing `getEmbeddedAddend` to apply sign
extension to 32-bit values. We were previously passing around wrong
64-bit addend values that became "right" after being truncated back to
32-bit.

I've also made `getEmbeddedAddend` return a signed int, which is similar
to what LLD-ELF does for its `getImplicitAddend`.

`reportRangeError`, `checkUInt`, and `checkInt` are counterparts of similar
functions in LLD-ELF.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D98387
2021-03-12 17:26:27 -05:00
Jez Ng
a723db92d8 [lld-macho][nfc] Refactor subtractor reloc handling
SUBTRACTOR relocations are always paired with UNSIGNED
relocations to indicate a pair of symbols whose address difference we
want. Functionally they are like a single relocation: only one pointer
gets written / relocated. Previously, we would handle these pairs by
skipping over the SUBTRACTOR relocation and writing the pointer when
handling the UNSIGNED reloc. This diff reverses things, so we write
while handling SUBTRACTORs and skip over the UNSIGNED relocs instead.

Being able to distinguish between SUBTRACTOR and UNSIGNED relocs in the
write phase (i.e. inside `relocateOne`) is useful for the upcoming range
check diff: we want to check that SUBTRACTOR relocs write signed values,
but UNSIGNED relocs (naturally) write unsigned values.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D98386
2021-03-11 13:28:13 -05:00
Jez Ng
e8a3058303 [lld-macho] Fix handling of X86_64_RELOC_SIGNED_{1,2,4}
The previous implementation miscalculated the addend, resulting
in an underflow. This meant that every SIGNED_N section relocation would
be associated with the last subsection (since the addend would now be a
huge number). We were "lucky" that this mistake was typically cancelled
out -- 64-to-32-bit-truncation meant that the final value was correct,
as long as subsections were not rearranged.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D98385
2021-03-11 13:28:11 -05:00