We recently added `--initial-heap` - an option that allows one to up the
initial memory size without the burden of having to know exactly how
much is needed.
However, in the process of implementing support for this in Emscripten
(https://github.com/emscripten-core/emscripten/pull/21071), we have
realized that `--initial-heap` cannot support the use-case of
non-growable memories by itself, since with it we don't know what to set
`--max-memory` to.
We have thus agreed to move the above work forward by introducing
another option to the linker (see
https://github.com/emscripten-core/emscripten/pull/21071#discussion_r1491755616),
one that would allow users to explicitly specify they want a
non-growable memory.
This change does this by introducing `--no-growable-memory`: an option
that is mutally exclusive with `--max-memory` (for simplicity - we can
also decide that it should override or be overridable by `--max-memory`.
In Emscripten a similar mix of options results in `--no-growable-memory`
taking precedence). The option specifies that the maximum memory size
should be set to the initial memory size, effectively disallowing memory
growth.
Closes#81932.
In WebAssembly, we have `WASM_SYMBOL_NO_STRIP` symbol flag to mark the
referenced content as retained. However, the flag is not enough to
express retained data that is not referenced by any symbol. This patch
adds a new segment flag`WASM_SEG_FLAG_RETAIN` to support "private"
linkage data that is retained by llvm.used.
This kind of data that is not referenced but must be retained is usually
used with encapsulation symbols (__start/__stop). Swift runtime uses
this technique and depends on the fact "all metadata sections in live
objects are retained", which was not guaranteed with `--gc-sections`
before this patch.
This is a revised version of https://reviews.llvm.org/D126950 (has been
reverted) based on @MaskRay's comments
Wasm has no unified virtual memory space as other object formats and
architectures do, so previously WasmObjectFile reported 0 for all
section addresses, and until 428cf71ff used section offsets for function
symbols. Now we use file offsets for function symbols, and this change
switches section addresses to do the same (in linked files). The main
result of this is that objdump now reports VMAs in section listings, and
also uses file offets rather than section offsets when disassembling
linked binaries (matching the behavior of other disassemblers and stack
traces produced by browwsers). To make this work, this PR also updates
objdump's generation of synthetics fallback symbols to match lib/Object
and also correctly plumbs symbol types for regular and dummy symbols
through to the backend to avoid needing special knowledge of address 0.
This also paves the way for generating symbols from name sections rather
than symbol tables or imports (see #76107) by allowing the
disassembler's synthetic fallback symbols match the name-section
generated symbols (in a followup PR).
Followup to #78658, which caused a regression in emscripten.
When a lazy symbol is added, which resolved and existing undefined
symbol, we don't need/want to replace the undefined symbol with the lazy
one. Instead we called extract, which replaces the undefined symbol with
the defined one.
The fact that we were first replacing the undefined symbol with a lazy
one before extracting the archive member doesn't normally matter but, in
the case of the function symbol, replacing the undefined symbol with a
lazy symbol means that `addDefinedFunction` sees the existing symbol as
lazy and simply replaces it.
Note that this is consistent with both the ELF code in
`Symbol::resolve(const LazySymbol &other)` and the wasm code prior to
#78658, neither of which replace the existing symbol with the lazy one
in this case.
The ELF linker transitioned away from archive indexes in
https://reviews.llvm.org/D117284.
This paves the way for supporting `--start-lib`/`--end-lib` (See #77960)
The ELF linker unified library handling with `--start-lib`/`--end-lib` and removed
the ArchiveFile class in https://reviews.llvm.org/D119074.
When undefined functions exist in the final link we need to create
stub functions (otherwise direct calls to those functions could
not be generated). We were creating those stub when
`--unresolved-symbols=ignore-all` was passed but overlooked the fact
that `--warn-unresolved-symbols` essentially has the same effect (i.e.
undefined function can exist in the final link).
Fixes: #53987
It is beneficial to preallocate a certain number of pages in the linear
memory (i. e. use the "minimum" field of WASM memories) so that fewer
"memory.grow"s are needed at startup.
So far, the way to do that has been to pass the "--initial-memory"
option to the linker. It works, but has the very significant downside of
requiring the user to know the size of static data beforehand, as it
must not exceed the number of bytes passed-in as "--initial-memory".
The new "--initial-heap" option avoids this downside by simply appending
the specified number of pages to static data (and stack), regardless of
how large they already are.
Ref: https://github.com/emscripten-core/emscripten/issues/20888.
When doing LTO on multiple archives, the order with which bitcodes are
linked to the LTO module is hard to control, given that processing
undefined symbols can lead to parsing of an object file, which in turn
lead to parsing of another object file before finishing parsing of the
previous file. This can result in encountering a non-prevailing comdat
first when linking, which can make the the symbol undefined, and the
real definition is added later with an additional prefix to avoid
duplication (e.g. `__cxx_global_var_init` and `__cxx_global_var_init.2`)
So this one-line fix ensures we compile bitcodes in the order that we
process comdats, so that when multiple archived bitcode files have the
same variable with the same comdat, we make sure that the prevailing
comdat will be linked first in the LTO.
Fixes#62243.
In emscripten we have a build mode (the default actually) where the
runtime never exits and therefore `__cxa_atexit` is a dummy/stub
function that does nothing. In this case we would like to be able
completely DCE any otherwise-unused global dtor functions.
Fixes: https://github.com/emscripten-core/emscripten/issues/19993
Fixes a crash in `-Wl,-emit-relocs` where the linker was not able to
write linker-synthetic absolute symbols to the symbol table.
This change adds a new symbol flag (`WASM_SYMBOL_ABS`), which means that
the symbol's offset is absolute and not relative to a given segment.
Such symbols include `__stack_low` and `__stack_low`.
Note that wasm object files never contains such symbols, only binaries
linked with `-Wl,-emit-relocs`.
Fixes: #67111
This change writes the module name to the name section of the wasm
binary. We use the `-soname` argument to determine the name and we
default the output file basename if this option is not specified.
In the future we will likely want to embed the soname in the dylink
section too, but this the first step in supporting `-soname`.
Differential Revision: https://reviews.llvm.org/D158001
When stub libraries trigger the fetching of new object files we can
potentially introduce new undefined symbols so process the stub in
loop until no new objects are pulled in.
Differential Revision: https://reviews.llvm.org/D153466
Similar to recent changes to ELF (e.g., D154813), Mach-O, and COFF to
improve hashing performance.
Reviewed By: dschuff
Differential Revision: https://reviews.llvm.org/D155752
Annotation attributes may be attached to a function to mark it with
custom data that will be contained in the final Wasm file. The
annotation causes a custom section named
"func_attr.annotate.<name>.<arg0>.<arg1>..." to be created that will
contain each function's index value that was marked with the annotation.
A new patchable relocation type for function indexes had to be created so
the custom section could be updated during linking.
Reviewed By: sbc100
Differential Revision: https://reviews.llvm.org/D150803
This is an ongoing series of commits that are reformatting our
Python code. This catches the last of the python files to
reformat. Since they where so few I bunched them together.
Reformatting is done with `black`.
If you end up having problems merging this commit because you
have made changes to a python file, the best way to handle that
is to run git checkout --ours <yourfile> and then reformat it
with black.
If you run into any problems, post to discourse about it and
we will try to help.
RFC Thread below:
https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Reviewed By: jhenderson, #libc, Mordante, sivachandra
Differential Revision: https://reviews.llvm.org/D150784
Also, fix checking of first line in ::parse. We can't use the
::getLines helper here since that already does comment stripping
internally.
Differential Revision: https://reviews.llvm.org/D147548
There are cases where stub library processing can trigger new exports
which might require them to be included at LTO time.
Specifically `processStubLibraries` marks symbols as `forceExports`
which even effect the LTO process.
And since the LTO process can generate new undefined symbols
(specifically libcall function) we need to also process the stub
libraries after LTO.
Differential Revision: https://reviews.llvm.org/D147190
Implement the --build-id flag similarly to ELF, and generate a
build_id section according to the WebAssembly tool convention
specified in https://github.com/WebAssembly/tool-conventions/pull/183
The default style ("fast" aka "tree") hashes the contents of the
output and (unlike ELF) generates a v5 UUID based on the hash (using a
random namespace). It also supports generating a random v4 UUID, a
sha1 hash, and a user-specified string (as ELF does).
Differential Revision: https://reviews.llvm.org/D107662
Fix MSVC build by std::copy on the underying buffer rather than
directly from std::array to llvm::MutableArrayRef
Implement the --build-id flag similarly to ELF, and generate a build_id
section according to the WebAssembly tool convention specified in
https://github.com/WebAssembly/tool-conventions/pull/183
The default style ("fast" aka "tree") hashes the contents of the output
and (unlike ELF) generates a v5 UUID based on the hash (using a random
namespace).
It also supports generating a random v4 UUID, a sha1 hash,
and a user-specified string (as ELF does).
Differential Revision: https://reviews.llvm.org/D107662
Allow controlling the CodeGenOpt::Level independent of the LTO
optimization level in LLD via new options for the COFF, ELF, MachO, and
wasm frontends to lld. Most are spelled as --lto-CGO[0-3], but COFF is
spelled as -opt:lldltocgo=[0-3].
See D57422 for discussion surrounding the issue of how to set the CG opt
level. The ultimate goal is to let each function control its CG opt
level, but until then the current default means it is impossible to
specify a CG opt level lower than 2 while using LTO. This option gives
the user a means to control it for as long as it is not handled on a
per-function basis.
Reviewed By: MaskRay, #lld-macho, int3
Differential Revision: https://reviews.llvm.org/D141970
This only effects folks building with wasm64 + shared memory which
is not currently a supported configuration in emscripten or any other
wasm toolchain.
Differential Revision: https://reviews.llvm.org/D141005
A specific case for ThinLTO cache pruning is that the current build is huge, and the cache wasn't big enough to hold the intermediate object files of that build. So in doing that build, a file would be cached, and later in that same build it would be evicted. This was significantly decreasing the effectiveness of the cache. By giving this warning, the user could identify the required cache size/files and improve ThinLTO link speed.
Differential Revision: https://reviews.llvm.org/D135590
This adds an `--export-memory` option to wasm-ld which allows passing
a name to give to the exported memory, and extends `--import-memory` to
allow passing a <module>,<name> pair specifying where the memory should
be imported from.
This is based on https://reviews.llvm.org/D131376, with the main
difference being that it only supports exporting memory by one name.
Differential Revision: https://reviews.llvm.org/D135898
Add some checks around this combination of flags
Also, honor `--global-base` when specified in `--stack-first` mode
rather than ignoring it. But error out if the specified base preseeds
the end of the stack.
Differential Revision: https://reviews.llvm.org/D136117
Define a `__heap_end` symbol that marks the end of the memory region
that starts at `__heap_base`. This will allow malloc implementations to
know how much memory they can use at `__heap_base` even if someone has
done a `memory.grow` before they can initialize their state.
Differential Revision: https://reviews.llvm.org/D136110
