There was an assumpiton that TUs and CUs share .debug_str_offsets
contribution. For ThinLTO builds it is not the case. Changed so that we
parse contributions for TUs also, and did some refactoring so that we
don't re-parse contributions that were not modified.
When option --dwarf-output-path is specified, if the path does not exist
BOLT will now create it. This is what also happens when
--plugin-opt=dwo_dir=<value> is specified to LLD.
Fixed handling of DWP as input. Before BOLT crashed. Now it will write
out
correct CU, and all the TUs. Potential future improvement is to scan all
the TUs
used in this CU, and only include those.
Fixed a bug where when --dwarf-output-path is specified
and DW_AT_dwo_name contains part of the path the output path would
contain both.
Which lead to llvm-bolt crash, because the path didn't exist.
Example:
llvm-bolt .... --dwarf-output-path=/some/path/
DW_AT_dwo_name ("objects/o1/split.dwo")
It would try to write .dwo file to /some/path/objects/o1/split.dwo.dwo
instead of to
/some/path/split.dwo.dwo
Closes https://github.com/llvm/llvm-project/issues/63097
Before merging please make sure the change to
bolt/include/bolt/Passes/StokeInfo.h is correct.
bolt/include/bolt/Passes/StokeInfo.h
```diff
// This Pass solves the two major problems to use the Stoke program without
- // proting its code:
+ // probing its code:
```
I'm still not happy about the awkward wording in this comment.
bolt/include/bolt/Passes/FixRelaxationPass.h
```
$ ed -s bolt/include/bolt/Passes/FixRelaxationPass.h <<<'9,12p'
// This file declares the FixRelaxations class, which locates instructions with
// wrong targets and fixes them. Such problems usually occures when linker
// relaxes (changes) instructions, but doesn't fix relocations types properly
// for them.
$
```
bolt/docs/doxygen.cfg.in
bolt/include/bolt/Core/BinaryContext.h
bolt/include/bolt/Core/BinaryFunction.h
bolt/include/bolt/Core/BinarySection.h
bolt/include/bolt/Core/DebugData.h
bolt/include/bolt/Core/DynoStats.h
bolt/include/bolt/Core/Exceptions.h
bolt/include/bolt/Core/MCPlusBuilder.h
bolt/include/bolt/Core/Relocation.h
bolt/include/bolt/Passes/FixRelaxationPass.h
bolt/include/bolt/Passes/InstrumentationSummary.h
bolt/include/bolt/Passes/ReorderAlgorithm.h
bolt/include/bolt/Passes/StackReachingUses.h
bolt/include/bolt/Passes/StokeInfo.h
bolt/include/bolt/Passes/TailDuplication.h
bolt/include/bolt/Profile/DataAggregator.h
bolt/include/bolt/Profile/DataReader.h
bolt/lib/Core/BinaryContext.cpp
bolt/lib/Core/BinarySection.cpp
bolt/lib/Core/DebugData.cpp
bolt/lib/Core/DynoStats.cpp
bolt/lib/Core/Relocation.cpp
bolt/lib/Passes/Instrumentation.cpp
bolt/lib/Passes/JTFootprintReduction.cpp
bolt/lib/Passes/ReorderData.cpp
bolt/lib/Passes/RetpolineInsertion.cpp
bolt/lib/Passes/ShrinkWrapping.cpp
bolt/lib/Passes/TailDuplication.cpp
bolt/lib/Rewrite/BoltDiff.cpp
bolt/lib/Rewrite/DWARFRewriter.cpp
bolt/lib/Rewrite/RewriteInstance.cpp
bolt/lib/Utils/CommandLineOpts.cpp
bolt/runtime/instr.cpp
bolt/test/AArch64/got-ld64-relaxation.test
bolt/test/AArch64/unmarked-data.test
bolt/test/X86/Inputs/dwarf5-cu-no-debug-addr-helper.s
bolt/test/X86/Inputs/linenumber.cpp
bolt/test/X86/double-jump.test
bolt/test/X86/dwarf5-call-pc-function-null-check.test
bolt/test/X86/dwarf5-split-dwarf4-monolithic.test
bolt/test/X86/dynrelocs.s
bolt/test/X86/fallthrough-to-noop.test
bolt/test/X86/tail-duplication-cache.s
bolt/test/runtime/X86/instrumentation-ind-calls.s
Create BinaryFunction::translateInputToOutputRange() and use it for
updating DWARF debug ranges and location lists while de-duplicating the
existing code. Additionally, move DWARF-specific code out of
BinaryFunction and add print functions to facilitate debugging.
Note that this change is deliberately kept "bug-level" compatible with
the existing solution to keep it NFCI and make it easier to track any
possible regressions in the future updates to the ranges-handling code.
This bug crept in when CU partitioning was introduced. It manifests itself when
there are CUs that use location lists that come before CUs that are part of
thin-lto. BOLT processes CUs with cross CU references first (these are produced
by thin-lto). When we wrote out all the location lists we did it in original
order. Since DWARF4 uses offsets directly in to .debug_loc those offsets in DIEs
became wrong.
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D157908
Changed to creating a new index all the time. This code was legacy of when we
couldn't change the size of .debug_info, and led to subtle bugs where index for
new entries was pointing to a wrong address.
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D157356
Changed to creating a new index all the time. This code was legacy of when we
couldn't change the size of .debug_info, and led to subtle bugs where index for
new entries was pointing to a wrong address.
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D157355
This is purely to make debugging easier for developers. Now that we moved to IR
the print out of DIEs is lacking. This function will lazily parse DIE and use
DWARFDie dump function.
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D157354
Fixed a bug where when Skelton CU had DW_AT_ranges, it the output CU DW_AT_ranges
offset was relative, and not absolute.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D156958
Now that we have new DWARF Rewriter we can remove DW_AT_low_pc when converting
DW_AT_low_pc/DW_AT_high_pc to DW_AT_ranges. Which closer follows DWARF spec.
Leaving CU DW_AT_low_pc in place. Reading the spec I think it's needed.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D156957
Clang can generate DW_TAG_inlined_subroutine with low_pc 0. With split dwarf
this led to range offset being a negative number.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D156742
When output range is only one entry, and input is low_pc/high_pc do not convert
to ranges. This helps with size of .debug_ranges/.debug_rnglists. It also helps
when either low_pc/high_pc is 0. We not generating potentially invalid ranges
that result in LLDB error.
Also fixed handling of DW_AT_subprogram with ranges. This can be created with
-fbasic-block-sections=all.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D156374
There are cases in DWARF4 when Skeleton CU has ranges, but dwo CU doesn't.
Bug was introduced in new DWARFRewriter where for DWARF4 it would fall through
to DWARF5 case.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D155033
The DWO Unit DIE, doesn't have low_pc/high_pc, so we were printing this error
for valid cases.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D155032
To reduce memory footprint changed so that we process and write out TUs first,
reset DIEBuilder and process CUs. CUs are processed in buckets. First bucket
contains all the CUs with cross CU references. Rest processd one at a time.
clang-17 build in debug mode, by clang-17.
before
8:25.81 real, 834.37 user, 86.03 sys, 0 amem, 79525064 mmem
8:02.20 real, 820.46 user, 81.81 sys, 0 amem, 79501616 mmem
7:52.69 real, 802.01 user, 83.99 sys, 0 amem, 79534392 mmem
after
7:49.35 real, 822.04 user, 66.19 sys, 0 amem, 34934260 mmem
7:42.16 real, 825.46 user, 63.52 sys, 0 amem, 34951660 mmem
7:46.71 real, 821.11 user, 63.14 sys, 0 amem, 34981164 mmem
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D151909
* Some cleanup and minor fixes for the new debug information re-writer before moving on
to productatization.
* The new rewriter wasn't handling binary with DWARF5 and DWARF4 with
-fdebug-types-sections.
* Removed dead cross cu reference code.
* Added support for DW_AT_sibling.
* With the new re-writer abbrev number can change which can lead to offset of Type
Units changing. Before we would just copy raw data. Changed to write out Type
Unit List. This is generated by gdb-add-index.
* Fixed how bolt handles gdb-index generated by gdb-11 with types sections.
Simplified logic that handles variations of gdb-index.
* Clang can generate two type units with the same hash, but different content. LLD
does not de-duplicate when ThinLTO is involved. Changed so that TU hash and
offset are used to make TU's unique.
* It is possible to have references within location expression to another DIE.
Fixed it so that relative offset is updated correctly.
* Removed all the code related to patching.
* Removed dead code. Changed how we handling writting out TUs and TU Index. It now
should fully work for DWARF4 and DWARF5.
* Removed unused arguments from some APIs, changed return type to void, and other
small cleanups.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D151906
This revision implement new mechanism for DWARFRewriter.
In the new mechanism, we adopt the same way with DWARFLinker did.
By parsing Debug information into IR, we are allowed to handle debug information more flexible.
Now the debug information updating process relies on IR and IR will be written out to binary once the updating finished.
A new class was added: DIEBuilder. This class is responsible for parsing debug information and raising it to the IR level.
This class is also used to write out the .debug_info and .debug_abbrev sections.
Since we output brand new Abbrev section we won't need to always convert low_pc/high_pc into ranges.
When conversion does happen we can also remove low_pc entry.
Reviewed By: maksfb, ayermolo
Differential Revision: https://reviews.llvm.org/D130315
This reverts commit 460a224443.
It breaks building on macOS, and it was landed with a review URL
pointing to some Facebook-internal service.
Also reverts a bunch of follow-ups:
Revert "[BOLT][DWARF] Don't check string offsets"
This reverts commit f9d6f48c8b.
Revert "[BOLT][DWARF] Change to process and write out TUs first then CUs in batches"
This reverts commit 88e95c1e4b.
Revert "[BOLT][DWARF] Output DWO files as they are being processed"
This reverts commit 46ca2e3fcd.
Revert "[BOLT][DWARF] Don't check string offsets"
This reverts commit cfe4a4b04f.
Revert "[BOLT][DWARF] Numerous fixes for a new DWARFRewriter"
This reverts commit 2701a661da.
Summary:
To reduce memory footprint changed so that we process and write out TUs first,
reset DIEBuilder and process CUs. CUs are processed in buckets. First bucket
contains all the CUs with cross CU references. Rest processd one at a time.
clang-17 build in debug mode, by clang-17.
before
8:25.81 real, 834.37 user, 86.03 sys, 0 amem, 79525064 mmem
8:02.20 real, 820.46 user, 81.81 sys, 0 amem, 79501616 mmem
7:52.69 real, 802.01 user, 83.99 sys, 0 amem, 79534392 mmem
after
7:49.35 real, 822.04 user, 66.19 sys, 0 amem, 34934260 mmem
7:42.16 real, 825.46 user, 63.52 sys, 0 amem, 34951660 mmem
7:46.71 real, 821.11 user, 63.14 sys, 0 amem, 34981164 mmem
Differential Revision: https://phabricator.intern.facebook.com/D45883198
Summary:
* Some cleanup and minor fixes for the new debug information re-writer before moving on
to productatization.
* The new rewriter wasn't handling binary with DWARF5 and DWARF4 with
-fdebug-types-sections.
* Removed dead cross cu reference code.
* Added support for DW_AT_sibling.
* With the new re-writer abbrev number can change which can lead to offset of Type
Units changing. Before we would just copy raw data. Changed to write out Type
Unit List. This is generated by gdb-add-index.
* Fixed how bolt handles gdb-index generated by gdb-11 with types sections.
Simplified logic that handles variations of gdb-index.
* Clang can generate two type units with the same hash, but different content. LLD
does not de-duplicate when ThinLTO is involved. Changed so that TU hash and
offset are used to make TU's unique.
* It is possible to have references within location expression to another DIE.
Fixed it so that relative offset is updated correctly.
* Removed all the code related to patching.
* Removed dead code. Changed how we handling writting out TUs and TU Index. It now
should fully work for DWARF4 and DWARF5.
* Removed unused arguments from some APIs, changed return type to void, and other
small cleanups.
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
Differential Revision: https://phabricator.intern.facebook.com/D46168257
Summary:
This revision implement new mechanism for DWARFRewriter.
In the new mechanism, we adopt the same way with DWARFLinker did.
By parsing Debug information into IR, we are allowed to handle debug information more flexible.
Now the debug information updating process relies on IR and IR will be written out to binary once the updating finished.
A new class was added: DIEBuilder. This class is responsible for parsing debug information and raising it to the IR level.
This class is also used to write out the .debug_info and .debug_abbrev sections.
Since we output brand new Abbrev section we won't need to always convert low_pc/high_pc into ranges.
When conversion does happen we can also remove low_pc entry.
Differential Revision: https://phabricator.intern.facebook.com/D39484421
Tasks: T117448832
A CU can have only one function so CU will have low_pc/high_pc. If this funciton
is GCed by LLD low_pc will become 0x0, and BOLT can't map this to output. We
kind of were getting away with it in monolithic DWARF, but with split DWARF
there is only skeleton CU, so we end up with rnglist with header and array, but
no body. This caused LLDB to report an error.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D151845
Spotted this one while working on new DWARF Rewriter. We were using wrong check
in assertion.
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D150167
When input is DWP with DWARF5 bolt wasn't handling correctly CUs that didn't
have TU references. Which resulted in a crash.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D148589
Changed contribution data structure to 64 bit. I added the 32bit and 64bit
accessors to make it explicit where we use 32bit and where we use 64bit. Also to
make sure sure we catch all the cases where this data structure is used.
Reviewed By: dblaikie
Differential Revision: https://reviews.llvm.org/D139379
Changed contribution data structure to 64 bit. I added the 32bit and 64bit
accessors to make it explicit where we use 32bit and where we use 64bit. Also to
make sure sure we catch all the cases where this data structure is used.
Reviewed By: dblaikie
Differential Revision: https://reviews.llvm.org/D139379
Managed to introduce an error when changing code to fix other tests and the unit
test was no adequate due to --nostdlib being passed in in llvm testing
enviroment.
Original diff: https://reviews.llvm.org/D132059
Updated a test to make sure that original address and the new address are
different.
Reviewed By: maksfb, #bolt
Differential Revision: https://reviews.llvm.org/D132782
With ThinLTO mutliple CUs can share the same .debug_str_offsets contribution. We
were creating a new one for each CU. This lead to a binary size increase.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D139214
Summary:
Changed contribution data structure to 64 bit. I added the 32bit and 64bit
accessors to make it explicit where we use 32bit and where we use 64bit. Also to
make sure sure we catch all the cases where this data structure is used.
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
We were trying to process .debug_addr for CU that doesn't have it. This resulted
in assert. Example came from GCC that also doesn't use DW_OP_addrx in
DW_FORM_exprloc.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D132422
Wasn't handling correctly creating DW_AT_rnglists_base in UnitDie when
converting access pattern for DW_AT_ranges from offset to index for DWARF5.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132087
We were resetting DW_AT_low_pc to zero when DW_AT_high_pc was zero, or
DW_AT_low_pc == DW_AT_high_pc. This resulted in LLDB to print error "adding
range [0x0-0x0) which has a base that is less than the function's low PC".
Changed it so that when this case arises we set DW_AT_low_pc to the start
address.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132059
DW_TAG_call_site/DW_AT_call_return_pc can contain address that is not in any
function. In this case getBinaryFunctionContainingAddress returns nullptr. For
this case preserving original address.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132057
