When using split DWARF with .dwp files we had an issue where sometimes
the DWO file within the .dwp file would be parsed _before_ the skeleton
compile unit. The DWO file expects to be able to always be able to get a
link back to the skeleton compile unit. Prior to this fix, the only time
the skeleton compile unit backlink would get set, was if the unit
headers for the main executable have been parsed _and_ if the unit DIE
was parsed in that DWARFUnit. This patch ensures that we can always get
the skeleton compile unit for a DWO file by adding a function:
```
DWARFCompileUnit *DWARFUnit::GetSkeletonUnit();
```
Prior to this fix DWARFUnit had some unsafe accessors that were used to
store two different things:
```
void *DWARFUnit::GetUserData() const;
void DWARFUnit::SetUserData(void *d);
```
This was used by SymbolFileDWARF to cache the `lldb_private::CompileUnit
*` for a SymbolFileDWARF and was also used to store the `DWARFUnit *`
for SymbolFileDWARFDwo. This patch clears up this unsafe usage by adding
two separate accessors and ivars for this:
```
lldb_private::CompileUnit *DWARFUnit::GetLLDBCompUnit() const { return m_lldb_cu; }
void DWARFUnit::SetLLDBCompUnit(lldb_private::CompileUnit *cu) { m_lldb_cu = cu; }
DWARFCompileUnit *DWARFUnit::GetSkeletonUnit();
void DWARFUnit::SetSkeletonUnit(DWARFUnit *skeleton_unit);
```
This will stop anyone from calling `void *DWARFUnit::GetUserData()
const;` and casting the value to an incorrect value.
A crash could occur in `SymbolFileDWARF::GetCompUnitForDWARFCompUnit()`
when the `non_dwo_cu`, which is a backlink to the skeleton compile unit,
was not set and was NULL. There is an assert() in the code, and then the
code just will kill the program if the assert isn't enabled because the
code looked like:
```
if (dwarf_cu.IsDWOUnit()) {
DWARFCompileUnit *non_dwo_cu =
static_cast<DWARFCompileUnit *>(dwarf_cu.GetUserData());
assert(non_dwo_cu);
return non_dwo_cu->GetSymbolFileDWARF().GetCompUnitForDWARFCompUnit(
*non_dwo_cu);
}
```
This is now fixed by calling the `DWARFUnit::GetSkeletonUnit()` which
will correctly always get the skeleton compile uint for a DWO file
regardless of if the skeleton unit headers have been parse or if the
skeleton unit DIE wasn't parsed yet.
To implement the ability to get the skeleton compile units, I added code
the DWARFDebugInfo.cpp/.h that make a map of DWO ID -> skeleton
DWARFUnit * that gets filled in for DWARF5 when the unit headers are
parsed. The `DWARFUnit::GetSkeletonUnit()` will end up parsing the unit
headers of the main executable to fill in this map if it already hasn't
been done. For DWARF4 and earlier we maintain a separate map that gets
filled in only for any DWARF4 compile units that have a DW_AT_dwo_id or
DW_AT_gnu_dwo_id attributes. This is more expensive, so this is done
lazily and in a thread safe manor. This allows us to be as efficient as
possible when using DWARF5 and also be backward compatible with DWARF4 +
split DWARF.
There was also an issue that stopped type lookups from succeeding in
`DWARFDIE SymbolFileDWARF::GetDIE(const DIERef &die_ref)` where it
directly was accessing the `m_dwp_symfile` ivar without calling the
accessor function that could end up needing to locate and load the .dwp
file. This was fixed by calling the
`SymbolFileDWARF::GetDwpSymbolFile()` accessor to ensure we always get a
valid value back if we can find the .dwp file. Prior to this fix it was
down which APIs were called and if any APIs were called that loaded the
.dwp file, it worked fine, but it might not if no APIs were called that
did cause it to get loaded.
When we have valid debug info indexes and when the lldb index cache was
enabled, this would cause this issue to show up more often.
I modified an existing test case to test that all of this works
correctly and doesn't crash.
`statistics dump` command relies on `SymbolFile::GetDebugInfoSize()` to
get total debug info size.
The current implementation is missing debug info for split dwarf
scenarios which requires getting debug info from separate dwo/dwp files.
This patch fixes this issue for split dwarf by parsing debug info from
dwp/dwo.
New yaml tests are added.
---------
Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
Store a SupportFile, rather than a FileSpec, in CompileUnit. This commit
works towards having the SourceManager operate on SupportFiles so that
it can (1) validate the Checksum and (2) materialize the content of
inline source information.
Per this RFC:
https://discourse.llvm.org/t/rfc-improve-lldb-progress-reporting/75717
on improving progress reports, this commit separates the title field and
details field so that the title specifies the category that the progress
report falls under. The details field is added as a part of the
constructor for progress reports and by default is an empty string. In addition, changes the total amount of progress completed into a std::optional. Also
updates the test to check for details being correctly reported from the
event structured data dictionary.
When I added the MD5 checksum I was on the fence between storing it in
FileSpec or creating a new SupportFile abstraction. The latter was
deemed overkill for just the MD5 hashes, but support for inline sources
in the DWARF 5 line table tipped the scales. This patch moves the MD5
checksum into the new SupportFile class.
…ntext
Following the specification chain seems to be clearly the expected
behavior of GetDeclContext(). Otherwise C++ methods have an empty
CompilerContext instead of being nested in their struct/class.
Theprimary motivation for this functionality is the Swift plugin. In
order to test the change I added a proof-of-concept implementation of a
Module::FindFunction() variant that takes a CompilerContext, expesed via
lldb-test.
rdar://120553412
This moves the functionally of finding a DIE based on a fully qualified
name from SymbolFileDWARF into DWARFIndex itself, so that
specializations of DWARFIndex can implement faster versions of this
query.
LLVM supports DWARF 5 linetable extension to store source files inline
in DWARF. This is particularly useful for compiler-generated source
code. This implementation tries to materialize them as temporary files
lazily, so SBAPI clients don't need to be aware of them.
rdar://110926168
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
This patch revives the effort to get this Phabricator patch into
upstream:
https://reviews.llvm.org/D137900
This patch was accepted before in Phabricator but I found some
-gsimple-template-names issues that are fixed in this patch.
A fixed up version of the description from the original patch starts
now.
This patch started off trying to fix Module::FindFirstType() as it
sometimes didn't work. The issue was the SymbolFile plug-ins didn't do
any filtering of the matching types they produced, and they only looked
up types using the type basename. This means if you have two types with
the same basename, your type lookup can fail when only looking up a
single type. We would ask the Module::FindFirstType to lookup "Foo::Bar"
and it would ask the symbol file to find only 1 type matching the
basename "Bar", and then we would filter out any matches that didn't
match "Foo::Bar". So if the SymbolFile found "Foo::Bar" first, then it
would work, but if it found "Baz::Bar" first, it would return only that
type and it would be filtered out.
Discovering this issue lead me to think of the patch Alex Langford did a
few months ago that was done for finding functions, where he allowed
SymbolFile objects to make sure something fully matched before parsing
the debug information into an AST type and other LLDB types. So this
patch aimed to allow type lookups to also be much more efficient.
As LLDB has been developed over the years, we added more ways to to type
lookups. These functions have lots of arguments. This patch aims to make
one API that needs to be implemented that serves all previous lookups:
- Find a single type
- Find all types
- Find types in a namespace
This patch introduces a `TypeQuery` class that contains all of the state
needed to perform the lookup which is powerful enough to perform all of
the type searches that used to be in our API. It contain a vector of
CompilerContext objects that can fully or partially specify the lookup
that needs to take place.
If you just want to lookup all types with a matching basename,
regardless of the containing context, you can specify just a single
CompilerContext entry that has a name and a CompilerContextKind mask of
CompilerContextKind::AnyType.
Or you can fully specify the exact context to use when doing lookups
like: CompilerContextKind::Namespace "std"
CompilerContextKind::Class "foo"
CompilerContextKind::Typedef "size_type"
This change expands on the clang modules code that already used a
vector<CompilerContext> items, but it modifies it to work with
expression type lookups which have contexts, or user lookups where users
query for types. The clang modules type lookup is still an option that
can be enabled on the `TypeQuery` objects.
This mirrors the most recent addition of type lookups that took a
vector<CompilerContext> that allowed lookups to happen for the
expression parser in certain places.
Prior to this we had the following APIs in Module:
```
void
Module::FindTypes(ConstString type_name, bool exact_match, size_t max_matches,
llvm::DenseSet<lldb_private::SymbolFile *> &searched_symbol_files,
TypeList &types);
void
Module::FindTypes(llvm::ArrayRef<CompilerContext> pattern, LanguageSet languages,
llvm::DenseSet<lldb_private::SymbolFile *> &searched_symbol_files,
TypeMap &types);
void Module::FindTypesInNamespace(ConstString type_name,
const CompilerDeclContext &parent_decl_ctx,
size_t max_matches, TypeList &type_list);
```
The new Module API is much simpler. It gets rid of all three above
functions and replaces them with:
```
void FindTypes(const TypeQuery &query, TypeResults &results);
```
The `TypeQuery` class contains all of the needed settings:
- The vector<CompilerContext> that allow efficient lookups in the symbol
file classes since they can look at basename matches only realize fully
matching types. Before this any basename that matched was fully realized
only to be removed later by code outside of the SymbolFile layer which
could cause many types to be realized when they didn't need to.
- If the lookup is exact or not. If not exact, then the compiler context
must match the bottom most items that match the compiler context,
otherwise it must match exactly
- If the compiler context match is for clang modules or not. Clang
modules matches include a Module compiler context kind that allows types
to be matched only from certain modules and these matches are not needed
when d oing user type lookups.
- An optional list of languages to use to limit the search to only
certain languages
The `TypeResults` object contains all state required to do the lookup
and store the results:
- The max number of matches
- The set of SymbolFile objects that have already been searched
- The matching type list for any matches that are found
The benefits of this approach are:
- Simpler API, and only one API to implement in SymbolFile classes
- Replaces the FindTypesInNamespace that used a CompilerDeclContext as a
way to limit the search, but this only worked if the TypeSystem matched
the current symbol file's type system, so you couldn't use it to lookup
a type in another module
- Fixes a serious bug in our FindFirstType functions where if we were
searching for "foo::bar", and we found a "baz::bar" first, the basename
would match and we would only fetch 1 type using the basename, only to
drop it from the matching list and returning no results
There was duplicated (and complex) code querying whether tags were
type-like tags (i.e. class or struct); this has been factored out into a
helper function.
There was also a comment about not comparing identical DIEs without ever
performing that check; this comment has been removed. It was likely a
result of copy paste from another function in this same file which
actually does that check.
The function FindDefinitionTypeForDWARFDeclContext loops over all DIEs
corresponding to types with a certain name and compares the context of
each found DIE with the context of a target DIE. However, the target DIE
never changes throughout this search, and yet we recompute its
DeclContext on every iteration of the search. This is wasteful because
the method is not exactly free (see
DWARFDebugInfoEntry::GetDWARFDeclContextStatic).
I've plumbed the LLVM DebugInfoD client into LLDB, and added automatic
downloading of DWP files to the SymbolFileDWARF.cpp plugin. If you have
DEBUGINFOD_URLS set to a space delimited set of web servers, LLDB will
try to use them as a last resort when searching for DWP files. If you do
*not* have that environment variable set, nothing should be changed.
There's also a setting, per @clayborg 's suggestion, that will override
the environment variable, or can be used instead of the environment
variable. The setting is why I also needed to add an API to the
llvm-debuginfod library
### Test Plan:
Suggestions are welcome here. I should probably have some positive and
negative tests, but I wanted to get the diff up for people who have a
clue what they're doing to rip it to pieces before spending too much
time validating the initial implementation.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
Co-authored-by: Alex Langford <nirvashtzero@gmail.com>
This patch relands https://github.com/llvm/llvm-project/pull/71004 which
was reverted because the clang change it depends on was reverted.
In addition to the original patch, this PR includes a change to
`SymbolFileDWARF::ParseVariableDIE` to support CU-level variable
definitions that don't have locations, but represent a constant value.
Previously, when debug-maps were available, we would assume that a
variable with "static lifetime" (which in this case means "has a linkage
name") has a valid address, which isn't the case for non-locationed
constants. We could omit this additional change if we stopped attaching
linkage names to global non-locationed constants.
Original commit message:
"""
https://github.com/llvm/llvm-project/pull/71780 proposes moving the
`DW_AT_const_value` on inline static members from the declaration DIE to
the definition DIE. This patch makes sure the LLDB's expression
evaluator can continue to support static initialisers even if the
declaration doesn't have a `DW_AT_const_value` anymore.
Previously the expression evaluator would find the constant for a
VarDecl from its declaration `DW_TAG_member` DIE. In cases where the
initialiser was specified out-of-class, LLDB could find it during symbol
resolution.
However, neither of those will work for constants, since we don't have a
constant attribute on the declaration anymore and we don't have
constants in the symbol table.
"""
Depends on:
* https://github.com/llvm/llvm-project/pull/71780
Read the MD5 checksum from DWARF line tables and store it in the
corresponding support files.
This is a re-land after fixing an off-by-one error in LLDB's
ParseSupportFilesFromPrologue (fixed in #71984).
This fixes a subtle and previously harmless off-by-one bug in
ParseSupportFilesFromPrologue. The function accounts for the start index
being one-based for DWARF v4 and earlier and zero-based for DWARF v5 and
later. However, the same care wasn't taken for the end index.
This bug existed unnoticed because GetFileByIndex gracefully handles an
invalid index. However, the bug manifested itself after #71458, which
added a call to getFileNameEntry which requires the index to be valid.
No test as the bug cannot be observed without the changes from #71458.
Once that PR is merged, this will be covered by all the DWARF v5 tests.
This completes the conversion of LocateSymbolFile into a SymbolLocator
plugin. The only remaining function is DownloadSymbolFileAsync which
doesn't really fit into the plugin model, and therefore moves into the
SymbolLocator class, while still relying on the plugins to do the
underlying work.
Often, we only care about the split-dwarf files that have failed to
load. This can be useful when diagnosing binaries with many separate
debug info files where only some have errors.
```
(lldb) help image dump separate-debug-info
List the separate debug info symbol files for one or more target modules.
Syntax: target modules dump separate-debug-info <cmd-options> [<filename> [<filename> [...]]]
Command Options Usage:
target modules dump separate-debug-info [-ej] [<filename> [<filename> [...]]]
-e ( --errors-only )
Filter to show only debug info files with errors.
-j ( --json )
Output the details in JSON format.
This command takes options and free-form arguments. If your arguments
resemble option specifiers (i.e., they start with a - or --), you must use
' -- ' between the end of the command options and the beginning of the
arguments.
'image' is an abbreviation for 'target modules'
```
I updated the following tests
```
# on Linux
bin/lldb-dotest -p TestDumpDwo
# on Mac
bin/lldb-dotest -p TestDumpOso
```
This change applies to both the table and JSON outputs.
---------
Co-authored-by: Tom Yang <toyang@fb.com>
This effectively moves a few functions from protected to public. In any
case, for the sake of having a cleaner SymbolFileDWARF API, it's better
if it's not a friend of a one of its consumers, DWARFASTParserClang.
Another effect of this change is that I can use SymbolFileDWARF for the
out-of-tree mojo dwarf parser, which relies on pretty much the same
functions that DWARFASTParserClang needs from SymbolFileDWARF.
When the debug info refers to a dwo with relative `DW_AT_comp_dir` and
`DW_AT_dwo_name`, we only print the `DW_AT_comp_dir` in our error
message if we can't find it. This often isn't very helpful, especially
when the `DW_AT_comp_dir` is ".":
```
(lldb) fr v
error: unable to locate .dwo debug file "." for skeleton DIE 0x000000000000003c
```
I'm updating the error message to include both `DW_AT_comp_dir` (if it
exists) and `DW_AT_dwo_name` when the `DW_AT_dwo_name` is relative. The
behavior when `DW_AT_dwo_name` is absolute should be the same.
As a followup of https://github.com/llvm/llvm-project/pull/67851, I'm
defining a new namespace `lldb_plugin::dwarf` for the classes in this
Plugins/SymbolFile/DWARF folder. This change is very NFC and helped me
with exporting the necessary symbols for my out-of-tree language plugin.
The only class that I didn't change is ClangDWARFASTParser, because that
shouldn't be in the same namespace as the generic language-agnostic
dwarf parser.
It would be a good idea if other plugins follow the same namespace
scheme.
The LLVM implementation of DWARFDebugAbbrev does not have a way of
listing all the DW_FORM values that have been parsed but are unsupported
or otherwise unknown. AFAICT this functionality does not exist in LLVM
at all. Since my primary goal is to unify the implementations and not
judge the usefulness or completeness of this functionality, I decided to
move it out of LLDB's implementation of DWARFDebugAbbrev for the time
being.
I want to work towards unifying the implementations. It would be a lot
easier to do if LLDB's DWARFDebugAbbrev looked more similar to LLVM's
implementation, so this change moves in that direction.
This reverts commit dc3f758ddc.
Lit decided to show me the least interesting part of the
test output, but from what I gather on Mac OS the DWARF
stays in the object files (https://stackoverflow.com/a/12827463).
So either split DWARF options do nothing or they produce
files I don't know the name of that aren't .dwo, so I'm
skipping these tests on Darwin.
Fixes#28667
There's a bunch of ways to end up building split DWARF where the
DWO file is not next to the program file. On top of that you may
distribute the program in various ways, move files about, switch
machines, flatten the directories, etc.
This change adds a few more strategies to find DWO files:
* Appending the DW_AT_COMP_DIR and DWO name to all the debug
search paths.
* Appending the same to the binary's dir.
* Appending the DWO name (e.g. a/b/foo.dwo) to all the debug
search paths.
* Appending the DWO name to the binary's location.
* Appending the DWO filename (e.g. foo.dwo) to the debug
search paths.
* Appending the DWO filename to the binary's location.
They are applied in that order and some will be skipped
if the DW_AT_COMP_DIR is relative or absolute, same for
the DWO name (though that seems to always be relative).
This uses the setting target.debug-file-search-paths, which
is used for DWP files already.
The added tests likely do not cover every part of the
strategies listed, it's a best effort.
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D157609
ConstString can be implicitly converted into a llvm::StringRef. This is
very useful in many places, but it also hides places where we are
creating a ConstString only to use it as a StringRef for the entire
lifespan of the ConstString object.
I locally removed the implicit conversion and found some of the places we
were doing this.
Differential Revision: https://reviews.llvm.org/D159237
StreamFile subclasses Stream (from lldbUtility) and is backed by a File
(from lldbHost). It does not depend on anything from lldbCore or any of its
sibling libraries, so I think it makes sense for this to live in
lldbHost instead.
Differential Revision: https://reviews.llvm.org/D157460
This code was introduced in 2fc93eabf7.
In order to improve readability of ParseVariableDIE, we move this code into a
helper function. The issue this code attempted to address was fixed between
Clangs 9 and 11; as such, if we ever want to delete this code, it is a lot
easier to do so after the refactor.
Differential Revision: https://reviews.llvm.org/D155100
This function does a _lot_ of different things:
1. Parses a DIE,
2. Builds an ExpressionList
3. Figures out lifetime of variable
4. Remaps addresses for debug maps
5. Handles external variables
6. Figures out scope of variables
A lot of this functionality is coded in a complex nest of conditions, variables
that are declared and then initialized much later, variables that are updated in
multiple code paths. All of this makes the code really hard to follow.
This commit attempts to improve the state of things by factoring out (3), adding
documentation on how the expression list is built, and by reducing the scope of
variables.
Differential Revision: https://reviews.llvm.org/D154513
Fix incorrect uses of LLDB_LOG_ERROR. The macro doesn't automatically
inject the error in the log message: it merely passes the error as the
first argument to formatv and therefore must be referenced with {0}.
Thanks to Nicholas Allegra for collecting a list of places where the
macro was misused.
rdar://111581655
Differential revision: https://reviews.llvm.org/D154530
A very old commit (9422dd64f8) changed the signature of this function in a
number of ways. This patch aims to improve it:
1. Reword the documentation, which still mentions old parameters that no longer
exist, and to elaborate upon the behavior of this function.
2. Remove the unnecessary parameter `op_addr_idx`. This parameter is odd in a
couple of ways: we never use it with a value that is non-zero, and the matching
`Update_DW_OP_addr` function doesn't use a similar parameter. We also document
that this new behavior. If we ever decide to handle multiple "DW_OP_addr", we
can introduce the complexity again.
Differential Revision: https://reviews.llvm.org/D154265
LLDB's implementation of DWARFDataExtractor has a method that returns a
llvm::DWARFDataExtractor. In some cases, like DebugNamesDWARFIndex::Create, we
were passing an LLVM::DWARFDataExtractor to a function that expects a
LLVM:DataExtractor by value. This is causing slicing of the derived class.
While slicing is not inherently bad, it can be dangerous if the constructor of
the derived class mutates the base class in a way that leaves it in an invalid
state after slicing.
Differential Revision: https://reviews.llvm.org/D153913
Currently, the method `GetAttributeAddressRanges` takes a DWARFRangeList as a
parameter, just to immediately clear it. The method also returns the size of
this list. Such an API was obfuscating the intent of the call sites (it's not
clear from the method name what it returns) and it was obfuscating redundant
checks on the size of the list.
This commit refactors the method to return the list and to also make the call
sites use the more explicit `IsEmpty` method.
Differential Revision: https://reviews.llvm.org/D151451
The purpose of this method is to get the list of attributes of a
DebugInfoEntry. Prior to this change we were passing in a mutable
reference to a DWARFAttributes object and having the method fill it in
for us while returning the size of the filled out list. But
instead of doing that, we can just return a `DWARFAttributes` object
ourselves since every caller creates a new list before calling
GetAttributes.
Differential Revision: https://reviews.llvm.org/D150402
Use templates to simplify {Get,Set}PropertyAtIndex. It has always
bothered me how cumbersome those calls are when adding new properties.
After this patch, SetPropertyAtIndex infers the type from its arguments
and GetPropertyAtIndex required a single template argument for the
return value. As an added benefit, this enables us to remove a bunch of
wrappers from UserSettingsController and OptionValueProperties.
Differential revision: https://reviews.llvm.org/D149774
The majority of call sites are nullptr as the execution context.
Refactor OptionValueProperties to make the argument optional and
simplify all the callers.
Similar to fdbe7c7faa, refactor OptionValueProperties to return a
std::optional instead of taking a fail value. This allows the caller to
handle situations where there's no value, instead of being unable to
distinguish between the absence of a value and the value happening the
match the fail value. When a fail value is required,
std::optional::value_or() provides the same functionality.
These probably do not need to be in the ConstString StringPool as they
don't really need any of the advantages that ConstStrings offer.
Lifetime for these things is always static and we never need to perform
comparisons for setting descriptions.
Differential Revision: https://reviews.llvm.org/D148679
**Summary**
In a program such as:
```
namespace A {
namespace B {
struct Bar {};
}
}
namespace B {
struct Foo {};
}
```
...LLDB would run into issues such as:
```
(lldb) expr ::B::Foo f
error: expression failed to parse:
error: <user expression 0>:1:6: no type named 'Foo' in namespace 'A::B'
::B::Foo f
~~~~~^
```
This is because the `SymbolFileDWARF::FindNamespace` implementation
will return *any* namespace it finds if the `parent_decl_ctx` provided
is empty. In `FindExternalVisibleDecls` we use this API to find the
namespace that symbol `B` refers to. If `A::B` happened to be the one
that `SymbolFileDWARF::FindNamespace` looked at first, we would try
to find `struct Foo` in `A::B`. Hence the error.
This patch proposes a new `SymbolFileDWARF::FindNamespace` API that
will only find a match for top-level namespaces, which is what
`FindExternalVisibleDecls` is attempting anyway; it just never
accounted for multiple namespaces of the same name.
**Testing**
* Added API test-case
Differential Revision: https://reviews.llvm.org/D147436
Implement SymbolFile::GetCompileOptions, which returns a map from
compilation units to compilation arguments associated with that unit.
Differential Revision: https://reviews.llvm.org/D147748
