Commit Graph

901 Commits

Author SHA1 Message Date
Jim Ingham
37b8e5feb1 Revert "[lldb][DWARF] Delay struct/class/union definition DIE searching when parsing declaration DIEs. (#90663)"
This reverts commit 9a7262c260.
2024-05-13 17:47:49 -07:00
Zequan Wu
9a7262c260 [lldb][DWARF] Delay struct/class/union definition DIE searching when parsing declaration DIEs. (#90663)
This is the implementation for
https://discourse.llvm.org/t/rfc-delay-definition-die-searching-when-parse-a-declaration-die-for-record-type/78526.

#### Motivation
Currently, lldb eagerly searches for definition DIE when parsing a
declaration DIE for struct/class/union definition DIE. It will search
for all definition DIEs with the same unqualified name (just
`DW_AT_name` ) and then find out those DIEs with same fully qualified
name. Then lldb will try to resolve those DIEs to create the Types from
definition DIEs. It works fine most time. However, when built with
`-gsimple-template-names`, the search graph expands very quickly,
because for the specialized-template classes, they don’t have template
parameter names encoded inside `DW_AT_name`. They have
`DW_TAG_template_type_parameter` to reference the types used as template
parameters. In order to identify if a definition DIE matches a
declaration DIE, lldb needs to resolve all template parameter types
first and those template parameter types might be template classes as
well, and so on… So, the search graph explodes, causing a lot
unnecessary searching/type-resolving to just get the fully qualified
names for a specialized-template class. This causes lldb stack overflow
for us internally on template-heavy libraries.

#### Implementation
Instead of searching for definition DIEs when parsing declaration DIEs,
we always construct the record type from the DIE regardless if it's
definition or declaration. The process of searching for definition DIE
is refactored to `DWARFASTParserClang::FindDefinitionTypeForDIE` which
is invoked when 1) completing the type on
`SymbolFileDWARF::CompleteType`. 2) the record type needs to start its
definition as a containing type so that nested classes can be added into
it in `PrepareContextToReceiveMembers`.

The key difference is `SymbolFileDWARF::ResolveType` return a `Type*`
that might be created from declaration DIE, which means it hasn't starts
its definition yet. We also need to change according in places where we
want the type to start definition, like `PrepareContextToReceiveMembers`
(I'm not aware of any other places, but this should be a simple call to
`SymbolFileDWARF::FindDefinitionDIE`)

#### Result
It fixes the stack overflow of lldb for the internal binary built with
simple template name. When constructing the fully qualified name built
with `-gsimple-template-names`, it gets the name of the type parameter
by resolving the referenced DIE, which might be a declaration (we won't
try to search for the definition DIE to just get the name).
I got rough measurement about the time using the same commands (set
breakpoint, run, expr this, exit). For the binary built without
`-gsimple-template-names`, this change has no impact on time, still
taking 41 seconds to complete. When built with
`-gsimple-template-names`, it also takes about 41 seconds to complete
wit this change.
2024-05-10 12:26:52 -04:00
Alex Langford
e2b3e4ea9f [lldb][NFCI] Unify DW_TAG -> string conversions (#90657)
The high level goal is to have 1 way of converting a DW_TAG value into a
human-readable string.

There are 3 ways this change accomplishes that:
1.) Changing DW_TAG_value_to_name to not create custom error strings.
  The way it was doing this is error-prone: Specifically, it was using a
  function-local static char buffer and handing out a pointer to it.
  Initialization of this is thread-safe, but mutating it is definitely
  not. Multiple threads that want to call this function could step on
  each others toes. The implementation in this patch sidesteps the issue
  by just returning a StringRef with no mention of the tag value in it.
2.) Changing all uses of DW_TAG_value_to_name to log the value of the
  tag since the function doesn't create a string with the value in it
  anymore.
3.) Removing `DWARFBaseDIE::GetTagAsCString()`. Callers should call
  DW_TAG_value_to_name on the tag directly.
2024-05-03 11:05:11 -07:00
David Spickett
327bfc971e Revert "[lldb] Fix TestSharedLibStrippedSymbols for #90622"
And "LLDB Debuginfod tests and a fix or two (#90622)".

f8fedfb680 /
2d4acb0865

As it has caused a test failure on 32 bit Arm:
https://lab.llvm.org/buildbot/#/builders/17/builds/52580

Expr/TestStringLiteralExpr.test. The follow up did fix
lang/c/shared_lib_stripped_symbols/TestSharedLibStrippedSymbols.py
but not the other failure.
2024-05-03 13:04:05 +00:00
Kevin Frei
2d4acb0865 LLDB Debuginfod tests and a fix or two (#90622)
I'm taking yet another swing at getting these tests going, on the
hypothesis that the problems with buildbots & whatnot are because
they're not configured with CURL support, which I've confirmed would
cause the previous tests to fail. (I have no access to an ARM64 linux
system, but I did repro the failure on MacOS configured without CURL
support)

So, the only difference between this diff and
[previous](https://github.com/llvm/llvm-project/pull/85693)
[diffs](https://github.com/llvm/llvm-project/pull/87676) that have
already been approved is that I've added a condition to the tests to
only run if Debuginfod capabilities should be built into the binary. I
had done this for these tests when they were [Shell
tests](https://github.com/llvm/llvm-project/pull/79181) and not API
tests, but I couldn't find a direct analog in any API test, so I used
the "plugins" model used by the intel-pt tests as well.

---------

Co-authored-by: Kevin Frei <freik@meta.com>
2024-05-02 11:02:17 -07:00
Shubham Rastogi
ca55ee8826 Revert "Debuginfod Testing & fixes: 3rd times the charm? (#87676)"
This reverts commit d6713ad80d.

This changed was reverted because of greendragon failures such
as

Unresolved Tests (2):
  lldb-api :: debuginfod/Normal/TestDebuginfod.py
  lldb-api :: debuginfod/SplitDWARF/TestDebuginfodDWP.py
2024-04-04 15:37:07 -07:00
Kevin Frei
d6713ad80d Debuginfod Testing & fixes: 3rd times the charm? (#87676)
I believe I've got the tests properly configured to only run on Linux
x86(_64), as I don't have a Linux AArch64/Arm device to diagnose what's
going wrong with the tests (I suspect there's some issue with generating
`.note.gnu.build-id` sections...)

The actual code fixes have now been reviewed 3 times:
https://github.com/llvm/llvm-project/pull/79181 (moved shell tests to
API tests), https://github.com/llvm/llvm-project/pull/85693 (Changed
some of the testing infra), and
https://github.com/llvm/llvm-project/pull/86812 (didn't get the tests
configured quite right). The Debuginfod integration for symbol
acquisition in LLDB now works with the `executable` and `debuginfo`
Debuginfod network requests working properly for normal, `objcopy
--only-keep-debug` stripped, split-dwarf, and `objcopy
--only-keep-debug` stripped *plus* split-dwarf symbols/binaries.

The reasons for the multiple attempts have been tests on platforms I
don't have access to (Linux AArch64/Arm + MacOS x86_64). I believe I've
got the tests properly disabled for everything except for Linux x86(_64)
now. I've built & tested on MacOS AArch64 and Linux x86_64.

---------

Co-authored-by: Kevin Frei <freik@meta.com>
2024-04-04 11:43:55 -07:00
Chelsea Cassanova
20433e9b24 Revert "DebugInfoD issues, take 2" (#87583)
Reverts llvm/llvm-project#86812. 

This commit caused a regression on the x86_64 MacOS buildbot:
https://green.lab.llvm.org/job/llvm.org/view/LLDB/job/lldb-cmake/784/
2024-04-03 16:34:03 -07:00
Kevin Frei
a8425d2fa2 DebugInfoD issues, take 2 (#86812)
The previous diff (and it's subsequent fix) were reverted as the tests
didn't work properly on the AArch64 & ARM LLDB buildbots. I made a
couple more minor changes to tests (from @clayborg's feedback) and
disabled them for non Linux-x86(_64) builds, as I don't have the ability
do anything about an ARM64 Linux failure. If I had to guess, I'd say the
toolchain on the buildbots isn't respecting the `-Wl,--build-id` flag.
Maybe, one day, when I have a Linux AArch64 system I'll dig in to it.

From the reverted PR:

I've migrated the tests in my
https://github.com/llvm/llvm-project/pull/79181 from shell to API (at
@JDevlieghere's suggestion) and addressed a couple issues that were
exposed during testing.

The tests first test the "normal" situation (no DebugInfoD involvement,
just normal debug files sitting around), then the "no debug info"
situation (to make sure the test is seeing failure properly), then it
tests to validate that when DebugInfoD returns the symbols, things work
properly. This is duplicated for DWP/split-dwarf scenarios.

---------

Co-authored-by: Kevin Frei <freik@meta.com>
2024-04-03 12:15:41 -07:00
Greg Clayton
75f7d53f0b Fix DWARF locations when we have large .dwp files. (#87164)
We have the ability to load .dwp files with a .debug_info.dwo section
that exceeds 4GB. There were 4 locations that were using 32 bit offsets
and lengths to extract variable locations, and if a DIE was over the 4GB
barrier, we would truncate the block offset for the variable locations
and the variable expression would be garbage. This fixes the issues. It
isn't possible to add a test for this as we don't want to create a 4GB
.dwp file on test machines.
2024-03-31 12:08:59 -07:00
Zequan Wu
9f84594e4e [lldb][Dwarf] Add missing timer when parsing .debug_abbrev. (#86568)
The time spent on parsing `.debug_abbrev` is also part of debug info
parsing time.
2024-03-27 10:33:25 -04:00
Muhammad Omair Javaid
7fc2fbb3f1 Revert "DebugInfoD tests + fixing issues exposed by tests (#85693)"
This reverts commit 6d939a6ec6.

This broke following LLDB bots:
https://lab.llvm.org/buildbot/#/builders/96/builds/54867
https://lab.llvm.org/buildbot/#/builders/17/builds/50824
2024-03-23 03:31:56 +05:00
Kevin Frei
6d939a6ec6 DebugInfoD tests + fixing issues exposed by tests (#85693)
Finally getting back to Debuginfod tests:
I've migrated the tests in my [earlier
PR](https://github.com/llvm/llvm-project/pull/79181) from shell to API
(at @JDevlieghere's suggestion) and addressed a couple issues that came
about during testing.

The tests first test the "normal" situation (no DebugInfoD involvement,
just normal debug files sitting around), then the "no debug info"
situation (to make sure the test is seeing failure properly), then it
tests to validate that when Debuginfod returns the symbols, things work
properly. This is duplicated for DWP/split-dwarf scenarios.

---------

Co-authored-by: Kevin Frei <freik@meta.com>
2024-03-21 13:09:04 -07:00
Greg Clayton
f0eb0c5689 Don't require a UUID in a .dwp file. (#83935)
DWP files don't usually have a GNU build ID built into them. When
searching for a .dwp file, don't require a UUID to be in the .dwp file.
The debug info search information was checking for a UUID in the .dwp
file when debug info search paths were being used. This is now fixed by
not specifying the UUID in the ModuleSpec being used for the .dwp file
search.
2024-03-06 13:49:26 -08:00
Greg Clayton
5a45d32b5b [lldb] Add more ways to find the .dwp file. (#81067)
When using split DWARF we can run into many different ways to store
debug info:
- lldb loads `<exe>` which contains skeleton DWARF and needs to find
`<exe>.dwp`
- lldb loads `<exe>` which is stripped but has .gnu_debuglink pointing
to `<exe>.debug` with skeleton DWARF and needs to find `<exe>.dwp`
- lldb loads `<exe>` which is stripped but has .gnu_debuglink pointing
to `<exe>.debug` with skeleton DWARF and needs to find `<exe>.debug.dwp`
- lldb loads `<exe>.debug` and needs to find `<exe>.dwp`

Previously we only handled the first two cases. This patch adds support
for the latter two.
2024-02-20 18:17:01 -08:00
Wanyi
dd7386d85f [Reland] Report only loaded debug info in statistics dump (#81706) (#82207)
Updates:

- The previous patch changed the default behavior to not load dwos in
`DWARFUnit`
~~`SymbolFileDWARFDwo *GetDwoSymbolFile(bool load_all_debug_info =
false);`~~
`SymbolFileDWARFDwo *GetDwoSymbolFile(bool load_all_debug_info = true);`
- This broke some lldb-shell tests (see
https://green.lab.llvm.org/green/view/LLDB/job/as-lldb-cmake/16273/)
- TestDebugInfoSize.py
- with symbol on-demand, by default statistics dump only reports
skeleton debug info size
- `statistics dump -f` will load all dwos. debug info = skeleton debug
info + all dwo debug info

Currently running `statistics dump` will trigger lldb to load debug info
that's not yet loaded (eg. dwo files). Resulted in a delay in the
command return, which, can be interrupting.

This patch also added a new option `--load-all-debug-info` asking
statistics to dump all possible debug info, which will force loading all
debug info available if not yet loaded.
2024-02-19 00:33:23 -05:00
Jonas Devlieghere
339baae3e2 Revert "Report only loaded debug info in statistics dump (#81706)" (#82150)
This reverts commit 21ddd7ff2b because it
breaks a bunch of tests:

https://lab.llvm.org/buildbot/#/builders/68/builds/69018
https://green.lab.llvm.org/green/view/LLDB/job/as-lldb-cmake/16273
2024-02-17 21:38:11 -08:00
Wanyi
21ddd7ff2b Report only loaded debug info in statistics dump (#81706)
Currently running `statistics dump` will trigger lldb to load debug info
that's not yet loaded (eg. dwo files). Resulted in a delay in the
command return, which, can be interrupting.

This patch also added a new option `--load-all-debug-info` asking
statistics to dump all possible debug info, which will force loading all
debug info available if not yet loaded.
2024-02-17 14:38:18 -05:00
Greg Clayton
9258f3e692 [lldb] Fix a crash when using .dwp files and make type lookup reliable with the index cache (#79544)
When using split DWARF with .dwp files we had an issue where sometimes
the DWO file within the .dwp file would be parsed _before_ the skeleton
compile unit. The DWO file expects to be able to always be able to get a
link back to the skeleton compile unit. Prior to this fix, the only time
the skeleton compile unit backlink would get set, was if the unit
headers for the main executable have been parsed _and_ if the unit DIE
was parsed in that DWARFUnit. This patch ensures that we can always get
the skeleton compile unit for a DWO file by adding a function:

```
DWARFCompileUnit *DWARFUnit::GetSkeletonUnit();
```

Prior to this fix DWARFUnit had some unsafe accessors that were used to
store two different things:

```
  void *DWARFUnit::GetUserData() const;
  void DWARFUnit::SetUserData(void *d);
```

This was used by SymbolFileDWARF to cache the `lldb_private::CompileUnit
*` for a SymbolFileDWARF and was also used to store the `DWARFUnit *`
for SymbolFileDWARFDwo. This patch clears up this unsafe usage by adding
two separate accessors and ivars for this:
```
lldb_private::CompileUnit *DWARFUnit::GetLLDBCompUnit() const { return m_lldb_cu; }
void DWARFUnit::SetLLDBCompUnit(lldb_private::CompileUnit *cu) { m_lldb_cu = cu; }
DWARFCompileUnit *DWARFUnit::GetSkeletonUnit();
void DWARFUnit::SetSkeletonUnit(DWARFUnit *skeleton_unit);
```
This will stop anyone from calling `void *DWARFUnit::GetUserData()
const;` and casting the value to an incorrect value.

A crash could occur in `SymbolFileDWARF::GetCompUnitForDWARFCompUnit()`
when the `non_dwo_cu`, which is a backlink to the skeleton compile unit,
was not set and was NULL. There is an assert() in the code, and then the
code just will kill the program if the assert isn't enabled because the
code looked like:
```
  if (dwarf_cu.IsDWOUnit()) {
    DWARFCompileUnit *non_dwo_cu =
        static_cast<DWARFCompileUnit *>(dwarf_cu.GetUserData());
    assert(non_dwo_cu);
    return non_dwo_cu->GetSymbolFileDWARF().GetCompUnitForDWARFCompUnit(
        *non_dwo_cu);
  }
```
This is now fixed by calling the `DWARFUnit::GetSkeletonUnit()` which
will correctly always get the skeleton compile uint for a DWO file
regardless of if the skeleton unit headers have been parse or if the
skeleton unit DIE wasn't parsed yet.

To implement the ability to get the skeleton compile units, I added code
the DWARFDebugInfo.cpp/.h that make a map of DWO ID -> skeleton
DWARFUnit * that gets filled in for DWARF5 when the unit headers are
parsed. The `DWARFUnit::GetSkeletonUnit()` will end up parsing the unit
headers of the main executable to fill in this map if it already hasn't
been done. For DWARF4 and earlier we maintain a separate map that gets
filled in only for any DWARF4 compile units that have a DW_AT_dwo_id or
DW_AT_gnu_dwo_id attributes. This is more expensive, so this is done
lazily and in a thread safe manor. This allows us to be as efficient as
possible when using DWARF5 and also be backward compatible with DWARF4 +
split DWARF.

There was also an issue that stopped type lookups from succeeding in
`DWARFDIE SymbolFileDWARF::GetDIE(const DIERef &die_ref)` where it
directly was accessing the `m_dwp_symfile` ivar without calling the
accessor function that could end up needing to locate and load the .dwp
file. This was fixed by calling the
`SymbolFileDWARF::GetDwpSymbolFile()` accessor to ensure we always get a
valid value back if we can find the .dwp file. Prior to this fix it was
down which APIs were called and if any APIs were called that loaded the
.dwp file, it worked fine, but it might not if no APIs were called that
did cause it to get loaded.

When we have valid debug info indexes and when the lldb index cache was
enabled, this would cause this issue to show up more often.

I modified an existing test case to test that all of this works
correctly and doesn't crash.
2024-02-01 13:58:18 -08:00
jeffreytan81
e0e6236fd6 Fix debug info size statistics for split dwarf (#80218)
`statistics dump` command relies on `SymbolFile::GetDebugInfoSize()` to
get total debug info size.
The current implementation is missing debug info for split dwarf
scenarios which requires getting debug info from separate dwo/dwp files.
This patch fixes this issue for split dwarf by parsing debug info from
dwp/dwo.

New yaml tests are added.

---------

Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
2024-02-01 09:11:25 -08:00
Jonas Devlieghere
ff9fcc74a3 [lldb] Store SupportFile in CompileUnit (NFC)
Store a SupportFile, rather than a FileSpec, in CompileUnit. This commit
works towards having the SourceManager operate on SupportFiles so that
it can (1) validate the Checksum and (2) materialize the content of
inline source information.
2024-01-16 21:27:20 -08:00
Chelsea Cassanova
f1ef910b97 [lldb][Progress] Separate title and details (#77547)
Per this RFC:
https://discourse.llvm.org/t/rfc-improve-lldb-progress-reporting/75717
on improving progress reports, this commit separates the title field and
details field so that the title specifies the category that the progress
report falls under. The details field is added as a part of the
constructor for progress reports and by default is an empty string. In addition, changes the total amount of progress completed into a std::optional. Also
updates the test to check for details being correctly reported from the
event structured data dictionary.
2024-01-16 07:57:18 -08:00
Jonas Devlieghere
e27561fc7d [lldb] Move MD5 Checksum from FileSpec to SupportFile
When I added the MD5 checksum I was on the fence between storing it in
FileSpec or creating a new SupportFile abstraction. The latter was
deemed overkill for just the MD5 hashes, but support for inline sources
in the DWARF 5 line table tipped the scales. This patch moves the MD5
checksum into the new SupportFile class.
2024-01-12 13:08:24 -08:00
Adrian Prantl
fa9284589f [lldb] DWARFDIE: Follow DW_AT_specification when computing CompilerCo… (#77157)
…ntext

Following the specification chain seems to be clearly the expected
behavior of GetDeclContext(). Otherwise C++ methods have an empty
CompilerContext instead of being nested in their struct/class.

Theprimary motivation for this functionality is the Swift plugin. In
order to test the change I added a proof-of-concept implementation of a
Module::FindFunction() variant that takes a CompilerContext, expesed via
lldb-test.

rdar://120553412
2024-01-09 10:45:30 -08:00
Felipe de Azevedo Piovezan
b4ee7d6119 [lldb][DWARFIndex][nfc] Factor out fully qualified name query (#76977)
This moves the functionally of finding a DIE based on a fully qualified
name from SymbolFileDWARF into DWARFIndex itself, so that
specializations of DWARFIndex can implement faster versions of this
query.
2024-01-08 11:16:22 -03:00
Adrian Prantl
917b404e2c Add support for inline DWARF source files. (#75880)
LLVM supports DWARF 5 linetable extension to store source files inline
in DWARF. This is particularly useful for compiler-generated source
code. This implementation tries to materialize them as temporary files
lazily, so SBAPI clients don't need to be aware of them.

rdar://110926168
2024-01-04 09:04:05 -08:00
Kazu Hirata
744f38913f [lldb] Use StringRef::{starts,ends}_with (NFC)
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.

I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
2023-12-16 14:39:37 -08:00
Greg Clayton
dd95877958 [lldb] Make only one function that needs to be implemented when searching for types (#74786)
This patch revives the effort to get this Phabricator patch into
upstream:

https://reviews.llvm.org/D137900

This patch was accepted before in Phabricator but I found some
-gsimple-template-names issues that are fixed in this patch.

A fixed up version of the description from the original patch starts
now.

This patch started off trying to fix Module::FindFirstType() as it
sometimes didn't work. The issue was the SymbolFile plug-ins didn't do
any filtering of the matching types they produced, and they only looked
up types using the type basename. This means if you have two types with
the same basename, your type lookup can fail when only looking up a
single type. We would ask the Module::FindFirstType to lookup "Foo::Bar"
and it would ask the symbol file to find only 1 type matching the
basename "Bar", and then we would filter out any matches that didn't
match "Foo::Bar". So if the SymbolFile found "Foo::Bar" first, then it
would work, but if it found "Baz::Bar" first, it would return only that
type and it would be filtered out.

Discovering this issue lead me to think of the patch Alex Langford did a
few months ago that was done for finding functions, where he allowed
SymbolFile objects to make sure something fully matched before parsing
the debug information into an AST type and other LLDB types. So this
patch aimed to allow type lookups to also be much more efficient.

As LLDB has been developed over the years, we added more ways to to type
lookups. These functions have lots of arguments. This patch aims to make
one API that needs to be implemented that serves all previous lookups:

- Find a single type
- Find all types
- Find types in a namespace

This patch introduces a `TypeQuery` class that contains all of the state
needed to perform the lookup which is powerful enough to perform all of
the type searches that used to be in our API. It contain a vector of
CompilerContext objects that can fully or partially specify the lookup
that needs to take place.

If you just want to lookup all types with a matching basename,
regardless of the containing context, you can specify just a single
CompilerContext entry that has a name and a CompilerContextKind mask of
CompilerContextKind::AnyType.

Or you can fully specify the exact context to use when doing lookups
like: CompilerContextKind::Namespace "std"
CompilerContextKind::Class "foo"
CompilerContextKind::Typedef "size_type"

This change expands on the clang modules code that already used a
vector<CompilerContext> items, but it modifies it to work with
expression type lookups which have contexts, or user lookups where users
query for types. The clang modules type lookup is still an option that
can be enabled on the `TypeQuery` objects.

This mirrors the most recent addition of type lookups that took a
vector<CompilerContext> that allowed lookups to happen for the
expression parser in certain places.

Prior to this we had the following APIs in Module:

```
void
Module::FindTypes(ConstString type_name, bool exact_match, size_t max_matches,
                  llvm::DenseSet<lldb_private::SymbolFile *> &searched_symbol_files,
                  TypeList &types);

void
Module::FindTypes(llvm::ArrayRef<CompilerContext> pattern, LanguageSet languages,
                  llvm::DenseSet<lldb_private::SymbolFile *> &searched_symbol_files,
                  TypeMap &types);

void Module::FindTypesInNamespace(ConstString type_name,
                                  const CompilerDeclContext &parent_decl_ctx,
                                  size_t max_matches, TypeList &type_list);
```

The new Module API is much simpler. It gets rid of all three above
functions and replaces them with:

```
void FindTypes(const TypeQuery &query, TypeResults &results);
```
The `TypeQuery` class contains all of the needed settings:

- The vector<CompilerContext> that allow efficient lookups in the symbol
file classes since they can look at basename matches only realize fully
matching types. Before this any basename that matched was fully realized
only to be removed later by code outside of the SymbolFile layer which
could cause many types to be realized when they didn't need to.
- If the lookup is exact or not. If not exact, then the compiler context
must match the bottom most items that match the compiler context,
otherwise it must match exactly
- If the compiler context match is for clang modules or not. Clang
modules matches include a Module compiler context kind that allows types
to be matched only from certain modules and these matches are not needed
when d oing user type lookups.
- An optional list of languages to use to limit the search to only
certain languages

The `TypeResults` object contains all state required to do the lookup
and store the results:
- The max number of matches
- The set of SymbolFile objects that have already been searched
- The matching type list for any matches that are found

The benefits of this approach are:
- Simpler API, and only one API to implement in SymbolFile classes
- Replaces the FindTypesInNamespace that used a CompilerDeclContext as a
way to limit the search, but this only worked if the TypeSystem matched
the current symbol file's type system, so you couldn't use it to lookup
a type in another module
- Fixes a serious bug in our FindFirstType functions where if we were
searching for "foo::bar", and we found a "baz::bar" first, the basename
would match and we would only fetch 1 type using the basename, only to
drop it from the matching list and returning no results
2023-12-12 16:51:49 -08:00
Felipe de Azevedo Piovezan
162248c22d [SymbolFileDWARF][NFC] Remove duplicated code checking for type tags (#74773)
There was duplicated (and complex) code querying whether tags were
type-like tags (i.e. class or struct); this has been factored out into a
helper function.

There was also a comment about not comparing identical DIEs without ever
performing that check; this comment has been removed. It was likely a
result of copy paste from another function in this same file which
actually does that check.
2023-12-11 08:01:04 -03:00
Felipe de Azevedo Piovezan
9982f8ee02 [lldb][SymbolFileDWARF][NFC] Remove unnecessary calls to GetDWARFDeclContext (#74523)
The function FindDefinitionTypeForDWARFDeclContext loops over all DIEs
corresponding to types with a certain name and compares the context of
each found DIE with the context of a target DIE. However, the target DIE
never changes throughout this search, and yet we recompute its
DeclContext on every iteration of the search. This is wasteful because
the method is not exactly free (see
DWARFDebugInfoEntry::GetDWARFDeclContextStatic).
2023-12-06 10:42:42 -08:00
Kevin Frei
c43c86c285 DEBUGINFOD based DWP acquisition for LLDB (#70996)
I've plumbed the LLVM DebugInfoD client into LLDB, and added automatic
downloading of DWP files to the SymbolFileDWARF.cpp plugin. If you have
DEBUGINFOD_URLS set to a space delimited set of web servers, LLDB will
try to use them as a last resort when searching for DWP files. If you do
*not* have that environment variable set, nothing should be changed.
There's also a setting, per @clayborg 's suggestion, that will override
the environment variable, or can be used instead of the environment
variable. The setting is why I also needed to add an API to the
llvm-debuginfod library

### Test Plan:

Suggestions are welcome here. I should probably have some positive and
negative tests, but I wanted to get the diff up for people who have a
clue what they're doing to rip it to pieces before spending too much
time validating the initial implementation.

---------

Co-authored-by: Kevin Frei <freik@meta.com>
Co-authored-by: Alex Langford <nirvashtzero@gmail.com>
2023-12-04 11:45:40 -08:00
Michael Buch
15c8085202 Reland "[lldb][DWARFASTParserClang] Fetch constant value from variable defintion if available" (#71800)
This patch relands https://github.com/llvm/llvm-project/pull/71004 which
was reverted because the clang change it depends on was reverted.

In addition to the original patch, this PR includes a change to
`SymbolFileDWARF::ParseVariableDIE` to support CU-level variable
definitions that don't have locations, but represent a constant value.
Previously, when debug-maps were available, we would assume that a
variable with "static lifetime" (which in this case means "has a linkage
name") has a valid address, which isn't the case for non-locationed
constants. We could omit this additional change if we stopped attaching
linkage names to global non-locationed constants.

Original commit message:
"""
https://github.com/llvm/llvm-project/pull/71780 proposes moving the
`DW_AT_const_value` on inline static members from the declaration DIE to
the definition DIE. This patch makes sure the LLDB's expression
evaluator can continue to support static initialisers even if the
declaration doesn't have a `DW_AT_const_value` anymore.

Previously the expression evaluator would find the constant for a
VarDecl from its declaration `DW_TAG_member` DIE. In cases where the
initialiser was specified out-of-class, LLDB could find it during symbol
resolution.

However, neither of those will work for constants, since we don't have a
constant attribute on the declaration anymore and we don't have
constants in the symbol table.
"""

Depends on:
* https://github.com/llvm/llvm-project/pull/71780
2023-11-13 06:09:58 +00:00
Jonas Devlieghere
64f62de966 [lldb] Read Checksum from DWARF line tables (#71458)
Read the MD5 checksum from DWARF line tables and store it in the
corresponding support files.

This is a re-land after fixing an off-by-one error in LLDB's
ParseSupportFilesFromPrologue (fixed in #71984).
2023-11-10 14:43:47 -08:00
Jonas Devlieghere
fa7e07ed99 [lldb] Fix a off-by-one error in ParseSupportFilesFromPrologue (#71984)
This fixes a subtle and previously harmless off-by-one bug in
ParseSupportFilesFromPrologue. The function accounts for the start index
being one-based for DWARF v4 and earlier and zero-based for DWARF v5 and
later. However, the same care wasn't taken for the end index.

This bug existed unnoticed because GetFileByIndex gracefully handles an
invalid index. However, the bug manifested itself after #71458, which
added a call to getFileNameEntry which requires the index to be valid.

No test as the bug cannot be observed without the changes from #71458.
Once that PR is merged, this will be covered by all the DWARF v5 tests.
2023-11-10 14:03:29 -08:00
Jonas Devlieghere
73519ba27a Revert "[lldb] Read Checksum from DWARF line tables" (#71864)
Reverts llvm/llvm-project#71458 as it might have caused
cross-project-test failures.
2023-11-09 12:43:53 -08:00
Jonas Devlieghere
5da98dec7a [lldb] Read Checksum from DWARF line tables (#71458)
Read the MD5 checksum from DWARF line tables and store it in the
corresponding support files.
2023-11-09 08:59:03 -08:00
Jonas Devlieghere
745e8bfd1a [lldb] Remove LocateSymbolFile (#71301)
This completes the conversion of LocateSymbolFile into a SymbolLocator
plugin. The only remaining function is DownloadSymbolFileAsync which
doesn't really fit into the plugin model, and therefore moves into the
SymbolLocator class, while still relying on the plugins to do the
underlying work.
2023-11-05 08:26:42 -08:00
Jonas Devlieghere
19df9aa3f4 [lldb] Move LocateExecutableSymbolFile to SymbolLocator plugin (#71266)
This builds on top of the work started in c3a302d to convert
LocateSymbolFile to a SymbolLocator plugin. This commit moves
LocateExecutableSymbolFile.
2023-11-03 19:48:36 -07:00
Tom Yang
9e0a5be0de [lldb][split-dwarf] Add --errors-only argument separate-debug-info list (#71000)
Often, we only care about the split-dwarf files that have failed to
load. This can be useful when diagnosing binaries with many separate
debug info files where only some have errors.

```
(lldb) help image dump separate-debug-info
List the separate debug info symbol files for one or more target modules.

Syntax: target modules dump separate-debug-info <cmd-options> [<filename> [<filename> [...]]]

Command Options Usage:
  target modules dump separate-debug-info [-ej] [<filename> [<filename> [...]]]

       -e ( --errors-only )
            Filter to show only debug info files with errors.

       -j ( --json )
            Output the details in JSON format.

     This command takes options and free-form arguments.  If your arguments
     resemble option specifiers (i.e., they start with a - or --), you must use
     ' -- ' between the end of the command options and the beginning of the
     arguments.

'image' is an abbreviation for 'target modules'
```

I updated the following tests
```
# on Linux
bin/lldb-dotest -p TestDumpDwo

# on Mac
bin/lldb-dotest -p TestDumpOso
```

This change applies to both the table and JSON outputs.

---------

Co-authored-by: Tom Yang <toyang@fb.com>
2023-11-02 11:36:24 -07:00
Walter Erquinigo
10508b6db7 [LLDB][NFC] Remove DWARFASTParserClang as friend from SymbolFileDWARF (#70157)
This effectively moves a few functions from protected to public. In any
case, for the sake of having a cleaner SymbolFileDWARF API, it's better
if it's not a friend of a one of its consumers, DWARFASTParserClang.
Another effect of this change is that I can use SymbolFileDWARF for the
out-of-tree mojo dwarf parser, which relies on pretty much the same
functions that DWARFASTParserClang needs from SymbolFileDWARF.
2023-10-25 18:04:25 -04:00
Adrian Prantl
49504674db Expose DWARFDIE::GetDeclContext() in lldb_private::Function. (#69981)
I need this API in the Swift plugin, but it seems generally useful
enough to expose it in the main branch.
2023-10-24 10:55:23 -07:00
Tom Yang
74ca07295f [lldb] improve dwo path in missing dwo error when relative (#69783)
When the debug info refers to a dwo with relative `DW_AT_comp_dir` and
`DW_AT_dwo_name`, we only print the `DW_AT_comp_dir` in our error
message if we can't find it. This often isn't very helpful, especially
when the `DW_AT_comp_dir` is ".":
```
(lldb) fr v
error: unable to locate .dwo debug file "." for skeleton DIE 0x000000000000003c
```

I'm updating the error message to include both `DW_AT_comp_dir` (if it
exists) and `DW_AT_dwo_name` when the `DW_AT_dwo_name` is relative. The
behavior when `DW_AT_dwo_name` is absolute should be the same.
2023-10-21 16:59:44 -07:00
Walter Erquinigo
1673a1ba5d [LLDB][NFC] Create a namespace for the DWARF plugin (#68150)
As a followup of https://github.com/llvm/llvm-project/pull/67851, I'm
defining a new namespace `lldb_plugin::dwarf` for the classes in this
Plugins/SymbolFile/DWARF folder. This change is very NFC and helped me
with exporting the necessary symbols for my out-of-tree language plugin.
The only class that I didn't change is ClangDWARFASTParser, because that
shouldn't be in the same namespace as the generic language-agnostic
dwarf parser.
It would be a good idea if other plugins follow the same namespace
scheme.
2023-10-13 16:51:24 -04:00
Tom Yang
64d78d8b3c Add target modules dump separate-debug-info (#66035)
Add a new command
```
target modules dump separate-debug-info [-j] [<filename> [<filename> [...]]]
```
or
```
image dump separate-debug-info [-j] [<filename> [<filename> [...]]]
```
(since `image` is an alias for `target modules`).
This lists the separate debug info files and their current status
(loaded or not loaded) for the specified modules. This diff implements
this command for mach-O files with OSO and ELF files with dwo.
Example dwo:
```
(lldb) image dump separate-debug-info
Symbol file: /home/toyang/workspace/dwo-scratch/a.out
Type: "dwo"
Dwo ID             Err Dwo Path
------------------ --- -----------------------------------------
0x9a429da5abb6faae     /home/toyang/workspace/scratch-dwo/a-main.dwo
0xbcc129959e76ff33     /home/toyang/workspace/scratch-dwo/a-foo.dwo

(lldb) image dump separate-debug-info -j
[
  {
    "separate-debug-info-files": [
      {
        "comp_dir": "/home/toyang/workspace/dwo-scratch",
        "dwo_id": 11115620165179865774,
        "dwo_name": "a-main.dwo",
        "loaded": true,
        "resolved_dwo_path": "/home/toyang/workspace/dwo-scratch/a-main.dwo"
      },
      {
        "comp_dir": "/home/toyang/workspace/dwo-scratch",
        "dwo_id": 13601198072221073203,
        "dwo_name": "a-foo.dwo",
        "loaded": true,
        "resolved_dwo_path": "/home/toyang/workspace/dwo-scratch/a-foo.dwo"
      }
    ],
    "symfile": "/home/toyang/workspace/dwo-scratch/a.out",
    "type": "dwo"
  }
]
```
Example dwo with missing dwo:
```
(lldb) image dump separate-debug-info
Symbol file: /home/toyang/workspace/dwo-scratch/a.out
Type: "dwo"
Dwo ID             Err Dwo Path
------------------ --- -----------------------------------------
0x9a429da5abb6faae E   unable to locate .dwo debug file "/home/toyang/workspace/scratch-dwo/b.out-main.dwo" for skeleton DIE 0x0000000000000014
0xbcc129959e76ff33 E   unable to locate .dwo debug file "/home/toyang/workspace/scratch-dwo/b.out-foo.dwo" for skeleton DIE 0x000000000000003c

(lldb) image dump separate-debug-info -j
[
  {
    "separate-debug-info-files": [
      {
        "comp_dir": "/home/toyang/workspace/dwo-scratch",
        "dwo_id": 11115620165179865774,
        "dwo_name": "a-main.dwo",
        "error": "unable to locate .dwo debug file \"/home/toyang/workspace/dwo-scratch/a-main.dwo\" for skeleton DIE 0x0000000000000014",
        "loaded": false
      },
      {
        "comp_dir": "/home/toyang/workspace/dwo-scratch",
        "dwo_id": 13601198072221073203,
        "dwo_name": "a-foo.dwo",
        "error": "unable to locate .dwo debug file \"/home/toyang/workspace/dwo-scratch/a-foo.dwo\" for skeleton DIE 0x000000000000003c",
        "loaded": false
      }
    ],
    "symfile": "/home/toyang/workspace/dwo-scratch/a.out",
    "type": "dwo"
  }
]
```
Example output with dwp:
```
(lldb) image dump separate-debug-info
Symbol file: /home/toyang/workspace/dwo-scratch/a.out
Type: "dwo"
Dwo ID             Err Dwo Path
------------------ --- -----------------------------------------
0x9a429da5abb6faae     /home/toyang/workspace/dwo-scratch/a.out.dwp(a-main.dwo)
0xbcc129959e76ff33     /home/toyang/workspace/dwo-scratch/a.out.dwp(a-foo.dwo)
(lldb) image dump separate-debug-info -j
[
  {
    "separate-debug-info-files": [
      {
        "comp_dir": "/home/toyang/workspace/dwo-scratch",
        "dwo_id": 11115620165179865774,
        "dwo_name": "a-main.dwo",
        "loaded": true,
        "resolved_dwo_path": "/home/toyang/workspace/dwo-scratch/a.out.dwp"
      },
      {
        "comp_dir": "/home/toyang/workspace/dwo-scratch",
        "dwo_id": 13601198072221073203,
        "dwo_name": "a-foo.dwo",
        "loaded": true,
        "resolved_dwo_path": "/home/toyang/workspace/dwo-scratch/a.out.dwp"
      }
    ],
    "symfile": "/home/toyang/workspace/dwo-scratch/a.out",
    "type": "dwo"
  }
]
```
Example oso on my Mac:
```
(lldb) image dump separate-debug-info
Symbol file: /Users/toyang/workspace/scratch/a.out
Type: "oso"
Mod Time           Err Oso Path
------------------ --- ---------------------
0x0000000064e64868     /Users/toyang/workspace/scratch/foo.a(foo.o)
0x0000000064e64868     /Users/toyang/workspace/scratch/foo.a(main.o)

(lldb) image dump separate-debug-info -j
[
  {
    "separate-debug-info-files": [
      {
        "loaded": true,
        "oso_mod_time": 1692813416,
        "oso_path": "/Users/toyang/workspace/scratch/foo.a(foo.o)",
        "so_file": "/Users/toyang/workspace/scratch/foo.cpp"
      },
      {
        "loaded": true,
        "oso_mod_time": 1692813416,
        "oso_path": "/Users/toyang/workspace/scratch/foo.a(main.o)",
        "so_file": "/Users/toyang/workspace/scratch/main.cpp"
      }
    ],
    "symfile": "/Users/toyang/workspace/scratch/a.out",
    "type": "oso"
  }
]
```

Test Plan:
Tested on Mac OS and Linux.
```
lldb-dotest -p TestDumpDwo
lldb-dotest -p TestDumpOso
```

---------

Co-authored-by: Tom Yang <toyang@fb.com>
2023-10-12 11:21:53 -07:00
Alex Langford
cdd3e964f2 [lldb] Replace lldb's DWARFDebugAbbrev implementation with llvm's (#67841)
The implementations are now close enough that replacing it is trivial.
2023-10-02 10:46:16 -07:00
Alex Langford
36a518317f [lldb][NFCI] Move functionality for getting unsupported DW_FORM values (#67579)
The LLVM implementation of DWARFDebugAbbrev does not have a way of
listing all the DW_FORM values that have been parsed but are unsupported
or otherwise unknown. AFAICT this functionality does not exist in LLVM
at all. Since my primary goal is to unify the implementations and not
judge the usefulness or completeness of this functionality, I decided to
move it out of LLDB's implementation of DWARFDebugAbbrev for the time
being.
2023-09-28 12:18:08 -07:00
Alex Langford
d3505c28a7 [lldb] Modify the DWARFDebugAbbrev interface to be closer to LLVM's (#67190)
I want to work towards unifying the implementations. It would be a lot
easier to do if LLDB's DWARFDebugAbbrev looked more similar to LLVM's
implementation, so this change moves in that direction.
2023-09-25 13:11:39 -07:00
David Spickett
b1f14d6473 Reland "[lldb] Add more ways to find split DWARF files"
This reverts commit dc3f758ddc.

Lit decided to show me the least interesting part of the
test output, but from what I gather on Mac OS the DWARF
stays in the object files (https://stackoverflow.com/a/12827463).

So either split DWARF options do nothing or they produce
files I don't know the name of that aren't .dwo, so I'm
skipping these tests on Darwin.
2023-09-08 08:15:08 +00:00
David Spickett
dc3f758ddc Revert "[lldb] Add more ways to find split DWARF files"
This reverts commit a723694321.

Tests are failing on x86_64 MacOS.
2023-09-08 08:00:38 +00:00
David Spickett
a723694321 [lldb] Add more ways to find split DWARF files
Fixes #28667

There's a bunch of ways to end up building split DWARF where the
DWO file is not next to the program file. On top of that you may
distribute the program in various ways, move files about, switch
machines, flatten the directories, etc.

This change adds a few more strategies to find DWO files:
* Appending the DW_AT_COMP_DIR and DWO name to all the debug
  search paths.
* Appending the same to the binary's dir.
* Appending the DWO name (e.g. a/b/foo.dwo) to all the debug
  search paths.
* Appending the DWO name to the binary's location.
* Appending the DWO filename (e.g. foo.dwo) to the debug
  search paths.
* Appending the DWO filename to the binary's location.

They are applied in that order and some will be skipped
if the DW_AT_COMP_DIR is relative or absolute, same for
the DWO name (though that seems to always be relative).

This uses the setting target.debug-file-search-paths, which
is used for DWP files already.

The added tests likely do not cover every part of the
strategies listed, it's a best effort.

Reviewed By: clayborg

Differential Revision: https://reviews.llvm.org/D157609
2023-09-07 08:19:11 +00:00