This patch consumes the `DW_AT_APPLE_enum_kind` attribute added in
https://github.com/llvm/llvm-project/pull/124752 and turns it into a
Clang attribute in the AST. This will currently be used by the Swift
language plugin when it creates `EnumDecl`s from debug-info and passes
it to Swift compiler, which expects these attributes
The `DWARFASTParserClang` reads enum values as `int64_t`s regardless of
the enumerators signedness. Then we pass it to
`AddEnumerationValueToEnumerationType` and only then do we create an
`APSInt` from it. However, there are other places where we read/pass
around the enum value as unsigned. This patch makes sure we consistently
use the same integer type for the enum value and let `APSInt` take care
of signedness. This shouldn't have any observable effect.
I tried using `CompleteEnumType` to replace some duplicated code in
`DWARFASTParserClang::ParseEnum` but tests started failing.
`CompleteEnumType` parses/attaches the child enumerators using the
signedness it got from `CompilerType::IsIntegerType`. However, this
would only report the correct signedness for builtin integer types
(never for `clang::EnumType`s). We have a different API for that in
`CompilerType::IsIntegerOrEnumerationType` which could've been used
there instead. This patch calls `IsEnumerationIntegerTypeSigned` to
determine signedness because we always pass an enum type into
`CompleteEnumType` anyway.
Based on git history this has been the case for a long time, but
possibly never caused issues because `ParseEnum` was completing the
definition manually instead of through `CompleteEnumType`.
I couldn't find a good way to test `CompleteEnumType` on its own because
it expects an enum type to be passed to it, which only gets created in
`ParseEnum` (at which point we already call `CompleteEnumType`). The
only other place we call `CompleteEnumType` at is in
[`CompleteTypeFromDWARF`](466217eb03/lldb/source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.cpp (L2260-L2262)).
Though I think we don't actually ever end up calling into that codepath
because we eagerly complete enum definitions. Maybe we can remove that
call to `CompleteEnumType` in a follow-up.
The purpose of this originally was to check for DWARF which refers to
garbage-collected functions (by checking whether we're able to get a
good address out of the function). The address check has been removed in
https://reviews.llvm.org/D112310, so the code computing it is not doing
anything.
While sifting through this part of the code I noticed that when we parse
C++ methods, `DWARFASTParserClang` creates two sets of `ParmVarDecls`,
one in `ParseChildParameters` and once in `AddMethodToCXXRecordType`.
The former is unused when we're dealing with methods. Moreover, the
`ParmVarDecls` we created in `ParseChildParameters` were created with an
incorrect `clang::DeclContext` (namely the DeclContext of the function,
and not the function itself). In Clang, there's
`ParmVarDecl::setOwningFunction` to adjust the DeclContext of a
parameter if the parameter was created before the FunctionDecl. But we
never used it.
This patch removes the `ParmVarDecl` creation from
`ParseChildParameters` and instead creates a
`TypeSystemClang::CreateParameterDeclarations` that ensures we set the
DeclContext correctly.
Note there is one differences in how `ParmVarDecl`s would be created
now: we won't set a ClangASTMetadata entry for any of the parameters. I
don't think this was ever actually useful for parameter DIEs anyway.
This wasn't causing any concrete issues (that I know of), but was quite
surprising. And this way of setting the parameters seems easier to
reason about (in my opinion).
This is the behavior expected by DWARF. It also requires some fixups to
algorithms which were storing the addresses of some objects (Blocks and
Variables) relative to the beginning of the function.
There are plenty of things that still don't work in this setups, but
this change is sufficient for the expression evaluator to correctly
recognize the entry point of a function in this case.
Reverts llvm/llvm-project#124096
Broke linux CI:
```
Note: This is test shard 7 of 42.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from DWARFASTParserClangTests
[ RUN ] DWARFASTParserClangTests.TestParseSubroutine_ExplicitObjectParameter
Expected<T> must be checked before access or destruction.
Expected<T> value was in success state. (Note: Expected<T> values in success mode must still be checked prior to being destroyed).
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0 SymbolFileDWARFTests 0x0000560271ee5ba7
1 SymbolFileDWARFTests 0x0000560271ee3a2c
2 SymbolFileDWARFTests 0x0000560271ee63ea
3 libc.so.6 0x00007f3e54e5b050
4 libc.so.6 0x00007f3e54ea9e2c
5 libc.so.6 0x00007f3e54e5afb2 gsignal + 18
6 libc.so.6 0x00007f3e54e45472 abort + 211
7 SymbolFileDWARFTests 0x0000560271e79d51
8 SymbolFileDWARFTests 0x0000560271e724f7
9 SymbolFileDWARFTests 0x0000560271f39e2c
10 SymbolFileDWARFTests 0x0000560271f3b368
11 SymbolFileDWARFTests 0x0000560271f3c053
12 SymbolFileDWARFTests 0x0000560271f4cf67
13 SymbolFileDWARFTests 0x0000560271f4c18a
14 SymbolFileDWARFTests 0x0000560271f2561c
15 libc.so.6 0x00007f3e54e4624a
16 libc.so.6 0x00007f3e54e46305 __libc_start_main + 133
17 SymbolFileDWARFTests 0x0000560271e65161
```
LLDB deduces the CV-qualifiers and storage class of a C++ method from
the object parameter. Currently it assumes that parameter is implicit
(and is a pointer type with the name "this"). This isn't true anymore in
C++23 with explicit object parameters. To support those we can simply
check the `DW_AT_object_pointer` of the subprogram DIE (works for both
declarations and definitions) when searching for the object parameter.
We can also remove the check for `eEncodingIsPointerUID`, because in C++
an artificial parameter called `this` is only ever the implicit object
parameter (at least for all the major compilers).
This patch continues simplifying `ParseChildParameters` by moving out
the logic that parses the first parameter of a function DIE into a
helper function. Since with GCC (and lately Clang) function declarations
have `DW_AT_object_pointer`s, we should be able to check for the
attribute's existence to determine if a function is static (and also
deduce CV-qualifiers from it). This will be useful for cases where the
object parameter is explicit (which is possible since C++23).
This should be NFC. I added a FIXME to places where we assume an
implicit object parameter (which will be addressed in a follow-up
patch).
We used to guard parsing of the CV-qualifiers of the "this" parameter
with a `encoding_mask & Type::eEncodingIsPointerUID`, which is
incorrect, because `eEncodingIsPointerUID` cannot be used as a bitmask
directly (see https://github.com/llvm/llvm-project/issues/120856). This
patch corrects this, but it should still be NFC because any parameter in
C++ called "this" *is* an implicit object parameter.
This patch refactors `ParseChildParameters` in a way which makes it (in
my opinion) more readable, removing some redundant local variables in
the process and reduces the scope of some variables.
**Motivation**
Since `DW_AT_object_pointer`s are now attached to declarations, we can
test for their existence to check whether a C++ method is static or not
(whereas currently we're deducing this from `ParseChildParameters` based
on some heuristics we know are true for most compilers). So my plan is
to move the code for determining `type_quals` and `is_static` out of
`ParseChildParameters`. The refactoring in this PR will make this
follow-up patch hopefully easier to review.
**Testing**
* This should be NFC. The main change is that we now no longer iterate
over `GetAttributes()` but instead retrieve the name, type and
is_artificial attributes of the parameters individually.
In https://github.com/llvm/llvm-project/pull/122742 we will start
attaching DW_AT_object_pointer to method declarations (in addition to
definitions).
Currently when LLDB parses a `DW_TAG_subprogram` definition, it will
parse all the attributes of the declaration as well. If we have
`DW_AT_object_pointer` on both, then we would overwrite the more
specific attribute that we got from the defintion with the one from the
specification. This is problematic because LLDB relies on getting the
`DW_AT_name` from the `DW_AT_object_pointer`, which doesn't exist on the
specification.
Note GCC does attach `DW_AT_object_pointer` on declarations *and*
definitions already (see https://godbolt.org/z/G1GvddY48), so there's
definitely some expressions that will fail for GCC compiled binaries.
This patch will fix those cases (e.g., I would expect `TestConstThis.py`
to fail with GCC).
`GetAttributes` returns all attributes on a given DIE, including any
attributes that the DIE references via `DW_AT_abstract_origin` and
`DW_AT_specification`. However, if an attribute exists on both the
referring DIE and the referenced DIE, the first one encountered will be
the one that takes precendence when querying the returned
`DWARFAttributes`. But there was no guarantee in which order those
attributes get visited. That means there's no convenient way of ensuring
that an attribute of a definition doesn't get shadowed by one found on
the declaration. One use-case where we don't want this to happen is for
`DW_AT_object_pointer` (which can exist on both definitions and
declarations, see https://github.com/llvm/llvm-project/pull/123089).
This patch makes sure we visit the current DIE's attributes before
following DIE references. I tried keeping as much of the original
`GetAttributes` unchanged and just add an outer `GetAttributes` that
keeps track of the DIEs we need to visit next.
There's precendent for this iteration order in
`llvm::DWARFDie::findRecursively` and also
`lldb_private::ElaboratingDIEIterator`. We could use the latter to
implement `GetAttributes`, though it also follows `DW_AT_signature` so I
decided to leave it for follow-up.
Anonymous namespaces are supposed to be optional when looking up types.
This was not working in combination with -gsimple-template-names,
because the way it was constructing the complete (with template args)
name scope (i.e., by generating thescope as a string and then reparsing
it) did not preserve the information about the scope kinds.
Essentially what the code wants here is to call `GetTypeLookupContext`
(that's the function used to get the context in the "regular" code
path), but to embelish each name with the template arguments (if they
don't have them already). This PR implements exactly that by adding an
argument to control which kind of names are we interested in. This
should also make the lookup faster as it avoids parsing of the long
string, but I haven't attempted to benchmark that.
I believe this function can also be used in some other places where
we're manually appending template names, but I'm leaving that for
another patch.
This doesn't make much of a difference now, but it makes it easier to
add -gsimple-template-names support to these functions (the idea is to
add an argument to say whether you want the name as spelled in the debug
info, or the one embellished with template arguments -- we have use
cases for both).
Many calls to Function::GetAddressRange() were not interested in the
range itself. Instead they wanted to find the address of the function
(its entry point) or the base address for relocation of function-scoped
entities (technically, the two don't need to be the same, but there's
isn't good reason for them not to be). This PR creates a separate
function for retrieving this, and changes the existing
(non-controversial) uses to call that instead.
The functions call GetName for everything except variables, where they
call GetPubname instead. The difference is that the latter prefers to
return the linkage name, if it is available.
This doesn't seem particularly useful given that the linkage name
already kind of contains the context of the variable, and I doubt that
anything depends on it as these functions are currently called on type
and subprogram DIEs -- not variables.
This makes it easier to simplify/deduplicate these functions later.
This addresses an issue encountered when investigating
https://github.com/llvm/llvm-project/pull/120569.
In `ParseTypeFromDWARF`, we insert the parsed type into
`UniqueDWARFASTTypeMap` using the typename and `Declaration` obtained
with `GetUniqueTypeNameAndDeclaration`. For C++
`GetUniqueTypeNameAndDeclaration` will zero the `Declaration` info
(presumably because forward declaration may not have it, and for C++,
ODR means we can rely on fully qualified typenames for uniqueing). When
we then called `CompleteType`, we would first `MapDeclDIEToDefDIE`,
updating the `UniqueDWARFASTType` we inserted previously with the
`Declaration` of the definition DIE (without zeroing it). We would then
call into `ParseTypeFromDWARF` for the same type again, and search the
`UniqueDWARFASTTypeMap`. But we do the search using a zeroed declaration
we get from `GetUniqueTypeNameAndDeclaration`.
This particular example was fixed by
https://github.com/llvm/llvm-project/pull/120569 but this still feels a
little inconsistent. I'm not entirely sure we can get into that
situation after that patch anymore. So the only test-case I could come
up with was the unit-test which calls `MapDeclDIEToDefDIE` directly.
The problem here manifests as follows:
1. We are stopped in main.o, so the first `ParseTypeFromDWARF` on
`FooImpl<char>` gets called on `main.o`'s SymbolFile. This adds a
mapping from *declaration die* -> `TypeSP` into `main.o`'s
`GetDIEToType` map.
2. We then `CompleteType(FooImpl<char>)`. Depending on the order of
entries in the debug-map, this might call `CompleteType` on `lib.o`'s
SymbolFile. In which case, `GetDIEToType().lookup(decl_die)` will return
a `nullptr`. This is already a bit iffy because some of the surrounding
code assumes we don't call `CompleteTypeFromDWARF` with a `nullptr`
`Type*`. E.g., `CompleteEnumType` blindly dereferences it (though enums
will never encounter this because their definition is fetched in
ParseEnum, unlike for structures).
3. While in `CompleteTypeFromDWARF`, we call `ParseTypeFromDWARF` again.
This will parse the member function `FooImpl::Create` and its return
type which is a typedef to `FooImpl*`. But now we're inside `lib.o`'s
SymbolFile, so we call it on the definition DIE. In step (2) we just
inserted a `nullptr` into `GetDIEToType` for the definition DIE, so we
trivially return a `nullptr` from `ParseTypeFromDWARF`. Instead of
reporting back this parse failure to the user LLDB trucks on and marks
`FooImpl::Ref` to be `void*`.
This test-case will trigger an assert in `TypeSystemClang::VerifyDecl`
even if we just `frame var` (but only in debug-builds). In release
builds where this function is a no-op, we'll create an incorrect Clang
AST node for the `Ref` typedef.
The proposed fix here is to share the `GetDIEToType` map between
SymbolFiles if a debug-map exists.
**Alternatives considered**
* Check the `GetDIEToType` map of the `SymbolFile` that the declaration
DIE belongs to. The assumption here being that if we called
`ParseTypeFromDWARF` on a declaration, the `GetDIEToType` map that the
result was inserted into was the one on that DIE's SymbolFile. This was
the first version of this patch, but that felt like a weaker version
sharing the map. It complicates the code in `CompleteType` and is less
consistent with the other bookkeeping structures we already share
between SymbolFiles
* Return from `SymbolFileDWARF::CompleteType` if there is no type in the
current `GetDIEToType`. Then `SymbolFileDWARFDebugMap::CompleteType`
could continue to the next `SymbolFile` which does own the type. But
that didn't quite work because we remove the
`GetForwardCompilerTypeToDie` entry in `SymbolFile::CompleteType`, which
`SymbolFileDWARFDebugMap::CompleteType` relies upon for iterating
The `llvm-gcc` front-end has been EOL'd at least since 2011 (based on
some `git` archeology). And Clang/LLVM has been removing references to
it ever since.
This patch removes the remaining references to it from LLDB. One benefit
of this is that it will allow us to remove the code checking for
`DW_AT_decl_file_attributes_are_invalid` and
`Supports_DW_AT_APPLE_objc_complete_type`.
With all the recent versions of Clang that I tested, ObjC forward
declarations like
```
@class ForwardObjcClass;
```
don't emit the kind of DWARF that this workaround was put in place for.
Also, zero-sized structures are valid in C (and thus Objective-C), so
this workaround makes things confusing to reason about when mixing the
two languages.
This workaround has been in place for at least a decade, and given that
recent compilers don't produce this anymore, we think it's a good time
to remove it.
For high-frequency multithreaded progress reports, the contention of
taking the progress mutex and the overhead of generating event can
significantly slow down the operation whose progress we are reporting.
This patch adds an (optional) capability to rate-limit them. It's
optional because this approach has one drawback: if the progress
reporting has a pattern where it generates a burst of activity and then
blocks (without reporting anything) for a large amount of time, it can
appear as if less progress was made that it actually was (because we
only reported the first event from the burst and dropped the other
ones).
I've also made a small refactor of the Progress class to better
distinguish between const (don't need protection), atomic (are used on
the hot path) and other (need mutex protection) members.
The main difference is that the llvm class (just a std::vector in
disguise) is not sorted. It turns out this isn't an issue because the
callers either:
- ignore the range list;
- convert it to a different format (which is then sorted);
- or query the minimum value (which is faster than sorting)
The last case is something I want to get rid of in a followup as a part
of removing the assumption that function's entry point is also its
lowest address.
Indexing a single DWARF unit is a fairly small task, which means the
overhead of enqueueing a task for each unit is not negligible (mainly
because introduces a lot of synchronization points for queue management,
memory allocation etc.). This is particularly true if the binary was
built with type units, as these are usually very small.
This essentially brings us back to the state before
https://reviews.llvm.org/D78337, but the new implementation is built on
the llvm ThreadPool, and I've added a small improvement -- we now
construct one "index set" per thread instead of one per unit, which
should lower the memory usage (fewer small allocations) and make the
subsequent merge step faster.
On its own this patch doesn't actually change the performance
characteristics because we still have one choke point -- progress
reporting. I'm leaving that for a separate patch, but I've tried that
simply removing the progress reporting gives us about a 30-60% speed
boost.
The ManualDWARFIndex class can create a index cache if the LLDB index
cache is enabled. This used to save the index to the same file,
regardless of wether the cache was a full index (no .debug_names) or a
partial index (have .debug_names, but not all .o files were had
.debug_names). So we could end up saving an index cache that was
partial, and then later load that partial index as if it were a full
index if the user set the 'settings set
plugin.symbol-file.dwarf.ignore-file-indexes true'. This would cause us
to ignore the .debug_names section, and if the index cache was enabled,
we could end up loading the partial index as if it were a full DWARF
index.
This patch detects when the ManualDWARFIndex is being used with
.debug_names, and saves out a cache file with a suffix of "-full" or
"-partial" to avoid this issue.
.. in the global namespace
The problem was the interaction of #116989 with an optimization in
GetTypesWithQuery. The optimization was only correct for non-exact
matches, but that didn't matter before this PR due to the "second layer
of defense". After that was removed, the query started returning more
types than it should.
This reverts commit 2526d5b168, reapplying
ba14dac481 after fixing the conflict with
#117532. The change is that Function::GetAddressRanges now recomputes
the returned value instead of returning the member. This means it now
returns a value instead of a reference type.
This is a follow-up/reimplementation of #115730. While working on that
patch, I did not realize that the correct (discontinuous) set of ranges
is already stored in the block representing the whole function. The
catch -- ranges for this block are only set later, when parsing all of
the blocks of the function.
This patch changes that by populating the function block ranges eagerly
-- from within the Function constructor. This also necessitates a
corresponding change in all of the symbol files -- so that they stop
populating the ranges of that block. This allows us to avoid some
unnecessary work (not parsing the function DW_AT_ranges twice) and also
results in some simplification of the parsing code.
It's basically true already (except for a brief time during
construction). This patch makes sure the objects are constructed with a
valid parent and enforces this in the type system, which allows us to
get rid of some nullptr checks.
In case of an error GetBlock would return a reference to a Block without
adding it to a parent. This doesn't seem like a good idea, and none of
the other plugins do that.
This patch fixes that by propagating errors (well, null pointers...) up
the stack.
I don't know of any specific problem that this solves, but given that
this occurs only when something goes very wrong (e.g. a corrupted PDB
file), it's quite possible noone has run into this situation, so we
can't say the code is correct either. It also gets in the way of a
refactor I'm contemplating.
In #108907, the index classes started filtering the DIEs according to
the full type query (instead of just the base name). This means that the
checks in SymbolFileDWARF are now redundant.
I've also moved the non-redundant checks so that now all checking is
done in the DWARFIndex class and the caller can expect to get the final
filtered list of types.
This is the second half of
https://github.com/llvm/llvm-project/pull/90008.
Essentially, it replaces the work of resolving template types when we
just need the qualified names with walking the DIE tree using
`DWARFTypePrinter`.
### Result
For an internal target, the time spent on `expr *this` for the first
time reduced from 28 secs to 17 secs.
Following up from https://github.com/llvm/llvm-project/pull/112928, we
can reuse the approach from Clang Sema to infer the MSInheritanceModel
and add the necessary attribute manually. This allows the inspection of
member function pointers with DWARF on Windows.
The class is only used from one place, which is trivial to implement
using the llvm class.
The main difference is that in the new implementation, the ranges are
parsed each time anew (instead of being parsed at startup and cached). I
believe this is fine because:
- this is already how things work with DWARF v5 debug_rnglists
- parsing debug_ranges is fairly fast (definitely faster than rnglists)
- generally, this result will be cached at a higher level anyway.
Browsing the code I did find one instance where that is not the case --
SymbolFileDWARF::ResolveFunctionAndBlock -- which is called each time we
resolve an address (to the block level). However, this function is
already pretty suboptimal: it first traverses the DIE tree (which
involves parsing all the DIE attributes) to find the correct block, then
it parses them again to construct the `lldb_private::Block`
representation, and *then* it uses the ID of the block DIE it found in
the first step to look up the `Block` object. If this turns out to be a
bottleneck, I think there are better ways to optimize it than caching
the debug_ranges parse.
The motiviation for this is that DWARFDebugRanges sorts the block
ranges, even though the order of the ranges is load-bearing (in the
absence of DW_AT_low_pc, the "base address" of a scope is determined by
the first range entry). Delaying the parsing (and sorting) step makes it
easier to access the first entry.
"statistics dump" currently report the statistics of all targets in
debugger instead of current target. This is wrong because there is a
"statistics dump --all-targets" option that supposed to include
everything.
This PR fixes the issue by only report statistics for current target
instead of all. It also includes the change to reset statistics debug
info/symbol table parsing/indexing time during debugger destroy. This is
required so that we report current statistics if we plan to reuse
lldb/lldb-dap across debug sessions
---------
Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
This is the beginning of a different, more fundamental approach to
handling. This PR tries to tries to minimize functional changes. It only
makes sure that we store the true set of ranges inside the function
object, so that subsequent patches can make use of it.
In DWARF 4 and earlier `static const` members of structs, classes and
unions have an entry tag `DW_TAG_member`, and are also tagged as
`DW_AT_declaration`, but otherwise follow the same rules as
`DW_TAG_variable`.
