This change by itself has no measurable effect on the LLDB
testsuite. I'm making it in preparation for threading through more
errors in the Swift language plugin.
# Added/changed options
The following options are **added** to the `statistics dump` command:
* `--targets=bool`: Boolean. Dumps the `targets` section.
* `--modules=bool`: Boolean. Dumps the `modules` section.
When both options are given, the field `moduleIdentifiers` will be
dumped for each target in the `targets` section.
The following options are **changed**:
* `--transcript=bool`: Changed to a boolean. Dumps the `transcript`
section.
# Behavior of `statistics dump` with various options
The behavior is **backward compatible**:
- When no options are provided, `statistics dump` dumps all sections.
- When `--summary` is provided, only dumps the summary info.
**New** behavior:
- `--targets=bool`, `--modules=bool`, `--transcript=bool` overrides the
above "default".
For **example**:
- `statistics dump --modules=false` dumps summary + targets +
transcript. No modules.
- `statistics dump --summary --targets=true --transcript=true` dumps
summary + targets (in summary mode) + transcript.
# Added options into public API
In `SBStatisticsOptions`, add:
* `Set/GetIncludeTargets`
* `Set/GetIncludeModules`
* `Set/GetIncludeTranscript`
**Alternative considered**: Thought about adding
`Set/GetIncludeSections(string sections_spec)`, which receives a
comma-separated list of section names to be included ("targets",
"modules", "transcript"). The **benefit** of this approach is that the
API is more future-proof when it comes to possible adding/changing of
section names. **However**, I feel the section names are likely to
remain unchanged for a while - it's not like we plan to make big changes
to the output of `statistics dump` any time soon. The **downsides** of
this approach are: 1\ the readability of the API is worse (requires
reading doc to understand what string can be accepted), 2\ string input
are more prone to human error (e.g. typo "target" instead of expected
"targets").
# Tests
```
bin/llvm-lit -sv ../external/llvm-project/lldb/test/API/commands/statistics/basic/TestStats.py
```
```
./tools/lldb/unittests/Interpreter/InterpreterTests
```
New test cases have been added to verify:
* Different sections are dumped/not dumped when different
`StatisticsOptions` are given through command line (CLI or
`HandleCommand`; see `test_sections_existence_through_command`) or API
(see `test_sections_existence_through_api`).
* The order in which the options are given in command line does not
matter (see `test_order_of_options_do_not_matter`).
---------
Co-authored-by: Roy Shi <royshi@meta.com>
Create additional helper functions for the ValueObject class, for:
- returning the value as an APSInt or APFloat
- additional type casting options
- additional ways to create ValueObjects from various types of data
- dereferencing a ValueObject
These helper functions are needed for implementing the Data Inspection
Language, described in
https://discourse.llvm.org/t/rfc-data-inspection-language/69893
Cppcheck recommends using a const reference for range variables in a
for-each loop.
This avoids unnecessary copying of elements, improving performance.
Caught by cppcheck -
lldb/source/API/SBBreakpoint.cpp:717:22: performance: Range variable
'name' should be declared as const reference. [iterateByValue]
lldb/source/API/SBTarget.cpp:1150:15: performance: Range variable 'name'
should be declared as const reference. [iterateByValue]
lldb/source/Breakpoint/Breakpoint.cpp:888:26: performance: Range
variable 'name' should be declared as const reference. [iterateByValue]
lldb/source/Breakpoint/BreakpointIDList.cpp:262:26: performance: Range
variable 'name' should be declared as const reference. [iterateByValue]
Fix#91213Fix#91217Fix#91219Fix#91220
This is useful if you have a transcript of a user session and want to
rerun those commands with RunCommandInterpreter. The same functionality
is also useful in testing.
I'm adding it primarily for the second reason. In a subsequent patch,
I'm adding the ability to Python based commands to provide their
"auto-repeat" command. Among other things, that will allow potentially
state destroying user commands to prevent auto-repeat. Testing this with
Shell or pexpect tests is not nearly as accurate or convenient as using
RunCommandInterpreter, but to use that I need to allow auto-repeat.
I think for consistency's sake, having interactive sessions always do
auto-repeats is the right choice, though that's a lightly held
opinion...
Re-apply https://github.com/llvm/llvm-project/pull/87550 with fixes.
Details:
Some tests in fuchsia failed because of the newly added assertion.
This was because `GetExceptionBreakpoint()` could be called before
`g_dap.debugger` was initted.
The fix here is to just lazily populate the list in
GetExceptionBreakpoint() rather than assuming it's already been initted.
(There is some nuisance here because we can't simply just populate it in
DAP::DAP(), which is a global ctor and is called before
`SBDebugger::Initialize()` is called. )
This adds new SB API calls and classes to allow a user of the SB API to obtain an address range from SBFunction and SBBlock. This is a second attempt to land the reverted PR #92014.
Update the folder titles for targets in the monorepository that have not
seen taken care of for some time. These are the folders that targets are
organized in Visual Studio and XCode
(`set_property(TARGET <target> PROPERTY FOLDER "<title>")`)
when using the respective CMake's IDE generator.
* Ensure that every target is in a folder
* Use a folder hierarchy with each LLVM subproject as a top-level folder
* Use consistent folder names between subprojects
* When using target-creating functions from AddLLVM.cmake, automatically
deduce the folder. This reduces the number of
`set_property`/`set_target_property`, but are still necessary when
`add_custom_target`, `add_executable`, `add_library`, etc. are used. A
LLVM_SUBPROJECT_TITLE definition is used for that in each subproject's
root CMakeLists.txt.
Here we go with attempt number five. Again, no changes to the LLDB code
diff, which has been reviewed several times.
For the tests, I added a `@skipIfCurlSupportMissing` annotation so that
the Debuginfod mocked server stuff won't run, and I also disabled
non-Linux/FreeBSD hosts altogether, as they fail for platform reasons on
macOS and Windows. In addition, I updated the process for extracting the
GNU BuildID to no create a target, per some feedback on the previous
diff.
For reference, previous PR's (landed, backed out after the fact for
various reasons) #90622, #87676, #86812, #85693
---------
Co-authored-by: Kevin Frei <freik@meta.com>
# Motivation
Individual callers of `SBDebugger::SetDestroyCallback()` might think
that they have registered their callback and expect it to be called when
the debugger is destroyed. In reality, only the last caller survives,
and all previous callers are forgotten, which might be a surprise to
them. Worse, if this is called in a race condition, which callback
survives is less predictable, which may case confusing behavior
elsewhere.
# This PR
Allows multiple destroy callbacks to be registered and all called when
the debugger is destroyed.
**EDIT**: Adds two new APIs: `AddDestroyCallback()` and
`ClearDestroyCallback()`. `SetDestroyCallback()` will first clear then
add the given callback. Tests are added for the new APIs.
## Tests
```
bin/llvm-lit -sv ../external/llvm-project/lldb/test/API/python_api/debugger/TestDebuggerAPI.py
```
## (out-dated, see comments below) Semantic change to
`SetDestroyCallback()`
~~Currently, the method overwrites the old callback with the new one.
With this PR, it will NOT overwrite. Instead, it will hold on to both.
Both callbacks get called during destroy.~~
~~**Risk**: Although the documentation of `SetDestroyCallback()` (see
[C++](https://lldb.llvm.org/cpp_reference/classlldb_1_1SBDebugger.html#afa1649d9453a376b5c95888b5a0cb4ec)
and
[python](https://lldb.llvm.org/python_api/lldb.SBDebugger.html#lldb.SBDebugger.SetDestroyCallback))
doesn't really specify the behavior, there is a risk: if existing call
sites rely on the "overwrite" behavior, they will be surprised because
now the old callback will get called. But as the above said, the current
behavior of "overwrite" itself might be unintended, so I don't
anticipate users to rely on this behavior. In short, this risk might be
less of a problem if we correct it sooner rather than later (which is
what this PR is trying to do).~~
## (out-dated, see comments below) Implementation
~~The implementation holds a `std::vector<std::pair<callback, baton>>`.
When `SetDestroyCallback()` is called, callbacks and batons are appended
to the `std::vector`. When destroy event happen, the `(callback, baton)`
pairs are invoked FIFO. Finally, the `std::vector` is cleared.~~
# (out-dated, see comments below) Alternatives considered
~~Instead of changing `SetDestroyCallback()`, a new method
`AddDestroyCallback()` can be added, which use the same
`std::vector<std::pair<>>` implementation. Together with
`ClearDestroyCallback()` (see below), they will replace and deprecate
`SetDestroyCallback()`. Meanwhile, in order to be backward compatible,
`SetDestroyCallback()` need to be updated to clear the `std::vector` and
then add the new callback. Pros: The end state is semantically more
correct. Cons: More steps to take; potentially maintaining an
"incorrect" behavior (of "overwrite").~~
~~A new method `ClearDestroyCallback()` can be added. Might be
unnecessary at this point, because workflows which need to set then
clear callbacks may exist but shouldn't be too common at least for now.
Such method can be added later when needed.~~
~~The `std::vector` may bring slight performance drawback if its
implementation doesn't handle small size efficiently. However, even if
that's the case, this path should be very cold (only used during init
and destroy). Such performance drawback should be negligible.~~
~~A different implementation was also considered. Instead of using
`std::vector`, the current `m_destroy_callback` field can be kept
unchanged. When `SetDestroyCallback()` is called, a lambda function can
be stored into `m_destroy_callback`. This lambda function will first
call the old callback, then the new one. This way, `std::vector` is
avoided. However, this implementation is more complex, thus less
readable, with not much perf to gain.~~
---------
Co-authored-by: Roy Shi <royshi@meta.com>
# Motivation
Currently, the user can already get the "transcript" (for "what is the
transcript", see `CommandInterpreter::SaveTranscript`). However, the
only way to obtain the transcript data as a user is to first destroy the
debugger, then read the save directory. Note that destroy-callbacks
cannot be used, because 1\ transcript data is private to the command
interpreter (see `CommandInterpreter.h`), and 2\ the writing of the
transcript is *after* the invocation of destory-callbacks (see
`Debugger::Destroy`).
So basically, there is no way to obtain the transcript:
* during the lifetime of a debugger (including the destroy-callbacks,
which often performs logging tasks, where the transcript can be useful)
* without relying on external storage
In theory, there are other ways for user to obtain transcript data
during a debugger's life cycle:
* Use Python API and intercept commands and results.
* Use CLI and record console input/output.
However, such ways rely on the client's setup and are not supported
natively by LLDB.
# Proposal
Add a new Python API `SBCommandInterpreter::GetTranscript()`.
Goals:
* It can be called at any time during the debugger's life cycle,
including in destroy-callbacks.
* It returns data in-memory.
Structured data:
* To make data processing easier, the return type is `SBStructuredData`.
See comments in code for how the data is organized.
* In the future, `SaveTranscript` can be updated to write different
formats using such data (e.g. JSON). This is probably accompanied by a
new setting (e.g. `interpreter.save-session-format`).
# Alternatives
The return type can also be `std::vector<std::pair<std::string,
SBCommandReturnObject>>`. This will make implementation easier, without
having to translate it to `SBStructuredData`. On the other hand,
`SBStructuredData` can convert to JSON easily, so it's more convenient
for user to process.
# Privacy
Both user commands and output/error in the transcript can contain
privacy data. However, as mentioned, the transcript is already available
to the user. The addition of the new API doesn't increase the level of
risk. In fact, it _lowers_ the risk of privacy data being leaked later
on, by avoiding writing such data to external storage.
Once the user (or their code) gets the transcript, it will be their
responsibility to make sure that any required privacy policies are
guaranteed.
# Tests
```
bin/llvm-lit -sv ../external/llvm-project/lldb/test/API/python_api/interpreter/TestCommandInterpreterAPI.py
```
```
bin/llvm-lit -sv ../external/llvm-project/lldb/test/API/commands/session/save/TestSessionSave.py
```
---------
Co-authored-by: Roy Shi <royshi@meta.com>
Co-authored-by: Med Ismail Bennani <ismail@bennani.ma>
If you change the generation script and re-run ninja (or whatever drives
your build), it currently will not regenerate SBLanguages.h. With
dependency tracking, it should re-run when the script changes.
Alex pointed out in #91254 that we only need the custom target if we had
more than one target depending on it. This isn't the case upstream, but
on our downstream fork, we have a second dependency. Reintroduce the
target so that everything can depend on that, without the
single-dependency foot-gun.
lldb already mostly(*) tracks this information. This just makes it
available to the SB users.
(*) It does not do that for typedefs right now see llvm.org/pr90958
that separates out language and version. To avoid reinventing the wheel
and introducing subtle incompatibilities, this API uses the table of
languages and versiond defined by the upcoming DWARF 6 standard
(https://dwarfstd.org/languages-v6.html). While the DWARF 6 spec is not
finialized, the list of languages is broadly considered stable.
The primary motivation for this is to allow the Swift language plugin to
switch between language dialects between, e.g., Swift 5.9 and 6.0 with
out introducing a ton of new language codes. On the main branch this
change is considered NFC.
Depends on https://github.com/llvm/llvm-project/pull/89980
I previously added this API via https://reviews.llvm.org/D142792 in
2023, along with changes to the ValueObject class to treat pointer types
as addresses, and to annotate those ValueObjects with the original
uint64_t byte sequence AND the name of the symbol once stripped, if that
points to a symbol.
I did this unconditionally for all pointer type ValueObjects, and it
caused several regressions in the Objective-C data formatters which have
a ValueObject of an object, it has the address of its class -- but with
ObjC, sometimes it is a "tagged pointer" which is metadata, not an
actual pointer. (e.g. a small NSInteger value is stored entirely in the
tagged pointer, instead of a separate object) Treating these
not-addresses as addresses -- clearing the non-addressable-bits -- is
invalid.
The original version of this patch we're using downstream only does this
bits clearing for pointer types that are specifically decorated with the
pointerauth typequal, but not all of those clang changes are upstreamed
to github main yet, so I tried this simpler approach and hit the tagged
pointer issue and bailed on the whole patch.
This patch, however, is simply adding SBValue::GetValueAsAddress so
script writers who know that an SBValue has an address in memory, can
strip off any metadata. It's an important API to have for script writers
when AArch64 ptrauth is in use, so I'm going to put this part of the
patch back on github main now until we can get the rest of that original
patch upstreamed.
This patch provides the initial implementation for the "Step Into
Specific/Step In Targets" feature in VSCode DAP.
The implementation disassembles all the call instructions in step range
and try to resolve operand name (assuming one operand) using debug info.
Later, the call target function name is chosen by end user and specified
in the StepInto() API call.
It is v1 because of using the existing step in target function name API.
This implementation has several limitations:
* Won't for indirect/virtual function call -- in most cases, our
disassembler won't be able to solve the indirect call target
address/name.
* Won't work for target function without debug info -- if the target
function has symbol but not debug info, the existing
ThreadPlanStepInRange won't stop.
* Relying on function names can be fragile -- if there is some middle
glue/thunk code, our disassembler can only resolve the glue/thunk code's
name not the real target function name. It can be fragile to depend
compiler/linker emits the same names for both.
* Does not support step into raw address call sites -- it is a valid
scenario that in Visual Studio debugger, user can explicitly choose a
raw address to step into which land in the function without debug
info/symbol, then choose UI to load the debug info on-demand for that
module/frame to continue exploring.
A more reliable design could be extending the ThreadPlanStepInRange to
support step in based on call-site instruction offset/PC which I will
propose in next iteration.
---------
Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
Summary:
When the target inferior process that is being debugged exits in lldb
command line, it emits following message:
`Process 4049526 exited with status = -1 (0xffffffff) debugserver died
with signal SIGTERM`
lldb-dap on the other hand does not emit a similar message. This PR adds
the same status message to lldb-dap.
Test Plan:
In VSCode debug any target and hit stop mode, kill lldb-server and
observe an exit status message similar to the following: Process 2167677
exited with status = -1 (0xffffffff) debugserver died with signal
SIGTERM
Reviewers:
@jeffreytan81,@clayborg,@kusmour,
Subscribers:
Tasks:
lldb-dap
Tags:
The main change is the addition of a new SBTypeStaticField class,
representing a static member of a class. It can be retrieved created
through SBType::GetStaticFieldWithName. It contains several methods
(GetName, GetMangledName, etc.) whose meaning is hopefully obvious. The
most interesting method is
lldb::SBValue GetConstantValue(lldb::SBTarget)
which returns a the value of the field -- if it is a compile time
constant. The reason for that is that only constants have their values
represented in the clang AST.
For non-constants, we need to go back to the module containing that
constant, and ask retrieve the associated ValueObjectVariable. That's
easy enough if the we are still in the type system of the module
(because then the type system will contain the pointer to the module
symbol file), but it's hard when the type has been copied into another
AST (e.g. during expression evaluation). To do that we would need to
walk the ast import chain backwards to find the source TypeSystem, and I
haven't found a nice way to do that.
Another possibility would be to use the mangled name of the variable to
perform a lookup (in all modules). That is sort of what happens when
evaluating the variable in an expression (which does work), but I did
not want to commit to that implementation as it's not necessary for my
use case (and if anyone wants to, he can use the GetMangledName function
and perform the lookup manually).
The patch adds a couple of new TypeSystem functions to surface the
information needed to implement this functionality.
These are hardcoded strings that are already present in the data section
of the binary, no need to immediately place them in the ConstString
StringPools. Lots of code still calls `GetBroadcasterClass` and places
the return value into a ConstString. Changing that would be a good
follow-up.
Additionally, calls to these functions are still wrapped in ConstStrings
at the SBAPI layer. This is because we must guarantee the lifetime of
all strings handed out publicly.
This implements coalescing of progress events using a timeout, as
discussed in the RFC on Discourse [1]. This PR consists of two commits
which, depending on the feedback, I may split up into two PRs. For now,
I think it's easier to review this as a whole.
1. The first commit introduces a new generic `Alarm` class. The class
lets you to schedule a function (callback) to be executed after a given
timeout expires. You can cancel and reset a callback before its
corresponding timeout expires. It achieves this with the help of a
worker thread that sleeps until the next timeout expires. The only
guarantee it provides is that your function is called no sooner than the
requested timeout. Because the callback is called directly from the
worker thread, a long running callback could potentially block the
worker thread. I intentionally kept the implementation as simple as
possible while addressing the needs for the `ProgressManager` use case.
If we want to rely on this somewhere else, we can reassess whether we
need to address those limitations.
2. The second commit uses the Alarm class to coalesce progress events.
To recap the Discourse discussion, when multiple progress events with
the same title execute in close succession, they get broadcast as one to
`eBroadcastBitProgressCategory`. The `ProgressManager` keeps track of
the in-flight progress events and when the refcount hits zero, the Alarm
class is used to schedule broadcasting the event. If a new progress
event comes in before the alarm fires, the alarm is reset (and the
process repeats when the new progress event ends). If no new event comes
in before the timeout expires, the progress event is broadcast.
[1]
https://discourse.llvm.org/t/rfc-improve-lldb-progress-reporting/75717/
This is another step towards supporting DWARF5 checksums and inline
source code in LLDB. This is a reland of #85468 but without the
functional change of storing the support file from the line table (yet).
Change GetNumChildren()/CalculateNumChildren() methods return
llvm::Expected
This is an NFC change that does not yet add any error handling or change
any code to return any errors.
This is the second big change in the patch series started with
https://github.com/llvm/llvm-project/pull/83501
A follow-up PR will wire up error handling.
Change GetNumChildren()/CalculateNumChildren() methods return
llvm::Expected
This is an NFC change that does not yet add any error handling or change
any code to return any errors.
This is the second big change in the patch series started with
https://github.com/llvm/llvm-project/pull/83501
A follow-up PR will wire up error handling.
[lldb] Add SBProcess methods for get/set/use address masks (#83095)
I'm reviving a patch from phabracator, https://reviews.llvm.org/D155905
which was approved but I wasn't thrilled with all the API I was adding
to SBProcess for all of the address mask types / memory regions. In this
update, I added enums to control type address mask type (code, data,
any) and address space specifiers (low, high, all) with defaulted
arguments for the most common case. I originally landed this via
https://github.com/llvm/llvm-project/pull/83095 but it failed on CIs
outside of arm64 Darwin so I had to debug it on more environments
and update the patch.
This patch is also fixing a bug in the "addressable bits to address
mask" calculation I added in AddressableBits::SetProcessMasks. If lldb
were told that 64 bits are valid for addressing, this method would
overflow the calculation and set an invalid mask. Added tests to check
this specific bug while I was adding these APIs.
This patch changes the value of "no mask set" from 0 to
LLDB_INVALID_ADDRESS_MASK, which is UINT64_MAX. A mask of all 1's
means "no bits are used for addressing" which is an impossible mask,
whereas a mask of 0 means "all bits are used for addressing" which
is possible.
I added a base class implementation of ABI::FixCodeAddress and
ABI::FixDataAddress that will apply the Process mask values if they
are set to a value other than LLDB_INVALID_ADDRESS_MASK.
I updated all the callers/users of the Mask methods which were
handling a value of 0 to mean invalid mask to use
LLDB_INVALID_ADDRESS_MASK.
I added code to the all AArch64 ABI Fix* methods to apply the
Highmem masks if they have been set. These will not be set on a
Linux environment, but in TestAddressMasks.py I test the highmem
masks feature for any AArch64 target, so all AArch64 ABI plugins
must handle it.
rdar://123530562
This reverts commit 9a12b0a600.
TestAddressMasks fails its first test on lldb-x86_64-debian,
lldb-arm-ubuntu, lldb-aarch64-ubuntu bots. Reverting while
investigating.
I'm reviving a patch from phabracator, https://reviews.llvm.org/D155905
which was approved but I wasn't thrilled with all the API I was adding
to SBProcess for all of the address mask types / memory regions. In this
update, I added enums to control type address mask type (code, data,
any) and address space specifiers (low, high, all) with defaulted
arguments for the most common case.
This patch is also fixing a bug in the "addressable bits to address
mask" calculation I added in AddressableBits::SetProcessMasks. If lldb
were told that 64 bits are valid for addressing, this method would
overflow the calculation and set an invalid mask. Added tests to check
this specific bug while I was adding these APIs.
rdar://123530562
Updates:
- The previous patch changed the default behavior to not load dwos in
`DWARFUnit`
~~`SymbolFileDWARFDwo *GetDwoSymbolFile(bool load_all_debug_info =
false);`~~
`SymbolFileDWARFDwo *GetDwoSymbolFile(bool load_all_debug_info = true);`
- This broke some lldb-shell tests (see
https://green.lab.llvm.org/green/view/LLDB/job/as-lldb-cmake/16273/)
- TestDebugInfoSize.py
- with symbol on-demand, by default statistics dump only reports
skeleton debug info size
- `statistics dump -f` will load all dwos. debug info = skeleton debug
info + all dwo debug info
Currently running `statistics dump` will trigger lldb to load debug info
that's not yet loaded (eg. dwo files). Resulted in a delay in the
command return, which, can be interrupting.
This patch also added a new option `--load-all-debug-info` asking
statistics to dump all possible debug info, which will force loading all
debug info available if not yet loaded.
Currently running `statistics dump` will trigger lldb to load debug info
that's not yet loaded (eg. dwo files). Resulted in a delay in the
command return, which, can be interrupting.
This patch also added a new option `--load-all-debug-info` asking
statistics to dump all possible debug info, which will force loading all
debug info available if not yet loaded.
We have a Python script that needs to locate coredump path during
debugging so that we can retrieve certain metadata files associated with
it. Currently, there is no API for this.
This patch adds a new `SBProcess::GetCoreFile()` to retrieve target dump
file spec used for dump debugging. Note: this is different from the main
executable module spec. To achieve this, the patch hoists m_core_file
into PostMortemProcess for sharing.
---------
Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
Adding command interpreter statistics into "statistics dump" command so
that we can track the command usage frequency for telemetry purpose.
This is useful to answer questions like what is the most frequently used
lldb commands across all our users.
---------
Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
