TestVSCode_breakpointEvents.py is failing on macOS Ventura because we
receive 3 breakpoint events instead of one. This is likely the result of
dyld moving into the shared cache.
This patch implements VSCode DAP logpoints feature (also called tracepoint
in other VS debugger).
This will provide a convenient way for user to do printf style logging
debugging without pausing debuggee.
Differential Revision: https://reviews.llvm.org/D127702
Eliminate boilerplate of having each test manually assign to `mydir` by calling
`compute_mydir` in lldbtest.py.
Differential Revision: https://reviews.llvm.org/D128077
This fixes an issue that optimized variable error message is not shown to end
users in lldb-vscode.
Differential Revision: https://reviews.llvm.org/D126014
To help user identify optimized code This diff adds a "[opt]" suffix to
optimized stack frames in lldb-vscode. This provides consistent experience
as command line lldb.
It also adds a new "optimized" attribute to DAP stack frame object so that
it is easy to identify from telemetry than parsing trailing "[opt]".
Differential Revision: https://reviews.llvm.org/D126013
Prior to this fix if we have a really large array or collection class, we would end up always creating all of the child variables for an array or collection class. If the number of children was very high this can cause delays when expanding variables. By adding the "indexedVariables" to variables with lots of children, we can keep good performance in the variables view at all times. This patch will add the "indexedVariables" key/value pair to any "Variable" JSON dictionairies when we have an array of synthetic child provider that will create more than 100 children.
We have to be careful to not call "uint32_t SBValue::GetNumChildren()" on any lldb::SBValue that we use because it can cause a class, struct or union to complete the type in order to be able to properly tell us how many children it has and this can be expensive if you have a lot of variables. By default LLDB won't need to complete a type if we have variables that are classes, structs or unions unless the user expands the variable in the variable view. So we try to only get the GetNumChildren() when we have an array, as this is a cheap operation, or a synthetic child provider, most of which are for showing collections that typically fall into this category. We add a variable reference, which indicates that something can be expanded, when the function "bool SBValue::MightHaveChildren()" is true as this call doesn't need to complete the type in order to return true. This way if no one ever expands class variables, we don't need to complete the type.
Differential Revision: https://reviews.llvm.org/D125347
This improves this test a lot because before when using the "attachCommands" to run the following commands:
(lldb) target create -d /path/to/a.out
(lldb) process launch
This was racy as it wasn't stopping the program at the entry point, and the process might run to completion before we can even debug it. With the recent changes to the "attachCommands" we were waiting for the process to stop, but the process might be exited already, and that _should_ have caused the attach to fail since there was no process to attach to. By adding "--stop-at-entry" to the process launch, we ensure this should be less racy and give us a valid process to attach to.
This is a modified version of a previous patch that was reverted: https://reviews.llvm.org/D119797
This version only waits for the process to stop when using "launchCommands" or "attachCommands"...
...and doesn't play with the async mode when doing normal launch/attach.
We discovered that when using "launchCommands" or "attachCommands" that there was an issue where these commands were not being run synchronously. There were further problems in this case where we would get thread events for the process that was just launched or attached before the IDE was ready, which is after "configurationDone" was sent to lldb-vscode.
This fix introduces the ability to wait for the process to stop after "launchCommands" or "attachCommands" are run to ensure that we have a stopped process point that is ready for the debug session to proceed. We spin up the thread that listens for process events before we start the launch or attach, but we don't want stop events being delivered through the DAP protocol until the "configurationDone" packet is received. We now always ignore the stop event with a stop ID of 1, which is the first stop. All normal launch and attach scenarios use the synchronous mode, and "launchCommands and "attachCommands" run an array of LLDB commands in async mode.
This should make our launch with "launchCommands" and attach with "attachCommands" avoid a race condition when the process is being launched or attached.
Differential Revision: https://reviews.llvm.org/D120755
The race is between these two pieces of code that are executed in two separate
lldb-vscode threads (the first is in the main thread and another is in the
event-handling thread):
```
// lldb-vscode.cpp
g_vsc.debugger.SetAsync(false);
g_vsc.target.Launch(launch_info, error);
g_vsc.debugger.SetAsync(true);
```
```
// Target.cpp
bool old_async = debugger.GetAsyncExecution();
debugger.SetAsyncExecution(true);
debugger.GetCommandInterpreter().HandleCommands(GetCommands(), exc_ctx,
options, result);
debugger.SetAsyncExecution(old_async);
```
The sequence that leads to the bug is this one:
1. Main thread enables synchronous mode and launches the process.
2. When the process is launched, it generates the first stop event.
3. This stop event is catched by the event-handling thread and DoOnRemoval
is invoked.
4. Inside DoOnRemoval, this thread runs stop hooks. And before running stop
hooks, the current synchronization mode is stored into old_async (and
right now it is equal to "false").
5. The main thread finishes the launch and returns to lldb-vscode, the
synchronization mode is restored to asynchronous by lldb-vscode.
6. Event-handling thread finishes stop hooks processing and restores the
synchronization mode according to old_async (i.e. makes the mode synchronous)
7. And now the mode is synchronous while lldb-vscode expects it to be
asynchronous. Synchronous mode forbids the process to broadcast public stop
events, so, VS Code just hangs because lldb-vscode doesn't notify it about
stops.
So, this diff makes the target intercept the first stop event if the process is
launched in the synchronous mode, thus preventing stop hooks execution.
The bug is only present on Windows because other platforms already
intercept this event using their own hijacking listeners.
So, this diff also fixes some problems with lldb-vscode tests on Windows to make
it possible to run the related test. Other tests still can't be enabled because
the debugged program prints something into stdout and LLDB can't intercept this
output and redirect it to lldb-vscode properly.
Reviewed By: jingham
Differential Revision: https://reviews.llvm.org/D119548
We discovered that when using "launchCommands" or "attachCommands" that there was an issue where these commands were not being run synchronously. There were further problems in this case where we would get thread events for the process that was just launched or attached before the IDE was ready, which is after "configurationDone" was sent to lldb-vscode.
This fix introduces the ability to wait for the process to stop after the run or attach to ensure that we have a stopped process at the entry point that is ready for the debug session to proceed. This also allows us to run the normal launch or attach without needing to play with the async flag the debugger. We spin up the thread that listens for process events before we start the launch or attach, but we stop the first eStateStopped (with stop ID of zero) event from being delivered through the DAP protocol because the "configurationDone" request handler will deliver it manually as the IDE expects a stop after configuration done. The request_configurationDone will also only deliver the stop packet if the "stopOnEntry" is False in the launch configuration.
Also added a new "timeout" to the launch and attach launch configuration arguments that can be set and defaults to 30 seconds. Since we now poll to detect when the process is stopped, we need a timeout that can be changed in case certain workflows take longer that 30 seconds to attach. If the process is not stopped by the timeout, an error will be retured for the launch or attach.
Added a flag to the vscode.py protocol classes that detects and ensures that no "stopped" events are sent prior to "configurationDone" has been sent and will raise an error if it does happen.
This should make our launching and attaching more reliable and avoid some deadlocks that were being seen (https://reviews.llvm.org/D119548).
Differential Revision: https://reviews.llvm.org/D119797
Scopes can have an optional hint for how to present this scope in the UI:
https://microsoft.github.io/debug-adapter-protocol/specification#Types_Scope
The IDEs can use the hint to present the data accordingly. For example,
Visual Studio has a separate Registers window, which is populated with the
data from the scope with `presentationHint: "registers"`.
Reviewed By: wallace
Differential Revision: https://reviews.llvm.org/D113400
SetSourceMapFromArguments is called after the core is loaded. This means
that the source file for the crashing code won't have the source map applied.
Move the call to SetSourceMapFromArguments in request_attach to just after
the call to RunInitCommands, matching request_launch behavior.
Reviewed By: clayborg, wallace
Differential Revision: https://reviews.llvm.org/D112834
Based on post-commit review discussion on
2bd8493847 with Richard Smith.
Other uses of forcing HasEmptyPlaceHolder to false seem OK to me -
they're all around pointer/reference types where the pointer/reference
token will appear at the rightmost side of the left side of the type
name, so they make nested types (eg: the "int" in "int *") behave as
though there is a non-empty placeholder (because the "*" is essentially
the placeholder as far as the "int" is concerned).
This was originally committed in 277623f4d5
Reverted in f9ad1d1c77 due to breakages
outside of clang - lldb seems to have some strange/strong dependence on
"char [N]" versus "char[N]" when printing strings (not due to that name
appearing in DWARF, but probably due to using clang to stringify type
names) that'll need to be addressed, plus a few other odds and ends in
other subprojects (clang-tools-extra, compiler-rt, etc).
The thread that Visual Studio Code displays on a stop is called the focus thread. When the previous focus thread exits and we stop in a new thread, lldb-vscode does not tell vscode to set the new thread as the focus thread, so it selects the first thread in the thread list.
This patch changes lldb-vscode to tell vscode that the new thread is the focus thread. It also includes a test that verifies the DAP stop message for this case contains the correct values.
Reviewed By: clayborg, wallace
Differential Revision: https://reviews.llvm.org/D109633
VScode now sends a "scopes" DAP request immediately after any expression evaluation.
This scopes request would clear and invalidate any non-scoped expandable variables in g_vsc.variables, causing later "variables" request to return empty result.
The symptom is that any expandable variables in VScode watch window/debug console UI to return empty content.
This diff fixes this issue by only clearing the expandable variables at process continue time. To achieve this, we have to repopulate all scoped variables
during context switch for each "scopes" request without clearing global expandable variables.
So the PR puts the scoped variables into its own locals/globals/registers; and all expandable variables into separate "expandableVariables" list.
Also, instead of using the variable index for "variableReference", it generates a new variableReference id each time as the key of "expandableVariables".
As a further new feature, this PR adds a new "expandablePermanentVariables" which has the lifetime of debug session. Any expandable variables from debug console
are added into this list. This enables users to snapshot expanable old variable in debug console and compare with new variables if desire.
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D105166
D104406 introduced an error in which, if there are multiple matchings rules for a given path, lldb was only checking for the validity in the filesystem of the first match instead of looking exhaustively one by one until a valid file is found.
Besides that, a call to consume_front was being done incorrectly, as it was modifying the input, which renders subsequent matches incorrect.
I added a test that checks for both cases.
Differential Revision: https://reviews.llvm.org/D106723
The variable.rst documentation says:
```
If it returns a value, and that value is True, LLDB will be allowed to cache the children and the children count it previously obtained, and will not return to the provider class to ask. If nothing, None, or anything other than True is returned, LLDB will discard the cached information and ask. Regardless, whenever necessary LLDB will call update.
```
However, several update methods in gnu_libstdcpp.py were returning True,
which made lldb unaware of any changes in the corresponding objects.
This problem was visible by lldb-vscode in the following way:
- If a breakpoint is hit and there's a vector with the contents {1, 2},
it'll be displayed correctly.
- Then the user steps and the next stop contains the vector modified.
The program changed it to {1, 2, 3}
- frame var then displays {1, 2} incorrectly, due to the caching caused
by the update method
It's worth mentioning that none of libcxx.py'd update methods return True. Same for LibCxxVector.cpp, which returns false.
Added a very simple test that fails without this fix.
Differential Revision: https://reviews.llvm.org/D103209
VSCode doesn't render multiple variables with the same name in the variables view. It only renders one of them. This is a situation that happens often when there are shadowed variables.
The nodejs debugger solves this by adding a number suffix to the variable, e.g. "x", "x2", "x3" are the different x variables in nested blocks.
In this patch I'm doing something similar, but the suffix is " @ <file_name:line>), e.g. "x @ main.cpp:17", "x @ main.cpp:21". The fallback would be an address if the source and line information is not present, which should be rare.
This fix is only needed for globals and locals. Children of variables don't suffer of this problem.
When there are shadowed variables
{F16182150}
Without shadowed variables
{F16182152}
Modifying these variables through the UI works
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D99989
In certain occasions times, like when LLDB is initializing and
evaluating the .lldbinit files, it tries to print to stderr and stdout
directly. This confuses the IDE with malformed data, as it talks to
lldb-vscode using stdin and stdout following the JSON RPC protocol. This
ends up terminating the debug session with the user unaware of what's
going on. There might be other situations in which this can happen, and
they will be harder to debug than the .lldbinit case.
After several discussions with @clayborg, @yinghuitan and @aadsm, we
realized that the best course of action is to simply redirect stdout and
stderr to the console, without modifying LLDB itself. This will prove to
be resilient to future bugs or features.
I made the simplest possible redirection logic I could come up with. It
only works for POSIX, and to make it work with Windows should be merely
changing pipe and dup2 for the windows equivalents like _pipe and _dup2.
Sadly I don't have a Windows machine, so I'll do it later once my office
reopens, or maybe someone else can do it.
I'm intentionally not adding a stop-redirecting logic, as I don't see it
useful for the lldb-vscode case (why would we want to do that, really?).
I added a test.
Note: this is a simpler version of D80659. I first tried to implement a
RIIA version of it, but it was problematic to manage the state of the
thread and reverting the redirection came with some non trivial
complexities, like what to do with unflushed data after the debug
session has finished on the IDE's side.
This diff ass postRunCommands, which are the counterpart of the preRunCommands. TThey will be executed right after the target is launched or attached correctly, which means that the targets can assume that the target is running.
Differential Revision: https://reviews.llvm.org/D100340
This fixes flakiness in TestVSCode_launch.test_progress_events
vscode.progress_events some times failed to populate in time for
follow up iterations.
Adding a minor delay before the the for the loop fixes the issue.
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D99497
TestVSCode_disconnect.test_launch fails with clean up error because
disconnect gets called twice once from the test case and once from
the tear down hook.
This patch disables disconnect after its been called from test_launch
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D99491
LLDB can often appear deadlocked to users that use IDEs when it is indexing DWARF, or parsing symbol tables. These long running operations can make a debug session appear to be doing nothing even though a lot of work is going on inside LLDB. This patch adds a public API to allow clients to listen to debugger events that report progress and will allow UI to create an activity window or display that can show users what is going on and keep them informed of expensive operations that are going on inside LLDB.
Differential Revision: https://reviews.llvm.org/D97739
Summary:
The request "evaluate" supports a "context" attribute, which is sent by VSCode. The attribute is defined here https://microsoft.github.io/debug-adapter-protocol/specification#Requests_Evaluate
The "clipboard" context is not yet supported by lldb-vscode, so we can forget about it for now. The 'repl' (i.e. Debug Console) and 'watch' (i.e. Watch Expression) contexts must use the expression parser in case the frame's variable path is not enough, as the user expects these expressions to never fail. On the other hand, the 'hover' expression is invoked whenever the user hovers on any keyword on the UI and the user is fine with the expression not being fully resolved, as they know that the 'repl' case is the fallback they can rely on.
Given that the 'hover' expression is invoked many many times without the user noticing it due to it being triggered by the mouse, I'm making it use only the frame's variable path functionality and not the expression parser. This should speed up tremendously the responsiveness of a debug session when the user only sets source breakpoints and inspect local variables, as the entire debug info is not needed to be parsed.
Regarding tests, I've tried to be as comprehensive as possible considering a multi-file project. Fortunately, the results from the "hover" case are enough most of the times.
Differential Revision: https://reviews.llvm.org/D98656
Use realpath() when spawning the executable create_after_attach
to workaround a FreeBSD plugin (and possibly others) problem.
If the executable is started via a path containing a symlink, it is
added to the module list twice -- via the real and apparent path.
This in turn cases the requested breakpoint to resolve twice.
Use realpath() for main program path in lldb-vscode breakpoint tests
to workaround a similar problem. If the passed path does not match
the realpath, lldb-vscode does not report the breakpoints as verified
and causes tests to fail.
Since the underlying problems are non-trivial to fix and the purpose
of these tests is not to reproduce symlink problems, let's apply
trivial workarounds to make them pass.
Differential Revision: https://reviews.llvm.org/D97230
VSCode was not being informed whenever a location had been resolved (after being initated as non-resolved), so even though it was actually resolved, the IDE would show a hollow dot (instead of a red dot) because it didn't know about the change.
Differential Revision: https://reviews.llvm.org/D96680
Convert `assertTrue(a == b)` to `assertEqual(a, b)` to produce better failure messages.
These were mostly done via regex search & replace, with some manual fixes.
Differential Revision: https://reviews.llvm.org/D95813
stella.stemenova mentioned in https://reviews.llvm.org/D93951 failures on Windows for this test.
I'm fixing the macro definitions and disabling the tests for python
versions lower than 3.7. I'll figure out that actual issue with
python3.6 after the buildbots are fine again.
Depends on D93874.
runInTerminal was using --wait-for, but it was some problems because it uses process polling looking for a single instance of the debuggee:
- it gets to know of the target late, which renders breakpoints in the main function almost impossible
- polling might fail if there are already other processes with the same name
- polling might also fail on some linux machine, as it's implemented with the ps command, and the ps command's args and output are not standard everywhere
As a better way to implement this so that it works well on Darwin and Linux, I'm using now the following process:
- lldb-vscode notices the runInTerminal, so it spawns lldb-vscode with a special flag --launch-target <target>. This flags tells lldb-vscode to wait to be attached and then it execs the target program. I'm using lldb-vscode itself to do this, because it makes finding the launcher program easier. Also no CMAKE INSTALL scripts are needed.
- Besides this, the debugger creates a temporary FIFO file where the launcher program will write its pid to. That way the debugger will be sure of which program to attach.
- Once attach happend, the debugger creates a second temporary file to notify the launcher program that it has been attached, so that it can then exec. I'm using this instead of using a signal or a similar mechanism because I don't want the launcher program to wait indefinitely to be attached in case the debugger crashed. That would pollute the process list with a lot of hanging processes. Instead, I'm setting a 20 seconds timeout (that's an overkill) and the launcher program seeks in intervals the second tepmorary file.
Some notes:
- I preferred not to use sockets because it requires a lot of code and I only need a pid. It would also require a lot of code when windows support is implemented.
- I didn't add Windows support, as I don't have a windows machine, but adding support for it should be easy, as the FIFO file can be implemented with a named pipe, which is standard on Windows and works pretty much the same way.
The existing test which didn't pass on Linux, now passes.
Differential Revision: https://reviews.llvm.org/D93951
lldb-vsdode was communicating the list of modules to the IDE with events, which in practice ended up having some drawbacks
- when debugging large targets, the number of these events were easily 10k, which polluted the messages being transmitted, which caused the following: a harder time debugging the messages, a lag after terminated the process because of these messages being processes (this could easily take several seconds). The latter was specially bad, as users were complaining about it even when they didn't check the modules view.
- these events were rarely used, as users only check the modules view when something is wrong and they try to debug things.
After getting some feedback from users, we realized that it's better to not used events but make this simply a request and is triggered by users whenever they needed.
This diff achieves that and does some small clean up in the existing code.
Differential Revision: https://reviews.llvm.org/D94033
