Following abba5de724, some tests started failing on green-dragon:
https://green.lab.llvm.org/green/job/lldb-cmake/55460/console
Looking at the backtrace, there seems to be a racing issue when deleting
the temporary directory containing all the JSON object files:
```
Traceback (most recent call last):
File "/Users/buildslave/jenkins/workspace/lldb-cmake/lldb-build/lib/python3.10/site-packages/lldb/macosx/crashlog.py", line 1115, in __call__
SymbolicateCrashLogs(debugger, shlex.split(command), result)
File "/Users/buildslave/jenkins/workspace/lldb-cmake/lldb-build/lib/python3.10/site-packages/lldb/macosx/crashlog.py", line 1457, in SymbolicateCrashLogs
SymbolicateCrashLog(crash_log, options)
File "/Users/buildslave/jenkins/workspace/lldb-cmake/lldb-build/lib/python3.10/site-packages/lldb/macosx/crashlog.py", line 1158, in SymbolicateCrashLog
with tempfile.TemporaryDirectory() as obj_dir:
File "/usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/tempfile.py", line 869, in __exit__
self.cleanup()
File "/usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/tempfile.py", line 873, in cleanup
self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
File "/usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/tempfile.py", line 855, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "/usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/shutil.py", line 731, in rmtree
onerror(os.rmdir, path, sys.exc_info())
File "/usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/shutil.py", line 729, in rmtree
os.rmdir(path)
OSError: [Errno 66] Directory not empty: '/var/folders/09/r4vw4v8n5kb67jl66zvlbljw0000gn/T/tmp6qfifxk7'
```
This patch should fix that issue since it won't delete the object file
directory until we're sure that the modules adding tasks completed.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch changes the way we generate the ObjectFileJSON files
containing the inlined symbols from the crash report to remove the
tempfile prefix from the object file name.
To do so, instead of creating a new tempfile for each module, we create a
temporary directory that contains each module object file with the same
name as the module.
This makes the backtraces only contain the module name without the
temfile prefix which makes it look like a regular stackframe.
Differential Revision: https://reviews.llvm.org/D151045
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch moves the `PassthroughScriptedProcess` & `PassthroughScriptedThread`
classes from the `interactive_scripted_process.py` test implementation
to the `lldb.scripted_process` python module.
This class is very versatile so it makes more sense to ship it with the
python module to make it easier for our adopters to derive their class
from it instead of copying it.
During the "migration", I've also noticed some bugs in the
`PassthroughScriptedThread` creation and update, so I also fixed that as
part of this patch.
Differential Revision: https://reviews.llvm.org/D151044
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch should fix an issue when parsing the process pid and setting
it in the scripted process.
It can happen that the `crashlog.process_id` attribute is sometimes
parsed as a string. That would cause the scripted process to pick the
default value (0).
To address that, this patch makes sure that the parsed attributed is
converted to the integer type before passing it to the scripted process.
Differential Revision: https://reviews.llvm.org/D151002
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Sometimes, crash reports come with inlined symbols. These provide the
exact stacktrace from the user binary.
However, when investigating a crash, it's very likely that the images related
to the crashed thread are not available on the debugging user system or
that the versions don't match. This causes interactive crashlog to show
a degraded backtrace in lldb.
This patch aims to address that issue, by parsing the inlined symbols
from the crash report and load them into lldb's target.
This patch is a follow-up to 27f27d1, focusing on inlined symbols
loading from legacy (non-json) crash reports.
To do so, it updates the stack frame regular expression to make the
capture groups more granular, to be able to extract the symbol name, the
offset and the source location if available, while making it more
maintainable.
So now, when parsing the crash report, we build a data structure
containing all the symbol information for each stackframe. Then, after
launching the scripted process for interactive mode, we write a JSON
symbol file for each module, only containing the symbols that it contains.
Finally, we load the json symbol file into lldb, before showing the user
the process status and backtrace.
rdar://97345586
Differential Revision: https://reviews.llvm.org/D146765
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
In 27f27d15f6, we added a new way to use textual (JSON) object files
and symbol files with the interactive crashlog command, using the
inlined symbols from the crash report.
However, there was a missing piece after successfully adding the textual
module to the target, we didn't mark it as available causing the module
loading to exit early.
This patch addresses that issue by marking the module as available when
added successfully to the target.
Differential Revision: https://reviews.llvm.org/D149477
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch improves breakpoint management when doing interactive
scripted process debugging.
In other to know which process set a breakpoint, we need to do some book
keeping on the multiplexer scripted process. When initializing the
multiplexer, we will first copy breakpoints that are already set on the
driving target.
Everytime we launch or resume, we should copy breakpoints from the
multiplexer to the driving process.
When creating a breakpoint from a child process, it needs to be set both
on the multiplexer and on the driving process. We also tag the created
breakpoint with the name and pid of the originator process.
This patch also implements all the requirement to achieve proper
breakpoint management. That involves:
- Adding python interator for breakpoints and watchpoints in SBTarget
- Add a new `ScriptedProcess.create_breakpoint` python method
Differential Revision: https://reviews.llvm.org/D148548
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
While debugging a Scripted Process, in order to update its state and
work nicely with lldb's execution model, it needs to toggle its private
state from running to stopped, which will result in broadcasting a
process state changed event to the debugger listener.
Originally, this state update was done systematically in the Scripted
Process C++ plugin, however in order to make scripted process
interactive, we need to be able to update their state dynamically.
This patch makes use of the recent addition of the
SBProcess::ForceScriptedState to programatically, and moves the
process private state update to the python implementation of the resume
method instead of doing it in ScriptedProcess::DoResume.
This patch also removes the unused ShouldStop & Stop scripted
process APIs, and adds new ScriptedInterface transform methods for
boolean arguments. This allow the user to programmatically decide if
after running the process, we should stop it (which is the default setting).
Differential Revision: https://reviews.llvm.org/D145295
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Create an artificial module using a JSON object file when we can't
locate the module and dSYM through dsymForUUID (or however
locate_module_and_debug_symbols is implemented). By parsing the symbols
from the crashlog and making them part of the JSON object file, LLDB can
symbolicate frames it otherwise wouldn't be able to, as there is no
module for it.
For non-interactive crashlogs, that never was a problem because we could
simply show the "pre-symbolicated" frame from the input. For interactive
crashlogs, we need a way to pass the symbol information to LLDB so that
it can symbolicate the frames, which is what motivated the JSON object
file format.
Differential revision: https://reviews.llvm.org/D148172
This patch should fix an assertion that causes some test failures:
https://ci.swift.org/view/LLDB/job/llvm-org-lldb-release-debuginfo/3587/console
This was caused by the changes introduces in `88f409194d5a` where we
replaced `DidLaunch` by `DidResume` in the `ScriptedProcess` class.
However, by the time we resume the process, the pid should be already
set. To address this, this patch brings back `DidLaunch` which will
initialize the ScriptedProcess pid with a placeholder value. That value
will be updated in `DidResume` to the final pid.
Note, this 2 stage PID initialization is necessary sometimes, when the
scripted process gets stopped at entry (launch) and gets assigned an
object that contains the PID value. In this case, we need to update the
PID when we resume the process after we've stopped at entry.
This also replaces the default scripted process id to an arbitrary
number (42) since the current value (0) is considered invalid.
Differential Revision: https://reviews.llvm.org/D148153
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Now that we can pass Python objects to the scripted process instance, we
don't need to parse the crashlog twice anymore.
Differential revision: https://reviews.llvm.org/D148063
While debugging a Scripted Process, in order to update its state and
work nicely with lldb's execution model, it needs to toggle its private
state from running to stopped, which will result in broadcasting a
process state changed event to the debugger listener.
Originally, this state update was done systematically in the Scripted
Process C++ plugin, however in order to make scripted process
interactive, we need to be able to update their state dynamically.
This patch makes use of the recent addition of the
`SBProcess::ForceScriptedState` to programatically, and moves the
process private state update to the python implementation of the `resume`
method instead of doing it in `ScriptedProcess::DoResume`.
This patch also removes the unused `ShouldStop` & `Stop` scripted
process APIs, and adds new ScriptedInterface transform methods for
boolean arguments. This allow the user to programmatically decide if
after running the process, we should stop it (which is the default setting).
Differential Revision: https://reviews.llvm.org/D145295
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch adds memory writing capabilities to the Scripted Process plugin.
This allows to user to get a target address and a memory buffer on the
python scripted process implementation that the user can make processing
on before performing the actual write.
This will also be used to write trap instruction to a real process
memory to set a breakpoint.
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
The goal of the simple patch is to clean-up the scripted process
interface by removing methods that were introduced with the interface
originally, but that were never really implemented (get_thread_with_id &
get_registers_for_thread).
This patch also changes `get_memory_region_containing_address` to have a
base implementation (that retunrs `None`), instead of forcing the user
to override it in their derived class.
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch adds process attach capabilities to the ScriptedProcess
plugin. This doesn't really expects a PID or process name, since the
process state is already script, however, this allows to create a
scripted process without requiring to have an executuble in the target.
In order to do so, this patch also turns the scripted process related
getters and setters from the `ProcessLaunchInfo` and
`ProcessAttachInfo` classes to a `ScriptedMetadata` instance and moves
it in the `ProcessInfo` class, so it can be accessed interchangeably.
This also adds the necessary SWIG wrappers to convert the internal
`Process{Attach,Launch}InfoSP` into a `SB{Attach,Launch}Info` to pass it
as argument the scripted process python implementation and convert it
back to the internal representation.
rdar://104577406
Differential Revision: https://reviews.llvm.org/D143104
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces a new method to the Scripted Process interface,
GetCapabilities.
This returns a dictionary that contains a list of flags that the
ScriptedProcess instance supports. This can be used for instance, to
force symbol lookup, when loading dynamic libraries in the scripted process.
Differential Revision: https://reviews.llvm.org/D142059
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This is a follow-up patch to 6f7835f309b9.
As explained previously, when running from an IDE, it can happen that
the IDE imports some lldb scripts by itself. If the user also tries to
import these commands, lldb will show the following message:
```
error: cannot add command: user command exists and force replace not set
```
This message is confusing to the user, because it suggests that the
command import failed and that the execution should stop. However, in
this case, lldb will continue the execution with the command added
previously by the user.
To prevent that, this patch updates every first-party lldb-packaged
custom commands to override commands that were pre-imported in lldb.
Differential Revision: https://reviews.llvm.org/D140293
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
When using interactive crashlog from an IDE, it can happen that the user
already have the `command script import lldb.macosx.crashlog` command on
their `lldbinit` file.
That leads to showing some message:
```
error: cannot add command: user command exists and force replace not set
error: cannot add command: user command exists and force replace not set
```
This leads to confusion because the crashlog symbolication continues and
succeeds even after these errors.
To address that, the crashlog commands get overridden everytime the
script get re-imported.
rdar://103403943
Differential Revision: https://reviews.llvm.org/D140113
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch should fix an undefined behaviour that's happening when
parsing a crash report from an IDE. In the previous implementation, the
CrashLogParser base class would use the `__new__` static class method to
create the right parser instance depending on the crash report type.
For some reasons, the derived parser initializer wouldn't be called when
running the command from an IDE, so this patch refactors the
CrashLogParser code to replace the use of the `__new__` method with a
factory `create` static method.
rdar://100527640
Differential Revision: https://reviews.llvm.org/D139951
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces both the Scripted Platform python base
implementation and an example for it.
The base implementation is embedded in lldb python module under
`lldb.plugins.scripted_platform`.
This patch also refactor the various SWIG methods to create scripted
objects into a single method, that is now shared between the Scripted
Platform, Process and Thread. It also replaces the target argument by a
execution context object.
Differential Revision: https://reviews.llvm.org/D139250
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch improves the ScriptedPythonInterface::Dispatch method to
support passing lldb_private types to the python implementation.
This will allow, for instance, the Scripted Process python implementation
to report errors when reading memory back to lldb.
To do so, the Dispatch method will transform the private types in the
parameter pack into `PythonObject`s to be able to pass them down to the
python methods.
Then, if the call succeeded, the transformed arguments will be converted
back to their original type and re-assigned in the parameter pack, to
ensure pointers and references behaviours are preserved.
This patch also updates various scripted process python class and tests
to reflect this change.
rdar://100030995
Differential Revision: https://reviews.llvm.org/D134033
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Allow `in_call_stack` to be imported in either of the following ways:
```
command script import path/to/in_call_stack.py
command script import lldb.utils.in_call_stack
```
rdar://102249295
Differential Revision: https://reviews.llvm.org/D137860
This patch replaces the backing file path key to "file_path" to keep it
consistent.
rdar://101652618
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
It can happen that the originator of a crash report doesn't have access
to certain images. When that's the case, ReportCrash won't show the
source info in the crash report stack frames, but only the stack address
and image name.
This patch fixes a bug in the crashlog stackframe parser regular
expression to optionally match the source info group.
rdar://101934135
Differential Revision: https://reviews.llvm.org/D137466
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
For an exception crashlog, the thread backtraces aren't usually very helpful
and instead, developpers look at the "Application Specific Backtrace" that
was generated by `objc_exception_throw`.
LLDB could already parse and symbolicate these Application Specific Backtraces
for regular textual-based crashlog, so this patch adds support to parse them
in JSON crashlogs, and materialize them a HistoryThread extending the
crashed ScriptedThread.
This patch also includes the Application Specific Information messages
as part of the process extended crash information log. To do so, the
ScriptedProcess Python interface has a new GetMetadata method that
returns an arbitrary dictionary with data related to the process.
rdar://93207586
Differential Revision: https://reviews.llvm.org/D126260
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch updates the image_regex_uuid matcher to match null-UUID
images in the plain text crashlog parser.
It updates the regex to match one or more '?' characters or the image
full path.
rdar://100904019
Differential Revision: https://reviews.llvm.org/D135482
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch adds support for 32bit stack frame addresses in the `crashlog`
command.
For crash reports that are generated from a arm64_32 process, `PAGEZERO`
is loaded at 0x00004000 so no code address will be less than 0x4000.
This patch changes the crashlog frame address regex group to match
addresses as small as 4 hex characters.
rdar://100805026
Differential Revision: https://reviews.llvm.org/D135310
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
The python "open" function will use the default encoding for the
locale (the result of "locale.getpreferredencoding()"). Explicitly set
the locale to utf-8 when opening the crashlog for writing, as there may
be non-ascii symbols in there (for example, Swift uses "τ" to indicate
generic parameters).
rdar://101402755
Differential Revision: https://reviews.llvm.org/D136798
I went over the output of the following mess of a command:
(ulimit -m 2000000; ulimit -v 2000000; git ls-files -z | parallel
--xargs -0 cat | aspell list --mode=none --ignore-case | grep -E
'^[A-Za-z][a-z]*$' | sort | uniq -c | sort -n | grep -vE '.{25}' |
aspell pipe -W3 | grep : | cut -d' ' -f2 | less)
and proceeded to spend a few days looking at it to find probable typos
and fixed a few hundred of them in all of the llvm project (note, the
ones I found are not anywhere near all of them, but it seems like a
good start).
Differential revision: https://reviews.llvm.org/D131122
This patch improve exception reporting when loading a crash report in a
scripted process. Now, we parse the `exception` dictionary from the
crash report use it the create a higher fidelity `MachException` stop info.
This patch also updates the test to reflect that change.
rdar://97096486
Differential Revision: https://reviews.llvm.org/D131086
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch parses CrashLog exception data from the raw
text format and adapts it to the new JSON format.
This is necessary for feature parity between the 2 formats.
Differential Revision: https://reviews.llvm.org/D131719
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
A spiritual follow up to D131032. I noticed some regex could be simplified.
This does some of the following:
1. Removes unused capture groups
2. Uses non-capturing `(?:...)` groups where grouping is needed but capturing isn't
3. Removes trailing `.*`
4. Uses `\d` over `[0-9]`
5. Uses raw strings
6. Uses `{N,}` to indicate N-or-more
Also improves the call site of a `re.findall`.
Differential Revision: https://reviews.llvm.org/D131305
This patch introduces a new option to the crashlog command to get the
the script version.
Since `crashlog.py` is not actually versioned, this returns lldb's
version instead.
rdar://98392669
Differential Revision: https://reviews.llvm.org/D131542
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch changes the CrashLogParser class to be both the base class
and a Factory for the JSONCrashLogParser & TextCrashLogParser.
That should help remove some code duplication and ensure both class
have a parse method.
Differential Revision: https://reviews.llvm.org/D131085
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Sometimes, it can happen that a crash report has null images in its list
of used binaries. This manifests like such:
```
0x0 - 0xffffffffffffffff ??? (*) <00000000-0000-0000-0000-000000000000> ???
```
When fetching debug symbols to symbolicate the crashlog stackframe,
having null images causes `dsymForUUID` to hang for few seconds.
This patch addresses that by skipping null images from being load by the
scripted process.
rdar://97419487
Differential Revision: https://reviews.llvm.org/D131038
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces a new option for the interactive crashlog mode,
that will prevent it from dumping the `process status` & `thread backtrace`
output to the debugger console.
This is necessary when lldb in running from an IDE, to prevent flooding
the console with information that should be already present in the UI.
rdar://96813296
Differential Revision: https://reviews.llvm.org/D131036
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
In can happen when creating stackshot crash report that that key is missing.
Moreover, we try to parse that key but don't use it, or need it, since we
fetch images and symbolicate the stackframes using the binaries UUIDs.
This is why this patch removes everything that is related to the
`process_path`/`procPath` parsing.
rdar://95054188
Differential Revision: https://reviews.llvm.org/D131033
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch updates the regular expression matching stackframes in
crashlog to allow addresses that are 7 characters long and more (vs. 8
characters previously).
It changes the `0x[0-9a-fA-F]{7}[0-9a-fA-F]+` by `0x[0-9a-fA-F]{7,}`.
rdar://97684839
Differential Revision: https://reviews.llvm.org/D131032
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch allows the crashlog script to surface its errors to lldb by
using the provided SBCommandReturnObject argument.
rdar://95048193
Differential Revision: https://reviews.llvm.org/D129614
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch introduces a new flag for the interactive crashlog mode, that
allow the user to specify, which target to use to create the scripted
process.
This can be very useful when lldb already have few targets created:
Instead of taking the first one (zeroth index), we will use that flag to
create a new target. If the user didn't provide a target path, we will rely
on the symbolicator to create a targer.If that fails and there are already
some targets loaded in lldb, we use the first one.
rdar://94682869
Differential Revision: https://reviews.llvm.org/D129611
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch changes the `crashlog` command behavior to print the help
message if no argument was provided with the command.
rdar://94576026
Differential Revision: https://reviews.llvm.org/D127362
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
