This patch re-lands f0731d5b61 with more fixes and improvements.
First, this patch removes `__eq__` implementations from classes that
didn't implemented `operator!=` on the C++ implementation.
This patch removes sphinx document generation for special members such
as `__len__`, since there is no straightforward way to skip class that
don't implement them. We also don't want to introduce a change in
behavior by implementing artifical special members for classes that are
missing them.
Finally, this patch improve the ergonomics of some classes by
implementing special members where it makes sense, i.e. `hex(SBFrame)`
is equivalent to `SBFrame.GetPC()`.
Differential Revision: https://reviews.llvm.org/D159017
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This reverts 3 commit:
- f0731d5b61.
- 8e0a087571.
- f2f5d6fb8d.
This changes were introduced to silence the warnings that are printed
when generating the lldb module documentation for the website but it
changed the python bindings and causes test failures on the macos bot:
https://green.lab.llvm.org/green/job/lldb-cmake/59438/
We will have to consider other options to silence these warnings.
This reverts commit 18f1c1ace7 and fix the
build failure issues introduced because of the `STRING_EXTENSION_OUTSIDE`
swig macros.
Differential Revision: https://reviews.llvm.org/D159017
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch does various things to silence the warnings that show up when
generating the website documentation.
First, this patch adds the missing definition for special member methods
in every SBAPI class. If the class cannot implement one of the special
member method, we just define it as a null operation (pass).
This should fix the following warnings:
```
WARNING: missing attribute __int__ in object lldb.SB*
WARNING: missing attribute __len__ in object lldb.SB*
WARNING: missing attribute __hex__ in object lldb.SB*
WARNING: missing attribute __oct__ in object lldb.SB*
WARNING: missing attribute __iter__ in object lldb.SB*
```
Then, it un-skips the various `static` methods that we didn't generate
the methods for, since it's not necessary thanks to the automod-api module.
Finally, this comments out the `_static` directory in the sphinx config,
since we don't need it anymore.
Differential Revision: https://reviews.llvm.org/D159017
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
SWIG 4 was released in 2019 and has been the de-facto standard for a
while now. All bots are running SWIG 4.0 or later.
This was motivated by #64279 which discovered that 662548c broke the
LLDB build with SWIG 3 on Windows.
Differential revision: https://reviews.llvm.org/D156804
This patch adds the ability to pass native types from the script
interpreter to methods that use a {SB,}StructuredData argument.
To do so, this patch changes the `ScriptedObject` struture that holds
the pointer to the script object as well as the originating script
interpreter language. It also exposes that to the SB API via a new class
called `SBScriptObject`.
This structure allows the debugger to parse the script object and
convert it to a StructuredData object. If the type is not compatible
with the StructuredData types, we will store its pointer in a
`StructuredData::Generic` object.
This patch also adds some SWIG typemaps that checks the input argument to
ensure it's either an SBStructuredData object, in which case it just
passes it throught, or a python object that is NOT another SB type, to
provide some guardrails for the user.
rdar://111467140
Differential Revision: https://reviews.llvm.org/D155161
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Many SB classes have public constructors or methods involving types that
are private. Some are more obvious (e.g. containing lldb_private in the
name) than others (lldb::FooSP is usually std::shared_pointer<lldb_private::Foo>).
This commit explicitly does not address FileSP, so I'm leaving that one
alone for now.
Some of these were for other SB classes to use and should have been made
protected/private with a friend class entry added. Some of these were
public for some of the swig python helpers to use. I put all of those
functions into a class and made them static methods. The relevant SB
classes mark that class as a friend so they can access those
private/protected members.
I've also removed an outdated SBStructuredData test (can you guess which
constructor it was using?) and updated the other relevant tests.
Differential Revision: https://reviews.llvm.org/D150157
This patch improves breakpoint management when doing interactive
scripted process debugging.
In other to know which process set a breakpoint, we need to do some book
keeping on the multiplexer scripted process. When initializing the
multiplexer, we will first copy breakpoints that are already set on the
driving target.
Everytime we launch or resume, we should copy breakpoints from the
multiplexer to the driving process.
When creating a breakpoint from a child process, it needs to be set both
on the multiplexer and on the driving process. We also tag the created
breakpoint with the name and pid of the originator process.
This patch also implements all the requirement to achieve proper
breakpoint management. That involves:
- Adding python interator for breakpoints and watchpoints in SBTarget
- Add a new `ScriptedProcess.create_breakpoint` python method
Differential Revision: https://reviews.llvm.org/D148548
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
If we have a `%typemap(freearg)` that frees the argument, we shouldn't
free it manually on an error path before calling `SWIG_fail`.
`SWIG_fail` will already free the memory in this case, and doing it
manually results in a double free.
Differential Revision: https://reviews.llvm.org/D147007
Python 3 doesn't have a distinction between PyInt and PyLong, it's all
PyLong now.
This also fixes a bug in SetNumberFromObject. This used to crash LLDB:
```
lldb -o "script data=lldb.SBData(); data.SetDataFromUInt64Array([2**63])"
```
The problem happened in the PyInt path:
```
if (PyInt_Check(obj))
number = static_cast<T>(PyInt_AsLong(obj));
```
when obj doesn't fit in a signed long, `PyInt_AsLong` would fail with
"OverflowError: Python int too large to convert to C long".
The existing long path does the right thing, as it will call
`PyLong_AsUnsignedLongLong` for uint64_t.
Differential Revision: https://reviews.llvm.org/D146590
Adding a new SBDebugger::SetDestroyCallback() API.
This API can be used by any client to query for statistics/metrics before
exiting debug sessions.
Differential Revision: https://reviews.llvm.org/D143520
When using SBProcess::GetScriptedImplementation in python, if the
process has a valid implementation, we returned a reference of the
object without incrementing the reference counting. That causes the
interpreter to crash after accessing the reference several times.
This patch address this by incrementing the reference count when passing
the valid object reference.
Differential Revision: https://reviews.llvm.org/D145260
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch adds memory writing capabilities to the Scripted Process plugin.
This allows to user to get a target address and a memory buffer on the
python scripted process implementation that the user can make processing
on before performing the actual write.
This will also be used to write trap instruction to a real process
memory to set a breakpoint.
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch adds process attach capabilities to the ScriptedProcess
plugin. This doesn't really expects a PID or process name, since the
process state is already script, however, this allows to create a
scripted process without requiring to have an executuble in the target.
In order to do so, this patch also turns the scripted process related
getters and setters from the `ProcessLaunchInfo` and
`ProcessAttachInfo` classes to a `ScriptedMetadata` instance and moves
it in the `ProcessInfo` class, so it can be accessed interchangeably.
This also adds the necessary SWIG wrappers to convert the internal
`Process{Attach,Launch}InfoSP` into a `SB{Attach,Launch}Info` to pass it
as argument the scripted process python implementation and convert it
back to the internal representation.
rdar://104577406
Differential Revision: https://reviews.llvm.org/D143104
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Instead of maintaining separate swig interface files, we can use the API
headers directly. They implement the exact same C++ APIs and we can
conditionally include the python extensions as needed. To remove the
swig extensions from the API headers when building the LLDB
framework, we can use the unifdef tool when it is available. Otherwise
we just copy them as-is.
Differential Revision: https://reviews.llvm.org/D142926
This patch introduces a new `GetScriptedImplementation` method to the
SBProcess class in the SBAPI. It will allow users of Scripted Processes to
fetch the scripted implementation object from to script interpreter to be
able to interact with it directly (without having to go through lldb).
This allows to user to perform action that are not specified in the
scripted process interface, like calling un-specified methods, but also
to enrich the implementation, by passing it complex objects.
Differential Revision: https://reviews.llvm.org/D143236
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch should fix the creation and addition of the `scripted_platform`
python module into the `lldb.plugins` module.
Previously, we were creating the `plugins` submodule, each time with a
different source file (either `scripted_process` or `scripted_platform`).
The removes the redundant `create_python_package` call and group both
python source files toghether.
Differential Revision: https://reviews.llvm.org/D143122
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch conditionaly removes the `-py3` swig flag that was used to
generate python3 bindings, since it's unsued since SWIG 4.1.0.
```
Deprecated command line option: -py3. Ignored, this option is no longer supported
```
This also removes the `-shadow` flag that have been deprecated since 2002.
Differential Revision: https://reviews.llvm.org/D142245
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
LLDB only supports Python3 now, so the `six` shim for Python2 is no longer necessary.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D142140
This patch introduces both the Scripted Platform python base
implementation and an example for it.
The base implementation is embedded in lldb python module under
`lldb.plugins.scripted_platform`.
This patch also refactor the various SWIG methods to create scripted
objects into a single method, that is now shared between the Scripted
Platform, Process and Thread. It also replaces the target argument by a
execution context object.
Differential Revision: https://reviews.llvm.org/D139250
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
The current implementation nicely takes into account when the python interpreter is symlinked (or transitively within a symlinked directory). Sadly, `os.path.islink` returns `false` on Windows if instead of Windows symlinks, junctions are used. This has caused me issues after I started using `scoop` as my package manager on Windows, which creates junctions instead of symlinks.
The fix proposed in this patch is to check whether `realpath` returns a different path to `exe`, and if it does, to simply try again with that path.
The code could also be simplified since `sys.executable` is guaranteed to be absolute, and `os.readlink`, which can return a relative path, is no longer used.
Tested on Windows 11 with Python 3.11 as interpreter and Ubuntu 18.04 with Python 3.6
Differential Revision: https://reviews.llvm.org/D141042
Some headers in LLDB work only when considered as textual inclusion, but not if one attempts to use them on their own or with a different context.
- python-typemaps.h: uses Python definitions without using "Python.h".
- RISCVCInstructions.h uses RISC-V register enums without including the enums header.
- RISCVInstructions.h includes EmulateInstructionRISCV.h, but is unnecessary since we forward-declare EmulateInstructionRISCV anyway. Including the header is problematic because EmulateInstructionRISCV.h uses DecodeResult which isn't defined until later in RISCVInstructions.h.
This makes LLDB build cleanly with the "parse_headers" feature [1]. I'm not sure what the analagous CMake option is.
[1] I didn't find public documentation but @MaskRay wrote this up: https://maskray.me/blog/2022-09-25-layering-check-with-clang#parse_headers
Reviewed By: labath, MaskRay
Differential Revision: https://reviews.llvm.org/D138310
This patch improves the ScriptedPythonInterface::Dispatch method to
support passing lldb_private types to the python implementation.
This will allow, for instance, the Scripted Process python implementation
to report errors when reading memory back to lldb.
To do so, the Dispatch method will transform the private types in the
parameter pack into `PythonObject`s to be able to pass them down to the
python methods.
Then, if the call succeeded, the transformed arguments will be converted
back to their original type and re-assigned in the parameter pack, to
ensure pointers and references behaviours are preserved.
This patch also updates various scripted process python class and tests
to reflect this change.
rdar://100030995
Differential Revision: https://reviews.llvm.org/D134033
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
This patch adds a new matching method for data formatters, in addition
to the existing exact typename and regex-based matching. The new method
allows users to specify the name of a Python callback function that
takes a `SBType` object and decides whether the type is a match or not.
Here is an overview of the changes performed:
- Add a new `eFormatterMatchCallback` matching type, and logic to handle
it in `TypeMatcher` and `SBTypeNameSpecifier`.
- Extend `FormattersMatchCandidate` instances with a pointer to the
current `ScriptInterpreter` and the `TypeImpl` corresponding to the
candidate type, so we can run registered callbacks and pass the type
to them. All matcher search functions now receive a
`FormattersMatchCandidate` instead of a type name.
- Add some glue code to ScriptInterpreterPython and the SWIG bindings to
allow calling a formatter matching callback. Most of this code is
modeled after the equivalent code for watchpoint callback functions.
- Add an API test for the new callback-based matching feature.
For more context, please check the RFC thread where this feature was
originally discussed:
https://discourse.llvm.org/t/rfc-python-callback-for-data-formatters-type-matching/64204/11
Differential Revision: https://reviews.llvm.org/D135648
When attempting to use SWIG's `-builtin` flag, there were a few compile
failures caused by a mismatch between return type and return value. In those
cases, the return type was `int` but many of the type maps assume returning
`NULL`/`nullptr` (only the latter caused compile failures).
This fix abstracts failure paths to use the `SWIG_fail` macro, which performs
`goto fail;`. Each of the generated functions contain a `fail` label, which
performs any resource cleanup and returns the appropriate failure value.
This change isn't strictly necessary at this point, but seems like the right
thing to do, and for anyone who tries `-builtin` later, it resolves those
issues.
Differential Revision: https://reviews.llvm.org/D133961
D120690 introduced `eBasicTypeChar8` but missed proper documentation order. This also introduces the missing bindings data on Swig, which should correspond with the documented information.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D116136
This patch introduces a new type of ScriptedProcess: CrashLogScriptedProcess.
It takes advantage of lldb's crashlog parsers and Scripted Processes to
reconstruct a static debugging session with symbolicated stackframes, instead
of just dumping out everything in the user's terminal.
The crashlog command also has an interactive mode that only provide a
very limited experience. This is why this patch removes all the logic
for this interactive mode and creates CrashLogScriptedProcess instead.
This will fetch and load all the libraries that were used by the crashed
thread and re-create all the frames artificially.
rdar://88721117
Differential Revision: https://reviews.llvm.org/D119501
Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>
Update `__init__.py` generation to implement `__lldb_init_module`, which calls
`__lldb_init_module` on submodules that define it.
This allows the use case where a user runs `command script import lldb.macosx`.
With this change, the `__lldb_init_module` function in `crashlog.py` and
`heap.py` will be run, which is where command registration is occurring.
Differential Revision: https://reviews.llvm.org/D119179
Return our PythonObject wrappers instead of raw PyObjects (obfuscated as
void *). This ensures that ownership (reference counts) of python
objects is automatically tracked.
Differential Revision: https://reviews.llvm.org/D117462
The lldbconfig module was necessary to run the LLDB test suite against a
reproducer. Since this functionality has been removed, the module is no
longer necessary.
Unlike the rest of our SB objects, SBEvent and SBCommandReturnObject
have the ability to hold non-owning pointers to their non-SB
counterparts. This makes it hard to ensure the SB objects do not become
dangling once their backing object goes away.
While we could make these two objects behave like others, that would
require plubming even more shared pointers through our internal code
(Event objects are mostly prepared for it, CommandReturnObject are not).
Doing so seems unnecessarily disruptive, given that (unlike for some of
the other objects) I don't see any good reason why would someone want to
hold onto these objects after the function terminates.
For that reason, this patch implements a different approach -- the SB
objects will still hold non-owning pointers, but they will be reset to
the empty/default state as soon as the function terminates. This python
code will not crash if the user decides to store these objects -- but
the objects themselves will be useless/empty.
Differential Revision: https://reviews.llvm.org/D116162
This starts to fix the other half of the lifetime problems in this code
-- dangling references. SB objects created on the stack will go away
when the function returns, which is a problem if the python code they
were meant for stashes a reference to them somewhere. Most of the time
this goes by unnoticed, as the code rarely has a reason to store these,
but in case it does, we shouldn't respond by crashing.
This patch fixes the management for a couple of SB objects (Debugger,
Frame, Thread). The SB objects are now created on the heap, and
their ownership is immediately passed on to SWIG, which will ensure they
are destroyed when the last python reference goes away. I will handle
the other objects in separate patches.
I include one test which demonstrates the lifetime issue for SBDebugger.
Strictly speaking, one should create a test case for each of these
objects and each of the contexts they are being used. That would require
figuring out how to persist (and later access) each of these objects.
Some of those may involve a lot of hoop-jumping (we can run python code
from within a frame-format string). I don't think that is
necessary/worth it since the new wrapper functions make it very hard to
get this wrong.
Differential Revision: https://reviews.llvm.org/D115925
I've found my recent ventures into the swig land painful because
of the strange way they are formatted. This patch attempts to alleviate
future headaches by formatting these files into something resembling the
normal llvm style.
Unfortunately, completely formatting these files automatically does not
work because clang format gets confused by swigs % syntax, so I have
employed a hybrid approach where I formatted blocks of c++ code with
clang-format and then manually massaged the code until it looked
reasonable (and compiled).
I don't expect these files to remain perfectly formatted (although, if
one's editor is configured to configure the current line/block on
request, one can get pretty good results by using it judiciously), but
at least it will prevent the (mangled form of the) old lldb style being
proliferated endlessly.
Differential Revision: https://reviews.llvm.org/D115736
StructuredDataImpl ownership semantics is unclear at best. Various
structures were holding a non-owning pointer to it, with a comment that
the object is owned somewhere else. From what I was able to gather that
"somewhere else" was the SBStructuredData object, but I am not sure that
all created object eventually made its way there. (It wouldn't matter
even if they did, as we are leaking most of our SBStructuredData
objects.)
Since StructuredDataImpl is just a collection of two (shared) pointers,
there's really no point in elaborate lifetime management, so this patch
replaces all StructuredDataImpl pointers with actual objects or
unique_ptrs to it. This makes it much easier to resolve SBStructuredData
leaks in a follow-up patch.
Differential Revision: https://reviews.llvm.org/D114791
Some pythons are configured to set platlib somewhere outside of their
sys.prefix. It's important that we at least use some reasonable
default for LLDB_PYTHON_RELATIVE_PATH even in that case, because
even if the user overrides it on the cmake invocation, cmake will
still be called without the override in order to build tablegen.
Reviewed By: JDevlieghere, clayborg
Differential Revision: https://reviews.llvm.org/D114973
The LLDBSWIGPython functions had (at least) two problems:
- There wasn't a single source of truth (a header file) for the
prototypes of these functions. This meant that subtle differences
in copies of function declarations could go by undetected. And
not-so-subtle differences would result in strange runtime failures.
- All of the declarations had to have an extern "C" interface, because
the function definitions were being placed inside and extert "C" block
generated by swig.
This patch fixes both problems by moving the function definitions to the
%header block of the swig files. This block is not surrounded by extern
"C", and seems more appropriate anyway, as swig docs say it is meant for
"user-defined support code" (whereas the previous %wrapper code was for
automatically-generated wrappers).
It also puts the declarations into the SWIGPythonBridge header file
(which seems to have been created for this purpose), and ensures it is
included by all code wishing to define or use these functions. This
means that any differences in the declaration become a compiler error
instead of a runtime failure.
Differential Revision: https://reviews.llvm.org/D114369
