This patch revives the effort to get this Phabricator patch into
upstream:
https://reviews.llvm.org/D137900
This patch was accepted before in Phabricator but I found some
-gsimple-template-names issues that are fixed in this patch.
A fixed up version of the description from the original patch starts
now.
This patch started off trying to fix Module::FindFirstType() as it
sometimes didn't work. The issue was the SymbolFile plug-ins didn't do
any filtering of the matching types they produced, and they only looked
up types using the type basename. This means if you have two types with
the same basename, your type lookup can fail when only looking up a
single type. We would ask the Module::FindFirstType to lookup "Foo::Bar"
and it would ask the symbol file to find only 1 type matching the
basename "Bar", and then we would filter out any matches that didn't
match "Foo::Bar". So if the SymbolFile found "Foo::Bar" first, then it
would work, but if it found "Baz::Bar" first, it would return only that
type and it would be filtered out.
Discovering this issue lead me to think of the patch Alex Langford did a
few months ago that was done for finding functions, where he allowed
SymbolFile objects to make sure something fully matched before parsing
the debug information into an AST type and other LLDB types. So this
patch aimed to allow type lookups to also be much more efficient.
As LLDB has been developed over the years, we added more ways to to type
lookups. These functions have lots of arguments. This patch aims to make
one API that needs to be implemented that serves all previous lookups:
- Find a single type
- Find all types
- Find types in a namespace
This patch introduces a `TypeQuery` class that contains all of the state
needed to perform the lookup which is powerful enough to perform all of
the type searches that used to be in our API. It contain a vector of
CompilerContext objects that can fully or partially specify the lookup
that needs to take place.
If you just want to lookup all types with a matching basename,
regardless of the containing context, you can specify just a single
CompilerContext entry that has a name and a CompilerContextKind mask of
CompilerContextKind::AnyType.
Or you can fully specify the exact context to use when doing lookups
like: CompilerContextKind::Namespace "std"
CompilerContextKind::Class "foo"
CompilerContextKind::Typedef "size_type"
This change expands on the clang modules code that already used a
vector<CompilerContext> items, but it modifies it to work with
expression type lookups which have contexts, or user lookups where users
query for types. The clang modules type lookup is still an option that
can be enabled on the `TypeQuery` objects.
This mirrors the most recent addition of type lookups that took a
vector<CompilerContext> that allowed lookups to happen for the
expression parser in certain places.
Prior to this we had the following APIs in Module:
```
void
Module::FindTypes(ConstString type_name, bool exact_match, size_t max_matches,
llvm::DenseSet<lldb_private::SymbolFile *> &searched_symbol_files,
TypeList &types);
void
Module::FindTypes(llvm::ArrayRef<CompilerContext> pattern, LanguageSet languages,
llvm::DenseSet<lldb_private::SymbolFile *> &searched_symbol_files,
TypeMap &types);
void Module::FindTypesInNamespace(ConstString type_name,
const CompilerDeclContext &parent_decl_ctx,
size_t max_matches, TypeList &type_list);
```
The new Module API is much simpler. It gets rid of all three above
functions and replaces them with:
```
void FindTypes(const TypeQuery &query, TypeResults &results);
```
The `TypeQuery` class contains all of the needed settings:
- The vector<CompilerContext> that allow efficient lookups in the symbol
file classes since they can look at basename matches only realize fully
matching types. Before this any basename that matched was fully realized
only to be removed later by code outside of the SymbolFile layer which
could cause many types to be realized when they didn't need to.
- If the lookup is exact or not. If not exact, then the compiler context
must match the bottom most items that match the compiler context,
otherwise it must match exactly
- If the compiler context match is for clang modules or not. Clang
modules matches include a Module compiler context kind that allows types
to be matched only from certain modules and these matches are not needed
when d oing user type lookups.
- An optional list of languages to use to limit the search to only
certain languages
The `TypeResults` object contains all state required to do the lookup
and store the results:
- The max number of matches
- The set of SymbolFile objects that have already been searched
- The matching type list for any matches that are found
The benefits of this approach are:
- Simpler API, and only one API to implement in SymbolFile classes
- Replaces the FindTypesInNamespace that used a CompilerDeclContext as a
way to limit the search, but this only worked if the TypeSystem matched
the current symbol file's type system, so you couldn't use it to lookup
a type in another module
- Fixes a serious bug in our FindFirstType functions where if we were
searching for "foo::bar", and we found a "baz::bar" first, the basename
would match and we would only fetch 1 type using the basename, only to
drop it from the matching list and returning no results
Add a new API in SBTarget to Load Core from a SBFile.
This will enable a target to load core from a file descriptor.
So that in coredumper, we don't need to write core file to disk, instead
we can pass the input file descriptor to lldb directly.
Test:
```
(lldb) script
Python Interactive Interpreter. To exit, type 'quit()', 'exit()' or Ctrl-D.
>>> file_object = open("/home/hyubo/210hda79ms32sr0h", "r")
>>> fd=file_object.fileno()
>>> file = lldb.SBFile(fd,'r', True)
>>> error = lldb.SBError()
>>> target = lldb.debugger.CreateTarget(None)
>>> target.LoadCore(file,error)
SBProcess: pid = 56415, state = stopped, threads = 1
```
Prior to this patch, each core file plugin (ObjectFileMachO.cpp and
ObjectFileMinindump.cpp) would calculate the address ranges to save in
different ways. This patch adds a new function to Process.h/.cpp:
```
Status Process::CalculateCoreFileSaveRanges(lldb::SaveCoreStyle core_style, CoreFileMemoryRanges &ranges);
```
The patch updates the ObjectFileMachO::SaveCore(...) and
ObjectFileMinindump::SaveCore(...) to use same code. This will allow
core files to be consistent with the lldb::SaveCoreStyle across
different core file creators and will allow us to add new core file
saving features that do more complex things in future patches.
Follows the format laid out in the Arm manual, AArch32 only fields are
ignored.
```
(lldb) register read fpcr
fpcr = 0x00000000
= (AHP = 0, DN = 0, FZ = 0, RMMode = 0, FZ16 = 0, IDE = 0, IXE = 0, UFE = 0, OFE = 0, DZE = 0, IOE = 0)
```
Tests use the first 4 fields that we know are always present.
Converted all the HCWAP defines to `UL` because I'm bound to
forget one if I don't do it now.
This one is easy because none of the fields depend on extensions. Only
thing to note is that I've ignored some AArch32 only fields.
```
(lldb) register read fpsr
fpsr = 0x00000000
= (QC = 0, IDC = 0, IXC = 0, UFC = 0, OFC = 0, DZC = 0, IOC = 0)
```
The contents of which are mostly SPSR_EL1 as shown in the Arm manual,
with a few adjustments for things Linux says userspace shouldn't concern
itself with.
```
(lldb) register read cpsr
cpsr = 0x80001000
= (N = 1, Z = 0, C = 0, V = 0, SS = 0, IL = 0, ...
```
Some fields are always present, some depend on extensions. I've checked
for those extensions using HWCAP and HWCAP2.
To provide this for core files and live processes I've added a new class
LinuxArm64RegisterFlags. This is a container for all the registers we'll
want to have fields and handles detecting fields and updating register
info.
This is used by the native process as follows:
* There is a global LinuxArm64RegisterFlags object.
* The first thread takes a mutex on it, and updates the fields.
* Subsequent threads see that detection is already done, and skip it.
* All threads then update their own copy of the register information
with pointers to the field information contained in the global object.
This means that even though every thread will have the same fields, we
only detect them once and have one copy of the information.
Core files instead have a LinuxArm64RegisterFlags as a member, because
each core file could have different saved capabilities. The logic from
there is the same but we get HWACP values from the corefile note.
This handler class is Linux specific right now, but it can easily be
made more generic if needed. For example by using LLVM's FeatureBitset
instead of HWCAPs.
Updating register info is done with string comparison, which isn't
ideal. For CPSR, we do know the register number ahead of time but we do
not for other registers in dynamic register sets. So in the interest of
consistency, I'm going to use string comparison for all registers
including cpsr.
I've added tests with a core file and live process. Only checking for
fields that are always present to account for CPU variance.
This patch tentatively fixes TestScriptedProcess.py which has been
failing on the `lldb-arm-ubuntu` & `lldb-aarch64-ubuntu` bots:
- https://lab.llvm.org/buildbot/#/builders/17/builds/44965
- https://lab.llvm.org/buildbot/#/builders/96/builds/48152
According to the test log, on those systems, the clang driver that build
the test binary doesn't have the `-m` flag to specify the architure so
this patch replaces it with the `-target` flag using `clang -dumpmachine`
to get the host triple.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch enforces that every scripted object implements all the
necessary abstract methods.
Every scripted affordance language interface can implement a list of
abstract methods name that checked when the object is instanciated.
Since some scripting affordances implementations can be derived from
template base classes, we can't check the object dictionary since it
will contain the definition of the base class, so instead, this checks
the scripting class dictionary.
Previously, for the various python interfaces, we used
`ABC.abstractmethod` decorators but this is too language specific and
doesn't work for scripting affordances that are not derived from
template base classes (i.e OperatingSystem, ScriptedThreadPlan, ...), so
this patch provides generic/language-agnostic checks for every scripted
affordance.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch enforces that every scripted object implements all the
necessary abstract methods.
Every scripted affordance language interface can implement a list of
abstract methods name that checked when the object is instanciated.
Since some scripting affordances implementations can be derived from
template base classes, we can't check the object dictionary since it
will contain the definition of the base class, so instead, this checks
the scripting class dictionary.
Previously, for the various python interfaces, we used
`ABC.abstractmethod` decorators but this is too language specific and
doesn't work for scripting affordances that are not derived from
template base classes (i.e OperatingSystem, ScriptedThreadPlan, ...), so
this patch provides generic/language-agnostic checks for every scripted
affordance.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Add the ability to get a C++ vtable ValueObject from another
ValueObject.
This patch adds the ability to ask a ValueObject for a ValueObject that
represents the virtual function table for a C++ class. If the
ValueObject is not a C++ class with a vtable, a valid ValueObject value
will be returned that contains an appropriate error. If it is successful
a valid ValueObject that represents vtable will be returned. The
ValueObject that is returned will have a name that matches the demangled
value for a C++ vtable mangled name like "vtable for <class-name>". It
will have N children, one for each virtual function pointer. Each
child's value is the function pointer itself, the summary is the
symbolication of this function pointer, and the type will be a valid
function pointer from the debug info if there is debug information
corresponding to the virtual function pointer.
The vtable SBValue will have the following:
- SBValue::GetName() returns "vtable for <class>"
- SBValue::GetValue() returns a string representation of the vtable
address
- SBValue::GetSummary() returns NULL
- SBValue::GetType() returns a type appropriate for a uintptr_t type for
the current process
- SBValue::GetLoadAddress() returns the address of the vtable adderess
- SBValue::GetValueAsUnsigned(...) returns the vtable address
- SBValue::GetNumChildren() returns the number of virtual function
pointers in the vtable
- SBValue::GetChildAtIndex(...) returns a SBValue that represents a
virtual function pointer
The child SBValue objects that represent a virtual function pointer has
the following values:
- SBValue::GetName() returns "[%u]" where %u is the vtable function
pointer index
- SBValue::GetValue() returns a string representation of the virtual
function pointer
- SBValue::GetSummary() returns a symbolicated respresentation of the
virtual function pointer
- SBValue::GetType() returns the function prototype type if there is
debug info, or a generic funtion prototype if there is no debug info
- SBValue::GetLoadAddress() returns the address of the virtual function
pointer
- SBValue::GetValueAsUnsigned(...) returns the virtual function pointer
- SBValue::GetNumChildren() returns 0
- SBValue::GetChildAtIndex(...) returns invalid SBValue for any index
Examples of using this API via python:
```
(lldb) script vtable = lldb.frame.FindVariable("shape_ptr").GetVTable()
(lldb) script vtable
vtable for Shape = 0x0000000100004088 {
[0] = 0x0000000100003d20 a.out`Shape::~Shape() at main.cpp:3
[1] = 0x0000000100003e4c a.out`Shape::~Shape() at main.cpp:3
[2] = 0x0000000100003e7c a.out`Shape::area() at main.cpp:4
[3] = 0x0000000100003e3c a.out`Shape::optional() at main.cpp:7
}
(lldb) script c = vtable.GetChildAtIndex(0)
(lldb) script c
(void ()) [0] = 0x0000000100003d20 a.out`Shape::~Shape() at main.cpp:3
```
This patch makes ScriptedThreadPlan conforming to the ScriptedInterface
& ScriptedPythonInterface facilities by introducing 2
ScriptedThreadPlanInterface & ScriptedThreadPlanPythonInterface classes.
This allows us to get rid of every ScriptedThreadPlan-specific SWIG
method and re-use the same affordances as other scripting offordances,
like Scripted{Process,Thread,Platform} & OperatingSystem.
To do so, this adds new transformer methods for `ThreadPlan`, `Stream` &
`Event`, to allow the bijection between C++ objects and their python
counterparts.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
These tests were failing on the LLDB public matrix build-bots for older
clang versions:
```
clang-7: warning: argument unused during compilation: '-nostdlib++' [-Wunused-command-line-argument]
error: invalid value 'c++20' in '-std=c++20'
note: use 'c++98' or 'c++03' for 'ISO C++ 1998 with amendments' standard
note: use 'gnu++98' or 'gnu++03' for 'ISO C++ 1998 with amendments and GNU extensions' standard
note: use 'c++11' for 'ISO C++ 2011 with amendments' standard
note: use 'gnu++11' for 'ISO C++ 2011 with amendments and GNU extensions' standard
note: use 'c++14' for 'ISO C++ 2014 with amendments' standard
note: use 'gnu++14' for 'ISO C++ 2014 with amendments and GNU extensions' standard
note: use 'c++17' for 'ISO C++ 2017 with amendments' standard
note: use 'gnu++17' for 'ISO C++ 2017 with amendments and GNU extensions' standard
note: use 'c++2a' for 'Working draft for ISO C++ 2020' standard
note: use 'gnu++2a' for 'Working draft for ISO C++ 2020 with GNU extensions' standard
make: *** [main.o] Error 1
```
The test fails because we try to compile it with `-std=c++20` (which is
required for std::chrono::{days,weeks,months,years}) on clang versions
that don't support the `-std=c++20` flag.
We could change the test to conditionally compile the C++20 parts of the
test based on the `-std=` flag and have two versions of the python
tests, one for the C++11 chrono features and one for the C++20 features.
This patch instead just disables the test on older clang versions
(because it's simpler and we don't really lose important coverage).
This patch introduces an `OperatingSystem` base implementation in the
`lldb` python module to make it easier for lldb users to write their own
implementation.
The `OperatingSystem` base implementation is derived itself from the
`ScriptedThread` base implementation since they share some common grounds.
To achieve that, this patch makes changes to the `ScriptedThread`
initializer since it gets called by the `OperatingSystem` initializer.
I also took the opportunity to document the `OperatingSystem` base
class and methods.
Differential Revision: https://reviews.llvm.org/D159315
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This is relanding of https://github.com/llvm/llvm-project/pull/69253.
`TestTemplatePackArgs.py` is passing now.
https://github.com/llvm/llvm-project/pull/68012/files added new data
formatters for LibStdC++ std::variant.
However, this formatter can crash if std::variant's index field has
invalid value (exceeds the number of template arguments).
This can happen if the current IP stops at a place std::variant is not
initialized yet.
This patch fixes the crash by ensuring the index is a valid value and
fix GetNthTemplateArgument() to make sure it is not crashing.
Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
The underlying timezone classes are being reimplemented in Swift, and these
strings will be Swift strings, without the ObjC `@` prefix. Leaving off the `@`
makes these tests usable both before and after the reimplmentation of
Foundation in Swift.
https://github.com/llvm/llvm-project/pull/68012/files added new data
formatters for LibStdC++ std::variant.
However, this formatter can crash if std::variant's index field has
invalid value (exceeds the number of template arguments).
This can happen if the current IP stops at a place std::variant is not
initialized yet.
This patch fixes the crash by ensuring the index is a valid value.
---------
Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
To get the number of children for a VectorType (i.e.,
a type declared with a `vector_size`/`ext_vector_type` attribute)
LLDB previously did following calculation:
1. Get byte-size of the vector container from Clang (`getTypeInfo`).
2. Get byte-size of the element type we want to interpret the array as.
(e.g., sometimes we want to interpret an `unsigned char vec[16]`
as a `float32[]`).
3. `numChildren = containerSize / reinterpretedElementSize`
However, for step 1, clang will return us the *aligned* container
byte-size.
So for a type such as `float __attribute__((ext_vector_type(3)))`
(which is an array of 3 4-byte floats), clang will round up the
byte-width of the array to `16`.
(see
[here](ab6a66dbec/clang/lib/AST/ASTContext.cpp (L1987-L1992)))
This means that for vectors where the size isn't a power-of-2, LLDB
will miscalculate the number of elements.
**Solution**
This patch changes step 1 such that we calculate the container size
as `numElementsInSource * byteSizeOfElement`.
The type formatter code is effectively considering empty strings as read
errors, which is wrong. The fix is very simple. We should rely on the
error object and stop checking the size. I also added a test.
https://github.com/llvm/llvm-project/pull/68012 works on my CentOS Linux
and Macbook but seems to fail for certain build bots. The error log
complains "No Value" check failure for `std::variant` but not very
actionable without a reproduce.
To unblock the build bots, I am commenting out the "No Value" checks.
Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
This reverts commit a7b78cac9a.
With updates to the tests.
TestWatchTaggedAddress.py: Updated the expected watchpoint types,
though I'm not sure there should be a differnt default for the two
ways of setting them, that needs to be confirmed.
TestStepOverWatchpoint.py: Skipped this everywhere because I think
what used to happen is you couldn't put 2 watchpoints on the same
address (after alignment). I guess that this is now allowed because
modify watchpoints aren't accounted for, but likely should be.
Needs investigating.
The test is expecting watchpoint hits that are valid on aarch64
systems but not on Intel. I need to update this test to match the
actual behavior on Intel macs, and don't want the CI bots to stay
broken while I get that figured out.
Watchpoints in lldb can be either 'read', 'write', or 'read/write'. This
is exposing the actual behavior of hardware watchpoints. gdb has a
different behavior: a "write" type watchpoint only stops when the
watched memory region *changes*.
A user is using a watchpoint for one of three reasons:
1. Want to find what is changing/corrupting this memory.
2. Want to find what is writing to this memory.
3. Want to find what is reading from this memory.
I believe (1) is the most common use case for watchpoints, and it
currently can't be done in lldb -- the user needs to continue every time
the same value is written to the watched-memory manually. I think gdb's
behavior is the correct one. There are some use cases where a developer
wants to find every function that writes/reads to/from a memory region,
regardless of value, I want to still allow that functionality.
This is also a bit of groundwork for my large watchpoint support
proposal
https://discourse.llvm.org/t/rfc-large-watchpoint-support-in-lldb/72116
where I will be adding support for AArch64 MASK watchpoints which watch
power-of-2 memory regions. A user might ask to watch 24 bytes, and a
MASK watchpoint stub can do this with a 32-byte MASK watchpoint if it is
properly aligned. And we need to ignore writes to the final 8 bytes of
that watched region, and not show those hits to the user.
This patch adds a new 'modify' watchpoint type and it is the default.
Re-landing this patch after addressing testsuite failures found in CI on
Linux, Intel machines, and windows.
rdar://108234227
