The remaining use of ConstString in StructuredData is the Dictionary
class. Internally it's backed by a `std::map<ConstString, ObjectSP>`.
I propose that we replace it with a `llvm::StringMap<ObjectSP>`.
Many StructuredData::Dictionary objects are ephemeral and only exist for
a short amount of time. Many of these Dictionaries are only produced
once and are never used again. That leaves us with a lot of string data
in the ConstString StringPool that is sitting there never to be used
again. Even if the same string is used many times for keys of different
Dictionary objects, that is something we can measure and adjust for
instead of assuming that every key may be reused at some point in the
future.
Quick comparisons of key data is likely not a concern with Dictionary,
but the use of `llvm::StringMap` means that lookups should be fast with
its hashing strategy.
Switching to a llvm::StringMap meant that the iteration order may be
different. To account for this when serializing/dumping the dictionary,
I added some code to sort the output by key before emitting anything.
Differential Revision: https://reviews.llvm.org/D159313
Add a new LC_NOTE for Mach-O corefiles, "proces metadata", which is a
JSON string. Currently there may be a `threads` key in the JSON,
and if `threads` is present, it is an array with the same number of
elements as there are LC_THREADs in the corefile. This patch adds
support for a `thread_id` key-value for each `thread` entry, to
supply a thread ID for that LC_THREAD.
Differential Revision: https://reviews.llvm.org/D158785
rdar://113037252
While working in support for SME's ZA register, I found a circumstance
where restoring ZA after SVE, when the current SVE mode is non-streaming,
will kick the process back into FPSIMD mode. Meaning the SVE values that
you just wrote are now cut off at 128 bit.
The fix for that is to write ZA then SVE. Problem with that
is, the current ReadAll/WriteAll makes a lot of assumptions about the
saved data length.
This patch changes the format so there is a "type" written before
each data block. This tells WriteAllRegisterValues what it's looking at
without brittle checks on length, or assumptions about ordering.
If we want to change the order of restoration, all we now have to
do is change the order of saving.
This exposes a bug where the TLS registers are not restored.
This will be fixed by https://reviews.llvm.org/D156512 in some form,
depending on what lands first.
Existing SVE tests certainly check restoration and when I got this
wrong, many, many tests failed. So I think we have enough coverage
already, and more will be coming with future ZA changes.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D156687
This always succeeds. While I'm here, document why we check the size
of p0 against the value of VG.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D157845
ConstString can be implicitly converted into a llvm::StringRef. This is
very useful in many places, but it also hides places where we are
creating a ConstString only to use it as a StringRef for the entire
lifespan of the ConstString object.
I locally removed the implicit conversion and found some of the places we
were doing this.
Differential Revision: https://reviews.llvm.org/D159237
This patch gets clang-6 building with LLDB. The move from makeArrayRef
to deduction guides in 984b800a03 is
tripping up clang-6 on Ubuntu 18.04.
Related to issue #64782.
qHostInfo / stop-reply packet / LC_NOTE "addrable bits" can all
specify either a single value for all address masks, or separate
masks for low and high memory addresses.
When the same number of addressing bits are used for all addresses,
we use the "low memory" address masks for everything. (or another
way, if the high address masks are not set, we use the low address
masks with the assumption that all memory is using the same mask --
the most common situation).
I was setting low and high address masks when I had a single value
from these metadata, but that gave the impression that the high
address mask was specified explicitly. After living on the code
a bit, it's clearly better to only set the high address masks when
we have a distinct high address mask value.
This patch is the minor adjustment to behave that way.
I'm rewriting this for 2 reasons:
1) Although a 1024 char buffer is probably enough space, the error
string may grow beyond that and we could lose some information. Using
StringStream (as we do in the rest of ProcessGDBRemote) seems like a
sensible solution.
2) I am planning on changing `UnixSignals::GetSignalAsCString`,
rewriting things with `llvm::formatv` will make that process easier.
Differential Revision: https://reviews.llvm.org/D158017
Add support for the `low_mem_addressing_bits` and
`high_mem_addressing_bits` keys in the stop reply packet,
in addition to the existing `addressing_bits`. Same
behavior as in the qHostInfo packet.
Clean up AddressableBits so we don't need to check if
any values have been set in the object before using it
to potentially update the Process address masks.
Differential Revision: https://reviews.llvm.org/D158041
Some Apple firmware environments store the UUID of the main binary
at a fixed address in low memory. Add that list of addresess to
ProcessMachCore to check for a UUID, and try to load it.
Differential Revision: https://reviews.llvm.org/D157756
On AArch64 systems, we may have different page table setups for
low memory and high memory, and therefore a different number of
bits used for addressing depending on which half of memory the
address is in.
This patch extends the qHostInfo and LC_NOTE "addrable bits" so
that it can specify the number of addressing bits in high memory
and in low memory separately. It builds on the patch I added in
https://reviews.llvm.org/D151292 where Process tracks the separate
address masks, and there is a user setting to set them manually.
Differential Revision: https://reviews.llvm.org/D157667
rdar://113225907
Since https://reviews.llvm.org/D157058 in libc++,
the base template for char_traits has been removed - it is only
provided for char, wchar_t, char8_t, char16_t and char32_t.
(Thus, to use basic_string with a type other than those, we'd need
to supply suitable traits ourselves.)
For this particular use, a vector works just as well as basic_string.
Differential Revision: https://reviews.llvm.org/D157589
StreamFile subclasses Stream (from lldbUtility) and is backed by a File
(from lldbHost). It does not depend on anything from lldbCore or any of its
sibling libraries, so I think it makes sense for this to live in
lldbHost instead.
Differential Revision: https://reviews.llvm.org/D157460
Mach-O corefiles have several possible types of metadata for the
binaries that were running when the corefile was written.
ProcessMachCore will try to find these binaries, and load them.
When we have a hint, but could not find the binary, this change
makes ProcessMachCore not fall back to scanning the corefile looking
for ANY binary that it could load. We sometimes have multiple binaries
present in the memory in a corefile, but only the correct binary
should be loaded, the others are data.
Differential Revision: https://reviews.llvm.org/D157168
rdar://112602508
DynamicLoader::LoadBinaryWithUUIDAndAddress can create a Module based
on the binary image in memory, which in some cases contains symbol
names and can be genuinely useful. If we don't have a filename, it
creates a name in the form `memory-image-0x...` with the header address.
In practice, this is most useful with Darwin userland corefiles
where the binary was stored in the corefile in whole, and we can't
find a binary with the matching UUID. Using the binary out of
the corefile memory in this case works well.
But in other cases, akin to firmware debugging, we merely end up
with an oddly named binary image and no symbols.
Add a flag to control whether we will create these memory images
and add them to the Target or not; only set it to true when working
with a userland Mach-O image with the "all image infos" LC_NOTE for
a userland corefile.
Differential Revision: https://reviews.llvm.org/D157167
Previously lldb was storing them but not restoring them. Meaning that this function:
```
void expr(uint64_t value) {
__asm__ volatile("msr tpidr_el0, %0" ::"r"(value));
}
```
When run from lldb:
```
(lldb) expression expr()
```
Would leave tpidr as `value` instead of the original value of the register.
A check for this scenario has been added to TestAArch64LinuxTLSRegisters.py,
which covers tpidr and the SME excluisve tpidr2 register when it's present.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D156512
7e229217f4 did live processes, this does
core files. Pretty simple, there is an NT_ARM_TLS note that contains
at least tpidr, and on systems with the Scalable Matrix Extension (SME), tpidr2
as well.
tpidr2 will be handled in future patches for SME support.
This NT_ARM_TLS note has always been present but it seems convenient to
handle it as "optional" inside of LLDB. We'll probably want the flexibility
when supporting tpidr2.
Normally the C library would set tpidr but all our test sources build
without it. So I've updated the neon test program to write to tpidr
and regenerated the corefile.
I've removed the LLDB_PTRACE_NT_ARM_TLS that was unused, we get
what we need from llvm's defs instead.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D156118
This changes the TLS regset to not only be dynamic in that it could
exist or not (though it always does) but also of a dynamic size.
If SME is present then the regset is 16 bytes and contains both tpidr
and tpidr2.
Testing is the same as tpidr. Write from assembly, read from lldb and
vice versa since we have no way to predict what its value should be
by just running a program.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D154930
The Scalable Matrix Extension (SME) adds a new Scalable Vector mode
called "streaming SVE mode".
In this mode a lot of things change, but my understanding overall
is that this mode assumes you are not going to move data out of
the vector unit very often or read flags.
Based on "E1.3" of "Arm® Architecture Reference Manual Supplement,
The Scalable Matrix Extension (SME), for Armv9-A".
https://developer.arm.com/documentation/ddi0616/latest/
The important details for debug are that this adds another set
of SVE registers. This set is only active when we are in streaming
mode and is read from a new ptrace regset NT_ARM_SSVE.
We are able to read the header of either mode at all times but
only one will be active and contain register data.
For this reason, I have reused the existing SVE state. Streaming
mode is just another mode value attached to that state.
The streaming mode registers do not have different names in the
architecture, so I do not plan to allow users to read or write the
inactive mode's registers. "z0" will always mean "z0" of the active
mode.
Ptrace does allow reading inactive modes, but the data is of little
use. Writing to inactive modes will switch to that mode which would
not be what a debugger user would expect. So lldb will do neither.
Existing SVE tests have been updated to check streaming mode and
mode switches. However, we are limited in what we can check given
that state for the other mode is invalidated on mode switch.
The only way to know what mode you are in for testing purposes would
be to execute a streaming only, or non-streaming only instruction in
the opposite mode. However, the CPU feature smefa64 actually allows
all non-streaming mode instructions in streaming mode.
This is enabled by default in QEMU emulation and rather than mess
about trying to disable it I'm just going to use the pseduo streaming
control register added in a later patch to make these tests more
robust.
A new test has been added to check SIMD read/write from all the modes
as there is a subtlety there that needs noting, though lldb
doesn't have to make extra effort to do so.
If you are in streaming mode and write to v0, when you later exit
streaming mode that value may not be in the non-streaming state.
This can depend on how the core works but is a valid behaviour.
For example, say I am stopped here:
mov x0, v0.d[0]
And I want to update v0 in lldb. "register write v0 ..." should update
the v0 that this instruction is about to see. Not the potential other
copy of v0 in the non-streaming state (which is what I attempted in
earlier versions of this patch).
Not to mention, switching out of streaming mode here would be unexpected
and difficult to signal to the user.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D154926
Increase the default timeouts when running under ASan. We had something
similar before we adopted tablegen, but the larger timeouts got lost in
the transition, possibly because tablegen's preprocessor support is very
limited. This patch passes a new define (LLDB_SANITIZED) to
lldb-tablegen on which we can base the default value.
Differential revision: https://reviews.llvm.org/D156279
Similar to ae316ac66f for
QEMU_(UN)SET_ENV.
The iteration order of StringMap is not guaranteed to be deterministic.
Sort the entries to give deterministic packets for the tests added by
D108018.
/data/llvm-project/lldb/source/Plugins/Process/Utility/RegisterInfos_x86_64_with_base_shared.cpp:319:1: error: non-void function does
not return a value in all control paths [-Werror,-Wreturn-type]
}
^
1 error generated.
Summary:
[lldb][x86_64] This patch adds fs_base/gs_base support for Linux x86_64.
Originally, I plan to split the diff into two parts, one to refactoring lldb_xxx_x86_64 => x86_64::lldb_xxx across code base and the other one for adding fs_base/gs_base, but it turns out to be a non-trivial effort to split and very error prone so I decided to keep a single diff to get feedback.
GDB supports fs_base/gs_base registers while LLDB does not. Since both linux coredump note section and ptrace
supports them it is a missing feature.
For context, this is a required feature to support getting pthread pointer on linux from both live and dump debugging.
See thread below for details:
https://discourse.llvm.org/t/how-to-get-pthread-pointer-from-lldb/70542/2?u=jeffreytan81
Implementation wise, we have initially tried `#ifdef` approach to reuse the code but it is introducing very tricky bugs and proves
hard to maintain. Instead the diff completely separates the registers between x86_64 and x86_64_with_base so that non-linux related
implementations can use x86_64 registers while linux uses x86_64_with_base.
Here are the list of changes done in the patch:
* Registers in lldb-x86-register-enums.h are separated into two: x86_64 and x86_64_with_base
* fs_base/gs_base are added into x86_64_with_base
* All linux files are change to use x86_64::lldb_xxx => x86_64_with_base::lldb_xxx
* Support linux elf-core:
* A new RegisterContextLinuxCore_x86_64 class is added for ThreadElfCore
* RegisterContextLinuxCore_x86_64 overrides and uses its own register set supports fs_base/gs_base
* RegisterInfos_x86_64_with_base/RegisterInfos_x86_64_with_base_shared ared added to provide g_contained_XXX/g_invalidate_XXX and RegInfo related code sharing.
* `RegisterContextPOSIX_x86 ::m_gpr_x86_64` seems to be unused so I removed it.
* `NativeRegisterContextDBReg_x86::GetDR()` is overridden in `NativeRegisterContextLinux_x86_64` to make watchpoint work.
Reviewers:clayborg,labath,jingham,jdoerfert,JDevlieghere,kusmour,GeorgeHuyubo
Subscribers:
Tasks:
Tags:
Differential Revision: https://reviews.llvm.org/D155256
I need to call this to figure out why the assert in
StopInfoMachException::CreateStopReasonWithMachException is triggering, but
it isn't appropriate to directly access the GDBRemoteCommunication there. And
dumping whatever history the process plugin has collected during the run isn't
gdb-remote specific...
Differential Revision: https://reviews.llvm.org/D154992
In preparation for removing the `#include "llvm/ADT/StringExtras.h"`
from the header to source file of `llvm/Support/Error.h`, first add in
all the missing includes that were previously included transitively
through this header.
This is fixing all files missed in b0abd4893f, 39d8e6e22c, and
a11efd4926.
Differential Revision: https://reviews.llvm.org/D154775
This patch attempts to fix a dead lock when loading modules in a Scripted
Process.
This issue was triggered by loading the modules after the process did resume,
but before the process actually stop, causing the language runtime mutex to
be locked by a separate thread, responsible to unwind the stack (using
the runtime unwind plan), while the module loading thread was trying to
notify the runtimes of the newly loaded module.
To address that, this patch moves the module loading logic to be done before
sending the stop event, to prevent the dead lock situation described above.
Differential Revision: https://reviews.llvm.org/D154649
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Fix incorrect uses of LLDB_LOG_ERROR. The macro doesn't automatically
inject the error in the log message: it merely passes the error as the
first argument to formatv and therefore must be referenced with {0}.
Thanks to Nicholas Allegra for collecting a list of places where the
macro was misused.
rdar://111581655
Differential revision: https://reviews.llvm.org/D154530
I'm not convinced that it makes sense for the paths to be ConstStrings. We're
going to be putting them into FileSpecs (which are backed by
ConstStrings, for now) but otherwise there's no need to store them as
ConstStrings upfront.
Differential Revision: https://reviews.llvm.org/D153827
While looking at https://github.com/llvm/llvm-project/issues/61955
I noticed that when we send qLaunchGDBServer we check that we got a response
but not what kind of response it was.
I think this was why the bug reporter saw:
(lldb) run
error: invalid host:port specification: '[192.168.64.2]'
The missing port is because we went down a path we only should have
chosen if the operation succeeded. Since we didn't check, we went ahead
with an empty port number.
To test this I've done the following:
* Make a temporary copy of lldb-server.
* Run that as a platform.
* Remove the copy.
* Attempt to create and run a target.
This fails because the running lldb-server will try to invoke itself
and it no longer exists.
Reviewed By: jasonmolenda
Differential Revision: https://reviews.llvm.org/D153513
Previously lldb was using arrays of size kMaxRegisterByteSize to handle
registers. This was set to 256 because the largest possible register
we support is Arm's scalable vectors (SVE) which can be up to 256 bytes long.
This means for most operations aside from SVE, we're wasting 192 bytes
of it. Which is ok given that we don't have to pay the cost of a heap
alocation and 256 bytes isn't all that much overall.
With the introduction of the Arm Scalable Matrix extension there is a new
array storage register, ZA. This register is essentially a square made up of
SVE vectors. Therefore ZA could be up to 64kb in size.
https://developer.arm.com/documentation/ddi0616/latest/
"The Effective Streaming SVE vector length, SVL, is a power of two in the range 128 to 2048 bits inclusive."
"The ZA storage is architectural register state consisting of a two-dimensional ZA array of [SVLB × SVLB] bytes."
99% of operations will never touch ZA and making every stack frame 64kb+ just
for that slim chance is a bad idea.
Instead I'm switching register handling to use SmallVector with a stack allocation
size of kTypicalRegisterByteSize. kMaxRegisterByteSize will be used in places
where we can't predict the size of register we're reading (in the GDB remote client).
The result is that the 99% of small register operations can use the stack
as before and the actual ZA operations will move to the heap as needed.
I tested this by first working out -wframe-larger-than values for all the
libraries using the arrays previously. With this change I was able to increase
kMaxRegisterByteSize to 256*256 without hitting those limits. With the
exception of the GDB server which needs to use a max size buffer.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D153626
This patch should address the failure of TestStackCoreScriptedProcess
that is happening specifically on x86_64.
It turns out that in 1370a1cb5b, I changed the way we extract integers
from a `StructuredData::Dictionary` and in order to get a stop info from
the scripted process, we call a method that returns a `SBStructuredData`
containing the stop reason data.
TestStackCoreScriptedProcess` was failing specifically on x86_64 because
the stop info dictionary contains the signal number, that the `Scripted
Thread` was trying to extract as a signed integer where it was actually
parsed as an unsigned integer. That caused `GetValueForKeyAsInteger` to
return the default value parameter, `LLDB_INVALID_SIGNAL_NUMBER`.
This patch address the issue by extracting the signal number with the
appropriate type and re-enables the test.
Differential Revision: https://reviews.llvm.org/D152848
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
In Android API level 23 and above, dynamic loader is able to load .so file
directly from APK, which is zip file.
https://android.googlesource.com/platform/bionic/+/master/
android-changes-for-ndk-developers.md#
opening-shared-libraries-directly-from-an-apk
The .so file is page aligned and uncompressed, so
ObjectFileELF::GetModuleSpecifications works with .so file offset and size
directly from zip file without extracting it. (D152757)
GDBRemoteCommunicationServerCommon::GetModuleInfo returns a module spec to LLDB
with "zip_path!/so_path" file spec, which is passed through from Android
dynamic loader, and the .so file offset and size.
PlatformAndroid::DownloadModuleSlice uses 'shell dd' to download the .so file
slice from the zip file with the .so file offset and size.
Differential Revision: https://reviews.llvm.org/D152759
This register is used as the pointer to the current thread
local storage block and is read from NT_ARM_TLS on Linux.
Though tpidr will be present on all AArch64 Linux, I am soon
going to add a second register tpidr2 to this set.
tpidr is only present when SME is implemented, therefore the
NT_ARM_TLS set will change size. This is why I've added this
as a dynamic register set to save changes later.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D152516
On one CI bot we're seeing a failure where the kernel reports that
we have completed an instruction step (via a mach exception) and
lldb doesn't think the thread was doing an instruction step. It
takes the conservative approach of stopping at this point, breaking
tests.
This patch adds an llvm fatal error for debug builds where it will
log the state of the thread and the AArch64 ESR, to confirm what
the hardware reported as the exception so we can double check the
kernel's interpretation.
I'll change this to an lldbassert without the runtime details in
the string once we have an idea what is happening. the hope is
that this will get hit on the CI bot soon.
Differential Revision: https://reviews.llvm.org/D153079
I want to add some error handling to DynamicRegisterInfo because there
are many operations that can fail and many of these operations do not
give meaningful information back to the caller.
To begin that process, I want to add a static method that is responsible
for creating a DynamicRegisterInfo from a StructuredData::Dictionary
(and ArchSpec). This is meant to replace the equivalent constructor
because constructors are ill-equipped to perform error handling.
Differential Revision: https://reviews.llvm.org/D152594
This patch should address the failure of TestStackCoreScriptedProcess
that is happening specifically on x86_64.
It turns out that in 1370a1cb5b, I changed the way we extract integers
from a `StructuredData::Dictionary` and in order to get a stop info from
the scripted process, we call a method that returns a `SBStructuredData`
containing the stop reason data.
TestStackCoreScriptedProcess` was failing specifically on x86_64 because
the stop info dictionary contains the signal number, that the `Scripted
Thread` was trying to extract as a signed integer where it was actually
parsed as an unsigned integer. That caused `GetValueForKeyAsInteger` to
return the default value parameter, `LLDB_INVALID_SIGNAL_NUMBER`.
This patch address the issue by extracting the signal number with the
appropriate type and re-enables the test.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
In ProcessMachCore::LoadBinariesViaMetadata(), if we did
load some binaries via metadata in the core file, don't
then search for a userland dyld in the corefile / kernel
and throw away that binary list. Also fix a little bug
with correctly recognizing corefiles using a `main bin spec`
LC_NOTE that explicitly declare that this is a userland
corefile.
LocateSymbolFileMacOSX.cpp's Symbols::DownloadObjectAndSymbolFile
clarify the comments on how the force_lookup and how the
dbgshell_command local both have the same effect.
In PlatformDarwinKernel::LoadPlatformBinaryAndSetup, don't
log a message unless we actually found a kernel fileset.
Reorganize ObjectFileMachO::LoadCoreFileImages so that it delegates
binary searching to DynamicLoader::LoadBinaryWithUUIDAndAddress and
doesn't duplicate those searches. For searches that fail, we would
perform them multiple times in both methods. When we have the
mach-o segment vmaddrs for a binary, don't let LoadBinaryWithUUIDAndAddress
load the binary first at its mach-o header address in the Target;
we'll load the segments at the correct addresses individually later
in this method.
DynamicLoaderDarwin::ImageInfo::PutToLog fix a LLDB_LOG logging
formatter.
In DynamicLoader::LoadBinaryWithUUIDAndAddress, instead of using
Target::GetOrCreateModule as a way to find a binary already registered
in lldb's global module cache (and implicitly add it to the Target
image list), use ModuleList::GetSharedModule() which only searches
the global module cache, don't add it to the Target. We may not
want to add an unstripped binary to the Target.
Add a call to Symbols::DownloadObjectAndSymbolFile() even if
"force_symbol_search" isn't set -- this will turn into a
DebugSymbols call / Spotlight search on a macOS system, which
we want.
Only set the Module's LoadAddress if the caller asked us to do that.
Differential Revision: https://reviews.llvm.org/D150928
rdar://109186357
This patch adds support to eStopReasonTrace to Scripted Threads.
This is necessary when using a Scrited Process with a Scripted Thread
Plan to report a special thread stop reason to the thread plan.
rdar://109425542
Differential Revision: https://reviews.llvm.org/D151043
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
