AddressableBits is in the Utility module of LLDB. It currently directly
refers to Process, which is from the Target LLDB module. This is a
layering violation which concretely means that it is impossible to link
anything that uses Utility without it also using Target as well. This is
generally not an issue for LLDB (since everything is built together) but
it may make it difficult to write unit tests for AddressableBits later
on.
Darwin AArch64 application processors are run with Top Byte Ignore mode
enabled so metadata may be stored in the top byte, it needs to be
ignored when reading/writing memory. David Spickett handled this already
in the base class Process::ReadMemory but ProcessMachCore overrides that
method (to avoid the memory cache) and did not pick up the same change.
I add a test case that creates a pointer with metadata in the top byte
and dereferences it with a live process and with a corefile.
rdar://123784501
We have a Python script that needs to locate coredump path during
debugging so that we can retrieve certain metadata files associated with
it. Currently, there is no API for this.
This patch adds a new `SBProcess::GetCoreFile()` to retrieve target dump
file spec used for dump debugging. Note: this is different from the main
executable module spec. To achieve this, the patch hoists m_core_file
into PostMortemProcess for sharing.
---------
Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
We need to generate events when finalizing, or we won't know that we
succeeded in stopping the process to detach/kill. Instead, we stall and
then after our 20 interrupt timeout, we kill the process (even if we
were supposed to detach) and exit.
OTOH, we have to not generate events when the Process is being
destructed because shared_from_this has already been torn down, and
using it will cause crashes.
This completes the conversion of LocateSymbolFile into a SymbolLocator
plugin. The only remaining function is DownloadSymbolFileAsync which
doesn't really fit into the plugin model, and therefore moves into the
SymbolLocator class, while still relying on the plugins to do the
underlying work.
Add a new LC_NOTE for Mach-O corefiles, "proces metadata", which is a
JSON string. Currently there may be a `threads` key in the JSON,
and if `threads` is present, it is an array with the same number of
elements as there are LC_THREADs in the corefile. This patch adds
support for a `thread_id` key-value for each `thread` entry, to
supply a thread ID for that LC_THREAD.
Differential Revision: https://reviews.llvm.org/D158785
rdar://113037252
Add support for the `low_mem_addressing_bits` and
`high_mem_addressing_bits` keys in the stop reply packet,
in addition to the existing `addressing_bits`. Same
behavior as in the qHostInfo packet.
Clean up AddressableBits so we don't need to check if
any values have been set in the object before using it
to potentially update the Process address masks.
Differential Revision: https://reviews.llvm.org/D158041
Some Apple firmware environments store the UUID of the main binary
at a fixed address in low memory. Add that list of addresess to
ProcessMachCore to check for a UUID, and try to load it.
Differential Revision: https://reviews.llvm.org/D157756
On AArch64 systems, we may have different page table setups for
low memory and high memory, and therefore a different number of
bits used for addressing depending on which half of memory the
address is in.
This patch extends the qHostInfo and LC_NOTE "addrable bits" so
that it can specify the number of addressing bits in high memory
and in low memory separately. It builds on the patch I added in
https://reviews.llvm.org/D151292 where Process tracks the separate
address masks, and there is a user setting to set them manually.
Differential Revision: https://reviews.llvm.org/D157667
rdar://113225907
Mach-O corefiles have several possible types of metadata for the
binaries that were running when the corefile was written.
ProcessMachCore will try to find these binaries, and load them.
When we have a hint, but could not find the binary, this change
makes ProcessMachCore not fall back to scanning the corefile looking
for ANY binary that it could load. We sometimes have multiple binaries
present in the memory in a corefile, but only the correct binary
should be loaded, the others are data.
Differential Revision: https://reviews.llvm.org/D157168
rdar://112602508
DynamicLoader::LoadBinaryWithUUIDAndAddress can create a Module based
on the binary image in memory, which in some cases contains symbol
names and can be genuinely useful. If we don't have a filename, it
creates a name in the form `memory-image-0x...` with the header address.
In practice, this is most useful with Darwin userland corefiles
where the binary was stored in the corefile in whole, and we can't
find a binary with the matching UUID. Using the binary out of
the corefile memory in this case works well.
But in other cases, akin to firmware debugging, we merely end up
with an oddly named binary image and no symbols.
Add a flag to control whether we will create these memory images
and add them to the Target or not; only set it to true when working
with a userland Mach-O image with the "all image infos" LC_NOTE for
a userland corefile.
Differential Revision: https://reviews.llvm.org/D157167
In ProcessMachCore::LoadBinariesViaMetadata(), if we did
load some binaries via metadata in the core file, don't
then search for a userland dyld in the corefile / kernel
and throw away that binary list. Also fix a little bug
with correctly recognizing corefiles using a `main bin spec`
LC_NOTE that explicitly declare that this is a userland
corefile.
LocateSymbolFileMacOSX.cpp's Symbols::DownloadObjectAndSymbolFile
clarify the comments on how the force_lookup and how the
dbgshell_command local both have the same effect.
In PlatformDarwinKernel::LoadPlatformBinaryAndSetup, don't
log a message unless we actually found a kernel fileset.
Reorganize ObjectFileMachO::LoadCoreFileImages so that it delegates
binary searching to DynamicLoader::LoadBinaryWithUUIDAndAddress and
doesn't duplicate those searches. For searches that fail, we would
perform them multiple times in both methods. When we have the
mach-o segment vmaddrs for a binary, don't let LoadBinaryWithUUIDAndAddress
load the binary first at its mach-o header address in the Target;
we'll load the segments at the correct addresses individually later
in this method.
DynamicLoaderDarwin::ImageInfo::PutToLog fix a LLDB_LOG logging
formatter.
In DynamicLoader::LoadBinaryWithUUIDAndAddress, instead of using
Target::GetOrCreateModule as a way to find a binary already registered
in lldb's global module cache (and implicitly add it to the Target
image list), use ModuleList::GetSharedModule() which only searches
the global module cache, don't add it to the Target. We may not
want to add an unstripped binary to the Target.
Add a call to Symbols::DownloadObjectAndSymbolFile() even if
"force_symbol_search" isn't set -- this will turn into a
DebugSymbols call / Spotlight search on a macOS system, which
we want.
Only set the Module's LoadAddress if the caller asked us to do that.
Differential Revision: https://reviews.llvm.org/D150928
rdar://109186357
It is possible to have a memory-only mach-o corefile, with the
threads provided by an os-plugin thread provider, or a scripted
process, in Python.
Differential Revision: https://reviews.llvm.org/D142662
rdar://102579544
As a last resort, with a Mach-O corefile, lldb will iterate through
all memory segments looking for a dyld binary (for a userland process
core dump) or an xnu kernel binary (kernel coredump). We often
have metadata via LC_NOTEs so this final search mechanism is not
needed. During the rewrite in https://reviews.llvm.org/D133680 I
did not handle the case of finding the xnu kernel by exhaustive
search correctly.
rdar://103813200
In the restructuring I did in https://reviews.llvm.org/D133680 , I
call ObjectFile::LoadBinariesViaMetadata, and the process m_dyld
may be set by a method under there -- in
ProcessMachCore::LoadBinariesViaMetadata I wanted to check to see
if m_dyld_up had been set. I did this by calling the GetDynamicLoader()
method, but that method will call FindPlugin() if there is no
dynamic loader yet, and the static dynamic loader plugin was being
loaded, preventing the scan for userland binaries in a userland
corefile.
Differential Revision: https://reviews.llvm.org/D137807
rdar://102210820
Add support for recognizing a platform binary in the ObjectFileMachO
method that parses the "load binary" LC_NOTEs in a corefile.
A bit of reorganization to ProcessMachCore::DoLoadCore to separate
all of the unrelated things being done in that method into their own
separate methods, as well as small fixes to improve the handling of
a corefile with multiple kernel images in the corefile.
Differential Revision: https://reviews.llvm.org/D133680
rdar://98754861
Add support to Mach-O corefiles and to live gdb remote serial protocol
connections for the corefile/remote stub to provide a list of load
addresses of binaries that should be found & loaded by lldb, and nothing
else. lldb will try to parse the binary out of memory, and if it can
find a UUID, try to find a binary & its debug information based on the
UUID, falling back to using the memory image if it must.
A bit of code unification from three parts of lldb that were loading
individual binaries already, so there is a shared method in
DynamicLoader to handle all of the variations they were doing.
Re-landing this with a uuid_is_null() implementation added to
Utility/UuidCompatibility.h for non-Darwin systems.
Differential Revision: https://reviews.llvm.org/D130813
rdar://94249937
rdar://94249384
This reverts commit d8879fba88.
Debian bot failure; I included <uuid/uuid.h> to get uuid_is_null() but
don't get it there. Will memcmp or whatever & recommit.
Add support to Mach-O corefiles and to live gdb remote serial protocol
connections for the corefile/remote stub to provide a list of load
addresses of binaries that should be found & loaded by lldb, and nothing
else. lldb will try to parse the binary out of memory, and if it can
find a UUID, try to find a binary & its debug information based on the
UUID, falling back to using the memory image if it must.
A bit of code unification from three parts of lldb that were loading
individual binaries already, so there is a shared method in
DynamicLoader to handle all of the variations they were doing.
Differential Revision: https://reviews.llvm.org/D130813
rdar://94249937
rdar://94249384
In ProcessMachCore::DoLoadCore when we have a standalone
binary and a 'main bin spec' LC_NOTE detailing the UUID and
load address, ProcessMachCore will do a (potentially slow)
lookup to try to find the binary and/or dSYM. For kernel and
userland corefile using 'main bin spec', we would follow the
normal schemes of locating them. DynamicLoaderDarwinKernel would
use the same (possibly expensive) calls to find the correct
binary. dyld by default would use the in-core-file binary image,
and so if the corefile didn't include the entire address space,
the LINKEDIT for dyld could be missing. This means we can't find
the dyld4::dyld_all_image_infos struct, which tells us where the
other binaries are loaded in memory.
Treat userland 'bin main spec' like we do standalone firmewares;
try the expensive checks to find the best dyld we can, before
falling back to using a memory module out of the corefile.
Also add a little TODO for myself in this load_standalone_binary
function that we should handle the case of a binary in the shared
cache correctly, creating a memory module in the corefile and
using the segment load addresses from that to set our segment
load addresses for the final binary.
rdar://89717101
This reverts commit 0df522969a.
Additional checks are added to fix the detection of the last memory region
in GetMemoryRegions or repeating the "memory region" command when the
target has non-address bits.
Normally you keep reading from address 0, looking up each region's end
address until you get LLDB_INVALID_ADDR as the region end address.
(0xffffffffffffffff)
This is what the remote will return once you go beyond the last mapped region:
[0x0000fffffffdf000-0x0001000000000000) rw- [stack]
[0x0001000000000000-0xffffffffffffffff) ---
Problem is that when we "fix" the lookup address, we remove some bits
from it. On an AArch64 system we have 48 bit virtual addresses, so when
we fix the end address of the [stack] region the result is 0.
So we loop back to the start.
[0x0000fffffffdf000-0x0001000000000000) rw- [stack]
[0x0000000000000000-0x0000000000400000) ---
To fix this I added an additional check for the last range.
If the end address of the region is different once you apply
FixDataAddress, we are at the last region.
Since the end of the last region will be the last valid mappable
address, plus 1. That 1 will be removed by the ABI plugin.
The only side effect is that on systems with non-address bits, you
won't get that last catch all unmapped region from the max virtual
address up to 0xf...f.
[0x0000fffff8000000-0x0000fffffffdf000) ---
[0x0000fffffffdf000-0x0001000000000000) rw- [stack]
<ends here>
Though in some way this is more correct because that region is not
just unmapped, it's not mappable at all.
No extra testing is needed because this is already covered by
TestMemoryRegion.py, I simply forgot to run it on system that had
both top byte ignore and pointer authentication.
This change has been tested on a qemu VM with top byte ignore,
memory tagging and pointer authentication enabled.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D115508
Most of our code was including Log.h even though that is not where the
"lldb" log channel is defined (Log.h defines the generic logging
infrastructure). This worked because Log.h included Logging.h, even
though it should.
After the recent refactor, it became impossible the two files include
each other in this direction (the opposite inclusion is needed), so this
patch removes the workaround that was put in place and cleans up all
files to include the right thing. It also renames the file to LLDBLog to
better reflect its purpose.
Version 2 of 'main bin spec' LC_NOTE allows for the specification
of a slide of where the binary is loaded in the corefile virtual
address space. It also adds a (currently unused) platform field
for the main binary.
Some corefile creators will only have a UUID and an offset to be
applied to the binary.
Changed TestFirmwareCorefiles.py to test this new form of
'main bin spec' with a slide, and also to run on both x86_64
and arm64 macOS systems.
Differential Revision: https://reviews.llvm.org/D116094
rdar://85938455
This reverts commit fac3f20de5.
I found this has broken how we detect the last memory region in
GetMemoryRegions/"memory region" command.
When you're debugging an AArch64 system with pointer authentication,
the ABI plugin will remove the top bit from the end address of the last
user mapped area.
(lldb)
[0x0000fffffffdf000-0x0001000000000000) rw- [stack]
ABI plugin removes anything above the 48th bit (48 bit virtual addresses
by default on AArch64, leaving an address of 0.
(lldb)
[0x0000000000000000-0x0000000000400000) ---
You get back a mapping for 0 and get into an infinite loop.
This reverts commit 5fbcf67734.
ProcessDebugger is used in ProcessWindows and NativeProcessWindows.
I thought I was simplifying things by renaming to DoGetMemoryRegionInfo
in ProcessDebugger but the Native process side expects "GetMemoryRegionInfo".
Follow the pattern that WriteMemory uses. So:
* ProcessWindows::DoGetMemoryRegioninfo calls ProcessDebugger::GetMemoryRegionInfo
* NativeProcessWindows::GetMemoryRegionInfo does the same
On AArch64 we have various things using the non address bits
of pointers. This means when you lookup their containing region
you won't find it if you don't remove them.
This changes Process GetMemoryRegionInfo to a non virtual method
that uses the current ABI plugin to remove those bits. Then it
calls DoGetMemoryRegionInfo.
That function does the actual work and is virtual to be overriden
by Process implementations.
A test case is added that runs on AArch64 Linux using the top
byte ignore feature.
Reviewed By: omjavaid
Differential Revision: https://reviews.llvm.org/D102757
There is no reason why this function should be returning a ConstString.
While modifying these files, I also fixed several instances where
GetPluginName and GetPluginNameStatic were returning different strings.
I am not changing the return type of GetPluginNameStatic in this patch, as that
would necessitate additional changes, and this patch is big enough as it is.
Differential Revision: https://reviews.llvm.org/D111877
A Mach-O corefile has an array of memory segments, representing
the memory of the process at the point of the capture. Each segment
has a virtual address + size, and a file offset + size. The file
size may be less than the virtual address size, indicating that
the memory was unavailable. When ProcessMachCore::DoLoadCore scans
this array of memory segments, it builds up a table to translate
virtual addresses to file offsets, for memory read requests.
This lookup table combines contiguous memory segments into a single
entry, to reduce the number of entries (some corefile writers will
emit a separate segement for each virtual meory page).
This contiguous check wasn't taking into account a segment that
isn't present in the corefile, e.g. filesize==0, and every contiguous
memory segment after that point would result in lldb reading the
wrong offset of the file because it didn't account for this.
I'd like to have an error message when someone tries to read memory from
one of these segments, instead of returning all zeroes, so this patch
intentionally leaves these out of the vmaddr -> fileoff table (and
avoids combining them with segments that actually do exist in the
corefile).
I'm a little unsure of writing a test for this one; I'd have to do
a yaml2obj of a corefile with the problem, or add an internal mode
to the Mach-O process save-core where it could write a filesize==0
segment while it was writing one.
rdar://83382487
A MachO userspace corefile may contain LC_THREAD commands which specify
thread exception state.
For arm64* only (for now), report a human-readable version of this state
as the thread stop reason, instead of 'SIGSTOP'.
As a follow-up, similar functionality can be implemented for x86 cores
by translating the trapno/err exception registers.
rdar://82898146
Differential Revision: https://reviews.llvm.org/D109795
A MachO userspace corefile may contain LC_THREAD commands which specify
thread exception state.
For arm64* only (for now), report a human-readable version of this state
as the thread stop reason, instead of 'SIGSTOP'.
As a follow-up, similar functionality can be implemented for x86 cores
by translating the trapno/err exception registers.
rdar://82898146
Differential Revision: https://reviews.llvm.org/D109795
In all these years, we haven't found a use for this function (it has
zero callers). Lets just remove the boilerplate.
Differential Revision: https://reviews.llvm.org/D109600
This patch adds code to process save-core for Mach-O files which
embeds an "addrable bits" LC_NOTE when the process is using a
code address mask (e.g. AArch64 v8.3 with ptrauth aka arm64e).
Add code to ObjectFileMachO to read that LC_NOTE from corefiles,
and ProcessMachCore to set the process masks based on it when reading
a corefile back in.
Also have "process status --verbose" print the current address masks
that lldb is using internally to strip ptrauth bits off of addresses.
Differential Revision: https://reviews.llvm.org/D106348
rdar://68630113
Add a new feature to process save-core on Darwin systems -- for
lldb to create a user process corefile with only the dirty (modified
memory) pages included. All of the binaries that were used in the
corefile are assumed to still exist on the system for the duration
of the use of the corefile. A new --style option to process save-core
is added, so a full corefile can be requested if portability across
systems, or across time, is needed for this corefile.
debugserver can now identify the dirty pages in a memory region
when queried with qMemoryRegionInfo, and the size of vm pages is
given in qHostInfo.
Create a new "all image infos" LC_NOTE for Mach-O which allows us
to describe all of the binaries that were loaded in the process --
load address, UUID, file path, segment load addresses, and optionally
whether code from the binary was executing on any thread. The old
"read dyld_all_image_infos and then the in-memory Mach-O load
commands to get segment load addresses" no longer works when we
only have dirty memory.
rdar://69670807
Differential Revision: https://reviews.llvm.org/D88387
The C headers are deprecated so as requested in D102845, this is replacing them
all with their (not deprecated) C++ equivalent.
Reviewed By: shafik
Differential Revision: https://reviews.llvm.org/D103084
Fill out ProcessMachCore::DoLoadCore to handle LC_NOTE hints with
a UUID or with a UUID+address, and load the binary at the specified
offset correctly. Add tests for all four combinations. Change
DynamicLoaderStatic to not re-set a Section's load address in the
Target if it's already been specified.
Differential Revision: https://reviews.llvm.org/D99571
rdar://51490545
Add a 'can_connect' parameter to Process plugin initialization, and use
it to filter plugins to these capable of remote connections. This is
used to prevent 'process connect' from picking up a plugin that can only
be used locally, e.g. the legacy FreeBSD plugin.
Differential Revision: https://reviews.llvm.org/D91810
Depends on D90490.
The stop command is simple and invokes the new method Trace::StopTracingThread(thread).
On the other hand, the start command works by delegating its implementation to a CommandObject provided by the Trace plugin. This is necessary because each trace plugin needs different options for this command. There's even the chance that a Trace plugin can't support live tracing, but instead supports offline decoding and analysis, which means that "thread trace dump instructions" works but "thread trace start" doest. Because of this and a few other reasons, it's better to have each plugin provide this implementation.
Besides, I'm using the GetSupportedTraceType method introduced in D90490 to quickly infer what's the trace plug-in that works for the current process.
As an implementation note, I moved CommandObjectIterateOverThreads to its header so that I can use it from the IntelPT plugin. Besides, the actual start and stop logic for intel-pt is not part of this diff.
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D90729
This is a polymorphic class, copying it is a bad idea.
This was not a problem because most classes inheriting from it were
deleting their copy operations themselves. However, this enables us to
delete those explicit deletions, and ensure noone forgets to add them in
the future.
