The idea behind the address-expression is that it handles all the common
expressions that produce addresses. It handles actual valid expressions
that return a scalar, and it handles useful cases that the various
source languages don't support. At present, the fallback handles:
<symbol_name>{+-}<offset>
which isn't valid C but is very handy.
This patch adds handling of:
$<reg_name>
and
$<reg_name>{+-}<offset>
That's kind of pointless in C because the C expression parser handles
that expression already. But some languages don't have a straightforward
way to represent register values like this (swift) so having this
fallback is quite a quality of life improvement.
I added a test which tests that I didn't mess up either of these
fallbacks, though it doesn't test the actually handling of registers
that I added, since the expression parser for C succeeds in that case
and returns before this code gets run.
I will add a test on the swift fork for that checks that this works the
same way for a swift frame after this check.
`EvaluateExpression` does not always create a new persistent result. If the expression
is a bare persistent variable, then a new persistent result is not created. This means
the caller can't assume a new persistent result is created for each evaluation.
However, `dwim-print` was doing exactly that: assuming a new persistent result for each
evaluation. This resulted in a bug:
```
(lldb) p int $j = 23
(lldb) p $j
(lldb) p $j
```
The first `p $j` would not create a persistent result, and so `dwim-print` would
inadvertently delete `$j`. The second `p $j` would fail.
The fix is to try `expr` as a persistent variable, after trying `expr` as a frame
variable. For persistent variables, this avoids calling `EvaluateExpression`.
Resolves https://github.com/llvm/llvm-project/issues/84806
rdar://124688427
There was a think-o in a previous commit that made us only able to
define 1 line commands when using command script add interactively.
There was also no test for this feature, so I fixed the think-o and
added a test.
I noticed that the term-width setting would always report its default
value (80) despite the driver correctly setting the value with
SBDebugger::SetTerminalWidth.
```
(lldb) settings show term-width
term-width (int) = 80
```
The issue is that the setting was defined as a SInt64 instead of a
UInt64 while the getter returned an unsigned value. There's no reason
the terminal width should be a signed value. My best guess it that it
was using SInt64 because UInt64 didn't support min and max values. I
fixed that and correct the type and now lldb reports the correct
terminal width:
```
(lldb) settings show term-width
term-width (unsigned) = 189
```
rdar://123488999
Any time we see the pattern `assertEqual(value, bool)`, we can replace
that with `assert<bool>(value)`. Likewise for `assertNotEqual`.
Technically this relaxes the test a bit, as we may want to make sure
`value` is either `True` or `False`, and not something that implicitly
converts to a bool. For example, `assertEqual("foo", True)` will fail,
but `assertTrue("foo")` will not. In most cases, this distinction is not
important.
There are two such places that this patch does **not** transform, since
it seems intentional that we want the result to be a bool:
*
5daf2001a1/lldb/test/API/python_api/sbstructureddata/TestStructuredDataAPI.py (L90)
*
5daf2001a1/lldb/test/API/commands/settings/TestSettings.py (L940)
Followup to 9c2468821e. I patched `teyit`
with a `visit_assertEqual` node handler to generate this.
This uses [teyit](https://pypi.org/project/teyit/) to modernize asserts,
as recommended by the [unittest release
notes](https://docs.python.org/3.12/whatsnew/3.12.html#id3).
For example, `assertTrue(a == b)` is replaced with `assertEqual(a, b)`.
This produces better error messages, e.g. `error: unexpectedly found 1
and 2 to be different` instead of `error: False`.
This is a follow-on to:
https://github.com/llvm/llvm-project/pull/82085
The completer for register names was missing from the argument table. I
somehow missed that the only register completer test was x86_64, so that
test broke.
I added the completer in to the right slot in the argument table, and
added a small completions test that just uses the alias register names.
If we end up having a platform that doesn't define register names, we'll
have to skip this test there, but it should add a sniff test for
register completion that will run most everywhere.
This reverts commit 21631494b0.
Reverted because of greendragon failure:
******************** TEST 'lldb-api :: functionalities/completion/TestCompletion.py' FAILED ********************
Script:
Most commands were adding argument completion handling by themselves,
resulting in a lot of unnecessary boilerplate. In many cases, this could
be done generically given the argument definition and the entries in the
g_argument_table.
I'm going to address this in a couple passes. In this first pass, I
added handling of commands that have only one argument list, with one
argument type, either single or repeated, and changed all the commands
that are of this sort (and don't have other bits of business in their
completers.)
I also added some missing connections between arg types and completions
to the table, and added a RemoteFilename and RemotePath to use in places
where we were using the Remote completers. Those arguments used to say
they were "files" but they were in fact remote files.
I also added a module arg type to use where we were using the module
completer. In that case, we should call the argument module.
Updates:
- The previous patch changed the default behavior to not load dwos in
`DWARFUnit`
~~`SymbolFileDWARFDwo *GetDwoSymbolFile(bool load_all_debug_info =
false);`~~
`SymbolFileDWARFDwo *GetDwoSymbolFile(bool load_all_debug_info = true);`
- This broke some lldb-shell tests (see
https://green.lab.llvm.org/green/view/LLDB/job/as-lldb-cmake/16273/)
- TestDebugInfoSize.py
- with symbol on-demand, by default statistics dump only reports
skeleton debug info size
- `statistics dump -f` will load all dwos. debug info = skeleton debug
info + all dwo debug info
Currently running `statistics dump` will trigger lldb to load debug info
that's not yet loaded (eg. dwo files). Resulted in a delay in the
command return, which, can be interrupting.
This patch also added a new option `--load-all-debug-info` asking
statistics to dump all possible debug info, which will force loading all
debug info available if not yet loaded.
assertEquals is a deprecated alias for assertEqual and has been removed
in Python 3.12. This wasn't an issue previously because we used a
vendored version of the unittest module. Now that we use the built-in
version this gets updated together with the Python version used to run
the test suite.
Switching to modern `unittest` in
5b386158aa needs xfail annotations to be
known prior to test running. In contrast, skipping can happen at any
time, even during test execution.
Thus, `expectedFailureIfFn` inherently doesn't work. Either we eagerly
evaluate the function and use `expectedFailureIf` instead, or we use a
skip annotation to lazily evaluate the function and potentially skip the
test right before it starts.
- For `expectedFailureAndroid`, the intent seems to be that certain
tests _should_ work on android, but don't. Thus, xfail is appropriate,
to ensure the test is re-enabled once those bugs are ever fixed.
- For the other uses in individual tests, those generally seem to be
cases where the test environment doesn't support the setup required by
the test, and so it isn't meaningful to run the test at all. For those,
a drop-in replacement to `skipTestIfFn` works.
Python3.9 does not allow you to put a reference to a class staticmethod
in a table and call it from there. Python3.10 and following do allow
this, but we still support 3.9. staticmethod was slightly cleaner,
but this will do.
When the parsed command python code is run on 3.9, I get:
File ".../lib/python3.9/site-packages/lldb/plugins/parsed_cmd.py", line 124, in translate_value
return cls.translators[value_type](value)
TypeError: 'staticmethod' object is not callable
But this works correctly in Python 3.10 on macOS and Linux. I'm guessing something
changed between those versions, and I'll have to do something to work around the difference.
But I'm going to skip the test on 3.9 while I figure that out.
This removes the dependency LLDB API tests have on
lldb/third_party/Python/module/unittest2, and instead uses the standard
one provided by Python.
This does not actually remove the vendored dep yet, nor update the docs.
I'll do both those once this sticks.
Non-trivial changes to call out:
- expected failures (i.e. "bugnumber") don't have a reason anymore, so
those params were removed
- `assertItemsEqual` is now called `assertCountEqual`
- When a test is marked xfail, our copy of unittest2 considers failures
during teardown to be OK, but modern unittest does not. See
TestThreadLocal.py. (Very likely could be a real bug/leak).
- Our copy of unittest2 was patched to print all test results, even ones
that don't happen, e.g. `(5 passes, 0 failures, 1 errors, 0 skipped,
...)`, but standard unittest prints a terser message that omits test
result types that didn't happen, e.g. `OK (skipped=1)`. Our lit
integration parses this stderr and needs to be updated w/ that
expectation.
I tested this w/ `ninja check-lldb-api` on Linux. There's a good chance
non-Linux tests have similar quirks, but I'm not able to uncover those.
This allows you to specify options and arguments and their definitions
and then have lldb handle the completions, help, etc. in the same way
that lldb does for its parsed commands internally.
This feature has some design considerations as well as the code, so I've
also set up an RFC, but I did this one first and will put the RFC
address in here once I've pushed it...
Note, the lldb "ParsedCommand interface" doesn't actually do all the
work that it should. For instance, saying the type of an option that has
a completer doesn't automatically hook up the completer, and ditto for
argument values. We also do almost no work to verify that the arguments
match their definition, or do auto-completion for them. This patch
allows you to make a command that's bug-for-bug compatible with built-in
ones, but I didn't want to stall it on getting the auto-command checking
to work all the way correctly.
As an overall design note, my primary goal here was to make an interface
that worked well in the script language. For that I needed, for
instance, to have a property-based way to get all the option values that
were specified. It was much more convenient to do that by making a
fairly bare-bones C interface to define the options and arguments of a
command, and set their values, and then wrap that in a Python class
(installed along with the other bits of the lldb python module) which
you can then derive from to make your new command. This approach will
also make it easier to experiment.
See the file test_commands.py in the test case for examples of how this
works.
Adding command interpreter statistics into "statistics dump" command so
that we can track the command usage frequency for telemetry purpose.
This is useful to answer questions like what is the most frequently used
lldb commands across all our users.
---------
Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
`statistics dump` command relies on `SymbolFile::GetDebugInfoSize()` to
get total debug info size.
The current implementation is missing debug info for split dwarf
scenarios which requires getting debug info from separate dwo/dwp files.
This patch fixes this issue for split dwarf by parsing debug info from
dwp/dwo.
New yaml tests are added.
---------
Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
The way the locals are laid out on the stack on x86-64 Debian is
resulting in a test failure with the new large watchpoint support.
Collecting more logging before I revert/debug it.
The LLDB expression parser relies on using the external AST source
support in LLDB. This allows us to find a class at the root namespace
level, but it wouldn't allow us to find nested classes all of the time.
When LLDB finds a class via this mechanism, it would be able to complete
this class when needed, but during completion, we wouldn't populate
nested types within this class which would prevent us from finding
contained types when needed as clang would expect them to be present if
a class was completed. When we parse a type for a class, struct or
union, we make a forward declaration to the class which can be
completed. Now when the class is completed, we also add any contained
types to the class' declaration context which now allows these types to
be found. If we have a struct that contains a struct, we will add the
forward declaration of the contained structure which can be c ompleted
later. Having this forward declaration makes it possible for LLDB to
find everything it needs now.
This should fix an existing issue:
https://github.com/llvm/llvm-project/issues/53904
Previously, contained types could be parsed by accident and allow
expression to complete successfully. Other times we would have to run an
expression multiple times because our old type lookup from our
expressions would cau se a type to be parsed, but not used in the
current expression, but this would have parsed a type into the
containing decl context and the expression might succeed if it is run
again.
If adding a user commands fails because a command with the same name
already exists, we only say that "force replace is not set" without
telling the user _how_ to set it. There are two ways to do so; this
commit changes the error message to mention both.
Small change to get `image dump separate-debug-info` working when using
`symbols.load-on-demand`.
Added tests to `TestDumpDwo`, and enabled the test for all platforms. If we fail to build, we skip the test, so this shouldn't cause the test to fail on unsupported platforms.
```
bin/lldb-dotest -p TestDumpDwo
```
It's easy to verify this manually by running
```
lldb --one-line-before-file "settings set symbols.load-on-demand true" <some_target>
(lldb) image dump separate-debug-info
...
```
---------
Co-authored-by: Tom Yang <toyang@fb.com>
These error messages are written in a way that makes sense to an lldb
developer, but not to an end user who asks lldb to run on a compressed
corefile or whatever. Simplfy the messages.
This register is a pseudo register but mirrors the architectural
register's contents. See:
https://developer.arm.com/documentation/ddi0616/latest/
For the full details. Example output:
```
(lldb) register read svcr
svcr = 0x0000000000000002
= (ZA = 1, SM = 0)
```
This is a Linux pseudo register provided by the NT_ARM_TAGGED_ADDR_CTRL
register set. It reflects the value passed to prctl
PR_SET_TAGGED_ADDR_CTRL.
https://docs.kernel.org/arch/arm64/memory-tagging-extension.html
The fields are made from the #defines the kernel provides for setting
the value. Its contents are constant so no runtime detection is needed
(once we've decided we have this register in the first place).
The permitted generated tags is technically a bitfield but at this time
we don't have a way to mark a field as preferring hex formatting.
```
(lldb) register read mte_ctrl
mte_ctrl = 0x000000000007fffb
= (TAGS = 65535, TCF_ASYNC = 0, TCF_SYNC = 1, TAGGED_ADDR_ENABLE = 1)
```
(4 bit tags mean 16 possible tags, 16 bit bitfield)
Testing has been added to TestMTECtrlRegister.py, which needed a more
granular way to check for XML support, so I've added hasXMLSupport that
can be used within a test case instead of skipping whole tests if XML
isn't supported.
Same for the core file tests.
Follows the format laid out in the Arm manual, AArch32 only fields are
ignored.
```
(lldb) register read fpcr
fpcr = 0x00000000
= (AHP = 0, DN = 0, FZ = 0, RMMode = 0, FZ16 = 0, IDE = 0, IXE = 0, UFE = 0, OFE = 0, DZE = 0, IOE = 0)
```
Tests use the first 4 fields that we know are always present.
Converted all the HCWAP defines to `UL` because I'm bound to
forget one if I don't do it now.
This one is easy because none of the fields depend on extensions. Only
thing to note is that I've ignored some AArch32 only fields.
```
(lldb) register read fpsr
fpsr = 0x00000000
= (QC = 0, IDC = 0, IXC = 0, UFC = 0, OFC = 0, DZC = 0, IOC = 0)
```
The contents of which are mostly SPSR_EL1 as shown in the Arm manual,
with a few adjustments for things Linux says userspace shouldn't concern
itself with.
```
(lldb) register read cpsr
cpsr = 0x80001000
= (N = 1, Z = 0, C = 0, V = 0, SS = 0, IL = 0, ...
```
Some fields are always present, some depend on extensions. I've checked
for those extensions using HWCAP and HWCAP2.
To provide this for core files and live processes I've added a new class
LinuxArm64RegisterFlags. This is a container for all the registers we'll
want to have fields and handles detecting fields and updating register
info.
This is used by the native process as follows:
* There is a global LinuxArm64RegisterFlags object.
* The first thread takes a mutex on it, and updates the fields.
* Subsequent threads see that detection is already done, and skip it.
* All threads then update their own copy of the register information
with pointers to the field information contained in the global object.
This means that even though every thread will have the same fields, we
only detect them once and have one copy of the information.
Core files instead have a LinuxArm64RegisterFlags as a member, because
each core file could have different saved capabilities. The logic from
there is the same but we get HWACP values from the corefile note.
This handler class is Linux specific right now, but it can easily be
made more generic if needed. For example by using LLVM's FeatureBitset
instead of HWCAPs.
Updating register info is done with string comparison, which isn't
ideal. For CPSR, we do know the register number ahead of time but we do
not for other registers in dynamic register sets. So in the interest of
consistency, I'm going to use string comparison for all registers
including cpsr.
I've added tests with a core file and live process. Only checking for
fields that are always present to account for CPU variance.
Often, we only care about the split-dwarf files that have failed to
load. This can be useful when diagnosing binaries with many separate
debug info files where only some have errors.
```
(lldb) help image dump separate-debug-info
List the separate debug info symbol files for one or more target modules.
Syntax: target modules dump separate-debug-info <cmd-options> [<filename> [<filename> [...]]]
Command Options Usage:
target modules dump separate-debug-info [-ej] [<filename> [<filename> [...]]]
-e ( --errors-only )
Filter to show only debug info files with errors.
-j ( --json )
Output the details in JSON format.
This command takes options and free-form arguments. If your arguments
resemble option specifiers (i.e., they start with a - or --), you must use
' -- ' between the end of the command options and the beginning of the
arguments.
'image' is an abbreviation for 'target modules'
```
I updated the following tests
```
# on Linux
bin/lldb-dotest -p TestDumpDwo
# on Mac
bin/lldb-dotest -p TestDumpOso
```
This change applies to both the table and JSON outputs.
---------
Co-authored-by: Tom Yang <toyang@fb.com>
SME2 is documented as part of the main SME supplement:
https://developer.arm.com/documentation/ddi0616/latest/
The one change for debug is this new ZT0 register. This register
contains data to be used with new table lookup instructions.
It's size is always 512 bits (not scalable) and can be
interpreted in many different ways depending on the instructions
that use it.
The kernel has implemented this as a new register set containing
this single register. It always returns register data (with no header,
unlike ZA which does have a header).
https://docs.kernel.org/arch/arm64/sme.html
ZT0 is only active when ZA is active (when SVCR.ZA is 1). In the
inactive state the kernel returns 0s for its contents. Therefore
lldb doesn't need to create 0s like it does for ZA.
However, we will skip restoring the value of ZT0 if we know that
ZA is inactive. As writing to an inactive ZT0 sets SVCR.ZA to 1,
which is not desireable as it would activate ZA also. Whether
SVCR.ZA is set will be determined only by the ZA data we restore.
Due to this, I've added a new save/restore kind SME2. This is easier
than accounting for the variable length ZA in the SME data. We'll only
save an SME2 data block if ZA is active. If it's not we can get fresh
0s back from the kernel for ZT0 anyway so there's nothing for us to
restore.
This new register will only show up if the system has SME2 therefore
the SME set presented to the user may change, and I've had to account
for that in in a few places.
I've referred to it internally as simply "ZT" as the kernel does in
NT_ARM_ZT, but the architecture refers to the specific register as "ZT0"
so that's what you'll see in lldb.
```
(lldb) register read -s 6
Scalable Matrix Extension Registers:
svcr = 0x0000000000000000
svg = 0x0000000000000004
za = {0x00 <...> 0x00}
zt0 = {0x00 <...> 0x00}
```
FEAT_SME_FA64 (smefa64 in Linux cpuinfo) allows the use of the full A64
instruction set while in streaming SVE mode.
See https://developer.arm.com/documentation/ddi0616/latest/ for details.
This means for example if we want to write to the ffr register during or
use floating point registers while in streaming mode, we need this
extension.
I initially was using QEMU which has it by default, and switched to
Arm's FVP which does not. So this change adds a more strict check and
converts most of the tests to use that. It would be possible in some
cases to avoid the offending instructions but it would be a lot of
effort and liable to fail randomly as the C library changes.
It is also my assumption that the majority of systems will have smefa64
as QEMU has chosen to have. If I turn out to be wrong, we can make the
effort to get the tests working without smefa64.
`isAArch64SME` remains for some tests, which are as follows:
* `test_aarch64_dynamic_regset_config` merely checks for the presence of
a register set, which appears for any SME system not just one with
smefa64.
* `test_aarch64_dynamic_regset_config_sme_za_disabled` only needs the ZA
register and does not enter streaming mode.
* `test_sme_not_present` tests for the absence of the SME register set,
so must be skipped if any form of SME is present.
* Various tests in `TestSVERegisters.py` need to know if SME is present
at all to generate an expected SVCR value. Earlier in the callstack
something else checked `isAArch64SMEFA64` already.
* `TestAArch64LinuxTLSRegisters.py` needs to test the `tpidr2` register
if any form of SME is present. msr/mrs instructions are used to do this
and are allowed even if smefa64 is not present.
This reverts commit 8d80a452b8.
The pointer to the invalidates lists needs to be non-const. Though in this case
I don't think it's ever modified.
Also I realised that the invalidate list was being set on svg not vg.
Should be the other way around.
This fixes a bug where writing vg during streaming mode
could prevent you reading za directly afterwards.
vg is invalidated just prior to us reading it in AArch64Reconfigure,
but svg was not. This lead to some situations where vg would be
updated or cleared and re-read, but svg would not be.
This meant it had some undefined value which lead to errors
that prevented us reading ZA. Likely we received a lot more
data than we were expecting.
There are at least 2 ways to get into this situation:
* Explicit write by the user to vg.
* We have just stopped and need to get the potentially new svg and vg.
The first is handled by invalidating svg client side before fetching the
new one. This also
covers some but not all of the second scenario. For the second, I've
made writes to vg
invalidate svg by noting this in the register information.
Whichever one of those kicks in, we'll get the latest value of svg.
The bug may depend on timing, I could not find a consistent way
to trigger it. I originally found it when checking whether za
is disabled after a vg change, so I've added checks for that
to TestZAThreadedDynamic.
The SVE VG version of the bug did show up on the buildbot,
but not consistently. So it's possible that TestZAThreadedDynamic
does in fact cover this, but I haven't run it enough times to know.
