This fix is based on a problem with cxx_compiler and cxx_linker macros
on Windows.
There was an issue with compiler detection in paths containing "icc". In
such case, Makefile.rules thought it was provided with icc compiler.
To solve that, utilities detection has been rewritten in Python.
The last element of compiler's path is separated, taking into account
the platform path delimiter, and compiler type is extracted, with regard
of possible cross-toolchain prefix.
---------
Co-authored-by: Pavel Labath <pavel@labath.sk>
@walter-erquinigo found the the [PR with testing and a fix for
DebugInfoD](https://github.com/llvm/llvm-project/pull/98344) caused an
issue when working with stripped binaries.
The issue is that when you're working with split-dwarf, there are *3*
possible files: The stripped binary the user is debugging, the
"only-keep-debug" *or* unstripped binary, plus the `.dwp` file. The
debuginfod plugin should provide the unstripped/OKD binary. However, if
the debuginfod plugin fails, the default symbol locator plugin will just
return the stripped binary, which doesn't help. So, to address that, the
SymbolVendorELF code checks to see if the SymbolLocator's
ExecutableObjectFile request returned the same file, and bails if that's
the case. You can see the specific diff as the second commit in the PR.
I'm investigating adding a test: I can't quite get a simple repro, and
I'm unwilling to make any additional changes to Makefile.rules to this
diff, for Pavlovian reasons.
These changes aim to support cross-compilation build on Windows host for
Linux target for API tests execution. They're not final: changes will
follow for refactoring and adjustments to make all tests pass.
Chocolatey make is recommended to be used since it is maintained better
than GnuWin32 mentioned here
https://lldb.llvm.org/resources/build.html#windows (latest GnuWin32
release is dated by 2010) and helps to avoid problems with building
tests (for example, GnuWin32 make doesn't support long paths and there
are some other failures with building for Linux with it).
Co-authored-by: Pavel Labath <pavel@labath.sk>
In MinGW make, the `%windir%` variable has a lowercase name `$(windir)`
when launched from cmd.exe, and `$(WINDIR)` name when launched from
MSYS2, since MSYS2 represents standard Windows environment variables
names in upper case.
This commit makes Makefile.rules consider both run variants.
Remove commands for OS/HOST_OS detection from Makefile.rules to simplify
it, since logic for these variables has been implemented in
`lldb/packages/Python/lldbsuite/test/lldbplatformutil.py`
(7021e44b2f).
This is motivated by the upcoming refactor of libc++'s
`__compressed_pair` in https://github.com/llvm/llvm-project/pull/76756
As this will require changes to numerous LLDB libc++ data-formatters
(see early draft https://github.com/llvm/llvm-project/pull/96538), it
would be nice to have a test-suite that will actually exercise both the
old and new layout. We have a matrix bot that tests old versions of
Clang (but currently those only date back to Clang-15). Having them in
the test-suite will give us quicker signal on what broke.
We have an existing test that exercises various layouts of `std::string`
over time in `TestDataFormatterLibcxxStringSimulator.py`, but that's the
only STL type we have it for. This patch proposes a new
`libcxx-simulators` directory which will take the same approach for all
the STL types that we can feasibly support in this way (as @labath
points out, for some types this might just not be possible due to their
implementation complexity). Nonetheless, it'd be great to have a record
of how the layout of libc++ types changed over time.
Some related discussion:
*
https://github.com/llvm/llvm-project/pull/97568#issuecomment-2213426804
This reverts commit 2fa1220a37.
This reverts commit b9496a74eb.
The patch #98344 causes a crash in LLDB when parsing some files like `numpy.libs/libgfortran-daac5196.so.5.0.0` on graviton (you can download it in https://drive.google.com/file/d/12ygLjJwWpzdYsrzBPp1JGiFHxcgM0-XY/view?usp=drive_link if you want to troubleshoot yourself).
The assert that is hit is the following:
```
llvm-project/lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp:2452: std::pair<unsigned int, std::map<long unsigned int, lldb_private::AddressClass> > ObjectFileELF::ParseSymbolTable(lldb_private::Symtab*, lldb::user_id_t, lldb_private::Section*): Assertion `strtab->GetObjectFile() == this' failed.
[383588:383636:20240716,025305.572639:ERROR crashpad_client_linux.cc:780] Crashpad isn't enabled
```
This object file doesn't have apparently a strings table but LLDB still tries to process it due to the code that is being reverted.
This is all the tests and fixes I've had percolating since my first
attempt at this in January. After 6 months of trying, I've given up on
adding the ability to test DWP files in LLDB API tests. I've left both
the tests (disabled) and the changes to Makefile.rules in place, in the
hopes that someone who can configure the build bots will be able to
enable the tests once a non-borked dwp tool is widely available.
Other than disabling the DWP tests, this continues to be the same diff
that I've tried to land and
[not](https://github.com/llvm/llvm-project/pull/90622)
[revert](https://github.com/llvm/llvm-project/pull/87676)
[five](https://github.com/llvm/llvm-project/pull/86812)
[times](https://github.com/llvm/llvm-project/pull/85693)
[before](https://github.com/llvm/llvm-project/pull/96802). There are a
couple of fixes that the testing exposed, and I've abandoned the DWP
tests because I want to get those fixes finally upstreamed, as without
them DebugInfoD is less useful.
Makefile.rules uses HOST_OS and OS variables for determining host and target
OSes for API tests compilation.
This commit moves the platform detection logic from Makefile to Python lldb
test suite.
This is useful for the case of Windows-to-Linux cross-testing.
Reverts llvm/llvm-project#96802
Attempt #5 fails. It's been 6 months. I despise Makefile.rules and have
no ability to even *detect* these failures without _landing_ a diff. In
the mean time, we have no testing for DWP files at all (and a regression
that was introduced, that I fix with this diff) so I'm going to just
remove some of the tests and try to land it again, but with less testing
I guess.
This is the same diff I've put up at many times before. I've been trying
to add some brand new functionality to the LLDB test infrastucture
(create split-dwarf files!), and we all know that no good deed goes
unpunished. The last attempt was reverted because it didn't work on the
Fuchsia build.
There are no code differences between this and
[the](https://github.com/llvm/llvm-project/pull/90622)
[previous](https://github.com/llvm/llvm-project/pull/87676)
[four](https://github.com/llvm/llvm-project/pull/86812)
[diffs](https://github.com/llvm/llvm-project/pull/85693) landed &
reverted (due to testing infra failures). The only change in this one is
the way `dwp` is being identified in `Makefile.rules`.
Thanks to @petrhosek for helping me figure out how the fuchsia builders
are configured. I now prefer to use llvm-dwp and fall back to gnu's dwp
if the former isn't found. Hopefully this will work everywhere it needs
to.
Here we go with attempt number five. Again, no changes to the LLDB code
diff, which has been reviewed several times.
For the tests, I added a `@skipIfCurlSupportMissing` annotation so that
the Debuginfod mocked server stuff won't run, and I also disabled
non-Linux/FreeBSD hosts altogether, as they fail for platform reasons on
macOS and Windows. In addition, I updated the process for extracting the
GNU BuildID to no create a target, per some feedback on the previous
diff.
For reference, previous PR's (landed, backed out after the fact for
various reasons) #90622, #87676, #86812, #85693
---------
Co-authored-by: Kevin Frei <freik@meta.com>
And "LLDB Debuginfod tests and a fix or two (#90622)".
f8fedfb680 /
2d4acb0865
As it has caused a test failure on 32 bit Arm:
https://lab.llvm.org/buildbot/#/builders/17/builds/52580
Expr/TestStringLiteralExpr.test. The follow up did fix
lang/c/shared_lib_stripped_symbols/TestSharedLibStrippedSymbols.py
but not the other failure.
I'm taking yet another swing at getting these tests going, on the
hypothesis that the problems with buildbots & whatnot are because
they're not configured with CURL support, which I've confirmed would
cause the previous tests to fail. (I have no access to an ARM64 linux
system, but I did repro the failure on MacOS configured without CURL
support)
So, the only difference between this diff and
[previous](https://github.com/llvm/llvm-project/pull/85693)
[diffs](https://github.com/llvm/llvm-project/pull/87676) that have
already been approved is that I've added a condition to the tests to
only run if Debuginfod capabilities should be built into the binary. I
had done this for these tests when they were [Shell
tests](https://github.com/llvm/llvm-project/pull/79181) and not API
tests, but I couldn't find a direct analog in any API test, so I used
the "plugins" model used by the intel-pt tests as well.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
This reverts commit d6713ad80d.
This changed was reverted because of greendragon failures such
as
Unresolved Tests (2):
lldb-api :: debuginfod/Normal/TestDebuginfod.py
lldb-api :: debuginfod/SplitDWARF/TestDebuginfodDWP.py
I believe I've got the tests properly configured to only run on Linux
x86(_64), as I don't have a Linux AArch64/Arm device to diagnose what's
going wrong with the tests (I suspect there's some issue with generating
`.note.gnu.build-id` sections...)
The actual code fixes have now been reviewed 3 times:
https://github.com/llvm/llvm-project/pull/79181 (moved shell tests to
API tests), https://github.com/llvm/llvm-project/pull/85693 (Changed
some of the testing infra), and
https://github.com/llvm/llvm-project/pull/86812 (didn't get the tests
configured quite right). The Debuginfod integration for symbol
acquisition in LLDB now works with the `executable` and `debuginfo`
Debuginfod network requests working properly for normal, `objcopy
--only-keep-debug` stripped, split-dwarf, and `objcopy
--only-keep-debug` stripped *plus* split-dwarf symbols/binaries.
The reasons for the multiple attempts have been tests on platforms I
don't have access to (Linux AArch64/Arm + MacOS x86_64). I believe I've
got the tests properly disabled for everything except for Linux x86(_64)
now. I've built & tested on MacOS AArch64 and Linux x86_64.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
The previous diff (and it's subsequent fix) were reverted as the tests
didn't work properly on the AArch64 & ARM LLDB buildbots. I made a
couple more minor changes to tests (from @clayborg's feedback) and
disabled them for non Linux-x86(_64) builds, as I don't have the ability
do anything about an ARM64 Linux failure. If I had to guess, I'd say the
toolchain on the buildbots isn't respecting the `-Wl,--build-id` flag.
Maybe, one day, when I have a Linux AArch64 system I'll dig in to it.
From the reverted PR:
I've migrated the tests in my
https://github.com/llvm/llvm-project/pull/79181 from shell to API (at
@JDevlieghere's suggestion) and addressed a couple issues that were
exposed during testing.
The tests first test the "normal" situation (no DebugInfoD involvement,
just normal debug files sitting around), then the "no debug info"
situation (to make sure the test is seeing failure properly), then it
tests to validate that when DebugInfoD returns the symbols, things work
properly. This is duplicated for DWP/split-dwarf scenarios.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
Finally getting back to Debuginfod tests:
I've migrated the tests in my [earlier
PR](https://github.com/llvm/llvm-project/pull/79181) from shell to API
(at @JDevlieghere's suggestion) and addressed a couple issues that came
about during testing.
The tests first test the "normal" situation (no DebugInfoD involvement,
just normal debug files sitting around), then the "no debug info"
situation (to make sure the test is seeing failure properly), then it
tests to validate that when Debuginfod returns the symbols, things work
properly. This is duplicated for DWP/split-dwarf scenarios.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
Support universal Mach-O binaries with a fat64 header. After
4d683f7fa7, dsymutil can now generate such binaries when the offsets
would otherwise overflow the 32-bit offsets in the regular fat header.
rdar://107289570
Differential revision: https://reviews.llvm.org/D147012
For tests marked as "USE_SYSTEM_STDLIB", the expectation is that the
system's standard library should be used. However, the implementation of
this flag is such that we simply don't pass _any_ libcxxx-related flags
to Clang; in turn, Clang will use its defaults.
For a Clang/Libcxx pair compiled together, Clang defaults to:
1. The headers of the sibling libcxx.
2. The libraries of the system.
This mismatch is actually a bug in the driver; once fixed, however, (2)
would point to the sibling libcxx as well, which is _not_ what test
authors intended with the USE_SYSTEM_STDLIB flag.
As such, this patch explicitly sets a path to the system's libraries.
This change is done only in Apple platforms so that we can test this
works in this case first.
Differential Revision: https://reviews.llvm.org/D146714
In certain configurations, libc++ headers all exist in the same directory, and libc++ binaries exist in the same directory as lldb libs. When `LLVM_ENABLE_PER_TARGET_RUNTIME_DIR` is enabled (*and* the host is not Apple, which is why I assume this wasn't caught by others?), this is not the case: most headers will exist in the usual `include/c++/v1` directory, but `__config_site` exists in `include/$TRIPLE/c++/v1`. Likewise, the libc++.so binary exists in `lib/$TRIPLE/`, not `lib/` (where LLDB libraries reside).
This also adds the just-built-libcxx functionality to the lldb-dotest tool.
The `LIBCXX_` cmake config is borrowed from `libcxx/CMakeLists.txt`. I could not figure out a way to share the cmake config; ideally we would reuse the same config instead of copy/paste.
Reviewed By: JDevlieghere, fdeazeve
Differential Revision: https://reviews.llvm.org/D133973
This commit improves upon cc0b5ebf7f, which added support for
specifying which libcxx to use when testing LLDB. That patch honored
requests by tests that had `USE_LIBCPP=1` defined in their makefiles.
Now, we also use a non-default libcxx if all conditions below are true:
1. The test is not explicitly requesting the use of libstdcpp
(USE_LIBSTDCPP=1).
2. The test is not explicitly requesting the use of the system's
library (USE_SYSTEM_STDLIB=1).
3. A path to libcxx was either provided by the user through CMake flags
or libcxx was built together with LLDB.
Condition (2) is new and introduced in this patch in order to support
tests that are either:
* Cross-platform (such as API/macosx/macCatalyst and
API/tools/lldb-server). The just-built libcxx is usually not built for
platforms other than the host's.
* Cross-language (such as API/lang/objc/exceptions). In this case, the
Objective C runtime throws an exceptions that always goes through the
system's libcxx, instead of the just built libcxx. Fixing this would
require either changing the install-name of the just built libcxx in Mac
systems, or tuning the DYLD_LIBRARY_PATH variable at runtime.
Some other tests exposes limitations of LLDB when running with a debug
standard library. TestDbgInfoContentForwardLists had an assertion
removed, as it was checking for buggy LLDB behavior (which now
crashes). TestFixIts had a variable renamed, as the old name clashes
with a standard library name when debug info is present. This is a known
issue: https://github.com/llvm/llvm-project/issues/34391.
For `TestSBModule`, the way the "main" module is found was changed to
look for the "a.out" module, instead of relying on the index being 0. In
some systems, the index 0 is dyld when a custom standard library is
used.
Differential Revision: https://reviews.llvm.org/D132940
This commit improves upon cc0b5ebf7f, which added support for
specifying which libcxx to use when testing LLDB. That patch honored
requests by tests that had `USE_LIBCPP=1` defined in their makefiles.
Now, we also use a non-default libcxx if all conditions below are true:
1. The test is not explicitly requesting the use of libstdcpp
(USE_LIBSTDCPP=1).
2. The test is not explicitly requesting the use of the system's
library (USE_SYSTEM_STDLIB=1).
3. A path to libcxx was either provided by the user through CMake flags
or libcxx was built together with LLDB.
Condition (2) is new and introduced in this patch in order to support
tests that are either:
* Cross-platform (such as API/macosx/macCatalyst and
API/tools/lldb-server). The just-built libcxx is usually not built for
platforms other than the host's.
* Cross-language (such as API/lang/objc/exceptions). In this case, the
Objective C runtime throws an exceptions that always goes through the
system's libcxx, instead of the just built libcxx. Fixing this would
require either changing the install-name of the just built libcxx in Mac
systems, or tuning the DYLD_LIBRARY_PATH variable at runtime.
Some other tests exposes limitations of LLDB when running with a debug
standard library. TestDbgInfoContentForwardLists had an assertion
removed, as it was checking for buggy LLDB behavior (which now
crashes). TestFixIts had a variable renamed, as the old name clashes
with a standard library name when debug info is present. This is a known
issue: https://github.com/llvm/llvm-project/issues/34391.
For `TestSBModule`, the way the "main" module is found was changed to
look for the "a.out" module, instead of relying on the index being 0. In
some systems, the index 0 is dyld when a custom standard library is
used.
Differential Revision: https://reviews.llvm.org/D132940
MSYS 'uname' on windows returns "MSYS_NT*" instead of windows32 and also
MSYS 'pwd' returns non-windows path string.
This patch fixes Makefile.rules to make adjustments required to run LLDB
API tests using MSYS tools.
Differential Revision: https://reviews.llvm.org/D133002
This patch combines D129166 (to always pick the just-built libc++) and
D132257 (to allow customizing the libc++ for testing). The common goal
is to avoid picking up an unexpected libc++ for testing.
Differential revision: https://reviews.llvm.org/D132263
When resolving symbols during IR execution, lldb makes a last effort attempt
to resolve external symbols from object files by approximate name matching.
It currently uses `CPlusPlusNameParser` to parse the demangled function name
and arguments for the unresolved symbol and its candidates. However, this
hand-rolled C++ parser doesn’t support ABI tags which, depending on the demangler,
get demangled into `[abi:tag]`. This lack of parsing support causes lldb to never
consider a candidate mangled function name that has ABI tags.
The issue reproduces by calling an ABI-tagged template function from the
expression evaluator. This is particularly problematic with the recent
addition of ABI tags to numerous libcxx APIs.
The issue stems from the fact that `clang::CodeGen` emits function
function calls using the mangled name inferred from the `FunctionDecl`
LLDB constructs from DWARF. Debug info often lacks information for
us to construct a perfect FunctionDecl resulting in subtle mangled
name inaccuracies.
This patch side-steps the problem of inaccurate `FunctionDecl`s by
attaching an `asm()` label to each `FunctionDecl` LLDB creates from DWARF.
`clang::CodeGen` consults this label to get the mangled name as one of
the first courses of action when emitting a function call.
LLDB already does this for C++ member functions as of
[675767a591](https://reviews.llvm.org/D40283)
**Testing**
* Added API tests
Differential Revision: https://reviews.llvm.org/D131974
This commit re-applies 9ee97ce3b8, which was reverted by 61d417ce
because it broke the LLDB data formatter tests. It also re-applies
6148c79a (the manual GN change associated to it).
Differential Revision: https://reviews.llvm.org/D127444
This patch fixes LLDB API tests MakeFile.rules to allow overriding of
debug symbol flags when compiling tests for Windows.
Previously windows tests were forced to emit only dwarf debug symbols
as majority of the tests rely on dwarf debug info. After this patch
any test can override debug symbol flag by setting DEBUG_INFO_FLAG
variable in its make file.
This patch removes use of -fno-builtin flag for building LLDB API
tests.
LLDB API tests are built using Makefile.rules where we were using
-fno-builtin flag to avoid gcc intrinsic optimization conflicting
with Android runtime in past.
Now that we no longer use gcc for building testsuite and compiling
LLDB API tests on AArch64/Windows require clang to optimize certain
calls like _setjmp to setjmpex as former is not implemented by
AArch64 windows runtime.
This patch fixes an issue, where if the thread has a signal blocked when
we try to inject it into the process (via vCont), then instead of
executing straight away, the injected signal will trigger another stop
when the thread unblocks the signal.
As (linux) threads start their life with SIGUSR1 (among others)
disabled, and only enable it during initialization, injecting the signal
during this window did not behave as expected. The fix is to change the
test to ensure the signal gets injected with the signal unblocked.
The simplest way to do this was to write a dedicated inferior for this
test. I also created a new header to factor out the function retrieving
the (os-specific) thread id.
On my Debian machine, system libc++/libc++abi is not installed (`libc++1-9 libc++abi-9`),
21 check-lldb-api tests fail because -stdlib=libc++ linked executables cannot
find runtime libc++.so.1 at runtime.
Use the `-Wl,-rpath,$(LLVM_LIBS_DIR)` mechanism in
`packages/Python/lldbsuite/test/make/Makefile.rules` (D58630 for NetBSD) to
allow such tests compile/link with fresh libc++ built beside lldb.
(A system libc++.so.1 is not guaranteed to match fresh libc++ header files.)
Some tweaks to the existing NetBSD rule when generalizing:
* Drop `-L$(LLVM_LIBS_DIR)` since Clang driver adds it correctly.
* Add `-stdlib=libc++` only for `USE_LIBCPP`.
Also, drop `-isystem /usr/include/c++/v1` introduced in D9426. It is not needed
by Clang driver. GCC using libc++ requires more setup.
I don't find any test needing `-Wl,-rpath` in `test/Shell/helper/{build,toolchain}.py` (D58630 for NetBSD added them).
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D94888
