Avoid deadlocks in the Alarm class by releasing the lock before invoking
callbacks. This deadlock manifested itself in the ProgressManager:
1. On the main thread, the ProgressManager acquires its lock in
ProgressManager::Decrement and calls Alarm::Create.
2. On the main thread, the Alarm acquires its lock in Alarm::Create.
3. On the alarm thread, the Alarm acquires its lock after waiting on the
condition variable and calls ProgressManager::Expire.
4. On the alarm thread, the ProgressManager acquires its lock in
ProgressManager::Expire.
Note how the two threads are acquiring the locks in different orders.
Deadlocks can be avoided by always acquiring locks in the same order,
but since the two mutexes here are private implementation details,
belong to different classes, that's not straightforward. Luckily, we
don't need to have the Alarm mutex locked when invoking the callbacks.
That exactly how this patch solves the issue.
llvm-project/lldb/source/Host/common/Alarm.cpp:37:5:
error: 'lock_guard' may not intend to support class template argument deduction [-Werror,-Wctad-maybe-unsupported]
std::lock_guard alarm_guard(m_alarm_mutex);
^
The commit introduces a new, generic, Alarm class. The class lets you to
schedule functions (callbacks) that will execute after a predefined
timeout. Once scheduled, you can cancel and reset a callback, given the
timeout hasn't expired yet.
The alarm class worker thread that sleeps until the next timeout
expires. When the thread wakes up, it checks for all the callbacks that
have expired and calls them in order. Because the callback is called
from the worker thread, the only guarantee is that a callback is called
no sooner than the timeout. A long running callback could potentially
block the worker threads and delay other callbacks from getting called.
I intentionally kept the implementation as simple as possible while
addressing the needs for the use case of coalescing progress events as
discussed in [1]. If we want to rely on this somewhere else, we can
reassess whether we need to address this class' limitations.
[1] https://discourse.llvm.org/t/rfc-improve-lldb-progress-reporting/75717/
If the `m_editor_status` is `EditorStatus::Editing`, PrintAsync clears
the currently edited line. In some situations, the edited line is not
saved. After the stream flushes, PrintAsync tries to display the unsaved
line, causing the loss of the edited line.
The issue arose while I was debugging REPRLRun in
[Fuzzilli](https://github.com/googleprojectzero/fuzzilli). I started
LLDB and attempted to set a breakpoint in libreprl-posix.c. I entered
`breakpoint set -f lib` and used the "tab" key for command completion.
After completion, the edited line was flushed, leaving a blank line.
Currently, calls to Host::SystemLog print to stderr on all host
platforms except Darwin. This severely limits its value on the command
line, where we don't want to overload the user with log messages. Switch
to using the syslog function on POSIX systems to send messages to the
system logger instead of stdout.
On Darwin systems this sends the log message to os_log, which matches
what we do today. Nevertheless I kept the current implementation that
uses os_log directly as it gives us more freedom.
I'm not sure if there's an equivalent on Windows, so I kept the existing
behavior of logging to stderr.
https://github.com/modularml/mojo/issues/1796 discovered that if you try
to complete a space-only line in the REPL on Linux, LLDB crashes. I
suspect that editline doesn't behave the same way on linux and on
darwin, because I can't replicate this on darwin.
Adding a boundary check in the completion code prevents the crash from
happening.
LLDB has a setting (symbols.enable-background-lookup) that calls
dsymForUUID on a background thread for images as they appear in the
current backtrace. Originally, the laziness of only looking up symbols
for images in the backtrace only existed to bring the number of
dsymForUUID calls down to a manageable number.
Users have requesting the same functionality but blocking. This gives
them the same user experience as enabling dsymForUUID globally, but
without the massive upfront cost of having to download all the images,
the majority of which they'll likely not need.
This patch renames the setting to have a more generic name
(symbols.auto-download) and changes its values from a boolean to an
enum. Users can now specify "off", "background" and "foreground". The
default remains "off" although I'll probably change that in the near
future.
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
If you type `settings show <tab>` LLDB might crash, depending on the
version of libedit you're compiled with, and whether you're compiled
with `-DLLDB_EDITLINE_USE_WCHAR=0` (and depending on how the optimizer
lays out the stack...)
The issue has to do with trying to figure out whether the libedit
`getchar` callback is supposed to read a wide or 8 bit character. In
order to maintain backward compatibility, there's really no 'clean' way
to do it. We just have to make sure that we're invoking el_[w]getc with
a buffer that is as wide as the getchar callback (registered by the
`SetGetCharacterFunction` function further down in `Editline.cpp`.
So, it's 'fixed' with a comment, and a wider version of the 'reply'
variable.
Co-authored-by: Kevin Frei <freik@meta.com>
The minimum GCC version was bumped up from 4.8 to 5.1 and then even newer
awhile ago so garbage collect the pre 4.8 workaround.
https://reviews.llvm.org/D66188
The code was incorrectly going into the wrong direction by removing one
component instead of appendeing /Developer to it. Due to fallback
mechanisms in xcrun this never seemed to have caused any issues.
We just forget to check for interrupt while waiting for the answer to the prompt. But if we are in the interrupt state then the lower
layers of the EditLine code just eat all characters so we never get out of the query prompt. You're pretty much stuck and have to kill lldb.
The solution is to check for the interrupt. The patch is a little bigger because where I needed to check the Interrupt state I only
had the ::EditLine object, but the editor state is held in lldb's EditLine wrapper, so I had to do a little work to get my hands on it.
This patch simplifies the color handling logic in Editline and
IOHandlerEditline:
- Remove the m_color_prompts property from Editline and use the prompt
ANSI prefix and suffix as the single source of truth. This avoids
having to redraw the prompt unnecessarily, for example when colors
are enabled but the prompt prefix and suffix are empty.
- Rename m_color_prompts to just m_color in IOHandlerEditline and use
it to ensure consistency between colored prompts and colored
auto-suggestions. Some IOHandler explicitly turn off colors (such as
IOHandlerConfirm) and it doesn't really make sense to have one or the
other.
Users often want to change the look of their prompt and currently the
only way to do that is by using ANSI escape codes in the prompt itself.
This is not only tedious, it also results in extra whitespace because
our Editline wrapper, when computing the cursor column, doesn't ignore
the invisible escape codes.
We already have various *-ansi-{prefix,suffix} settings that allow the
users to customize the color of auto-suggestions and progress events,
using mnemonics like ${ansi.fg.yellow}. This patch brings the same
mechanism to the prompt.
rdar://115390406
Account for Unicode when computing the prompt column width. Previously,
the string length (i.e. number of bytes) rather than the width of the
Glyph was used to compute the cursor position. The result was that the
cursor would be offset to the right when using a prompt containing
Unicode.
When sending file from a Linux host to a Windows remote, Linux host will try to copy the source file's permission bits, which will contain `_S_I?GRP` and `_S_I?OTH` bits. Those bits are rejected by `_wsopen_s`, causing it to return EINVAL.
This patch masks out the rejected bits.
GitHub issue: #64313
Reviewed By: jasonmolenda, DavidSpickett
Differential Revision: https://reviews.llvm.org/D156817
This patch fixes warnings like:
lldb/source/Core/ModuleList.cpp:1086:3: error: 'scoped_lock' may not
intend to support class template argument deduction
[-Werror,-Wctad-maybe-unsupported]
/data/home/jiefu/llvm-project/lldb/source/Host/common/File.cpp:251:3: error: 'scoped_lock' may not intend to support class template argument deduction [-Werror,-Wctad-maybe-unsupported]
std::scoped_lock lock(m_descriptor_mutex, m_stream_mutex);
^
/opt/rh/gcc-toolset-12/root/usr/lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/mutex:692:11: note: add a deduction guide to suppress this warning
class scoped_lock
^
/data/home/jiefu/llvm-project/lldb/source/Host/common/File.cpp:316:3: error: 'scoped_lock' may not intend to support class template argument deduction [-Werror,-Wctad-maybe-unsupported]
std::scoped_lock lock(m_descriptor_mutex, m_stream_mutex);
^
/opt/rh/gcc-toolset-12/root/usr/lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/mutex:692:11: note: add a deduction guide to suppress this warning
class scoped_lock
^
2 errors generated.
Just about every file in the lldb project is built with C++ enabled.
Unless I've missed something, these macro guards don't really accomplish
very much.
Differential Revision: https://reviews.llvm.org/D157538
TSan reports the following data race:
Write of size 4 at 0x000109e0b160 by thread T2 (...):
#0 lldb_private::NativeFile::Close() File.cpp:329
#1 lldb_private::ConnectionFileDescriptor::Disconnect(...) ConnectionFileDescriptorPosix.cpp:232
#2 lldb_private::Communication::Disconnect(...) Communication.cpp:61
#3 lldb_private::process_gdb_remote::ProcessGDBRemote::DidExit() ProcessGDBRemote.cpp:1164
#4 lldb_private::Process::SetExitStatus(...) Process.cpp:1097
#5 lldb_private::process_gdb_remote::ProcessGDBRemote::MonitorDebugserverProcess(...) ProcessGDBRemote.cpp:3387
Previous read of size 4 at 0x000109e0b160 by main thread (...):
#0 lldb_private::NativeFile::IsValid() const File.h:393
#1 lldb_private::ConnectionFileDescriptor::IsConnected() const ConnectionFileDescriptorPosix.cpp:121
#2 lldb_private::Communication::IsConnected() const Communication.cpp:79
#3 lldb_private::process_gdb_remote::GDBRemoteCommunication::WaitForPacketNoLock(...) GDBRemoteCommunication.cpp:256
#4 lldb_private::process_gdb_remote::GDBRemoteCommunication::WaitForPacketNoLock(...) GDBRemoteCommunication.cpp:244
#5 lldb_private::process_gdb_remote::GDBRemoteClientBase::SendPacketAndWaitForResponseNoLock(...) GDBRemoteClientBase.cpp:246
I originally tried fixing the problem at the ConnectionFileDescriptor
level, but that operates on an IOObject which can have different thread
safety guarantees depending on its implementation.
For this particular issue, the problem is specific to NativeFile.
NativeFile can hold a file descriptor and/or a file stream. Throughout
its implementation, it checks if the descriptor or stream is valid and
do some operation on it if it is. While that works in a single threaded
environment, nothing prevents another thread from modifying the
descriptor or stream between the IsValid check and when it's actually
being used.
This patch prevents such issues by returning a ValueGuard RAII object.
As long as the object is in scope, the value is guaranteed by a lock.
Differential revision: https://reviews.llvm.org/D157347
StreamFile subclasses Stream (from lldbUtility) and is backed by a File
(from lldbHost). It does not depend on anything from lldbCore or any of its
sibling libraries, so I think it makes sense for this to live in
lldbHost instead.
Differential Revision: https://reviews.llvm.org/D157460
TSan reports the following data race:
Write of size 4 at 0x000109e0b160 by thread T2 (mutexes: write M0, write M1):
#0 NativeFile::Close() File.cpp:329
#1 ConnectionFileDescriptor::Disconnect(lldb_private::Status*) ConnectionFileDescriptorPosix.cpp:232
#2 Communication::Disconnect(lldb_private::Status*) Communication.cpp:61
#3 process_gdb_remote::ProcessGDBRemote::DidExit() ProcessGDBRemote.cpp:1164
#4 Process::SetExitStatus(int, char const*) Process.cpp:1097
#5 process_gdb_remote::ProcessGDBRemote::MonitorDebugserverProcess(...) ProcessGDBRemote.cpp:3387
Previous read of size 4 at 0x000109e0b160 by main thread (mutexes: write M2):
#0 NativeFile::IsValid() const File.h:393
#1 ConnectionFileDescriptor::IsConnected() const ConnectionFileDescriptorPosix.cpp:121
#2 Communication::IsConnected() const Communication.cpp:79
#3 process_gdb_remote::GDBRemoteCommunication::WaitForPacketNoLock(...) GDBRemoteCommunication.cpp:256
#4 process_gdb_remote::GDBRemoteCommunication::WaitForPacketNoLock(...l) GDBRemoteCommunication.cpp:244
#5 process_gdb_remote::GDBRemoteClientBase::SendPacketAndWaitForResponseNoLock(llvm::StringRef, StringExtractorGDBRemote&) GDBRemoteClientBase.cpp:246
The problem is that in WaitForPacketNoLock's run loop, it checks that
the connection is still connected. This races with the
ConnectionFileDescriptor disconnecting. Most (but not all) access to the
IOObject in ConnectionFileDescriptorPosix is already gated by the mutex.
This patch just protects IsConnected in the same way.
Differential revision: https://reviews.llvm.org/D157347
There were some checks removed previously in
bc196970b5.
However, all these SPIs are actually defined in macOS 12 and onward, not
macOSX 10.12 as the previous commit would suggest. As a result, if you
have access to these SPIs lldb will fail to compile correctly.
Instead of adding back the __builtin_availability checks, it seems
easier just to check the minimum deployment target with Availability
macros.
Differential Revision: https://reviews.llvm.org/D156838
This commit does a few related things:
- Removes unused function `uuid_is_null`
- Removes unneeded includes of UuidCompatibility.h
- Renames UuidCompatibility to AppleUuidCompatibility and adds a comment
to clarify intent of header.
- Moves AppleUuidCompatibility to the include directory
Differential Revision: https://reviews.llvm.org/D156562
In preparation for removing the `#include "llvm/ADT/StringExtras.h"`
from the header to source file of `llvm/Support/Error.h`, first add in
all the missing includes that were previously included transitively
through this header.
This is fixing all files missed in b0abd4893f and
39d8e6e22c.
Differential Revision: https://reviews.llvm.org/D154763
Fix incorrect uses of formatv specifiers in LLDB_LOG. Unlike Python,
arguments must be numbered. All the affected log statements take
llvm:Errors so use the LLDB_LOG_ERROR macro instead.
Differential revision: https://reviews.llvm.org/D154532
Instead of just returning a raw `const char *`, I think llvm::StringRef
would make more sense. Most of the time that we use the return value of
`GetProcessPluginName` we're passing it to `CreateProcess` which takes a
StringRef anyway.
Differential Revision: https://reviews.llvm.org/D153825
Currently, source files are cached by their resolved path. This means
that before we can query the cache, we potentially have to resolve the
path, which can be slow. This patch avoids the call to FileSystem::Resolve
by caching both the resolved and unresolved path. We now only resolve
the path once when we create and cache a new file.
rdar://110787562
Differential revision: https://reviews.llvm.org/D153726
Previously lldb was using arrays of size kMaxRegisterByteSize to handle
registers. This was set to 256 because the largest possible register
we support is Arm's scalable vectors (SVE) which can be up to 256 bytes long.
This means for most operations aside from SVE, we're wasting 192 bytes
of it. Which is ok given that we don't have to pay the cost of a heap
alocation and 256 bytes isn't all that much overall.
With the introduction of the Arm Scalable Matrix extension there is a new
array storage register, ZA. This register is essentially a square made up of
SVE vectors. Therefore ZA could be up to 64kb in size.
https://developer.arm.com/documentation/ddi0616/latest/
"The Effective Streaming SVE vector length, SVL, is a power of two in the range 128 to 2048 bits inclusive."
"The ZA storage is architectural register state consisting of a two-dimensional ZA array of [SVLB × SVLB] bytes."
99% of operations will never touch ZA and making every stack frame 64kb+ just
for that slim chance is a bad idea.
Instead I'm switching register handling to use SmallVector with a stack allocation
size of kTypicalRegisterByteSize. kMaxRegisterByteSize will be used in places
where we can't predict the size of register we're reading (in the GDB remote client).
The result is that the 99% of small register operations can use the stack
as before and the actual ZA operations will move to the heap as needed.
I tested this by first working out -wframe-larger-than values for all the
libraries using the arrays previously. With this change I was able to increase
kMaxRegisterByteSize to 256*256 without hitting those limits. With the
exception of the GDB server which needs to use a max size buffer.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D153626
EnumerateDirectory gets the vfs::Status of each directory entry to
decide how to process it. If it is unable to get the Status for
a directory entry, it will currently halt the directory iteration
entirely. It should only skip this entry.
Differential Revision: https://reviews.llvm.org/D153822
rdar://110861210
D152759 introduced the Android .zip so file support, but it only considered
POSIX path. The code also runs on Windows, so the path could be Windows path.
Support both patterns on Windows.
Differential Revision: https://reviews.llvm.org/D153390
In Android API level 23 and above, dynamic loader is able to load .so file
directly from APK, which is zip file.
https://android.googlesource.com/platform/bionic/+/master/
android-changes-for-ndk-developers.md#
opening-shared-libraries-directly-from-an-apk
The .so file is page aligned and uncompressed, so
ObjectFileELF::GetModuleSpecifications works with .so file offset and size
directly from zip file without extracting it. (D152757)
GDBRemoteCommunicationServerCommon::GetModuleInfo returns a module spec to LLDB
with "zip_path!/so_path" file spec, which is passed through from Android
dynamic loader, and the .so file offset and size.
PlatformAndroid::DownloadModuleSlice uses 'shell dd' to download the .so file
slice from the zip file with the .so file offset and size.
Differential Revision: https://reviews.llvm.org/D152759
