When `FileAction` opens file with write access, it doesn't clear the
file nor append to the end of the file if it already exists. Instead, it
writes from cursor index 0.
For example, by using the settings `target.output-path` and
`target.error-path`, lldb will redirect process stdout/stderr to files.
It then calls this function to write to the files which the above
symptoms appear.
## Test
- Added unit test checking the file flags
- Added 2 api tests checking
- File content overwritten if the file path already exists
- Stdout and stderr redirection to the same file doesn't change its
behavior
Disable -Wdeprecated-declarations for codecvt_utf8 in Editline. This is
in preparation for #112276 which narrows the scope of
-Wno-deprecated-declarations for building LLDB.
As discussed in #111911, we have consensus that as it stands, the system
log is only meaningful on Darwin and that by default it should be a NOOP
on other platforms.
Windows doesn't have a built-in system log. Previously we got away with
writing to stdout and stderr because it was used only sporadically. As
we're trying to make the system log more useful on the other platforms,
the increased use become a concern. Make it a NOOP until someone figures
out a reasonable alternative.
Add an "always on" log category and channel. Unlike other, existing log
channels, it is not exposed to users. The channel is meant to be used
sparsely and deliberately for logging high-value information to the
system log.
We have a similar concept in the downstream Swift fork and this has
proven to be extremely valuable. This is especially true on macOS where
system log messages are automatically captured as part of a sysdiagnose.
(this is lldb part)
Without these explicit includes, removing other headers, who implicitly
include llvm-config.h, may have non-trivial side effects. For example,
`clangd` may report even `llvm-config.h` as "no used" in case it defines
a macro, that is explicitly used with #ifdef. It is actually amplified
with different build configs which use different set of macros.
The existing function already used the MainLoop class, which allows one
to wait on multiple events at once. It needed to do this in order to
wait for v4 and v6 connections simultaneously. However, since it was
creating its own instance of MainLoop, this meant that it was impossible
to multiplex these sockets with anything else.
This patch simply adds a version of this function which uses an
externally provided main loop instance, which allows the caller to add
any events it deems necessary. The previous function becomes a very thin
wrapper over the new one.
This patch removes all of the Set.* methods from Status.
This cleanup is part of a series of patches that make it harder use the
anti-pattern of keeping a long-lives Status object around and updating
it while dropping any errors it contains on the floor.
This patch is largely NFC, the more interesting next steps this enables
is to:
1. remove Status.Clear()
2. assert that Status::operator=() never overwrites an error
3. remove Status::operator=()
Note that step (2) will bring 90% of the benefits for users, and step
(3) will dramatically clean up the error handling code in various
places. In the end my goal is to convert all APIs that are of the form
` ResultTy DoFoo(Status& error)
`
to
` llvm::Expected<ResultTy> DoFoo()
`
How to read this patch?
The interesting changes are in Status.h and Status.cpp, all other
changes are mostly
` perl -pi -e 's/\.SetErrorString/ = Status::FromErrorString/g' $(git
grep -l SetErrorString lldb/source)
`
plus the occasional manual cleanup.
Don't pass `LOG_CONS` to the openlog call.
> Write directly to the system console if there is an error while
> sending to the system logger.
This seemed like a good idea at the time, but it turns out to be
extremely annoying when this happens and LLDB is overwhelmed by log
messages in the console.
rdar://132243490
Remove setupterm workaround on macOS which caused an issues after the
removal of the terminfo dependency. There's a comment that explains why
the workaround is present, but neither Jim nor I were able to reproduce
the issue by setting TERM to vt100.
For the significant amount of call sites that want to create an
incontrovertible error, such a wrapper function creates a significant
readability improvement and lowers the cost of entry to add error
handling in more places.
Avoid deadlocks in the Alarm class by releasing the lock before invoking
callbacks. This deadlock manifested itself in the ProgressManager:
1. On the main thread, the ProgressManager acquires its lock in
ProgressManager::Decrement and calls Alarm::Create.
2. On the main thread, the Alarm acquires its lock in Alarm::Create.
3. On the alarm thread, the Alarm acquires its lock after waiting on the
condition variable and calls ProgressManager::Expire.
4. On the alarm thread, the ProgressManager acquires its lock in
ProgressManager::Expire.
Note how the two threads are acquiring the locks in different orders.
Deadlocks can be avoided by always acquiring locks in the same order,
but since the two mutexes here are private implementation details,
belong to different classes, that's not straightforward. Luckily, we
don't need to have the Alarm mutex locked when invoking the callbacks.
That exactly how this patch solves the issue.
llvm-project/lldb/source/Host/common/Alarm.cpp:37:5:
error: 'lock_guard' may not intend to support class template argument deduction [-Werror,-Wctad-maybe-unsupported]
std::lock_guard alarm_guard(m_alarm_mutex);
^
The commit introduces a new, generic, Alarm class. The class lets you to
schedule functions (callbacks) that will execute after a predefined
timeout. Once scheduled, you can cancel and reset a callback, given the
timeout hasn't expired yet.
The alarm class worker thread that sleeps until the next timeout
expires. When the thread wakes up, it checks for all the callbacks that
have expired and calls them in order. Because the callback is called
from the worker thread, the only guarantee is that a callback is called
no sooner than the timeout. A long running callback could potentially
block the worker threads and delay other callbacks from getting called.
I intentionally kept the implementation as simple as possible while
addressing the needs for the use case of coalescing progress events as
discussed in [1]. If we want to rely on this somewhere else, we can
reassess whether we need to address this class' limitations.
[1] https://discourse.llvm.org/t/rfc-improve-lldb-progress-reporting/75717/
If the `m_editor_status` is `EditorStatus::Editing`, PrintAsync clears
the currently edited line. In some situations, the edited line is not
saved. After the stream flushes, PrintAsync tries to display the unsaved
line, causing the loss of the edited line.
The issue arose while I was debugging REPRLRun in
[Fuzzilli](https://github.com/googleprojectzero/fuzzilli). I started
LLDB and attempted to set a breakpoint in libreprl-posix.c. I entered
`breakpoint set -f lib` and used the "tab" key for command completion.
After completion, the edited line was flushed, leaving a blank line.
Currently, calls to Host::SystemLog print to stderr on all host
platforms except Darwin. This severely limits its value on the command
line, where we don't want to overload the user with log messages. Switch
to using the syslog function on POSIX systems to send messages to the
system logger instead of stdout.
On Darwin systems this sends the log message to os_log, which matches
what we do today. Nevertheless I kept the current implementation that
uses os_log directly as it gives us more freedom.
I'm not sure if there's an equivalent on Windows, so I kept the existing
behavior of logging to stderr.
https://github.com/modularml/mojo/issues/1796 discovered that if you try
to complete a space-only line in the REPL on Linux, LLDB crashes. I
suspect that editline doesn't behave the same way on linux and on
darwin, because I can't replicate this on darwin.
Adding a boundary check in the completion code prevents the crash from
happening.
LLDB has a setting (symbols.enable-background-lookup) that calls
dsymForUUID on a background thread for images as they appear in the
current backtrace. Originally, the laziness of only looking up symbols
for images in the backtrace only existed to bring the number of
dsymForUUID calls down to a manageable number.
Users have requesting the same functionality but blocking. This gives
them the same user experience as enabling dsymForUUID globally, but
without the massive upfront cost of having to download all the images,
the majority of which they'll likely not need.
This patch renames the setting to have a more generic name
(symbols.auto-download) and changes its values from a boolean to an
enum. Users can now specify "off", "background" and "foreground". The
default remains "off" although I'll probably change that in the near
future.
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
If you type `settings show <tab>` LLDB might crash, depending on the
version of libedit you're compiled with, and whether you're compiled
with `-DLLDB_EDITLINE_USE_WCHAR=0` (and depending on how the optimizer
lays out the stack...)
The issue has to do with trying to figure out whether the libedit
`getchar` callback is supposed to read a wide or 8 bit character. In
order to maintain backward compatibility, there's really no 'clean' way
to do it. We just have to make sure that we're invoking el_[w]getc with
a buffer that is as wide as the getchar callback (registered by the
`SetGetCharacterFunction` function further down in `Editline.cpp`.
So, it's 'fixed' with a comment, and a wider version of the 'reply'
variable.
Co-authored-by: Kevin Frei <freik@meta.com>
The minimum GCC version was bumped up from 4.8 to 5.1 and then even newer
awhile ago so garbage collect the pre 4.8 workaround.
https://reviews.llvm.org/D66188
We just forget to check for interrupt while waiting for the answer to the prompt. But if we are in the interrupt state then the lower
layers of the EditLine code just eat all characters so we never get out of the query prompt. You're pretty much stuck and have to kill lldb.
The solution is to check for the interrupt. The patch is a little bigger because where I needed to check the Interrupt state I only
had the ::EditLine object, but the editor state is held in lldb's EditLine wrapper, so I had to do a little work to get my hands on it.
This patch simplifies the color handling logic in Editline and
IOHandlerEditline:
- Remove the m_color_prompts property from Editline and use the prompt
ANSI prefix and suffix as the single source of truth. This avoids
having to redraw the prompt unnecessarily, for example when colors
are enabled but the prompt prefix and suffix are empty.
- Rename m_color_prompts to just m_color in IOHandlerEditline and use
it to ensure consistency between colored prompts and colored
auto-suggestions. Some IOHandler explicitly turn off colors (such as
IOHandlerConfirm) and it doesn't really make sense to have one or the
other.
Users often want to change the look of their prompt and currently the
only way to do that is by using ANSI escape codes in the prompt itself.
This is not only tedious, it also results in extra whitespace because
our Editline wrapper, when computing the cursor column, doesn't ignore
the invisible escape codes.
We already have various *-ansi-{prefix,suffix} settings that allow the
users to customize the color of auto-suggestions and progress events,
using mnemonics like ${ansi.fg.yellow}. This patch brings the same
mechanism to the prompt.
rdar://115390406
Account for Unicode when computing the prompt column width. Previously,
the string length (i.e. number of bytes) rather than the width of the
Glyph was used to compute the cursor position. The result was that the
cursor would be offset to the right when using a prompt containing
Unicode.
/data/home/jiefu/llvm-project/lldb/source/Host/common/File.cpp:251:3: error: 'scoped_lock' may not intend to support class template argument deduction [-Werror,-Wctad-maybe-unsupported]
std::scoped_lock lock(m_descriptor_mutex, m_stream_mutex);
^
/opt/rh/gcc-toolset-12/root/usr/lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/mutex:692:11: note: add a deduction guide to suppress this warning
class scoped_lock
^
/data/home/jiefu/llvm-project/lldb/source/Host/common/File.cpp:316:3: error: 'scoped_lock' may not intend to support class template argument deduction [-Werror,-Wctad-maybe-unsupported]
std::scoped_lock lock(m_descriptor_mutex, m_stream_mutex);
^
/opt/rh/gcc-toolset-12/root/usr/lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/mutex:692:11: note: add a deduction guide to suppress this warning
class scoped_lock
^
2 errors generated.
TSan reports the following data race:
Write of size 4 at 0x000109e0b160 by thread T2 (...):
#0 lldb_private::NativeFile::Close() File.cpp:329
#1 lldb_private::ConnectionFileDescriptor::Disconnect(...) ConnectionFileDescriptorPosix.cpp:232
#2 lldb_private::Communication::Disconnect(...) Communication.cpp:61
#3 lldb_private::process_gdb_remote::ProcessGDBRemote::DidExit() ProcessGDBRemote.cpp:1164
#4 lldb_private::Process::SetExitStatus(...) Process.cpp:1097
#5 lldb_private::process_gdb_remote::ProcessGDBRemote::MonitorDebugserverProcess(...) ProcessGDBRemote.cpp:3387
Previous read of size 4 at 0x000109e0b160 by main thread (...):
#0 lldb_private::NativeFile::IsValid() const File.h:393
#1 lldb_private::ConnectionFileDescriptor::IsConnected() const ConnectionFileDescriptorPosix.cpp:121
#2 lldb_private::Communication::IsConnected() const Communication.cpp:79
#3 lldb_private::process_gdb_remote::GDBRemoteCommunication::WaitForPacketNoLock(...) GDBRemoteCommunication.cpp:256
#4 lldb_private::process_gdb_remote::GDBRemoteCommunication::WaitForPacketNoLock(...) GDBRemoteCommunication.cpp:244
#5 lldb_private::process_gdb_remote::GDBRemoteClientBase::SendPacketAndWaitForResponseNoLock(...) GDBRemoteClientBase.cpp:246
I originally tried fixing the problem at the ConnectionFileDescriptor
level, but that operates on an IOObject which can have different thread
safety guarantees depending on its implementation.
For this particular issue, the problem is specific to NativeFile.
NativeFile can hold a file descriptor and/or a file stream. Throughout
its implementation, it checks if the descriptor or stream is valid and
do some operation on it if it is. While that works in a single threaded
environment, nothing prevents another thread from modifying the
descriptor or stream between the IsValid check and when it's actually
being used.
This patch prevents such issues by returning a ValueGuard RAII object.
As long as the object is in scope, the value is guaranteed by a lock.
Differential revision: https://reviews.llvm.org/D157347
StreamFile subclasses Stream (from lldbUtility) and is backed by a File
(from lldbHost). It does not depend on anything from lldbCore or any of its
sibling libraries, so I think it makes sense for this to live in
lldbHost instead.
Differential Revision: https://reviews.llvm.org/D157460
In preparation for removing the `#include "llvm/ADT/StringExtras.h"`
from the header to source file of `llvm/Support/Error.h`, first add in
all the missing includes that were previously included transitively
through this header.
This is fixing all files missed in b0abd4893f and
39d8e6e22c.
Differential Revision: https://reviews.llvm.org/D154763
Fix incorrect uses of formatv specifiers in LLDB_LOG. Unlike Python,
arguments must be numbered. All the affected log statements take
llvm:Errors so use the LLDB_LOG_ERROR macro instead.
Differential revision: https://reviews.llvm.org/D154532
Instead of just returning a raw `const char *`, I think llvm::StringRef
would make more sense. Most of the time that we use the return value of
`GetProcessPluginName` we're passing it to `CreateProcess` which takes a
StringRef anyway.
Differential Revision: https://reviews.llvm.org/D153825
Currently, source files are cached by their resolved path. This means
that before we can query the cache, we potentially have to resolve the
path, which can be slow. This patch avoids the call to FileSystem::Resolve
by caching both the resolved and unresolved path. We now only resolve
the path once when we create and cache a new file.
rdar://110787562
Differential revision: https://reviews.llvm.org/D153726
Previously lldb was using arrays of size kMaxRegisterByteSize to handle
registers. This was set to 256 because the largest possible register
we support is Arm's scalable vectors (SVE) which can be up to 256 bytes long.
This means for most operations aside from SVE, we're wasting 192 bytes
of it. Which is ok given that we don't have to pay the cost of a heap
alocation and 256 bytes isn't all that much overall.
With the introduction of the Arm Scalable Matrix extension there is a new
array storage register, ZA. This register is essentially a square made up of
SVE vectors. Therefore ZA could be up to 64kb in size.
https://developer.arm.com/documentation/ddi0616/latest/
"The Effective Streaming SVE vector length, SVL, is a power of two in the range 128 to 2048 bits inclusive."
"The ZA storage is architectural register state consisting of a two-dimensional ZA array of [SVLB × SVLB] bytes."
99% of operations will never touch ZA and making every stack frame 64kb+ just
for that slim chance is a bad idea.
Instead I'm switching register handling to use SmallVector with a stack allocation
size of kTypicalRegisterByteSize. kMaxRegisterByteSize will be used in places
where we can't predict the size of register we're reading (in the GDB remote client).
The result is that the 99% of small register operations can use the stack
as before and the actual ZA operations will move to the heap as needed.
I tested this by first working out -wframe-larger-than values for all the
libraries using the arrays previously. With this change I was able to increase
kMaxRegisterByteSize to 256*256 without hitting those limits. With the
exception of the GDB server which needs to use a max size buffer.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D153626
EnumerateDirectory gets the vfs::Status of each directory entry to
decide how to process it. If it is unable to get the Status for
a directory entry, it will currently halt the directory iteration
entirely. It should only skip this entry.
Differential Revision: https://reviews.llvm.org/D153822
rdar://110861210
