This patch improves LLDB launch time on Linux machines for **preload scenarios**, particularly for executables with a lot of shared library dependencies (or modules). Specifically: * Launching a binary with `target.preload-symbols = true` * Attaching to a process with `target.preload-symbols = true`. It's completely controlled by a new flag added in the first commit `plugin.dynamic-loader.posix-dyld.parallel-module-load`, which *defaults to false*. This was inspired by similar work on Darwin #110646. Some rough numbers to showcase perf improvement, run on a very beefy machine: * Executable with ~5600 modules: baseline 45s, improvement 15s * Executable with ~3800 modules: baseline 25s, improvement 10s * Executable with ~6650 modules: baseline 67s, improvement 20s * Executable with ~12500 modules: baseline 185s, improvement 85s * Executable with ~14700 modules: baseline 235s, improvement 120s A lot of targets we deal with have a *ton* of modules, and unfortunately we're unable to convince other folks to reduce the number of modules, so performance improvements like this can be very impactful for user experience. This patch achieves the performance improvement by parallelizing `DynamicLoaderPOSIXDYLD::RefreshModules` for the launch scenario, and `DynamicLoaderPOSIXDYLD::LoadAllCurrentModules` for the attach scenario. The commits have some context on their specific changes as well -- hopefully this helps the review. # More context on implementation We discovered the bottlenecks by via `perf record -g -p <lldb's pid>` on a Linux machine. With an executable known to have 1000s of shared library dependencies, I ran ``` (lldb) b main (lldb) r # taking a while ``` and showed the resulting perf trace (snippet shown) ``` Samples: 85K of event 'cycles:P', Event count (approx.): 54615855812 Children Self Command Shared Object Symbol - 93.54% 0.00% intern-state libc.so.6 [.] clone3 clone3 start_thread lldb_private::HostNativeThreadBase::ThreadCreateTrampoline(void*) r std::_Function_handler<void* (), lldb_private::Process::StartPrivateStateThread(bool)::$_0>::_M_invoke(std::_Any_data const&) lldb_private::Process::RunPrivateStateThread(bool) n - lldb_private::Process::HandlePrivateEvent(std::shared_ptr<lldb_private::Event>&) - 93.54% lldb_private::Process::ShouldBroadcastEvent(lldb_private::Event*) - 93.54% lldb_private::ThreadList::ShouldStop(lldb_private::Event*) - lldb_private::Thread::ShouldStop(lldb_private::Event*) * - 93.53% lldb_private::StopInfoBreakpoint::ShouldStopSynchronous(lldb_private::Event*) t - 93.52% lldb_private::BreakpointSite::ShouldStop(lldb_private::StoppointCallbackContext*) i lldb_private::BreakpointLocationCollection::ShouldStop(lldb_private::StoppointCallbackContext*) k lldb_private::BreakpointLocation::ShouldStop(lldb_private::StoppointCallbackContext*) b lldb_private::BreakpointOptions::InvokeCallback(lldb_private::StoppointCallbackContext*, unsigned long, unsigned long) i DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit(void*, lldb_private::StoppointCallbackContext*, unsigned long, unsigned lo - DynamicLoaderPOSIXDYLD::RefreshModules() O - 93.42% DynamicLoaderPOSIXDYLD::RefreshModules()::$_0::operator()(DYLDRendezvous::SOEntry const&) const u - 93.40% DynamicLoaderPOSIXDYLD::LoadModuleAtAddress(lldb_private::FileSpec const&, unsigned long, unsigned long, bools - lldb_private::DynamicLoader::LoadModuleAtAddress(lldb_private::FileSpec const&, unsigned long, unsigned long, boos - 83.90% lldb_private::DynamicLoader::FindModuleViaTarget(lldb_private::FileSpec const&) o - 83.01% lldb_private::Target::GetOrCreateModule(lldb_private::ModuleSpec const&, bool, lldb_private::Status* - 77.89% lldb_private::Module::PreloadSymbols() - 44.06% lldb_private::Symtab::PreloadSymbols() - 43.66% lldb_private::Symtab::InitNameIndexes() ... ``` We saw that majority of time was spent in `RefreshModules`, with the main culprit within it `LoadModuleAtAddress` which eventually calls `PreloadSymbols`. At first, `DynamicLoaderPOSIXDYLD::LoadModuleAtAddress` appears fairly independent -- most of it deals with different files and then getting or creating Modules from these files. The portions that aren't independent seem to deal with ModuleLists, which appear concurrency safe. There were members of `DynamicLoaderPOSIXDYLD` I had to synchronize though: namely `m_loaded_modules` which `DynamicLoaderPOSIXDYLD` maintains to map its loaded modules to their link addresses. Without synchronizing this, I ran into SEGFAULTS and other issues when running `check-lldb`. I also locked the assignment and comparison of `m_interpreter_module`, which may be unnecessary. # Alternate implementations When creating this patch, another implementation I considered was directly background-ing the call to `Module::PreloadSymbol` in `Target::GetOrCreateModule`. It would have the added benefit of working across platforms generically, and appeared to be concurrency safe. It was done via `Debugger::GetThreadPool().async` directly. However, there were a ton of concurrency issues, so I abandoned that approach for now. # Testing With the feature active, I tested via `ninja check-lldb` on both Debug and Release builds several times (~5 or 6 altogether?), and didn't spot additional failing or flaky tests. I also tested manually on several different binaries, some with around 14000 modules, but just basic operations: launching, reaching main, setting breakpoint, stepping, showing some backtraces. I've also tested with the flag off just to make sure things behave properly synchronously.
195 lines
6.9 KiB
C++
195 lines
6.9 KiB
C++
//===-- DynamicLoaderPOSIXDYLD.h --------------------------------*- C++ -*-===//
|
|
//
|
|
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
|
|
// See https://llvm.org/LICENSE.txt for license information.
|
|
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
|
//
|
|
//===----------------------------------------------------------------------===//
|
|
|
|
#ifndef LLDB_SOURCE_PLUGINS_DYNAMICLOADER_POSIX_DYLD_DYNAMICLOADERPOSIXDYLD_H
|
|
#define LLDB_SOURCE_PLUGINS_DYNAMICLOADER_POSIX_DYLD_DYNAMICLOADERPOSIXDYLD_H
|
|
|
|
#include <map>
|
|
#include <memory>
|
|
|
|
#include "DYLDRendezvous.h"
|
|
#include "Plugins/Process/Utility/AuxVector.h"
|
|
#include "lldb/Breakpoint/StoppointCallbackContext.h"
|
|
#include "lldb/Core/ModuleList.h"
|
|
#include "lldb/Target/DynamicLoader.h"
|
|
|
|
class AuxVector;
|
|
|
|
class DynamicLoaderPOSIXDYLD : public lldb_private::DynamicLoader {
|
|
public:
|
|
DynamicLoaderPOSIXDYLD(lldb_private::Process *process);
|
|
|
|
~DynamicLoaderPOSIXDYLD() override;
|
|
|
|
static void Initialize();
|
|
|
|
static void Terminate();
|
|
|
|
static llvm::StringRef GetPluginNameStatic() { return "posix-dyld"; }
|
|
|
|
static llvm::StringRef GetPluginDescriptionStatic();
|
|
|
|
static lldb_private::DynamicLoader *
|
|
CreateInstance(lldb_private::Process *process, bool force);
|
|
|
|
// DynamicLoader protocol
|
|
|
|
void DidAttach() override;
|
|
|
|
void DidLaunch() override;
|
|
|
|
lldb::ThreadPlanSP GetStepThroughTrampolinePlan(lldb_private::Thread &thread,
|
|
bool stop_others) override;
|
|
|
|
lldb_private::Status CanLoadImage() override;
|
|
|
|
lldb::addr_t GetThreadLocalData(const lldb::ModuleSP module,
|
|
const lldb::ThreadSP thread,
|
|
lldb::addr_t tls_file_addr) override;
|
|
|
|
// PluginInterface protocol
|
|
llvm::StringRef GetPluginName() override { return GetPluginNameStatic(); }
|
|
|
|
lldb::ModuleSP LoadModuleAtAddress(const lldb_private::FileSpec &file,
|
|
lldb::addr_t link_map_addr,
|
|
lldb::addr_t base_addr,
|
|
bool base_addr_is_offset) override;
|
|
|
|
void CalculateDynamicSaveCoreRanges(
|
|
lldb_private::Process &process,
|
|
std::vector<lldb_private::MemoryRegionInfo> &ranges,
|
|
llvm::function_ref<bool(const lldb_private::Thread &)>
|
|
save_thread_predicate) override;
|
|
|
|
protected:
|
|
/// Runtime linker rendezvous structure.
|
|
DYLDRendezvous m_rendezvous;
|
|
|
|
/// Virtual load address of the inferior process.
|
|
lldb::addr_t m_load_offset;
|
|
|
|
/// Virtual entry address of the inferior process.
|
|
lldb::addr_t m_entry_point;
|
|
|
|
/// Auxiliary vector of the inferior process.
|
|
std::unique_ptr<AuxVector> m_auxv;
|
|
|
|
/// Rendezvous breakpoint.
|
|
lldb::break_id_t m_dyld_bid;
|
|
|
|
/// Contains AT_SYSINFO_EHDR, which means a vDSO has been
|
|
/// mapped to the address space
|
|
lldb::addr_t m_vdso_base;
|
|
|
|
/// Contains AT_BASE, which means a dynamic loader has been
|
|
/// mapped to the address space
|
|
lldb::addr_t m_interpreter_base;
|
|
|
|
/// Contains the pointer to the interpret module, if loaded.
|
|
std::weak_ptr<lldb_private::Module> m_interpreter_module;
|
|
|
|
/// Returns true if the process is for a core file.
|
|
bool IsCoreFile() const;
|
|
|
|
/// If possible sets a breakpoint on a function called by the runtime
|
|
/// linker each time a module is loaded or unloaded.
|
|
bool SetRendezvousBreakpoint();
|
|
|
|
/// Callback routine which updates the current list of loaded modules based
|
|
/// on the information supplied by the runtime linker.
|
|
static bool RendezvousBreakpointHit(
|
|
void *baton, lldb_private::StoppointCallbackContext *context,
|
|
lldb::user_id_t break_id, lldb::user_id_t break_loc_id);
|
|
|
|
/// Indicates whether the initial set of modules was reported added.
|
|
bool m_initial_modules_added;
|
|
|
|
/// Helper method for RendezvousBreakpointHit. Updates LLDB's current set
|
|
/// of loaded modules.
|
|
void RefreshModules();
|
|
|
|
/// Updates the load address of every allocatable section in \p module.
|
|
///
|
|
/// \param module The module to traverse.
|
|
///
|
|
/// \param link_map_addr The virtual address of the link map for the @p
|
|
/// module.
|
|
///
|
|
/// \param base_addr The virtual base address \p module is loaded at.
|
|
void UpdateLoadedSections(lldb::ModuleSP module, lldb::addr_t link_map_addr,
|
|
lldb::addr_t base_addr,
|
|
bool base_addr_is_offset) override;
|
|
|
|
/// Removes the loaded sections from the target in \p module.
|
|
///
|
|
/// \param module The module to traverse.
|
|
void UnloadSections(const lldb::ModuleSP module) override;
|
|
|
|
/// Resolves the entry point for the current inferior process and sets a
|
|
/// breakpoint at that address.
|
|
void ProbeEntry();
|
|
|
|
/// Callback routine invoked when we hit the breakpoint on process entry.
|
|
///
|
|
/// This routine is responsible for resolving the load addresses of all
|
|
/// dependent modules required by the inferior and setting up the rendezvous
|
|
/// breakpoint.
|
|
static bool
|
|
EntryBreakpointHit(void *baton,
|
|
lldb_private::StoppointCallbackContext *context,
|
|
lldb::user_id_t break_id, lldb::user_id_t break_loc_id);
|
|
|
|
/// Helper for the entry breakpoint callback. Resolves the load addresses
|
|
/// of all dependent modules.
|
|
virtual void LoadAllCurrentModules();
|
|
|
|
void LoadVDSO();
|
|
|
|
// Loading an interpreter module (if present) assuming m_interpreter_base
|
|
// already points to its base address.
|
|
lldb::ModuleSP LoadInterpreterModule();
|
|
|
|
/// Computes a value for m_load_offset returning the computed address on
|
|
/// success and LLDB_INVALID_ADDRESS on failure.
|
|
lldb::addr_t ComputeLoadOffset();
|
|
|
|
/// Computes a value for m_entry_point returning the computed address on
|
|
/// success and LLDB_INVALID_ADDRESS on failure.
|
|
lldb::addr_t GetEntryPoint();
|
|
|
|
/// Evaluate if Aux vectors contain vDSO and LD information
|
|
/// in case they do, read and assign the address to m_vdso_base
|
|
/// and m_interpreter_base.
|
|
void EvalSpecialModulesStatus();
|
|
|
|
/// Loads Module from inferior process.
|
|
void ResolveExecutableModule(lldb::ModuleSP &module_sp);
|
|
|
|
bool AlwaysRelyOnEHUnwindInfo(lldb_private::SymbolContext &sym_ctx) override;
|
|
|
|
private:
|
|
DynamicLoaderPOSIXDYLD(const DynamicLoaderPOSIXDYLD &) = delete;
|
|
const DynamicLoaderPOSIXDYLD &
|
|
operator=(const DynamicLoaderPOSIXDYLD &) = delete;
|
|
|
|
/// Loaded module list. (link map for each module)
|
|
/// This may be accessed in a multi-threaded context. Use the accessor methods
|
|
/// to access `m_loaded_modules` safely.
|
|
std::map<lldb::ModuleWP, lldb::addr_t, std::owner_less<lldb::ModuleWP>>
|
|
m_loaded_modules;
|
|
llvm::sys::RWMutex m_loaded_modules_rw_mutex;
|
|
|
|
void SetLoadedModule(const lldb::ModuleSP &module_sp,
|
|
lldb::addr_t link_map_addr);
|
|
void UnloadModule(const lldb::ModuleSP &module_sp);
|
|
std::optional<lldb::addr_t>
|
|
GetLoadedModuleLinkAddr(const lldb::ModuleSP &module_sp);
|
|
};
|
|
|
|
#endif // LLDB_SOURCE_PLUGINS_DYNAMICLOADER_POSIX_DYLD_DYNAMICLOADERPOSIXDYLD_H
|