## Summary Implement the complete index system for cross-file LSP features. This adds persistent two-tier indexing (ProjectIndex + per-file MergedIndex shards), background indexing triggered on idle, and index-based query handlers for major LSP requests. ### Index Data Layer (`src/index/`) - **TUIndex**: Add binary serialization/deserialization via FlatBuffers, enabling IPC between stateless worker and master server - **ProjectIndex**: Add symbol name/kind storage, `PathPool` path normalization (backslash -> forward slash), and binary persistence - **MergedIndex**: Add `content` field to store file content for reliable offset<->position mapping; add `removed` bitmap for garbage collection of deleted entries; filter removed IDs in `lookup()` queries - **schema.fbs**: Add TUIndex tables, `Symbol.name` field, `MergedIndex.removed` bitmap and `MergedIndex.content` string ### Server (`src/server/`) - **Background indexing**: Idle-triggered coroutine dequeues files from CDB, dispatches `IndexParams` to stateless workers, merges returned `TUIndex` into ProjectIndex/MergedIndex, and persists to `.clice/index/` - **Index persistence**: `save_index()` / `load_index()` for startup restoration; only rewrites shards flagged `need_rewrite()` - **LSP handlers**: - `textDocument/definition` -- index-first lookup with stateful worker fallback - `textDocument/references` -- cross-file reference query via index - `callHierarchy/prepare`, `incomingCalls`, `outgoingCalls` -- Caller/Callee relation traversal - `typeHierarchy/prepare`, `supertypes`, `subtypes` -- Base/Derived relation traversal - `workspace/symbol` -- case-insensitive substring search over ProjectIndex symbols - **Stateless worker**: Add `Index` request handler that builds `TUIndex` from compiled AST and returns serialized data - **Config**: Add `enable_indexing` (default true) and `idle_timeout_ms` (default 3000ms) ### Fixes and Cross-platform - **ElaboratedType handling** in `decl_of()` for correct Base/Derived relation emission - **Windows path normalization** in `PathPool::intern()` and `ProjectIndex::from()` (backslash -> forward slash) - **`.gitattributes`**: Force LF in `tests/data/**` to prevent CRLF byte-offset mismatches on Windows CI - **Test fixture**: Clean `.clice/` before each test for hermetic index state ### Tests - **370-line** `index_query_tests.cpp`: unit tests for occurrence lookup, relation queries, content retrieval, removed bitmap filtering - **282-line** `test_index.py`: E2E integration tests for GoToDefinition, FindReferences, CallHierarchy (prepare/incoming/outgoing), TypeHierarchy (prepare/supertypes/subtypes), WorkspaceSymbol - Updated existing MergedIndex and ProjectIndex tests for new schema fields ## Test plan - [x] 414 C++ unit tests pass (including new IndexQuery, MergedIndex, ProjectIndex tests) - [x] 69 Python integration tests pass (including 10 new index feature tests) - [x] CI green on Linux, macOS, Windows - [ ] Manual smoke test with VSCode extension --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
95 lines
3.0 KiB
C++
95 lines
3.0 KiB
C++
#include "server/config.h"
|
|
|
|
#include <algorithm>
|
|
#include <thread>
|
|
|
|
#include "eventide/serde/toml.h"
|
|
#include "support/filesystem.h"
|
|
#include "support/logging.h"
|
|
|
|
namespace clice {
|
|
|
|
/// Replace all occurrences of ${workspace} with the workspace root.
|
|
static void substitute_workspace(std::string& value, const std::string& workspace_root) {
|
|
constexpr std::string_view placeholder = "${workspace}";
|
|
std::string::size_type pos = 0;
|
|
while((pos = value.find(placeholder, pos)) != std::string::npos) {
|
|
value.replace(pos, placeholder.size(), workspace_root);
|
|
pos += workspace_root.size();
|
|
}
|
|
}
|
|
|
|
void CliceConfig::apply_defaults(const std::string& workspace_root) {
|
|
auto cpu_count = std::thread::hardware_concurrency();
|
|
if(cpu_count == 0)
|
|
cpu_count = 4;
|
|
|
|
if(stateful_worker_count == 0) {
|
|
stateful_worker_count = std::max(1u, cpu_count / 4);
|
|
}
|
|
if(stateless_worker_count == 0) {
|
|
stateless_worker_count = std::max(1u, cpu_count / 4);
|
|
}
|
|
if(worker_memory_limit == 0) {
|
|
worker_memory_limit = 4ULL * 1024 * 1024 * 1024; // 4GB default
|
|
}
|
|
if(cache_dir.empty() && !workspace_root.empty()) {
|
|
cache_dir = path::join(workspace_root, ".clice");
|
|
}
|
|
|
|
if(index_dir.empty() && !cache_dir.empty()) {
|
|
index_dir = path::join(cache_dir, "index");
|
|
}
|
|
|
|
// Apply variable substitution to string fields
|
|
substitute_workspace(compile_commands_path, workspace_root);
|
|
substitute_workspace(cache_dir, workspace_root);
|
|
substitute_workspace(index_dir, workspace_root);
|
|
}
|
|
|
|
std::optional<CliceConfig> CliceConfig::load(const std::string& path,
|
|
const std::string& workspace_root) {
|
|
auto content = fs::read(path);
|
|
if(!content) {
|
|
return std::nullopt;
|
|
}
|
|
|
|
auto result = eventide::serde::toml::parse<CliceConfig>(*content);
|
|
if(!result) {
|
|
LOG_WARN("Failed to parse config file {}", path);
|
|
return std::nullopt;
|
|
}
|
|
|
|
auto config = std::move(*result);
|
|
config.apply_defaults(workspace_root);
|
|
|
|
LOG_INFO("Loaded config from {}", path);
|
|
return config;
|
|
}
|
|
|
|
CliceConfig CliceConfig::load_from_workspace(const std::string& workspace_root) {
|
|
if(!workspace_root.empty()) {
|
|
// Try standard config file locations
|
|
for(auto* name: {"clice.toml", ".clice/config.toml"}) {
|
|
auto config_path = path::join(workspace_root, name);
|
|
if(llvm::sys::fs::exists(config_path)) {
|
|
auto config = load(config_path, workspace_root);
|
|
if(config)
|
|
return std::move(*config);
|
|
}
|
|
}
|
|
}
|
|
|
|
// No config file found; use defaults
|
|
CliceConfig config;
|
|
config.apply_defaults(workspace_root);
|
|
LOG_INFO(
|
|
"No clice.toml found, using default configuration " "(stateful={}, stateless={}, memory_limit={}MB)",
|
|
config.stateful_worker_count,
|
|
config.stateless_worker_count,
|
|
config.worker_memory_limit / (1024 * 1024));
|
|
return config;
|
|
}
|
|
|
|
} // namespace clice
|