8f714c3b4ac599e3c181a5577dba4f3b43ba925d
22 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
ykiko
|
4d8c335c0d |
fix: re-lookup session after co_await to avoid invalidated iterator
The sessions DenseMap iterator may be invalidated during co_await (other coroutines can modify the map). Re-lookup by path_id after the await completes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
ykiko
|
13527b7084 |
feat(feature): preserve PCH document links and add #embed/#has_embed support
PCH compilation now serializes document links and stores them in PCHState. The master server merges PCH links with main-file links on DocumentLink requests, fixing missing links for includes inside the preamble. Also adds document link support for #embed and __has_embed directives. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
ykiko
|
8b3e3a9595 |
refactor(tests): reorganize integration tests into domain-based subdirectories (#409)
## Summary - Extract shared test utilities into `tests/integration/utils/` (client, workspace, assertions, wait, cache) - Migrate 12 test files into categorized subdirectories: `lifecycle/`, `compilation/`, `features/`, `modules/`, `extensions/`, `stress/` - Merge `test_include_completion.py` + `test_import_completion.py` → `features/test_completion.py` - Remove stale directory-tree comments and section divider comments ## Test plan - [x] `pytest --collect-only` collects all 113 tests - [x] All test module imports verified - [x] `pixi run format` applied 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Reorganized integration suites: added new feature tests (completion, server) and removed older duplicated modules. * Centralized shared test utilities and assertion helpers for diagnostics, workspace operations, waiting/synchronization, and cache inspection. * **Chores / Refactor** * Standardized test client lifecycle and helper usage across suites for more reliable test flows. * Improved server session lifecycle handling for more predictable document/session resets during tests. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
ykiko
|
2bbdf6c02b |
refactor(command): split CompilationContext into ResolvedFlags → CompileCommand → to_argv() (#408)
## Summary
- Replace flat `CompilationContext { directory, arguments }` with a
three-layer abstraction: `ResolvedFlags` (file-independent flags) →
`CompileCommand` (+ source file) → `to_argv()` (full argv on demand)
- `ResolvedFlags.flags` never contains source file path or
`-main-file-name`, making it directly usable as a clean cache key input
(e.g. PCH sharing across files with identical preambles)
- `to_argv()` handles `-main-file-name` insertion for cc1 mode
automatically — consumers no longer need to search/replace in the
argument list
- Eliminates the pollute-then-clean anti-pattern in `lookup()` and the
manual source-file replacement in `fill_header_context_args()`
## Test plan
- [x] `pixi run format` — no changes
- [x] `pixi run unit-test` — 481 passed
- [x] `pixi run integration-test` — 113 passed
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Unified compile-command handling across the server and tools for more
consistent argument and flag behavior (driver vs frontend modes).
* **New Features**
* Added an LRU-backed in-memory cache to improve performance and
eviction control.
* **Chores**
* Added an option to control injection of resource-directory flags
(enabled by default).
* **Tests**
* Updated unit and integration tests to adopt the new command
representation and verify cache behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
||
|
ykiko
|
9c9e6b0bcb |
refactor: introduce Workspace/Session state model and clarify component responsibilities (#406)
## Summary Introduces a two-layer state model that cleanly separates disk-based project state from per-open-file editing state, and redistributes responsibilities across server components so each has a single, clear role. ## New types **Workspace** — all persistent, project-wide shared state: - CompilationDatabase, PathPool, DependencyGraph, CompileGraph - path_to_module mapping, PCH cache, PCM cache, PCM paths - ProjectIndex, MergedIndex shards - CliceConfig - Methods: on_file_saved(), on_file_closed(), load/save/cleanup_cache(), build_module_map(), fill_pcm_deps(), cancel_all() **Session** — volatile per-open-file editing state: - text, version, generation, ast_dirty - pch_ref (references Workspace.pch_cache), ast_deps, header_context - file_index (OpenFileIndex for unsaved buffer) - path_id member for self-identification ## Component responsibilities after refactor | Component | Role | Owns state? | |-----------|------|-------------| | **Workspace** | Disk truth + shared caches | Yes (all project state) | | **Session** | One open file editing state | Yes (per-file only) | | **Compiler** | Compilation pipeline, worker communication | No (references only) | | **Indexer** | Index queries + background indexing scheduling | Scheduling state only | | **MasterServer** | LSP protocol dispatch + lifecycle coordination | sessions map | ## What moved where **Into Workspace** (from Compiler/MasterServer): - PCH/PCM cache management (load_cache, save_cache, cleanup_cache) - Module map building (build_module_map, fill_pcm_deps) - File lifecycle hooks (on_file_saved, on_file_closed) - cancel_all, OpenFileIndex/MergedIndexShard type definitions **Into Session** (from Compiler documents map): - Document text, version, generation, ast_dirty - PCH reference, dependency snapshot, header context **Into Indexer** (from MasterServer): - Background indexing queue, scheduling state, idle timer - schedule(), enqueue(), run_background_indexing() **Into syntax/completion.h** (from Compiler): - detect_completion_context() — pure text parsing - complete_module_import() — prefix match on module names - complete_include_path() — directory listing against search paths **Inlined into MasterServer** (from Compiler): - didOpen/didChange/didClose/didSave handlers - switchContext/currentContext - publish_diagnostics/clear_diagnostics **Deleted from Compiler** (9 methods): - open_document, apply_changes, close_document, on_save - switch_context, get_active_context, invalidate_host_contexts - on_file_closed, on_file_saved, complete_include, complete_import ## Tests - 481 tests pass (465 existing + 16 new completion tests) - New: tests/unit/syntax/completion_tests.cpp ## Diff stats 15 files changed, +1857, -1555 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Enhanced completion support for include paths and module imports with improved context detection. * Added background indexing system for automatic project symbol indexing. * **Bug Fixes** * Improved reliability of document change tracking and compilation state management. * Better handling of header file compilation contexts. * **Tests** * Added unit tests for completion context detection and module/include path completion. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
ykiko
|
bb0b160a28 |
refactor(server): extract Indexer and Compiler from MasterServer (#403)
## Summary - **Extract `Indexer` class** — owns all index state (ProjectIndex, MergedIndex shards, OpenFileIndex) and query methods (definition, references, call/type hierarchy, workspace symbol search) - **Extract `Compiler` class** — owns document state, PCH/PCM cache, compile argument resolution, header context, `ensure_compiled`, and worker forwarding - **MasterServer is now a pure LSP handler registration layer** (~700 lines, down from ~3200) - **`MergedIndexShard`** wraps `index::MergedIndex` with a lazily-cached PositionMapper; `OpenFileIndex` gains matching `find_occurrence()`/`find_relations()` APIs — callers get pre-converted LSP ranges directly - **Indexer returns typed values** (`vector<Location>`, `vector<CallHierarchyIncomingCall>`, etc.) instead of pre-serialized JSON, fixing the references handler from JSON string surgery to simple vector concatenation - **Fix**: duplicate `workspace/symbol` loop in the original code ## Test plan - [x] 465 unit tests pass - [x] 113 integration tests pass - [x] 2/2 smoke tests pass - [x] `clang-format` applied 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Server-side C++ compilation orchestration (module & precompiled header builds) with LSP-integrated document handling. * **Improvements** * Deterministic, persistent, dependency-aware caching to avoid redundant rebuilds and speed up incremental work. * Better cross-file indexing and navigation, improved diagnostics and more reliable include/import-aware completions. * **Tests** * Unit tests updated to the unified worker query/build request shapes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
ykiko
|
ada202e489 |
feat(index): piggyback indexing on PCH/PCM builds and open-file compiles (#402)
## Summary Piggyback index construction onto existing compilation steps, eliminating redundant recompilation in background indexing: - **`TUIndex::build` gains `interested_only` parameter**: `true` traverses only the main file's top-level decls; `false` (default) traverses the full AST - **PCH build indexes preamble headers**: stateless worker calls `TUIndex::build(unit)` (full traversal) after successful `BuildPCH`, clears `main_file_index`, serializes and sends back; master merges into MergedIndex - **PCM build indexes module interface**: stateless worker calls `TUIndex::build(unit, true)` after successful `BuildPCM`; master merges into MergedIndex - **Open-file compile indexes main file**: stateful worker calls `TUIndex::build(unit, true)` after successful `Compile`, serialized in `CompileResult` - **New `OpenFileIndex` in-memory structure**: master holds `FileIndex + SymbolTable + buffer text` per open file — not persisted to disk, not merged, discarded on close - **Dual-source query path**: `query_index_relations`, `lookup_symbol_at_position`, `find_symbol_definition_location`, all hierarchy handlers, and `workspace/symbol` check `OpenFileIndex` first (fresher), then fall back to `MergedIndex` (disk-indexed) - **Background indexing skips open files**: checked via `documents.count()`; on `didClose` the file is re-queued into `index_queue` - **`didSave` re-queues non-open dependents**: dirtied files from `compile_graph->update()` that are not open get pushed into `index_queue` for background re-indexing - **Extract `lookup_occurrence` helper**: binary search + forward scan picking the innermost (narrowest) match, replacing a broken `while/break/break` pattern - **Extract `find_symbol_info` helper**: consolidates 6 duplicated "search open file indices then ProjectIndex" lookups into one method - **`resolve_hierarchy_item` checks open file indices**: no longer limited to ProjectIndex only ## Test plan - [x] 465 unit tests pass - [x] 105 integration tests pass (including all `test_index` cases: GoToDefinition, FindReferences, CallHierarchy, TypeHierarchy, WorkspaceSymbol) - [x] Manual: open a file and immediately use GoToDefinition — should work without waiting for background indexing - [x] Manual: close a file and verify background indexing picks it up and produces a MergedIndex shard 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
ykiko
|
836f415e50 |
feat: header context protocol — queryContext, currentContext, switchContext (#398)
## Summary Add three LSP protocol extensions that allow users to manage compilation contexts for header files and source files with multiple CDB entries. ### Protocol extensions (`protocol.h`) | Command | Purpose | |---------|---------| | `clice/queryContext` | List all possible contexts for a file. Headers → host source files; sources → CDB entries. Paginated (10 per page, `offset` param). | | `clice/currentContext` | Query the active context override for a file (null if default). | | `clice/switchContext` | Set the active context, invalidate caches, trigger recompilation. | ### Header context resolution (`master_server.cpp`, `dependency_graph.cpp`) - `find_host_sources()`: BFS the reverse include graph to find source files that transitively include a header - `find_include_chain()`: BFS the forward include graph to find the shortest include chain from host to header - `resolve_header_context()`: walks the include chain, extracts content before each `#include` directive, concatenates with `#line` markers into a preamble file (hash-addressed under `.clice/header_context/`) - `fill_header_context_args()`: uses the host source's CDB entry, replaces source path with header path, injects `-include preamble.h` ### Compilation flow - Default: headers compile as standalone files (no context) - After `switchContext`: `fill_compile_args` checks `active_contexts` first → uses host's CDB entry + preamble injection - Fallback: if no CDB entry and no active context, auto-resolves via `resolve_header_context` - `#include` directive matching uses precise filename extraction from `"..."` / `<...>`, not substring matching ### Source file multiple contexts (`multi_context` workspace) - `queryContext` on a source file returns all CDB entries with distinguishing labels (extracted from `-D`, `-O`, `-std=` flags) ### Test data - `header_context/`: non-self-contained 3-level chain (`main.cpp` → `utils.h` → `inner.h`), `types.h` provides `Point` struct - `multi_context/`: single source with two CDB entries (`-DCONFIG_A`, `-DCONFIG_B`) ### Tests (9 integration tests) - queryContext returns host sources for headers - queryContext returns CDB entries for source files - currentContext defaults to null - switchContext sets active context, currentContext reflects it - Full flow: open → query → switch → hover works in non-self-contained header - Deep nested: switchContext + hover on `inner.h` (3 levels deep) - Multiple CDB entries: queryContext returns both CONFIG_A and CONFIG_B ## Test plan - [x] Unit tests: 465 passed - [x] Integration tests: 113 passed (9 new header context tests) - [x] Smoke test: 1/1 passed - [ ] Manual VSCode testing 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
ykiko
|
e239b0d32c |
feat: smart PCH rebuild, #include/import completion, rapid-edit robustness (#394)
## Summary ### Preamble completeness check - `is_preamble_complete()` in `scan.cpp`: checks whether `#include`/`import`/`export module` directives in the preamble region are syntactically complete (have closing `>`/`"`/`;`) - `ensure_pch` defers PCH rebuild when preamble is incomplete (user still typing), reuses old PCH instead of failing ### #include / import completion - Master intercepts completion requests in `#include "..."` / `#include <...>` / `import ...` contexts before forwarding to worker - `complete_include()`: searches include paths (from compile args via `SearchConfig`) using `DirListingCache`, supports quoted/angled/multi-level paths - `complete_import()`: filters `path_to_module` map by prefix - Word boundary checks prevent false matches (e.g. `important` not treated as `import`) ### Detached compile task (rapid-edit fix) - Compile operations (`ensure_deps` + `send_stateful` + `publish_diagnostics`) run as detached tasks via `loop.schedule()`, independent of the LSP request coroutine chain - LSP `$/cancelRequest` can no longer kill in-flight compilations — previously, cancellation would destroy the `ensure_compiled` coroutine frame, leaving `doc.compiling` permanently set and hanging all subsequent requests - `CompileGuard` RAII ensures `doc.compiling` is always cleaned up even if the detached task fails - Stale feature requests (where `ast_dirty` became true after compile finished) are dropped before forwarding to worker ### Other fixes - `signal(SIGPIPE, SIG_IGN)` on POSIX: prevents server crash when LSP client disconnects mid-write - `CompilationUnitRef::file_path()` / `deps()`: null-check `FileEntryRef` to prevent segfault on invalid FileID - `stateless_worker.cpp`: log BuildPCH diagnostic errors for debuggability - Default worker counts changed to 2 stateful + 3 stateless - `logging_dir` default changed to `.clice/logs` in config ### Tests - 19 unit tests for `is_preamble_complete` (incomplete `#include`, `import`, `export module`, mixed cases) - Integration tests: `test_include_completion.py` (5 tests), `test_import_completion.py` (4 tests), `test_rapid_edit.py` (2 tests), `test_pch.py` (4 new tests) - Smoke test: `rapid_edit.jsonl` — recorded VSCode session with 40 rapid edits + 61 cancel requests ## Test plan - [x] Unit tests: 463 passed - [x] Integration tests: 104 passed - [x] Smoke test (rapid_edit.jsonl): PASS - [x] Manual VSCode testing with `#include <iostream>` project 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Myriad-Dreamin
|
d04bc6f774 |
feat: register server capability correctly (#397)
## Summary - register workspace and text document capabilities through the structured LSP capability types - advertise completion, signature help, declaration, definition, implementation, type definition, and reference support more explicitly - add placeholder handlers for declaration, type definition, and implementation requests so the advertised capabilities have matching routes ## Testing - Not run (not requested) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added workspace folder support for improved project tracking. * Registered navigation handlers for type-definition, implementation, and declaration (currently return a “not supported yet” placeholder). * **Improvements** * Enhanced completion and signature help with explicit trigger characters and clearer capability declarations. * **Tests** * Relaxed capability assertions to recognize more nuanced enabled/disabled states. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
ykiko
|
b6886d222b |
feat: per-session file-based logging with crash capture (#393)
## Summary
Implement structured file-based logging with per-component separation
and crash stacktrace capture.
### Log output structure
```
.clice/logs/2026-04-05_10-30-00_<pid>/
master.log
SF-0.log
SF-1.log
SL-0.log
```
### Changes
**Logging infrastructure** (`logging.h`, `logging.cpp`)
- `file_logger()` creates a dual-sink logger (file + stderr), so logs go
to both the file and terminal
- Pre-checks log directory creation and file writability before
constructing spdlog sinks; falls back to existing stderr logger on
failure
- `install_crash_handler()` uses LLVM's `AddSignalHandler` +
`PrintStackTraceOnErrorSignal` to write crash stacktraces into the
component's log file (and also to stderr)
- Fix `LOG_MESSAGE` macro: wrap in `do { } while(0)` to prevent
dangling-else
- Fix typo: `file_loggger` → `file_logger`
**Config** (`config.h`, `config.cpp`)
- Add `logging_dir` field to `CliceConfig`, defaulting to
`<cache_dir>/logs/`
- Apply `${workspace}` variable substitution to `logging_dir`
**Master server** (`master_server.h`, `master_server.cpp`)
- After config loads, create a session directory named
`<timestamp>_<pid>` under `logging_dir` and switch master to file
logging
- Pass session log directory to worker pool
**Worker pool** (`worker_pool.h`, `worker_pool.cpp`)
- Pass `--worker-name` (e.g. `SF-0`, `SL-1`) and `--log-dir` to spawned
worker processes
- Add `log_dir` to `WorkerPoolOptions`
**Workers** (`stateful_worker.h/cpp`, `stateless_worker.h/cpp`)
- Accept `worker_name` and `log_dir` parameters; switch to file logging
when `log_dir` is provided
**CLI cleanup** (`clice.cc`)
- Remove `--stateful-worker-count`, `--stateless-worker-count` from CLI
(config-file only)
- Group internal worker args (`--worker-memory-limit`, `--worker-name`,
`--log-dir`) separately
**Docs** (`docs/clice.toml`)
- Fix `logging_dir` example: `.clice/logging` → `.clice/logs`
## Test plan
- [x] `pixi run cmake-build RelWithDebInfo` compiles successfully
- [ ] Verify log files created under `.clice/logs/<timestamp>_<pid>/`
- [ ] Verify each component writes to its own file
- [ ] Verify crash stacktrace appears in component log file
- [ ] Verify `logging_dir` override in `clice.toml` works
- [ ] Verify graceful fallback when log directory is not writable
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Session-specific logging directories (timestamped) and per-worker log
files
* CLI options to set worker name and log directory; general log level
control
* Configurable logging directory with default `<cache_dir>/logs/`
* **Bug Fixes**
* Fixed file-logging name/initialization issues; ensures directory
creation and deterministic filenames
* Added crash-handler support to append stack traces to logs
* **Documentation**
* Updated example config to use `${workspace}/.clice/logs`
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
ykiko
|
3838bedcbf |
feat: persistent PCH/PCM cache across sessions (#391)
## Summary
PCH and PCM artifacts are now cached to disk at
`.clice/cache/{pch/,pcm/}` with content-addressed filenames, so they
survive server restarts. Dependency metadata is persisted in
`cache.json` (using eventide serde) with a shared path table for
deduplication.
### Key changes
- **protocol.h**: `output_path` field on `BuildPCHParams` /
`BuildPCMParams` so master specifies where workers write
- **stateless_worker.cpp**: Atomic write via `.tmp` + `fs::rename`;
`CompilationUnit` destroyed before rename to flush the file to disk;
fallback to temp file when `output_path` is empty (unit tests)
- **master_server.h**: `PCMState` struct, `pcm_states` map,
`load_cache()` / `save_cache()` / `cleanup_cache()` methods
- **master_server.cpp**: Cache lifecycle — load from `cache.json` on
startup, save after each PCH/PCM build and on exit; deterministic path
computation (`xxh3` preamble hash for PCH, module name + source path
hash for PCM); stale files (>7 days) cleaned on startup; `cache.json`
uses shared path table to avoid redundant storage of header paths across
entries
- **filesystem.h**: `fs::rename()` helper; `ThreadSafeFS` broadened to
match `.pch` extension instead of `preamble-` prefix
- **tests**: 11 new integration tests covering PCH/PCM persistence,
cross-session reuse, staleness detection, shared preamble dedup, and
restart survival; unit tests updated with `output_path`
### Naming scheme
- **PCH**: `.clice/cache/pch/<016x(xxh3(preamble))>.pch`
- **PCM**:
`.clice/cache/pcm/<module_name>-<016x(xxh3(source_path))>.pcm`
## Test plan
- [x] Unit tests — 448 passed
- [x] Integration tests — 92 passed (including 11 new persistent cache
tests)
- [x] Smoke tests — 1 passed
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
ykiko
|
31d9c609b6 |
fix: data race in stateful worker between Compile and DocumentUpdate (#389)
## Summary Fix two data races in the stateful worker that caused spurious "redefinition" errors during rapid edits, and remove a didChange workaround that is no longer needed after clice-io/eventide#95. ### stateful_worker.cpp **Compile handler**: move `params` → `doc` field copy **after** `strand.lock()`. Previously the copy happened before the lock, so a concurrent Compile request waiting on the strand could overwrite `doc.text` while `et::queue` was reading it on the thread pool: ``` T1: Compile A → doc.text = text_A → lock → et::queue reads doc.text T2: Compile B → doc.text = text_B → waits for strand (overwrites!) T3: et::queue sees text_B instead of text_A → PCH/text mismatch ``` **DocumentUpdate handler**: only mark `dirty`, stop modifying `doc.text`/`doc.version`. The event loop notification can fire while `et::queue` work is running on the thread pool — writing `doc.text` from one thread while reading it from another is a data race. ### master_server.cpp Remove the `{0,0}-{0,0}` range workaround for whole-document `didChange`. eventide's variant deserialization now correctly rejects `TextDocumentContentChangePartial` when the `range` field is absent (clice-io/eventide#95), so `TextDocumentContentChangeWholeDocument` is matched as intended. ### protocol.h Remove `text` field from `DocumentUpdateParams` — the worker no longer needs it since DocumentUpdate only sets the dirty flag. ### Integration tests (+312 lines) Extend test_staleness.py from 5 to 14 tests covering document lifecycle: - `didChange` body edit → recompilation with updated diagnostics - `didChange` preamble edit → PCH rebuild + clean recompilation - `didClose` + reopen → compiles fresh from disk - `didClose` → hover returns None - `didSave` header → dependent file recompiles - `didSave` module → CompileGraph dependents invalidated ## Test plan - [x] 422 unit tests pass (426 on CI with extra test suites) - [x] 14 integration tests pass locally - [x] Depends on clice-io/eventide#95 (merged) 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Smaller document-update notifications sent to background workers (only path and version). * **Bug Fixes** * Reduced races and unnecessary work between update and compile flows. * Prevented notifications from overwriting in-memory document text, improving state consistency. * Safer concurrent handling to avoid mid-request eviction of active documents. * **Tests** * Added integration tests for staleness, dependency propagation, and LSP lifecycle. * Updated unit tests to match revised update behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
ykiko
|
1dd94e54c0 |
feat: two-layer staleness tracking with concurrent compilation dedup (#386)
## Summary Replace the `didSave` sledgehammer (`pch_hashes.clear()` + mark-all-dirty) with precise per-file dependency tracking that avoids unnecessary recompilation. ### Two-layer staleness detection After each successful compilation, a `DepsSnapshot` is captured (interned path IDs + xxh3 content hashes + timestamp). On the next feature request, `deps_changed()` checks: - **Layer 1 (fast):** stat each dep, compare mtime against `build_at`. If mtime is older → skip. Uses strict `<` so same-second modifications fall through. - **Layer 2 (precise):** for files with newer mtime, re-hash content and compare. Catches touch-without-change (git checkout, backup restore) without false rebuilds. Special cases: files unreadable at build time (hash=0) always fall through to Layer 2; disappeared files are detected via stat failure. ### Consolidated PCH state Scatter of four maps (`pch_paths`, `pch_bounds`, `pch_hashes`, `pch_building`) → single `PCHState` struct with `path`, `bound`, `hash`, `deps`, `building` fields. `DepsSnapshot` and `SymbolInfo` moved out of `MasterServer` to namespace scope. ### Concurrent compilation dedup - **`ensure_compiled`:** `DocumentState::compiling` event prevents duplicate AST compilations. Waiters `co_await` the event and check `ast_dirty` after waking. When deps change is detected during an in-flight build, `generation` is bumped so the builder's generation check prevents it from incorrectly clearing `ast_dirty`. - **`ensure_pch`:** `PCHState::building` event deduplicates PCH builds. Waiters re-validate `preamble_hash` after waking to handle edits during the wait. The `bound==0` path waits for in-flight builds before erasing. Old PCH is deleted and path cleared before rebuild starts, so waiters never see a stale path on failure. ### `didSave` changes Removed the blanket `pch_hashes.clear()` + mark-all-dirty on save. Staleness is now detected lazily via `deps_changed()` at the next feature request. `didSave` still invalidates `CompileGraph` dependents for module deps. ### FIXME noted Rapid `didChange` edits (especially preamble changes) can cause the stateful worker to compile with stale/concatenated text. Root cause is in the worker, not in staleness tracking — noted as FIXME for a follow-up PR. ## Test plan 13 integration tests covering: - [x] Header mtime change → AST recompilation - [x] Preamble header change → PCH rebuild - [x] No change → fast path (cached AST reused) - [x] Touch without content change → Layer 2 hash skips recompile - [x] Header replaced with different content → detected - [x] Fix error in header → diagnostics clear - [x] Multiple files sharing header → each detects independently - [x] Transitive header change → detected through include chain - [x] didChange body edit → recompilation with updated diagnostics - [x] didClose + reopen → compiles new disk content - [x] didClose → hover returns None - [x] didSave header → dependent file recompiles - [x] didSave module → CompileGraph dependents invalidated 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
ykiko
|
e24eff6c16 |
refactor: pull-based compilation for document lifecycle (#385)
## Summary Replace the push-based compilation model with a pull-based (lazy) model where compilation is driven entirely by feature requests. ### Server core (`master_server.cpp/h`) - **Remove** `schedule_build()`, `run_build_drain()`, debounce timers, and `DocumentState` flags (`build_running`, `build_requested`, `drain_scheduled`) - **Remove** `debounce_ms` config field - `didOpen`/`didChange` only update `DocumentState` and mark `ast_dirty` — no compilation triggered - `didSave` marks dependent docs dirty via `CompileGraph::update()`, invalidates PCH hashes, marks **all** open documents `ast_dirty` (header saves), and queues background indexing - **Implement** `ensure_compiled(path_id)` — the pull-based entry point called by `forward_stateful()`/`forward_stateless()` before every feature request: 1. Fast-path if `!ast_dirty` 2. Compile C++20 module deps via `compile_graph->compile_deps()` 3. Build/reuse PCH via `ensure_pch()` (only attach on success) 4. Send `CompileParams` to stateful worker 5. Publish diagnostics, clear dirty, schedule indexing 6. Generation mismatch → return `false`, keep dirty for retry - `forward_stateless()` now also calls `compile_graph->compile_deps()` before stateless requests (completion/signatureHelp) - Move module-implementation-unit implicit dependency handling into `resolve_fn` (was duplicated in `run_build_drain` and `ensure_compiled`) ### CompileGraph (`compile_graph.cpp/h`) - **Add** `compile_deps(path_id)` — compiles all transitive module dependencies but NOT the file itself (used for plain .cpp files that `import` modules) - Unify `compile`/`compile_deps` via `compile_impl(path_id, ancestors, dispatch_self)` parameter - `compile_deps` compiles dependencies concurrently via `when_all` - Extract `finish()` lambda to deduplicate `compiling=false; completion->set()` cleanup across all exit paths - Use `std::ranges::remove` instead of legacy `std::remove` ### Test infrastructure (`conftest.py`) - `open_and_wait()` now sends a hover request to trigger `ensure_compiled()` (pull-based model requires a feature request to compile) - Fix URI handling: send percent-encoded URI on the wire, normalize for internal lookups, store diagnostics under both raw and normalized URI keys - Add `_normalize_uri()` helper using `urllib.parse.unquote` ### Integration tests - Update all tests for pull-based model: no more waiting on `didOpen` diagnostics - `_wait_for_index()` sends hover to trigger compilation before polling `workspace/symbol` - `test_hover_save_close` simplified — hover directly triggers compilation - `test_save_recompile` and `test_pch_*` wait for fresh diagnostics after hover-triggered recompilation ### Unit tests (`compile_graph_tests.cpp`) - Extract `compiled`/`graph` as TEST_SUITE members with `std::optional<CompileGraph>` - Extract `execute(callback)` helper to deduplicate event_loop boilerplate - Add 8 new `compile_deps` tests: no-deps, single dep, chain, diamond, failure, plain-cpp, concurrent dedup, resolve-once - Remove redundant `inline` on file-scope helpers ## Test plan - [x] Unit tests: 426 passed, 5 skipped - [x] Smoke tests: 1/1 passed - [x] Integration tests: 69 passed, 0 failed, no hangs 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
ykiko
|
e43bb14998 |
feat: implement index system with LSP query handlers (#382)
## Summary Implement the complete index system for cross-file LSP features. This adds persistent two-tier indexing (ProjectIndex + per-file MergedIndex shards), background indexing triggered on idle, and index-based query handlers for major LSP requests. ### Index Data Layer (`src/index/`) - **TUIndex**: Add binary serialization/deserialization via FlatBuffers, enabling IPC between stateless worker and master server - **ProjectIndex**: Add symbol name/kind storage, `PathPool` path normalization (backslash -> forward slash), and binary persistence - **MergedIndex**: Add `content` field to store file content for reliable offset<->position mapping; add `removed` bitmap for garbage collection of deleted entries; filter removed IDs in `lookup()` queries - **schema.fbs**: Add TUIndex tables, `Symbol.name` field, `MergedIndex.removed` bitmap and `MergedIndex.content` string ### Server (`src/server/`) - **Background indexing**: Idle-triggered coroutine dequeues files from CDB, dispatches `IndexParams` to stateless workers, merges returned `TUIndex` into ProjectIndex/MergedIndex, and persists to `.clice/index/` - **Index persistence**: `save_index()` / `load_index()` for startup restoration; only rewrites shards flagged `need_rewrite()` - **LSP handlers**: - `textDocument/definition` -- index-first lookup with stateful worker fallback - `textDocument/references` -- cross-file reference query via index - `callHierarchy/prepare`, `incomingCalls`, `outgoingCalls` -- Caller/Callee relation traversal - `typeHierarchy/prepare`, `supertypes`, `subtypes` -- Base/Derived relation traversal - `workspace/symbol` -- case-insensitive substring search over ProjectIndex symbols - **Stateless worker**: Add `Index` request handler that builds `TUIndex` from compiled AST and returns serialized data - **Config**: Add `enable_indexing` (default true) and `idle_timeout_ms` (default 3000ms) ### Fixes and Cross-platform - **ElaboratedType handling** in `decl_of()` for correct Base/Derived relation emission - **Windows path normalization** in `PathPool::intern()` and `ProjectIndex::from()` (backslash -> forward slash) - **`.gitattributes`**: Force LF in `tests/data/**` to prevent CRLF byte-offset mismatches on Windows CI - **Test fixture**: Clean `.clice/` before each test for hermetic index state ### Tests - **370-line** `index_query_tests.cpp`: unit tests for occurrence lookup, relation queries, content retrieval, removed bitmap filtering - **282-line** `test_index.py`: E2E integration tests for GoToDefinition, FindReferences, CallHierarchy (prepare/incoming/outgoing), TypeHierarchy (prepare/supertypes/subtypes), WorkspaceSymbol - Updated existing MergedIndex and ProjectIndex tests for new schema fields ## Test plan - [x] 414 C++ unit tests pass (including new IndexQuery, MergedIndex, ProjectIndex tests) - [x] 69 Python integration tests pass (including 10 new index feature tests) - [x] CI green on Linux, macOS, Windows - [ ] Manual smoke test with VSCode extension --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
ykiko
|
21a969af27 |
feat: integrate PCH into MasterServer build drain (#381)
## Summary - Add `ensure_pch()` helper to MasterServer that builds/reuses precompiled headers via stateless workers, with preamble hash-based staleness detection (xxh3_64bits) - Fix `BuildPCHParams` to carry `preamble_bound` so the stateless worker truncates content at the preamble boundary (fixes redefinition errors when PCH included full file) - Wire PCH into both `run_build_drain` (stateful compile path) and `forward_stateless` (completion/signatureHelp path) - Add PCH state cleanup on `didClose` and hash invalidation on `didSave` ## Test plan - [x] 398 unit tests pass (including 6 new PCH tests: PreambleHash x3, PCHWorker x2, BuildPCHRequest assertion) - [x] 5 new integration tests pass (`test_pch.py`: diagnostics on open, body edit recompile, no-include file, hover with PCH, completion with PCH) - [x] 21 existing integration tests pass unchanged - [x] Build succeeds with 0 errors 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Precompiled header (PCH) caching to speed compilations and reduce edit latency * Automatic attachment of cached PCH to compile requests, improving hover and completion responsiveness * Module-aware completions expanded to include available module artifacts from other files * **Bug Fixes** * PCH cache cleared on file close; saving now triggers broader PCH invalidation to prevent stale PCH use * **Tests** * Added unit and integration tests exercising PCH build, reuse, and editor interactions <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
ykiko
|
6d3b6acc82 |
feat: initial CompileGraph integration into MasterServer (#376)
## Summary Initial integration of `CompileGraph` (#375) into `MasterServer`, enabling basic end-to-end C++20 module support: on-demand PCM building, dependency-ordered compilation, cascade invalidation on save, and diagnostic integration. This is a **first-pass implementation** — the core pipeline works, but there are known areas for follow-up: - PCM files go to system temp dir instead of `.clice/cache/`; no disk cleanup on invalidation - `run_build_drain` scans imports itself rather than delegating fully to CompileGraph - No incremental/partial rebuild (full PCM rebuild on any change) - Cycle detection is tested at unit level but integration-level coverage is minimal ## Changes ### Module dependency compilation (`master_server.cpp`) Before sending a file to the stateful worker, `run_build_drain` now: 1. Scans imports via `scan_precise()` to discover module dependencies 2. Compiles each dep through `compile_graph->compile()`, which recursively builds transitive PCMs 3. Handles implementation units — `module M;` implicitly needs the interface PCM 4. Passes all built PCMs to the stateful worker, excluding the file's own PCM 5. Skips compile on dep failure and resets `build_running` / `drain_scheduled` 6. Re-lookups iterators after `co_await` to avoid use-after-invalidation ### Cascade invalidation (`didSave` / `didClose`) - `didSave`: calls `compile_graph->update()` to mark transitive dependents dirty, removes stale PCM paths, schedules rebuilds for open dirtied files - `didClose`: cancels in-flight compilations for the closed file ### Other fixes in this PR - Debounce timers switched to `shared_ptr` to prevent use-after-free when `didClose` destroys the timer mid-wait - `fill_compile_args` returns `bool`; callers handle empty CDB gracefully - Adapt all `PositionMapper` call sites to the new `optional` return API from eventide ## Test plan - [x] 25 C++ unit tests for CompileGraph (cycles, partial failure, cancel, update, empty graph) - [x] 24 C++ integration tests with real clang PCM compilation - [x] 3 worker-level module tests (BuildPCM, PCM-dependent compile, multi-module) - [x] 26 Python LSP integration tests (single module through circular deps, hover, error diagnostics) - [x] 371 unit tests + 54 integration tests pass 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
ykiko
|
f8a39147a7 |
feat: add include resolver, dependency graph, BFS scanner (#368)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
ykiko
|
46ba1e4db6 |
refactor: simplify CompilationDatabase, extract ArgumentParser, remove pimpl (#371)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
ykiko
|
848065265c |
refactor: move resource_dir to CompilationDatabase, rename test dirs (#369)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
ykiko
|
020c2cb3cc |
feat: implement multi-process LSP server architecture (#364)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |