Commit Graph

322 Commits

Author SHA1 Message Date
ykiko
94bc872cdb feat: add LSP trace recording and smoke test replay (#383)
## Summary

Add LSP trace recording and replay-based smoke testing infrastructure.

### clice changes (`src/clice.cc`)
- Add `--log-level` CLI option with validation (rejects unknown levels
instead of silently defaulting to off)
- Add `--record <path>` CLI option that wraps the transport with
`RecordingTransport` to capture client→server messages as timestamped
JSONL
- Works in both pipe and socket modes
- Fix exit code: `loop.run()` returns non-zero after `uv_stop()`,
explicitly return 0 after clean shutdown

### Compile logging (`src/compile/compilation.cpp`)
- Print compile commands at debug log level

### Replay script (`tests/replay.py`)
- Timestamp-based pacing: sleeps between messages based on recorded
intervals, faithful to original editor session
- Automatic workspace path rewriting: infers repo root from script
location, rewrites absolute paths in trace so CI replay works without
extra arguments
- Handles server→client requests (workDoneProgress/create,
registerCapability, etc.) with default responses
- Waits for all pending responses before sending shutdown/exit
- Detects server exit mid-replay and fails pending futures immediately
instead of hanging
- Reports PASS/FAIL/SKIP with stderr tail on failure

### CI & config
- Add `smoke-test` pixi task and CI workflow step (runs after
integration tests)
- `.gitattributes`: mark `tests/smoke/*.jsonl` as `linguist-generated
binary` to suppress diffs
- Add sample trace file `tests/smoke/session.jsonl`

### VSCode extension
- Add restart command (`clice.restart`)
- Support `CLICE_MODE` env var to override mode setting (for debug
launch configs)
- Split launch configs into socket/pipe variants with
`--disable-extensions`

## Test plan
- [x] `python tests/replay.py tests/smoke/session.jsonl --clice
./build/RelWithDebInfo/bin/clice` passes locally
- [ ] CI smoke test passes on Linux/macOS/Windows

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 14:21:28 +08:00
ykiko
e43bb14998 feat: implement index system with LSP query handlers (#382)
## Summary

Implement the complete index system for cross-file LSP features. This
adds persistent two-tier indexing (ProjectIndex + per-file MergedIndex
shards), background indexing triggered on idle, and index-based query
handlers for major LSP requests.

### Index Data Layer (`src/index/`)

- **TUIndex**: Add binary serialization/deserialization via FlatBuffers,
enabling IPC between stateless worker and master server
- **ProjectIndex**: Add symbol name/kind storage, `PathPool` path
normalization (backslash -> forward slash), and binary persistence
- **MergedIndex**: Add `content` field to store file content for
reliable offset<->position mapping; add `removed` bitmap for garbage
collection of deleted entries; filter removed IDs in `lookup()` queries
- **schema.fbs**: Add TUIndex tables, `Symbol.name` field,
`MergedIndex.removed` bitmap and `MergedIndex.content` string

### Server (`src/server/`)

- **Background indexing**: Idle-triggered coroutine dequeues files from
CDB, dispatches `IndexParams` to stateless workers, merges returned
`TUIndex` into ProjectIndex/MergedIndex, and persists to `.clice/index/`
- **Index persistence**: `save_index()` / `load_index()` for startup
restoration; only rewrites shards flagged `need_rewrite()`
- **LSP handlers**:
- `textDocument/definition` -- index-first lookup with stateful worker
fallback
  - `textDocument/references` -- cross-file reference query via index
- `callHierarchy/prepare`, `incomingCalls`, `outgoingCalls` --
Caller/Callee relation traversal
- `typeHierarchy/prepare`, `supertypes`, `subtypes` -- Base/Derived
relation traversal
- `workspace/symbol` -- case-insensitive substring search over
ProjectIndex symbols
- **Stateless worker**: Add `Index` request handler that builds
`TUIndex` from compiled AST and returns serialized data
- **Config**: Add `enable_indexing` (default true) and `idle_timeout_ms`
(default 3000ms)

### Fixes and Cross-platform

- **ElaboratedType handling** in `decl_of()` for correct Base/Derived
relation emission
- **Windows path normalization** in `PathPool::intern()` and
`ProjectIndex::from()` (backslash -> forward slash)
- **`.gitattributes`**: Force LF in `tests/data/**` to prevent CRLF
byte-offset mismatches on Windows CI
- **Test fixture**: Clean `.clice/` before each test for hermetic index
state

### Tests

- **370-line** `index_query_tests.cpp`: unit tests for occurrence
lookup, relation queries, content retrieval, removed bitmap filtering
- **282-line** `test_index.py`: E2E integration tests for
GoToDefinition, FindReferences, CallHierarchy
(prepare/incoming/outgoing), TypeHierarchy
(prepare/supertypes/subtypes), WorkspaceSymbol
- Updated existing MergedIndex and ProjectIndex tests for new schema
fields

## Test plan

- [x] 414 C++ unit tests pass (including new IndexQuery, MergedIndex,
ProjectIndex tests)
- [x] 69 Python integration tests pass (including 10 new index feature
tests)
- [x] CI green on Linux, macOS, Windows
- [ ] Manual smoke test with VSCode extension

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 00:20:41 +08:00
ykiko
21a969af27 feat: integrate PCH into MasterServer build drain (#381)
## Summary
- Add `ensure_pch()` helper to MasterServer that builds/reuses
precompiled headers via stateless workers, with preamble hash-based
staleness detection (xxh3_64bits)
- Fix `BuildPCHParams` to carry `preamble_bound` so the stateless worker
truncates content at the preamble boundary (fixes redefinition errors
when PCH included full file)
- Wire PCH into both `run_build_drain` (stateful compile path) and
`forward_stateless` (completion/signatureHelp path)
- Add PCH state cleanup on `didClose` and hash invalidation on `didSave`

## Test plan
- [x] 398 unit tests pass (including 6 new PCH tests: PreambleHash x3,
PCHWorker x2, BuildPCHRequest assertion)
- [x] 5 new integration tests pass (`test_pch.py`: diagnostics on open,
body edit recompile, no-include file, hover with PCH, completion with
PCH)
- [x] 21 existing integration tests pass unchanged
- [x] Build succeeds with 0 errors

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Precompiled header (PCH) caching to speed compilations and reduce edit
latency
* Automatic attachment of cached PCH to compile requests, improving
hover and completion responsiveness
* Module-aware completions expanded to include available module
artifacts from other files

* **Bug Fixes**
* PCH cache cleared on file close; saving now triggers broader PCH
invalidation to prevent stale PCH use

* **Tests**
* Added unit and integration tests exercising PCH build, reuse, and
editor interactions
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-01 18:33:33 +08:00
ykiko
0a891d8b4a refactor(tests): use Tester fixture, normalize helpers, add index tests (#377)
## Summary

- **Use `Tester` as fixture base** for all test suites that need
compilation, replacing `TesterFixture` and removing redundant
`tester.clear()` calls (eventide zest now creates fresh instances per
TEST_CASE)
- **Remove local `Tester` variables** in `compilation_tests`,
`template_resolver_tests`, `selection_tests` — use inherited fixture
members directly
- **Normalize helper naming**: `expect_xxx` → `EXPECT_XXX`,
`go_to_definition` → `GO_TO_DEFINITION` for consistency
- **Extract shared `test/cdb_helper.h`**: deduplicate `CDBEntry`,
`json_escape`, `build_cdb_json` from `dependency_graph_tests` and
`compile_graph_integration_tests`
- **Add new test files/cases**: `project_index_tests.cpp`, expanded
`tu_index_tests`, `merged_index_tests`, `compilation_tests`

## Test plan

- [x] All existing unit tests pass
- [x] New index tests (TUIndex, MergedIndex, ProjectIndex) pass
- [x] Compilation tests (PCH, PCM, stop) pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Tests**
* Standardized test fixtures and helper naming, moved suites to a shared
fixture, and unified in-memory VFS and compile flows.
* Added broad new coverage: indexing, project indexing, compilation/PCH,
diagnostics, semantic features, and many targeted unit cases.
* Introduced a small compile-database helper and improved driver-style
test compilation paths.

* **Chores**
* Consolidated and reorganized test utilities and tester APIs for easier
maintenance and reuse.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 10:29:49 +08:00
ykiko
6d3b6acc82 feat: initial CompileGraph integration into MasterServer (#376)
## Summary

Initial integration of `CompileGraph` (#375) into `MasterServer`,
enabling basic end-to-end C++20 module support: on-demand PCM building,
dependency-ordered compilation, cascade invalidation on save, and
diagnostic integration.

This is a **first-pass implementation** — the core pipeline works, but
there are known areas for follow-up:

- PCM files go to system temp dir instead of `.clice/cache/`; no disk
cleanup on invalidation
- `run_build_drain` scans imports itself rather than delegating fully to
CompileGraph
- No incremental/partial rebuild (full PCM rebuild on any change)
- Cycle detection is tested at unit level but integration-level coverage
is minimal

## Changes

### Module dependency compilation (`master_server.cpp`)

Before sending a file to the stateful worker, `run_build_drain` now:

1. Scans imports via `scan_precise()` to discover module dependencies
2. Compiles each dep through `compile_graph->compile()`, which
recursively builds transitive PCMs
3. Handles implementation units — `module M;` implicitly needs the
interface PCM
4. Passes all built PCMs to the stateful worker, excluding the file's
own PCM
5. Skips compile on dep failure and resets `build_running` /
`drain_scheduled`
6. Re-lookups iterators after `co_await` to avoid use-after-invalidation

### Cascade invalidation (`didSave` / `didClose`)

- `didSave`: calls `compile_graph->update()` to mark transitive
dependents dirty, removes stale PCM paths, schedules rebuilds for open
dirtied files
- `didClose`: cancels in-flight compilations for the closed file

### Other fixes in this PR

- Debounce timers switched to `shared_ptr` to prevent use-after-free
when `didClose` destroys the timer mid-wait
- `fill_compile_args` returns `bool`; callers handle empty CDB
gracefully
- Adapt all `PositionMapper` call sites to the new `optional` return API
from eventide

## Test plan

- [x] 25 C++ unit tests for CompileGraph (cycles, partial failure,
cancel, update, empty graph)
- [x] 24 C++ integration tests with real clang PCM compilation
- [x] 3 worker-level module tests (BuildPCM, PCM-dependent compile,
multi-module)
- [x] 26 Python LSP integration tests (single module through circular
deps, hover, error diagnostics)
- [x] 371 unit tests + 54 integration tests pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 20:05:58 +08:00
ykiko
7ed558c1e7 feat: add CompileGraph for pull-based module dependency compilation (#375)
## Summary

Add `CompileGraph`, a pull-based async scheduler for C++20 module
compilation. When a file is compiled that imports modules, the graph
automatically resolves, builds, and caches PCM dependencies in the
correct order before the main compile proceeds.

## Design

### Data model

Each compilation unit (`CompileUnit`) tracks:
- `dependencies` / `dependents` — forward and reverse dependency edges
- `dirty` / `compiling` — current state flags
- `generation` — monotonic counter incremented by `update()`, used for
ABA-safe stale detection
- `source` + `completion` — cancellation token source and completion
event for cooperative async

### `compile(path_id)` — pull-based compilation

Lazily resolves dependencies (via `resolve_fn`) on first access, then
recursively compiles all transitive deps before dispatching the unit
itself:

- **Concurrent**: sibling deps compiled in parallel via `when_all`
- **Dedup**: diamond dependencies (A->B->D, A->C->D) — the second branch
waits on the first via `completion.wait()` instead of re-compiling
- **Cycle detection**: per-branch `ancestors` set (passed by value)
catches direct cycles; `has_wait_cycle()` BFS catches cross-branch
cycles (e.g. `1->{2,3}, 2->3, 3->2`) that would deadlock at
`completion.wait()`
- **Cancellation**: all `co_await` wrapped with `with_token()`, so
`update()` can cancel in-flight compilations immediately
- **Generation check**: captures generation counter before `co_await`;
if `update()` bumped it during dispatch, the result is discarded (unit
stays dirty)

### `update(path_id)` — cascade invalidation

BFS along `dependents` edges to mark the entire reverse-transitive
closure as dirty. For the source node, clears `resolved` and dependency
edges so they are re-scanned on next compile. Cancels any in-flight
compilations via `source->cancel()`.

## Test plan

22 unit tests covering:
- [x] No deps, single dep, chain, diamond (compile ordering + dedup)
- [x] Update invalidation, cascade through chains and diamonds
- [x] Re-resolution after update (deps can change)
- [x] Stale back-edge cleanup
- [x] Direct cycle detection (A->B->A)
- [x] Cross-branch cycle detection (when_all deadlock case)
- [x] Self-loop
- [x] Dispatch failure propagation
- [x] cancel_all + recompile
- [x] Update during in-flight compile (cancellation + generation check)
- [x] CI green on Linux, macOS, Windows

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 14:38:15 +08:00
ykiko
a536865fca feat: add scan_module_decl() fallback for conditional module declarations (#373)
## Summary

- Add `scan_module_decl()` — a lightweight preprocessor-based fallback
that resolves module declarations inside `#if`/`#ifdef` conditionals.
When `scan()` detects `need_preprocess=true`, this function runs clang's
preprocessor to evaluate conditions and extract the actual module name.
It stops lexing as soon as the module declaration is found, making it
much cheaper than `scan_precise()`.
- Integrate the fallback into `scan_dependency_graph()` for wave 0
source files, so conditional module declarations (e.g. `#ifdef
USE_MODULES / export module M; / #endif`) are correctly registered in
the dependency graph.
- Add comprehensive test cases covering all C++20 module declaration
forms from cppreference, including `scan_module_decl()` tests for
conditional resolution and `scan_precise()` tests for module import
semantics.

## Test plan

- [x] All 310 unit tests pass (0 failures, 9 skipped)
- [x] `scan()` tests cover: primary interface, implementation, dotted
names, partitions, GMF, conditional module declarations, private module
fragment
- [x] `scan_module_decl()` tests cover: basic, conditional with `-D`,
conditional with `#if` expression, GMF with conditional, implementation
unit, dotted name, partition, no-module file
- [x] `scan_precise()` tests cover: named import, multiple imports,
dotted import, partition import, export-import, export-import partition,
implementation import, GMF with import, mixed includes/imports,
no-module file

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Improved detection of module declarations hidden by conditional
compilation via a lightweight fallback scan. Resolved module vs.
interface classification is cached to avoid repeated work and is used
consistently in dependency mapping.
* Better handling and classification of module imports, partitions, and
global-fragment includes when building module relationships.

* **Tests**
* Added comprehensive unit tests covering module declaration extraction,
fallback resolution under preprocessor guards, imports, partitions,
includes, and macro-driven cases.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 22:27:49 +08:00
ykiko
f8a39147a7 feat: add include resolver, dependency graph, BFS scanner (#368)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 17:40:29 +08:00
ykiko
46ba1e4db6 refactor: simplify CompilationDatabase, extract ArgumentParser, remove pimpl (#371)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 13:15:49 +08:00
ykiko
498c975042 feat: add SearchConfig, ToolchainProvider, PathPool and related tests (#370)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 21:27:18 +08:00
ykiko
848065265c refactor: move resource_dir to CompilationDatabase, rename test dirs (#369)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 10:40:20 +08:00
ykiko
f7a8d104ce refactor: move command files to src/command/, remove scan_fuzzy (#366)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 23:54:32 +08:00
ykiko
020c2cb3cc feat: implement multi-process LSP server architecture (#364)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 23:37:08 +08:00
ykiko
73afcfbb58 refactor: introduce syntax/scan module with DependencyDirectivesGetter (#357)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 21:45:31 +08:00
ykiko
ce2f355988 refactor: introduce eventide and rewrite server architecture (#355) 2026-03-01 18:17:02 +08:00
ykiko
c0ffd2369b refactor: unify the CompilationUnitRef usage (#346) 2026-01-12 00:21:35 +08:00
Myriad-Dreamin
d6733dd43d feat: replace compile_commands_dirs with compile_commands_paths (#343) 2026-01-10 23:23:32 +08:00
Myriad-Dreamin
53689f2256 fix: prevent bad argument pop back when querying toolchain (#342) 2026-01-10 22:43:47 +08:00
Myriad-Dreamin
f30f68f573 fix: compiling C source file without -std=c++20 (#334) 2026-01-07 23:13:21 +08:00
ykiko
dd8f0dd90d refactor: diagnostic handling (#337) 2026-01-03 16:23:06 +08:00
Myriad-Dreamin
c6d87cccf3 fix: stuck caused by Network:on_read (#333) 2025-12-29 22:58:47 +08:00
ykiko
7105e36803 chore: use pixi to manage the format tools and format the world (#330) 2025-12-28 19:36:06 +08:00
ykiko
cec13ec29b refactor: tests and format the world (#314) 2025-11-30 15:21:27 +08:00
ykiko
8aff090a08 refactor: incremental update for compilation database and introduce query toolchain (#311) 2025-11-23 18:43:36 +08:00
ykiko
9e1039f861 refactor: improve logging (#301) 2025-11-08 00:37:07 +08:00
ykiko
336ca639f0 refactor: CompilationDatabase and scan (#286) 2025-11-05 23:01:28 +08:00
ykiko
39ec9bf7c5 fix: reset clang dependency output options (#293) 2025-11-02 23:25:39 +08:00
ykiko
397eb71dad build: update llvm to 21.1.4 (#292) 2025-11-02 22:23:11 +08:00
Myriad-Dreamin
3b1e379408 Fix warnings in Compiler (#290) 2025-10-31 20:52:16 +08:00
Myriad-Dreamin
8b998e658c [Fix] Use server's encoding kind (#289) 2025-10-31 20:51:11 +08:00
Myriad-Dreamin
9806e45fa3 [Feature] Enable clang-tidy (#200)
Co-authored-by: star9029 <hengxings783@gmail.com>
2025-10-31 20:50:07 +08:00
ykiko
dffa884e5a Store indices to disk (#279) 2025-10-19 21:53:57 +08:00
qingfengzl
eda7f03a37 Fix logging color (#276) 2025-10-10 22:23:09 +08:00
ykiko
9674328c18 Add ProjectIndex and basic symbol lookup implementation (#274) 2025-10-08 16:39:50 +08:00
ykiko
4c63c52487 Serialize index to binary (#273) 2025-10-07 18:21:44 +08:00
ykiko
b705560557 Introduce flatbuffer (#272)
Co-authored-by: star9029 <hengxings783@gmail.com>
2025-10-06 13:10:29 +08:00
ykiko
49caa2d9ba FileIndex and MergedIndex for clice (#271) 2025-10-05 22:46:32 +08:00
ykiko
4a184a26ce Fix unit_test fail doesn't result in error in CI (#270) 2025-10-03 14:50:38 +08:00
ykiko
9569988509 Fix query driver on windows (#269) 2025-10-03 03:26:21 +08:00
ykiko
7cd20c8565 Add remove and append options for CompilationDatabase (#268) 2025-10-02 18:00:01 +08:00
qingfengzl
2b350400bd Fix unit test of driver query (#267) 2025-09-27 11:13:39 +08:00
qingfengzl
e5348bd305 Fix glob pattern (#263) 2025-09-20 23:10:31 +08:00
ykiko
bba3ee3174 Remove config option and read config file from ${workspace}/clice.toml (#258) 2025-09-14 22:42:02 +08:00
ykiko
e8f5f6ba64 Allow log output to a file (#252)
Co-authored-by: star9029 <hengxings783@gmail.com>
2025-09-14 16:45:07 +08:00
zaragoza
d502eee53f Fix query driver on Windows (#255)
Co-authored-by: zaragoza-xu <xuyiyan920@gmail.com>
2025-09-11 00:10:54 +08:00
ykiko
f65876903c Improve integration test (#246) 2025-09-09 02:03:59 +08:00
ykiko
3cc3bae441 Format and simplify intergration test (#245) 2025-09-07 23:27:39 +08:00
Myriad-Dreamin
5e273415ee Implement isRegisteredTidyCheck and isFastTidyCheck (#204) 2025-09-07 14:22:52 +08:00
zaragoza
5786d852f2 Record the time of handling request and notification (#236)
Co-authored-by: zaragoza-xu <xuyiyan920@gmail.com>
2025-09-06 16:29:29 +08:00
Myriad-Dreamin
658c33a01f Init clang tidy (#195) 2025-09-06 16:23:20 +08:00