feat: implement index system with LSP query handlers (#382)

## Summary

Implement the complete index system for cross-file LSP features. This
adds persistent two-tier indexing (ProjectIndex + per-file MergedIndex
shards), background indexing triggered on idle, and index-based query
handlers for major LSP requests.

### Index Data Layer (`src/index/`)

- **TUIndex**: Add binary serialization/deserialization via FlatBuffers,
enabling IPC between stateless worker and master server
- **ProjectIndex**: Add symbol name/kind storage, `PathPool` path
normalization (backslash -> forward slash), and binary persistence
- **MergedIndex**: Add `content` field to store file content for
reliable offset<->position mapping; add `removed` bitmap for garbage
collection of deleted entries; filter removed IDs in `lookup()` queries
- **schema.fbs**: Add TUIndex tables, `Symbol.name` field,
`MergedIndex.removed` bitmap and `MergedIndex.content` string

### Server (`src/server/`)

- **Background indexing**: Idle-triggered coroutine dequeues files from
CDB, dispatches `IndexParams` to stateless workers, merges returned
`TUIndex` into ProjectIndex/MergedIndex, and persists to `.clice/index/`
- **Index persistence**: `save_index()` / `load_index()` for startup
restoration; only rewrites shards flagged `need_rewrite()`
- **LSP handlers**:
- `textDocument/definition` -- index-first lookup with stateful worker
fallback
  - `textDocument/references` -- cross-file reference query via index
- `callHierarchy/prepare`, `incomingCalls`, `outgoingCalls` --
Caller/Callee relation traversal
- `typeHierarchy/prepare`, `supertypes`, `subtypes` -- Base/Derived
relation traversal
- `workspace/symbol` -- case-insensitive substring search over
ProjectIndex symbols
- **Stateless worker**: Add `Index` request handler that builds
`TUIndex` from compiled AST and returns serialized data
- **Config**: Add `enable_indexing` (default true) and `idle_timeout_ms`
(default 3000ms)

### Fixes and Cross-platform

- **ElaboratedType handling** in `decl_of()` for correct Base/Derived
relation emission
- **Windows path normalization** in `PathPool::intern()` and
`ProjectIndex::from()` (backslash -> forward slash)
- **`.gitattributes`**: Force LF in `tests/data/**` to prevent CRLF
byte-offset mismatches on Windows CI
- **Test fixture**: Clean `.clice/` before each test for hermetic index
state

### Tests

- **370-line** `index_query_tests.cpp`: unit tests for occurrence
lookup, relation queries, content retrieval, removed bitmap filtering
- **282-line** `test_index.py`: E2E integration tests for
GoToDefinition, FindReferences, CallHierarchy
(prepare/incoming/outgoing), TypeHierarchy
(prepare/supertypes/subtypes), WorkspaceSymbol
- Updated existing MergedIndex and ProjectIndex tests for new schema
fields

## Test plan

- [x] 414 C++ unit tests pass (including new IndexQuery, MergedIndex,
ProjectIndex tests)
- [x] 69 Python integration tests pass (including 10 new index feature
tests)
- [x] CI green on Linux, macOS, Windows
- [ ] Manual smoke test with VSCode extension

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
ykiko
2026-04-02 00:20:41 +08:00
committed by GitHub
parent 21a969af27
commit e43bb14998
23 changed files with 2395 additions and 79 deletions

View File

@@ -163,6 +163,43 @@ TEST_CASE(FileIdsMapCorrectness) {
}
}
TEST_CASE(NameSurvivesRoundTrip) {
index::TUIndex tu;
ASSERT_TRUE(build_and_index(R"(
int my_variable = 42;
void my_function() {}
)",
tu));
index::ProjectIndex project;
project.merge(tu);
// Verify names are populated after merge.
bool found_var = false;
bool found_func = false;
for(auto& [hash, symbol]: project.symbols) {
if(symbol.name == "my_variable")
found_var = true;
if(symbol.name == "my_function")
found_func = true;
}
ASSERT_TRUE(found_var);
ASSERT_TRUE(found_func);
// Serialize and deserialize.
llvm::SmallString<4096> buf;
llvm::raw_svector_ostream os(buf);
project.serialize(os);
auto restored = index::ProjectIndex::from(buf.data());
// Verify names survive round-trip.
for(auto& [hash, symbol]: project.symbols) {
ASSERT_TRUE(restored.symbols.contains(hash));
ASSERT_EQ(restored.symbols[hash].name, symbol.name);
ASSERT_EQ(restored.symbols[hash].kind.value(), symbol.kind.value());
}
}
}; // TEST_SUITE(ProjectIndex)
} // namespace
} // namespace clice::testing