17 Commits

Author SHA1 Message Date
Myriad-Dreamin
04af0e51f8 test(agentic): remove lint integration test 2026-06-05 09:46:38 +08:00
Myriad-Dreamin
7a37222fd3 fix(prune): keep clang-tidy module libraries 2026-06-05 09:11:07 +08:00
Myriad-Dreamin
8e49d7b9e9 Force link clang-tidy check modules 2026-06-05 08:58:52 +08:00
Myriad-Dreamin
98e258d50c Format agentic lint test 2026-06-05 02:29:55 +08:00
Myriad-Dreamin
cb190d3d99 Test agentic clang-tidy lint diagnostics 2026-06-05 02:24:58 +08:00
Myriad-Dreamin
2baf947bff Run clang-tidy for agentic lint 2026-06-05 02:05:38 +08:00
Myriad-Dreamin
a17b3f6b7b Add agentic lint request 2026-06-05 01:26:54 +08:00
Myriad-Dreamin
3b45888622 fix(semantic-tokens): filter ineligible highlight references (#434)
Add a reusable declaration-name eligibility helper that mirrors clangd's
`canHighlightName`, use it to suppress unsupported reference tokens in
the semantic-token collector, and cover the change with focused
semantic-token regression tests plus a constructor/destructor positive
case.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved semantic token highlighting to suppress ineligible operator
references lacking meaningful source text.
* Ensured constructor and destructor names remain properly highlighted
with correct visual modifiers.

* **Tests**
* Added test coverage for semantic token highlighting behavior across
various declaration types.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/clice-io/clice/pull/434?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-05 00:03:18 +08:00
ykiko
cc5b25d5c3 refactor: public feature types and snapshot testing infrastructure (#442)
## Summary

- **Public feature types**: Move `SemanticToken`, `FoldingRange`,
`DocumentSymbol`, `InlayHint`, and `HintCategory` from internal `.cpp`
files to `feature.h` as public API types. Each feature now exposes two
overloads: a raw overload returning offset-based types and a protocol
overload that converts to LSP wire-format with explicit
`PositionEncoding`.
- **Snapshot testing**: Add corpus-driven snapshot tests using
`ASSERT_SNAPSHOT_GLOB` for semantic tokens, folding ranges, inlay hints,
document symbols, and TU index. Tests compile real C++ corpus files,
format output as YAML flow mappings, and diff against `.snap.yml`
baselines.
- **Test infrastructure**: Add `compile_file()` to `Tester`,
`yaml_str()` utility, `--corpus-dir` / `--snapshot-dir` CLI options, and
`--verbose` flag for unit tests. Migrate to kotatsu's unified
`kota::zest::Options` API.
- **Toolchain robustness**: Filter unknown cc1 args via
`clang::driver::getDriverOptTable()` to handle system compilers newer
than embedded LLVM.
- **Dependency bump**: Update kotatsu to 7381404 (unified zest Options,
out-param `from_json` API).

## Details

### Feature type changes
All five feature modules (`semantic_tokens`, `folding_ranges`,
`document_symbols`, `inlay_hints`, `document_links`) now follow the same
two-overload pattern. The raw overload returns offset-based structs
suitable for indexing and testing; the protocol overload adds
`PositionEncoding` conversion for LSP responses. `stateful_worker.cpp`
explicitly passes `PositionEncoding::UTF16` at every call site.

### Snapshot tests
Corpus files live in `tests/corpus/` (organized by language construct).
Snapshot baselines live in `tests/snapshots/<feature>/`. Format lambdas
are inlined directly in test bodies — no separate format functions for
single-use formatters. YAML output uses flow mappings (`- { key: value
}`) for compact, diffable baselines.

### cc1 arg filtering
`src/command/toolchain.cpp` now parses the cc1 argument list through
LLVM's driver option table and drops any args classified as
`UnknownClass`. This prevents compilation failures when the system
compiler emits flags that the embedded LLVM version doesn't recognize.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-24 19:36:27 +08:00
ykiko
3305465d1f feat(formatting): wire up textDocument/formatting and rangeFormatting (#441)
## Summary

- Wire up the existing `document_format` feature to LSP via stateless
workers
- Add `Format` kind to stateless worker dispatch, with a lightweight
`forward_format` path in `Compiler` (no compilation/deps needed — just
file path + content)
- Register `textDocument/formatting` and `textDocument/rangeFormatting`
handlers with `scoped_pause`
- Style lookup uses `clang::format::getStyle` which walks parent
directories for `.clang-format`, matching clangd's behavior

## Test plan

- [x] 4 unit tests: simple format, range format, idempotent (no edits),
include sort
- [x] 3 integration tests: full document format (verifies applied edits
match expected output), range format, already-formatted no-op
- [x] Capability assertions added to `test_capabilities`
- [x] All existing tests pass (554 unit, 170 integration, 2 smoke)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

* **New Features**
* Added document formatting and range formatting capabilities to the LSP
server
* Formatting can target the entire document or a specific range of code
  * Server now advertises formatting support to LSP clients

* **Tests**
  * Added comprehensive test coverage for formatting functionality

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-04 19:15:07 +08:00
ykiko
47ad905f5b feat(server): agentic query API, daemon mode, and relay mode (#438)
## Summary
- **Agentic query API**: 11 JSON-RPC handlers over TCP for AI agent
integration — compileCommand, projectFiles, fileDeps, impactAnalysis,
symbolSearch, readSymbol, documentSymbols, definition, references,
callGraph, typeHierarchy
- **Daemon mode**: MasterServer listens on a unix domain socket,
accepting multiple agent connections (`--mode daemon`)
- **Relay mode**: Bidirectional stdin/stdout ↔ unix socket proxy for
editor integration (`--mode relay`)
- **Full-body definition text**: readSymbol/definition return complete
function/class bodies via brace matching, not just the declaration line
- **24 integration tests** with concrete value assertions covering all
handlers

## Test plan
- [x] All 551 unit tests pass
- [x] All 2 smoke tests pass
- [x] All 148 integration tests pass (including 24 new agentic tests)
- [x] Raw JSON responses manually inspected for correctness (line
numbers, text content, field names, structural completeness)
- [x] Formatted with `pixi run format`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Daemon mode: background service with workspace watching and socket
support
  * Relay mode: stdio ↔ Unix-socket message relay
* Expanded agentic RPC/CLI: project files, deps/impact, symbol
search/read, document symbols, definitions, references, call graphs,
type hierarchy, status, shutdown; richer query options

* **Behavioral**
* Improved indexing lifecycle and status reporting; safer shutdown
coordination and client handling

* **Tests**
* New integration tests covering agentic RPC endpoints, CLI status,
shutdown, and error cases
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-04 00:15:37 +08:00
ykiko
75b9ea05b8 refactor(server): split into service layer, add agentic protocol, adopt task_group (#437)
## Summary

- **Restructure `src/server/` into subdirectories** (`service/`,
`compiler/`, `worker/`, `workspace/`, `protocol/`) to separate concerns:
transport/session management, compilation, worker orchestration, and
persistent workspace state.
- **Decouple MasterServer from transport**: MasterServer no longer holds
a `JsonPeer&` reference or registers handlers itself. New `LSPClient`
and `AgentClient` classes own their peer references and register
protocol handlers, accessing MasterServer internals via `friend class`.
- **Add agentic protocol**: A TCP-based side channel
(`agentic/compileCommand`) that lets external tools (AI agents, build
systems) query compile commands from a running clice server. Includes a
CLI client mode (`--mode agentic --port N --path FILE`), server-side
listener when `--port` is specified in pipe mode, and integration tests
for happy path, fallback, concurrency, and connection-refused.
- **Replace fire-and-forget `loop.schedule()` with `kota::task_group`**:
Compiler compile tasks, Indexer background indexing + resource monitor,
WorkerPool worker monitors, and socket accept loops now use structured
concurrency. This eliminates manual `alive_count_`/generation counters
and ensures all spawned tasks are joined on shutdown.
- **Fix flaky integration test**: `CliceClient.initialize()` now always
sets `cache_dir` to a workspace-local `.clice/` directory, preventing
stale PCH artifacts from the global `~/.cache/clice/` from polluting
test runs.

## Details

**Compiler peer lifetime**: `Compiler` and `Indexer` previously took
`JsonPeer&` in their constructors, coupling them to a single connection.
They now store a `JsonPeer*` set via `set_peer()`, with null checks
before sending diagnostics/progress. This supports the multi-connection
model where agentic clients don't need diagnostics.

**Socket mode single-LSP enforcement**: `accept_connections()` takes a
`register_lsp` flag; when true, only the first connection gets an
`LSPClient`. All connections get an `AgentClient`. This prevents
multiple LSP sessions from racing on shared server state.

**Structured shutdown**: `Compiler::stop()` cancels in-flight compile
tasks and joins them. `WorkerPool::stop()` signals workers and joins the
monitor task group. `Indexer` uses a `cancellation_source` to stop its
resource monitor when a background indexing run completes.

**Pin kotatsu**: Changed from `GIT_TAG main` + `GIT_SHALLOW TRUE` to an
exact commit hash for reproducible builds.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-02 01:06:18 +08:00
ykiko
939ab6d0d4 feat(server): concurrent background indexing with priority control (#432)
## Summary

- Rewrite serial background indexing to concurrent dispatch (up to
`stateless_worker_count / 2` parallel tasks)
- Add depth-counted pause/resume mechanism: completion and
signature-help handlers pause new index dispatches to prioritize user
requests
- Report indexing progress via LSP `$/progress` notifications
(percentage + file count)
- Lower thread scheduling priority (`nice +10`) for index tasks in
stateless workers via RAII `ScopedNice` guard

## Test plan

- [x] `pixi run format` — no changes
- [x] `pixi run unit-test Debug` — 551 passed, 9 skipped (pre-existing)
- [x] `pixi run smoke-test Debug` — 2/2 passed
- [x] `pixi run integration-test Debug` — 121 passed, 3 failed (all
pre-existing on main: header_context x2, staleness x1)
- [ ] Manual test: open a large project (e.g. LLVM), verify progress bar
appears and completion remains responsive during indexing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Pause/resume controls for background indexing
* Concurrent, adaptive background indexing with configurable concurrency
* LSP progress reporting (create/begin/report/end) and updated
completion metrics

* **Behavior Change**
* Code completion and signature help temporarily pause indexing for
responsiveness
* Background indexing runs with reduced scheduling priority on
non-Windows and logs "files dispatched" at finish

* **Tests**
* Test client fixture defaults init options and sets workspace cache dir
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-23 13:28:59 +08:00
ykiko
e1202d2fa5 fix: prevent worker crashes from null ASTConsumer, invalid FileID, and missing PCH cache dir (#435)
## Summary

Three pre-existing bugs cause worker processes to crash with SEGV or
SIGABRT. On the main branch these crashes are silent (workers die,
requests fail fast with "transport closed", tests still pass because
null responses are accepted). However when combined with #432's worker
respawn mechanism, the crash-respawn-crash cycle on low-core CI machines
causes request timeouts and smoke test hangs.

### Fixes

- **compilation.cpp**: `ProxyAction::CreateASTConsumer` now checks for
null before passing to `MultiplexConsumer`. When the wrapped action's
`CreateASTConsumer` fails (e.g. missing system headers during PCH
generation), this previously caused a null pointer dereference, SEGV,
ASAN kills the stateless worker.
- **compilation_unit.cpp**: `file_path()` returns empty `StringRef` on
invalid `FileID` instead of asserting. The assert fired when
`IncludeGraph::from()` called `file_path(interested_file())` on an AST
compiled with synthesized default commands (no compile_commands.json,
clang++ -std=c++20 fallback, no system headers, invalid main file ID),
SIGABRT, stateful worker crash.
- **compiler.cpp**: `ensure_pch` now creates the PCH cache directory
before sending the build request. Previously, when `load_workspace()`
exited early (no compile_commands.json), the cache subdirectories were
never created, causing every PCH write to fail with "No such file or
directory".
- **master_server.cpp/h**: `load_workspace()` changed from
`kota::task<>` to plain `void` -- it contains only synchronous
filesystem operations and no co_await, so the coroutine wrapper was
unnecessary. Called directly instead of via `loop.schedule()`.

## Test plan

- [x] Verified zero SEGV/SIGABRT/assertion crashes in worker stderr
after fix
- [x] rapid_edit.jsonl smoke test passes 3/3 runs consistently (34s
each)
- [x] Behavior matches main branch (both return 134 responses, 0
pending)
- [x] Debug build with ASAN (detect_leaks=0) -- clean run, no sanitizer
reports

<!-- codesmith:footer -->
---
<a
href="https://app.blacksmith.sh/clice-io/codesmith/clice/pr/435"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-in-codesmith-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-in-codesmith-light.svg"><img
alt="View in Codesmith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/view-in-codesmith-dark.svg"></picture></a>
<sup>Codesmith can help with this PR — just tag <code>@codesmith</code>
or enable autofix.</sup>

- [ ] Autofix CI and bot reviews
<!-- /codesmith:footer -->

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved error handling for AST consumer creation with null checks and
a clear failure path.
* Safer file-path access that returns empty for invalid identifiers
instead of asserting.
* PCH cache handling now validates cache configuration, attempts
directory creation, logs warnings, and aborts PCH builds on failure.

* **Refactor**
* Workspace loading changed from asynchronous to synchronous execution.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-23 10:36:03 +08:00
ykiko
17e68010a0 feat(server): improve configuration file handling (#423)
## Summary

- **`[[rules]]`**: TOML array-of-tables config for per-file compilation
flag rules with glob pattern matching (`append`/`remove`). Patterns are
pre-compiled at config load time. Rules whose patterns all fail to
compile are dropped entirely (no silent no-op entries), and rules now
apply uniformly to every compilation — including the header-context
fallback path used when editing a header without its own CDB entry.
- **CDB auto-scan**: Default search scans workspace root + all immediate
subdirectories for `compile_commands.json`, replacing the hardcoded
directory list.
- **LSP `initializationOptions`**: Clients can pass config as JSON via
the LSP initialize request; priority is `initializationOptions >
clice.toml > defaults`.
- **XDG cache paths**: Default cache/index/logging paths prefer
`$XDG_CACHE_HOME/clice/<workspace-hash>/`; fall back to
`$HOME/.cache/clice/<hash>/`, then `<workspace>/.clice/`.
- **`${workspace}` substitution**: supported in `cache_dir`,
`index_dir`, `logging_dir`, and every `compile_commands_paths` entry.
No-op when `workspace_root` is empty.
- **Partial config support**: All TOML/JSON fields are optional via
`kota::meta::defaulted<T>`, so minimal config files work correctly.
- **Detailed diagnostics**: malformed `clice.toml` now logs line, column
and parser description (via toml++ direct parse); a malformed workspace
config surfaces a clear fallback warning instead of silently reverting
to defaults.

## Test plan

- [x] 28 unit tests for config (full suite 545 unit tests pass, Debug)
- [x] 119 integration tests pass
- [x] 2 smoke tests pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* XDG-based, workspace-scoped project cache (PCH/PCM and header caches
moved under project cache) with workspace fallback
* Initialization options JSON can override config (takes precedence over
file/defaults)
* Per-file pattern rules to append/remove compile flags; expanded
discovery of compilation databases (multiple paths)

* **Refactor**
* Configuration fields reorganized under a project scope; runtime
behavior now respects project-scoped values

* **Tests**
* New unit and integration tests for config parsing, rule matching, and
persistent cache behavior

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-20 00:21:31 +08:00
ykiko
3fa653bcaf feat(completion): mark deprecated symbols with strikethrough (#414)
## Summary
- Check `CXAvailability_Deprecated` on `CodeCompletionResult` and set
`CompletionItemTag::Deprecated`
- Editors render deprecated completions with a strikethrough on the
label

## Test plan
- [x] `DeprecatedTag` — `[[deprecated]]` function gets the tag
- [x] `NotDeprecated` — normal function has no Deprecated tag
- [x] All 491 unit tests pass
- [x] `pixi run format` clean

Stacked on #411.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Code completion now marks deprecated declarations with a deprecated
tag so users can see deprecated items in completion lists.

* **Tests**
* Added unit tests ensuring deprecated declarations produce completion
items with the deprecated tag and non-deprecated items do not.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-19 13:42:32 +08:00
ykiko
592b37417e feat: cross-compile & upgrade LLVM to 21.1.8 (#390)
## Summary

This PR adds cross-compilation support for three new target platforms,
upgrades LLVM to 21.1.8, and overhauls the CI pipelines around
cross-builds and testing.

## Cross-compilation

New target triples accepted via `-DCLICE_TARGET_TRIPLE=...`:

| Target triple | Host | Output |
|---|---|---|
| `x86_64-apple-darwin` | macos-15 (arm64) | macOS x64 |
| `aarch64-linux-gnu` | ubuntu-24.04 (x64) | Linux arm64 |
| `aarch64-pc-windows-msvc` | windows-2025 (x64) | Windows arm64 |

- `cmake/toolchain.cmake` — maps `CLICE_TARGET_TRIPLE` to
`CMAKE_SYSTEM_NAME`/`CMAKE_SYSTEM_PROCESSOR`/compiler `--target`; picks
up conda aarch64 sysroot when cross-compiling Linux.
- `cmake/llvm.cmake` — forwards target platform/arch to `setup-llvm.py`
so the right prebuilt LLVM is downloaded for the target.
- `CMakeLists.txt` — uses a host-side `flatc` from `PATH` under
`CMAKE_CROSSCOMPILING` instead of the in-tree target build.
- `pixi.toml`:
  - Adds `osx-64`, `linux-aarch64`, `win-arm64` platforms.
- New environments: `cross-macos-x64`, `cross-linux-aarch64` (adds
`gcc_linux-aarch64` + `sysroot_linux-aarch64`), `cross-windows-arm64`.
- New lightweight `test-run` env used on native ARM/x64 runners to
execute cross-built artifacts (pulls in upstream clang+lld on macOS so
tests don't fall back to Apple clang).
- `scripts/activate_cross_linux.sh` — exports `CONDA_PREFIX`-relative
paths for the aarch64 toolchain.
- `scripts/build-llvm.py` — `--target-triple` support and a
`build_native_tools()` helper that produces host `llvm-tblgen` /
`clang-tblgen` needed when cross-compiling LLVM itself.

## LLVM upgrade 21.1.4 → 21.1.8

- `cmake/package.cmake` bumps `setup_llvm("21.1.8")`.
- `config/llvm-manifest.json` regenerated with 6 new cross-compiled
entries and a new `arch` field on every entry so lookup is `(version,
platform, arch, lto, build_type)`.
- `scripts/setup-llvm.py` — honours the new `arch` field when resolving
artifacts.
- `scripts/update-llvm-version.py` (new) — single-call version bump
across `package.cmake` + manifest.
- `scripts/validate-llvm-components.py` (new) — scans the LLVM source
tree for library targets and diffs them against
`scripts/llvm-components.json` to catch stale/misspelled component names
before a build.
- `scripts/llvm-components.json` (new) — explicit allow-list of required
LLVM/Clang library targets used by `build-llvm.py`.

## CI changes

- `.github/workflows/build-llvm.yml`:
- Adds `workflow_dispatch` with `llvm_version`, `skip_upload`, `skip_pr`
inputs.
- Matrix extended with the 6 cross-compile entries (2 per new platform:
RelWithDebInfo ± LTO).
- `build clice` / test / prune steps gated on `!matrix.target_triple`
for cross-builds; cross-built LTO entries apply the native prune
manifest (arch-independent).
  - Cross-compiled binary architecture is verified with `file(1)`.
- New `upload` job triggered by `workflow_dispatch` pushes artifacts to
`clice-io/clice-llvm` and hands the manifest off to the next job.
- `.github/workflows/test-cmake.yml`:
- Build matrix gains three `build_only: true` cross entries that upload
`bin/` + `lib/` artifacts.
- New `test-cross` job runs on native `macos-15-intel`,
`ubuntu-24.04-arm`, `windows-11-arm` runners, downloads the cross-built
artifacts, and runs unit / integration / smoke tests under the
`test-run` pixi env.
- Cache keys now include `target_triple` so native and cross builds
don't collide.
- `.github/workflows/publish-clice.yml`:
- Three additional release artifacts for the new targets
(`clice-x86_64-macos-darwin`, `clice-aarch64-linux-gnu`,
`clice-aarch64-windows-msvc`), each with a matching `-symbol` archive.

## Compatibility

- All existing native builds and tests are preserved; cross entries are
additive.
- `Debug` + ASAN remains disabled on Windows (`llvm_mode == Debug && os
== windows-*` no longer appends `-asan`).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-19 00:17:39 +08:00
124 changed files with 12869 additions and 1383 deletions

View File

@@ -13,7 +13,7 @@ runs:
- name: Setup Pixi
uses: prefix-dev/setup-pixi@v0.9.3
with:
pixi-version: v0.62.0
pixi-version: v0.67.0
environments: ${{ inputs.environments }}
activate-environment: true
cache: true

View File

@@ -21,7 +21,7 @@ jobs:
- name: Build scan_benchmark
run: |
pixi run cmake-config RelWithDebInfo ON "-DCLICE_ENABLE_BENCHMARK=ON"
pixi run cmake-config RelWithDebInfo ON -- -DCLICE_ENABLE_BENCHMARK=ON
cmake --build build/RelWithDebInfo --target scan_benchmark
- name: Clone LLVM

View File

@@ -1,6 +1,22 @@
name: build llvm
on:
workflow_dispatch:
inputs:
llvm_version:
description: "LLVM version to build (e.g., 21.1.8)"
required: true
type: string
skip_upload:
description: "Skip upload and PR creation (build-only mode)"
required: false
type: boolean
default: false
skip_pr:
description: "Skip PR creation (upload only, no PR)"
required: false
type: boolean
default: false
pull_request:
# if you want to run this workflow, change the branch name to main,
# if you want to turn off it, change it to non existent branch.
@@ -12,9 +28,7 @@ jobs:
fail-fast: false
matrix:
include:
- os: windows-2025
llvm_mode: Debug
lto: OFF
# Native builds
- os: windows-2025
llvm_mode: RelWithDebInfo
lto: OFF
@@ -39,6 +53,42 @@ jobs:
- os: macos-15
llvm_mode: RelWithDebInfo
lto: ON
# Cross-compilation builds
# macOS x64 (from arm64 macos-15)
- os: macos-15
llvm_mode: RelWithDebInfo
lto: OFF
target_triple: x86_64-apple-darwin
- os: macos-15
llvm_mode: RelWithDebInfo
lto: ON
target_triple: x86_64-apple-darwin
# Linux aarch64 (from x64 ubuntu-24.04)
- os: ubuntu-24.04
llvm_mode: RelWithDebInfo
lto: OFF
target_triple: aarch64-linux-gnu
pixi_env: cross-linux-aarch64
- os: ubuntu-24.04
llvm_mode: RelWithDebInfo
lto: ON
target_triple: aarch64-linux-gnu
pixi_env: cross-linux-aarch64
# Windows arm64 (from x64 windows-2025)
- os: windows-2025
llvm_mode: RelWithDebInfo
lto: OFF
target_triple: aarch64-pc-windows-msvc
pixi_env: cross-windows-arm64
- os: windows-2025
llvm_mode: RelWithDebInfo
lto: ON
target_triple: aarch64-pc-windows-msvc
pixi_env: cross-windows-arm64
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
@@ -67,49 +117,91 @@ jobs:
free -h
df -h
- name: Setup Pixi
uses: prefix-dev/setup-pixi@v0.9.3
- uses: ./.github/actions/setup-pixi
with:
pixi-version: v0.59.0
environments: package
activate-environment: true
cache: true
locked: true
environments: ${{ matrix.pixi_env || 'package' }}
- name: Clone llvm-project (21.1.4)
- name: Clone llvm-project
shell: bash
run: |
git clone --branch llvmorg-21.1.4 --depth 1 https://github.com/llvm/llvm-project.git .llvm
VERSION="${{ inputs.llvm_version || '21.1.8' }}"
echo "Cloning LLVM ${VERSION}..."
git clone --branch "llvmorg-${VERSION}" --depth 1 https://github.com/llvm/llvm-project.git .llvm
- name: Validate distribution components
shell: bash
run: |
python3 scripts/validate-llvm-components.py \
--llvm-src=.llvm \
--components-file=scripts/llvm-components.json
- name: Build LLVM (install-distribution)
shell: bash
run: |
pixi run build-llvm --llvm-src=.llvm --mode="${{ matrix.llvm_mode }}" --lto="${{ matrix.lto }}" --build-dir=build
ENV="${{ matrix.pixi_env || 'package' }}"
EXTRA_ARGS=""
if [[ -n "${{ matrix.target_triple }}" ]]; then
EXTRA_ARGS="--target-triple=${{ matrix.target_triple }}"
fi
pixi run -e "$ENV" build-llvm \
--llvm-src=.llvm \
--mode="${{ matrix.llvm_mode }}" \
--lto="${{ matrix.lto }}" \
--build-dir=build \
${EXTRA_ARGS}
- name: Build clice using installed LLVM
if: ${{ !matrix.target_triple }}
shell: bash
run: |
cmake -B build -G Ninja \
-DCMAKE_BUILD_TYPE=${{ matrix.llvm_mode }} \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchain.cmake \
-DCLICE_ENABLE_TEST=ON \
-DCLICE_CI_ENVIRONMENT=ON \
-DCLICE_ENABLE_LTO=${{ matrix.lto }} \
-DLLVM_INSTALL_PATH=".llvm/build-install"
cmake --build build
pixi run cmake-config ${{ matrix.llvm_mode }} ON -- \
"-DCLICE_ENABLE_LTO=${{ matrix.lto }}" \
"-DLLVM_INSTALL_PATH=.llvm/build-install"
pixi run cmake-build ${{ matrix.llvm_mode }}
- name: Build clice using installed LLVM (cross-compile)
if: ${{ matrix.target_triple }}
shell: bash
run: |
ENV="${{ matrix.pixi_env || 'package' }}"
pixi run -e "$ENV" cmake-config ${{ matrix.llvm_mode }} ON -- \
"-DCLICE_ENABLE_LTO=${{ matrix.lto }}" \
"-DCLICE_TARGET_TRIPLE=${{ matrix.target_triple }}" \
"-DLLVM_INSTALL_PATH=.llvm/build-install"
pixi run -e "$ENV" cmake-build ${{ matrix.llvm_mode }}
- name: Verify cross-compiled binary architecture
if: ${{ matrix.target_triple && runner.os != 'Windows' }}
shell: bash
run: |
BINARY="build/${{ matrix.llvm_mode }}/bin/clice"
echo "Binary info:"
file "$BINARY"
case "${{ matrix.target_triple }}" in
aarch64-linux-gnu) file "$BINARY" | grep -q "aarch64" ;;
x86_64-apple-darwin) file "$BINARY" | grep -q "x86_64" ;;
esac
- name: Upload cross-compiled clice for functional test
if: ${{ matrix.target_triple && matrix.lto == 'OFF' }}
uses: actions/upload-artifact@v4
with:
name: cross-clice-${{ matrix.target_triple }}-${{ matrix.llvm_mode }}
path: |
build/${{ matrix.llvm_mode }}/bin/
build/${{ matrix.llvm_mode }}/lib/
if-no-files-found: error
retention-days: 1
- name: Run tests
if: ${{ !matrix.target_triple }}
shell: bash
run: |
EXE_EXT=""
if [[ "${{ runner.os }}" == "Windows" ]]; then
EXE_EXT=".exe"
fi
./build/bin/unit_tests${EXE_EXT} --test-dir="./tests/data"
uv run --project tests pytest -s --log-cli-level=INFO tests/integration --executable=./build/bin/clice${EXE_EXT}
run: pixi run test ${{ matrix.llvm_mode }}
# Prune is only supported for native builds (requires linking clice to test).
# Cross-compiled targets reuse the native prune manifest of the same OS.
- name: Prune LLVM static libraries (Debug/RelWithDebInfo no LTO)
if: matrix.llvm_mode == 'Debug' || (matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF')
if: (!matrix.target_triple) && (matrix.llvm_mode == 'Debug' || (matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF'))
shell: bash
run: |
MANIFEST="pruned-libs-${{ matrix.os }}.json"
@@ -117,13 +209,13 @@ jobs:
python3 scripts/prune-llvm-bin.py \
--action discover \
--install-dir ".llvm/build-install/lib" \
--build-dir "build" \
--build-dir "build/${{ matrix.llvm_mode }}" \
--max-attempts 60 \
--sleep-seconds 60 \
--manifest "${MANIFEST}"
- name: Upload pruned-libs manifest
if: matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF'
if: (!matrix.target_triple) && matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF'
uses: actions/upload-artifact@v4
with:
name: llvm-pruned-libs-${{ matrix.os }}
@@ -131,8 +223,8 @@ jobs:
if-no-files-found: error
compression-level: 0
- name: Apply pruned-libs manifest (RelWithDebInfo + LTO)
if: matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'ON'
- name: Apply pruned-libs manifest (RelWithDebInfo + LTO, native only)
if: (!matrix.target_triple) && matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'ON'
shell: bash
env:
GH_TOKEN: ${{ github.token }}
@@ -142,7 +234,27 @@ jobs:
--action apply \
--manifest "${MANIFEST}" \
--install-dir ".llvm/build-install/lib" \
--build-dir "build" \
--build-dir "build/${{ matrix.llvm_mode }}" \
--gh-run-id "${{ github.run_id }}" \
--gh-artifact "llvm-pruned-libs-${{ matrix.os }}" \
--gh-download-dir "artifacts" \
--max-attempts 60 \
--sleep-seconds 60
# For cross-compiled LTO builds, apply the native prune manifest.
# The unused library set is arch-independent (same API surface).
- name: Apply pruned-libs manifest (cross-compile + LTO)
if: matrix.target_triple && matrix.lto == 'ON'
shell: bash
env:
GH_TOKEN: ${{ github.token }}
run: |
MANIFEST="pruned-libs-${{ matrix.os }}.json"
python3 scripts/prune-llvm-bin.py \
--action apply \
--manifest "${MANIFEST}" \
--install-dir ".llvm/build-install/lib" \
--build-dir "build/${{ matrix.llvm_mode }}" \
--gh-run-id "${{ github.run_id }}" \
--gh-artifact "llvm-pruned-libs-${{ matrix.os }}" \
--gh-download-dir "artifacts" \
@@ -157,23 +269,35 @@ jobs:
MODE_TAG="debug"
fi
ARCH="x64"
PLATFORM="linux"
TOOLCHAIN="gnu"
if [[ "${{ matrix.os }}" == windows-* ]]; then
PLATFORM="windows"
TOOLCHAIN="msvc"
elif [[ "${{ matrix.os }}" == macos-* ]]; then
ARCH="arm64"
PLATFORM="macos"
TOOLCHAIN="clang"
# Determine arch/platform/toolchain from target triple or runner OS
if [[ -n "${{ matrix.target_triple }}" ]]; then
case "${{ matrix.target_triple }}" in
x86_64-apple-darwin)
ARCH="x64"; PLATFORM="macos"; TOOLCHAIN="clang" ;;
aarch64-linux-gnu)
ARCH="aarch64"; PLATFORM="linux"; TOOLCHAIN="gnu" ;;
aarch64-pc-windows-msvc)
ARCH="aarch64"; PLATFORM="windows"; TOOLCHAIN="msvc" ;;
esac
else
ARCH="x64"
PLATFORM="linux"
TOOLCHAIN="gnu"
if [[ "${{ matrix.os }}" == windows-* ]]; then
PLATFORM="windows"
TOOLCHAIN="msvc"
elif [[ "${{ matrix.os }}" == macos-* ]]; then
ARCH="arm64"
PLATFORM="macos"
TOOLCHAIN="clang"
fi
fi
SUFFIX=""
if [[ "${{ matrix.lto }}" == "ON" ]]; then
SUFFIX="-lto"
fi
if [[ "${{ matrix.llvm_mode }}" == "Debug" ]]; then
if [[ "${{ matrix.llvm_mode }}" == "Debug" && "${{ matrix.os }}" != windows-* ]]; then
SUFFIX="${SUFFIX}-asan"
fi
@@ -189,3 +313,134 @@ jobs:
name: ${{ env.LLVM_INSTALL_ARCHIVE }}
path: ${{ env.LLVM_INSTALL_ARCHIVE }}
if-no-files-found: error
test-cross:
needs: build
strategy:
fail-fast: false
matrix:
include:
- os: macos-15-intel
llvm_mode: RelWithDebInfo
target_triple: x86_64-apple-darwin
- os: ubuntu-24.04-arm
llvm_mode: RelWithDebInfo
target_triple: aarch64-linux-gnu
- os: windows-11-arm
llvm_mode: RelWithDebInfo
target_triple: aarch64-pc-windows-msvc
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: test-run
- name: Download cross-compiled clice
uses: actions/download-artifact@v4
with:
name: cross-clice-${{ matrix.target_triple }}-${{ matrix.llvm_mode }}
path: build/${{ matrix.llvm_mode }}/
- name: Make binaries executable
if: runner.os != 'Windows'
run: chmod +x build/${{ matrix.llvm_mode }}/bin/*
- name: Run tests
run: pixi run -e test-run test ${{ matrix.llvm_mode }}
upload:
needs: build
if: ${{ !cancelled() && inputs.llvm_version && !inputs.skip_upload }}
runs-on: ubuntu-24.04
permissions:
contents: read
steps:
- uses: actions/checkout@v4
- name: Download all build artifacts
env:
GH_TOKEN: ${{ github.token }}
run: scripts/download-llvm.sh "${{ github.run_id }}"
- name: Upload to clice-llvm
env:
GH_TOKEN: ${{ secrets.UPLOAD_LLVM }}
TARGET_REPO: clice-io/clice-llvm
run: python3 scripts/upload-llvm.py "${{ inputs.llvm_version }}" "${TARGET_REPO}" "${{ github.run_id }}"
- name: Save manifest for update-clice job
uses: actions/upload-artifact@v4
with:
name: llvm-manifest-final
path: artifacts/llvm-manifest.json
if-no-files-found: error
compression-level: 0
update-clice:
needs: upload
if: ${{ !inputs.skip_pr }}
runs-on: ubuntu-24.04
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v4
- name: Download manifest
uses: actions/download-artifact@v4
with:
name: llvm-manifest-final
path: .
- name: Update manifest and version
run: |
python3 scripts/update-llvm-version.py \
--version "${{ inputs.llvm_version }}" \
--manifest-src llvm-manifest.json \
--manifest-dest config/llvm-manifest.json \
--package-cmake cmake/package.cmake
- name: Create or update PR
env:
GH_TOKEN: ${{ github.token }}
run: |
VERSION="${{ inputs.llvm_version }}"
BRANCH="chore/update-llvm-${VERSION}"
RUN_URL="https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}"
RELEASE_URL="https://github.com/clice-io/clice-llvm/releases/tag/${VERSION}"
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git checkout -b "${BRANCH}"
git add config/llvm-manifest.json cmake/package.cmake
git commit -m "chore: update LLVM to ${VERSION}"
git push --force-with-lease origin "${BRANCH}"
# Check if PR already exists for this branch
EXISTING_PR=$(gh pr list --head "${BRANCH}" --json number --jq '.[0].number // empty')
BODY="$(cat <<EOF
## Summary
- Update LLVM prebuilt binaries to version ${VERSION}
- Updated \`config/llvm-manifest.json\` with new SHA256 hashes
- Updated \`cmake/package.cmake\` version string
**Artifacts:** [clice-llvm release](${RELEASE_URL})
**Build:** [workflow run](${RUN_URL})
> Auto-generated by build-llvm workflow
EOF
)"
if [[ -n "${EXISTING_PR}" ]]; then
echo "Updating existing PR #${EXISTING_PR}"
gh pr edit "${EXISTING_PR}" --body "${BODY}"
else
gh pr create \
--title "chore: update LLVM to ${VERSION}" \
--body "${BODY}" \
--base main
fi

View File

@@ -14,6 +14,12 @@ jobs:
with:
environments: format
- name: Validate update-llvm-version.py can still patch package.cmake
run: |
python3 scripts/update-llvm-version.py --check \
--manifest-dest config/llvm-manifest.json \
--package-cmake cmake/package.cmake
- name: Run formatter
run: pixi run format
continue-on-error: true

View File

@@ -7,6 +7,10 @@ on:
pull_request:
branches: [main]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
jobs:
changes:
if: ${{ !startsWith(github.ref, 'refs/tags/') }}

View File

@@ -9,6 +9,7 @@ jobs:
fail-fast: false
matrix:
include:
# Native builds
- os: windows-2025
artifact_name: clice.zip
asset_name: clice-x64-windows-msvc.zip
@@ -27,6 +28,31 @@ jobs:
symbol_artifact_name: clice-symbol.tar.gz
symbol_asset_name: clice-arm64-macos-darwin-symbol.tar.gz
# Cross-compilation builds
- os: macos-15
target_triple: x86_64-apple-darwin
pixi_env: cross-macos-x64
artifact_name: clice.tar.gz
asset_name: clice-x86_64-macos-darwin.tar.gz
symbol_artifact_name: clice-symbol.tar.gz
symbol_asset_name: clice-x86_64-macos-darwin-symbol.tar.gz
- os: ubuntu-24.04
target_triple: aarch64-linux-gnu
pixi_env: cross-linux-aarch64
artifact_name: clice.tar.gz
asset_name: clice-aarch64-linux-gnu.tar.gz
symbol_artifact_name: clice-symbol.tar.gz
symbol_asset_name: clice-aarch64-linux-gnu-symbol.tar.gz
- os: windows-2025
target_triple: aarch64-pc-windows-msvc
pixi_env: cross-windows-arm64
artifact_name: clice.zip
asset_name: clice-aarch64-windows-msvc.zip
symbol_artifact_name: clice-symbol.zip
symbol_asset_name: clice-aarch64-windows-msvc-symbol.zip
runs-on: ${{ matrix.os }}
defaults:
@@ -39,11 +65,20 @@ jobs:
- uses: ./.github/actions/setup-pixi
with:
environments: package
environments: ${{ matrix.pixi_env || 'package' }}
- name: Package
- name: Package (native)
if: ${{ !matrix.target_triple }}
run: pixi run package
- name: Package (cross-compile)
if: ${{ matrix.target_triple }}
run: |
ENV="${{ matrix.pixi_env }}"
pixi run -e "$ENV" package-config -- \
"-DCLICE_TARGET_TRIPLE=${{ matrix.target_triple }}"
pixi run -e "$ENV" cmake-build
- name: Upload Main Package to Release
if: startsWith(github.ref, 'refs/tags/v')
uses: svenstaro/upload-release-action@v2

View File

@@ -17,53 +17,154 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [windows-2025, ubuntu-24.04, macos-15]
build_type: [Debug, RelWithDebInfo]
include:
# Native builds
- os: windows-2025
build_type: RelWithDebInfo
- os: ubuntu-24.04
build_type: Debug
- os: ubuntu-24.04
build_type: RelWithDebInfo
- os: macos-15
build_type: Debug
- os: macos-15
build_type: RelWithDebInfo
# Cross-compile (build only; tests run on native runners)
- os: macos-15
build_type: RelWithDebInfo
target_triple: x86_64-apple-darwin
build_only: true
- os: ubuntu-24.04
build_type: RelWithDebInfo
target_triple: aarch64-linux-gnu
build_only: true
pixi_env: cross-linux-aarch64
- os: windows-2025
build_type: RelWithDebInfo
target_triple: aarch64-pc-windows-msvc
build_only: true
pixi_env: cross-windows-arm64
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: ${{ matrix.pixi_env || 'default' }}
- name: Restore compiler cache
uses: actions/cache@v4
with:
path: ${{ runner.os == 'Windows' && '.cache/sccache' || '.cache/ccache' }}
key: ${{ runner.os }}-${{ matrix.build_type }}-ccache-${{ github.sha }}
key: ${{ runner.os }}-${{ matrix.build_type }}-${{ matrix.target_triple || 'native' }}-ccache-${{ github.sha }}
restore-keys: |
${{ runner.os }}-${{ matrix.build_type }}-ccache-
${{ runner.os }}-${{ matrix.build_type }}-${{ matrix.target_triple || 'native' }}-ccache-
- name: Zero cache stats
run: |
ENV="${{ matrix.pixi_env || 'default' }}"
if [ "$RUNNER_OS" = "Windows" ]; then
pixi run -- sccache --stop-server || true
pixi run -- sccache --zero-stats || true
pixi run -e "$ENV" -- sccache --stop-server || true
pixi run -e "$ENV" -- sccache --zero-stats || true
else
pixi run -- ccache --zero-stats || true
pixi run -e "$ENV" -- ccache --zero-stats || true
fi
shell: bash
- name: Build
- name: Build (native)
if: ${{ !matrix.target_triple }}
run: pixi run build ${{ matrix.build_type }} ON
- name: Unit Test
- name: Build (cross-compile)
if: ${{ matrix.target_triple }}
shell: bash
run: |
ENV="${{ matrix.pixi_env || 'default' }}"
pixi run -e "$ENV" cmake-config ${{ matrix.build_type }} OFF -- \
"-DCLICE_TARGET_TRIPLE=${{ matrix.target_triple }}"
pixi run -e "$ENV" cmake-build ${{ matrix.build_type }}
- name: Upload cross-compiled binaries
if: ${{ matrix.build_only }}
uses: actions/upload-artifact@v4
with:
name: cross-build-${{ matrix.target_triple }}
path: |
build/${{ matrix.build_type }}/bin/
build/${{ matrix.build_type }}/lib/
if-no-files-found: error
retention-days: 1
- name: Unit tests
if: ${{ !matrix.build_only }}
timeout-minutes: 5
run: pixi run unit-test ${{ matrix.build_type }}
- name: Integration Test
- name: Integration tests
if: ${{ !matrix.build_only }}
timeout-minutes: 20
run: pixi run integration-test ${{ matrix.build_type }}
- name: Smoke Test
if: success() || failure()
- name: Smoke tests
if: ${{ !matrix.build_only }}
timeout-minutes: 15
run: pixi run smoke-test ${{ matrix.build_type }}
- name: Print cache stats and stop server
if: always()
run: |
ENV="${{ matrix.pixi_env || 'default' }}"
if [ "$RUNNER_OS" = "Windows" ]; then
pixi run -- sccache --show-stats
pixi run -- sccache --stop-server || true
pixi run -e "$ENV" -- sccache --show-stats
pixi run -e "$ENV" -- sccache --stop-server || true
else
pixi run -- ccache --show-stats
pixi run -e "$ENV" -- ccache --show-stats
fi
shell: bash
test-cross:
needs: build
strategy:
fail-fast: false
matrix:
include:
- os: macos-15-intel
build_type: RelWithDebInfo
target_triple: x86_64-apple-darwin
- os: ubuntu-24.04-arm
build_type: RelWithDebInfo
target_triple: aarch64-linux-gnu
- os: windows-11-arm
build_type: RelWithDebInfo
target_triple: aarch64-pc-windows-msvc
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: test-run
- name: Download cross-compiled binaries
uses: actions/download-artifact@v4
with:
name: cross-build-${{ matrix.target_triple }}
path: build/${{ matrix.build_type }}/
- name: Make binaries executable
if: runner.os != 'Windows'
run: chmod +x build/${{ matrix.build_type }}/bin/*
- name: Unit tests
timeout-minutes: 5
run: pixi run -e test-run unit-test ${{ matrix.build_type }}
- name: Integration tests
timeout-minutes: 20
run: pixi run -e test-run integration-test ${{ matrix.build_type }}
- name: Smoke tests
timeout-minutes: 10
run: pixi run -e test-run smoke-test ${{ matrix.build_type }}

3
.gitignore vendored
View File

@@ -68,8 +68,7 @@ tests/unit/Local/
.pixi/*
!.pixi/config.toml
.codex/
.codex
.claude/*
!.claude/CLAUDE.md
!.claude/commands/
openspec/

View File

@@ -124,22 +124,63 @@ if(CLICE_CI_ENVIRONMENT)
target_compile_definitions(clice_options INTERFACE CLICE_CI_ENVIRONMENT=1)
endif()
set(CLICE_CLANG_TIDY_MODULE_LIBRARIES)
set(CLICE_MISSING_CLANG_TIDY_MODULES)
foreach(module IN LISTS CLICE_CLANG_TIDY_MODULE_COMPONENTS)
find_library(CLICE_${module}_LIBRARY
NAMES "${module}"
PATHS "${LLVM_INSTALL_PATH}/lib"
NO_DEFAULT_PATH
)
if(CLICE_${module}_LIBRARY)
list(APPEND CLICE_CLANG_TIDY_MODULE_LIBRARIES "${CLICE_${module}_LIBRARY}")
else()
list(APPEND CLICE_MISSING_CLANG_TIDY_MODULES "${module}")
endif()
endforeach()
if(CLICE_MISSING_CLANG_TIDY_MODULES)
message(STATUS "Clang-tidy module libraries not available: ${CLICE_MISSING_CLANG_TIDY_MODULES}")
else()
target_compile_definitions(clice_options INTERFACE CLICE_HAS_CLANG_TIDY_MODULES=1)
endif()
set(FBS_SCHEMA_FILE "${PROJECT_SOURCE_DIR}/src/index/schema.fbs")
set(GENERATED_HEADER "${PROJECT_BINARY_DIR}/generated/schema_generated.h")
set(CLANG_TIDY_CONFIG_SOURCE_FILE "${PROJECT_SOURCE_DIR}/config/clang-tidy-config.h")
set(CLANG_TIDY_CONFIG_GENERATED_FILE "${PROJECT_BINARY_DIR}/generated/clang-tidy-config.h")
if(CMAKE_CROSSCOMPILING)
find_program(FLATC_EXECUTABLE flatc REQUIRED)
set(FLATC_CMD "${FLATC_EXECUTABLE}")
else()
set(FLATC_CMD "$<TARGET_FILE:flatc>")
endif()
add_custom_command(
OUTPUT "${GENERATED_HEADER}"
COMMAND $<TARGET_FILE:flatc> --cpp -o "${PROJECT_BINARY_DIR}/generated" "${FBS_SCHEMA_FILE}"
COMMAND ${FLATC_CMD} --cpp -o "${PROJECT_BINARY_DIR}/generated" "${FBS_SCHEMA_FILE}"
DEPENDS "${FBS_SCHEMA_FILE}"
COMMENT "Generating C++ header from ${FBS_SCHEMA_FILE}"
)
add_custom_target(generate_flatbuffers_schema DEPENDS "${GENERATED_HEADER}")
add_custom_command(
OUTPUT "${CLANG_TIDY_CONFIG_GENERATED_FILE}"
COMMAND ${CMAKE_COMMAND} -E copy_if_different
"${CLANG_TIDY_CONFIG_SOURCE_FILE}"
"${CLANG_TIDY_CONFIG_GENERATED_FILE}"
DEPENDS "${CLANG_TIDY_CONFIG_SOURCE_FILE}"
COMMENT "Generating C++ header from ${CLANG_TIDY_CONFIG_SOURCE_FILE}"
)
add_custom_target(generate_clang_tidy_config DEPENDS "${CLANG_TIDY_CONFIG_GENERATED_FILE}")
file(GLOB_RECURSE CLICE_CORE_SOURCES CONFIGURE_DEPENDS "${PROJECT_SOURCE_DIR}/src/*.cpp")
add_library(clice-core STATIC ${CLICE_CORE_SOURCES})
add_library(clice::core ALIAS clice-core)
add_dependencies(clice-core generate_flatbuffers_schema)
add_dependencies(clice-core generate_flatbuffers_schema generate_clang_tidy_config)
target_include_directories(clice-core PUBLIC
"${PROJECT_SOURCE_DIR}/src"
@@ -155,6 +196,9 @@ target_link_libraries(clice-core PUBLIC
kota::codec::toml
simdjson::simdjson
)
if(CLICE_CLANG_TIDY_MODULE_LIBRARIES)
target_link_libraries(clice-core PUBLIC ${CLICE_CLANG_TIDY_MODULE_LIBRARIES})
endif()
add_executable(clice "${PROJECT_SOURCE_DIR}/src/clice.cc")
target_link_libraries(clice PRIVATE clice::core kota::deco)

View File

@@ -26,7 +26,7 @@
#include "support/path_pool.h"
#include "syntax/dependency_graph.h"
#include "kota/codec/json/serializer.h"
#include "kota/codec/json/json.h"
#include "kota/deco/deco.h"
#include "llvm/Support/FileSystem.h"

View File

@@ -1,5 +1,34 @@
include_guard()
set(CLICE_CLANG_TIDY_MODULE_COMPONENTS
# Keep this in sync with scripts/llvm-components.json and the old
# ALL_CLANG_TIDY_CHECKS list. MPIModule is intentionally excluded because
# clice disables static analyzer checks in ClangTidyForceLinker.h.
clangTidyAndroidModule
clangTidyAbseilModule
clangTidyAlteraModule
clangTidyBoostModule
clangTidyBugproneModule
clangTidyCERTModule
clangTidyConcurrencyModule
clangTidyCppCoreGuidelinesModule
clangTidyDarwinModule
clangTidyFuchsiaModule
clangTidyGoogleModule
clangTidyHICPPModule
clangTidyLinuxKernelModule
clangTidyLLVMModule
clangTidyLLVMLibcModule
clangTidyMiscModule
clangTidyModernizeModule
clangTidyObjCModule
clangTidyOpenMPModule
clangTidyPerformanceModule
clangTidyPortabilityModule
clangTidyReadabilityModule
clangTidyZirconModule
)
function(setup_llvm LLVM_VERSION)
find_package(Python3 COMPONENTS Interpreter REQUIRED)
@@ -25,6 +54,22 @@ function(setup_llvm LLVM_VERSION)
list(APPEND LLVM_SETUP_ARGS "--offline")
endif()
if(DEFINED CLICE_TARGET_TRIPLE)
if(CLICE_TARGET_TRIPLE MATCHES "linux")
list(APPEND LLVM_SETUP_ARGS "--target-platform" "Linux")
elseif(CLICE_TARGET_TRIPLE MATCHES "darwin")
list(APPEND LLVM_SETUP_ARGS "--target-platform" "macosx")
elseif(CLICE_TARGET_TRIPLE MATCHES "windows")
list(APPEND LLVM_SETUP_ARGS "--target-platform" "Windows")
endif()
if(CLICE_TARGET_TRIPLE MATCHES "^aarch64")
list(APPEND LLVM_SETUP_ARGS "--target-arch" "arm64")
elseif(CLICE_TARGET_TRIPLE MATCHES "^x86_64")
list(APPEND LLVM_SETUP_ARGS "--target-arch" "x64")
endif()
endif()
execute_process(
COMMAND "${Python3_EXECUTABLE}" "${LLVM_SETUP_SCRIPT}" ${LLVM_SETUP_ARGS}
RESULT_VARIABLE LLVM_SETUP_RESULT
@@ -71,29 +116,6 @@ function(setup_llvm LLVM_VERSION)
clangSerialization
clangTidy
clangTidyUtils
clangTidyAndroidModule
clangTidyAbseilModule
clangTidyAlteraModule
clangTidyBoostModule
clangTidyBugproneModule
clangTidyCERTModule
clangTidyConcurrencyModule
clangTidyCppCoreGuidelinesModule
clangTidyDarwinModule
clangTidyFuchsiaModule
clangTidyGoogleModule
clangTidyHICPPModule
clangTidyLinuxKernelModule
clangTidyLLVMModule
clangTidyLLVMLibcModule
clangTidyMiscModule
clangTidyModernizeModule
clangTidyObjCModule
clangTidyOpenMPModule
clangTidyPerformanceModule
clangTidyPortabilityModule
clangTidyReadabilityModule
clangTidyZirconModule
clangTooling
clangToolingCore
clangToolingInclusions

View File

@@ -1,7 +1,7 @@
include_guard()
include(${CMAKE_CURRENT_LIST_DIR}/llvm.cmake)
setup_llvm("21.1.4+r1")
setup_llvm("21.1.8")
# install dependencies
include(FetchContent)
@@ -41,8 +41,7 @@ set(FLATBUFFERS_BUILD_FLATHASH OFF CACHE BOOL "" FORCE)
FetchContent_Declare(
kotatsu
GIT_REPOSITORY https://github.com/clice-io/kotatsu
GIT_TAG main
GIT_SHALLOW TRUE
GIT_TAG 73814044ce8142f4438a3028f44668675fc09fff
)
set(KOTA_ENABLE_ZEST ON)

View File

@@ -1,5 +1,29 @@
cmake_minimum_required(VERSION 3.30)
# Cross-compilation support via CLICE_TARGET_TRIPLE.
# Examples:
# -DCLICE_TARGET_TRIPLE=x86_64-apple-darwin (macOS x64 from arm64)
# -DCLICE_TARGET_TRIPLE=aarch64-linux-gnu (Linux arm64 from x64)
# -DCLICE_TARGET_TRIPLE=aarch64-pc-windows-msvc (Windows arm64 from x64)
if(DEFINED CLICE_TARGET_TRIPLE)
if(CLICE_TARGET_TRIPLE MATCHES "^x86_64-apple-darwin")
set(CMAKE_OSX_ARCHITECTURES "x86_64" CACHE STRING "")
elseif(CLICE_TARGET_TRIPLE MATCHES "^aarch64-.*linux")
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
set(CMAKE_C_COMPILER_TARGET "aarch64-linux-gnu" CACHE STRING "")
set(CMAKE_CXX_COMPILER_TARGET "aarch64-linux-gnu" CACHE STRING "")
if(DEFINED ENV{CONDA_PREFIX} AND NOT DEFINED CMAKE_SYSROOT)
set(CMAKE_SYSROOT "$ENV{CONDA_PREFIX}/aarch64-conda-linux-gnu/sysroot" CACHE PATH "")
endif()
elseif(CLICE_TARGET_TRIPLE MATCHES "^aarch64-.*-windows")
set(CMAKE_SYSTEM_NAME Windows)
set(CMAKE_SYSTEM_PROCESSOR ARM64)
set(CMAKE_C_COMPILER_TARGET "aarch64-pc-windows-msvc" CACHE STRING "")
set(CMAKE_CXX_COMPILER_TARGET "aarch64-pc-windows-msvc" CACHE STRING "")
endif()
endif()
set(CMAKE_C_COMPILER clang CACHE STRING "")
set(CMAKE_CXX_COMPILER clang++ CACHE STRING "")

View File

@@ -1,83 +1,142 @@
[
{
"version": "21.1.4+r1",
"version": "21.1.8",
"filename": "aarch64-linux-gnu-releasedbg-lto.tar.xz",
"sha256": "f3444ee840b50933c23656cbee7c4d010e752ac55ca66095b97f7c0e997b13b5",
"lto": true,
"asan": false,
"platform": "linux",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"filename": "aarch64-linux-gnu-releasedbg.tar.xz",
"sha256": "b9012bf059e4d8673fb564b5780e5fc78c6a2e47f5cc6a39f444d1879b42dd2a",
"lto": false,
"asan": false,
"platform": "linux",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"filename": "aarch64-windows-msvc-releasedbg-lto.tar.xz",
"sha256": "8870d16141ba7f9ea12f5147b8d91329abbbaa4376cd4576667dd323d896dd08",
"lto": true,
"asan": false,
"platform": "windows",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"filename": "aarch64-windows-msvc-releasedbg.tar.xz",
"sha256": "ad394e79ec85dd40f942671bb0342ffe54a103eb2baabacb773999d57d80134b",
"lto": false,
"asan": false,
"platform": "windows",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"filename": "arm64-macos-clang-debug-asan.tar.xz",
"sha256": "7da4b7d63edefecaf11773e7e701c575140d1a07329bbbb038673b6ee4516ff5",
"sha256": "b02d20e4f7294ee33f49a09dfdd765b3b44135e003ef50e3a760aeee39e3f993",
"lto": false,
"asan": true,
"platform": "macosx",
"arch": "arm64",
"build_type": "Debug"
},
{
"version": "21.1.4+r1",
"version": "21.1.8",
"filename": "arm64-macos-clang-releasedbg-lto.tar.xz",
"sha256": "300455b169448f9f01ae95e3bc269f489558a4ca3955e3032171cc75feca0e30",
"sha256": "e40c21eb0d0b91d9d4ab31212a5cb01ea46707f5c29839414567857e4147604d",
"lto": true,
"asan": false,
"platform": "macosx",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.4+r1",
"version": "21.1.8",
"filename": "arm64-macos-clang-releasedbg.tar.xz",
"sha256": "9abfc6cd65b957d734ffb97610a634fb4a66d3fbe0fcfb5a1c9124ef693c1495",
"sha256": "e1b01de34f0edfd41c118e4981a93afb35556ae369597e864f4a393db623b926",
"lto": false,
"asan": false,
"platform": "macosx",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.4+r1",
"version": "21.1.8",
"filename": "x64-linux-gnu-debug-asan.tar.xz",
"sha256": "c1ad3ec476911596a842ac67dd9c9c9475ce9f0a77b81101d3c801840292e7bc",
"sha256": "76bb82d822b5377fb5e0fac8abcfba125142e6a0acc02bb36d1fa1532a268646",
"lto": false,
"asan": true,
"platform": "linux",
"arch": "x64",
"build_type": "Debug"
},
{
"version": "21.1.4+r1",
"version": "21.1.8",
"filename": "x64-linux-gnu-releasedbg-lto.tar.xz",
"sha256": "8a869c2184d139dbba704e2d712e7a68336458ad2d70622b3eb906c3e3511e54",
"sha256": "32f5edddec1e689124f045b586fb402ae30febc05203af7391b088bc8494cd53",
"lto": true,
"asan": false,
"platform": "linux",
"arch": "x64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.4+r1",
"version": "21.1.8",
"filename": "x64-linux-gnu-releasedbg.tar.xz",
"sha256": "552bab86f715d4f2c027f07eaaf5b3d6b8e430af0b74b470142f3f00da4feec6",
"sha256": "8ba3c84f23a2a81a86c54780754a61adf99048aa2ac0dc9b9708d0f842d553de",
"lto": false,
"asan": false,
"platform": "linux",
"arch": "x64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.4+r1",
"filename": "x64-windows-msvc-debug-asan.tar.xz",
"sha256": "093667a493d336c22ff3c604c5f1fea2a7d2c927c1179cec44e9a03726906ac1",
"lto": false,
"asan": true,
"platform": "windows",
"build_type": "Debug"
"version": "21.1.8",
"filename": "x64-macos-clang-releasedbg-lto.tar.xz",
"sha256": "97e81d6296896d7237f118f728d05291707b9e4e5791e07ce4be8aee0517505d",
"lto": true,
"asan": false,
"platform": "macosx",
"arch": "x64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.4+r1",
"version": "21.1.8",
"filename": "x64-macos-clang-releasedbg.tar.xz",
"sha256": "53c13f8e1082fa2fe2f9c05303de48cb3133bf5f24271f4b3062f1dec578159c",
"lto": false,
"asan": false,
"platform": "macosx",
"arch": "x64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"filename": "x64-windows-msvc-releasedbg-lto.tar.xz",
"sha256": "010539e85621dc3c6ecf359d899feb4075aeca5d0bba6625cdbec0e570e79129",
"sha256": "16bcf0e4cbc3d2b1204edd619a3837004dacea28eeff0a101c8d0212f936427d",
"lto": true,
"asan": false,
"platform": "windows",
"arch": "x64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.4+r1",
"version": "21.1.8",
"filename": "x64-windows-msvc-releasedbg.tar.xz",
"sha256": "f473c09fbea10053fac00be409d75dc228d4a38bcbc5e4aeb58b56a4b0dde78e",
"sha256": "81d31fad05e200726c8178314b0b2045c947483dddd8cb974f4c376ae5f441fa",
"lto": false,
"asan": false,
"platform": "windows",
"arch": "x64",
"build_type": "RelWithDebInfo"
}
]

View File

@@ -153,7 +153,7 @@ String values support `${workspace}` substitution.
## IPC Protocol
The master and workers communicate using custom RPC messages defined in `src/server/protocol.h`. Each message type has a `RequestTraits` or `NotificationTraits` specialization that defines the method name and result type.
The master and workers communicate using custom RPC messages defined in `src/server/protocol/`. Each message type has a `RequestTraits` or `NotificationTraits` specialization that defines the method name and result type.
### Stateful Worker Messages

5486
pixi.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -14,17 +14,24 @@ readme = "README.md"
documentation = "https://docs.clice.io/clice/"
repository = "https://github.com/clice-io/clice"
channels = ["conda-forge"]
platforms = ["win-64", "linux-64", "osx-arm64"]
platforms = ["win-64", "linux-64", "osx-arm64", "osx-64", "linux-aarch64", "win-arm64"]
[environments]
default = ["build", "test"]
package = ["build", "test", "package"]
cross-macos-x64 = ["build", "package", "cross-macos-x64"]
cross-linux-aarch64 = ["build", "package", "cross-linux-aarch64"]
cross-windows-arm64 = ["build", "package", "cross-windows-arm64"]
node = ["node"]
format = ["format"]
test-run = ["test"]
# ============================================================================== #
# DEPENDENCIES #
# ============================================================================== #
[feature.build]
platforms = ["win-64", "linux-64", "osx-arm64", "osx-64", "linux-aarch64"]
[feature.build.dependencies]
python = ">=3.13"
cmake = ">=3.30"
@@ -35,6 +42,7 @@ lld = "==20.1.8"
llvm-tools = "==20.1.8"
clang-tools = "==20.1.8"
compiler-rt = "==20.1.8"
flatbuffers = "==25.9.23"
[feature.build.target.win-64.dependencies]
sccache = "*"
@@ -54,24 +62,69 @@ scripts = ["scripts/activate_linux.sh"]
[feature.build.target.win-64.activation]
scripts = ["scripts/activate_asan.bat"]
# macOS x64 (from arm64): clang natively supports cross-arch, no extra deps.
[feature.cross-macos-x64.target.osx-arm64.dependencies]
[feature.cross-macos-x64.target.osx-arm64.activation]
scripts = ["scripts/activate_cross_macos.sh"]
# Linux aarch64 (from x64): needs aarch64 sysroot and cross gcc for libstdc++.
[feature.cross-linux-aarch64.target.linux-64.dependencies]
sysroot_linux-aarch64 = "==2.17"
gcc_linux-aarch64 = "==14.2.0"
gxx_linux-aarch64 = "==14.2.0"
[feature.cross-linux-aarch64.target.linux-64.activation]
scripts = ["scripts/activate_cross_linux.sh"]
# Windows arm64 (from x64): Windows SDK on CI already includes ARM64 libs.
[feature.cross-windows-arm64.target.win-64.dependencies]
[feature.cross-windows-arm64.target.win-64.activation]
scripts = ["scripts/activate_cross_windows.bat"]
[feature.test.dependencies]
python = ">=3.13"
# On macOS, the system Apple clang emits vendor-specific flags that upstream
# LLVM cannot parse. Providing upstream clang + lld in PATH prevents
# fallback to /usr/bin/clang++ and satisfies toolchain.cmake's -fuse-ld=lld.
[feature.test.target.osx-64.dependencies]
clang = "==20.1.8"
clangxx = "==20.1.8"
lld = "==20.1.8"
[feature.test.target.osx-arm64.dependencies]
clang = "==20.1.8"
clangxx = "==20.1.8"
lld = "==20.1.8"
[feature.test.pypi-dependencies]
pytest = "*"
pytest-asyncio = ">=1.1.0"
pytest-timeout = "*"
pygls = ">=2.0.0"
lsprotocol = ">=2024.0.0"
[feature.package.dependencies]
xz = ">=5.8.1,<6"
[feature.package.tasks.package]
[feature.package.tasks.package-config]
args = [{ arg = "type", default = "RelWithDebInfo" }]
cmd = """
cmake -B build/RelWithDebInfo -G Ninja \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
cmake -B build/{{ type }} -G Ninja \
-DCMAKE_BUILD_TYPE={{ type }} \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchain.cmake \
-DCLICE_RELEASE=ON && \
cmake --build build/RelWithDebInfo
-DCLICE_RELEASE=ON
"""
[feature.package.tasks.package]
args = [{ arg = "type", default = "RelWithDebInfo" }]
depends-on = [
{ task = "package-config", args = ["{{ type }}"] },
{ task = "cmake-build", args = ["{{ type }}"] },
]
# ============================================================================== #
# CMAKE #
# ============================================================================== #
@@ -79,14 +132,13 @@ cmake --build build/RelWithDebInfo
args = [
{ arg = "type", default = "RelWithDebInfo" },
{ arg = "ci", default = "OFF" },
{ arg = "extra", default = "" },
]
cmd = """
cmake -B build/{{ type }} -G Ninja \
-DCMAKE_BUILD_TYPE={{ type }} \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchain.cmake \
-DCLICE_ENABLE_TEST=ON \
-DCLICE_CI_ENVIRONMENT={{ ci }} {{extra}} \
-DCLICE_CI_ENVIRONMENT={{ ci }}
"""
[feature.build.tasks.cmake-build]
@@ -97,10 +149,9 @@ cmd = "cmake --build build/{{ type }}"
args = [
{ arg = "type", default = "RelWithDebInfo" },
{ arg = "ci", default = "OFF" },
{ arg = "extra", default = "" },
]
depends-on = [
{ task = "cmake-config", args = ["{{ type }}", "{{ ci }}", "{{extra}}"] },
{ task = "cmake-config", args = ["{{ type }}", "{{ ci }}"] },
{ task = "cmake-build", args = ["{{ type }}"] },
]
@@ -108,15 +159,15 @@ depends-on = [
args = [{ arg = "type", default = "RelWithDebInfo" }]
depends-on = [{ task = "lint-cpp", args = ["{{ type }}"] }]
[feature.build.tasks.unit-test]
[feature.test.tasks.unit-test]
args = [{ arg = "type", default = "RelWithDebInfo" }]
cmd = './build/{{ type }}/bin/unit_tests --test-dir="./tests/data"'
cmd = './build/{{ type }}/bin/unit_tests --test-dir="./tests/data" --snapshot-dir="./tests/snapshots" --corpus-dir="./tests/corpus" --verbose'
[feature.test.tasks.integration-test]
args = [{ arg = "type", default = "RelWithDebInfo" }]
cmd = """
pytest -s --log-cli-level=INFO tests/integration \
--executable=./build/{{ type }}/bin/clice
pytest -s --log-cli-level=INFO --timeout=300 --timeout-method=thread \
tests/integration --executable=./build/{{ type }}/bin/clice
"""
[feature.test.tasks.smoke-test]
@@ -131,6 +182,7 @@ args = [{ arg = "type", default = "RelWithDebInfo" }]
depends-on = [
{ task = "unit-test", args = ["{{ type }}"] },
{ task = "integration-test", args = ["{{ type }}"] },
{ task = "smoke-test", args = ["{{ type }}"] },
]
# ============================================================================== #
@@ -152,9 +204,14 @@ gh workflow run upload-llvm.yml \
args = ["file_name"]
cmd = ["scripts/delete-artifacts.bash", "{{ file_name }}"]
[dependencies]
# ============================================================================== #
# DOCS & VSCODE EXTENSION #
# ============================================================================== #
[feature.node]
platforms = ["win-64", "linux-64", "osx-arm64", "osx-64", "linux-aarch64"]
[feature.node.dependencies]
nodejs = ">=20"
pnpm = "*"
@@ -180,6 +237,9 @@ outputs = ["editors/vscode/node_modules/.modules.yaml"]
# ============================================================================== #
# FORMAT #
# ============================================================================== #
[feature.format]
platforms = ["win-64", "linux-64", "osx-arm64", "osx-64", "linux-aarch64"]
[feature.format.dependencies]
ruff = "*"
tombi = "*"
@@ -197,7 +257,7 @@ format-markdown = "fd -H -e md -x prettier --write"
format-json = "fd -H -e json -E package-lock.json -x prettier --write"
format-toml = "fd -H -e toml -x tombi format"
format-yaml = """
fd -H -e yaml -e yml -E pnpm-lock.yaml -x prettier --write && \
fd -H -e yaml -e yml -E pnpm-lock.yaml -E '*.snap.yml' -x prettier --write && \
fd -H "^\\.clang-(format|tidy)$" -x prettier --write --parser yaml
"""
format = { depends-on = [

View File

@@ -0,0 +1,12 @@
#!/bin/sh
# Clear conda cross-gcc flags so host x86_64 paths don't leak into the
# aarch64 build. conda's gcc_linux-aarch64 activation sets
# CFLAGS/CXXFLAGS/CPPFLAGS/LDFLAGS with -isystem/-L pointing at $CONDA_PREFIX
# (x86_64 host paths). LIBRARY_PATH from ld_impl_linux-64 likewise points at
# host libs. Empty-string export reliably overrides conda-installed values
# regardless of whether pixi sources or calls this script.
export CFLAGS=
export CXXFLAGS=
export CPPFLAGS=
export LDFLAGS=
export LIBRARY_PATH=

View File

@@ -0,0 +1,8 @@
#!/bin/sh
# Clear conda host flags so arm64 host paths don't leak into the x86_64-macos
# cross build. See scripts/activate_cross_linux.sh for rationale.
export CFLAGS=
export CXXFLAGS=
export CPPFLAGS=
export LDFLAGS=
export LIBRARY_PATH=

View File

@@ -0,0 +1,8 @@
@echo off
REM Clear conda host flags so host x64 paths don't leak into the aarch64-windows
REM cross build. See scripts/activate_cross_linux.sh for rationale.
set "CFLAGS="
set "CXXFLAGS="
set "CPPFLAGS="
set "LDFLAGS="
set "LIBRARY_PATH="

View File

@@ -4,6 +4,7 @@ import subprocess
import shutil
import argparse
import os
import json
from pathlib import Path
@@ -22,6 +23,66 @@ def normalize_mode(value: str) -> str:
)
def build_native_tools(project_root: Path, build_dir: Path) -> Path:
"""Build native host tablegen tools for cross-compilation.
When cross-compiling LLVM, build tools like llvm-tblgen must run on the
host but would otherwise be compiled for the target architecture. This
function performs a minimal native build and returns the bin directory
containing host-runnable executables.
"""
native_dir = build_dir.parent / f"{build_dir.name}-native-tools"
native_dir.mkdir(exist_ok=True)
source_dir = project_root / "llvm"
cmake_args = [
"-G",
"Ninja",
"-DCMAKE_BUILD_TYPE=Release",
"-DLLVM_ENABLE_PROJECTS=clang;clang-tools-extra",
"-DLLVM_TARGETS_TO_BUILD=Native",
"-DLLVM_DISABLE_ASSEMBLY_FILES=ON",
"-DCMAKE_C_FLAGS=-w",
"-DCMAKE_CXX_FLAGS=-w",
]
if sys.platform == "win32":
cmake_args += [
"-DCMAKE_C_COMPILER=clang-cl",
"-DCMAKE_CXX_COMPILER=clang-cl",
]
else:
cmake_args += [
"-DCMAKE_C_COMPILER=clang",
"-DCMAKE_CXX_COMPILER=clang++",
]
print(f"\nConfiguring native host tools in {native_dir}...")
subprocess.check_call(
["cmake", "-S", str(source_dir), "-B", str(native_dir)] + cmake_args
)
required_tools = ["llvm-tblgen", "llvm-min-tblgen", "clang-tblgen"]
optional_tools = ["clang-tidy-confusable-chars-gen"]
for tool in required_tools:
print(f"Building native {tool}...")
subprocess.check_call(["cmake", "--build", str(native_dir), "--target", tool])
for tool in optional_tools:
try:
print(f"Building native {tool} (optional)...")
subprocess.check_call(
["cmake", "--build", str(native_dir), "--target", tool]
)
except subprocess.CalledProcessError:
print(f" {tool} not available, skipping.")
bin_dir = native_dir / "bin"
print(f"Native host tools ready in {bin_dir}")
return bin_dir
def main():
parser = argparse.ArgumentParser(
description="Build LLVM with specific configurations."
@@ -48,6 +109,10 @@ def main():
"--build-dir",
help="Custom build directory (relative to project root or absolute)",
)
parser.add_argument(
"--target-triple",
help="Cross-compilation target triple (e.g. x86_64-apple-darwin, aarch64-linux-gnu, aarch64-pc-windows-msvc)",
)
args = parser.parse_args()
@@ -85,118 +150,46 @@ def main():
print("--- Configuration ---")
print(f"Mode: {args.mode}")
print(f"LTO: {args.lto}")
print(f"Target Triple: {args.target_triple or '(native)'}")
print(f"Root: {project_root}")
print(f"Build Dir: {build_dir}")
print(f"Install Prefix: {install_prefix}")
print(f"Toolchain: {toolchain_file}")
print("---------------------")
llvm_distribution_components = [
"LLVMDemangle",
"LLVMSupport",
"LLVMCore",
"LLVMOption",
"LLVMBinaryFormat",
"LLVMMC",
"LLVMMCParser",
"LLVMObject",
"LLVMProfileData",
"LLVMBitReader",
"LLVMBitstreamReader",
"LLVMRemarks",
"LLVMObjectYAML",
"LLVMAggressiveInstCombine",
"LLVMInstCombine",
"LLVMIRReader",
"LLVMTextAPI",
"LLVMSymbolize",
"LLVMDebugInfoDWARF",
"LLVMDebugInfoDWARFLowLevel",
"LLVMDebugInfoCodeView",
"LLVMDebugInfoGSYM",
"LLVMDebugInfoPDB",
"LLVMDebugInfoBTF",
"LLVMDebugInfoMSF",
"LLVMAsmParser",
"LLVMTargetParser",
"LLVMTransformUtils",
"LLVMAnalysis",
"LLVMScalarOpts",
"LLVMFrontendHLSL",
"LLVMFrontendOpenMP",
"LLVMFrontendOffloading",
"LLVMFrontendAtomic",
"LLVMFrontendDirective",
"LLVMWindowsDriver",
"clangIndex",
"clangAPINotes",
"clangAST",
"clangASTMatchers",
"clangBasic",
"clangDriver",
"clangFormat",
"clangFrontend",
"clangLex",
"clangParse",
"clangSema",
"clangSerialization",
"clangRewrite",
"clangAnalysis",
"clangEdit",
"clangSupport",
"clangStaticAnalyzerCore",
"clangStaticAnalyzerFrontend",
"clangTidy",
"clangTidyUtils",
"clangTidyAndroidModule",
"clangTidyAbseilModule",
"clangTidyAlteraModule",
"clangTidyBoostModule",
"clangTidyBugproneModule",
"clangTidyCERTModule",
"clangTidyConcurrencyModule",
"clangTidyCppCoreGuidelinesModule",
"clangTidyDarwinModule",
"clangTidyFuchsiaModule",
"clangTidyGoogleModule",
"clangTidyHICPPModule",
"clangTidyLinuxKernelModule",
"clangTidyLLVMModule",
"clangTidyLLVMLibcModule",
"clangTidyMiscModule",
"clangTidyModernizeModule",
"clangTidyObjCModule",
"clangTidyOpenMPModule",
"clangTidyPerformanceModule",
"clangTidyPortabilityModule",
"clangTidyReadabilityModule",
"clangTidyZirconModule",
"clangTooling",
"clangToolingCore",
"clangToolingInclusions",
"clangToolingInclusionsStdlib",
"clangToolingSyntax",
"clangToolingRefactoring",
"clangTransformer",
"clangCrossTU",
"clangAnalysisFlowSensitive",
"clangAnalysisFlowSensitiveModels",
"clangStaticAnalyzerCheckers",
"clangIncludeCleaner",
"llvm-headers",
"clang-headers",
"clang-tidy-headers",
"clang-resource-headers",
]
components_path = Path(__file__).resolve().parent / "llvm-components.json"
with components_path.open() as f:
llvm_distribution_components = json.load(f)["components"]
components_joined = ";".join(llvm_distribution_components)
cmake_args = [
"-G",
"Ninja",
f"-DCMAKE_TOOLCHAIN_FILE={toolchain_file.as_posix()}",
f"-DCMAKE_INSTALL_PREFIX={install_prefix}",
"-DCMAKE_C_FLAGS=-w",
"-DCMAKE_CXX_FLAGS=-w",
]
if sys.platform == "win32":
# Use clang-cl (MSVC driver) on Windows so that LLVM's CMake
# generates correct MSVC-style linker flags for LTO, etc.
c_flags = "-w"
if args.target_triple:
c_flags += f" --target={args.target_triple}"
cmake_args += [
"-DCMAKE_C_COMPILER=clang-cl",
"-DCMAKE_CXX_COMPILER=clang-cl",
f"-DCMAKE_C_FLAGS={c_flags}",
f"-DCMAKE_CXX_FLAGS={c_flags}",
"-DLLVM_USE_LINKER=lld-link",
]
else:
cmake_args += [
f"-DCMAKE_TOOLCHAIN_FILE={toolchain_file.as_posix()}",
"-DCMAKE_C_FLAGS=-w",
"-DCMAKE_CXX_FLAGS=-w",
"-DLLVM_USE_LINKER=lld",
]
cmake_args += [
"-DLLVM_ENABLE_ZLIB=OFF",
"-DLLVM_ENABLE_ZSTD=OFF",
"-DLLVM_ENABLE_LIBXML2=OFF",
@@ -231,7 +224,6 @@ def main():
"-DCMAKE_JOB_POOL_LINK=console",
"-DLLVM_ENABLE_PROJECTS=clang;clang-tools-extra",
"-DLLVM_TARGETS_TO_BUILD=all",
"-DLLVM_USE_LINKER=lld",
"-DLLVM_DISABLE_ASSEMBLY_FILES=ON",
# Distribution
f"-DLLVM_DISTRIBUTION_COMPONENTS={components_joined}",
@@ -256,8 +248,10 @@ def main():
is_shared = "OFF"
if args.mode == "Debug":
cmake_args.append("-DCMAKE_BUILD_TYPE=Debug")
cmake_args.append("-DLLVM_USE_SANITIZER=Address")
is_shared = "ON"
# ASAN is incompatible with -MDd on Windows (clang-cl), skip it there.
if sys.platform != "win32":
cmake_args.append("-DLLVM_USE_SANITIZER=Address")
is_shared = "ON"
elif args.mode == "Release":
cmake_args.append("-DCMAKE_BUILD_TYPE=Release")
elif args.mode == "RelWithDebInfo":
@@ -272,6 +266,24 @@ def main():
else:
cmake_args.append("-DLLVM_ENABLE_LTO=OFF")
if args.target_triple:
cmake_args.append(f"-DCLICE_TARGET_TRIPLE={args.target_triple}")
cmake_args.append(f"-DLLVM_HOST_TRIPLE={args.target_triple}")
# When cross-compiling, clear conda's host-platform flags so they
# don't leak into the target build (e.g. -L pointing to x86_64 libs).
# This must happen before the native-tools build too so we don't
# contaminate the native configure with target-arch link flags.
for var in ["LIBRARY_PATH", "LDFLAGS", "CFLAGS", "CXXFLAGS", "CPPFLAGS"]:
os.environ.pop(var, None)
# Cross-compilation needs native host tools (tablegen, etc.) that can
# run on the build machine. macOS handles this transparently via
# Rosetta 2, but Linux and Windows require a separate native build.
if sys.platform != "darwin":
native_bin_dir = build_native_tools(project_root, build_dir)
cmake_args.append(f"-DLLVM_NATIVE_TOOL_DIR={native_bin_dir}")
build_dir.mkdir(exist_ok=True)
print(f"\nConfiguring in {build_dir}...")

View File

@@ -0,0 +1,99 @@
{
"components": [
"LLVMDemangle",
"LLVMSupport",
"LLVMCore",
"LLVMOption",
"LLVMBinaryFormat",
"LLVMMC",
"LLVMMCParser",
"LLVMObject",
"LLVMProfileData",
"LLVMBitReader",
"LLVMBitstreamReader",
"LLVMRemarks",
"LLVMObjectYAML",
"LLVMAggressiveInstCombine",
"LLVMInstCombine",
"LLVMIRReader",
"LLVMTextAPI",
"LLVMSymbolize",
"LLVMDebugInfoDWARF",
"LLVMDebugInfoDWARFLowLevel",
"LLVMDebugInfoCodeView",
"LLVMDebugInfoGSYM",
"LLVMDebugInfoPDB",
"LLVMDebugInfoBTF",
"LLVMDebugInfoMSF",
"LLVMAsmParser",
"LLVMTargetParser",
"LLVMTransformUtils",
"LLVMAnalysis",
"LLVMScalarOpts",
"LLVMFrontendHLSL",
"LLVMFrontendOpenMP",
"LLVMFrontendOffloading",
"LLVMFrontendAtomic",
"LLVMFrontendDirective",
"LLVMWindowsDriver",
"clangIndex",
"clangAPINotes",
"clangAST",
"clangASTMatchers",
"clangBasic",
"clangDriver",
"clangFormat",
"clangFrontend",
"clangLex",
"clangParse",
"clangSema",
"clangSerialization",
"clangRewrite",
"clangAnalysis",
"clangEdit",
"clangSupport",
"clangStaticAnalyzerCore",
"clangStaticAnalyzerFrontend",
"clangTidy",
"clangTidyUtils",
"clangTidyAndroidModule",
"clangTidyAbseilModule",
"clangTidyAlteraModule",
"clangTidyBoostModule",
"clangTidyBugproneModule",
"clangTidyCERTModule",
"clangTidyConcurrencyModule",
"clangTidyCppCoreGuidelinesModule",
"clangTidyDarwinModule",
"clangTidyFuchsiaModule",
"clangTidyGoogleModule",
"clangTidyHICPPModule",
"clangTidyLinuxKernelModule",
"clangTidyLLVMModule",
"clangTidyLLVMLibcModule",
"clangTidyMiscModule",
"clangTidyModernizeModule",
"clangTidyObjCModule",
"clangTidyOpenMPModule",
"clangTidyPerformanceModule",
"clangTidyPortabilityModule",
"clangTidyReadabilityModule",
"clangTidyZirconModule",
"clangTooling",
"clangToolingCore",
"clangToolingInclusions",
"clangToolingInclusionsStdlib",
"clangToolingSyntax",
"clangToolingRefactoring",
"clangTransformer",
"clangCrossTU",
"clangAnalysisFlowSensitive",
"clangAnalysisFlowSensitiveModels",
"clangStaticAnalyzerCheckers",
"clangIncludeCleaner",
"llvm-headers",
"clang-headers",
"clang-tidy-headers",
"clang-resource-headers"
]
}

View File

@@ -16,7 +16,10 @@ import subprocess
import time
from datetime import datetime, timezone
from pathlib import Path
from typing import Iterable, List, Optional
from typing import Iterable, List, Optional, Set
LLVM_COMPONENTS_FILE = Path(__file__).with_name("llvm-components.json")
def parse_args() -> argparse.Namespace:
@@ -102,12 +105,33 @@ def run_build(build_dir: Path) -> bool:
return False
def protected_library_names() -> Set[str]:
data = json.loads(LLVM_COMPONENTS_FILE.read_text())
components = data.get("components", [])
if not isinstance(components, list):
raise ValueError(f"{LLVM_COMPONENTS_FILE} missing 'components' list")
names: Set[str] = set()
for component in components:
if not isinstance(component, str):
continue
if not (component.startswith("clangTidy") and component.endswith("Module")):
continue
names.add(f"lib{component}.a")
names.add(f"{component}.lib")
return names
def candidate_files(install_dir: Path) -> Iterable[Path]:
if not install_dir.is_dir():
raise FileNotFoundError(f"lib dir not found: {install_dir}")
protected = protected_library_names()
for path in sorted(install_dir.iterdir()):
if not path.is_file():
continue
if path.name in protected:
print(f"Keeping protected clang-tidy module library: {path.name}")
continue
if path.suffix.lower() in {".a", ".lib"}:
yield path
else:
@@ -156,7 +180,11 @@ def apply_manifest(manifest: Path, install_dir: Path) -> None:
removed = data.get("removed", [])
if not isinstance(removed, list):
raise ValueError("Manifest missing 'removed' list")
protected = protected_library_names()
for name in removed:
if name in protected:
print(f"Keeping protected clang-tidy module library from manifest: {name}")
continue
target = install_dir / name
if target.exists():
print(f"Deleting {target}")

View File

@@ -40,23 +40,52 @@ def detect_platform() -> str:
raise RuntimeError(f"Unsupported platform: {plat}")
def detect_arch() -> str:
import platform
machine = platform.machine().lower()
if machine in ("x86_64", "amd64"):
return "x64"
if machine in ("aarch64", "arm64"):
return "arm64"
raise RuntimeError(f"Unsupported architecture: {machine}")
def pick_artifact(
manifest: list[dict], version: str, build_type: str, is_lto: bool, platform: str
manifest: list[dict],
version: str,
build_type: str,
is_lto: bool,
platform: str,
arch: str,
) -> dict:
base_version = version.split("+", 1)[0]
saw_missing_arch = False
for entry in manifest:
if entry.get("version") != version:
continue
if entry.get("platform") != platform.lower():
continue
entry_arch = entry.get("arch")
if entry_arch is None:
saw_missing_arch = True
continue
if entry_arch != arch:
continue
if entry.get("build_type") != build_type:
continue
if bool(entry.get("lto")) != is_lto:
continue
return entry
if saw_missing_arch:
raise RuntimeError(
f"Manifest contains entries without an 'arch' field for version={base_version}, "
f"platform={platform}. The manifest format changed to require explicit "
f"architectures; regenerate it via scripts/update-llvm-version.py."
)
raise RuntimeError(
f"No matching LLVM artifact in manifest for version={base_version}, platform={platform}, "
f"build_type={build_type}, lto={is_lto}"
f"arch={arch}, build_type={build_type}, lto={is_lto}"
)
@@ -264,6 +293,14 @@ def main() -> None:
parser.add_argument("--install-path")
parser.add_argument("--enable-lto", action="store_true")
parser.add_argument("--offline", action="store_true")
parser.add_argument(
"--target-platform",
help="Override platform for cross-compilation (e.g. macosx, linux, windows)",
)
parser.add_argument(
"--target-arch",
help="Override architecture for cross-compilation (e.g. x64, arm64)",
)
parser.add_argument("--output", required=True)
args = parser.parse_args()
@@ -275,8 +312,11 @@ def main() -> None:
)
token = os.environ.get("GH_TOKEN") or os.environ.get("GITHUB_TOKEN")
build_type = args.build_type
platform_name = detect_platform()
log(f"Platform detected: {platform_name}, normalized build type: {build_type}")
platform_name = args.target_platform if args.target_platform else detect_platform()
arch_name = args.target_arch if args.target_arch else detect_arch()
log(
f"Platform: {platform_name}, arch: {arch_name}, normalized build type: {build_type}"
)
manifest = read_manifest(Path(args.manifest))
binary_dir = Path(args.binary_dir).resolve()
@@ -304,7 +344,12 @@ def main() -> None:
if install_path is None:
needs_install = True
artifact = pick_artifact(
manifest, args.version, build_type, args.enable_lto, platform_name
manifest,
args.version,
build_type,
args.enable_lto,
platform_name,
arch_name,
)
log(f"Selected artifact: {artifact.get('filename')} for download")
filename = artifact["filename"]
@@ -317,7 +362,12 @@ def main() -> None:
install_path = install_root
elif needs_install:
artifact = pick_artifact(
manifest, args.version, build_type, args.enable_lto, platform_name
manifest,
args.version,
build_type,
args.enable_lto,
platform_name,
arch_name,
)
log(f"Selected artifact: {artifact.get('filename')} for download")
filename = artifact["filename"]

162
scripts/update-llvm-version.py Executable file
View File

@@ -0,0 +1,162 @@
#!/usr/bin/env python3
import argparse
import json
import re
import sys
from pathlib import Path
def copy_manifest(src: Path, dest: Path) -> None:
text = src.read_text(encoding="utf-8")
try:
data = json.loads(text)
except json.JSONDecodeError as err:
print(f"Error: {src} is not valid JSON: {err}", file=sys.stderr)
sys.exit(1)
if not isinstance(data, list) or len(data) == 0:
print(f"Error: {src} must be a non-empty JSON array", file=sys.stderr)
sys.exit(1)
dest.parent.mkdir(parents=True, exist_ok=True)
with dest.open("w", encoding="utf-8") as handle:
json.dump(data, handle, indent=2)
handle.write("\n")
print(f"Copied manifest: {src} -> {dest} ({len(data)} entries)")
def update_package_cmake(path: Path, version: str) -> None:
text = path.read_text(encoding="utf-8")
pattern = r'setup_llvm\("[^"]*"\)'
matches = re.findall(pattern, text)
if len(matches) == 0:
print(f"Error: no setup_llvm(...) call found in {path}", file=sys.stderr)
sys.exit(1)
if len(matches) > 1:
print(
f"Error: expected exactly 1 setup_llvm(...) call in {path}, "
f"found {len(matches)}",
file=sys.stderr,
)
sys.exit(1)
old_call = matches[0]
new_call = f'setup_llvm("{version}")'
if old_call == new_call:
print(f"Version in {path} is already {version}, no change needed")
return
updated = text.replace(old_call, new_call)
path.write_text(updated, encoding="utf-8")
print(f"Updated {path}: {old_call} -> {new_call}")
def check_package_cmake(path: Path) -> None:
"""Verify package.cmake has exactly one setup_llvm(...) call that the
update script can rewrite. Used by CI to catch drift before the next bump."""
text = path.read_text(encoding="utf-8")
matches = re.findall(r'setup_llvm\("[^"]*"\)', text)
if len(matches) == 0:
print(f"Error: no setup_llvm(...) call found in {path}", file=sys.stderr)
sys.exit(1)
if len(matches) > 1:
print(
f"Error: expected exactly 1 setup_llvm(...) call in {path}, "
f"found {len(matches)}: {matches}",
file=sys.stderr,
)
sys.exit(1)
print(f"OK: {path} has a single setup_llvm(...) call: {matches[0]}")
def check_manifest(path: Path) -> None:
"""Verify the manifest is a well-formed non-empty array with required fields."""
try:
data = json.loads(path.read_text(encoding="utf-8"))
except json.JSONDecodeError as err:
print(f"Error: {path} is not valid JSON: {err}", file=sys.stderr)
sys.exit(1)
if not isinstance(data, list) or len(data) == 0:
print(f"Error: {path} must be a non-empty JSON array", file=sys.stderr)
sys.exit(1)
required = ("version", "platform", "arch", "build_type", "filename", "sha256")
for idx, entry in enumerate(data):
missing = [k for k in required if k not in entry]
if missing:
print(
f"Error: {path} entry {idx} is missing fields: {missing}",
file=sys.stderr,
)
sys.exit(1)
print(f"OK: {path} has {len(data)} well-formed entries")
def main() -> None:
parser = argparse.ArgumentParser(
description="Update LLVM version references in the clice project."
)
parser.add_argument(
"--check",
action="store_true",
help="Validate existing state without modifying files (for CI drift checks)",
)
parser.add_argument(
"--version",
help="New LLVM version string (e.g. 21.2.0); required unless --check",
)
parser.add_argument(
"--manifest-src",
help="Path to the source llvm-manifest.json; required unless --check",
)
parser.add_argument(
"--manifest-dest",
required=True,
help="Path to destination manifest (e.g. config/llvm-manifest.json)",
)
parser.add_argument(
"--package-cmake",
required=True,
help="Path to cmake/package.cmake",
)
args = parser.parse_args()
manifest_dest = Path(args.manifest_dest)
package_cmake = Path(args.package_cmake)
if not package_cmake.is_file():
print(f"Error: package.cmake not found: {package_cmake}", file=sys.stderr)
sys.exit(1)
if args.check:
check_package_cmake(package_cmake)
check_manifest(manifest_dest)
print("Done (check mode).")
return
if not args.version or not args.manifest_src:
print(
"Error: --version and --manifest-src are required unless --check is set",
file=sys.stderr,
)
sys.exit(1)
manifest_src = Path(args.manifest_src)
if not manifest_src.is_file():
print(f"Error: manifest source not found: {manifest_src}", file=sys.stderr)
sys.exit(1)
copy_manifest(manifest_src, manifest_dest)
update_package_cmake(package_cmake, args.version)
print("Done.")
if __name__ == "__main__":
main()

View File

@@ -27,6 +27,15 @@ def parse_platform(name: str) -> str:
raise ValueError(f"Unable to determine platform from filename: {name}")
def parse_arch(name: str) -> str:
lowered = name.lower()
if lowered.startswith("aarch64-") or lowered.startswith("arm64-"):
return "arm64"
if lowered.startswith("x64-") or lowered.startswith("x86_64-"):
return "x64"
raise ValueError(f"Unable to determine arch from filename: {name}")
def parse_build_type(name: str) -> str:
lowered = name.lower()
if "debug" in lowered:
@@ -43,6 +52,7 @@ def build_metadata_entry(path: Path, version: str) -> dict:
"lto": "-lto" in filename.lower(),
"asan": "-asan" in filename.lower(),
"platform": parse_platform(filename),
"arch": parse_arch(filename),
"build_type": parse_build_type(filename),
}

View File

@@ -0,0 +1,163 @@
#!/usr/bin/env python3
"""
Validate the LLVM distribution component list against the actual LLVM source tree.
Scans the LLVM source for CMake library targets and compares them against
a components JSON file to detect stale or misspelled entries.
"""
import argparse
import difflib
import json
import re
import sys
from pathlib import Path
# CMake function calls that define library targets.
# The captured group uses [^\s)]+ to grab the target name without
# trailing parentheses or whitespace.
LLVM_LIB_PATTERNS = [
re.compile(r"add_llvm_component_library\(\s*([^\s)]+)"),
re.compile(r"add_llvm_library\(\s*([^\s)]+)"),
]
CLANG_LIB_PATTERNS = [
re.compile(r"add_clang_library\(\s*([^\s)]+)"),
]
# Header-only / custom install targets.
HEADER_PATTERNS = [
re.compile(r"add_llvm_install_targets\(\s*([^\s)]+)"),
re.compile(r"add_custom_target\(\s*([^\s)]+)"),
re.compile(r"add_library\(\s*([^\s)]+)"),
]
# Targets we recognise as header-only distribution components.
KNOWN_HEADER_TARGETS = {
"llvm-headers",
"clang-headers",
"clang-tidy-headers",
"clang-resource-headers",
}
def scan_targets(directory: Path, patterns: list[re.Pattern]) -> set[str]:
"""Recursively scan *directory* for CMakeLists.txt files and extract target names."""
targets: set[str] = set()
if not directory.is_dir():
return targets
for cmake_file in directory.rglob("CMakeLists.txt"):
text = cmake_file.read_text(errors="replace")
for pattern in patterns:
for match in pattern.finditer(text):
targets.add(match.group(1))
return targets
def scan_header_targets(llvm_src: Path) -> set[str]:
"""Scan for well-known header / custom-install targets across the tree."""
found: set[str] = set()
for cmake_file in llvm_src.rglob("CMakeLists.txt"):
text = cmake_file.read_text(errors="replace")
for pattern in HEADER_PATTERNS:
for match in pattern.finditer(text):
name = match.group(1)
if name in KNOWN_HEADER_TARGETS:
found.add(name)
return found
def collect_source_targets(llvm_src: Path) -> set[str]:
"""Return the full set of library / header targets found in the LLVM source tree."""
targets: set[str] = set()
targets |= scan_targets(llvm_src / "llvm" / "lib", LLVM_LIB_PATTERNS)
targets |= scan_targets(llvm_src / "clang" / "lib", CLANG_LIB_PATTERNS)
targets |= scan_targets(llvm_src / "clang-tools-extra", CLANG_LIB_PATTERNS)
targets |= scan_header_targets(llvm_src)
return targets
def load_components(path: Path) -> list[str]:
with path.open("r", encoding="utf-8") as handle:
data = json.load(handle)
if isinstance(data, dict):
data = data.get("components", [])
if not isinstance(data, list) or not data:
print(f"Error: no component list found in {path}", file=sys.stderr)
sys.exit(1)
return data
def main() -> None:
parser = argparse.ArgumentParser(
description="Validate LLVM distribution components against the source tree."
)
parser.add_argument(
"--llvm-src",
required=True,
help="Path to the llvm-project source root",
)
parser.add_argument(
"--components-file",
required=True,
help="Path to llvm-components.json",
)
args = parser.parse_args()
llvm_src = Path(args.llvm_src).expanduser().resolve()
components_file = Path(args.components_file).expanduser().resolve()
if not llvm_src.is_dir():
print(f"Error: LLVM source directory not found: {llvm_src}")
sys.exit(1)
if not (llvm_src / "llvm" / "CMakeLists.txt").exists():
print(f"Error: {llvm_src} does not look like an llvm-project root.")
sys.exit(1)
if not components_file.is_file():
print(f"Error: components file not found: {components_file}")
sys.exit(1)
components = load_components(components_file)
source_targets = collect_source_targets(llvm_src)
print(f"Found {len(source_targets)} targets in LLVM source tree")
print(f"Components file lists {len(components)} entries")
# Check for components that are missing from the source tree.
missing: list[tuple[str, list[str]]] = []
for name in components:
if name not in source_targets:
suggestions = difflib.get_close_matches(
name, source_targets, n=3, cutoff=0.6
)
missing.append((name, suggestions))
if missing:
print(f"\nError: {len(missing)} component(s) not found in the source tree:\n")
for name, suggestions in missing:
print(f" - {name}")
if suggestions:
print(f" Did you mean: {', '.join(suggestions)}?")
sys.exit(1)
# Warn about source targets not present in the component list.
component_set = set(components)
new_targets = sorted(source_targets - component_set - KNOWN_HEADER_TARGETS)
# Filter to targets that follow LLVM/Clang naming conventions to reduce noise.
noteworthy = [t for t in new_targets if t.startswith(("LLVM", "clang", "Clang"))]
if noteworthy:
print(
f"\nWarning: {len(noteworthy)} target(s) in source not listed in components:"
)
for name in noteworthy:
print(f" + {name}")
print("\nAll components validated successfully.")
sys.exit(0)
if __name__ == "__main__":
main()

View File

@@ -4,33 +4,33 @@
#include <print>
#include <string>
#include "server/master_server.h"
#include "server/stateful_worker.h"
#include "server/stateless_worker.h"
#include "server/service/agentic.h"
#include "server/service/master_server.h"
#include "server/worker/stateful_worker.h"
#include "server/worker/stateless_worker.h"
#include "support/logging.h"
#include "kota/async/async.h"
#include "kota/deco/deco.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/peer.h"
#include "kota/ipc/recording_transport.h"
#include "kota/ipc/transport.h"
namespace clice {
using kota::deco::decl::KVStyle;
struct Options {
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Running mode: pipe, socket, stateless-worker, stateful-worker",
required = false)
DecoKV(
style = KVStyle::JoinedOrSeparate,
help =
"Running mode: pipe, socket, daemon, relay, agentic, stateless-worker, stateful-worker",
required = false)
<std::string> mode;
DecoKV(style = KVStyle::JoinedOrSeparate, help = "Socket mode address", required = false)
<std::string> host = "127.0.0.1";
DecoKV(style = KVStyle::JoinedOrSeparate, help = "Socket mode port", required = false)
<int> port = 50051;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Agentic TCP port (0 = disabled)",
required = false)
<int> port = 0;
DecoKV(style = KVStyle::JoinedOrSeparate,
names = {"--log-level", "--log-level="},
@@ -43,6 +43,50 @@ struct Options {
required = false)
<std::string> record;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "File path for agentic queries",
required = false)
<std::string> path;
DecoKV(
style = KVStyle::JoinedOrSeparate,
help =
"Agentic method (compileCommand, symbolSearch, definition, references, "
"documentSymbols, readSymbol, callGraph, typeHierarchy, projectFiles, "
"lint, fileDeps, impactAnalysis, status, shutdown)",
required = false)
<std::string> method;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Symbol name for agentic queries",
required = false)
<std::string> name;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Search query for symbolSearch",
required = false)
<std::string> query;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Line number for position-based lookup",
required = false)
<int> line;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Direction: callers/callees or supertypes/subtypes",
required = false)
<std::string> direction;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Unix domain socket path for daemon mode",
required = false)
<std::string> socket;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Workspace root directory for daemon mode",
required = false)
<std::string> workspace;
// Internal options (passed from master to worker processes)
DecoKV(style = KVStyle::JoinedOrSeparate,
names = {"--worker-memory-limit", "--worker-memory-limit="},
@@ -68,9 +112,6 @@ struct Options {
int main(int argc, const char** argv) {
#ifndef _WIN32
// On POSIX systems, ignore SIGPIPE so that writing to a closed pipe
// (e.g. when the LSP client disconnects) returns EPIPE instead of
// killing the process. This is standard practice for pipe-based servers.
signal(SIGPIPE, SIG_IGN);
#endif
@@ -110,8 +151,6 @@ int main(int argc, const char** argv) {
return 1;
}
std::string self_path = argv[0];
auto& mode = *opts.mode;
auto worker_name = opts.worker_name.value_or("");
@@ -131,77 +170,51 @@ int main(int argc, const char** argv) {
log_dir);
}
if(mode == "pipe") {
clice::logging::stderr_logger("master", clice::logging::options);
kota::event_loop loop;
auto transport = kota::ipc::StreamTransport::open_stdio(loop);
if(!transport) {
LOG_ERROR("failed to open stdio transport");
return 1;
}
std::unique_ptr<kota::ipc::Transport> final_transport = std::move(*transport);
if(opts.record.has_value()) {
final_transport =
std::make_unique<kota::ipc::RecordingTransport>(std::move(final_transport),
*opts.record);
}
kota::ipc::JsonPeer peer(loop, std::move(final_transport));
clice::MasterServer server(loop, peer, std::move(self_path));
server.register_handlers();
loop.schedule(peer.run());
loop.run();
return 0;
if(mode == "pipe" || mode == "socket") {
clice::ServerOptions server_opts;
server_opts.mode = mode;
server_opts.host = opts.host.value_or("127.0.0.1");
server_opts.port = opts.port.value_or(0);
server_opts.self_path = argv[0];
server_opts.record = opts.record.value_or("");
return clice::run_server_mode(server_opts);
}
if(mode == "socket") {
clice::logging::stderr_logger("master", clice::logging::options);
kota::event_loop loop;
auto host = opts.host.value_or("127.0.0.1");
auto port = opts.port.value_or(50051);
auto acceptor = kota::tcp::listen(host, port, {}, loop);
if(!acceptor) {
LOG_ERROR("failed to listen on {}:{}", host, port);
if(mode == "daemon") {
auto workspace = opts.workspace.value_or("");
if(workspace.empty()) {
LOG_ERROR("--workspace is required for daemon mode");
return 1;
}
LOG_INFO("Listening on {}:{} ...", host, port);
clice::DaemonOptions daemon_opts;
daemon_opts.socket_path = opts.socket.value_or("");
daemon_opts.workspace = std::move(workspace);
daemon_opts.self_path = argv[0];
return clice::run_daemon_mode(daemon_opts);
}
auto task = [&]() -> kota::task<> {
auto client = co_await acceptor->accept();
if(!client.has_value()) {
LOG_ERROR("failed to accept connection");
loop.stop();
co_return;
}
if(mode == "agentic") {
auto port = opts.port.value_or(0);
if(port <= 0) {
LOG_ERROR("--port is required for agentic mode");
return 1;
}
clice::AgenticQueryOptions aq;
aq.host = opts.host.value_or("127.0.0.1");
aq.port = port;
aq.method = opts.method.value_or("compileCommand");
aq.path = opts.path.value_or("");
aq.name = opts.name.value_or("");
aq.query = opts.query.value_or("");
aq.line = opts.line.value_or(0);
aq.direction = opts.direction.value_or("");
return clice::run_agentic_mode(aq);
}
LOG_INFO("Client connected");
std::unique_ptr<kota::ipc::Transport> transport =
std::make_unique<kota::ipc::StreamTransport>(std::move(client.value()));
if(opts.record.has_value()) {
transport = std::make_unique<kota::ipc::RecordingTransport>(std::move(transport),
*opts.record);
}
kota::ipc::JsonPeer peer(loop, std::move(transport));
clice::MasterServer server(loop, peer, std::string(self_path));
server.register_handlers();
co_await peer.run();
peer.close();
loop.stop();
};
loop.schedule(task());
loop.run();
return 0;
if(mode == "relay") {
auto socket = opts.socket.value_or("");
return clice::run_relay_mode(socket);
}
LOG_ERROR("unknown mode '{}'", mode);

View File

@@ -9,6 +9,7 @@
#include "support/logging.h"
#include "kota/meta/enum.h"
#include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/ScopeExit.h"
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/FileSystem.h"
@@ -17,6 +18,7 @@
#include "llvm/TargetParser/Host.h"
#include "clang/Driver/Compilation.h"
#include "clang/Driver/Driver.h"
#include "clang/Driver/Options.h"
#include "clang/Driver/Tool.h"
#ifndef _WIN32
@@ -470,11 +472,32 @@ std::vector<const char*> query_clang_toolchain(const QueryParams& params) {
continue;
}
for(auto arg: args) {
if(arg == "-###"sv) {
// FIXME: the system compiler may be newer than our embedded LLVM,
// producing cc1 flags we don't recognize. Filter them out here.
// Long-term we should unify the command pipeline so the driver
// version always matches the embedded LLVM.
auto& table = clang::driver::getDriverOptTable();
auto cc1_args = llvm::ArrayRef(args).drop_front(2);
unsigned missing_index = 0, missing_count = 0;
auto parsed = table.ParseArgs(cc1_args, missing_index, missing_count);
llvm::DenseSet<unsigned> unknown_indices;
for(auto* a: parsed) {
if(a->getOption().getKind() == llvm::opt::Option::UnknownClass) {
unknown_indices.insert(a->getIndex());
}
}
result.emplace_back(params.callback(args[0]));
result.emplace_back(params.callback(args[1]));
for(unsigned i = 0; i < cc1_args.size(); ++i) {
if(unknown_indices.contains(i)) {
continue;
}
result.emplace_back(params.callback(arg));
if(cc1_args[i] == "-###"sv) {
continue;
}
result.emplace_back(params.callback(cc1_args[i]));
}
}
}

View File

@@ -219,9 +219,10 @@ public:
auto CreateASTConsumer(clang::CompilerInstance& instance, llvm::StringRef file)
-> std::unique_ptr<clang::ASTConsumer> final {
return std::make_unique<ProxyASTConsumer>(
WrapperFrontendAction::CreateASTConsumer(instance, file),
unit);
auto consumer = WrapperFrontendAction::CreateASTConsumer(instance, file);
if(!consumer)
return nullptr;
return std::make_unique<ProxyASTConsumer>(std::move(consumer), unit);
}
/// Make this public.
@@ -417,6 +418,8 @@ CompilationUnit compile(CompilationParams& params, PCMInfo& out) {
}
CompilationUnit complete(CompilationParams& params, clang::CodeCompleteConsumer* consumer) {
params.kind = CompilationKind::Completion;
auto& [file, offset] = params.completion;
/// The location of clang is 1-1 based.

View File

@@ -65,7 +65,7 @@ struct PCMInfo : ModuleInfo {
struct CompilationParams {
/// The kind of this compilation.
CompilationKind kind;
CompilationKind kind = CompilationKind::Content;
/// Whether to run clang-tidy.
bool clang_tidy = false;

View File

@@ -81,7 +81,8 @@ auto CompilationUnitRef::file_offset(clang::SourceLocation location) -> std::uin
}
auto CompilationUnitRef::file_path(clang::FileID fid) -> llvm::StringRef {
assert(fid.isValid() && "Invalid fid");
if(!fid.isValid())
return {};
if(auto it = self->path_cache.find(fid); it != self->path_cache.end()) {
return it->second;
}

View File

@@ -12,6 +12,10 @@
#include "clang-tidy/ClangTidyDiagnosticConsumer.h"
#include "clang-tidy/ClangTidyModuleRegistry.h"
#include "clang-tidy/ClangTidyOptions.h"
#ifdef CLICE_HAS_CLANG_TIDY_MODULES
#define CLANG_TIDY_DISABLE_STATIC_ANALYZER_CHECKS
#include "clang-tidy/ClangTidyForceLinker.h"
#endif
namespace clice::tidy {
@@ -92,15 +96,11 @@ tidy::ClangTidyOptions create_options() {
// include-cleaner is directly integrated in IncludeCleaner.cpp
"-misc-include-cleaner",
// ----- False Positives -----
// Check relies on seeing ifndef/define/endif directives,
// clangd doesn't replay those when using a preamble.
"-llvm-header-guard",
"-modernize-macro-to-enum",
// ----- Crashing Checks -----
// Check can choke on invalid (intermediate) c++
// code, which is often the case when clangd
// tries to build an AST.

View File

@@ -296,7 +296,8 @@ public:
llvm::StringRef overload_key,
llvm::StringRef signature = {},
llvm::StringRef return_type = {},
bool is_snippet = false) {
bool is_snippet = false,
bool is_deprecated = false) {
if(label.empty()) {
return;
}
@@ -327,6 +328,9 @@ public:
}
item.label_details = std::move(details);
}
if(is_deprecated) {
item.tags = std::vector{protocol::CompletionItemTag::Deprecated};
}
overloads.push_back({
.item = std::move(item),
.score = *score,
@@ -355,6 +359,9 @@ public:
}
item.label_details = std::move(details);
}
if(is_deprecated) {
item.tags = std::vector{protocol::CompletionItemTag::Deprecated};
}
collected.push_back(std::move(item));
};
@@ -431,13 +438,15 @@ public:
bool has_snippet = !snippet.empty();
auto insert = has_snippet ? llvm::StringRef(snippet) : llvm::StringRef(label);
bool deprecated = candidate.Availability == CXAvailability_Deprecated;
try_add(label,
kind,
insert,
qualified_name.str(),
signature,
return_type,
has_snippet);
has_snippet,
deprecated);
break;
}
}

View File

@@ -93,18 +93,9 @@ auto symbol_detail(clang::ASTContext& context, const clang::NamedDecl& decl) ->
return detail;
}
struct InternalSymbol {
std::string name;
std::string detail;
SymbolKind kind = SymbolKind::Invalid;
LocalSourceRange range;
LocalSourceRange selection_range;
std::vector<InternalSymbol> children;
};
struct SymbolFrame {
std::vector<InternalSymbol> symbols;
std::vector<InternalSymbol>* cursor = &symbols;
std::vector<DocumentSymbol> symbols;
std::vector<DocumentSymbol>* cursor = &symbols;
};
class DocumentSymbolCollector : public FilteredASTVisitor<DocumentSymbolCollector> {
@@ -143,7 +134,7 @@ public:
return ok;
}
auto collect() -> std::vector<InternalSymbol> {
auto collect() -> std::vector<DocumentSymbol> {
TraverseDecl(unit.tu());
return std::move(result.symbols);
}
@@ -174,8 +165,8 @@ private:
SymbolFrame result;
};
void sort_symbols(std::vector<InternalSymbol>& symbols) {
std::ranges::sort(symbols, [](const InternalSymbol& lhs, const InternalSymbol& rhs) {
void sort_symbols(std::vector<DocumentSymbol>& symbols) {
std::ranges::sort(symbols, [](const DocumentSymbol& lhs, const DocumentSymbol& rhs) {
if(lhs.range.begin != rhs.range.begin) {
return lhs.range.begin < rhs.range.begin;
}
@@ -187,7 +178,7 @@ void sort_symbols(std::vector<InternalSymbol>& symbols) {
}
}
auto to_protocol_symbol(const InternalSymbol& symbol, const PositionMapper& converter)
auto to_protocol_symbol(const DocumentSymbol& symbol, const PositionMapper& converter)
-> protocol::DocumentSymbol {
protocol::DocumentSymbol result{
.name = symbol.name,
@@ -215,10 +206,15 @@ auto to_protocol_symbol(const InternalSymbol& symbol, const PositionMapper& conv
} // namespace
auto document_symbols(CompilationUnitRef unit) -> std::vector<DocumentSymbol> {
auto result = DocumentSymbolCollector(unit).collect();
sort_symbols(result);
return result;
}
auto document_symbols(CompilationUnitRef unit, PositionEncoding encoding)
-> std::vector<protocol::DocumentSymbol> {
auto internal = DocumentSymbolCollector(unit).collect();
sort_symbols(internal);
auto internal = document_symbols(unit);
PositionMapper converter(unit.interested_content(), encoding);
std::vector<protocol::DocumentSymbol> symbols;

View File

@@ -7,6 +7,7 @@
#include "compile/compilation.h"
#include "compile/compilation_unit.h"
#include "semantic/symbol_kind.h"
#include "kota/ipc/lsp/position.h"
#include "kota/ipc/lsp/protocol.h"
@@ -59,18 +60,66 @@ struct InlayHintsOptions {
struct SignatureHelpOptions {};
auto semantic_tokens(CompilationUnitRef unit, PositionEncoding encoding = PositionEncoding::UTF16)
struct SemanticToken {
LocalSourceRange range;
SymbolKind kind = SymbolKind::Invalid;
std::uint32_t modifiers = 0;
};
struct FoldingRange {
LocalSourceRange range;
std::optional<protocol::FoldingRangeKind> kind;
std::string collapsed_text;
};
struct DocumentSymbol {
std::string name;
std::string detail;
SymbolKind kind = SymbolKind::Invalid;
LocalSourceRange range;
LocalSourceRange selection_range;
std::vector<DocumentSymbol> children;
};
enum class HintCategory : std::uint8_t {
Parameter,
DefaultArgument,
Type,
Designator,
BlockEnd,
};
struct InlayHint {
std::uint32_t offset = 0;
HintCategory kind = HintCategory::Type;
std::string label;
bool padding_left = false;
bool padding_right = false;
};
auto semantic_tokens(CompilationUnitRef unit) -> std::vector<SemanticToken>;
auto semantic_tokens(CompilationUnitRef unit, PositionEncoding encoding)
-> protocol::SemanticTokens;
auto folding_ranges(CompilationUnitRef unit) -> std::vector<FoldingRange>;
auto folding_ranges(CompilationUnitRef unit, PositionEncoding encoding)
-> std::vector<protocol::FoldingRange>;
auto document_symbols(CompilationUnitRef unit) -> std::vector<DocumentSymbol>;
auto document_symbols(CompilationUnitRef unit, PositionEncoding encoding)
-> std::vector<protocol::DocumentSymbol>;
auto inlay_hints(CompilationUnitRef unit,
LocalSourceRange target,
const InlayHintsOptions& options = {}) -> std::vector<InlayHint>;
auto inlay_hints(CompilationUnitRef unit,
LocalSourceRange target,
const InlayHintsOptions& options,
PositionEncoding encoding) -> std::vector<protocol::InlayHint>;
auto document_links(CompilationUnitRef unit, PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::DocumentLink>;
auto document_symbols(CompilationUnitRef unit, PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::DocumentSymbol>;
auto folding_ranges(CompilationUnitRef unit, PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::FoldingRange>;
auto diagnostics(CompilationUnitRef unit, PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::Diagnostic>;
@@ -89,12 +138,6 @@ auto hover(CompilationUnitRef unit,
const HoverOptions& options = {},
PositionEncoding encoding = PositionEncoding::UTF16) -> std::optional<protocol::Hover>;
auto inlay_hints(CompilationUnitRef unit,
LocalSourceRange target,
const InlayHintsOptions& options = {},
PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::InlayHint>;
auto signature_help(CompilationParams& params, const SignatureHelpOptions& options = {})
-> protocol::SignatureHelp;

View File

@@ -53,12 +53,6 @@ auto to_kind(FoldingKind kind) -> protocol::FoldingRangeKind {
return protocol::FoldingRangeKind(protocol::FoldingRangeKind::region);
}
struct RawFoldingRange {
LocalSourceRange range;
std::optional<protocol::FoldingRangeKind> kind;
std::string collapsed_text;
};
class FoldingRangeCollector : public FilteredASTVisitor<FoldingRangeCollector> {
public:
explicit FoldingRangeCollector(CompilationUnitRef unit) : FilteredASTVisitor(unit, true) {}
@@ -185,7 +179,7 @@ public:
return true;
}
auto collect() -> std::vector<RawFoldingRange> {
auto collect() -> std::vector<FoldingRange> {
TraverseDecl(unit.tu());
auto directives_it = unit.directives().find(unit.interested_file());
@@ -193,7 +187,7 @@ public:
collect_directives(directives_it->second);
}
std::ranges::sort(ranges, [](const RawFoldingRange& lhs, const RawFoldingRange& rhs) {
std::ranges::sort(ranges, [](const FoldingRange& lhs, const FoldingRange& rhs) {
if(lhs.range.begin != rhs.range.begin) {
return lhs.range.begin < rhs.range.begin;
}
@@ -343,14 +337,18 @@ private:
}
private:
std::vector<RawFoldingRange> ranges;
std::vector<FoldingRange> ranges;
};
} // namespace
auto folding_ranges(CompilationUnitRef unit) -> std::vector<FoldingRange> {
return FoldingRangeCollector(unit).collect();
}
auto folding_ranges(CompilationUnitRef unit, PositionEncoding encoding)
-> std::vector<protocol::FoldingRange> {
auto collected = FoldingRangeCollector(unit).collect();
auto collected = folding_ranges(unit);
PositionMapper converter(unit.interested_content(), encoding);
std::vector<protocol::FoldingRange> result;

View File

@@ -49,7 +49,7 @@ auto document_format(llvm::StringRef file,
range ? tooling::Range(range->begin, range->length()) : tooling::Range(0, content.size());
auto replacements = format_content(file, content, selection);
if(!replacements) {
LOG_INFO("Fail to format for {}\n{}", file, replacements.error());
LOG_WARN("Failed to format {}: {}", file, replacements.error());
return edits;
}

View File

@@ -26,22 +26,6 @@ using llvm::dyn_cast_or_null;
// For now, inlay hints are always anchored at the left or right of their range.
enum class HintSide { Left, Right };
enum class HintCategory : std::uint8_t {
Parameter,
DefaultArgument,
Type,
Designator,
BlockEnd,
};
struct RawInlayHint {
std::uint32_t offset = 0;
HintCategory kind = HintCategory::Type;
std::string label;
bool padding_left = false;
bool padding_right = false;
};
bool is_expanded_from_param_pack(const clang::ParmVarDecl* param) {
return ast::underlying_pack_type(param) != nullptr;
}
@@ -123,7 +107,7 @@ struct Callee {
class Builder {
public:
Builder(std::vector<RawInlayHint>& result,
Builder(std::vector<InlayHint>& result,
CompilationUnitRef unit,
LocalSourceRange restrict_range,
const InlayHintsOptions& options) :
@@ -499,7 +483,7 @@ public:
bool pad_left = prefix.consume_front(" ");
bool pad_right = suffix.consume_back(" ");
RawInlayHint hint{
InlayHint hint{
.offset = offset,
.kind = kind,
.label = (prefix + label + suffix).str(),
@@ -554,7 +538,7 @@ public:
}
private:
std::vector<RawInlayHint>& result;
std::vector<InlayHint>& result;
CompilationUnitRef unit;
LocalSourceRange restrict_range;
const InlayHintsOptions& options;
@@ -913,36 +897,43 @@ private:
} // namespace
auto inlay_hints(CompilationUnitRef unit,
LocalSourceRange target,
const InlayHintsOptions& options,
PositionEncoding encoding) -> std::vector<protocol::InlayHint> {
auto inlay_hints(CompilationUnitRef unit, LocalSourceRange target, const InlayHintsOptions& options)
-> std::vector<InlayHint> {
if(!options.enabled) {
return {};
}
std::vector<RawInlayHint> raw_hints;
std::vector<InlayHint> raw_hints;
Builder builder(raw_hints, unit, target, options);
Visitor visitor(builder, unit, target, options);
visitor.TraverseDecl(unit.tu());
std::ranges::sort(raw_hints, [](const RawInlayHint& lhs, const RawInlayHint& rhs) {
std::ranges::sort(raw_hints, [](const InlayHint& lhs, const InlayHint& rhs) {
return std::tie(lhs.offset, lhs.label, lhs.kind, lhs.padding_left, lhs.padding_right) <
std::tie(rhs.offset, rhs.label, rhs.kind, rhs.padding_left, rhs.padding_right);
});
auto unique_begin =
std::ranges::unique(raw_hints, [](const RawInlayHint& lhs, const RawInlayHint& rhs) {
std::ranges::unique(raw_hints, [](const InlayHint& lhs, const InlayHint& rhs) {
return lhs.offset == rhs.offset && lhs.kind == rhs.kind && lhs.label == rhs.label &&
lhs.padding_left == rhs.padding_left && lhs.padding_right == rhs.padding_right;
});
raw_hints.erase(unique_begin.begin(), unique_begin.end());
return raw_hints;
}
auto inlay_hints(CompilationUnitRef unit,
LocalSourceRange target,
const InlayHintsOptions& options,
PositionEncoding encoding) -> std::vector<protocol::InlayHint> {
auto collected = inlay_hints(unit, target, options);
PositionMapper converter(unit.interested_content(), encoding);
std::vector<protocol::InlayHint> hints;
hints.reserve(raw_hints.size());
hints.reserve(collected.size());
for(const auto& hint: raw_hints) {
for(const auto& hint: collected) {
protocol::InlayHint out{
.position = *converter.to_position(hint.offset),
.label = hint.label,

View File

@@ -18,12 +18,6 @@ namespace clice::feature {
namespace {
struct RawToken {
LocalSourceRange range;
SymbolKind kind = SymbolKind::Invalid;
std::uint32_t modifiers = 0;
};
void add_modifier(std::uint32_t& modifiers, SymbolModifiers::Kind kind) {
modifiers |= SymbolModifiers::to_mask(kind);
}
@@ -40,6 +34,34 @@ bool is_dependent(const clang::Decl* D) {
return isa<clang::UnresolvedUsingValueDecl>(D);
}
/// Whether a declaration name is backed by source text that should be highlighted.
bool can_highlight_name(clang::DeclarationName name) {
switch(name.getNameKind()) {
case clang::DeclarationName::Identifier: {
auto* info = name.getAsIdentifierInfo();
return info && !info->getName().empty();
}
case clang::DeclarationName::CXXConstructorName:
case clang::DeclarationName::CXXDestructorName: {
return true;
}
case clang::DeclarationName::CXXConversionFunctionName:
case clang::DeclarationName::CXXOperatorName:
case clang::DeclarationName::CXXDeductionGuideName:
case clang::DeclarationName::CXXLiteralOperatorName:
case clang::DeclarationName::CXXUsingDirective:
case clang::DeclarationName::ObjCZeroArgSelector:
case clang::DeclarationName::ObjCOneArgSelector:
case clang::DeclarationName::ObjCMultiArgSelector: {
return false;
}
}
std::unreachable();
}
/// Returns true if `decl` is considered to be from a default/system library.
/// This currently checks the systemness of the file by include type, although
/// different heuristics may be used in the future (e.g. sysroot paths).
@@ -166,7 +188,7 @@ class SemanticTokensCollector : public SemanticVisitor<SemanticTokensCollector>
public:
explicit SemanticTokensCollector(CompilationUnitRef unit) : SemanticVisitor(unit, true) {}
auto collect() -> std::vector<RawToken> {
auto collect() -> std::vector<SemanticToken> {
highlight_lexical(unit.interested_file());
run();
highlight_modules();
@@ -177,6 +199,10 @@ public:
void handleDeclOccurrence(const clang::NamedDecl* decl,
RelationKind relation,
clang::SourceLocation location) {
if(relation.isReference() && !can_highlight_name(decl->getDeclName())) {
return;
}
std::uint32_t modifiers = 0;
if(relation.is_one_of(RelationKind::Definition)) {
// todo: clangd add both Declaration and Definition modifiers for definitions.
@@ -398,7 +424,7 @@ private:
}
}
static void resolve_conflict(RawToken& last, const RawToken& current) {
static void resolve_conflict(SemanticToken& last, const SemanticToken& current) {
if(last.kind == SymbolKind::Conflict) {
return;
}
@@ -414,14 +440,14 @@ private:
}
void merge_tokens() {
std::ranges::sort(tokens, [](const RawToken& lhs, const RawToken& rhs) {
std::ranges::sort(tokens, [](const SemanticToken& lhs, const SemanticToken& rhs) {
if(lhs.range.begin != rhs.range.begin) {
return lhs.range.begin < rhs.range.begin;
}
return lhs.range.end < rhs.range.end;
});
std::vector<RawToken> merged;
std::vector<SemanticToken> merged;
merged.reserve(tokens.size());
for(const auto& token: tokens) {
@@ -448,7 +474,7 @@ private:
}
public:
std::vector<RawToken> tokens;
std::vector<SemanticToken> tokens;
};
class SemanticTokenEncoder {
@@ -458,7 +484,7 @@ public:
protocol::SemanticTokens& output) :
content(content), converter(content, encoding), output(output) {}
void append(const RawToken& token) {
void append(const SemanticToken& token) {
if(!token.range.valid() || token.range.end <= token.range.begin ||
token.range.end > content.size()) {
return;
@@ -542,10 +568,14 @@ private:
} // namespace
auto semantic_tokens(CompilationUnitRef unit) -> std::vector<SemanticToken> {
SemanticTokensCollector collector(unit);
return collector.collect();
}
auto semantic_tokens(CompilationUnitRef unit, PositionEncoding encoding)
-> protocol::SemanticTokens {
SemanticTokensCollector collector(unit);
auto tokens = collector.collect();
auto tokens = semantic_tokens(unit);
protocol::SemanticTokens result;
result.data.reserve(tokens.size() * 5);

View File

@@ -308,6 +308,10 @@ const clang::NamedDecl* decl_of_impl(const void* T) {
}
auto decl_of(clang::QualType type) -> const clang::NamedDecl* {
if(type.isNull()) {
return nullptr;
}
// Strip type-sugar that wraps the underlying type without adding a decl
// (e.g. ElaboratedType for "struct Foo" vs plain "Foo").
if(auto ET = type->getAs<clang::ElaboratedType>()) {

View File

@@ -1111,8 +1111,6 @@ public:
return Base::TransformDecltypeType(TLB, TL);
}
// --- State ---
private:
clang::Sema& sema;
clang::ASTContext& context;

View File

@@ -1,4 +1,4 @@
#include "server/compile_graph.h"
#include "server/compiler/compile_graph.h"
#include <algorithm>

View File

@@ -1,4 +1,4 @@
#include "server/compiler.h"
#include "server/compiler/compiler.h"
#include <format>
#include <ranges>
@@ -6,7 +6,7 @@
#include "command/search_config.h"
#include "index/tu_index.h"
#include "server/protocol.h"
#include "server/protocol/worker.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "syntax/include_resolver.h"
@@ -28,16 +28,20 @@ using serde_raw = kota::codec::RawValue;
/// Detect whether the cursor is inside a preamble directive (include/import).
Compiler::Compiler(kota::event_loop& loop,
kota::ipc::JsonPeer& peer,
Workspace& workspace,
WorkerPool& pool,
llvm::DenseMap<std::uint32_t, Session>& sessions) :
loop(loop), peer(peer), workspace(workspace), pool(pool), sessions(sessions) {}
loop(loop), workspace(workspace), pool(pool), sessions(sessions) {}
Compiler::~Compiler() {
workspace.cancel_all();
}
kota::task<> Compiler::stop() {
compile_tasks.cancel();
co_await compile_tasks.join();
}
void Compiler::init_compile_graph() {
if(workspace.path_to_module.empty()) {
LOG_INFO("No C++20 modules detected, skipping CompileGraph");
@@ -47,8 +51,13 @@ void Compiler::init_compile_graph() {
// Lazy dependency resolver: scans a module file on demand to discover imports.
auto resolve = [this](std::uint32_t path_id) -> llvm::SmallVector<std::uint32_t> {
auto file_path = workspace.path_pool.resolve(path_id);
auto results =
workspace.cdb.lookup(file_path, {.query_toolchain = true, .suppress_logging = true});
std::vector<std::string> rule_append, rule_remove;
workspace.config.match_rules(file_path, rule_append, rule_remove);
auto results = workspace.cdb.lookup(file_path,
{.query_toolchain = true,
.suppress_logging = true,
.remove = rule_remove,
.append = rule_append});
if(results.empty())
return {};
@@ -97,7 +106,8 @@ void Compiler::init_compile_graph() {
}
auto args_hash = llvm::xxh3_64bits(llvm::StringRef(hash_input));
auto pcm_filename = std::format("{}-{:016x}.pcm", safe_module_name, args_hash);
auto pcm_path = path::join(workspace.config.cache_dir, "cache", "pcm", pcm_filename);
auto pcm_path =
path::join(workspace.config.project.cache_dir, "cache", "pcm", pcm_filename);
// Check if cached PCM is still valid.
if(auto pcm_it = workspace.pcm_cache.find(path_id); pcm_it != workspace.pcm_cache.end()) {
@@ -156,7 +166,11 @@ bool Compiler::fill_compile_args(llvm::StringRef path,
}
// 2. Normal CDB lookup for the file itself.
auto results = workspace.cdb.lookup(path, {.query_toolchain = true});
// Apply rules from config (append/remove flags based on file patterns).
std::vector<std::string> rule_append, rule_remove;
workspace.config.match_rules(path, rule_append, rule_remove);
CommandOptions opts{.query_toolchain = true, .remove = rule_remove, .append = rule_append};
auto results = workspace.cdb.lookup(path, opts);
if(!results.empty()) {
auto& cmd = results.front();
directory = cmd.resolved.directory.str();
@@ -205,7 +219,13 @@ bool Compiler::fill_header_context_args(llvm::StringRef path,
}
auto host_path = workspace.path_pool.resolve(ctx_ptr->host_path_id);
auto host_results = workspace.cdb.lookup(host_path, {.query_toolchain = true});
// Apply rules matching the HEADER path (what the user is editing) on top of
// the host's command — rules are expected to apply uniformly to every file.
std::vector<std::string> rule_append, rule_remove;
workspace.config.match_rules(path, rule_append, rule_remove);
auto host_results = workspace.cdb.lookup(
host_path,
{.query_toolchain = true, .remove = rule_remove, .append = rule_append});
if(host_results.empty()) {
LOG_WARN("fill_header_context_args: host {} has no CDB entry", host_path);
return false;
@@ -355,7 +375,7 @@ std::optional<HeaderFileContext> Compiler::resolve_header_context(std::uint32_t
// Hash the preamble and write to cache directory.
auto preamble_hash = llvm::xxh3_64bits(llvm::StringRef(preamble));
auto preamble_filename = std::format("{:016x}.h", preamble_hash);
auto preamble_dir = path::join(workspace.config.cache_dir, "header_context");
auto preamble_dir = path::join(workspace.config.project.cache_dir, "header_context");
auto preamble_path = path::join(preamble_dir, preamble_filename);
if(!llvm::sys::fs::exists(preamble_path)) {
@@ -394,6 +414,8 @@ std::string uri_to_path(const std::string& uri) {
void Compiler::publish_diagnostics(const std::string& uri,
int version,
const kota::codec::RawValue& diagnostics_json) {
if(!peer)
return;
std::vector<protocol::Diagnostic> diagnostics;
if(!diagnostics_json.empty()) {
auto status = kota::codec::json::from_json(diagnostics_json.data, diagnostics);
@@ -405,14 +427,16 @@ void Compiler::publish_diagnostics(const std::string& uri,
params.uri = uri;
params.version = version;
params.diagnostics = std::move(diagnostics);
peer.send_notification(params);
peer->send_notification(params);
}
void Compiler::clear_diagnostics(const std::string& uri) {
if(!peer)
return;
protocol::PublishDiagnosticsParams params;
params.uri = uri;
params.diagnostics = {};
peer.send_notification(params);
peer->send_notification(params);
}
kota::task<bool> Compiler::ensure_pch(Session& session,
@@ -438,7 +462,7 @@ kota::task<bool> Compiler::ensure_pch(Session& session,
auto preamble_hash = llvm::xxh3_64bits(preamble_text);
// Deterministic content-addressed PCH path.
auto pch_path = path::join(workspace.config.cache_dir,
auto pch_path = path::join(workspace.config.project.cache_dir,
"cache",
"pch",
std::format("{:016x}.pch", preamble_hash));
@@ -474,6 +498,22 @@ kota::task<bool> Compiler::ensure_pch(Session& session,
auto completion = std::make_shared<kota::event>();
workspace.pch_cache[path_id].building = completion;
if(workspace.config.project.cache_dir.empty()) {
LOG_WARN("PCH build skipped: cache_dir is not configured");
workspace.pch_cache[path_id].building.reset();
completion->set();
co_return false;
}
// Ensure the PCH cache directory exists.
auto pch_dir = path::join(workspace.config.project.cache_dir, "cache", "pch");
if(auto ec = llvm::sys::fs::create_directories(pch_dir)) {
LOG_WARN("Cannot create PCH cache dir {}: {}", pch_dir, ec.message());
workspace.pch_cache[path_id].building.reset();
completion->set();
co_return false;
}
// Build a new PCH via stateless worker.
worker::BuildParams bp;
bp.kind = worker::BuildKind::BuildPCH;
@@ -597,6 +637,102 @@ void Compiler::record_deps(Session& session, llvm::ArrayRef<std::string> deps) {
/// Called lazily by forward_query() / forward_build() before every
/// feature request (hover, semantic tokens, etc.). Guarantees that when it
/// returns true the stateful worker assigned to `path_id` holds an up-to-date
kota::task<> Compiler::run_compile(std::uint32_t pid, std::shared_ptr<Session::PendingCompile> pc) {
auto find_session = [&]() -> Session* {
auto it = sessions.find(pid);
return it != sessions.end() ? &it->second : nullptr;
};
auto* sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
auto finish_compile = [&]() {
auto* s = find_session();
if(s && s->compiling == pc) {
s->compiling.reset();
}
LOG_INFO("ensure_compiled: finish path_id={}", pid);
pc->done.set();
};
auto gen = sess->generation;
LOG_INFO("ensure_compiled: starting compile path_id={} gen={}", pid, gen);
auto file_path = std::string(workspace.path_pool.resolve(pid));
auto uri = lsp::URI::from_file_path(file_path);
std::string uri_str = uri.has_value() ? uri->str() : file_path;
worker::CompileParams params;
params.path = file_path;
params.version = sess->version;
params.text = sess->text;
params.clang_tidy = workspace.config.project.clang_tidy.value;
if(!fill_compile_args(file_path, params.directory, params.arguments, sess)) {
finish_compile();
co_return;
}
if(!co_await ensure_deps(*sess, params.directory, params.arguments, params.pch, params.pcms)) {
LOG_WARN("Dependency preparation failed for {}, skipping compile", uri_str);
finish_compile();
co_return;
}
sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
auto result = co_await pool.send_stateful(pid, params);
sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
if(sess->generation != gen) {
LOG_INFO("ensure_compiled: generation mismatch ({} vs {}) for {}",
sess->generation,
gen,
uri_str);
finish_compile();
co_return;
}
if(!result.has_value()) {
LOG_WARN("Compile failed for {}: {}", uri_str, result.error().message);
clear_diagnostics(uri_str);
finish_compile();
co_return;
}
sess->ast_dirty = false;
pc->succeeded = true;
record_deps(*sess, result.value().deps);
if(!result.value().tu_index_data.empty()) {
auto tu_index = index::TUIndex::from(result.value().tu_index_data.data());
OpenFileIndex ofi;
ofi.file_index = std::move(tu_index.main_file_index);
ofi.symbols = std::move(tu_index.symbols);
ofi.content = sess->text;
ofi.mapper.emplace(ofi.content, lsp::PositionEncoding::UTF16);
sess->file_index = std::move(ofi);
}
auto version = sess->version;
finish_compile();
publish_diagnostics(uri_str, version, result.value().diagnostics);
if(on_indexing_needed)
on_indexing_needed();
}
/// AST and diagnostics have been published to the client.
///
/// Lifecycle overview (pull-based model):
@@ -616,9 +752,9 @@ void Compiler::record_deps(Session& session, llvm::ArrayRef<std::string> deps) {
/// worker); every other file is read from disk by the compiler.
///
/// Concurrency: multiple concurrent feature requests for the same file will
/// each call ensure_compiled(). The first one launches a detached compile
/// task via loop.schedule(); subsequent ones wait on the shared event.
/// The detached task cannot be cancelled by LSP $/cancelRequest, preventing
/// each call ensure_compiled(). The first one spawns a compile task into the
/// Compiler's task_group; subsequent ones wait on the shared event.
/// The spawned task is not cancelled by LSP $/cancelRequest, preventing
/// the race where cancellation wakes all waiters and they all start compiles.
kota::task<bool> Compiler::ensure_compiled(Session& session) {
auto path_id = session.path_id;
@@ -647,124 +783,12 @@ kota::task<bool> Compiler::ensure_compiled(Session& session) {
co_return true;
}
// No compile in flight and AST is dirty — launch a detached compile task.
// The detached task is scheduled via loop.schedule() so it is NOT subject
// to LSP $/cancelRequest cancellation. This eliminates the race where
// cancellation fires the RAII guard, waking all waiters simultaneously
// and causing them all to start new compiles.
auto pending_compile = std::make_shared<Session::PendingCompile>();
session.compiling = pending_compile;
LOG_INFO("ensure_compiled: launching detached compile path_id={} gen={}",
path_id,
session.generation);
LOG_INFO("ensure_compiled: launching compile path_id={} gen={}", path_id, session.generation);
// Capture path_id by value so the detached lambda can re-lookup the session
// from the sessions map after co_await (DenseMap may invalidate pointers).
loop.schedule([](Compiler* self,
std::uint32_t pid,
std::shared_ptr<Session::PendingCompile> pc) -> kota::task<> {
// Re-lookup session from the sessions map (pointer may have been
// invalidated by DenseMap growth during co_await).
auto find_session = [&]() -> Session* {
auto it = self->sessions.find(pid);
return it != self->sessions.end() ? &it->second : nullptr;
};
auto* sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
auto finish_compile = [&]() {
auto* s = find_session();
if(s && s->compiling == pc) {
s->compiling.reset();
}
LOG_INFO("ensure_compiled: finish_compile (detached) path_id={}", pid);
pc->done.set();
};
auto gen = sess->generation;
LOG_INFO("ensure_compiled: starting compile (detached) path_id={} gen={}", pid, gen);
auto file_path = std::string(self->workspace.path_pool.resolve(pid));
auto uri = lsp::URI::from_file_path(file_path);
std::string uri_str = uri.has_value() ? uri->str() : file_path;
worker::CompileParams params;
params.path = file_path;
params.version = sess->version;
params.text = sess->text;
if(!self->fill_compile_args(file_path, params.directory, params.arguments, sess)) {
finish_compile();
co_return;
}
if(!co_await self
->ensure_deps(*sess, params.directory, params.arguments, params.pch, params.pcms)) {
LOG_WARN("Dependency preparation failed for {}, skipping compile", uri_str);
finish_compile();
co_return;
}
// Re-lookup after co_await (DenseMap may have grown).
sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
auto result = co_await self->pool.send_stateful(pid, params);
// Re-lookup after co_await.
sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
if(sess->generation != gen) {
LOG_INFO("ensure_compiled: generation mismatch ({} vs {}) for {}",
sess->generation,
gen,
uri_str);
finish_compile();
co_return;
}
if(!result.has_value()) {
LOG_WARN("Compile failed for {}: {}", uri_str, result.error().message);
self->clear_diagnostics(uri_str);
finish_compile();
co_return;
}
sess->ast_dirty = false;
pc->succeeded = true;
self->record_deps(*sess, result.value().deps);
// Store open file index from the stateful worker's TUIndex.
if(!result.value().tu_index_data.empty()) {
auto tu_index = index::TUIndex::from(result.value().tu_index_data.data());
OpenFileIndex ofi;
ofi.file_index = std::move(tu_index.main_file_index);
ofi.symbols = std::move(tu_index.symbols);
ofi.content = sess->text;
ofi.mapper.emplace(ofi.content, lsp::PositionEncoding::UTF16);
sess->file_index = std::move(ofi);
}
auto version = sess->version;
finish_compile();
// Publish diagnostics AFTER marking compile as done, so that concurrent
// forward_query() calls can proceed immediately.
self->publish_diagnostics(uri_str, version, result.value().diagnostics);
if(self->on_indexing_needed)
self->on_indexing_needed();
}(this, path_id, pending_compile));
compile_tasks.spawn(run_compile(path_id, pending_compile));
// Wait for the detached compile to finish. If this wait is cancelled
// by LSP $/cancelRequest, the detached task continues unaffected.
@@ -859,6 +883,32 @@ Compiler::RawResult Compiler::forward_build(worker::BuildKind kind,
co_return std::move(result.value().result_json);
}
Compiler::RawResult Compiler::forward_format(Session& session,
std::optional<protocol::Range> range) {
auto path_id = session.path_id;
auto path = std::string(workspace.path_pool.resolve(path_id));
worker::BuildParams wp;
wp.kind = worker::BuildKind::Format;
wp.file = path;
wp.text = session.text;
if(range) {
lsp::PositionMapper mapper(wp.text, lsp::PositionEncoding::UTF16);
auto begin = mapper.to_offset(range->start);
auto end = mapper.to_offset(range->end);
if(!begin || !end)
co_return serde_raw{"null"};
wp.format_range = {*begin, *end};
}
auto result = co_await pool.send_stateless(wp);
if(!result.has_value()) {
co_return serde_raw{"null"};
}
co_return std::move(result.value().result_json);
}
Compiler::RawResult Compiler::handle_completion(const protocol::Position& position,
Session& session) {
auto path_id = session.path_id;

View File

@@ -8,13 +8,13 @@
#include <vector>
#include "command/command.h"
#include "server/session.h"
#include "server/worker_pool.h"
#include "server/workspace.h"
#include "server/service/session.h"
#include "server/worker/worker_pool.h"
#include "server/workspace/workspace.h"
#include "syntax/completion.h"
#include "kota/async/async.h"
#include "kota/codec/raw_value.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/peer.h"
@@ -50,10 +50,14 @@ std::string uri_to_path(const std::string& uri);
class Compiler {
public:
Compiler(kota::event_loop& loop,
kota::ipc::JsonPeer& peer,
Workspace& workspace,
WorkerPool& pool,
llvm::DenseMap<std::uint32_t, Session>& sessions);
void set_peer(kota::ipc::JsonPeer* p) {
peer = p;
}
~Compiler();
void init_compile_graph();
@@ -86,6 +90,9 @@ public:
const protocol::Position& position,
Session& session);
/// Forward a formatting request to a stateless worker.
RawResult forward_format(Session& session, std::optional<protocol::Range> range = {});
/// Handle completion requests. Detects preamble context (include/import)
/// and serves those locally; delegates code completion to a stateless worker.
RawResult handle_completion(const protocol::Position& position, Session& session);
@@ -96,7 +103,12 @@ public:
/// Callback invoked when indexing should be scheduled.
std::function<void()> on_indexing_needed;
/// Cancel in-flight compile tasks and wait for them to finish.
kota::task<> stop();
private:
kota::task<> run_compile(std::uint32_t path_id, std::shared_ptr<Session::PendingCompile> pc);
kota::task<bool> ensure_deps(Session& session,
const std::string& directory,
const std::vector<std::string>& arguments,
@@ -125,10 +137,11 @@ private:
private:
kota::event_loop& loop;
kota::ipc::JsonPeer& peer;
kota::ipc::JsonPeer* peer = nullptr;
Workspace& workspace;
WorkerPool& pool;
llvm::DenseMap<std::uint32_t, Session>& sessions;
kota::task_group<> compile_tasks{loop};
};
} // namespace clice

View File

@@ -1,14 +1,15 @@
#include "server/indexer.h"
#include "server/compiler/indexer.h"
#include <algorithm>
#include <string>
#include <variant>
#include <vector>
#include "index/tu_index.h"
#include "server/compiler.h"
#include "server/protocol.h"
#include "server/session.h"
#include "server/worker_pool.h"
#include "server/compiler/compiler.h"
#include "server/protocol/worker.h"
#include "server/service/session.h"
#include "server/worker/worker_pool.h"
#include "support/filesystem.h"
#include "support/logging.h"
@@ -446,6 +447,152 @@ std::optional<SymbolInfo> Indexer::resolve_symbol(index::SymbolHash hash) {
return SymbolInfo{hash, std::move(name), kind, def_loc->uri, def_loc->range};
}
static std::string extract_line(llvm::StringRef content, std::uint32_t offset) {
if(content.empty() || offset >= content.size())
return {};
std::size_t line_start = 0;
if(offset > 0) {
auto pos = content.rfind('\n', offset - 1);
if(pos != llvm::StringRef::npos)
line_start = pos + 1;
}
auto line_end = content.find('\n', offset);
if(line_end == llvm::StringRef::npos)
line_end = content.size();
return content.slice(line_start, line_end).str();
}
std::optional<Indexer::DefinitionText> Indexer::get_definition_text(index::SymbolHash hash) {
for(auto& [id, sess]: sessions) {
if(!sess.file_index || !sess.file_index->mapper)
continue;
auto it = sess.file_index->file_index.relations.find(hash);
if(it == sess.file_index->file_index.relations.end())
continue;
for(auto& rel: it->second) {
if(rel.kind.value() != RelationKind::Definition)
continue;
auto def_range = std::bit_cast<LocalSourceRange>(rel.target_symbol);
if(def_range.begin >= def_range.end)
continue;
llvm::StringRef content = sess.file_index->content;
if(def_range.end > content.size())
continue;
auto start = sess.file_index->mapper->to_position(def_range.begin);
auto end = sess.file_index->mapper->to_position(def_range.end);
if(!start || !end)
continue;
return DefinitionText{
.file = std::string(workspace.path_pool.resolve(id)),
.start_line = static_cast<int>(start->line) + 1,
.end_line = static_cast<int>(end->line) + 1,
.text =
std::string(content.substr(def_range.begin, def_range.end - def_range.begin)),
};
}
}
auto sym_it = workspace.project_index.symbols.find(hash);
if(sym_it == workspace.project_index.symbols.end())
return std::nullopt;
for(auto file_id: sym_it->second.reference_files) {
if(is_proj_path_open(file_id))
continue;
auto shard_it = workspace.merged_indices.find(file_id);
if(shard_it == workspace.merged_indices.end())
continue;
auto* m = shard_it->second.mapper();
if(!m)
continue;
auto content = shard_it->second.index.content();
std::optional<DefinitionText> result;
shard_it->second.index.lookup(
hash,
RelationKind::Definition,
[&](const index::Relation& r) {
auto def_range = std::bit_cast<LocalSourceRange>(r.target_symbol);
if(def_range.begin >= def_range.end || def_range.end > content.size())
return true;
auto start = m->to_position(def_range.begin);
auto end = m->to_position(def_range.end);
if(!start || !end)
return true;
result = DefinitionText{
.file = workspace.project_index.path_pool.path(file_id).str(),
.start_line = static_cast<int>(start->line) + 1,
.end_line = static_cast<int>(end->line) + 1,
.text = std::string(
content.substr(def_range.begin, def_range.end - def_range.begin)),
};
return false;
});
if(result)
return result;
}
return std::nullopt;
}
std::vector<Indexer::ReferenceWithContext> Indexer::collect_references(index::SymbolHash hash,
RelationKind kind) {
std::vector<ReferenceWithContext> results;
auto sym_it = workspace.project_index.symbols.find(hash);
if(sym_it != workspace.project_index.symbols.end()) {
for(auto file_id: sym_it->second.reference_files) {
if(is_proj_path_open(file_id))
continue;
auto shard_it = workspace.merged_indices.find(file_id);
if(shard_it == workspace.merged_indices.end())
continue;
auto* m = shard_it->second.mapper();
if(!m)
continue;
auto content = shard_it->second.index.content();
auto file_path = workspace.project_index.path_pool.path(file_id);
shard_it->second.index.lookup(hash, kind, [&](const index::Relation& r) {
auto start = m->to_position(r.range.begin);
if(!start)
return true;
results.push_back(ReferenceWithContext{
.file = file_path.str(),
.line = static_cast<int>(start->line) + 1,
.context = extract_line(content, r.range.begin),
});
return true;
});
}
}
for(auto& [id, sess]: sessions) {
if(!sess.file_index || !sess.file_index->mapper)
continue;
auto it = sess.file_index->file_index.relations.find(hash);
if(it == sess.file_index->file_index.relations.end())
continue;
auto file_path = workspace.path_pool.resolve(id);
llvm::StringRef content = sess.file_index->content;
for(auto& rel: it->second) {
if(rel.kind != kind)
continue;
auto start = sess.file_index->mapper->to_position(rel.range.begin);
if(!start)
continue;
results.push_back(ReferenceWithContext{
.file = file_path.str(),
.line = static_cast<int>(start->line) + 1,
.context = extract_line(content, rel.range.begin),
});
}
}
return results;
}
std::vector<protocol::CallHierarchyIncomingCall>
Indexer::find_incoming_calls(index::SymbolHash hash) {
llvm::DenseMap<index::SymbolHash, std::vector<protocol::Range>> caller_ranges;
@@ -624,16 +771,108 @@ void Indexer::enqueue(std::uint32_t server_path_id) {
index_queue.push_back(server_path_id);
}
void Indexer::pause_indexing() {
++pause_depth;
if(pause_depth == 1) {
resume_event.reset();
LOG_DEBUG("Background indexing paused");
}
}
void Indexer::resume_indexing() {
if(pause_depth > 0)
--pause_depth;
if(pause_depth == 0) {
resume_event.set();
LOG_DEBUG("Background indexing resumed");
}
}
kota::task<> Indexer::stop() {
bg_tasks.cancel();
co_await bg_tasks.join();
}
void Indexer::schedule() {
if(!workspace.config.enable_indexing || indexing_active || indexing_scheduled)
if(!*workspace.config.project.enable_indexing || indexing_active || indexing_scheduled)
return;
indexing_scheduled = true;
if(!index_idle_timer) {
index_idle_timer = std::make_shared<kota::timer>(kota::timer::create(loop));
}
index_idle_timer->start(std::chrono::milliseconds(workspace.config.idle_timeout_ms));
loop.schedule(run_background_indexing());
index_idle_timer->start(std::chrono::milliseconds(*workspace.config.project.idle_timeout_ms));
if(!bg_tasks.spawn(run_background_indexing())) {
indexing_scheduled = false;
LOG_WARN("Failed to spawn background indexing task (task group stopped)");
}
}
kota::task<> Indexer::index_one(std::uint32_t server_path_id) {
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
if(sessions.contains(server_path_id))
co_return;
if(!need_update(file_path))
co_return;
// For module interface units, compile their PCM (and transitive deps)
// first so the stateless worker has the artifacts it needs.
if(workspace.compile_graph && workspace.path_to_module.contains(server_path_id)) {
co_await workspace.compile_graph->compile(server_path_id);
}
worker::BuildParams params;
params.kind = worker::BuildKind::Index;
params.file = file_path;
if(!compiler.fill_compile_args(file_path, params.directory, params.arguments, nullptr))
co_return;
workspace.fill_pcm_deps(params.pcms);
LOG_INFO("Background indexing: {}", file_path);
auto result = co_await pool.send_stateless(params);
if(result.has_value() && result.value().success && !result.value().tu_index_data.empty()) {
LOG_INFO("Background indexing got TUIndex for {}: {} bytes",
file_path,
result.value().tu_index_data.size());
merge(result.value().tu_index_data.data(), result.value().tu_index_data.size());
} else if(result.has_value() && !result.value().success) {
LOG_WARN("Background index failed for {}: {}", file_path, result.value().error);
} else if(result.has_value() && result.value().tu_index_data.empty()) {
LOG_WARN("Background index returned empty TUIndex for {}", file_path);
} else {
LOG_WARN("Background index IPC error for {}: {}", file_path, result.error().message);
}
}
kota::task<> Indexer::monitor_resources() {
while(true) {
co_await kota::sleep(std::chrono::milliseconds(3000));
auto mem = kota::sys::memory();
if(mem.total == 0)
continue;
auto effective_total =
(mem.constrained > 0 && mem.constrained < mem.total) ? mem.constrained : mem.total;
auto ratio = static_cast<double>(mem.available) / static_cast<double>(effective_total);
if(ratio < 0.15 && max_concurrent > 1) {
--max_concurrent;
LOG_INFO("Index concurrency -> {} (memory pressure: {:.0f}% available)",
max_concurrent,
ratio * 100);
} else if(ratio > 0.30 && max_concurrent < baseline_concurrent) {
++max_concurrent;
LOG_DEBUG("Index concurrency -> {} (memory OK: {:.0f}% available)",
max_concurrent,
ratio * 100);
}
}
}
kota::task<> Indexer::run_background_indexing() {
@@ -648,49 +887,75 @@ kota::task<> Indexer::run_background_indexing() {
}
indexing_active = true;
std::size_t processed = 0;
while(index_queue_pos < index_queue.size()) {
auto server_path_id = index_queue[index_queue_pos];
index_queue_pos++;
kota::cancellation_source monitor_cancel;
bg_tasks.spawn(kota::with_token(monitor_resources(), monitor_cancel.token()));
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
std::stable_partition(
index_queue.begin() + index_queue_pos,
index_queue.end(),
[this](std::uint32_t id) { return workspace.path_to_module.contains(id); });
if(sessions.contains(server_path_id))
continue;
auto total = index_queue.size() - index_queue_pos;
std::size_t dispatched = 0;
std::size_t completed = 0;
if(!need_update(file_path))
continue;
worker::BuildParams params;
params.kind = worker::BuildKind::Index;
params.file = file_path;
if(!compiler.fill_compile_args(file_path, params.directory, params.arguments, nullptr))
continue;
workspace.fill_pcm_deps(params.pcms);
LOG_INFO("Background indexing: {}", file_path);
auto result = co_await pool.send_stateless(params);
if(result.has_value() && result.value().success && !result.value().tu_index_data.empty()) {
LOG_INFO("Background indexing got TUIndex for {}: {} bytes",
file_path,
result.value().tu_index_data.size());
merge(result.value().tu_index_data.data(), result.value().tu_index_data.size());
++processed;
} else if(result.has_value() && !result.value().success) {
LOG_WARN("Background index failed for {}: {}", file_path, result.value().error);
} else if(result.has_value() && result.value().tu_index_data.empty()) {
LOG_WARN("Background index returned empty TUIndex for {}", file_path);
std::optional<lsp::ProgressReporter<kota::ipc::JsonPeer>> progress;
if(peer) {
progress.emplace(*peer, protocol::ProgressToken(std::string("clice/backgroundIndex")));
auto create_result = co_await progress->create();
if(!create_result.has_error()) {
progress->begin("Indexing", std::format("0/{} files", total), 0);
} else {
LOG_WARN("Background index IPC error for {}: {}", file_path, result.error().message);
progress.reset();
}
}
kota::task_group<> workers(loop);
std::size_t in_flight = 0;
kota::event slot_available;
while(index_queue_pos < index_queue.size()) {
if(pause_depth > 0)
co_await resume_event.wait();
auto server_path_id = index_queue[index_queue_pos++];
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
if(sessions.contains(server_path_id) || !need_update(file_path)) {
++completed;
continue;
}
while(in_flight >= max_concurrent) {
slot_available.reset();
co_await slot_available.wait();
}
++in_flight;
++dispatched;
workers.spawn([&, server_path_id]() -> kota::task<> {
co_await index_one(server_path_id);
--in_flight;
++completed;
if(progress) {
auto pct = total > 0 ? static_cast<std::uint32_t>(completed * 100 / total) : 100;
progress->report(std::format("{}/{} files", completed, total), pct);
}
slot_available.set();
}());
}
co_await workers.join();
if(progress) {
progress->end(std::format("Indexed {} files", dispatched));
}
monitor_cancel.cancel();
indexing_active = false;
LOG_INFO("Background indexing complete: {} files processed", processed);
save(workspace.config.index_dir);
LOG_INFO("Background indexing complete: {} files dispatched", dispatched);
save(workspace.config.project.index_dir);
}
} // namespace clice

View File

@@ -9,10 +9,12 @@
#include "semantic/relation_kind.h"
#include "semantic/symbol_kind.h"
#include "server/workspace.h"
#include "server/workspace/workspace.h"
#include "kota/async/async.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/lsp/position.h"
#include "kota/ipc/lsp/progress.h"
#include "kota/ipc/lsp/protocol.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/SmallVector.h"
@@ -59,8 +61,49 @@ public:
WorkerPool& pool,
Compiler& compiler,
std::function<bool(std::uint32_t)> is_file_open = {}) :
loop(loop), workspace(workspace), sessions(sessions), pool(pool), compiler(compiler),
is_file_open(std::move(is_file_open)) {}
loop(loop), bg_tasks(loop), workspace(workspace), sessions(sessions), pool(pool),
compiler(compiler), is_file_open(std::move(is_file_open)) {}
/// Set the LSP peer for progress reporting. Must be called before
/// schedule() if progress notifications are desired.
void set_peer(kota::ipc::JsonPeer* p) {
peer = p;
}
/// Temporarily pause background indexing to give priority to user
/// requests. Indexing tasks already dispatched to workers continue,
/// but no new tasks will be sent until resume_indexing() is called.
void pause_indexing();
/// Resume background indexing after a pause.
void resume_indexing();
/// RAII guard that pauses indexing for its lifetime.
struct [[nodiscard]] ScopedPause {
Indexer& indexer;
explicit ScopedPause(Indexer& idx) : indexer(idx) {
indexer.pause_indexing();
}
~ScopedPause() {
indexer.resume_indexing();
}
ScopedPause(const ScopedPause&) = delete;
ScopedPause& operator=(const ScopedPause&) = delete;
};
ScopedPause scoped_pause() {
return ScopedPause{*this};
}
/// Set the maximum number of concurrent index tasks.
/// Also sets the baseline that dynamic adjustment will restore to.
void set_max_concurrency(std::size_t n) {
max_concurrent = std::max<std::size_t>(n, 1);
baseline_concurrent = max_concurrent;
}
/// Add a file to the background indexing queue.
void enqueue(std::uint32_t server_path_id);
@@ -124,6 +167,43 @@ public:
std::vector<protocol::SymbolInformation> search_symbols(llvm::StringRef query,
std::size_t max_results = 100);
struct DefinitionText {
std::string file;
int start_line;
int end_line;
std::string text;
};
/// Get full definition text for a symbol, using stored index ranges and content.
std::optional<DefinitionText> get_definition_text(index::SymbolHash hash);
struct ReferenceWithContext {
std::string file;
int line;
std::string context;
};
/// Collect references (or definitions) with context lines from stored content.
std::vector<ReferenceWithContext> collect_references(index::SymbolHash hash, RelationKind kind);
/// Cancel background indexing and wait for all tasks to settle.
kota::task<> stop();
/// Whether background indexing is currently idle (no active or queued work).
bool is_idle() const {
return !indexing_active && index_queue_pos >= index_queue.size();
}
/// Number of files remaining in the indexing queue.
std::size_t pending_files() const {
return index_queue_pos < index_queue.size() ? index_queue.size() - index_queue_pos : 0;
}
/// Total files that were enqueued in the current (or last) indexing round.
std::size_t total_queued() const {
return index_queue.size();
}
/// Convert internal SymbolKind to LSP SymbolKind.
static protocol::SymbolKind to_lsp_symbol_kind(SymbolKind kind);
@@ -165,6 +245,7 @@ private:
private:
kota::event_loop& loop;
kota::task_group<> bg_tasks;
Workspace& workspace;
llvm::DenseMap<std::uint32_t, Session>& sessions;
WorkerPool& pool;
@@ -175,6 +256,9 @@ private:
/// server-path-id-keyed sessions map to project-level path_ids.
std::function<bool(std::uint32_t)> is_file_open;
/// LSP peer for progress reporting (optional, not owned).
kota::ipc::JsonPeer* peer = nullptr;
/// Background indexing queue and scheduling state.
std::vector<std::uint32_t> index_queue;
std::size_t index_queue_pos = 0;
@@ -182,7 +266,18 @@ private:
bool indexing_scheduled = false;
std::shared_ptr<kota::timer> index_idle_timer;
/// Concurrency control for background indexing.
std::size_t max_concurrent = 2;
std::size_t baseline_concurrent = 2;
/// Pause/resume: when paused, new index tasks wait on this event.
/// Uses a counter so nested pause/resume pairs work correctly.
std::size_t pause_depth = 0;
kota::event resume_event{true};
kota::task<> run_background_indexing();
kota::task<> index_one(std::uint32_t server_path_id);
kota::task<> monitor_resources();
};
} // namespace clice

View File

@@ -1,100 +0,0 @@
#include "server/config.h"
#include <algorithm>
#include <thread>
#include "support/filesystem.h"
#include "support/logging.h"
#include "kota/codec/toml.h"
namespace clice {
/// Replace all occurrences of ${workspace} with the workspace root.
static void substitute_workspace(std::string& value, const std::string& workspace_root) {
constexpr std::string_view placeholder = "${workspace}";
std::string::size_type pos = 0;
while((pos = value.find(placeholder, pos)) != std::string::npos) {
value.replace(pos, placeholder.size(), workspace_root);
pos += workspace_root.size();
}
}
void CliceConfig::apply_defaults(const std::string& workspace_root) {
auto cpu_count = std::thread::hardware_concurrency();
if(cpu_count == 0)
cpu_count = 4;
if(stateful_worker_count == 0) {
stateful_worker_count = 2;
}
if(stateless_worker_count == 0) {
stateless_worker_count = 3;
}
if(worker_memory_limit == 0) {
worker_memory_limit = 4ULL * 1024 * 1024 * 1024; // 4GB default
}
if(cache_dir.empty() && !workspace_root.empty()) {
cache_dir = path::join(workspace_root, ".clice");
}
if(index_dir.empty() && !cache_dir.empty()) {
index_dir = path::join(cache_dir, "index");
}
if(logging_dir.empty() && !cache_dir.empty()) {
logging_dir = path::join(cache_dir, "logs");
}
// Apply variable substitution to string fields
substitute_workspace(compile_commands_path, workspace_root);
substitute_workspace(cache_dir, workspace_root);
substitute_workspace(index_dir, workspace_root);
substitute_workspace(logging_dir, workspace_root);
}
std::optional<CliceConfig> CliceConfig::load(const std::string& path,
const std::string& workspace_root) {
auto content = fs::read(path);
if(!content) {
return std::nullopt;
}
auto result = kota::codec::toml::parse<CliceConfig>(*content);
if(!result) {
LOG_WARN("Failed to parse config file {}", path);
return std::nullopt;
}
auto config = std::move(*result);
config.apply_defaults(workspace_root);
LOG_INFO("Loaded config from {}", path);
return config;
}
CliceConfig CliceConfig::load_from_workspace(const std::string& workspace_root) {
if(!workspace_root.empty()) {
// Try standard config file locations
for(auto* name: {"clice.toml", ".clice/config.toml"}) {
auto config_path = path::join(workspace_root, name);
if(llvm::sys::fs::exists(config_path)) {
auto config = load(config_path, workspace_root);
if(config)
return std::move(*config);
}
}
}
// No config file found; use defaults
CliceConfig config;
config.apply_defaults(workspace_root);
LOG_INFO(
"No clice.toml found, using default configuration " "(stateful={}, stateless={}, memory_limit={}MB)",
config.stateful_worker_count,
config.stateless_worker_count,
config.worker_memory_limit / (1024 * 1024));
return config;
}
} // namespace clice

View File

@@ -1,46 +0,0 @@
#pragma once
#include <cstdint>
#include <optional>
#include <string>
namespace clice {
/// Configuration for the clice LSP server, loadable from clice.toml.
struct CliceConfig {
// Worker configuration (0 = auto-detect from system resources)
std::uint32_t stateful_worker_count = 0;
std::uint32_t stateless_worker_count = 0;
std::uint64_t worker_memory_limit = 0; // bytes; 0 = auto
// Compilation database path (empty = auto-detect)
std::string compile_commands_path;
// Cache directory (empty = default: <workspace>/.clice/)
std::string cache_dir;
// Index storage directory (default: <cache_dir>/index/)
std::string index_dir;
// Logging directory (default: <cache_dir>/logs/)
std::string logging_dir;
// Background indexing
bool enable_indexing = true;
int idle_timeout_ms = 3000;
/// Compute default values for any field left at its zero/empty sentinel.
void apply_defaults(const std::string& workspace_root);
/// Try to load configuration from a TOML file.
/// Performs ${workspace} variable substitution in string fields.
/// Returns std::nullopt if the file does not exist or cannot be parsed.
static std::optional<CliceConfig> load(const std::string& path,
const std::string& workspace_root);
/// Load config from the workspace, trying standard locations.
/// Returns a default config (with apply_defaults) if no file is found.
static CliceConfig load_from_workspace(const std::string& workspace_root);
};
} // namespace clice

View File

@@ -1,80 +0,0 @@
#pragma once
#include <cstdint>
#include <memory>
#include <string>
#include <vector>
#include "server/compiler.h"
#include "server/indexer.h"
#include "server/session.h"
#include "server/worker_pool.h"
#include "server/workspace.h"
#include "kota/async/async.h"
#include "kota/codec/raw_value.h"
#include "kota/ipc/peer.h"
#include "llvm/ADT/DenseMap.h"
namespace clice {
enum class ServerLifecycle : std::uint8_t {
Uninitialized,
Initialized,
Ready,
ShuttingDown,
Exited,
};
/// Top-level LSP server — the single orchestration point for the language
/// server process.
///
/// Responsibilities:
/// - Owns the two-layer state model: Workspace (disk truth) and Sessions
/// (per-open-file volatile state).
/// - Manages Session lifecycle directly: didOpen creates, didChange mutates,
/// didSave syncs to Workspace, didClose destroys.
/// - Dispatches compilation and feature queries to Compiler.
/// - Dispatches index lookups and background indexing to Indexer.
///
/// Design principle:
/// Open files are never depended upon by other files. Dependencies always
/// point to disk files. The only path from Session to Workspace is didSave.
class MasterServer {
public:
MasterServer(kota::event_loop& loop, kota::ipc::JsonPeer& peer, std::string self_path);
~MasterServer();
void register_handlers();
private:
kota::event_loop& loop;
kota::ipc::JsonPeer& peer;
/// Persistent project-wide state (config, CDB, path pool, dependency
/// graphs, compilation caches, symbol index).
Workspace workspace;
/// Per-file editing sessions, keyed by server-level path_id.
llvm::DenseMap<std::uint32_t, Session> sessions;
/// Worker process pool for offloading compilation and queries.
WorkerPool pool;
/// Compilation lifecycle manager (reads/writes workspace and sessions).
Compiler compiler;
/// Index query and background scheduling (reads from workspace and sessions).
Indexer indexer;
ServerLifecycle lifecycle = ServerLifecycle::Uninitialized;
std::string self_path;
std::string workspace_root;
std::string session_log_dir;
kota::task<> load_workspace();
using RawResult = kota::task<kota::codec::RawValue, kota::ipc::Error>;
};
} // namespace clice

View File

@@ -0,0 +1,311 @@
#pragma once
#include <cstdint>
#include <optional>
#include <string>
#include <vector>
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/protocol.h"
namespace clice::agentic {
struct CompileCommandParams {
std::string path;
};
struct CompileCommandResult {
std::string file;
std::string directory;
std::vector<std::string> arguments;
};
struct FileInfo {
std::string path;
std::string kind;
std::optional<std::string> module_name;
};
struct ProjectFilesParams {
std::optional<std::string> filter;
};
struct ProjectFilesResult {
std::vector<FileInfo> files;
int total = 0;
};
struct DepEntry {
std::string path;
int depth = 0;
};
struct FileDepsParams {
std::string path;
std::optional<std::string> direction;
std::optional<int> depth;
};
struct FileDepsResult {
std::string file;
std::vector<DepEntry> includes;
std::vector<DepEntry> includers;
};
struct ImpactAnalysisParams {
std::string path;
};
struct ImpactAnalysisResult {
std::vector<std::string> direct_dependents;
std::vector<std::string> transitive_dependents;
std::vector<std::string> affected_modules;
};
struct SymbolEntry {
std::string name;
std::string kind;
std::string file;
int line = 0;
std::optional<std::string> container;
std::uint64_t symbol_id = 0;
};
struct SymbolSearchParams {
std::string query;
std::optional<std::vector<std::string>> kind_filter;
std::optional<int> max_results;
};
struct SymbolSearchResult {
std::vector<SymbolEntry> symbols;
};
struct ReadSymbolParams {
std::optional<std::string> name;
std::optional<std::string> path;
std::optional<int> line;
std::optional<std::uint64_t> symbol_id;
};
struct ReadSymbolResult {
std::string name;
std::string kind;
std::string file;
int start_line = 0;
int end_line = 0;
std::string text;
std::optional<std::string> signature;
std::uint64_t symbol_id = 0;
};
struct DocumentSymbolEntry {
std::string name;
std::string kind;
int start_line = 0;
int end_line = 0;
std::uint64_t symbol_id = 0;
};
struct DocumentSymbolsParams {
std::string path;
};
struct DocumentSymbolsResult {
std::vector<DocumentSymbolEntry> symbols;
};
struct DefinitionParams {
std::optional<std::string> name;
std::optional<std::string> path;
std::optional<int> line;
std::optional<std::uint64_t> symbol_id;
};
struct LocationEntry {
std::string file;
int start_line = 0;
int end_line = 0;
std::string text;
};
struct DefinitionResult {
std::string name;
std::string kind;
std::uint64_t symbol_id = 0;
std::optional<LocationEntry> definition;
};
struct ReferenceEntry {
std::string file;
int line = 0;
std::string context;
};
struct ReferencesParams {
std::optional<std::string> name;
std::optional<std::string> path;
std::optional<int> line;
std::optional<std::uint64_t> symbol_id;
std::optional<bool> include_declaration;
};
struct ReferencesResult {
std::string name;
std::string kind;
std::uint64_t symbol_id = 0;
std::vector<ReferenceEntry> references;
int total = 0;
};
struct CallGraphEntry {
std::string name;
std::string kind;
std::string file;
int line = 0;
std::uint64_t symbol_id = 0;
};
struct CallGraphParams {
std::optional<std::string> name;
std::optional<std::string> path;
std::optional<int> line;
std::optional<std::uint64_t> symbol_id;
std::optional<std::string> direction;
std::optional<int> depth;
};
struct CallGraphResult {
CallGraphEntry root;
std::vector<CallGraphEntry> callers;
std::vector<CallGraphEntry> callees;
};
struct TypeHierarchyEntry {
std::string name;
std::string kind;
std::string file;
int line = 0;
std::uint64_t symbol_id = 0;
};
struct TypeHierarchyParams {
std::optional<std::string> name;
std::optional<std::string> path;
std::optional<int> line;
std::optional<std::uint64_t> symbol_id;
std::optional<std::string> direction;
};
struct TypeHierarchyResult {
TypeHierarchyEntry root;
std::vector<TypeHierarchyEntry> supertypes;
std::vector<TypeHierarchyEntry> subtypes;
};
struct LintParams {
std::string path;
std::optional<int> line;
};
using LintResult = std::vector<kota::ipc::protocol::Diagnostic>;
struct StatusParams {};
struct StatusResult {
bool idle = true;
int pending = 0;
int total = 0;
int indexed = 0;
};
struct ShutdownParams {};
} // namespace clice::agentic
namespace kota::ipc::protocol {
template <>
struct RequestTraits<clice::agentic::CompileCommandParams> {
using Result = clice::agentic::CompileCommandResult;
constexpr inline static std::string_view method = "agentic/compileCommand";
};
template <>
struct RequestTraits<clice::agentic::ProjectFilesParams> {
using Result = clice::agentic::ProjectFilesResult;
constexpr inline static std::string_view method = "agentic/projectFiles";
};
template <>
struct RequestTraits<clice::agentic::FileDepsParams> {
using Result = clice::agentic::FileDepsResult;
constexpr inline static std::string_view method = "agentic/fileDeps";
};
template <>
struct RequestTraits<clice::agentic::ImpactAnalysisParams> {
using Result = clice::agentic::ImpactAnalysisResult;
constexpr inline static std::string_view method = "agentic/impactAnalysis";
};
template <>
struct RequestTraits<clice::agentic::SymbolSearchParams> {
using Result = clice::agentic::SymbolSearchResult;
constexpr inline static std::string_view method = "agentic/symbolSearch";
};
template <>
struct RequestTraits<clice::agentic::ReadSymbolParams> {
using Result = clice::agentic::ReadSymbolResult;
constexpr inline static std::string_view method = "agentic/readSymbol";
};
template <>
struct RequestTraits<clice::agentic::DocumentSymbolsParams> {
using Result = clice::agentic::DocumentSymbolsResult;
constexpr inline static std::string_view method = "agentic/documentSymbols";
};
template <>
struct RequestTraits<clice::agentic::DefinitionParams> {
using Result = clice::agentic::DefinitionResult;
constexpr inline static std::string_view method = "agentic/definition";
};
template <>
struct RequestTraits<clice::agentic::ReferencesParams> {
using Result = clice::agentic::ReferencesResult;
constexpr inline static std::string_view method = "agentic/references";
};
template <>
struct RequestTraits<clice::agentic::CallGraphParams> {
using Result = clice::agentic::CallGraphResult;
constexpr inline static std::string_view method = "agentic/callGraph";
};
template <>
struct RequestTraits<clice::agentic::TypeHierarchyParams> {
using Result = clice::agentic::TypeHierarchyResult;
constexpr inline static std::string_view method = "agentic/typeHierarchy";
};
template <>
struct RequestTraits<clice::agentic::LintParams> {
using Result = clice::agentic::LintResult;
constexpr inline static std::string_view method = "agentic/lint";
};
template <>
struct RequestTraits<clice::agentic::StatusParams> {
using Result = clice::agentic::StatusResult;
constexpr inline static std::string_view method = "agentic/status";
};
template <>
struct NotificationTraits<clice::agentic::ShutdownParams> {
constexpr inline static std::string_view method = "agentic/shutdown";
};
} // namespace kota::ipc::protocol

View File

@@ -0,0 +1,42 @@
#pragma once
#include <optional>
#include <string>
#include <vector>
namespace clice::ext {
struct ContextItem {
std::string label;
std::string description;
std::string uri;
};
struct QueryContextParams {
std::string uri;
std::optional<int> offset;
};
struct QueryContextResult {
std::vector<ContextItem> contexts;
int total = 0;
};
struct CurrentContextParams {
std::string uri;
};
struct CurrentContextResult {
std::optional<ContextItem> context;
};
struct SwitchContextParams {
std::string uri;
std::string context_uri;
};
struct SwitchContextResult {
bool success = false;
};
} // namespace clice::ext

View File

@@ -1,7 +1,6 @@
#pragma once
#include <cstdint>
#include <optional>
#include <string>
#include <unordered_map>
#include <utility>
@@ -9,8 +8,7 @@
#include "syntax/token.h"
#include "kota/codec/raw_value.h"
#include "kota/ipc/lsp/protocol.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/protocol.h"
namespace clice::worker {
@@ -45,6 +43,7 @@ struct CompileParams {
std::string text;
std::string directory;
std::vector<std::string> arguments;
bool clang_tidy = false;
std::pair<std::string, uint32_t> pch;
std::unordered_map<std::string, std::string> pcms;
};
@@ -66,6 +65,7 @@ enum class BuildKind : uint8_t {
Index,
Completion,
SignatureHelp,
Format,
};
/// Unified parameters for all stateless build/compilation tasks.
@@ -76,6 +76,7 @@ enum class BuildKind : uint8_t {
/// - Index: + pcms
/// - Completion: + text, version, offset, pch, pcms
/// - SignatureHelp: + text, version, offset, pch, pcms
/// - Format: + text, format_range (optional)
struct BuildParams {
BuildKind kind;
std::string file;
@@ -92,6 +93,7 @@ struct BuildParams {
std::string output_path; ///< BuildPCH, BuildPCM
std::string module_name; ///< BuildPCM
uint32_t preamble_bound = UINT32_MAX; ///< BuildPCH
LocalSourceRange format_range; ///< Format (default = full document)
};
/// Unified result for stateless build tasks.
@@ -122,43 +124,6 @@ struct EvictedParams {
} // namespace clice::worker
namespace clice::ext {
struct ContextItem {
std::string label;
std::string description;
std::string uri;
};
struct QueryContextParams {
std::string uri;
std::optional<int> offset;
};
struct QueryContextResult {
std::vector<ContextItem> contexts;
int total;
};
struct CurrentContextParams {
std::string uri;
};
struct CurrentContextResult {
std::optional<ContextItem> context;
};
struct SwitchContextParams {
std::string uri;
std::string context_uri;
};
struct SwitchContextResult {
bool success;
};
} // namespace clice::ext
namespace kota::ipc::protocol {
template <>

View File

@@ -0,0 +1,820 @@
#include "server/service/agent_client.h"
#include <algorithm>
#include <format>
#include <ranges>
#include <string>
#include <vector>
#include "compile/compilation.h"
#include "feature/feature.h"
#include "server/protocol/agentic.h"
#include "server/service/master_server.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "kota/async/async.h"
#include "kota/ipc/lsp/uri.h"
#include "kota/meta/enum.h"
#include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/SmallVector.h"
namespace clice {
using kota::ipc::RequestResult;
using RequestContext = kota::ipc::JsonPeer::RequestContext;
namespace lsp = kota::ipc::lsp;
namespace protocol = kota::ipc::protocol;
static std::string_view symbol_kind_name(SymbolKind kind) {
constexpr auto names = kota::meta::reflection<SymbolKind::Kind>::member_names;
auto idx = static_cast<std::size_t>(kind.value());
if(idx < names.size())
return names[idx];
return "Unknown";
}
struct ResolvedSymbol {
index::SymbolHash hash = 0;
std::string name;
SymbolKind kind;
std::string file;
int line = 0;
};
static std::vector<ResolvedSymbol> resolve_locator(const agentic::ReadSymbolParams& loc,
Workspace& workspace,
llvm::DenseMap<std::uint32_t, Session>& sessions,
Indexer& indexer) {
if(loc.symbol_id.has_value() && *loc.symbol_id != 0) {
auto hash = static_cast<index::SymbolHash>(*loc.symbol_id);
std::string name;
SymbolKind kind;
if(!indexer.find_symbol_info(hash, name, kind))
return {};
auto def_loc = indexer.find_definition_location(hash);
if(!def_loc)
return {};
auto file = uri_to_path(def_loc->uri);
int line_num = static_cast<int>(def_loc->range.start.line) + 1;
return {
{hash, std::move(name), kind, std::move(file), line_num}
};
}
if(loc.name.has_value() && !loc.name->empty()) {
std::string query_lower = llvm::StringRef(*loc.name).lower();
std::vector<ResolvedSymbol> candidates;
std::vector<ResolvedSymbol> exact_matches;
llvm::DenseSet<index::SymbolHash> seen;
auto try_symbol = [&](index::SymbolHash hash, const index::Symbol& symbol) {
if(symbol.name.empty())
return;
if(llvm::StringRef(symbol.name).lower().find(query_lower) == std::string::npos)
return;
auto def_loc = indexer.find_definition_location(hash);
if(!def_loc)
return;
if(!seen.insert(hash).second)
return;
auto file = uri_to_path(def_loc->uri);
int line_num = static_cast<int>(def_loc->range.start.line) + 1;
if(loc.path.has_value() && !loc.path->empty()) {
llvm::StringRef wanted(*loc.path);
bool basename_only = wanted.find_last_of("/\\") == llvm::StringRef::npos;
if(basename_only) {
if(llvm::sys::path::filename(file) != wanted)
return;
} else if(!llvm::StringRef(file).ends_with(wanted)) {
return;
}
}
bool is_exact = llvm::StringRef(symbol.name).lower() == query_lower ||
llvm::StringRef(symbol.name).ends_with("::" + *loc.name);
ResolvedSymbol rs{hash, symbol.name, symbol.kind, std::move(file), line_num};
if(is_exact)
exact_matches.push_back(std::move(rs));
else
candidates.push_back(std::move(rs));
};
for(auto& [hash, symbol]: workspace.project_index.symbols)
try_symbol(hash, symbol);
for(auto& [_, sess]: sessions) {
if(!sess.file_index)
continue;
for(auto& [hash, symbol]: sess.file_index->symbols)
try_symbol(hash, symbol);
}
if(!exact_matches.empty())
return exact_matches;
return candidates;
}
if(loc.path.has_value() && loc.line.has_value()) {
auto path_str = *loc.path;
auto target_line = static_cast<protocol::uinteger>(*loc.line - 1);
auto pool_it = workspace.path_pool.cache.find(path_str);
auto server_id = pool_it != workspace.path_pool.cache.end() ? pool_it->second : ~0u;
auto* sess =
server_id != ~0u && sessions.contains(server_id) ? &sessions[server_id] : nullptr;
if(sess && sess->file_index) {
auto& fi = *sess->file_index;
if(fi.mapper) {
for(auto& [hash, rels]: fi.file_index.relations) {
for(auto& rel: rels) {
if(rel.kind.value() != RelationKind::Definition)
continue;
auto start = fi.mapper->to_position(rel.range.begin);
if(start && start->line == target_line) {
std::string name;
SymbolKind kind;
if(indexer.find_symbol_info(hash, name, kind))
return {
{hash, std::move(name), kind, path_str, *loc.line}
};
}
}
}
}
}
auto it = workspace.project_index.path_pool.find(path_str);
if(it == workspace.project_index.path_pool.cache.end())
return {};
auto proj_id = it->second;
auto shard_it = workspace.merged_indices.find(proj_id);
if(shard_it == workspace.merged_indices.end())
return {};
for(auto& [hash, symbol]: workspace.project_index.symbols) {
if(!symbol.reference_files.contains(proj_id))
continue;
bool found = false;
shard_it->second.find_relations(hash,
RelationKind::Definition,
[&](const index::Relation&, protocol::Range range) {
if(range.start.line == target_line) {
found = true;
return false;
}
return true;
});
if(found)
return {
{hash, symbol.name, symbol.kind, path_str, *loc.line}
};
}
return {};
}
return {};
}
static std::uint64_t extract_symbol_id(const std::optional<protocol::LSPAny>& data) {
if(!data.has_value())
return 0;
if(auto* val = std::get_if<std::int64_t>(&static_cast<const protocol::LSPVariant&>(*data)))
return static_cast<std::uint64_t>(*val);
LOG_WARN("extract_symbol_id: unexpected LSPAny variant type");
return 0;
}
AgentClient::AgentClient(MasterServer& server, kota::ipc::JsonPeer& peer) :
server(server), peer(peer) {
using namespace agentic;
auto& srv = this->server;
peer.on_request(
[&srv](RequestContext&,
const CompileCommandParams& params) -> RequestResult<CompileCommandParams> {
std::string directory;
std::vector<std::string> arguments;
if(!srv.compiler.fill_compile_args(params.path, directory, arguments)) {
co_return kota::outcome_error(
kota::ipc::Error{std::format("no compile command found for {}", params.path)});
}
co_return CompileCommandResult{
.file = params.path,
.directory = std::move(directory),
.arguments = std::move(arguments),
};
});
peer.on_request([&srv](RequestContext&,
const ProjectFilesParams& params) -> RequestResult<ProjectFilesParams> {
auto& ws = srv.workspace;
auto filter = params.filter.value_or("all");
ProjectFilesResult result;
llvm::DenseSet<std::uint32_t> seen;
for(auto& entry: ws.cdb.get_entries()) {
auto file_path = ws.cdb.resolve_path(entry.file);
if(file_path.empty())
continue;
auto proj_it = ws.project_index.path_pool.find(file_path);
if(proj_it != ws.project_index.path_pool.cache.end()) {
if(!seen.insert(proj_it->second).second)
continue;
}
std::string kind_str;
auto mod_it = ws.path_to_module.find(ws.path_pool.intern(file_path));
if(mod_it != ws.path_to_module.end()) {
kind_str = "module";
} else {
auto ext = llvm::sys::path::extension(file_path);
if(ext == ".h" || ext == ".hpp" || ext == ".hxx" || ext == ".hh")
kind_str = "header";
else
kind_str = "source";
}
if(filter != "all" && filter != kind_str)
continue;
FileInfo fi;
fi.path = file_path.str();
fi.kind = std::move(kind_str);
if(mod_it != ws.path_to_module.end())
fi.module_name = mod_it->second;
result.files.push_back(std::move(fi));
}
if(filter == "all" || filter == "header") {
for(auto& [path_id, shard]: ws.merged_indices) {
if(seen.contains(path_id))
continue;
auto path_str = ws.project_index.path_pool.path(path_id);
auto ext = llvm::sys::path::extension(path_str);
if(ext == ".h" || ext == ".hpp" || ext == ".hxx" || ext == ".hh") {
seen.insert(path_id);
result.files.push_back(FileInfo{
.path = path_str.str(),
.kind = "header",
});
}
}
}
result.total = static_cast<int>(result.files.size());
co_return result;
});
peer.on_request(
[&srv](RequestContext&, const FileDepsParams& params) -> RequestResult<FileDepsParams> {
auto& ws = srv.workspace;
auto pool_it = ws.path_pool.cache.find(params.path);
if(pool_it == ws.path_pool.cache.end())
co_return FileDepsResult{.file = params.path};
auto path_id = pool_it->second;
auto direction = params.direction.value_or("both");
auto max_depth = params.depth.value_or(1);
FileDepsResult result;
result.file = params.path;
if(direction == "includes" || direction == "both") {
auto includes = ws.dep_graph.get_all_includes(path_id);
for(auto inc_id: includes) {
auto real_id = inc_id & DependencyGraph::PATH_ID_MASK;
auto inc_path = ws.path_pool.resolve(real_id);
result.includes.push_back(DepEntry{.path = inc_path.str(), .depth = 1});
}
if(max_depth == 0 || max_depth > 1) {
llvm::DenseSet<std::uint32_t> visited;
visited.insert(path_id);
for(auto& dep: result.includes)
visited.insert(ws.path_pool.intern(dep.path));
for(std::size_t i = 0; i < result.includes.size(); ++i) {
if(max_depth > 0 && result.includes[i].depth >= max_depth)
continue;
auto dep_id = ws.path_pool.intern(result.includes[i].path);
auto sub = ws.dep_graph.get_all_includes(dep_id);
for(auto sub_id: sub) {
auto real_id = sub_id & DependencyGraph::PATH_ID_MASK;
if(!visited.insert(real_id).second)
continue;
auto sub_path = ws.path_pool.resolve(real_id);
result.includes.push_back(DepEntry{
.path = sub_path.str(),
.depth = result.includes[i].depth + 1,
});
}
}
}
}
if(direction == "includers" || direction == "both") {
auto includers = ws.dep_graph.get_includers(path_id);
for(auto inc_id: includers) {
auto inc_path = ws.path_pool.resolve(inc_id);
result.includers.push_back(DepEntry{.path = inc_path.str(), .depth = 1});
}
if(max_depth == 0 || max_depth > 1) {
llvm::DenseSet<std::uint32_t> visited;
visited.insert(path_id);
for(auto& dep: result.includers) {
auto it = ws.path_pool.cache.find(dep.path);
if(it != ws.path_pool.cache.end())
visited.insert(it->second);
}
for(std::size_t i = 0; i < result.includers.size(); ++i) {
if(max_depth > 0 && result.includers[i].depth >= max_depth)
continue;
auto dep_it = ws.path_pool.cache.find(result.includers[i].path);
if(dep_it == ws.path_pool.cache.end())
continue;
auto sub = ws.dep_graph.get_includers(dep_it->second);
for(auto sub_id: sub) {
if(!visited.insert(sub_id).second)
continue;
auto sub_path = ws.path_pool.resolve(sub_id);
result.includers.push_back(DepEntry{
.path = sub_path.str(),
.depth = result.includers[i].depth + 1,
});
}
}
}
}
co_return result;
});
peer.on_request(
[&srv](RequestContext&,
const ImpactAnalysisParams& params) -> RequestResult<ImpactAnalysisParams> {
auto& ws = srv.workspace;
auto pool_it = ws.path_pool.cache.find(params.path);
if(pool_it == ws.path_pool.cache.end())
co_return ImpactAnalysisResult{};
auto path_id = pool_it->second;
ImpactAnalysisResult result;
auto direct_includers = ws.dep_graph.get_includers(path_id);
for(auto inc_id: direct_includers) {
result.direct_dependents.push_back(ws.path_pool.resolve(inc_id).str());
}
auto hosts = ws.dep_graph.find_host_sources(path_id);
llvm::DenseSet<std::uint32_t> seen;
seen.insert(path_id);
for(auto inc_id: direct_includers)
seen.insert(inc_id);
for(auto host_id: hosts) {
if(seen.insert(host_id).second)
result.transitive_dependents.push_back(ws.path_pool.resolve(host_id).str());
}
for(auto host_id: hosts) {
auto it = ws.path_to_module.find(host_id);
if(it != ws.path_to_module.end())
result.affected_modules.push_back(it->second);
}
auto mod_it = ws.path_to_module.find(path_id);
if(mod_it != ws.path_to_module.end())
result.affected_modules.push_back(mod_it->second);
co_return result;
});
peer.on_request([&srv](RequestContext&,
const SymbolSearchParams& params) -> RequestResult<SymbolSearchParams> {
auto max = params.max_results.value_or(100);
std::string query_lower = llvm::StringRef(params.query).lower();
SymbolSearchResult result;
llvm::DenseSet<index::SymbolHash> seen;
auto try_symbol = [&](index::SymbolHash hash, const index::Symbol& symbol) {
if(static_cast<int>(result.symbols.size()) >= max)
return;
if(symbol.name.empty())
return;
if(!query_lower.empty() &&
llvm::StringRef(symbol.name).lower().find(query_lower) == std::string::npos)
return;
if(params.kind_filter.has_value()) {
auto kind_name = std::string(symbol_kind_name(symbol.kind));
auto& filter = *params.kind_filter;
if(std::ranges::find(filter, kind_name) == filter.end())
return;
}
auto def_loc = srv.indexer.find_definition_location(hash);
if(!def_loc)
return;
if(!seen.insert(hash).second)
return;
auto file = uri_to_path(def_loc->uri);
result.symbols.push_back(SymbolEntry{
.name = symbol.name,
.kind = std::string(symbol_kind_name(symbol.kind)),
.file = std::move(file),
.line = static_cast<int>(def_loc->range.start.line) + 1,
.symbol_id = hash,
});
};
for(auto& [hash, symbol]: srv.workspace.project_index.symbols)
try_symbol(hash, symbol);
for(auto& [_, sess]: srv.sessions) {
if(!sess.file_index)
continue;
for(auto& [hash, symbol]: sess.file_index->symbols)
try_symbol(hash, symbol);
}
co_return result;
});
peer.on_request(
[&srv](RequestContext&, const ReadSymbolParams& params) -> RequestResult<ReadSymbolParams> {
auto candidates = resolve_locator(params, srv.workspace, srv.sessions, srv.indexer);
if(candidates.empty())
co_return kota::outcome_error(kota::ipc::Error{"symbol not found"});
if(candidates.size() > 1) {
co_return kota::outcome_error(kota::ipc::Error{
std::format("ambiguous: {} candidates, use symbolId to disambiguate",
candidates.size())});
}
auto& rs = candidates[0];
auto def_text = srv.indexer.get_definition_text(rs.hash);
if(!def_text)
co_return kota::outcome_error(kota::ipc::Error{"definition not found"});
co_return ReadSymbolResult{
.name = rs.name,
.kind = std::string(symbol_kind_name(rs.kind)),
.file = std::move(def_text->file),
.start_line = def_text->start_line,
.end_line = def_text->end_line,
.text = std::move(def_text->text),
.symbol_id = rs.hash,
};
});
peer.on_request(
[&srv](RequestContext&,
const DocumentSymbolsParams& params) -> RequestResult<DocumentSymbolsParams> {
auto is_document_level = [](SymbolKind kind) {
return kind == SymbolKind::Namespace || kind == SymbolKind::Class ||
kind == SymbolKind::Struct || kind == SymbolKind::Union ||
kind == SymbolKind::Enum || kind == SymbolKind::Type ||
kind == SymbolKind::Field || kind == SymbolKind::EnumMember ||
kind == SymbolKind::Function || kind == SymbolKind::Method ||
kind == SymbolKind::Variable || kind == SymbolKind::Macro ||
kind == SymbolKind::Concept || kind == SymbolKind::Module ||
kind == SymbolKind::Operator || kind == SymbolKind::Attribute;
};
DocumentSymbolsResult result;
auto pool_it = srv.workspace.path_pool.cache.find(params.path);
if(pool_it == srv.workspace.path_pool.cache.end())
co_return result;
auto server_id = pool_it->second;
auto sess_it = srv.sessions.find(server_id);
if(sess_it != srv.sessions.end() && sess_it->second.file_index) {
auto& fi = *sess_it->second.file_index;
for(auto& [hash, rels]: fi.file_index.relations) {
for(auto& rel: rels) {
if(rel.kind.value() != RelationKind::Definition)
continue;
std::string name;
SymbolKind kind;
if(!srv.indexer.find_symbol_info(hash, name, kind))
continue;
if(!is_document_level(kind))
continue;
if(fi.mapper) {
auto start = fi.mapper->to_position(rel.range.begin);
auto end = fi.mapper->to_position(rel.range.end);
if(start && end) {
result.symbols.push_back(DocumentSymbolEntry{
.name = std::move(name),
.kind = std::string(symbol_kind_name(kind)),
.start_line = static_cast<int>(start->line) + 1,
.end_line = static_cast<int>(end->line) + 1,
.symbol_id = hash,
});
break;
}
}
}
}
co_return result;
}
auto it = srv.workspace.project_index.path_pool.find(params.path);
if(it == srv.workspace.project_index.path_pool.cache.end())
co_return result;
auto proj_id = it->second;
auto shard_it = srv.workspace.merged_indices.find(proj_id);
if(shard_it == srv.workspace.merged_indices.end())
co_return result;
for(auto& [hash, symbol]: srv.workspace.project_index.symbols) {
if(symbol.name.empty())
continue;
if(!is_document_level(symbol.kind))
continue;
if(!symbol.reference_files.contains(proj_id))
continue;
shard_it->second.find_relations(
hash,
RelationKind::Definition,
[&](const index::Relation&, protocol::Range range) {
result.symbols.push_back(DocumentSymbolEntry{
.name = symbol.name,
.kind = std::string(symbol_kind_name(symbol.kind)),
.start_line = static_cast<int>(range.start.line) + 1,
.end_line = static_cast<int>(range.end.line) + 1,
.symbol_id = hash,
});
return true;
});
}
co_return result;
});
peer.on_request(
[&srv](RequestContext&, const DefinitionParams& params) -> RequestResult<DefinitionParams> {
auto candidates = resolve_locator(
ReadSymbolParams{params.name, params.path, params.line, params.symbol_id},
srv.workspace,
srv.sessions,
srv.indexer);
if(candidates.empty())
co_return kota::outcome_error(kota::ipc::Error{"symbol not found"});
if(candidates.size() > 1) {
co_return kota::outcome_error(kota::ipc::Error{
std::format("ambiguous: {} candidates, use symbolId to disambiguate",
candidates.size())});
}
auto& rs = candidates[0];
DefinitionResult result;
result.name = rs.name;
result.kind = std::string(symbol_kind_name(rs.kind));
result.symbol_id = rs.hash;
if(auto def_text = srv.indexer.get_definition_text(rs.hash)) {
result.definition = LocationEntry{
.file = std::move(def_text->file),
.start_line = def_text->start_line,
.end_line = def_text->end_line,
.text = std::move(def_text->text),
};
}
co_return result;
});
peer.on_request(
[&srv](RequestContext&, const ReferencesParams& params) -> RequestResult<ReferencesParams> {
auto candidates = resolve_locator(
ReadSymbolParams{params.name, params.path, params.line, params.symbol_id},
srv.workspace,
srv.sessions,
srv.indexer);
if(candidates.empty())
co_return kota::outcome_error(kota::ipc::Error{"symbol not found"});
if(candidates.size() > 1) {
co_return kota::outcome_error(kota::ipc::Error{
std::format("ambiguous: {} candidates, use symbolId to disambiguate",
candidates.size())});
}
auto& rs = candidates[0];
ReferencesResult result;
result.name = rs.name;
result.kind = std::string(symbol_kind_name(rs.kind));
result.symbol_id = rs.hash;
for(auto& ref: srv.indexer.collect_references(rs.hash, RelationKind::Reference)) {
result.references.push_back(ReferenceEntry{
.file = std::move(ref.file),
.line = ref.line,
.context = std::move(ref.context),
});
}
if(params.include_declaration.value_or(false)) {
for(auto& ref: srv.indexer.collect_references(rs.hash, RelationKind::Definition)) {
result.references.push_back(ReferenceEntry{
.file = std::move(ref.file),
.line = ref.line,
.context = std::move(ref.context),
});
}
}
result.total = static_cast<int>(result.references.size());
co_return result;
});
peer.on_request(
[&srv](RequestContext&, const CallGraphParams& params) -> RequestResult<CallGraphParams> {
auto candidates = resolve_locator(
ReadSymbolParams{params.name, params.path, params.line, params.symbol_id},
srv.workspace,
srv.sessions,
srv.indexer);
if(candidates.empty())
co_return kota::outcome_error(kota::ipc::Error{"symbol not found"});
if(candidates.size() > 1) {
co_return kota::outcome_error(kota::ipc::Error{
std::format("ambiguous: {} candidates, use symbolId to disambiguate",
candidates.size())});
}
auto& rs = candidates[0];
auto direction = params.direction.value_or("both");
CallGraphResult result;
result.root = CallGraphEntry{
.name = rs.name,
.kind = std::string(symbol_kind_name(rs.kind)),
.file = rs.file,
.line = rs.line,
.symbol_id = rs.hash,
};
auto resolve_kind = [&](std::uint64_t sym_id) -> std::string {
if(sym_id == 0)
return "Function";
std::string name;
SymbolKind kind;
if(srv.indexer.find_symbol_info(sym_id, name, kind))
return std::string(symbol_kind_name(kind));
return "Function";
};
if(direction == "callers" || direction == "both") {
auto incoming = srv.indexer.find_incoming_calls(rs.hash);
for(auto& call: incoming) {
auto sid = extract_symbol_id(call.from.data);
result.callers.push_back(CallGraphEntry{
.name = call.from.name,
.kind = resolve_kind(sid),
.file = uri_to_path(call.from.uri),
.line = static_cast<int>(call.from.range.start.line) + 1,
.symbol_id = sid,
});
}
}
if(direction == "callees" || direction == "both") {
auto outgoing = srv.indexer.find_outgoing_calls(rs.hash);
for(auto& call: outgoing) {
auto sid = extract_symbol_id(call.to.data);
result.callees.push_back(CallGraphEntry{
.name = call.to.name,
.kind = resolve_kind(sid),
.file = uri_to_path(call.to.uri),
.line = static_cast<int>(call.to.range.start.line) + 1,
.symbol_id = sid,
});
}
}
co_return result;
});
peer.on_request(
[&srv](RequestContext&,
const TypeHierarchyParams& params) -> RequestResult<TypeHierarchyParams> {
auto candidates = resolve_locator(
ReadSymbolParams{params.name, params.path, params.line, params.symbol_id},
srv.workspace,
srv.sessions,
srv.indexer);
if(candidates.empty())
co_return kota::outcome_error(kota::ipc::Error{"symbol not found"});
if(candidates.size() > 1) {
co_return kota::outcome_error(kota::ipc::Error{
std::format("ambiguous: {} candidates, use symbolId to disambiguate",
candidates.size())});
}
auto& rs = candidates[0];
auto direction = params.direction.value_or("both");
TypeHierarchyResult result;
result.root = TypeHierarchyEntry{
.name = rs.name,
.kind = std::string(symbol_kind_name(rs.kind)),
.file = rs.file,
.line = rs.line,
.symbol_id = rs.hash,
};
auto resolve_kind = [&](std::uint64_t sym_id) -> std::string {
if(sym_id == 0)
return "Class";
std::string name;
SymbolKind kind;
if(srv.indexer.find_symbol_info(sym_id, name, kind))
return std::string(symbol_kind_name(kind));
return "Class";
};
if(direction == "supertypes" || direction == "both") {
for(auto& item: srv.indexer.find_supertypes(rs.hash)) {
auto sid = extract_symbol_id(item.data);
result.supertypes.push_back(TypeHierarchyEntry{
.name = item.name,
.kind = resolve_kind(sid),
.file = uri_to_path(item.uri),
.line = static_cast<int>(item.range.start.line) + 1,
.symbol_id = sid,
});
}
}
if(direction == "subtypes" || direction == "both") {
for(auto& item: srv.indexer.find_subtypes(rs.hash)) {
auto sid = extract_symbol_id(item.data);
result.subtypes.push_back(TypeHierarchyEntry{
.name = item.name,
.kind = resolve_kind(sid),
.file = uri_to_path(item.uri),
.line = static_cast<int>(item.range.start.line) + 1,
.symbol_id = sid,
});
}
}
co_return result;
});
peer.on_request([&srv](RequestContext&, const LintParams& params) -> RequestResult<LintParams> {
std::string directory;
std::vector<std::string> arguments;
if(!srv.compiler.fill_compile_args(params.path, directory, arguments)) {
co_return kota::outcome_error(
kota::ipc::Error{std::format("no compile command found for {}", params.path)});
}
auto result = co_await kota::queue([path = params.path,
directory = std::move(directory),
arguments = std::move(arguments)]() mutable {
CompilationParams cp;
cp.kind = CompilationKind::Content;
cp.clang_tidy = true;
cp.directory = std::move(directory);
for(auto& arg: arguments) {
cp.arguments.push_back(arg.c_str());
}
auto unit = compile(cp);
if(!unit.completed() && !unit.fatal_error()) {
LOG_WARN("Lint compilation failed: {}", path);
return LintResult{};
}
return feature::diagnostics(unit);
});
co_return result.value();
});
peer.on_request([&srv](RequestContext&, const StatusParams&) -> RequestResult<StatusParams> {
StatusResult result;
result.idle = srv.indexer.is_idle();
result.pending = static_cast<int>(srv.indexer.pending_files());
result.total = static_cast<int>(srv.indexer.total_queued());
result.indexed = std::max(0, result.total - result.pending);
co_return result;
});
peer.on_notification([&srv](const ShutdownParams&) {
LOG_INFO("agentic/shutdown received, shutting down");
srv.schedule_shutdown();
});
}
} // namespace clice

View File

@@ -0,0 +1,18 @@
#pragma once
#include "kota/ipc/codec/json.h"
namespace clice {
class MasterServer;
class AgentClient {
public:
AgentClient(MasterServer& server, kota::ipc::JsonPeer& peer);
private:
MasterServer& server;
kota::ipc::JsonPeer& peer;
};
} // namespace clice

View File

@@ -0,0 +1,180 @@
#include "server/service/agentic.h"
#include <memory>
#include <print>
#include <string>
#include "server/protocol/agentic.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "kota/async/async.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/transport.h"
namespace clice {
template <typename Params>
static kota::task<bool> send_and_print(kota::ipc::JsonPeer& peer, Params params) {
auto result = co_await peer.send_request(std::move(params));
if(!result) {
LOG_ERROR("request failed: {}", result.error().message);
co_return false;
}
auto json = kota::codec::json::to_string<kota::ipc::lsp_config>(*result);
std::println("{}", json ? *json : "null");
co_return true;
}
static kota::task<> agentic_request(kota::ipc::JsonPeer& peer,
int& exit_code,
const AgenticQueryOptions& opts) {
bool ok = false;
if(opts.method == "compileCommand") {
ok = co_await send_and_print(peer, agentic::CompileCommandParams{.path = opts.path});
} else if(opts.method == "projectFiles") {
auto filter = opts.query.empty() ? std::nullopt : std::optional(opts.query);
ok = co_await send_and_print(peer, agentic::ProjectFilesParams{.filter = filter});
} else if(opts.method == "symbolSearch") {
ok = co_await send_and_print(peer, agentic::SymbolSearchParams{.query = opts.query});
} else if(opts.method == "definition") {
auto name = opts.name.empty() ? std::nullopt : std::optional(opts.name);
auto path = opts.path.empty() ? std::nullopt : std::optional(opts.path);
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
ok = co_await send_and_print(
peer,
agentic::DefinitionParams{.name = name, .path = path, .line = line});
} else if(opts.method == "references") {
auto name = opts.name.empty() ? std::nullopt : std::optional(opts.name);
auto path = opts.path.empty() ? std::nullopt : std::optional(opts.path);
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
ok = co_await send_and_print(
peer,
agentic::ReferencesParams{.name = name, .path = path, .line = line});
} else if(opts.method == "readSymbol") {
auto name = opts.name.empty() ? std::nullopt : std::optional(opts.name);
auto path = opts.path.empty() ? std::nullopt : std::optional(opts.path);
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
ok = co_await send_and_print(
peer,
agentic::ReadSymbolParams{.name = name, .path = path, .line = line});
} else if(opts.method == "documentSymbols") {
ok = co_await send_and_print(peer, agentic::DocumentSymbolsParams{.path = opts.path});
} else if(opts.method == "callGraph") {
auto name = opts.name.empty() ? std::nullopt : std::optional(opts.name);
auto path = opts.path.empty() ? std::nullopt : std::optional(opts.path);
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
auto dir = opts.direction.empty() ? std::nullopt : std::optional(opts.direction);
ok = co_await send_and_print(peer,
agentic::CallGraphParams{
.name = name,
.path = path,
.line = line,
.direction = dir,
});
} else if(opts.method == "typeHierarchy") {
auto name = opts.name.empty() ? std::nullopt : std::optional(opts.name);
auto path = opts.path.empty() ? std::nullopt : std::optional(opts.path);
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
auto dir = opts.direction.empty() ? std::nullopt : std::optional(opts.direction);
ok = co_await send_and_print(peer,
agentic::TypeHierarchyParams{
.name = name,
.path = path,
.line = line,
.direction = dir,
});
} else if(opts.method == "lint") {
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
ok = co_await send_and_print(peer, agentic::LintParams{.path = opts.path, .line = line});
} else if(opts.method == "fileDeps") {
auto dir = opts.direction.empty() ? std::nullopt : std::optional(opts.direction);
ok = co_await send_and_print(peer,
agentic::FileDepsParams{.path = opts.path, .direction = dir});
} else if(opts.method == "impactAnalysis") {
ok = co_await send_and_print(peer, agentic::ImpactAnalysisParams{.path = opts.path});
} else if(opts.method == "status") {
ok = co_await send_and_print(peer, agentic::StatusParams{});
} else if(opts.method == "shutdown") {
peer.send_notification(agentic::ShutdownParams{});
ok = true;
} else {
LOG_ERROR("unknown agentic method '{}'", opts.method);
}
if(ok)
exit_code = 0;
peer.close();
}
static kota::task<> agentic_client(int& exit_code,
std::unique_ptr<kota::ipc::JsonPeer>& peer_out,
const AgenticQueryOptions& opts) {
auto& loop = kota::event_loop::current();
auto transport = co_await kota::ipc::StreamTransport::connect_tcp(opts.host, opts.port, loop);
if(!transport) {
LOG_ERROR("failed to connect to {}:{}", opts.host, opts.port);
co_return;
}
peer_out = std::make_unique<kota::ipc::JsonPeer>(loop, std::move(*transport));
co_await kota::when_all(peer_out->run(), agentic_request(*peer_out, exit_code, opts));
}
int run_agentic_mode(const AgenticQueryOptions& opts) {
logging::stderr_logger("agentic", logging::options);
kota::event_loop loop;
int exit_code = 1;
std::unique_ptr<kota::ipc::JsonPeer> peer;
loop.schedule(agentic_client(exit_code, peer, opts));
loop.run();
return exit_code;
}
static kota::task<> relay_forward(kota::ipc::Transport& from, kota::ipc::Transport& to) {
while(true) {
auto msg = co_await from.read_message();
if(!msg)
break;
co_await to.write_message(*msg);
}
to.close();
}
static kota::task<> relay_main(kota::event_loop& loop, int& exit_code, std::string socket_path) {
auto stdio = kota::ipc::StreamTransport::open_stdio(loop);
if(!stdio) {
LOG_ERROR("failed to open stdio transport");
loop.stop();
co_return;
}
auto conn = co_await kota::pipe::connect(socket_path, {}, loop);
if(!conn) {
LOG_ERROR("failed to connect to {}", socket_path);
loop.stop();
co_return;
}
auto socket = std::make_unique<kota::ipc::StreamTransport>(std::move(*conn));
co_await kota::when_all(relay_forward(**stdio, *socket), relay_forward(*socket, **stdio));
exit_code = 0;
loop.stop();
}
int run_relay_mode(llvm::StringRef socket_path) {
logging::stderr_logger("relay", logging::options);
auto path = socket_path.empty() ? path::default_socket_path() : socket_path.str();
kota::event_loop loop;
int exit_code = 1;
loop.schedule(relay_main(loop, exit_code, std::move(path)));
loop.run();
return exit_code;
}
} // namespace clice

View File

@@ -0,0 +1,24 @@
#pragma once
#include <string>
#include "llvm/ADT/StringRef.h"
namespace clice {
struct AgenticQueryOptions {
std::string host;
int port = 0;
std::string method;
std::string path;
std::string name;
std::string query;
int line = 0;
std::string direction;
};
int run_agentic_mode(const AgenticQueryOptions& opts);
int run_relay_mode(llvm::StringRef socket_path);
} // namespace clice

View File

@@ -1,4 +1,4 @@
#include "server/master_server.h"
#include "server/service/lsp_client.h"
#include <algorithm>
#include <format>
@@ -7,7 +7,9 @@
#include <variant>
#include "semantic/symbol_kind.h"
#include "server/protocol.h"
#include "server/protocol/extension.h"
#include "server/protocol/worker.h"
#include "server/service/master_server.h"
#include "support/filesystem.h"
#include "support/logging.h"
@@ -16,7 +18,6 @@
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/lsp/uri.h"
#include "kota/meta/enum.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/Process.h"
@@ -29,143 +30,39 @@ using kota::ipc::RequestResult;
using RequestContext = kota::ipc::JsonPeer::RequestContext;
using serde_raw = kota::codec::RawValue;
/// Serialize a value to a JSON RawValue using LSP config.
template <typename T>
static serde_raw to_raw(const T& value) {
auto json = kota::codec::json::to_json<kota::ipc::lsp_config>(value);
return serde_raw{json ? std::move(*json) : "null"};
}
MasterServer::MasterServer(kota::event_loop& loop,
kota::ipc::JsonPeer& peer,
std::string self_path) :
loop(loop), peer(peer), pool(loop), compiler(loop, peer, workspace, pool, sessions),
indexer(loop,
workspace,
sessions,
pool,
compiler,
[this](uint32_t proj_path_id) {
// Bridge project-level path_id to server-level path_id.
// The two PathPools may assign different IDs to the same path.
auto path = workspace.project_index.path_pool.path(proj_path_id);
auto server_id = workspace.path_pool.intern(path);
return sessions.contains(server_id);
}),
self_path(std::move(self_path)) {}
LSPClient::LSPClient(MasterServer& server, kota::ipc::JsonPeer& peer) : server(server), peer(peer) {
server.compiler.set_peer(&peer);
server.indexer.set_peer(&peer);
MasterServer::~MasterServer() = default;
kota::task<> MasterServer::load_workspace() {
if(workspace_root.empty())
co_return;
if(!workspace.config.cache_dir.empty()) {
auto ec = llvm::sys::fs::create_directories(workspace.config.cache_dir);
if(ec) {
LOG_WARN("Failed to create cache directory {}: {}",
workspace.config.cache_dir,
ec.message());
} else {
LOG_INFO("Cache directory: {}", workspace.config.cache_dir);
}
for(auto* subdir: {"cache/pch", "cache/pcm"}) {
auto dir = path::join(workspace.config.cache_dir, subdir);
auto ec2 = llvm::sys::fs::create_directories(dir);
if(ec2) {
LOG_WARN("Failed to create {}: {}", dir, ec2.message());
}
}
// Clean up stale files first, then load — load_cache() only restores
// entries still listed in cache.json, so cleanup won't delete live files.
workspace.cleanup_cache();
workspace.load_cache();
}
std::string cdb_path;
if(!workspace.config.compile_commands_path.empty()) {
if(llvm::sys::fs::exists(workspace.config.compile_commands_path)) {
cdb_path = workspace.config.compile_commands_path;
} else {
LOG_WARN("Configured compile_commands_path not found: {}",
workspace.config.compile_commands_path);
}
}
if(cdb_path.empty()) {
for(auto* subdir: {"build", "cmake-build-debug", "cmake-build-release", "out", "."}) {
auto candidate = path::join(workspace_root, subdir, "compile_commands.json");
if(llvm::sys::fs::exists(candidate)) {
cdb_path = std::move(candidate);
break;
}
}
}
if(cdb_path.empty()) {
LOG_WARN("No compile_commands.json found in workspace {}", workspace_root);
co_return;
}
auto count = workspace.cdb.load(cdb_path);
LOG_INFO("Loaded CDB from {} with {} entries", cdb_path, count);
auto report = scan_dependency_graph(workspace.cdb, workspace.path_pool, workspace.dep_graph);
workspace.dep_graph.build_reverse_map();
auto unresolved = report.includes_found - report.includes_resolved;
double accuracy =
report.includes_found > 0
? 100.0 * static_cast<double>(report.includes_resolved) / report.includes_found
: 100.0;
LOG_INFO(
"Dependency scan: {}ms, {} files ({} source + {} header), " "{} edges, {}/{} resolved ({:.1f}%), {} waves",
report.elapsed_ms,
report.total_files,
report.source_files,
report.header_files,
report.total_edges,
report.includes_resolved,
report.includes_found,
accuracy,
report.waves);
if(unresolved > 0) {
LOG_WARN("{} unresolved includes", unresolved);
}
workspace.build_module_map();
indexer.load(workspace.config.index_dir);
if(workspace.config.enable_indexing) {
for(auto& entry: workspace.cdb.get_entries()) {
auto file = workspace.cdb.resolve_path(entry.file);
auto server_id = workspace.path_pool.intern(file);
indexer.enqueue(server_id);
}
indexer.schedule();
}
compiler.init_compile_graph();
}
void MasterServer::register_handlers() {
using StringVec = std::vector<std::string>;
peer.on_request([this](RequestContext& ctx, const protocol::InitializeParams& params)
-> RequestResult<protocol::InitializeParams> {
if(lifecycle != ServerLifecycle::Uninitialized) {
auto& srv = this->server;
if(srv.lifecycle != ServerLifecycle::Uninitialized) {
co_return kota::outcome_error(protocol::Error{"Server already initialized"});
}
auto& init = params.lsp__initialize_params;
if(init.root_uri.has_value()) {
workspace_root = uri_to_path(*init.root_uri);
srv.workspace_root = uri_to_path(*init.root_uri);
}
lifecycle = ServerLifecycle::Initialized;
LOG_INFO("Initialized with workspace: {}", workspace_root);
if(init.initialization_options.has_value()) {
auto json =
kota::codec::json::to_json<kota::ipc::lsp_config>(*init.initialization_options);
if(json)
srv.init_options_json = std::move(*json);
}
srv.lifecycle = ServerLifecycle::Initialized;
LOG_INFO("Initialized with workspace: {}", srv.workspace_root);
protocol::InitializeResult result;
auto& caps = result.capabilities;
@@ -188,7 +85,6 @@ void MasterServer::register_handlers() {
caps.signature_help_provider = protocol::SignatureHelpOptions{
.trigger_characters = StringVec{"(", ")", "{", "}", "<", ">", ","},
};
/// FIXME: In the future, we would support work done progress.
caps.declaration_provider = protocol::DeclarationOptions{
.work_done_progress = false,
};
@@ -212,6 +108,8 @@ void MasterServer::register_handlers() {
caps.call_hierarchy_provider = true;
caps.type_hierarchy_provider = true;
caps.workspace_symbol_provider = true;
caps.document_formatting_provider = true;
caps.document_range_formatting_provider = true;
protocol::SemanticTokensOptions sem_opts;
{
@@ -243,80 +141,32 @@ void MasterServer::register_handlers() {
co_return result;
});
peer.on_notification([this](const protocol::InitializedParams& params) {
workspace.config = CliceConfig::load_from_workspace(workspace_root);
if(!workspace.config.logging_dir.empty()) {
auto now = std::chrono::system_clock::now();
auto pid = llvm::sys::Process::getProcessId();
auto session_dir = path::join(workspace.config.logging_dir,
std::format("{:%Y-%m-%d_%H-%M-%S}_{}", now, pid));
logging::file_logger("master", session_dir, logging::options);
session_log_dir = session_dir;
}
LOG_INFO("Server ready (stateful={}, stateless={}, idle={}ms)",
workspace.config.stateful_worker_count,
workspace.config.stateless_worker_count,
workspace.config.idle_timeout_ms);
WorkerPoolOptions pool_opts;
pool_opts.self_path = self_path;
pool_opts.stateful_count = workspace.config.stateful_worker_count;
pool_opts.stateless_count = workspace.config.stateless_worker_count;
pool_opts.worker_memory_limit = workspace.config.worker_memory_limit;
pool_opts.log_dir = session_log_dir;
if(!pool.start(pool_opts)) {
LOG_ERROR("Failed to start worker pool");
return;
}
lifecycle = ServerLifecycle::Ready;
compiler.on_indexing_needed = [this]() {
indexer.schedule();
};
loop.schedule(load_workspace());
peer.on_notification([this]([[maybe_unused]] const protocol::InitializedParams& params) {
this->server.initialize();
});
peer.on_request(
[this](RequestContext& ctx,
const protocol::ShutdownParams& params) -> RequestResult<protocol::ShutdownParams> {
lifecycle = ServerLifecycle::ShuttingDown;
this->server.lifecycle = ServerLifecycle::ShuttingDown;
LOG_INFO("Shutdown requested");
co_return nullptr;
});
peer.on_notification([this](const protocol::ExitParams& params) {
lifecycle = ServerLifecycle::Exited;
peer.on_notification([this]([[maybe_unused]] const protocol::ExitParams& params) {
LOG_INFO("Exit notification received");
indexer.save(workspace.config.index_dir);
workspace.save_cache();
loop.schedule([this]() -> kota::task<> {
co_await pool.stop();
loop.stop();
}());
this->server.schedule_shutdown();
});
/// Document lifecycle — handled directly by MasterServer.
peer.on_notification([this](const protocol::DidOpenTextDocumentParams& params) {
if(lifecycle != ServerLifecycle::Ready)
auto& srv = this->server;
if(srv.lifecycle != ServerLifecycle::Ready)
return;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto path_id = srv.workspace.path_pool.intern(path);
auto [it, inserted] = sessions.try_emplace(path_id);
auto& session = it->second;
if(!inserted) {
// DenseMap tombstone may retain stale data — reset to a fresh Session.
session = Session{};
}
session.path_id = path_id;
auto& session = srv.open_session(path_id);
session.version = params.text_document.version;
session.text = params.text_document.text;
session.generation++;
@@ -325,18 +175,18 @@ void MasterServer::register_handlers() {
});
peer.on_notification([this](const protocol::DidChangeTextDocumentParams& params) {
if(lifecycle != ServerLifecycle::Ready)
auto& srv = this->server;
if(srv.lifecycle != ServerLifecycle::Ready)
return;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto path_id = srv.workspace.path_pool.intern(path);
auto it = sessions.find(path_id);
if(it == sessions.end())
auto* session = srv.find_session(path_id);
if(!session)
return;
auto& session = it->second;
session.version = params.text_document.version;
session->version = params.text_document.version;
for(auto& change: params.content_changes) {
std::visit(
@@ -344,186 +194,157 @@ void MasterServer::register_handlers() {
using T = std::remove_cvref_t<decltype(c)>;
if constexpr(std::is_same_v<T,
protocol::TextDocumentContentChangeWholeDocument>) {
session.text = c.text;
session->text = c.text;
} else {
auto& range = c.range;
lsp::PositionMapper mapper(session.text, lsp::PositionEncoding::UTF16);
lsp::PositionMapper mapper(session->text, lsp::PositionEncoding::UTF16);
auto start = mapper.to_offset(range.start);
auto end = mapper.to_offset(range.end);
if(start && end && *start <= *end) {
session.text.replace(*start, *end - *start, c.text);
session->text.replace(*start, *end - *start, c.text);
}
}
},
change);
}
session.generation++;
session.ast_dirty = true;
session->generation++;
session->ast_dirty = true;
LOG_DEBUG("didChange: path={} version={} gen={}",
path,
session.version,
session.generation);
session->version,
session->generation);
worker::DocumentUpdateParams update;
update.path = path;
update.version = session.version;
pool.notify_stateful(path_id, update);
update.version = session->version;
srv.pool.notify_stateful(path_id, update);
});
peer.on_notification([this](const protocol::DidCloseTextDocumentParams& params) {
if(lifecycle != ServerLifecycle::Ready)
auto& srv = this->server;
if(srv.lifecycle != ServerLifecycle::Ready)
return;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
workspace.on_file_closed(path_id);
pool.notify_stateful(path_id, worker::EvictParams{path});
// Clear diagnostics for the closed file.
protocol::PublishDiagnosticsParams diag_params;
diag_params.uri = params.text_document.uri;
peer.send_notification(diag_params);
sessions.erase(path_id);
indexer.enqueue(path_id);
indexer.schedule();
LOG_DEBUG("didClose: {}", path);
auto path_id = srv.workspace.path_pool.intern(uri_to_path(params.text_document.uri));
srv.close_session(path_id, this->peer);
});
peer.on_notification([this](const protocol::DidSaveTextDocumentParams& params) {
if(lifecycle != ServerLifecycle::Ready)
auto& srv = this->server;
if(srv.lifecycle != ServerLifecycle::Ready)
return;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto dirtied = workspace.on_file_saved(path_id);
for(auto dirty_id: dirtied) {
if(auto sit = sessions.find(dirty_id); sit != sessions.end()) {
sit->second.ast_dirty = true;
} else {
indexer.enqueue(dirty_id);
}
}
// Invalidate header contexts for sessions whose host is this file.
for(auto& [hdr_id, session]: sessions) {
if(session.header_context && session.header_context->host_path_id == path_id) {
session.header_context.reset();
session.ast_dirty = true;
}
}
indexer.schedule();
auto path_id = srv.workspace.path_pool.intern(path);
srv.on_file_saved(path_id);
LOG_DEBUG("didSave: {}", path);
});
/// Feature requests — stateful forwarding.
peer.on_request([this](RequestContext& ctx, const protocol::HoverParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document_position_params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::Hover,
sit->second,
params.text_document_position_params.position);
co_return co_await srv.compiler.forward_query(
worker::QueryKind::Hover,
*session,
params.text_document_position_params.position);
});
peer.on_request([this](RequestContext& ctx,
const protocol::SemanticTokensParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::SemanticTokens, sit->second);
co_return co_await srv.compiler.forward_query(worker::QueryKind::SemanticTokens, *session);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::InlayHintParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::InlayHints,
sit->second,
{},
params.range);
co_return co_await srv.compiler.forward_query(worker::QueryKind::InlayHints,
*session,
{},
params.range);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::FoldingRangeParams& params) -> RawResult {
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::FoldingRange, sit->second);
});
peer.on_request([this](RequestContext& ctx,
const protocol::FoldingRangeParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await srv.compiler.forward_query(worker::QueryKind::FoldingRange, *session);
});
peer.on_request([this](RequestContext& ctx,
const protocol::DocumentSymbolParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::DocumentSymbol, sit->second);
});
peer.on_request([this](RequestContext& ctx,
const protocol::DocumentLinkParams& params) -> RawResult {
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
co_return serde_raw{"null"};
auto& session = sit->second;
auto result = co_await compiler.forward_query(worker::QueryKind::DocumentLink, session);
if(!result.has_value())
co_return serde_raw{"null"};
// Merge document links from PCH if available.
auto& links = result.value();
// Re-lookup session after co_await since iterators may be invalidated.
auto sit2 = sessions.find(path_id);
if(sit2 != sessions.end() && sit2->second.pch_ref) {
auto pch_it = workspace.pch_cache.find(sit2->second.pch_ref->path_id);
if(pch_it != workspace.pch_cache.end() && !pch_it->second.document_links_json.empty()) {
auto& pch_json = pch_it->second.document_links_json;
// Merge two JSON arrays.
if(!links.data.empty() && links.data != "null" && links.data.size() > 2) {
// "[a,b]" + "[c,d]" -> "[a,b,c,d]"
links.data.pop_back(); // remove trailing ']'
links.data += ',';
links.data.append(pch_json.begin() + 1, pch_json.end()); // skip '['
} else {
links.data = pch_json;
}
}
}
co_return std::move(links);
co_return co_await srv.compiler.forward_query(worker::QueryKind::DocumentSymbol, *session);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::CodeActionParams& params) -> RawResult {
[this](RequestContext& ctx, const protocol::DocumentLinkParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::CodeAction, sit->second);
auto result =
co_await srv.compiler.forward_query(worker::QueryKind::DocumentLink, *session);
if(!result.has_value())
co_return serde_raw{"null"};
auto& links = result.value();
auto* session2 = srv.find_session(path_id);
if(session2 && session2->pch_ref) {
auto& pch_cache = srv.workspace.pch_cache;
auto pch_it = pch_cache.find(session2->pch_ref->path_id);
if(pch_it != pch_cache.end() && !pch_it->second.document_links_json.empty()) {
auto& pch_json = pch_it->second.document_links_json;
if(!links.data.empty() && links.data != "null" && links.data.size() > 2) {
links.data.pop_back();
links.data += ',';
links.data.append(pch_json.begin() + 1, pch_json.end());
} else {
links.data = pch_json;
}
}
}
co_return std::move(links);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::CodeActionParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await srv.compiler.forward_query(worker::QueryKind::CodeAction, *session);
});
/// Helper: resolve URI to path, path_id, and Session pointer.
auto resolve_uri = [this](const std::string& uri) {
struct Result {
std::string path;
@@ -531,22 +352,21 @@ void MasterServer::register_handlers() {
Session* session;
};
auto path = uri_to_path(uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
Session* session = (sit != sessions.end()) ? &sit->second : nullptr;
auto path_id = this->server.workspace.path_pool.intern(path);
auto* session = this->server.find_session(path_id);
return Result{std::move(path), path_id, session};
};
auto lookup_at = [this, resolve_uri](const std::string& uri, const protocol::Position& pos) {
auto [path, path_id, session] = resolve_uri(uri);
return indexer.lookup_symbol(uri, path, pos, session);
return this->server.indexer.lookup_symbol(uri, path, pos, session);
};
auto query_at = [this, resolve_uri](const std::string& uri,
const protocol::Position& pos,
RelationKind kind) -> std::vector<protocol::Location> {
auto [path, path_id, session] = resolve_uri(uri);
return indexer.query_relations(path, pos, kind, session);
return this->server.indexer.query_relations(path, pos, kind, session);
};
auto resolve_item =
@@ -555,11 +375,9 @@ void MasterServer::register_handlers() {
const protocol::Range& range,
const std::optional<protocol::LSPAny>& data) -> std::optional<SymbolInfo> {
auto [path, path_id, session] = resolve_uri(uri);
return indexer.resolve_hierarchy_item(uri, path, range, data, session);
return this->server.indexer.resolve_hierarchy_item(uri, path, range, data, session);
};
/// Feature requests — index-based with AST fallback.
peer.on_request([this, query_at](RequestContext& ctx,
const protocol::DefinitionParams& params) -> RawResult {
auto& uri = params.text_document_position_params.text_document.uri;
@@ -570,14 +388,15 @@ void MasterServer::register_handlers() {
co_return to_raw(result);
}
auto& srv = this->server;
auto path = uri_to_path(uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::GoToDefinition,
sit->second,
pos);
co_return co_await srv.compiler.forward_query(worker::QueryKind::GoToDefinition,
*session,
pos);
});
peer.on_request([this, query_at](RequestContext& ctx,
@@ -614,32 +433,60 @@ void MasterServer::register_handlers() {
co_return serde_raw{"null"};
});
/// Feature requests — stateless forwarding.
peer.on_request([this](RequestContext& ctx,
const protocol::CompletionParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document_position_params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.handle_completion(params.text_document_position_params.position,
sit->second);
auto pause = srv.indexer.scoped_pause();
auto result =
co_await srv.compiler.handle_completion(params.text_document_position_params.position,
*session);
co_return std::move(result);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::SignatureHelpParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document_position_params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_build(worker::BuildKind::SignatureHelp,
params.text_document_position_params.position,
sit->second);
auto pause = srv.indexer.scoped_pause();
auto result =
co_await srv.compiler.forward_build(worker::BuildKind::SignatureHelp,
params.text_document_position_params.position,
*session);
co_return std::move(result);
});
/// Hierarchy queries — index-based.
peer.on_request(
[this](RequestContext& ctx, const protocol::DocumentFormattingParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
auto pause = srv.indexer.scoped_pause();
co_return co_await srv.compiler.forward_format(*session);
});
peer.on_request([this](RequestContext& ctx,
const protocol::DocumentRangeFormattingParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
auto pause = srv.indexer.scoped_pause();
co_return co_await srv.compiler.forward_format(*session, params.range);
});
peer.on_request(
[this, lookup_at](RequestContext& ctx,
@@ -664,7 +511,7 @@ void MasterServer::register_handlers() {
auto info = resolve_item(params.item.uri, params.item.range, params.item.data);
if(!info)
co_return serde_raw{"null"};
auto results = indexer.find_incoming_calls(info->hash);
auto results = this->server.indexer.find_incoming_calls(info->hash);
if(results.empty())
co_return serde_raw{"null"};
co_return to_raw(results);
@@ -676,7 +523,7 @@ void MasterServer::register_handlers() {
auto info = resolve_item(params.item.uri, params.item.range, params.item.data);
if(!info)
co_return serde_raw{"null"};
auto results = indexer.find_outgoing_calls(info->hash);
auto results = this->server.indexer.find_outgoing_calls(info->hash);
if(results.empty())
co_return serde_raw{"null"};
co_return to_raw(results);
@@ -706,7 +553,7 @@ void MasterServer::register_handlers() {
auto info = resolve_item(params.item.uri, params.item.range, params.item.data);
if(!info)
co_return serde_raw{"null"};
auto results = indexer.find_supertypes(info->hash);
auto results = this->server.indexer.find_supertypes(info->hash);
if(results.empty())
co_return serde_raw{"null"};
co_return to_raw(results);
@@ -718,7 +565,7 @@ void MasterServer::register_handlers() {
auto info = resolve_item(params.item.uri, params.item.range, params.item.data);
if(!info)
co_return serde_raw{"null"};
auto results = indexer.find_subtypes(info->hash);
auto results = this->server.indexer.find_subtypes(info->hash);
if(results.empty())
co_return serde_raw{"null"};
co_return to_raw(results);
@@ -726,29 +573,29 @@ void MasterServer::register_handlers() {
peer.on_request(
[this](RequestContext& ctx, const protocol::WorkspaceSymbolParams& params) -> RawResult {
auto results = indexer.search_symbols(params.query);
auto results = this->server.indexer.search_symbols(params.query);
if(results.empty())
co_return serde_raw{"null"};
co_return to_raw(results);
});
/// clice/ extension commands.
peer.on_request(
"clice/queryContext",
[this](RequestContext& ctx, const ext::QueryContextParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.uri);
auto path_id = workspace.path_pool.intern(path);
auto path_id = srv.workspace.path_pool.intern(path);
int offset_val = std::max(0, params.offset.value_or(0));
constexpr int page_size = 10;
ext::QueryContextResult result;
std::vector<ext::ContextItem> all_items;
auto hosts = workspace.dep_graph.find_host_sources(path_id);
auto& ws = srv.workspace;
auto hosts = ws.dep_graph.find_host_sources(path_id);
for(auto host_id: hosts) {
auto host_path = workspace.path_pool.resolve(host_id);
auto host_cdb = workspace.cdb.lookup(host_path, {.suppress_logging = true});
auto host_path = ws.path_pool.resolve(host_id);
auto host_cdb = ws.cdb.lookup(host_path, {.suppress_logging = true});
if(host_cdb.empty())
continue;
auto host_uri_opt = lsp::URI::from_file_path(std::string(host_path));
@@ -762,7 +609,7 @@ void MasterServer::register_handlers() {
}
if(hosts.empty()) {
auto entries = workspace.cdb.lookup(path, {.suppress_logging = true});
auto entries = ws.cdb.lookup(path, {.suppress_logging = true});
for(std::size_t i = 0; i < entries.size(); ++i) {
auto& cmd = entries[i];
auto argv = cmd.to_argv();
@@ -804,13 +651,14 @@ void MasterServer::register_handlers() {
peer.on_request(
"clice/currentContext",
[this](RequestContext& ctx, const ext::CurrentContextParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.uri);
auto path_id = workspace.path_pool.intern(path);
auto path_id = srv.workspace.path_pool.intern(path);
ext::CurrentContextResult result;
auto sit = sessions.find(path_id);
if(sit != sessions.end() && sit->second.active_context) {
auto ctx_path = workspace.path_pool.resolve(*sit->second.active_context);
auto* session = srv.find_session(path_id);
if(session && session->active_context) {
auto ctx_path = srv.workspace.path_pool.resolve(*session->active_context);
auto ctx_uri_opt = lsp::URI::from_file_path(std::string(ctx_path));
if(ctx_uri_opt) {
ext::ContextItem item;
@@ -826,34 +674,41 @@ void MasterServer::register_handlers() {
peer.on_request(
"clice/switchContext",
[this](RequestContext& ctx, const ext::SwitchContextParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.uri);
auto path_id = workspace.path_pool.intern(path);
auto path_id = srv.workspace.path_pool.intern(path);
auto context_path = uri_to_path(params.context_uri);
auto context_path_id = workspace.path_pool.intern(context_path);
auto context_path_id = srv.workspace.path_pool.intern(context_path);
ext::SwitchContextResult result;
auto context_cdb = workspace.cdb.lookup(context_path, {.suppress_logging = true});
auto& ws = srv.workspace;
auto context_cdb = ws.cdb.lookup(context_path, {.suppress_logging = true});
if(context_cdb.empty()) {
result.success = false;
co_return to_raw(result);
}
auto sit = sessions.find(path_id);
if(sit == sessions.end()) {
auto* session = srv.find_session(path_id);
if(!session) {
result.success = false;
co_return to_raw(result);
}
sit->second.active_context = context_path_id;
sit->second.header_context.reset();
sit->second.pch_ref.reset();
sit->second.ast_deps.reset();
sit->second.ast_dirty = true;
session->active_context = context_path_id;
session->header_context.reset();
session->pch_ref.reset();
session->ast_deps.reset();
session->ast_dirty = true;
result.success = true;
co_return to_raw(result);
});
}
LSPClient::~LSPClient() {
server.compiler.set_peer(nullptr);
server.indexer.set_peer(nullptr);
}
} // namespace clice

View File

@@ -0,0 +1,23 @@
#pragma once
#include "kota/async/async.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
namespace clice {
class MasterServer;
class LSPClient {
public:
LSPClient(MasterServer& server, kota::ipc::JsonPeer& peer);
~LSPClient();
private:
using RawResult = kota::task<kota::codec::RawValue, kota::ipc::Error>;
MasterServer& server;
kota::ipc::JsonPeer& peer;
};
} // namespace clice

View File

@@ -0,0 +1,551 @@
#include "server/service/master_server.h"
#include <cerrno>
#include <cstring>
#include <list>
#include <memory>
#include <string>
#include <vector>
#ifndef _WIN32
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#endif
#include "server/protocol/worker.h"
#include "server/service/agent_client.h"
#include "server/service/lsp_client.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "kota/async/async.h"
#include "kota/async/io/fs_event.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/lsp/uri.h"
#include "kota/ipc/recording_transport.h"
#include "kota/ipc/transport.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/Process.h"
namespace clice {
namespace lsp = kota::ipc::lsp;
namespace protocol = kota::ipc::protocol;
MasterServer::MasterServer(kota::event_loop& loop, std::string self_path) :
loop(loop), pool(loop), compiler(loop, workspace, pool, sessions),
indexer(loop,
workspace,
sessions,
pool,
compiler,
[this](uint32_t proj_path_id) {
auto path = workspace.project_index.path_pool.path(proj_path_id);
auto server_id = workspace.path_pool.intern(path);
return sessions.contains(server_id);
}),
self_path(std::move(self_path)) {}
MasterServer::~MasterServer() = default;
void MasterServer::initialize() {
workspace.config = Config::load_from_workspace(workspace_root);
if(!init_options_json.empty()) {
if(auto ov = kota::codec::json::parse(init_options_json, workspace.config); !ov) {
LOG_WARN("Failed to apply initializationOptions: {}", ov.error().to_string());
} else {
workspace.config.apply_defaults(workspace_root);
LOG_INFO("Applied initializationOptions overlay");
}
init_options_json.clear();
}
auto& cfg = workspace.config.project;
if(!cfg.logging_dir.empty()) {
auto now = std::chrono::system_clock::now();
auto pid = llvm::sys::Process::getProcessId();
session_log_dir =
path::join(cfg.logging_dir, std::format("{:%Y-%m-%d_%H-%M-%S}_{}", now, pid));
logging::file_logger("master", session_log_dir, logging::options);
}
LOG_INFO("Server ready (stateful={}, stateless={}, idle={}ms)",
cfg.stateful_worker_count.value,
cfg.stateless_worker_count.value,
*cfg.idle_timeout_ms);
WorkerPoolOptions pool_opts;
pool_opts.self_path = self_path;
pool_opts.stateful_count = cfg.stateful_worker_count;
pool_opts.stateless_count = cfg.stateless_worker_count;
pool_opts.worker_memory_limit = cfg.worker_memory_limit;
pool_opts.log_dir = session_log_dir;
if(!pool.start(pool_opts)) {
LOG_ERROR("Failed to start worker pool");
return;
}
lifecycle = ServerLifecycle::Ready;
compiler.on_indexing_needed = [this]() {
indexer.schedule();
};
indexer.set_max_concurrency(cfg.stateless_worker_count.value);
load_workspace();
}
void MasterServer::initialize(llvm::StringRef root) {
workspace_root = root.str();
initialize();
}
void MasterServer::start_file_watcher() {
if(workspace_root.empty())
return;
loop.schedule([this]() -> kota::task<> {
auto watcher = kota::fs_event::create(workspace_root, {}, loop);
if(!watcher) {
LOG_WARN("Failed to start file watcher for {}", workspace_root);
co_return;
}
LOG_INFO("File watcher started for {}", workspace_root);
while(true) {
auto changes = co_await watcher->next();
if(!changes)
break;
for(auto& change: *changes) {
if(change.type != kota::fs_event::effect::modify &&
change.type != kota::fs_event::effect::create)
continue;
llvm::StringRef file(change.path);
if(file.ends_with("compile_commands.json")) {
LOG_INFO("CDB changed, reloading workspace");
load_workspace();
continue;
}
if(file.ends_with(".cpp") || file.ends_with(".cc") || file.ends_with(".cxx") ||
file.ends_with(".c") || file.ends_with(".h") || file.ends_with(".hpp") ||
file.ends_with(".hxx") || file.ends_with(".cppm") || file.ends_with(".ixx")) {
auto path_id = workspace.path_pool.intern(file);
on_file_saved(path_id);
}
}
}
}());
}
Session* MasterServer::find_session(std::uint32_t path_id) {
auto it = sessions.find(path_id);
return it != sessions.end() ? &it->second : nullptr;
}
Session& MasterServer::open_session(std::uint32_t path_id) {
auto [it, inserted] = sessions.try_emplace(path_id);
auto& session = it->second;
if(!inserted)
session = Session{};
session.path_id = path_id;
return session;
}
void MasterServer::close_session(std::uint32_t path_id, kota::ipc::JsonPeer& peer) {
namespace protocol = kota::ipc::protocol;
auto path = workspace.path_pool.resolve(path_id);
workspace.on_file_closed(path_id);
pool.notify_stateful(path_id, worker::EvictParams{std::string(path)});
protocol::PublishDiagnosticsParams diag_params;
auto uri = lsp::URI::from_file_path(std::string(path));
if(uri)
diag_params.uri = uri->str();
diag_params.diagnostics = {};
peer.send_notification(diag_params);
sessions.erase(path_id);
indexer.enqueue(path_id);
indexer.schedule();
LOG_DEBUG("didClose: {}", path);
}
void MasterServer::on_file_saved(std::uint32_t path_id) {
auto dirtied = workspace.on_file_saved(path_id);
for(auto dirty_id: dirtied) {
if(auto* session = find_session(dirty_id)) {
session->ast_dirty = true;
} else {
indexer.enqueue(dirty_id);
}
}
for(auto& [hdr_id, session]: sessions) {
if(session.header_context && session.header_context->host_path_id == path_id) {
session.header_context.reset();
session.ast_dirty = true;
}
}
indexer.schedule();
}
void MasterServer::schedule_shutdown() {
if(lifecycle == ServerLifecycle::Exited)
return;
lifecycle = ServerLifecycle::Exited;
indexer.save(workspace.config.project.index_dir);
workspace.save_cache();
shutdown_event.set();
loop.schedule([this]() -> kota::task<> {
co_await kota::when_all(indexer.stop(), compiler.stop(), pool.stop());
loop.stop();
}());
}
void MasterServer::load_workspace() {
if(workspace_root.empty())
return;
auto& cfg = workspace.config.project;
if(!cfg.cache_dir.empty()) {
auto ec = llvm::sys::fs::create_directories(cfg.cache_dir);
if(ec) {
LOG_WARN("Failed to create cache directory {}: {}",
std::string_view(cfg.cache_dir),
ec.message());
} else {
LOG_INFO("Cache directory: {}", std::string_view(cfg.cache_dir));
}
for(auto* subdir: {"cache/pch", "cache/pcm"}) {
auto dir = path::join(cfg.cache_dir, subdir);
if(auto ec2 = llvm::sys::fs::create_directories(dir))
LOG_WARN("Failed to create {}: {}", dir, ec2.message());
}
workspace.cleanup_cache();
workspace.load_cache();
}
std::string cdb_path;
for(auto& configured: cfg.compile_commands_paths) {
if(llvm::sys::fs::is_directory(configured)) {
auto candidate = path::join(configured, "compile_commands.json");
if(llvm::sys::fs::exists(candidate)) {
cdb_path = std::move(candidate);
break;
}
} else if(llvm::sys::fs::exists(configured)) {
cdb_path = configured;
break;
} else {
LOG_WARN("Configured compile_commands_path not found: {}", configured);
}
}
if(cdb_path.empty()) {
auto try_candidate = [&](llvm::StringRef dir) -> bool {
auto candidate = path::join(dir, "compile_commands.json");
if(llvm::sys::fs::exists(candidate)) {
cdb_path = std::move(candidate);
return true;
}
return false;
};
if(!try_candidate(workspace_root)) {
std::error_code ec;
for(llvm::sys::fs::directory_iterator it(workspace_root, ec), end; it != end && !ec;
it.increment(ec)) {
if(it->type() == llvm::sys::fs::file_type::directory_file) {
if(try_candidate(it->path()))
break;
}
}
}
}
if(cdb_path.empty()) {
LOG_WARN("No compile_commands.json found in workspace {}", workspace_root);
return;
}
auto count = workspace.cdb.load(cdb_path);
LOG_INFO("Loaded CDB from {} with {} entries", cdb_path, count);
auto report = scan_dependency_graph(workspace.cdb,
workspace.path_pool,
workspace.dep_graph,
/*cache=*/nullptr,
[this](llvm::StringRef path,
std::vector<std::string>& append,
std::vector<std::string>& remove) {
workspace.config.match_rules(path, append, remove);
});
workspace.dep_graph.build_reverse_map();
auto unresolved = report.includes_found - report.includes_resolved;
double accuracy =
report.includes_found > 0
? 100.0 * static_cast<double>(report.includes_resolved) / report.includes_found
: 100.0;
LOG_INFO(
"Dependency scan: {}ms, {} files ({} source + {} header), " "{} edges, {}/{} resolved ({:.1f}%), {} waves",
report.elapsed_ms,
report.total_files,
report.source_files,
report.header_files,
report.total_edges,
report.includes_resolved,
report.includes_found,
accuracy,
report.waves);
if(unresolved > 0)
LOG_WARN("{} unresolved includes", unresolved);
workspace.build_module_map();
indexer.load(cfg.index_dir);
if(*cfg.enable_indexing) {
for(auto& entry: workspace.cdb.get_entries()) {
auto file = workspace.cdb.resolve_path(entry.file);
auto server_id = workspace.path_pool.intern(file);
indexer.enqueue(server_id);
}
indexer.schedule();
}
compiler.init_compile_graph();
}
struct Connection {
std::unique_ptr<kota::ipc::JsonPeer> peer;
std::unique_ptr<LSPClient> lsp_client;
std::unique_ptr<AgentClient> agent_client;
};
static kota::task<> run_connection(kota::ipc::JsonPeer* peer,
std::list<Connection>& connections,
std::list<Connection>::iterator pos) {
co_await peer->run();
LOG_INFO("Client disconnected");
connections.erase(pos);
}
static kota::task<> accept_connections(MasterServer& server,
kota::tcp::acceptor acceptor,
bool register_lsp,
std::list<Connection>& connections) {
auto& loop = kota::event_loop::current();
kota::task_group<> connection_group(loop);
bool lsp_registered = false;
while(true) {
auto conn = co_await acceptor.accept();
if(!conn.has_value())
break;
LOG_INFO("Client connected");
auto transport = std::make_unique<kota::ipc::StreamTransport>(std::move(*conn));
auto peer = std::make_unique<kota::ipc::JsonPeer>(loop, std::move(transport));
std::unique_ptr<LSPClient> lsp;
if(register_lsp && !lsp_registered) {
lsp = std::make_unique<LSPClient>(server, *peer);
lsp_registered = true;
}
auto agent = std::make_unique<AgentClient>(server, *peer);
auto* peer_ptr = peer.get();
auto it = connections.emplace(connections.end(),
Connection{
.peer = std::move(peer),
.lsp_client = std::move(lsp),
.agent_client = std::move(agent),
});
connection_group.spawn(run_connection(peer_ptr, connections, it));
}
co_await connection_group.join();
}
int run_server_mode(const ServerOptions& opts) {
logging::stderr_logger("master", logging::options);
kota::event_loop loop;
MasterServer server(loop, opts.self_path);
std::list<Connection> connections;
if(opts.mode == "pipe") {
auto transport = kota::ipc::StreamTransport::open_stdio(loop);
if(!transport) {
LOG_ERROR("failed to open stdio transport");
return 1;
}
std::unique_ptr<kota::ipc::Transport> final_transport = std::move(*transport);
if(!opts.record.empty()) {
final_transport =
std::make_unique<kota::ipc::RecordingTransport>(std::move(final_transport),
opts.record);
}
kota::ipc::JsonPeer lsp_peer(loop, std::move(final_transport));
LSPClient lsp_client(server, lsp_peer);
if(opts.port > 0) {
auto acceptor = kota::tcp::listen(opts.host, opts.port, {}, loop);
if(acceptor) {
LOG_INFO("Agentic protocol listening on {}:{}", opts.host, opts.port);
loop.schedule(accept_connections(server, std::move(*acceptor), false, connections));
} else {
LOG_WARN("Failed to start agentic listener on {}:{}", opts.host, opts.port);
}
}
loop.schedule(lsp_peer.run());
loop.run();
return 0;
}
if(opts.mode == "socket") {
auto acceptor = kota::tcp::listen(opts.host, opts.port, {}, loop);
if(!acceptor) {
LOG_ERROR("failed to listen on {}:{}", opts.host, opts.port);
return 1;
}
LOG_INFO("Listening on {}:{} ...", opts.host, opts.port);
loop.schedule(accept_connections(server, std::move(*acceptor), true, connections));
loop.run();
return 0;
}
LOG_ERROR("unknown server mode '{}'", opts.mode);
return 1;
}
struct DaemonConnection {
std::unique_ptr<kota::ipc::JsonPeer> peer;
std::unique_ptr<AgentClient> agent_client;
};
static kota::task<> run_daemon_connection(kota::ipc::JsonPeer* peer,
std::list<DaemonConnection>& connections,
std::list<DaemonConnection>::iterator pos) {
co_await peer->run();
LOG_INFO("Daemon client disconnected");
connections.erase(pos);
}
static kota::task<> daemon_main(MasterServer& server, kota::pipe::acceptor acceptor) {
auto& loop = kota::event_loop::current();
std::list<DaemonConnection> connections;
kota::task_group<> connection_group(loop);
co_await kota::when_all(
[&]() -> kota::task<> {
while(true) {
auto conn = co_await acceptor.accept();
if(!conn.has_value())
break;
LOG_INFO("Daemon client connected");
auto transport = std::make_unique<kota::ipc::StreamTransport>(std::move(*conn));
auto peer = std::make_unique<kota::ipc::JsonPeer>(loop, std::move(transport));
auto agent = std::make_unique<AgentClient>(server, *peer);
auto* peer_ptr = peer.get();
auto it = connections.emplace(connections.end(),
DaemonConnection{
.peer = std::move(peer),
.agent_client = std::move(agent),
});
connection_group.spawn(run_daemon_connection(peer_ptr, connections, it));
}
}(),
[&]() -> kota::task<> {
co_await server.get_shutdown_event().wait();
acceptor.stop();
for(auto& conn: connections) {
conn.peer->close();
}
}());
co_await connection_group.join();
}
int run_daemon_mode(const DaemonOptions& opts) {
logging::stderr_logger("daemon", logging::options);
auto socket_path = opts.socket_path.empty() ? path::default_socket_path() : opts.socket_path;
auto socket_dir = llvm::sys::path::parent_path(socket_path);
if(auto ec = llvm::sys::fs::create_directories(socket_dir)) {
LOG_ERROR("Failed to create socket directory {}: {}", socket_dir, ec.message());
return 1;
}
if(llvm::sys::fs::exists(socket_path)) {
#ifndef _WIN32
int fd = ::socket(AF_UNIX, SOCK_STREAM, 0);
if(fd >= 0) {
struct sockaddr_un addr{};
addr.sun_family = AF_UNIX;
auto len = std::min(socket_path.size(), sizeof(addr.sun_path) - 1);
std::memcpy(addr.sun_path, socket_path.data(), len);
bool live = ::connect(fd, reinterpret_cast<struct sockaddr*>(&addr), sizeof(addr)) == 0;
::close(fd);
if(live) {
LOG_ERROR("Another daemon is already running on {}", socket_path);
return 1;
}
}
#endif
llvm::sys::fs::remove(socket_path);
}
kota::event_loop loop;
MasterServer server(loop, opts.self_path);
if(!opts.workspace.empty()) {
server.initialize(opts.workspace);
server.start_file_watcher();
}
auto acceptor = kota::pipe::listen(socket_path, {}, loop);
if(!acceptor) {
LOG_ERROR("Failed to listen on {}", socket_path);
return 1;
}
LOG_INFO("Daemon listening on {}", socket_path);
loop.schedule(daemon_main(server, std::move(*acceptor)));
loop.run();
llvm::sys::fs::remove(socket_path);
return 0;
}
} // namespace clice

View File

@@ -0,0 +1,93 @@
#pragma once
#include <cstdint>
#include <string>
#include "server/compiler/compiler.h"
#include "server/compiler/indexer.h"
#include "server/service/session.h"
#include "server/worker/worker_pool.h"
#include "server/workspace/workspace.h"
#include "kota/async/async.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/StringRef.h"
namespace clice {
enum class ServerLifecycle : std::uint8_t {
Uninitialized,
Initialized,
Ready,
ShuttingDown,
Exited,
};
/// Core server state — owns the two-layer state model (Workspace + Sessions),
/// the worker pool, compilation engine, and indexer.
///
/// Does NOT own any transport or peer. Protocol-specific handler registration
/// is done by LSPClient and AgentClient, which access private members directly.
class MasterServer {
friend class LSPClient;
friend class AgentClient;
public:
MasterServer(kota::event_loop& loop, std::string self_path);
~MasterServer();
void initialize();
void initialize(llvm::StringRef root);
void start_file_watcher();
Session* find_session(std::uint32_t path_id);
Session& open_session(std::uint32_t path_id);
void close_session(std::uint32_t path_id, kota::ipc::JsonPeer& peer);
void on_file_saved(std::uint32_t path_id);
void schedule_shutdown();
kota::event& get_shutdown_event() {
return shutdown_event;
}
private:
kota::event shutdown_event;
void load_workspace();
kota::event_loop& loop;
Workspace workspace;
llvm::DenseMap<std::uint32_t, Session> sessions;
WorkerPool pool;
Compiler compiler;
Indexer indexer;
ServerLifecycle lifecycle = ServerLifecycle::Uninitialized;
std::string self_path;
std::string workspace_root;
std::string session_log_dir;
std::string init_options_json;
};
struct ServerOptions {
std::string mode;
std::string host = "127.0.0.1";
int port = 0;
std::string self_path;
std::string record;
};
int run_server_mode(const ServerOptions& opts);
struct DaemonOptions {
std::string socket_path;
std::string workspace;
std::string self_path;
};
int run_daemon_mode(const DaemonOptions& opts);
} // namespace clice

View File

@@ -5,7 +5,7 @@
#include <optional>
#include <string>
#include "server/workspace.h"
#include "server/workspace/workspace.h"
#include "kota/async/async.h"
#include "llvm/ADT/SmallVector.h"

View File

@@ -1,4 +1,4 @@
#include "server/stateful_worker.h"
#include "server/worker/stateful_worker.h"
#include <atomic>
#include <cstdint>
@@ -10,8 +10,8 @@
#include "compile/compilation.h"
#include "feature/feature.h"
#include "index/tu_index.h"
#include "server/protocol.h"
#include "server/worker_common.h"
#include "server/protocol/worker.h"
#include "server/worker/worker_common.h"
#include "support/logging.h"
#include "kota/async/async.h"
@@ -152,6 +152,7 @@ void StatefulWorker::register_handlers() {
CompilationParams cp;
cp.kind = CompilationKind::Content;
cp.clang_tidy = params.clang_tidy;
fill_args(cp, doc->directory, doc->arguments);
if(!doc->pch.first.empty()) {
cp.pch = doc->pch;
@@ -245,26 +246,33 @@ void StatefulWorker::register_handlers() {
co_return kota::codec::RawValue{"[]"};
case K::SemanticTokens:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::semantic_tokens(doc.unit));
return to_raw(
feature::semantic_tokens(doc.unit, feature::PositionEncoding::UTF16));
});
case K::InlayHints:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
auto range = params.range;
if(range.begin == static_cast<uint32_t>(-1))
range = LocalSourceRange{0, static_cast<uint32_t>(doc.text.size())};
return to_raw(feature::inlay_hints(doc.unit, range));
return to_raw(feature::inlay_hints(doc.unit,
range,
{},
feature::PositionEncoding::UTF16));
});
case K::FoldingRange:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::folding_ranges(doc.unit));
return to_raw(
feature::folding_ranges(doc.unit, feature::PositionEncoding::UTF16));
});
case K::DocumentSymbol:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::document_symbols(doc.unit));
return to_raw(
feature::document_symbols(doc.unit, feature::PositionEncoding::UTF16));
});
case K::DocumentLink:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::document_links(doc.unit));
return to_raw(
feature::document_links(doc.unit, feature::PositionEncoding::UTF16));
});
case K::CodeAction:
// TODO: Implement code actions

View File

@@ -1,10 +1,10 @@
#include "server/stateless_worker.h"
#include "server/worker/stateless_worker.h"
#include "compile/compilation.h"
#include "feature/feature.h"
#include "index/tu_index.h"
#include "server/protocol.h"
#include "server/worker_common.h"
#include "server/protocol/worker.h"
#include "server/worker/worker_common.h"
#include "support/logging.h"
#include "kota/async/async.h"
@@ -15,6 +15,22 @@
namespace clice {
/// RAII guard that lowers the current process's scheduling priority and
/// restores it on destruction.
struct ScopedNice {
int saved;
explicit ScopedNice(int increment = 10) {
auto p = kota::sys::priority();
saved = p ? *p : 0;
kota::sys::set_priority(saved + increment);
}
~ScopedNice() {
kota::sys::set_priority(saved);
}
};
using kota::ipc::RequestResult;
using RequestContext = kota::ipc::BincodePeer::RequestContext;
@@ -258,6 +274,22 @@ static worker::BuildResult handle_signature_help(const worker::BuildParams& para
return result;
}
static worker::BuildResult handle_format(const worker::BuildParams& params) {
ScopedTimer timer;
std::optional<LocalSourceRange> range;
if(params.format_range.valid()) {
range = params.format_range;
}
auto edits = feature::document_format(params.file, params.text, range);
LOG_DEBUG("Format done: {} edits, {}ms", edits.size(), timer.ms());
worker::BuildResult result;
result.result_json = to_raw(edits);
return result;
}
int run_stateless_worker_mode(const std::string& worker_name, const std::string& log_dir) {
logging::stderr_logger(worker_name, logging::options);
if(!log_dir.empty()) {
@@ -283,9 +315,13 @@ int run_stateless_worker_mode(const std::string& worker_name, const std::string&
switch(params.kind) {
case K::BuildPCH: return handle_build_pch(params);
case K::BuildPCM: return handle_build_pcm(params);
case K::Index: return handle_index(params);
case K::Index: {
ScopedNice guard;
return handle_index(params);
}
case K::Completion: return handle_completion(params);
case K::SignatureHelp: return handle_signature_help(params);
case K::Format: return handle_format(params);
}
return {false, "Unknown build kind"};
});

View File

@@ -8,8 +8,7 @@
#include "compile/compilation.h"
#include "kota/codec/json/serializer.h"
#include "kota/codec/raw_value.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
namespace clice {

View File

@@ -1,4 +1,4 @@
#include "server/worker_pool.h"
#include "server/worker/worker_pool.h"
#include <csignal>
#include <string>
@@ -13,14 +13,13 @@ namespace {
/// Coroutine that drains a worker's stderr pipe.
/// Workers write their own log files, so this only captures unexpected output
/// (crash stacktraces, assertion failures, etc.) that bypasses spdlog.
/// (crash stacktraces, assertion failures, sanitizer reports, etc.).
kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
std::string buffer;
while(true) {
auto result = co_await stderr_pipe.read();
if(!result.has_value()) {
if(!result.has_value())
break;
}
auto& chunk = result.value();
if(chunk.empty())
break;
@@ -34,7 +33,7 @@ kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
break;
auto line = buffer.substr(pos, nl - pos);
if(!line.empty()) {
LOG_DEBUG("{} {}", prefix, line);
LOG_WARN("{} {}", prefix, line);
}
pos = nl + 1;
}
@@ -42,7 +41,7 @@ kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
}
if(!buffer.empty()) {
LOG_DEBUG("{} {}", prefix, buffer);
LOG_WARN("{} {}", prefix, buffer);
}
}
@@ -97,9 +96,8 @@ bool WorkerPool::spawn_worker(const std::string& self_path,
std::move(spawn.stdin_pipe));
auto peer = std::make_unique<kota::ipc::BincodePeer>(loop, std::move(transport));
// Schedule stderr log collection
std::string prefix = "[" + worker_name + "]";
loop.schedule(drain_stderr(std::move(spawn.stderr_pipe), prefix));
io_group.spawn(drain_stderr(std::move(spawn.stderr_pipe), prefix));
workers.push_back(WorkerProcess{
.proc = std::move(spawn.proc),
@@ -108,24 +106,28 @@ bool WorkerPool::spawn_worker(const std::string& self_path,
});
auto& w = workers.back();
loop.schedule(w.peer->run());
w.alive = true;
io_group.spawn(w.peer->run());
return true;
}
bool WorkerPool::start(const WorkerPoolOptions& options) {
options_ = options;
log_dir_ = options.log_dir;
for(std::uint32_t i = 0; i < options.stateless_count; ++i) {
if(!spawn_worker(options.self_path, false, 0)) {
return false;
}
monitor_group.spawn(monitor_worker(stateless_workers.size() - 1, false));
}
for(std::uint32_t i = 0; i < options.stateful_count; ++i) {
if(!spawn_worker(options.self_path, true, options.worker_memory_limit)) {
return false;
}
monitor_group.spawn(monitor_worker(stateful_workers.size() - 1, true));
}
// Register evicted notification handler for each stateful worker
@@ -145,30 +147,19 @@ bool WorkerPool::start(const WorkerPoolOptions& options) {
kota::task<> WorkerPool::stop() {
LOG_INFO("WorkerPool stopping...");
shutting_down_ = true;
// Close output pipes to signal workers to exit gracefully
for(auto& w: stateless_workers) {
for(auto& w: stateless_workers)
w.peer->close_output();
}
for(auto& w: stateful_workers) {
for(auto& w: stateful_workers)
w.peer->close_output();
}
// Send SIGTERM to all workers
for(auto& w: stateless_workers) {
for(auto& w: stateless_workers)
w.proc.kill(SIGTERM);
}
for(auto& w: stateful_workers) {
for(auto& w: stateful_workers)
w.proc.kill(SIGTERM);
}
// Wait for all worker processes to exit
for(auto& w: stateless_workers) {
co_await w.proc.wait();
}
for(auto& w: stateful_workers) {
co_await w.proc.wait();
}
co_await kota::when_all(monitor_group.join(), io_group.join());
LOG_INFO("WorkerPool stopped");
}
@@ -198,7 +189,10 @@ std::size_t WorkerPool::assign_worker(std::uint32_t path_id) {
std::size_t WorkerPool::pick_least_loaded() {
std::size_t best = 0;
for(std::size_t i = 1; i < stateful_workers.size(); ++i) {
if(stateful_workers[i].owned_documents < stateful_workers[best].owned_documents) {
if(!stateful_workers[i].alive)
continue;
if(!stateful_workers[best].alive ||
stateful_workers[i].owned_documents < stateful_workers[best].owned_documents) {
best = i;
}
}
@@ -233,4 +227,122 @@ void WorkerPool::clear_owner(std::size_t worker_index) {
}
}
kota::task<> WorkerPool::monitor_worker(std::size_t index, bool stateful) {
auto& workers = stateful ? stateful_workers : stateless_workers;
auto& w = workers[index];
auto name = std::string(stateful ? "SF-" : "SL-") + std::to_string(index);
auto result = co_await w.proc.wait();
w.alive = false;
if(shutting_down_)
co_return;
if(result.has_value()) {
auto& exit = result.value();
if(exit.term_signal != 0) {
LOG_ERROR("Worker {} killed by signal {} (restarts: {})",
name,
exit.term_signal,
w.restart_count);
} else {
LOG_ERROR("Worker {} exited with code {} (restarts: {})",
name,
exit.status,
w.restart_count);
}
} else {
LOG_ERROR("Worker {} lost: {} (restarts: {})",
name,
result.error().message(),
w.restart_count);
}
if(stateful)
clear_owner(index);
constexpr unsigned max_restarts = 5;
if(w.restart_count >= max_restarts) {
LOG_ERROR("Worker {} exceeded max restarts ({}), giving up", name, max_restarts);
co_return;
}
if(!respawn_worker(index, stateful)) {
LOG_ERROR("Worker {} respawn failed", name);
}
}
bool WorkerPool::respawn_worker(std::size_t index, bool stateful) {
auto& workers = stateful ? stateful_workers : stateless_workers;
auto old_restart_count = workers[index].restart_count + 1;
auto worker_name = std::string(stateful ? "SF-" : "SL-") + std::to_string(index);
// Close the old peer and retire it so its coroutines (run/write_loop)
// can finish naturally before the object is destroyed.
if(workers[index].peer) {
workers[index].peer->close();
retired_peers.push_back(std::move(workers[index].peer));
}
kota::process::options opts;
opts.file = options_.self_path;
if(stateful) {
opts.args = {options_.self_path,
"--mode",
"stateful-worker",
"--worker-memory-limit",
std::to_string(options_.worker_memory_limit)};
} else {
opts.args = {options_.self_path, "--mode", "stateless-worker"};
}
opts.args.push_back("--worker-name");
opts.args.push_back(worker_name);
if(!log_dir_.empty()) {
opts.args.push_back("--log-dir");
opts.args.push_back(log_dir_);
}
opts.streams = {
kota::process::stdio::pipe(true, false),
kota::process::stdio::pipe(false, true),
kota::process::stdio::pipe(false, true),
};
auto result = kota::process::spawn(opts, loop);
if(!result) {
LOG_ERROR("Failed to respawn worker {}: {}", worker_name, result.error().message());
return false;
}
auto& spawn = *result;
auto transport = std::make_unique<kota::ipc::StreamTransport>(std::move(spawn.stdout_pipe),
std::move(spawn.stdin_pipe));
auto peer = std::make_unique<kota::ipc::BincodePeer>(loop, std::move(transport));
std::string prefix = "[" + worker_name + "]";
io_group.spawn(drain_stderr(std::move(spawn.stderr_pipe), prefix));
workers[index] = WorkerProcess{
.proc = std::move(spawn.proc),
.peer = std::move(peer),
.owned_documents = 0,
.alive = true,
.restart_count = old_restart_count,
};
auto& w = workers[index];
io_group.spawn(w.peer->run());
if(stateful) {
w.peer->on_notification([this](const worker::EvictedParams& params) {
if(on_evicted)
on_evicted(params.path);
});
}
monitor_group.spawn(monitor_worker(index, stateful));
LOG_INFO("Worker {} restarted (attempt {})", worker_name, old_restart_count);
return true;
}
} // namespace clice

View File

@@ -6,7 +6,7 @@
#include <list>
#include <memory>
#include "server/protocol.h"
#include "server/protocol/worker.h"
#include "kota/async/async.h"
#include "kota/ipc/codec/bincode.h"
@@ -64,6 +64,8 @@ private:
kota::process proc;
std::unique_ptr<kota::ipc::BincodePeer> peer;
std::size_t owned_documents = 0;
bool alive = true;
unsigned restart_count = 0;
};
kota::event_loop& loop;
@@ -80,8 +82,19 @@ private:
void clear_owner(std::size_t worker_index);
std::size_t pick_least_loaded();
bool shutting_down_ = false;
kota::task_group<> monitor_group{loop};
kota::task_group<> io_group{loop};
WorkerPoolOptions options_;
std::string log_dir_;
/// Peers moved here during respawn so their coroutines can finish
/// before the object is destroyed.
llvm::SmallVector<std::unique_ptr<kota::ipc::BincodePeer>> retired_peers;
bool spawn_worker(const std::string& self_path, bool stateful, std::uint64_t memory_limit);
bool respawn_worker(std::size_t index, bool stateful);
kota::task<> monitor_worker(std::size_t index, bool stateful);
};
template <typename Params>
@@ -91,11 +104,10 @@ RequestResult<Params> WorkerPool::send_stateful(std::uint32_t path_id,
if(stateful_workers.empty()) {
co_return kota::outcome_error(kota::ipc::Error{"No stateful workers available"});
}
// No timeout: compile tasks run as detached tasks (loop.schedule) that
// are immune to LSP $/cancelRequest. Adding a timeout here would use
// kotatsu's with_token/when_any which has a spurious-cancellation bug
// that kills requests within milliseconds instead of the configured period.
auto idx = assign_worker(path_id);
if(!stateful_workers[idx].alive) {
co_return kota::outcome_error(kota::ipc::Error{"Assigned stateful worker is down"});
}
co_return co_await stateful_workers[idx].peer->send_request(params, opts);
}
@@ -105,9 +117,16 @@ RequestResult<Params> WorkerPool::send_stateless(const Params& params,
if(stateless_workers.empty()) {
co_return kota::outcome_error(kota::ipc::Error{"No stateless workers available"});
}
auto idx = next_stateless;
next_stateless = (next_stateless + 1) % stateless_workers.size();
co_return co_await stateless_workers[idx].peer->send_request(params, opts);
// Round-robin, skipping dead workers.
auto start = next_stateless;
for(std::size_t i = 0; i < stateless_workers.size(); ++i) {
auto idx = (start + i) % stateless_workers.size();
if(stateless_workers[idx].alive) {
next_stateless = (idx + 1) % stateless_workers.size();
co_return co_await stateless_workers[idx].peer->send_request(params, opts);
}
}
co_return kota::outcome_error(kota::ipc::Error{"All stateless workers are down"});
}
template <typename Params>
@@ -115,6 +134,8 @@ void WorkerPool::notify_stateful(std::uint32_t path_id, const Params& params) {
auto it = owner.find(path_id);
if(it == owner.end())
return;
if(!stateful_workers[it->second].alive)
return;
stateful_workers[it->second].peer->send_notification(params);
}

View File

@@ -0,0 +1,195 @@
#include "server/workspace/config.h"
#include <algorithm>
#include "support/filesystem.h"
#include "support/glob_pattern.h"
#include "support/logging.h"
#include "kota/async/io/system.h"
#include "kota/codec/json/json.h"
#include "kota/codec/toml/toml.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/Process.h"
#include "llvm/Support/xxhash.h"
namespace clice {
/// Replace all occurrences of ${workspace} with the workspace root.
/// No-op when workspace_root is empty, to avoid producing paths like "/cache"
/// from "${workspace}/cache".
static void substitute_workspace(std::string& value, llvm::StringRef workspace_root) {
if(workspace_root.empty())
return;
constexpr std::string_view placeholder = "${workspace}";
std::size_t pos = 0;
while((pos = value.find(placeholder, pos)) != std::string::npos) {
value.replace(pos, placeholder.size(), workspace_root);
pos += workspace_root.size();
}
}
/// Try to resolve the default cache directory using XDG_CACHE_HOME.
/// Returns empty string on failure.
static std::string resolve_xdg_cache_dir(llvm::StringRef workspace_root) {
// Determine base: $XDG_CACHE_HOME or ~/.cache
std::string base;
if(auto xdg = llvm::sys::Process::GetEnv("XDG_CACHE_HOME"); xdg && !xdg->empty()) {
base = std::move(*xdg);
} else if(auto home = llvm::sys::Process::GetEnv("HOME"); home && !home->empty()) {
base = path::join(*home, ".cache");
} else {
return {};
}
// Use a hash of workspace_root to create a unique subdirectory.
auto hash = llvm::xxh3_64bits(workspace_root);
auto dir = path::join(base, "clice", std::format("{:016x}", hash));
if(auto ec = llvm::sys::fs::create_directories(dir)) {
LOG_WARN("Failed to create XDG cache directory {}: {}", dir, ec.message());
return {};
}
return dir;
}
void Config::apply_defaults(llvm::StringRef workspace_root) {
auto& p = project;
if(p.max_active_file == 0)
p.max_active_file = 8;
if(!p.enable_indexing)
p.enable_indexing = true;
if(!p.idle_timeout_ms)
p.idle_timeout_ms = 3000;
if(p.stateful_worker_count == 0)
p.stateful_worker_count = 2;
if(p.stateless_worker_count == 0) {
auto cores = kota::sys::parallelism();
p.stateless_worker_count = std::max(cores / 2, 2u);
}
if(p.worker_memory_limit == 0)
p.worker_memory_limit = 4ULL * 1024 * 1024 * 1024; // 4GB
if(p.cache_dir.empty() && !workspace_root.empty()) {
p.cache_dir = resolve_xdg_cache_dir(workspace_root);
if(p.cache_dir.empty())
p.cache_dir = path::join(workspace_root, ".clice");
}
if(p.index_dir.empty() && !p.cache_dir.empty())
p.index_dir = path::join(p.cache_dir, "index");
if(p.logging_dir.empty() && !p.cache_dir.empty())
p.logging_dir = path::join(p.cache_dir, "logs");
// Variable substitution on string fields.
substitute_workspace(p.cache_dir, workspace_root);
substitute_workspace(p.index_dir, workspace_root);
substitute_workspace(p.logging_dir, workspace_root);
for(auto& entry: p.compile_commands_paths)
substitute_workspace(entry, workspace_root);
// Pre-compile glob patterns from rules.
compiled_rules.clear();
for(auto& rule: rules) {
CompiledRule compiled;
for(auto& pattern_str: rule.patterns) {
auto pat = GlobPattern::create(pattern_str);
if(!pat) {
LOG_WARN("Invalid glob pattern in rule: {}", pattern_str);
continue;
}
compiled.patterns.push_back(std::move(*pat));
}
// Drop the whole rule if no pattern compiled successfully — otherwise the
// append/remove flags would be silently attached to a rule that can never match.
if(compiled.patterns.empty()) {
if(!rule.patterns.empty())
LOG_WARN("Rule dropped: all glob patterns failed to compile");
continue;
}
compiled.append.assign(rule.append.begin(), rule.append.end());
compiled.remove.assign(rule.remove.begin(), rule.remove.end());
compiled_rules.push_back(std::move(compiled));
}
}
void Config::match_rules(llvm::StringRef file_path,
std::vector<std::string>& append,
std::vector<std::string>& remove) const {
// Rules are processed in declaration order so that a later rule can
// override an earlier one. Specifically, when a later rule removes
// an argument, we also strip any string-equal entry already added
// to `append` by an earlier matching rule — otherwise the append
// would silently survive (lookup applies removes to the base flags
// only, not to entries contributed via `append`).
for(auto& rule: compiled_rules) {
bool matched =
std::ranges::any_of(rule.patterns, [&](auto& pat) { return pat.match(file_path); });
if(!matched)
continue;
for(auto& r: rule.remove) {
std::erase(append, r);
remove.push_back(r);
}
append.insert(append.end(), rule.append.begin(), rule.append.end());
}
}
std::optional<Config> Config::load(llvm::StringRef path, llvm::StringRef workspace_root) {
auto content = fs::read(path);
if(!content)
return std::nullopt;
auto result = kota::codec::toml::parse<Config>(*content);
if(!result) {
LOG_ERROR("Invalid clice.toml {}: {}", path, result.error().to_string());
return std::nullopt;
}
auto config = std::move(*result);
config.apply_defaults(workspace_root);
LOG_INFO("Loaded config from {}", path);
return config;
}
std::optional<Config> Config::load_from_json(llvm::StringRef json, llvm::StringRef workspace_root) {
Config config{};
auto result = kota::codec::json::from_json(json, config);
if(!result) {
LOG_WARN("Failed to parse initializationOptions JSON: {}", result.error().message);
return std::nullopt;
}
config.apply_defaults(workspace_root);
LOG_INFO("Loaded config from initializationOptions");
return config;
}
Config Config::load_from_workspace(llvm::StringRef workspace_root) {
if(!workspace_root.empty()) {
for(auto* name: {"clice.toml", ".clice/config.toml"}) {
auto config_path = path::join(workspace_root, name);
if(!llvm::sys::fs::exists(config_path))
continue;
if(auto config = load(config_path, workspace_root))
return std::move(*config);
// Present but malformed: fall through to defaults, but surface
// the situation clearly so users know their config wasn't applied.
LOG_WARN("Falling back to default configuration because {} is invalid", config_path);
}
}
Config config;
config.apply_defaults(workspace_root);
LOG_INFO(
"No clice.toml found, using default configuration " "(stateful={}, stateless={}, memory_limit={}MB)",
config.project.stateful_worker_count.value,
config.project.stateless_worker_count.value,
config.project.worker_memory_limit.value / (1024 * 1024));
return config;
}
} // namespace clice

View File

@@ -0,0 +1,79 @@
#pragma once
#include <cstdint>
#include <optional>
#include <string>
#include <vector>
#include "support/glob_pattern.h"
#include "kota/meta/annotation.h"
#include "llvm/ADT/StringRef.h"
namespace clice {
using kota::meta::defaulted;
/// A file-pattern rule that appends/removes compilation flags.
/// Corresponds to `[[rules]]` in clice.toml.
struct ConfigRule {
defaulted<std::vector<std::string>> patterns;
defaulted<std::vector<std::string>> append;
defaulted<std::vector<std::string>> remove;
};
/// Corresponds to the `[project]` section in clice.toml.
struct ProjectConfig {
defaulted<bool> clang_tidy = {};
defaulted<int> max_active_file = {};
defaulted<std::string> cache_dir;
defaulted<std::string> index_dir;
defaulted<std::string> logging_dir;
defaulted<std::vector<std::string>> compile_commands_paths;
std::optional<bool> enable_indexing;
std::optional<int> idle_timeout_ms;
defaulted<std::uint32_t> stateful_worker_count = {};
defaulted<std::uint32_t> stateless_worker_count = {};
defaulted<std::uint64_t> worker_memory_limit = {};
};
struct CompiledRule {
std::vector<GlobPattern> patterns;
std::vector<std::string> append;
std::vector<std::string> remove;
};
/// Configuration for the clice LSP server, loadable from clice.toml
/// or passed via LSP initializationOptions.
struct Config {
defaulted<ProjectConfig> project;
defaulted<std::vector<ConfigRule>> rules;
kota::meta::annotation<std::vector<CompiledRule>, kota::meta::attrs::skip> compiled_rules;
/// Compute default values for any field left at its zero/empty sentinel.
void apply_defaults(llvm::StringRef workspace_root);
/// Collect append/remove flags from all rules whose patterns match `path`.
void match_rules(llvm::StringRef path,
std::vector<std::string>& append,
std::vector<std::string>& remove) const;
/// Try to load configuration from a TOML file.
static std::optional<Config> load(llvm::StringRef path, llvm::StringRef workspace_root);
/// Try to load configuration from a JSON string (e.g. initializationOptions).
static std::optional<Config> load_from_json(llvm::StringRef json,
llvm::StringRef workspace_root);
/// Load config from the workspace, trying standard locations.
/// Returns a default config (with apply_defaults) if no file is found.
static Config load_from_workspace(llvm::StringRef workspace_root);
};
} // namespace clice

View File

@@ -1,4 +1,4 @@
#include "server/workspace.h"
#include "server/workspace/workspace.h"
#include <algorithm>
#include <chrono>
@@ -183,10 +183,10 @@ struct CacheData {
} // namespace
void Workspace::load_cache() {
if(config.cache_dir.empty())
if(config.project.cache_dir.empty())
return;
auto cache_path = path::join(config.cache_dir, "cache", "cache.json");
auto cache_path = path::join(config.project.cache_dir, "cache", "cache.json");
auto content = fs::read(cache_path);
if(!content) {
LOG_DEBUG("No cache.json found at {}", cache_path);
@@ -218,7 +218,7 @@ void Workspace::load_cache() {
};
for(auto& entry: data.pch) {
auto pch_path = path::join(config.cache_dir, "cache", "pch", entry.filename);
auto pch_path = path::join(config.project.cache_dir, "cache", "pch", entry.filename);
auto source = resolve(entry.source_file);
if(!llvm::sys::fs::exists(pch_path) || source.empty())
continue;
@@ -234,7 +234,7 @@ void Workspace::load_cache() {
}
for(auto& entry: data.pcm) {
auto pcm_path = path::join(config.cache_dir, "cache", "pcm", entry.filename);
auto pcm_path = path::join(config.project.cache_dir, "cache", "pcm", entry.filename);
auto source = resolve(entry.source_file);
if(!llvm::sys::fs::exists(pcm_path) || source.empty())
continue;
@@ -252,7 +252,7 @@ void Workspace::load_cache() {
}
void Workspace::save_cache() {
if(config.cache_dir.empty())
if(config.project.cache_dir.empty())
return;
CacheData data;
@@ -306,7 +306,7 @@ void Workspace::save_cache() {
return;
}
auto cache_path = path::join(config.cache_dir, "cache", "cache.json");
auto cache_path = path::join(config.project.cache_dir, "cache", "cache.json");
auto tmp_path = cache_path + ".tmp";
auto write_result = fs::write(tmp_path, *json_str);
if(!write_result) {
@@ -321,14 +321,14 @@ void Workspace::save_cache() {
}
void Workspace::cleanup_cache(int max_age_days) {
if(config.cache_dir.empty())
if(config.project.cache_dir.empty())
return;
auto now = std::chrono::system_clock::now();
auto max_age = std::chrono::hours(max_age_days * 24);
for(auto* subdir: {"cache/pch", "cache/pcm"}) {
auto dir = path::join(config.cache_dir, subdir);
auto dir = path::join(config.project.cache_dir, subdir);
std::error_code ec;
for(auto it = llvm::sys::fs::directory_iterator(dir, ec);
!ec && it != llvm::sys::fs::directory_iterator();

View File

@@ -11,8 +11,8 @@
#include "index/merged_index.h"
#include "index/project_index.h"
#include "semantic/relation_kind.h"
#include "server/compile_graph.h"
#include "server/config.h"
#include "server/compiler/compile_graph.h"
#include "server/workspace/config.h"
#include "support/path_pool.h"
#include "syntax/dependency_graph.h"
@@ -170,7 +170,7 @@ struct PCMState {
/// - didSave (on_file_saved: rescan disk, cascade invalidation)
/// - Background index (merge TUIndex results from stateless workers)
struct Workspace {
CliceConfig config;
Config config;
CompilationDatabase cdb;
PathPool path_pool;

View File

@@ -37,6 +37,14 @@ inline std::string real_path(llvm::StringRef file) {
return path.str().str();
}
inline std::string default_socket_path() {
llvm::SmallString<128> home;
if(!llvm::sys::path::home_directory(home))
return "/tmp/clice.sock";
llvm::sys::path::append(home, ".clice", "clice.sock");
return home.str().str();
}
} // namespace path
namespace fs {

View File

@@ -289,7 +289,7 @@ std::expected<GlobPattern::SubGlobPattern, std::string>
return pat;
}
bool GlobPattern::match(llvm::StringRef str) {
bool GlobPattern::match(llvm::StringRef str) const {
if(!str.consume_front(prefix)) {
return false;
}

View File

@@ -54,7 +54,7 @@ public:
}
/// \returns \p true if \p str matches this glob pattern
bool match(llvm::StringRef s);
bool match(llvm::StringRef s) const;
private:
/// GlobPattern is seperated into `Prefix + SubGlobPattern`

View File

@@ -256,7 +256,8 @@ kota::task<> scan_impl(CompilationDatabase& cdb,
DependencyGraph& graph,
ScanReport& report,
ScanCache* ext_cache,
kota::event_loop& loop) {
kota::event_loop& loop,
const RuleMatcher& rule_matcher) {
auto start_time = std::chrono::steady_clock::now();
// Reuse context groups and configs from cache when available (warm runs).
@@ -355,9 +356,19 @@ kota::task<> scan_impl(CompilationDatabase& cdb,
std::uint32_t config_id = next_config_id++;
context_to_config_id[context] = config_id;
auto representative_path = path_pool.resolve(file_ids[0]);
// Apply per-file rules so that `[[rules]]`-modified -I/-isystem/-std
// flags are reflected in the search config used by the scan.
// Rules are applied to the representative file and assumed to hold
// for the whole context group (same CompilationInfo).
std::vector<std::string> rule_append, rule_remove;
if(rule_matcher)
rule_matcher(representative_path, rule_append, rule_remove);
auto t0 = std::chrono::steady_clock::now();
configs[config_id] =
cdb.lookup_search_config(representative_path, {.query_toolchain = true});
configs[config_id] = cdb.lookup_search_config(
representative_path,
{.query_toolchain = true, .remove = rule_remove, .append = rule_append});
auto t1 = std::chrono::steady_clock::now();
lookup_us += std::chrono::duration_cast<std::chrono::microseconds>(t1 - t0).count();
}
@@ -819,14 +830,15 @@ kota::task<> scan_impl(CompilationDatabase& cdb,
ScanReport scan_dependency_graph(CompilationDatabase& cdb,
PathPool& path_pool,
DependencyGraph& graph,
ScanCache* cache) {
ScanCache* cache,
const RuleMatcher& rule_matcher) {
ScanReport report;
if(cdb.get_entries().empty()) {
return report;
}
kota::event_loop loop;
loop.schedule(scan_impl(cdb, path_pool, graph, report, cache, loop));
loop.schedule(scan_impl(cdb, path_pool, graph, report, cache, loop, rule_matcher));
loop.run();
return report;
}

View File

@@ -1,6 +1,7 @@
#pragma once
#include <cstdint>
#include <functional>
#include <string>
#include <vector>
@@ -253,6 +254,12 @@ struct ScanCache {
std::vector<WaveEntry> initial_wave;
};
/// Callback for per-file rule-based flag modification. Given a file path,
/// populates `append`/`remove` with rule-configured arguments so they can be
/// layered on top of the CDB command when extracting the search config.
using RuleMatcher = std::function<
void(llvm::StringRef path, std::vector<std::string>& append, std::vector<std::string>& remove)>;
/// Run the wavefront BFS scan over all files in the compilation database.
/// Internally creates a local event loop for async I/O (file reads via worker
/// thread pool, stat calls via libuv). Blocks until the scan is complete.
@@ -261,9 +268,14 @@ struct ScanCache {
/// avoids repeated readdir() and include-resolution work across
/// successive calls. PathPool must NOT be reset between calls
/// when a persistent cache is used (path_id values must remain stable).
/// @param rule_matcher Optional callback applied per context group so that
/// `[[rules]]`-modified include/std flags are reflected in the
/// dependency graph (otherwise rule-affected files would have
/// stale resolution).
ScanReport scan_dependency_graph(CompilationDatabase& cdb,
PathPool& path_pool,
DependencyGraph& graph,
ScanCache* cache = nullptr);
ScanCache* cache = nullptr,
const RuleMatcher& rule_matcher = {});
} // namespace clice

View File

@@ -1,6 +1,7 @@
import asyncio
import json
import shutil
import socket
import subprocess
import sys
from pathlib import Path
@@ -93,29 +94,67 @@ def workspace(request: pytest.FixtureRequest, test_data_dir: Path) -> Path | Non
@pytest.fixture
async def client(
request: pytest.FixtureRequest, executable: Path, workspace: Path | None
request: pytest.FixtureRequest,
executable: Path,
workspace: Path | None,
):
"""Spawn clice server, auto-initialize if @pytest.mark.workspace is present."""
config = request.config
mode = config.getoption("--mode")
host = config.getoption("--host")
cmd = [str(executable), "--mode", mode]
if mode == "socket":
host = config.getoption("--host")
port = config.getoption("--port")
cmd += ["--host", host, "--port", str(port)]
cmd = [str(executable), "--mode", mode, "--host", host]
c = CliceClient()
await c.start_io(*cmd)
if workspace is not None:
await c.initialize(workspace)
init_options_marker = request.node.get_closest_marker("init_options")
init_options = dict(init_options_marker.args[0]) if init_options_marker else {}
# Force cache_dir into the workspace so .clice/ cleanup prevents stale PCH.
project = dict(init_options.get("project", {}))
project.setdefault("cache_dir", str(workspace / ".clice"))
init_options["project"] = project
await c.initialize(workspace, initialization_options=init_options)
yield c
await _shutdown_client(c)
def _find_free_port() -> int:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(("127.0.0.1", 0))
return s.getsockname()[1]
@pytest.fixture
async def agentic(
request: pytest.FixtureRequest,
executable: Path,
workspace: Path | None,
):
"""Start a server with agentic TCP port, yield (executable, host, port)."""
host = "127.0.0.1"
port = _find_free_port()
cmd = [str(executable), "--mode", "pipe", "--host", host, "--port", str(port)]
c = CliceClient()
await c.start_io(*cmd)
if workspace is not None:
init_options_marker = request.node.get_closest_marker("init_options")
init_options = dict(init_options_marker.args[0]) if init_options_marker else {}
project = dict(init_options.get("project", {}))
project.setdefault("cache_dir", str(workspace / ".clice"))
init_options["project"] = project
await c.initialize(workspace, initialization_options=init_options)
yield executable, host, port
await _shutdown_client(c)
def generate_cdb(workspace: Path) -> None:
"""Generate compile_commands.json using CMake with Ninja backend."""
cmake = shutil.which("cmake")
@@ -163,12 +202,17 @@ async def _shutdown_client(c: CliceClient) -> None:
try:
server = getattr(c, "_server", None)
if server and server.stderr:
stderr_data = await asyncio.wait_for(server.stderr.read(), timeout=2.0)
if stderr_data:
for line in stderr_data.decode("utf-8", errors="replace").splitlines():
if "[warn]" in line or "[error]" in line:
print(f"[server] {line}", flush=True)
if server:
if server.returncode is not None:
print(f"[server] exit code: {server.returncode}", flush=True)
if server.stderr:
stderr_data = await asyncio.wait_for(server.stderr.read(), timeout=2.0)
if stderr_data:
for line in stderr_data.decode(
"utf-8", errors="replace"
).splitlines():
if "[warn]" in line or "[error]" in line or "Sanitizer" in line:
print(f"[server] {line}", flush=True)
except Exception:
pass
@@ -239,6 +283,21 @@ def _generate_test_data_cdbs(data_dir: Path) -> None:
dl_dir, [_entry(dl_dir, dl_main, [f"-I{dl_dir.as_posix()}", "-std=c++23"])]
)
# config_rules_toml / config_rules_no_config — rules tests must start
# from a CDB that does NOT include the flag the rule will append, so the
# rule's effect is observable through diagnostics.
for name in ("config_rules_toml", "config_rules_no_config"):
cr_dir = data_dir / name
cr_main = cr_dir / "main.cpp"
if cr_main.exists():
_write(cr_dir, [_entry(cr_dir, cr_main)])
# formatting
fmt_dir = data_dir / "formatting"
fmt_main = fmt_dir / "main.cpp"
if fmt_main.exists():
_write(fmt_dir, [_entry(fmt_dir, fmt_main)])
# pch_test
pt_dir = data_dir / "pch_test"
if pt_dir.exists():

View File

@@ -0,0 +1,36 @@
// basic if and if-else
namespace basic_if {
int abs_val(int x) {
if(x < 0)
return -x;
return x;
}
const char* sign(int x) {
if(x > 0) {
return "positive";
} else if(x < 0) {
return "negative";
} else {
return "zero";
}
}
// dangling else: else binds to nearest if
int nested_if(int a, int b) {
if(a > 0)
if(b > 0)
return 1;
else
return 2;
return 0;
}
void test() {
[[maybe_unused]] int r1 = abs_val(-3);
[[maybe_unused]] auto r2 = sign(5);
[[maybe_unused]] int r3 = nested_if(1, -1);
}
} // namespace basic_if

View File

@@ -0,0 +1,7 @@
int value() {
return FROM_INIT;
}
int main() {
return value();
}

View File

@@ -0,0 +1,3 @@
[[rules]]
patterns = ["**/*.cpp"]
append = ["-DFROM_TOML"]

View File

@@ -0,0 +1,7 @@
int value() {
return FROM_TOML;
}
int main() {
return value();
}

View File

@@ -0,0 +1,3 @@
BasedOnStyle: LLVM
IndentWidth: 4
ColumnLimit: 80

View File

@@ -0,0 +1 @@
int add(int a, int b) { return a + b; }

View File

View File

@@ -0,0 +1,592 @@
"""Tests for the agentic protocol handlers."""
import asyncio
import json
import socket
import subprocess
from concurrent.futures import ThreadPoolExecutor
import pytest
from tests.integration.utils.wait import wait_for_index
class AgenticRpcClient:
"""Minimal JSON-RPC client that speaks Content-Length framing over TCP."""
def __init__(self, host: str, port: int):
self.sock = socket.create_connection((host, port), timeout=10)
self.request_id = 0
self.buffer = b""
def request(self, method: str, params: dict):
self.request_id += 1
body = json.dumps(
{
"jsonrpc": "2.0",
"id": self.request_id,
"method": method,
"params": params,
}
)
payload = f"Content-Length: {len(body)}\r\n\r\n{body}".encode("utf-8")
self.sock.sendall(payload)
return self._read_response()
def _read_response(self):
while b"\r\n\r\n" not in self.buffer:
data = self.sock.recv(4096)
if not data:
raise ConnectionError("connection closed")
self.buffer += data
header_end = self.buffer.index(b"\r\n\r\n")
headers = self.buffer[:header_end].decode("utf-8")
self.buffer = self.buffer[header_end + 4 :]
content_length = 0
for line in headers.split("\r\n"):
if line.lower().startswith("content-length:"):
content_length = int(line.split(":")[1].strip())
while len(self.buffer) < content_length:
data = self.sock.recv(4096)
if not data:
raise ConnectionError("connection closed")
self.buffer += data
body = self.buffer[:content_length].decode("utf-8")
self.buffer = self.buffer[content_length:]
return json.loads(body)
def close(self):
self.sock.close()
def run_agentic(executable, host, port, path, timeout=10):
result = subprocess.run(
[
str(executable),
"--mode",
"agentic",
"--host",
host,
"--port",
str(port),
"--path",
path,
],
capture_output=True,
text=True,
timeout=timeout,
)
return result
@pytest.mark.workspace("hello_world")
async def test_compile_command(agentic, workspace):
executable, host, port = agentic
main_cpp = (workspace / "main.cpp").as_posix()
result = run_agentic(executable, host, port, main_cpp)
assert result.returncode == 0, f"stderr: {result.stderr}"
data = json.loads(result.stdout)
assert data["file"] == main_cpp
assert data["directory"] == workspace.as_posix()
assert len(data["arguments"]) > 0
@pytest.mark.workspace("hello_world")
async def test_compile_command_fallback(agentic, workspace):
executable, host, port = agentic
result = run_agentic(executable, host, port, "/nonexistent/file.cpp")
assert result.returncode == 0, f"stderr: {result.stderr}"
data = json.loads(result.stdout)
assert data["file"] == "/nonexistent/file.cpp"
@pytest.mark.workspace("hello_world")
async def test_multiple_requests(agentic, workspace):
executable, host, port = agentic
main_cpp = (workspace / "main.cpp").as_posix()
for _ in range(3):
result = run_agentic(executable, host, port, main_cpp)
assert result.returncode == 0, f"stderr: {result.stderr}"
data = json.loads(result.stdout)
assert data["file"] == main_cpp
async def test_connection_refused(executable):
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(("127.0.0.1", 0))
free_port = s.getsockname()[1]
result = run_agentic(executable, "127.0.0.1", free_port, "/some/file.cpp")
assert result.returncode != 0
@pytest.mark.workspace("hello_world")
async def test_concurrent_connections(agentic, workspace):
executable, host, port = agentic
main_cpp = (workspace / "main.cpp").as_posix()
def do_request(_):
return run_agentic(executable, host, port, main_cpp)
with ThreadPoolExecutor(max_workers=4) as pool:
results = list(pool.map(do_request, range(4)))
for r in results:
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["file"] == main_cpp
@pytest.fixture
async def indexed_agentic(request, executable, workspace):
"""Start server with LSP+agentic, compile a file, wait for indexing."""
from tests.integration.utils.client import CliceClient
from tests.conftest import _shutdown_client, _find_free_port
host = "127.0.0.1"
port = _find_free_port()
cmd = [str(executable), "--mode", "pipe", "--host", host, "--port", str(port)]
c = CliceClient()
await c.start_io(*cmd)
init_options = {"project": {"cache_dir": str(workspace / ".clice")}}
await c.initialize(workspace, initialization_options=init_options)
uri, _ = await c.open_and_wait(workspace / "main.cpp")
assert await wait_for_index(c, uri, "add"), "Index not ready"
rpc = AgenticRpcClient(host, port)
for _ in range(30):
resp = rpc.request("agentic/symbolSearch", {"query": "add"})
if "result" in resp and resp["result"]["symbols"]:
break
await asyncio.sleep(1)
else:
pytest.fail("agentic/symbolSearch never returned indexed symbols")
yield rpc, workspace
rpc.close()
c.close(uri)
await _shutdown_client(c)
@pytest.mark.workspace("index_features")
async def test_rpc_compile_command(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/compileCommand", {"path": path})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["file"] == path
assert len(result["arguments"]) > 0
@pytest.mark.workspace("index_features")
async def test_rpc_project_files(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/projectFiles", {})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["total"] > 0
paths = [f["path"] for f in result["files"]]
assert any("main.cpp" in p for p in paths)
@pytest.mark.workspace("index_features")
async def test_rpc_project_files_filter(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/projectFiles", {"filter": "source"})
assert "result" in resp
for f in resp["result"]["files"]:
assert f["kind"] == "source"
@pytest.mark.workspace("index_features")
async def test_rpc_symbol_search(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/symbolSearch", {"query": "add"})
assert "result" in resp, f"unexpected response: {resp}"
symbols = resp["result"]["symbols"]
add_sym = next((s for s in symbols if s["name"] == "add"), None)
assert add_sym is not None, f"'add' not found in {[s['name'] for s in symbols]}"
assert add_sym["kind"] == "Function"
assert add_sym["line"] == 19
assert add_sym["symbolId"] != 0
assert "main.cpp" in add_sym["file"]
@pytest.mark.workspace("index_features")
async def test_rpc_symbol_search_kind(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request(
"agentic/symbolSearch", {"query": "Animal", "kindFilter": ["Struct"]}
)
assert "result" in resp
for s in resp["result"]["symbols"]:
assert s["kind"] == "Struct"
@pytest.mark.workspace("index_features")
async def test_rpc_symbol_search_max(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/symbolSearch", {"query": "", "maxResults": 3})
assert "result" in resp
assert len(resp["result"]["symbols"]) <= 3
@pytest.mark.workspace("index_features")
async def test_rpc_read_symbol(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/readSymbol", {"name": "add"})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["name"] == "add"
assert result["symbolId"] != 0
assert result["startLine"] == 19
assert result["endLine"] == 21
assert "int add(int a, int b)" in result["text"]
assert "return a + b;" in result["text"]
@pytest.mark.workspace("index_features")
async def test_rpc_read_symbol_by_id(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp1 = rpc.request("agentic/readSymbol", {"name": "add"})
assert "result" in resp1
sid = resp1["result"]["symbolId"]
resp2 = rpc.request("agentic/readSymbol", {"symbolId": sid})
assert "result" in resp2
assert resp2["result"]["name"] == "add"
assert resp2["result"]["symbolId"] == sid
@pytest.mark.workspace("index_features")
async def test_rpc_document_symbols(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/documentSymbols", {"path": path})
assert "result" in resp, f"unexpected response: {resp}"
symbols = resp["result"]["symbols"]
names = [s["name"] for s in symbols]
kinds = [s["kind"] for s in symbols]
assert "add" in names, f"expected 'add' in {names}"
assert "main" in names, f"expected 'main' in {names}"
assert "global_var" in names, f"expected 'global_var' in {names}"
assert "Parameter" not in kinds, (
f"Parameters should be filtered: {list(zip(names, kinds))}"
)
@pytest.mark.workspace("index_features")
async def test_rpc_definition(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/definition", {"name": "add"})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["name"] == "add"
assert result["definition"] is not None
defn = result["definition"]
assert "main.cpp" in defn["file"]
assert defn["startLine"] == 19
assert defn["endLine"] == 21
assert "int add(int a, int b)" in defn["text"]
assert "return a + b;" in defn["text"]
@pytest.mark.workspace("index_features")
async def test_rpc_definition_by_position(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/definition", {"path": path, "line": 19})
assert "result" in resp, f"unexpected response: {resp}"
assert resp["result"]["name"] == "add"
@pytest.mark.workspace("index_features")
async def test_rpc_references(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/references", {"name": "global_var"})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["name"] == "global_var"
assert result["total"] == 2
lines = sorted(r["line"] for r in result["references"])
assert lines == [34, 38]
contexts = [r["context"] for r in result["references"]]
assert any("global_var + 1" in c for c in contexts)
assert any("global_var * 2" in c for c in contexts)
@pytest.mark.workspace("index_features")
async def test_rpc_references_include_decl(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request(
"agentic/references", {"name": "global_var", "includeDeclaration": True}
)
assert "result" in resp
result = resp["result"]
assert result["total"] == 3
lines = sorted(r["line"] for r in result["references"])
assert 31 in lines, f"expected declaration line 31 in {lines}"
@pytest.mark.workspace("index_features")
async def test_rpc_call_graph_incoming(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/callGraph", {"name": "add", "direction": "callers"})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["root"]["name"] == "add"
assert result["root"]["line"] == 19
assert result["root"]["symbolId"] != 0
callers = result["callers"]
caller_names = [c["name"] for c in callers]
assert "compute" in caller_names, f"expected 'compute' in {caller_names}"
compute = next(c for c in callers if c["name"] == "compute")
assert compute["line"] == 24
assert compute["symbolId"] != 0
assert result["callees"] == []
@pytest.mark.workspace("index_features")
async def test_rpc_call_graph_outgoing(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/callGraph", {"name": "compute", "direction": "callees"})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["root"]["name"] == "compute"
callees = result["callees"]
callee_names = [c["name"] for c in callees]
assert "add" in callee_names, f"expected 'add' in {callee_names}"
add_entry = next(c for c in callees if c["name"] == "add")
assert add_entry["line"] == 19
assert result["callers"] == []
@pytest.mark.workspace("index_features")
async def test_rpc_type_hierarchy_supertypes(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request(
"agentic/typeHierarchy", {"name": "Dog", "direction": "supertypes"}
)
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["root"]["name"] == "Dog"
assert result["root"]["line"] == 9
supertypes = result["supertypes"]
supertype_names = [t["name"] for t in supertypes]
assert "Animal" in supertype_names, f"expected 'Animal' in {supertype_names}"
animal = next(t for t in supertypes if t["name"] == "Animal")
assert animal["line"] == 2
assert animal["symbolId"] != 0
@pytest.mark.workspace("index_features")
async def test_rpc_type_hierarchy_subtypes(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request(
"agentic/typeHierarchy", {"name": "Animal", "direction": "subtypes"}
)
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["root"]["name"] == "Animal"
assert result["root"]["line"] == 2
subtypes = result["subtypes"]
subtype_names = [t["name"] for t in subtypes]
assert "Dog" in subtype_names, f"expected 'Dog' in {subtype_names}"
assert "Cat" in subtype_names, f"expected 'Cat' in {subtype_names}"
dog = next(t for t in subtypes if t["name"] == "Dog")
assert dog["line"] == 9
cat = next(t for t in subtypes if t["name"] == "Cat")
assert cat["line"] == 14
@pytest.mark.workspace("index_features")
async def test_rpc_status(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/status", {})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert isinstance(result["idle"], bool)
assert result["total"] > 0
assert isinstance(result["pending"], int)
assert isinstance(result["indexed"], int)
@pytest.mark.workspace("hello_world")
async def test_rpc_shutdown(executable, workspace):
"""Shutdown notification should cause the server to exit."""
from tests.integration.utils.client import CliceClient
from tests.conftest import _shutdown_client, _find_free_port
host = "127.0.0.1"
port = _find_free_port()
cmd = [str(executable), "--mode", "pipe", "--host", host, "--port", str(port)]
c = CliceClient()
await c.start_io(*cmd)
init_options = {"project": {"cache_dir": str(workspace / ".clice")}}
await c.initialize(workspace, initialization_options=init_options)
rpc = AgenticRpcClient(host, port)
body = json.dumps({"jsonrpc": "2.0", "method": "agentic/shutdown", "params": {}})
rpc.sock.sendall(f"Content-Length: {len(body)}\r\n\r\n{body}".encode())
rpc.sock.settimeout(5)
try:
rpc.sock.recv(4096)
except (socket.timeout, OSError):
pass
rpc.sock.close()
import asyncio
for _ in range(20):
if c._server.returncode is not None:
break
await asyncio.sleep(0.5)
assert c._server.returncode is not None, "Server did not exit after shutdown"
@pytest.mark.workspace("index_features")
async def test_rpc_symbol_not_found(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/definition", {"name": "nonexistent_symbol_xyz"})
assert "error" in resp
@pytest.mark.workspace("index_features")
async def test_rpc_symbol_id_roundtrip(indexed_agentic, workspace):
"""Search -> get symbolId -> definition -> verify consistency."""
rpc, _ = indexed_agentic
search = rpc.request("agentic/symbolSearch", {"query": "compute"})
assert "result" in search
symbols = search["result"]["symbols"]
compute = next((s for s in symbols if s["name"] == "compute"), None)
assert compute is not None, f"'compute' not found in {[s['name'] for s in symbols]}"
defn = rpc.request("agentic/definition", {"symbolId": compute["symbolId"]})
assert "result" in defn
assert defn["result"]["name"] == "compute"
assert defn["result"]["symbolId"] == compute["symbolId"]
@pytest.mark.workspace("index_features")
async def test_rpc_file_deps(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/fileDeps", {"path": path})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["file"] == path
assert isinstance(result["includes"], list)
assert isinstance(result["includers"], list)
@pytest.mark.workspace("index_features")
async def test_rpc_file_deps_direction(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/fileDeps", {"path": path, "direction": "includes"})
assert "result" in resp
assert resp["result"]["includers"] == []
@pytest.mark.workspace("index_features")
async def test_rpc_file_deps_unknown(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/fileDeps", {"path": "/nonexistent/file.cpp"})
assert "result" in resp
assert resp["result"]["includes"] == []
assert resp["result"]["includers"] == []
@pytest.mark.workspace("index_features")
async def test_rpc_impact_analysis(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/impactAnalysis", {"path": path})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert isinstance(result["directDependents"], list)
assert isinstance(result["transitiveDependents"], list)
assert isinstance(result["affectedModules"], list)
@pytest.mark.workspace("index_features")
async def test_rpc_impact_analysis_unknown(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/impactAnalysis", {"path": "/nonexistent/file.cpp"})
assert "result" in resp
assert resp["result"]["directDependents"] == []
async def test_shutdown_during_indexing(executable, tmp_path):
"""Shutdown during active background indexing must exit cleanly."""
from tests.integration.utils.client import CliceClient
from tests.conftest import _find_free_port
workspace = tmp_path / "ws"
workspace.mkdir()
entries = []
for i in range(20):
src = workspace / f"file_{i}.cpp"
src.write_text(
f"struct Type_{i} {{ int v = {i}; void m() {{}} }};\n"
f"int func_{i}(int x) {{ return x + {i}; }}\n"
f"int caller_{i}() {{ return func_{i}({i}); }}\n"
)
entries.append(
{
"directory": workspace.as_posix(),
"file": src.as_posix(),
"arguments": ["clang++", "-std=c++17", "-fsyntax-only", src.as_posix()],
}
)
(workspace / "compile_commands.json").write_text(json.dumps(entries))
host = "127.0.0.1"
port = _find_free_port()
cmd = [str(executable), "--mode", "pipe", "--host", host, "--port", str(port)]
c = CliceClient()
await c.start_io(*cmd)
init_options = {
"project": {
"cache_dir": str(workspace / ".clice"),
"idle_timeout_ms": 0,
}
}
await c.initialize(workspace, initialization_options=init_options)
# Give indexing a moment to start, then send shutdown
await asyncio.sleep(0.5)
rpc = AgenticRpcClient(host, port)
body = json.dumps({"jsonrpc": "2.0", "method": "agentic/shutdown", "params": {}})
rpc.sock.sendall(f"Content-Length: {len(body)}\r\n\r\n{body}".encode())
rpc.sock.settimeout(5)
try:
rpc.sock.recv(4096)
except (socket.timeout, OSError):
pass
rpc.sock.close()
for _ in range(30):
if c._server.returncode is not None:
break
await asyncio.sleep(0.5)
assert c._server.returncode is not None, "Server did not exit after shutdown"
assert c._server.returncode >= 0, (
f"Server crashed with signal {-c._server.returncode}"
)

View File

@@ -0,0 +1,189 @@
"""CLI-based tests for agentic mode — run clice --mode agentic as a subprocess."""
import json
import subprocess
import pytest
from tests.integration.utils.wait import wait_for_index
def run_cli(executable, host, port, method, **kwargs):
cmd = [
str(executable),
"--mode",
"agentic",
"--host",
host,
"--port",
str(port),
"--method",
method,
]
for k, v in kwargs.items():
cmd.extend([f"--{k}", str(v)])
result = subprocess.run(cmd, capture_output=True, text=True, timeout=10)
return result
@pytest.fixture
async def indexed_server(request, executable, workspace):
"""Start server with LSP+agentic, compile a file, wait for indexing."""
import asyncio
from tests.integration.utils.client import CliceClient
from tests.conftest import _shutdown_client, _find_free_port
host = "127.0.0.1"
port = _find_free_port()
cmd = [str(executable), "--mode", "pipe", "--host", host, "--port", str(port)]
c = CliceClient()
await c.start_io(*cmd)
init_options = {"project": {"cache_dir": str(workspace / ".clice")}}
await c.initialize(workspace, initialization_options=init_options)
uri, _ = await c.open_and_wait(workspace / "main.cpp")
assert await wait_for_index(c, uri, "add"), "Index not ready"
from tests.integration.agentic.test_agentic import AgenticRpcClient
rpc = AgenticRpcClient(host, port)
for _ in range(30):
resp = rpc.request("agentic/symbolSearch", {"query": "add"})
if "result" in resp and resp["result"]["symbols"]:
break
await asyncio.sleep(1)
rpc.close()
yield executable, host, port, workspace
c.close(uri)
await _shutdown_client(c)
@pytest.mark.workspace("index_features")
async def test_cli_compile_command(indexed_server, workspace):
exe, host, port, _ = indexed_server
path = (workspace / "main.cpp").as_posix()
r = run_cli(exe, host, port, "compileCommand", path=path)
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["file"] == path
assert len(data["arguments"]) > 0
@pytest.mark.workspace("index_features")
async def test_cli_symbol_search(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "symbolSearch", query="add")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
names = [s["name"] for s in data["symbols"]]
assert "add" in names
add_sym = next(s for s in data["symbols"] if s["name"] == "add")
assert add_sym["kind"] == "Function"
assert add_sym["line"] == 19
@pytest.mark.workspace("index_features")
async def test_cli_definition(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "definition", name="add")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["name"] == "add"
defn = data["definition"]
assert defn["startLine"] == 19
assert defn["endLine"] == 21
assert "return a + b;" in defn["text"]
@pytest.mark.workspace("index_features")
async def test_cli_definition_by_position(indexed_server, workspace):
exe, host, port, _ = indexed_server
path = (workspace / "main.cpp").as_posix()
r = run_cli(exe, host, port, "definition", path=path, line=19)
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["name"] == "add"
@pytest.mark.workspace("index_features")
async def test_cli_references(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "references", name="global_var")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["name"] == "global_var"
assert data["total"] == 2
lines = sorted(ref["line"] for ref in data["references"])
assert lines == [34, 38]
@pytest.mark.workspace("index_features")
async def test_cli_read_symbol(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "readSymbol", name="compute")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["name"] == "compute"
assert "add(1, 2)" in data["text"]
@pytest.mark.workspace("index_features")
async def test_cli_document_symbols(indexed_server, workspace):
exe, host, port, _ = indexed_server
path = (workspace / "main.cpp").as_posix()
r = run_cli(exe, host, port, "documentSymbols", path=path)
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
names = [s["name"] for s in data["symbols"]]
assert "add" in names
assert "main" in names
assert "global_var" in names
@pytest.mark.workspace("index_features")
async def test_cli_call_graph(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "callGraph", name="add", direction="callers")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["root"]["name"] == "add"
caller_names = [c["name"] for c in data["callers"]]
assert "compute" in caller_names
@pytest.mark.workspace("index_features")
async def test_cli_type_hierarchy(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "typeHierarchy", name="Dog", direction="supertypes")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["root"]["name"] == "Dog"
supertype_names = [t["name"] for t in data["supertypes"]]
assert "Animal" in supertype_names
@pytest.mark.workspace("index_features")
async def test_cli_project_files(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "projectFiles")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["total"] > 0
paths = [f["path"] for f in data["files"]]
assert any("main.cpp" in p for p in paths)
@pytest.mark.workspace("index_features")
async def test_cli_status(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "status")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert isinstance(data["idle"], bool)
assert data["total"] > 0
assert isinstance(data["pending"], int)
assert isinstance(data["indexed"], int)

View File

@@ -24,9 +24,17 @@ from tests.integration.utils.cache import (
from tests.integration.utils.assertions import assert_clean_compile
def _pin_cache_to_workspace(tmp_path):
"""Write a clice.toml that pins cache_dir to <workspace>/.clice/."""
(tmp_path / "clice.toml").write_text(
'[project]\ncache_dir = "${workspace}/.clice"\n'
)
async def test_pch_written_to_cache_dir(client, tmp_path):
"""After opening a file with #include, a .pch file should appear
in .clice/cache/pch/ with a hex-hash filename."""
_pin_cache_to_workspace(tmp_path)
(tmp_path / "header.h").write_text("#pragma once\nstruct Foo { int x; };\n")
(tmp_path / "main.cpp").write_text(
'#include "header.h"\nint main() { Foo f; return f.x; }\n'
@@ -48,6 +56,7 @@ async def test_pch_written_to_cache_dir(client, tmp_path):
async def test_cache_json_persisted(client, tmp_path):
"""After a PCH build, cache.json should be written with the entry."""
_pin_cache_to_workspace(tmp_path)
(tmp_path / "header.h").write_text("#pragma once\nint global_val = 42;\n")
(tmp_path / "main.cpp").write_text(
'#include "header.h"\nint main() { return global_val; }\n'
@@ -74,6 +83,7 @@ async def test_cache_json_persisted(client, tmp_path):
async def test_pch_reused_on_close_reopen(client, tmp_path):
"""Closing and reopening a file within the same session should reuse
the cached PCH — no additional .pch files should be created."""
_pin_cache_to_workspace(tmp_path)
(tmp_path / "header.h").write_text("#pragma once\nstruct Bar { int y; };\n")
(tmp_path / "main.cpp").write_text(
'#include "header.h"\nint main() { Bar b; return b.y; }\n'
@@ -108,6 +118,7 @@ async def test_pch_reused_on_close_reopen(client, tmp_path):
async def test_pch_survives_server_restart(executable, tmp_path):
"""PCH cache should survive a full server restart — cache.json is
loaded on startup and the existing .pch file is reused."""
_pin_cache_to_workspace(tmp_path)
(tmp_path / "header.h").write_text("#pragma once\nstruct Baz { int z; };\n")
(tmp_path / "main.cpp").write_text(
'#include "header.h"\nint main() { Baz b; return b.z; }\n'
@@ -150,6 +161,7 @@ async def test_pch_survives_server_restart(executable, tmp_path):
async def test_shared_preamble_shares_pch(client, tmp_path):
"""Two files with identical preambles should share the same PCH file
(content-addressed by preamble hash)."""
_pin_cache_to_workspace(tmp_path)
(tmp_path / "header.h").write_text("#pragma once\nint shared_val = 1;\n")
(tmp_path / "a.cpp").write_text(
'#include "header.h"\nint fa() { return shared_val; }\n'
@@ -176,6 +188,7 @@ async def test_shared_preamble_shares_pch(client, tmp_path):
async def test_different_preamble_different_pch(client, tmp_path):
"""Files with different preambles should produce different PCH files."""
_pin_cache_to_workspace(tmp_path)
(tmp_path / "a.h").write_text("#pragma once\nint val_a = 1;\n")
(tmp_path / "b.h").write_text("#pragma once\nint val_b = 2;\n")
(tmp_path / "a.cpp").write_text('#include "a.h"\nint fa() { return val_a; }\n')
@@ -199,6 +212,7 @@ async def test_different_preamble_different_pch(client, tmp_path):
async def test_pch_rebuilt_on_header_change(client, tmp_path):
"""When a preamble header changes, a new PCH should be built
(different hash → different filename). The old one remains for cleanup."""
_pin_cache_to_workspace(tmp_path)
(tmp_path / "header.h").write_text("#pragma once\nstruct V1 { int a; };\n")
(tmp_path / "main.cpp").write_text(
'#include "header.h"\nint main() { V1 v; return v.a; }\n'
@@ -240,6 +254,7 @@ async def test_pch_rebuilt_on_header_change(client, tmp_path):
async def test_no_tmp_files_after_build(client, tmp_path):
"""After a successful PCH build, no .tmp files should remain in the cache dir."""
_pin_cache_to_workspace(tmp_path)
(tmp_path / "header.h").write_text("#pragma once\nint val = 1;\n")
(tmp_path / "main.cpp").write_text(
'#include "header.h"\nint main() { return val; }\n'
@@ -265,6 +280,7 @@ async def test_no_tmp_files_after_build(client, tmp_path):
async def test_cache_dirs_created_on_startup(client, tmp_path):
"""The .clice/cache/pch/ and .clice/cache/pcm/ directories should be created
when the server initializes a workspace."""
_pin_cache_to_workspace(tmp_path)
(tmp_path / "main.cpp").write_text("int main() { return 0; }\n")
write_cdb(tmp_path, ["main.cpp"])
await client.initialize(tmp_path)

View File

@@ -0,0 +1,74 @@
import pytest
from lsprotocol.types import Position, Range
from tests.integration.utils.workspace import did_change
UNFORMATTED = "int add( int a , int b ) {\nreturn a+b ;\n}\n"
FORMATTED = "int add(int a, int b) { return a + b; }\n"
def apply_edits(text, edits):
"""Apply LSP TextEdits to a string, processing from end to start."""
lines = text.split("\n")
for edit in sorted(
edits, key=lambda e: (e.range.start.line, e.range.start.character), reverse=True
):
start = edit.range.start
end = edit.range.end
before = (
"\n".join(lines[: start.line])
+ ("\n" if start.line > 0 else "")
+ lines[start.line][: start.character]
)
after = (
lines[end.line][end.character :]
+ ("\n" if end.line < len(lines) - 1 else "")
+ "\n".join(lines[end.line + 1 :])
)
text = before + edit.new_text + after
lines = text.split("\n")
return text
@pytest.mark.workspace("formatting")
async def test_format_document(client, workspace):
uri, _ = await client.open_and_wait(workspace / "main.cpp")
did_change(client, uri, 1, UNFORMATTED)
edits = await client.format_document(uri)
assert edits is not None
assert len(edits) > 0
result = apply_edits(UNFORMATTED, edits)
assert result == FORMATTED
client.close(uri)
@pytest.mark.workspace("formatting")
async def test_format_range(client, workspace):
uri, _ = await client.open_and_wait(workspace / "main.cpp")
did_change(client, uri, 1, UNFORMATTED)
edits = await client.format_range(
uri,
Range(start=Position(line=1, character=0), end=Position(line=2, character=0)),
)
assert edits is not None
assert len(edits) > 0
client.close(uri)
@pytest.mark.workspace("formatting")
async def test_format_already_formatted(client, workspace):
uri, _ = await client.open_and_wait(workspace / "main.cpp")
did_change(client, uri, 1, FORMATTED)
edits = await client.format_document(uri)
assert edits is not None
assert len(edits) == 0
client.close(uri)

View File

@@ -34,6 +34,8 @@ async def test_capabilities(client, workspace):
assert capability_enabled(caps.folding_range_provider)
assert capability_enabled(caps.inlay_hint_provider)
assert capability_enabled(caps.code_action_provider)
assert caps.document_formatting_provider is True
assert caps.document_range_formatting_provider is True
assert caps.semantic_tokens_provider is not None

View File

@@ -0,0 +1,68 @@
"""Integration tests for clice configuration (clice.toml + initializationOptions).
Each workspace's main.cpp references a macro that is only defined when the
rule's `-D<macro>=...` is applied. When rules are applied, compilation is
clean; otherwise an undeclared-identifier diagnostic surfaces.
"""
import pytest
from tests.integration.utils.assertions import (
assert_clean_compile,
assert_has_errors,
get_errors,
)
@pytest.mark.workspace("config_rules_no_config")
async def test_baseline_without_rules(client, workspace):
uri, _ = await client.open_and_wait(workspace / "main.cpp")
assert_has_errors(client, uri, "Expected diagnostics without any rules applied")
errors = get_errors(client.diagnostics[uri])
assert any("FROM_INIT" in (d.message or "") for d in errors), (
f"Expected a diagnostic referencing FROM_INIT, got: {errors}"
)
@pytest.mark.workspace("config_rules_toml")
async def test_rules_from_toml(client, workspace):
uri, _ = await client.open_and_wait(workspace / "main.cpp")
assert_clean_compile(client, uri)
symbols = await client.document_symbols(uri)
assert symbols, "Expected document symbols for value()/main()"
hover = await client.hover_at(uri, line=4, character=4) # on 'main'
assert hover is not None
@pytest.mark.workspace("config_rules_no_config")
@pytest.mark.init_options(
{"rules": [{"patterns": ["**/*.cpp"], "append": ["-DFROM_INIT=1"]}]}
)
async def test_rules_from_init_options(client, workspace):
uri, _ = await client.open_and_wait(workspace / "main.cpp")
assert_clean_compile(client, uri)
@pytest.mark.workspace("config_rules_toml")
@pytest.mark.init_options(
{"rules": [{"patterns": ["**/*.cpp"], "append": ["-DUNRELATED"]}]}
)
async def test_init_options_replaces_toml_rules(client, workspace):
uri, _ = await client.open_and_wait(workspace / "main.cpp")
assert_has_errors(
client, uri, "initializationOptions should have overridden clice.toml rules"
)
errors = get_errors(client.diagnostics[uri])
assert any("FROM_TOML" in (d.message or "") for d in errors), (
f"Expected FROM_TOML diagnostic after override, got: {errors}"
)
@pytest.mark.workspace("config_rules_no_config")
@pytest.mark.init_options(
{"rules": [{"patterns": ["**/does_not_match.cpp"], "append": ["-DFROM_INIT=1"]}]}
)
async def test_rules_pattern_mismatch(client, workspace):
uri, _ = await client.open_and_wait(workspace / "main.cpp")
assert_has_errors(client, uri, "Rule pattern should not have matched main.cpp")

View File

@@ -16,9 +16,12 @@ from lsprotocol.types import (
Diagnostic,
DidCloseTextDocumentParams,
DidOpenTextDocumentParams,
DocumentFormattingParams,
DocumentLinkParams,
DocumentRangeFormattingParams,
DocumentSymbolParams,
FoldingRangeParams,
FormattingOptions,
HoverParams,
InlayHintParams,
InitializeParams,
@@ -86,16 +89,25 @@ class CliceClient(BaseLanguageClient):
# ── Lifecycle ────────────────────────────────────────────────────
async def initialize(self, workspace: Path) -> InitializeResult:
result = await self.initialize_async(
InitializeParams(
capabilities=ClientCapabilities(),
root_uri=workspace.as_uri(),
workspace_folders=[
WorkspaceFolder(uri=workspace.as_uri(), name="test")
],
)
async def initialize(
self,
workspace: Path,
*,
initialization_options: dict | None = None,
) -> InitializeResult:
if initialization_options is None:
initialization_options = {}
project = dict(initialization_options.get("project", {}))
project.setdefault("cache_dir", str(workspace / ".clice"))
initialization_options["project"] = project
params = InitializeParams(
capabilities=ClientCapabilities(),
root_uri=workspace.as_uri(),
workspace_folders=[WorkspaceFolder(uri=workspace.as_uri(), name="test")],
)
params.initialization_options = initialization_options
result = await self.initialize_async(params)
self.initialized(InitializedParams())
self.init_result = result
return result
@@ -303,6 +315,29 @@ class CliceClient(BaseLanguageClient):
timeout=timeout,
)
async def format_document(self, uri: str, *, timeout: float = 30.0):
return await asyncio.wait_for(
self.text_document_formatting_async(
DocumentFormattingParams(
text_document=TextDocumentIdentifier(uri=uri),
options=FormattingOptions(tab_size=4, insert_spaces=True),
)
),
timeout=timeout,
)
async def format_range(self, uri: str, range_: Range, *, timeout: float = 30.0):
return await asyncio.wait_for(
self.text_document_range_formatting_async(
DocumentRangeFormattingParams(
text_document=TextDocumentIdentifier(uri=uri),
range=range_,
options=FormattingOptions(tab_size=4, insert_spaces=True),
)
),
timeout=timeout,
)
# ── Extension protocol ───────────────────────────────────────────
async def query_context(self, uri: str, *, timeout: float = 30.0):

View File

@@ -1,3 +1,5 @@
[pytest]
asyncio_mode = auto
markers = workspace
markers =
workspace
init_options

View File

@@ -13,6 +13,9 @@ import re
import signal
import sys
import time
# Force line-buffered stdout so CI sees output immediately.
sys.stdout.reconfigure(line_buffering=True)
from pathlib import Path
from urllib.parse import quote, unquote
@@ -109,7 +112,9 @@ async def write_lsp_message(writer: asyncio.StreamWriter, payload: str):
await writer.drain()
async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool | None:
async def replay_one(
trace_path: Path, clice_bin: Path, timeout: int, wall_timeout: int = 300
) -> bool | None:
"""Replay a single trace. Returns True=PASS, False=FAIL, None=SKIP."""
records = load_trace(trace_path)
if not records:
@@ -179,8 +184,21 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
last_method = None
sent_count = 0
wall_deadline = wall_start + wall_timeout
def remaining_wall():
return max(0, wall_deadline - time.monotonic())
try:
for i, rec in enumerate(records):
if remaining_wall() <= 0:
elapsed = time.monotonic() - wall_start
print(
f" result: TIMEOUT (wall-clock {wall_timeout}s exceeded, {elapsed:.1f}s)"
)
success = False
break
if i > 0:
delay = rec["ts"] - records[i - 1]["ts"]
if delay > 0:
@@ -196,7 +214,7 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
try:
await asyncio.wait_for(
asyncio.gather(*pending.values(), return_exceptions=True),
timeout=timeout,
timeout=min(timeout, remaining_wall()),
)
except asyncio.TimeoutError:
elapsed = time.monotonic() - wall_start
@@ -210,7 +228,19 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
if msg_id is not None and method is not None:
pending[msg_id] = asyncio.get_event_loop().create_future()
await write_lsp_message(proc.stdin, rec["msg"])
try:
await asyncio.wait_for(
write_lsp_message(proc.stdin, rec["msg"]),
timeout=min(30, remaining_wall()),
)
except asyncio.TimeoutError:
elapsed = time.monotonic() - wall_start
print(
f" result: HANG (write blocked at {last_method},"
f" sent={sent_count}/{len(records)}, {elapsed:.1f}s)"
)
success = False
break
sent_count = i + 1
except (ConnectionError, BrokenPipeError):
@@ -231,7 +261,7 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
try:
await asyncio.wait_for(
asyncio.gather(*pending.values(), return_exceptions=True),
timeout=timeout,
timeout=min(timeout, remaining_wall()),
)
except asyncio.TimeoutError:
elapsed = time.monotonic() - wall_start
@@ -294,7 +324,7 @@ async def async_main(args):
print(f"SKIP: {trace} (not found)")
skipped += 1
continue
result = await replay_one(trace, args.clice, args.timeout)
result = await replay_one(trace, args.clice, args.timeout, args.wall_timeout)
if result is None:
skipped += 1
elif result:
@@ -317,7 +347,16 @@ def main():
p.add_argument("traces", nargs="+", type=Path, help="JSONL trace files")
p.add_argument("--clice", required=True, type=Path, help="Path to clice binary")
p.add_argument(
"--timeout", type=int, default=120, help="Timeout in seconds (default: 120)"
"--timeout",
type=int,
default=120,
help="Per-request timeout in seconds (default: 120)",
)
p.add_argument(
"--wall-timeout",
type=int,
default=300,
help="Max wall-clock time per trace in seconds (default: 300)",
)
args = p.parse_args()
sys.exit(asyncio.run(async_main(args)))

View File

@@ -0,0 +1,14 @@
---
source: document_symbol_tests.cpp
created_at: 2026-05-20
input_file: statements/if/basic_if.cpp
---
- { name: "basic_if", kind: Namespace, range: "1:0-35:1", selection_range: "1:10-1:18" }
- { name: "abs_val", kind: Function, range: "3:0-7:1", selection_range: "3:4-3:11", detail: "int (int)" }
- { name: "sign", kind: Function, range: "9:0-17:1", selection_range: "9:12-9:16", detail: "const char *(int)" }
- { name: "nested_if", kind: Function, range: "20:0-27:1", selection_range: "20:4-20:13", detail: "int (int, int)" }
- { name: "test", kind: Function, range: "29:0-33:1", selection_range: "29:5-29:9", detail: "void ()" }
- { name: "r1", kind: Variable, range: "30:21-30:41", selection_range: "30:25-30:27", detail: "int" }
- { name: "r2", kind: Variable, range: "31:21-31:38", selection_range: "31:26-31:28", detail: "const char *" }
- { name: "r3", kind: Variable, range: "32:21-32:46", selection_range: "32:25-32:27", detail: "int" }

View File

@@ -0,0 +1,11 @@
---
source: folding_range_tests.cpp
created_at: 2026-05-20
input_file: statements/if/basic_if.cpp
---
- { range: "1:19-35:1", kind: namespace, collapsed_text: "{...}" }
- { range: "3:19-7:1", kind: functionBody, collapsed_text: "{...}" }
- { range: "9:24-17:1", kind: functionBody, collapsed_text: "{...}" }
- { range: "20:28-27:1", kind: functionBody, collapsed_text: "{...}" }
- { range: "29:12-33:1", kind: functionBody, collapsed_text: "{...}" }

View File

@@ -0,0 +1,11 @@
---
source: inlay_hint_tests.cpp
created_at: 2026-05-20
input_file: statements/if/basic_if.cpp
---
- { pos: "30:38", kind: Parameter, label: "x:", padding_right: true }
- { pos: "31:28", kind: Type, label: ": const char *" }
- { pos: "31:36", kind: Parameter, label: "x:", padding_right: true }
- { pos: "32:40", kind: Parameter, label: "a:", padding_right: true }
- { pos: "32:43", kind: Parameter, label: "b:", padding_right: true }

View File

@@ -0,0 +1,75 @@
---
source: semantic_tokens_tests.cpp
created_at: 2026-05-20
input_file: statements/if/basic_if.cpp
---
- { loc: "0:0", text: "// basic if and if-else", kind: Comment }
- { loc: "1:0", text: "namespace", kind: Keyword }
- { loc: "1:10", text: "basic_if", kind: Namespace, modifiers: [Definition] }
- { loc: "3:0", text: "int", kind: Keyword }
- { loc: "3:4", text: "abs_val", kind: Function, modifiers: [Definition] }
- { loc: "3:12", text: "int", kind: Keyword }
- { loc: "3:16", text: "x", kind: Parameter, modifiers: [Definition] }
- { loc: "4:4", text: "if", kind: Keyword }
- { loc: "4:7", text: "x", kind: Parameter }
- { loc: "4:11", text: "0", kind: Number }
- { loc: "5:8", text: "return", kind: Keyword }
- { loc: "5:16", text: "x", kind: Parameter }
- { loc: "6:4", text: "return", kind: Keyword }
- { loc: "6:11", text: "x", kind: Parameter }
- { loc: "9:0", text: "const", kind: Keyword }
- { loc: "9:6", text: "char", kind: Keyword }
- { loc: "9:12", text: "sign", kind: Function, modifiers: [Definition, Readonly] }
- { loc: "9:17", text: "int", kind: Keyword }
- { loc: "9:21", text: "x", kind: Parameter, modifiers: [Definition] }
- { loc: "10:4", text: "if", kind: Keyword }
- { loc: "10:7", text: "x", kind: Parameter }
- { loc: "10:11", text: "0", kind: Number }
- { loc: "11:8", text: "return", kind: Keyword }
- { loc: "11:15", text: "\"positive\"", kind: String }
- { loc: "12:6", text: "else", kind: Keyword }
- { loc: "12:11", text: "if", kind: Keyword }
- { loc: "12:14", text: "x", kind: Parameter }
- { loc: "12:18", text: "0", kind: Number }
- { loc: "13:8", text: "return", kind: Keyword }
- { loc: "13:15", text: "\"negative\"", kind: String }
- { loc: "14:6", text: "else", kind: Keyword }
- { loc: "15:8", text: "return", kind: Keyword }
- { loc: "15:15", text: "\"zero\"", kind: String }
- { loc: "19:0", text: "// dangling else: else binds to nearest if", kind: Comment }
- { loc: "20:0", text: "int", kind: Keyword }
- { loc: "20:4", text: "nested_if", kind: Function, modifiers: [Definition] }
- { loc: "20:14", text: "int", kind: Keyword }
- { loc: "20:18", text: "a", kind: Parameter, modifiers: [Definition] }
- { loc: "20:21", text: "int", kind: Keyword }
- { loc: "20:25", text: "b", kind: Parameter, modifiers: [Definition] }
- { loc: "21:4", text: "if", kind: Keyword }
- { loc: "21:7", text: "a", kind: Parameter }
- { loc: "21:11", text: "0", kind: Number }
- { loc: "22:8", text: "if", kind: Keyword }
- { loc: "22:11", text: "b", kind: Parameter }
- { loc: "22:15", text: "0", kind: Number }
- { loc: "23:12", text: "return", kind: Keyword }
- { loc: "23:19", text: "1", kind: Number }
- { loc: "24:8", text: "else", kind: Keyword }
- { loc: "25:12", text: "return", kind: Keyword }
- { loc: "25:19", text: "2", kind: Number }
- { loc: "26:4", text: "return", kind: Keyword }
- { loc: "26:11", text: "0", kind: Number }
- { loc: "29:0", text: "void", kind: Keyword }
- { loc: "29:5", text: "test", kind: Function, modifiers: [Definition] }
- { loc: "30:21", text: "int", kind: Keyword }
- { loc: "30:25", text: "r1", kind: Variable, modifiers: [Definition] }
- { loc: "30:30", text: "abs_val", kind: Function }
- { loc: "30:39", text: "3", kind: Number }
- { loc: "31:21", text: "auto", kind: Keyword }
- { loc: "31:26", text: "r2", kind: Variable, modifiers: [Definition, Readonly] }
- { loc: "31:31", text: "sign", kind: Function, modifiers: [Readonly] }
- { loc: "31:36", text: "5", kind: Number }
- { loc: "32:21", text: "int", kind: Keyword }
- { loc: "32:25", text: "r3", kind: Variable, modifiers: [Definition] }
- { loc: "32:30", text: "nested_if", kind: Function }
- { loc: "32:40", text: "1", kind: Number }
- { loc: "32:44", text: "1", kind: Number }
- { loc: "35:3", text: "// namespace basic_if", kind: Comment }

Some files were not shown because too many files have changed in this diff Show More