8 Commits

Author SHA1 Message Date
21d776dfc0 move to gitea workflow folder
Some checks failed
main / changes (push) Successful in 13s
main / conventional-commit (push) Failing after 2s
main / release-clice (push) Has been skipped
main / release-vscode (push) Has been skipped
main / deploy (push) Has been cancelled
main / vscode (push) Has been cancelled
main / format (push) Has been cancelled
main / checks-passed (push) Failing after 3s
main / cmake (push) Has been skipped
2026-06-04 22:55:36 -03:00
Myriad-Dreamin
3b45888622 fix(semantic-tokens): filter ineligible highlight references (#434)
Add a reusable declaration-name eligibility helper that mirrors clangd's
`canHighlightName`, use it to suppress unsupported reference tokens in
the semantic-token collector, and cover the change with focused
semantic-token regression tests plus a constructor/destructor positive
case.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved semantic token highlighting to suppress ineligible operator
references lacking meaningful source text.
* Ensured constructor and destructor names remain properly highlighted
with correct visual modifiers.

* **Tests**
* Added test coverage for semantic token highlighting behavior across
various declaration types.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/clice-io/clice/pull/434?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-05 00:03:18 +08:00
ykiko
cc5b25d5c3 refactor: public feature types and snapshot testing infrastructure (#442)
## Summary

- **Public feature types**: Move `SemanticToken`, `FoldingRange`,
`DocumentSymbol`, `InlayHint`, and `HintCategory` from internal `.cpp`
files to `feature.h` as public API types. Each feature now exposes two
overloads: a raw overload returning offset-based types and a protocol
overload that converts to LSP wire-format with explicit
`PositionEncoding`.
- **Snapshot testing**: Add corpus-driven snapshot tests using
`ASSERT_SNAPSHOT_GLOB` for semantic tokens, folding ranges, inlay hints,
document symbols, and TU index. Tests compile real C++ corpus files,
format output as YAML flow mappings, and diff against `.snap.yml`
baselines.
- **Test infrastructure**: Add `compile_file()` to `Tester`,
`yaml_str()` utility, `--corpus-dir` / `--snapshot-dir` CLI options, and
`--verbose` flag for unit tests. Migrate to kotatsu's unified
`kota::zest::Options` API.
- **Toolchain robustness**: Filter unknown cc1 args via
`clang::driver::getDriverOptTable()` to handle system compilers newer
than embedded LLVM.
- **Dependency bump**: Update kotatsu to 7381404 (unified zest Options,
out-param `from_json` API).

## Details

### Feature type changes
All five feature modules (`semantic_tokens`, `folding_ranges`,
`document_symbols`, `inlay_hints`, `document_links`) now follow the same
two-overload pattern. The raw overload returns offset-based structs
suitable for indexing and testing; the protocol overload adds
`PositionEncoding` conversion for LSP responses. `stateful_worker.cpp`
explicitly passes `PositionEncoding::UTF16` at every call site.

### Snapshot tests
Corpus files live in `tests/corpus/` (organized by language construct).
Snapshot baselines live in `tests/snapshots/<feature>/`. Format lambdas
are inlined directly in test bodies — no separate format functions for
single-use formatters. YAML output uses flow mappings (`- { key: value
}`) for compact, diffable baselines.

### cc1 arg filtering
`src/command/toolchain.cpp` now parses the cc1 argument list through
LLVM's driver option table and drops any args classified as
`UnknownClass`. This prevents compilation failures when the system
compiler emits flags that the embedded LLVM version doesn't recognize.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-24 19:36:27 +08:00
ykiko
3305465d1f feat(formatting): wire up textDocument/formatting and rangeFormatting (#441)
## Summary

- Wire up the existing `document_format` feature to LSP via stateless
workers
- Add `Format` kind to stateless worker dispatch, with a lightweight
`forward_format` path in `Compiler` (no compilation/deps needed — just
file path + content)
- Register `textDocument/formatting` and `textDocument/rangeFormatting`
handlers with `scoped_pause`
- Style lookup uses `clang::format::getStyle` which walks parent
directories for `.clang-format`, matching clangd's behavior

## Test plan

- [x] 4 unit tests: simple format, range format, idempotent (no edits),
include sort
- [x] 3 integration tests: full document format (verifies applied edits
match expected output), range format, already-formatted no-op
- [x] Capability assertions added to `test_capabilities`
- [x] All existing tests pass (554 unit, 170 integration, 2 smoke)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

* **New Features**
* Added document formatting and range formatting capabilities to the LSP
server
* Formatting can target the entire document or a specific range of code
  * Server now advertises formatting support to LSP clients

* **Tests**
  * Added comprehensive test coverage for formatting functionality

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-04 19:15:07 +08:00
ykiko
47ad905f5b feat(server): agentic query API, daemon mode, and relay mode (#438)
## Summary
- **Agentic query API**: 11 JSON-RPC handlers over TCP for AI agent
integration — compileCommand, projectFiles, fileDeps, impactAnalysis,
symbolSearch, readSymbol, documentSymbols, definition, references,
callGraph, typeHierarchy
- **Daemon mode**: MasterServer listens on a unix domain socket,
accepting multiple agent connections (`--mode daemon`)
- **Relay mode**: Bidirectional stdin/stdout ↔ unix socket proxy for
editor integration (`--mode relay`)
- **Full-body definition text**: readSymbol/definition return complete
function/class bodies via brace matching, not just the declaration line
- **24 integration tests** with concrete value assertions covering all
handlers

## Test plan
- [x] All 551 unit tests pass
- [x] All 2 smoke tests pass
- [x] All 148 integration tests pass (including 24 new agentic tests)
- [x] Raw JSON responses manually inspected for correctness (line
numbers, text content, field names, structural completeness)
- [x] Formatted with `pixi run format`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Daemon mode: background service with workspace watching and socket
support
  * Relay mode: stdio ↔ Unix-socket message relay
* Expanded agentic RPC/CLI: project files, deps/impact, symbol
search/read, document symbols, definitions, references, call graphs,
type hierarchy, status, shutdown; richer query options

* **Behavioral**
* Improved indexing lifecycle and status reporting; safer shutdown
coordination and client handling

* **Tests**
* New integration tests covering agentic RPC endpoints, CLI status,
shutdown, and error cases
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-04 00:15:37 +08:00
ykiko
75b9ea05b8 refactor(server): split into service layer, add agentic protocol, adopt task_group (#437)
## Summary

- **Restructure `src/server/` into subdirectories** (`service/`,
`compiler/`, `worker/`, `workspace/`, `protocol/`) to separate concerns:
transport/session management, compilation, worker orchestration, and
persistent workspace state.
- **Decouple MasterServer from transport**: MasterServer no longer holds
a `JsonPeer&` reference or registers handlers itself. New `LSPClient`
and `AgentClient` classes own their peer references and register
protocol handlers, accessing MasterServer internals via `friend class`.
- **Add agentic protocol**: A TCP-based side channel
(`agentic/compileCommand`) that lets external tools (AI agents, build
systems) query compile commands from a running clice server. Includes a
CLI client mode (`--mode agentic --port N --path FILE`), server-side
listener when `--port` is specified in pipe mode, and integration tests
for happy path, fallback, concurrency, and connection-refused.
- **Replace fire-and-forget `loop.schedule()` with `kota::task_group`**:
Compiler compile tasks, Indexer background indexing + resource monitor,
WorkerPool worker monitors, and socket accept loops now use structured
concurrency. This eliminates manual `alive_count_`/generation counters
and ensures all spawned tasks are joined on shutdown.
- **Fix flaky integration test**: `CliceClient.initialize()` now always
sets `cache_dir` to a workspace-local `.clice/` directory, preventing
stale PCH artifacts from the global `~/.cache/clice/` from polluting
test runs.

## Details

**Compiler peer lifetime**: `Compiler` and `Indexer` previously took
`JsonPeer&` in their constructors, coupling them to a single connection.
They now store a `JsonPeer*` set via `set_peer()`, with null checks
before sending diagnostics/progress. This supports the multi-connection
model where agentic clients don't need diagnostics.

**Socket mode single-LSP enforcement**: `accept_connections()` takes a
`register_lsp` flag; when true, only the first connection gets an
`LSPClient`. All connections get an `AgentClient`. This prevents
multiple LSP sessions from racing on shared server state.

**Structured shutdown**: `Compiler::stop()` cancels in-flight compile
tasks and joins them. `WorkerPool::stop()` signals workers and joins the
monitor task group. `Indexer` uses a `cancellation_source` to stop its
resource monitor when a background indexing run completes.

**Pin kotatsu**: Changed from `GIT_TAG main` + `GIT_SHALLOW TRUE` to an
exact commit hash for reproducible builds.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-02 01:06:18 +08:00
ykiko
939ab6d0d4 feat(server): concurrent background indexing with priority control (#432)
## Summary

- Rewrite serial background indexing to concurrent dispatch (up to
`stateless_worker_count / 2` parallel tasks)
- Add depth-counted pause/resume mechanism: completion and
signature-help handlers pause new index dispatches to prioritize user
requests
- Report indexing progress via LSP `$/progress` notifications
(percentage + file count)
- Lower thread scheduling priority (`nice +10`) for index tasks in
stateless workers via RAII `ScopedNice` guard

## Test plan

- [x] `pixi run format` — no changes
- [x] `pixi run unit-test Debug` — 551 passed, 9 skipped (pre-existing)
- [x] `pixi run smoke-test Debug` — 2/2 passed
- [x] `pixi run integration-test Debug` — 121 passed, 3 failed (all
pre-existing on main: header_context x2, staleness x1)
- [ ] Manual test: open a large project (e.g. LLVM), verify progress bar
appears and completion remains responsive during indexing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Pause/resume controls for background indexing
* Concurrent, adaptive background indexing with configurable concurrency
* LSP progress reporting (create/begin/report/end) and updated
completion metrics

* **Behavior Change**
* Code completion and signature help temporarily pause indexing for
responsiveness
* Background indexing runs with reduced scheduling priority on
non-Windows and logs "files dispatched" at finish

* **Tests**
* Test client fixture defaults init options and sets workspace cache dir
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-23 13:28:59 +08:00
ykiko
e1202d2fa5 fix: prevent worker crashes from null ASTConsumer, invalid FileID, and missing PCH cache dir (#435)
## Summary

Three pre-existing bugs cause worker processes to crash with SEGV or
SIGABRT. On the main branch these crashes are silent (workers die,
requests fail fast with "transport closed", tests still pass because
null responses are accepted). However when combined with #432's worker
respawn mechanism, the crash-respawn-crash cycle on low-core CI machines
causes request timeouts and smoke test hangs.

### Fixes

- **compilation.cpp**: `ProxyAction::CreateASTConsumer` now checks for
null before passing to `MultiplexConsumer`. When the wrapped action's
`CreateASTConsumer` fails (e.g. missing system headers during PCH
generation), this previously caused a null pointer dereference, SEGV,
ASAN kills the stateless worker.
- **compilation_unit.cpp**: `file_path()` returns empty `StringRef` on
invalid `FileID` instead of asserting. The assert fired when
`IncludeGraph::from()` called `file_path(interested_file())` on an AST
compiled with synthesized default commands (no compile_commands.json,
clang++ -std=c++20 fallback, no system headers, invalid main file ID),
SIGABRT, stateful worker crash.
- **compiler.cpp**: `ensure_pch` now creates the PCH cache directory
before sending the build request. Previously, when `load_workspace()`
exited early (no compile_commands.json), the cache subdirectories were
never created, causing every PCH write to fail with "No such file or
directory".
- **master_server.cpp/h**: `load_workspace()` changed from
`kota::task<>` to plain `void` -- it contains only synchronous
filesystem operations and no co_await, so the coroutine wrapper was
unnecessary. Called directly instead of via `loop.schedule()`.

## Test plan

- [x] Verified zero SEGV/SIGABRT/assertion crashes in worker stderr
after fix
- [x] rapid_edit.jsonl smoke test passes 3/3 runs consistently (34s
each)
- [x] Behavior matches main branch (both return 134 responses, 0
pending)
- [x] Debug build with ASAN (detect_leaks=0) -- clean run, no sanitizer
reports

<!-- codesmith:footer -->
---
<a
href="https://app.blacksmith.sh/clice-io/codesmith/clice/pr/435"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-in-codesmith-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-in-codesmith-light.svg"><img
alt="View in Codesmith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/view-in-codesmith-dark.svg"></picture></a>
<sup>Codesmith can help with this PR — just tag <code>@codesmith</code>
or enable autofix.</sup>

- [ ] Autofix CI and bot reviews
<!-- /codesmith:footer -->

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved error handling for AST consumer creation with null checks and
a clear failure path.
* Safer file-path access that returns empty for invalid identifiers
instead of asserting.
* PCH cache handling now validates cache configuration, attempts
directory creation, logs warnings, and aborts PCH builds on failure.

* **Refactor**
* Workspace loading changed from asynchronous to synchronous execution.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-23 10:36:03 +08:00
114 changed files with 6732 additions and 1421 deletions

View File

@@ -0,0 +1,40 @@
---
name: Bug report
about: Create a report to help us improve
title: ""
labels: ""
assignees: ""
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.

View File

@@ -0,0 +1,19 @@
---
name: Feature request
about: Suggest an idea for this project
title: ""
labels: ""
assignees: ""
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

View File

@@ -0,0 +1,20 @@
name: "Setup Pixi"
description: "setup pixi"
inputs:
environments:
description: "The pixi environments to install (e.g. default, docs, test)"
required: false
default: "default"
runs:
using: "composite"
steps:
- name: Setup Pixi
uses: prefix-dev/setup-pixi@v0.9.3
with:
pixi-version: v0.67.0
environments: ${{ inputs.environments }}
activate-environment: true
cache: true
locked: true

View File

@@ -0,0 +1,44 @@
name: benchmark
on:
workflow_dispatch:
jobs:
benchmark:
strategy:
fail-fast: false
matrix:
os: [ubuntu-24.04, macos-15, windows-2025]
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
- name: Build scan_benchmark
run: |
pixi run cmake-config RelWithDebInfo ON -- -DCLICE_ENABLE_BENCHMARK=ON
cmake --build build/RelWithDebInfo --target scan_benchmark
- name: Clone LLVM
run: git clone --depth 1 https://github.com/llvm/llvm-project.git
- name: Generate CDB
run: |
cmake -B llvm-build -G Ninja \
-DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
-DCMAKE_TOOLCHAIN_FILE="$(pwd)/cmake/toolchain.cmake" \
-DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;lldb;mlir;polly;flang;bolt" \
-DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi;libunwind" \
llvm-project/llvm
- name: Run benchmark
run: ./build/RelWithDebInfo/bin/scan_benchmark --runs 20 llvm-build/compile_commands.json
- name: Stop sccache server
if: runner.os == 'Windows'
run: pixi run -- sccache --stop-server || true

View File

@@ -0,0 +1,446 @@
name: build llvm
on:
workflow_dispatch:
inputs:
llvm_version:
description: "LLVM version to build (e.g., 21.1.8)"
required: true
type: string
skip_upload:
description: "Skip upload and PR creation (build-only mode)"
required: false
type: boolean
default: false
skip_pr:
description: "Skip PR creation (upload only, no PR)"
required: false
type: boolean
default: false
pull_request:
# if you want to run this workflow, change the branch name to main,
# if you want to turn off it, change it to non existent branch.
branches: [main-turn-off]
jobs:
build:
strategy:
fail-fast: false
matrix:
include:
# Native builds
- os: windows-2025
llvm_mode: RelWithDebInfo
lto: OFF
- os: windows-2025
llvm_mode: RelWithDebInfo
lto: ON
- os: ubuntu-24.04
llvm_mode: Debug
lto: OFF
- os: ubuntu-24.04
llvm_mode: RelWithDebInfo
lto: OFF
- os: ubuntu-24.04
llvm_mode: RelWithDebInfo
lto: ON
- os: macos-15
llvm_mode: Debug
lto: OFF
- os: macos-15
llvm_mode: RelWithDebInfo
lto: OFF
- os: macos-15
llvm_mode: RelWithDebInfo
lto: ON
# Cross-compilation builds
# macOS x64 (from arm64 macos-15)
- os: macos-15
llvm_mode: RelWithDebInfo
lto: OFF
target_triple: x86_64-apple-darwin
- os: macos-15
llvm_mode: RelWithDebInfo
lto: ON
target_triple: x86_64-apple-darwin
# Linux aarch64 (from x64 ubuntu-24.04)
- os: ubuntu-24.04
llvm_mode: RelWithDebInfo
lto: OFF
target_triple: aarch64-linux-gnu
pixi_env: cross-linux-aarch64
- os: ubuntu-24.04
llvm_mode: RelWithDebInfo
lto: ON
target_triple: aarch64-linux-gnu
pixi_env: cross-linux-aarch64
# Windows arm64 (from x64 windows-2025)
- os: windows-2025
llvm_mode: RelWithDebInfo
lto: OFF
target_triple: aarch64-pc-windows-msvc
pixi_env: cross-windows-arm64
- os: windows-2025
llvm_mode: RelWithDebInfo
lto: ON
target_triple: aarch64-pc-windows-msvc
pixi_env: cross-windows-arm64
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Free Disk Space
if: runner.os == 'Linux'
uses: jlumbroso/free-disk-space@main
- name: Increase Swap Space
if: runner.os == 'Linux'
run: |
echo "===== Initial Status ====="
sudo swapon --show
free -h
echo "===== Creating Swap File ====="
sudo swapoff -a
sudo fallocate -l 16G /mnt/swapfile
sudo chmod 600 /mnt/swapfile
sudo mkswap /mnt/swapfile
sudo swapon /mnt/swapfile
echo "===== Final Status ====="
sudo swapon --show
free -h
df -h
- uses: ./.github/actions/setup-pixi
with:
environments: ${{ matrix.pixi_env || 'package' }}
- name: Clone llvm-project
shell: bash
run: |
VERSION="${{ inputs.llvm_version || '21.1.8' }}"
echo "Cloning LLVM ${VERSION}..."
git clone --branch "llvmorg-${VERSION}" --depth 1 https://github.com/llvm/llvm-project.git .llvm
- name: Validate distribution components
shell: bash
run: |
python3 scripts/validate-llvm-components.py \
--llvm-src=.llvm \
--components-file=scripts/llvm-components.json
- name: Build LLVM (install-distribution)
shell: bash
run: |
ENV="${{ matrix.pixi_env || 'package' }}"
EXTRA_ARGS=""
if [[ -n "${{ matrix.target_triple }}" ]]; then
EXTRA_ARGS="--target-triple=${{ matrix.target_triple }}"
fi
pixi run -e "$ENV" build-llvm \
--llvm-src=.llvm \
--mode="${{ matrix.llvm_mode }}" \
--lto="${{ matrix.lto }}" \
--build-dir=build \
${EXTRA_ARGS}
- name: Build clice using installed LLVM
if: ${{ !matrix.target_triple }}
shell: bash
run: |
pixi run cmake-config ${{ matrix.llvm_mode }} ON -- \
"-DCLICE_ENABLE_LTO=${{ matrix.lto }}" \
"-DLLVM_INSTALL_PATH=.llvm/build-install"
pixi run cmake-build ${{ matrix.llvm_mode }}
- name: Build clice using installed LLVM (cross-compile)
if: ${{ matrix.target_triple }}
shell: bash
run: |
ENV="${{ matrix.pixi_env || 'package' }}"
pixi run -e "$ENV" cmake-config ${{ matrix.llvm_mode }} ON -- \
"-DCLICE_ENABLE_LTO=${{ matrix.lto }}" \
"-DCLICE_TARGET_TRIPLE=${{ matrix.target_triple }}" \
"-DLLVM_INSTALL_PATH=.llvm/build-install"
pixi run -e "$ENV" cmake-build ${{ matrix.llvm_mode }}
- name: Verify cross-compiled binary architecture
if: ${{ matrix.target_triple && runner.os != 'Windows' }}
shell: bash
run: |
BINARY="build/${{ matrix.llvm_mode }}/bin/clice"
echo "Binary info:"
file "$BINARY"
case "${{ matrix.target_triple }}" in
aarch64-linux-gnu) file "$BINARY" | grep -q "aarch64" ;;
x86_64-apple-darwin) file "$BINARY" | grep -q "x86_64" ;;
esac
- name: Upload cross-compiled clice for functional test
if: ${{ matrix.target_triple && matrix.lto == 'OFF' }}
uses: actions/upload-artifact@v4
with:
name: cross-clice-${{ matrix.target_triple }}-${{ matrix.llvm_mode }}
path: |
build/${{ matrix.llvm_mode }}/bin/
build/${{ matrix.llvm_mode }}/lib/
if-no-files-found: error
retention-days: 1
- name: Run tests
if: ${{ !matrix.target_triple }}
shell: bash
run: pixi run test ${{ matrix.llvm_mode }}
# Prune is only supported for native builds (requires linking clice to test).
# Cross-compiled targets reuse the native prune manifest of the same OS.
- name: Prune LLVM static libraries (Debug/RelWithDebInfo no LTO)
if: (!matrix.target_triple) && (matrix.llvm_mode == 'Debug' || (matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF'))
shell: bash
run: |
MANIFEST="pruned-libs-${{ matrix.os }}.json"
echo "LLVM_PRUNED_MANIFEST=${MANIFEST}" >> "${GITHUB_ENV}"
python3 scripts/prune-llvm-bin.py \
--action discover \
--install-dir ".llvm/build-install/lib" \
--build-dir "build/${{ matrix.llvm_mode }}" \
--max-attempts 60 \
--sleep-seconds 60 \
--manifest "${MANIFEST}"
- name: Upload pruned-libs manifest
if: (!matrix.target_triple) && matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF'
uses: actions/upload-artifact@v4
with:
name: llvm-pruned-libs-${{ matrix.os }}
path: ${{ env.LLVM_PRUNED_MANIFEST }}
if-no-files-found: error
compression-level: 0
- name: Apply pruned-libs manifest (RelWithDebInfo + LTO, native only)
if: (!matrix.target_triple) && matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'ON'
shell: bash
env:
GH_TOKEN: ${{ github.token }}
run: |
MANIFEST="pruned-libs-${{ matrix.os }}.json"
python3 scripts/prune-llvm-bin.py \
--action apply \
--manifest "${MANIFEST}" \
--install-dir ".llvm/build-install/lib" \
--build-dir "build/${{ matrix.llvm_mode }}" \
--gh-run-id "${{ github.run_id }}" \
--gh-artifact "llvm-pruned-libs-${{ matrix.os }}" \
--gh-download-dir "artifacts" \
--max-attempts 60 \
--sleep-seconds 60
# For cross-compiled LTO builds, apply the native prune manifest.
# The unused library set is arch-independent (same API surface).
- name: Apply pruned-libs manifest (cross-compile + LTO)
if: matrix.target_triple && matrix.lto == 'ON'
shell: bash
env:
GH_TOKEN: ${{ github.token }}
run: |
MANIFEST="pruned-libs-${{ matrix.os }}.json"
python3 scripts/prune-llvm-bin.py \
--action apply \
--manifest "${MANIFEST}" \
--install-dir ".llvm/build-install/lib" \
--build-dir "build/${{ matrix.llvm_mode }}" \
--gh-run-id "${{ github.run_id }}" \
--gh-artifact "llvm-pruned-libs-${{ matrix.os }}" \
--gh-download-dir "artifacts" \
--max-attempts 60 \
--sleep-seconds 60
- name: Package LLVM install directory
shell: bash
run: |
MODE_TAG="releasedbg"
if [[ "${{ matrix.llvm_mode }}" == "Debug" ]]; then
MODE_TAG="debug"
fi
# Determine arch/platform/toolchain from target triple or runner OS
if [[ -n "${{ matrix.target_triple }}" ]]; then
case "${{ matrix.target_triple }}" in
x86_64-apple-darwin)
ARCH="x64"; PLATFORM="macos"; TOOLCHAIN="clang" ;;
aarch64-linux-gnu)
ARCH="aarch64"; PLATFORM="linux"; TOOLCHAIN="gnu" ;;
aarch64-pc-windows-msvc)
ARCH="aarch64"; PLATFORM="windows"; TOOLCHAIN="msvc" ;;
esac
else
ARCH="x64"
PLATFORM="linux"
TOOLCHAIN="gnu"
if [[ "${{ matrix.os }}" == windows-* ]]; then
PLATFORM="windows"
TOOLCHAIN="msvc"
elif [[ "${{ matrix.os }}" == macos-* ]]; then
ARCH="arm64"
PLATFORM="macos"
TOOLCHAIN="clang"
fi
fi
SUFFIX=""
if [[ "${{ matrix.lto }}" == "ON" ]]; then
SUFFIX="-lto"
fi
if [[ "${{ matrix.llvm_mode }}" == "Debug" && "${{ matrix.os }}" != windows-* ]]; then
SUFFIX="${SUFFIX}-asan"
fi
ARCHIVE="${ARCH}-${PLATFORM}-${TOOLCHAIN}-${MODE_TAG}${SUFFIX}.tar.xz"
set -eo pipefail
tar -C .llvm -cf - build-install | xz -T0 -9 -c > "${ARCHIVE}"
echo "LLVM_INSTALL_ARCHIVE=${ARCHIVE}" >> "${GITHUB_ENV}"
- name: Upload LLVM install artifact
uses: actions/upload-artifact@v4
with:
name: ${{ env.LLVM_INSTALL_ARCHIVE }}
path: ${{ env.LLVM_INSTALL_ARCHIVE }}
if-no-files-found: error
test-cross:
needs: build
strategy:
fail-fast: false
matrix:
include:
- os: macos-15-intel
llvm_mode: RelWithDebInfo
target_triple: x86_64-apple-darwin
- os: ubuntu-24.04-arm
llvm_mode: RelWithDebInfo
target_triple: aarch64-linux-gnu
- os: windows-11-arm
llvm_mode: RelWithDebInfo
target_triple: aarch64-pc-windows-msvc
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: test-run
- name: Download cross-compiled clice
uses: actions/download-artifact@v4
with:
name: cross-clice-${{ matrix.target_triple }}-${{ matrix.llvm_mode }}
path: build/${{ matrix.llvm_mode }}/
- name: Make binaries executable
if: runner.os != 'Windows'
run: chmod +x build/${{ matrix.llvm_mode }}/bin/*
- name: Run tests
run: pixi run -e test-run test ${{ matrix.llvm_mode }}
upload:
needs: build
if: ${{ !cancelled() && inputs.llvm_version && !inputs.skip_upload }}
runs-on: ubuntu-24.04
permissions:
contents: read
steps:
- uses: actions/checkout@v4
- name: Download all build artifacts
env:
GH_TOKEN: ${{ github.token }}
run: scripts/download-llvm.sh "${{ github.run_id }}"
- name: Upload to clice-llvm
env:
GH_TOKEN: ${{ secrets.UPLOAD_LLVM }}
TARGET_REPO: clice-io/clice-llvm
run: python3 scripts/upload-llvm.py "${{ inputs.llvm_version }}" "${TARGET_REPO}" "${{ github.run_id }}"
- name: Save manifest for update-clice job
uses: actions/upload-artifact@v4
with:
name: llvm-manifest-final
path: artifacts/llvm-manifest.json
if-no-files-found: error
compression-level: 0
update-clice:
needs: upload
if: ${{ !inputs.skip_pr }}
runs-on: ubuntu-24.04
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v4
- name: Download manifest
uses: actions/download-artifact@v4
with:
name: llvm-manifest-final
path: .
- name: Update manifest and version
run: |
python3 scripts/update-llvm-version.py \
--version "${{ inputs.llvm_version }}" \
--manifest-src llvm-manifest.json \
--manifest-dest config/llvm-manifest.json \
--package-cmake cmake/package.cmake
- name: Create or update PR
env:
GH_TOKEN: ${{ github.token }}
run: |
VERSION="${{ inputs.llvm_version }}"
BRANCH="chore/update-llvm-${VERSION}"
RUN_URL="https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}"
RELEASE_URL="https://github.com/clice-io/clice-llvm/releases/tag/${VERSION}"
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git checkout -b "${BRANCH}"
git add config/llvm-manifest.json cmake/package.cmake
git commit -m "chore: update LLVM to ${VERSION}"
git push --force-with-lease origin "${BRANCH}"
# Check if PR already exists for this branch
EXISTING_PR=$(gh pr list --head "${BRANCH}" --json number --jq '.[0].number // empty')
BODY="$(cat <<EOF
## Summary
- Update LLVM prebuilt binaries to version ${VERSION}
- Updated \`config/llvm-manifest.json\` with new SHA256 hashes
- Updated \`cmake/package.cmake\` version string
**Artifacts:** [clice-llvm release](${RELEASE_URL})
**Build:** [workflow run](${RUN_URL})
> Auto-generated by build-llvm workflow
EOF
)"
if [[ -n "${EXISTING_PR}" ]]; then
echo "Updating existing PR #${EXISTING_PR}"
gh pr edit "${EXISTING_PR}" --body "${BODY}"
else
gh pr create \
--title "chore: update LLVM to ${VERSION}" \
--body "${BODY}" \
--base main
fi

View File

@@ -0,0 +1,40 @@
name: format
on:
workflow_call:
jobs:
check-format:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: format
- name: Validate update-llvm-version.py can still patch package.cmake
run: |
python3 scripts/update-llvm-version.py --check \
--manifest-dest config/llvm-manifest.json \
--package-cmake cmake/package.cmake
- name: Run formatter
run: pixi run format
continue-on-error: true
- name: Auto correct
uses: huacnlee/autocorrect-action@v2
with:
args: --lint ./docs
continue-on-error: true
- name: Check diff
run: |
if ! git diff --quiet; then
echo "::error::Formatting changes detected. Please run 'pixi run format' and commit the result."
git --no-pager diff --stat
git --no-pager diff
exit 1
fi

View File

@@ -0,0 +1,30 @@
name: deploy
on:
workflow_call:
jobs:
deploy-docs:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: node
- name: Build docs
run: pixi run build-docs
- name: Deploy to GitHub Pages
uses: peaceiris/actions-gh-pages@v4
if: github.ref == 'refs/heads/main'
with:
personal_token: ${{ secrets.PUBLISH_DOCS }}
external_repository: clice-io/docs
publish_dir: ./docs/.vitepress/dist
destination_dir: clice
keep_files: true

137
.gitea/workflows/main.yml Normal file
View File

@@ -0,0 +1,137 @@
name: main
on:
push:
branches: [main]
tags: ["v*"]
pull_request:
branches: [main]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
jobs:
changes:
if: ${{ !startsWith(github.ref, 'refs/tags/') }}
runs-on: ubuntu-latest
permissions:
pull-requests: read
outputs:
format: ${{ steps.filter.outputs.format }}
docs: ${{ steps.filter.outputs.docs }}
clice: ${{ steps.filter.outputs.clice }}
vscode: ${{ steps.filter.outputs.vscode }}
cmake: ${{ steps.filter.outputs.cmake }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
id: filter
with:
filters: |
format:
- '**/*.{h,c,cpp,hpp,ts,js,lua,md,yml,yaml}'
docs:
- 'docs/**'
- '.github/workflows/deploy-docs.yml'
clice:
- 'src/**'
- 'include/**'
- 'CMakeLists.txt'
- '.github/workflows/publish-clice.yml'
vscode:
- 'editors/vscode/**'
- '.github/workflows/publish-vscode.yml'
cmake:
- 'CMakeLists.txt'
- 'src/**'
- 'include/**'
- 'tests/**'
- 'config/**'
- '.github/workflows/test-cmake.yml'
conventional-commit:
if: ${{ !startsWith(github.ref, 'refs/tags/') }}
runs-on: ubuntu-latest
steps:
- name: Check conventional commit format
env:
IS_PR: ${{ github.event_name == 'pull_request' }}
PR_TITLE: ${{ github.event.pull_request.title }}
COMMIT_MSG: ${{ github.event.head_commit.message }}
run: |
pattern='^(feat|fix|refactor|chore|build|ci|docs|test|perf|style|revert)(\(.+\))?: .+'
if [[ "$IS_PR" == "true" ]]; then
subject="$PR_TITLE"
label="PR title"
else
subject=$(echo "$COMMIT_MSG" | head -n1)
label="Commit message"
fi
if [[ ! "$subject" =~ $pattern ]]; then
echo "::error::$label must follow conventional commit format: type(scope)?: description"
echo " Valid types: feat, fix, refactor, chore, build, ci, docs, test, perf, style, revert"
echo " Got: '$subject'"
exit 1
fi
format:
needs: changes
if: ${{ needs.changes.outputs.format == 'true' }}
uses: ./.github/workflows/check-format.yml
deploy:
needs: changes
if: ${{ needs.changes.outputs.docs == 'true' }}
permissions:
contents: write
uses: ./.github/workflows/deploy-docs.yml
secrets: inherit
# clice:
# needs: changes
# if: ${{ needs.changes.outputs.clice == 'true' }}
# uses: ./.github/workflows/publish-clice.yml
vscode:
needs: changes
if: ${{ needs.changes.outputs.vscode == 'true' }}
uses: ./.github/workflows/publish-vscode.yml
cmake:
needs: changes
if: ${{ needs.changes.outputs.cmake == 'true' }}
uses: ./.github/workflows/test-cmake.yml
release-clice:
permissions:
contents: write
if: startsWith(github.ref, 'refs/tags/v')
uses: ./.github/workflows/publish-clice.yml
secrets: inherit
release-vscode:
permissions:
contents: write
if: startsWith(github.ref, 'refs/tags/v')
uses: ./.github/workflows/publish-vscode.yml
secrets: inherit
checks-passed:
if: ${{ always() && !startsWith(github.ref, 'refs/tags/') }}
needs:
- conventional-commit
- format
- deploy
# - clice
- vscode
- cmake
runs-on: ubuntu-latest
steps:
- name: Check results
uses: re-actors/alls-green@release/v1
with:
allowed-skips: conventional-commit,format,deploy,clice,vscode,cmake
jobs: ${{ toJSON(needs) }}

View File

@@ -0,0 +1,100 @@
name: clice
on:
workflow_call:
jobs:
publish-clice:
strategy:
fail-fast: false
matrix:
include:
# Native builds
- os: windows-2025
artifact_name: clice.zip
asset_name: clice-x64-windows-msvc.zip
symbol_artifact_name: clice-symbol.zip
symbol_asset_name: clice-x64-windows-msvc-symbol.zip
- os: ubuntu-24.04
artifact_name: clice.tar.gz
asset_name: clice-x86_64-linux-gnu.tar.gz
symbol_artifact_name: clice-symbol.tar.gz
symbol_asset_name: clice-x86_64-linux-gnu-symbol.tar.gz
- os: macos-15
artifact_name: clice.tar.gz
asset_name: clice-arm64-macos-darwin.tar.gz
symbol_artifact_name: clice-symbol.tar.gz
symbol_asset_name: clice-arm64-macos-darwin-symbol.tar.gz
# Cross-compilation builds
- os: macos-15
target_triple: x86_64-apple-darwin
pixi_env: cross-macos-x64
artifact_name: clice.tar.gz
asset_name: clice-x86_64-macos-darwin.tar.gz
symbol_artifact_name: clice-symbol.tar.gz
symbol_asset_name: clice-x86_64-macos-darwin-symbol.tar.gz
- os: ubuntu-24.04
target_triple: aarch64-linux-gnu
pixi_env: cross-linux-aarch64
artifact_name: clice.tar.gz
asset_name: clice-aarch64-linux-gnu.tar.gz
symbol_artifact_name: clice-symbol.tar.gz
symbol_asset_name: clice-aarch64-linux-gnu-symbol.tar.gz
- os: windows-2025
target_triple: aarch64-pc-windows-msvc
pixi_env: cross-windows-arm64
artifact_name: clice.zip
asset_name: clice-aarch64-windows-msvc.zip
symbol_artifact_name: clice-symbol.zip
symbol_asset_name: clice-aarch64-windows-msvc-symbol.zip
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: ${{ matrix.pixi_env || 'package' }}
- name: Package (native)
if: ${{ !matrix.target_triple }}
run: pixi run package
- name: Package (cross-compile)
if: ${{ matrix.target_triple }}
run: |
ENV="${{ matrix.pixi_env }}"
pixi run -e "$ENV" package-config -- \
"-DCLICE_TARGET_TRIPLE=${{ matrix.target_triple }}"
pixi run -e "$ENV" cmake-build
- name: Upload Main Package to Release
if: startsWith(github.ref, 'refs/tags/v')
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: build/RelWithDebInfo/${{ matrix.artifact_name }}
asset_name: ${{ matrix.asset_name }}
tag: ${{ github.ref }}
overwrite: true
- name: Upload Symbol Package to Release
if: startsWith(github.ref, 'refs/tags/v')
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: build/RelWithDebInfo/${{ matrix.symbol_artifact_name }}
asset_name: ${{ matrix.symbol_asset_name }}
tag: ${{ github.ref }}
overwrite: true

View File

@@ -0,0 +1,38 @@
name: vscode
on:
workflow_call:
jobs:
publish-vscode:
runs-on: ubuntu-latest
defaults:
run:
working-directory: editors/vscode
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: node
- name: Publish and Package to Marketplace
env:
VSCE_PAT: ${{ secrets.VSCE_PAT }}
run: |
FLAG="${{ contains(github.ref_name, '-') && '--pre-release' || '' }}"
pixi run build-vscode $FLAG
if [[ "${{ github.ref }}" == "refs/tags/"* ]]; then
pixi run publish-vscode -p "$VSCE_PAT" $FLAG
fi
- name: Upload .vsix to Release
uses: softprops/action-gh-release@v2
if: startsWith(github.ref, 'refs/tags/v')
with:
files: "editors/vscode/*.vsix"
tag_name: ${{ github.ref }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -0,0 +1,170 @@
name: cmake
on:
workflow_call:
env:
CCACHE_DIR: ${{ github.workspace }}/.cache/ccache
SCCACHE_DIR: ${{ github.workspace }}/.cache/sccache
CCACHE_BASEDIR: ${{ github.workspace }}
SCCACHE_BASEDIRS: ${{ github.workspace }}
CCACHE_COMPILERCHECK: content
CCACHE_MAXSIZE: 2G
SCCACHE_CACHE_SIZE: 2G
jobs:
build:
strategy:
fail-fast: false
matrix:
include:
# Native builds
- os: windows-2025
build_type: RelWithDebInfo
- os: ubuntu-24.04
build_type: Debug
- os: ubuntu-24.04
build_type: RelWithDebInfo
- os: macos-15
build_type: Debug
- os: macos-15
build_type: RelWithDebInfo
# Cross-compile (build only; tests run on native runners)
- os: macos-15
build_type: RelWithDebInfo
target_triple: x86_64-apple-darwin
build_only: true
- os: ubuntu-24.04
build_type: RelWithDebInfo
target_triple: aarch64-linux-gnu
build_only: true
pixi_env: cross-linux-aarch64
- os: windows-2025
build_type: RelWithDebInfo
target_triple: aarch64-pc-windows-msvc
build_only: true
pixi_env: cross-windows-arm64
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: ${{ matrix.pixi_env || 'default' }}
- name: Restore compiler cache
uses: actions/cache@v4
with:
path: ${{ runner.os == 'Windows' && '.cache/sccache' || '.cache/ccache' }}
key: ${{ runner.os }}-${{ matrix.build_type }}-${{ matrix.target_triple || 'native' }}-ccache-${{ github.sha }}
restore-keys: |
${{ runner.os }}-${{ matrix.build_type }}-${{ matrix.target_triple || 'native' }}-ccache-
- name: Zero cache stats
run: |
ENV="${{ matrix.pixi_env || 'default' }}"
if [ "$RUNNER_OS" = "Windows" ]; then
pixi run -e "$ENV" -- sccache --stop-server || true
pixi run -e "$ENV" -- sccache --zero-stats || true
else
pixi run -e "$ENV" -- ccache --zero-stats || true
fi
shell: bash
- name: Build (native)
if: ${{ !matrix.target_triple }}
run: pixi run build ${{ matrix.build_type }} ON
- name: Build (cross-compile)
if: ${{ matrix.target_triple }}
shell: bash
run: |
ENV="${{ matrix.pixi_env || 'default' }}"
pixi run -e "$ENV" cmake-config ${{ matrix.build_type }} OFF -- \
"-DCLICE_TARGET_TRIPLE=${{ matrix.target_triple }}"
pixi run -e "$ENV" cmake-build ${{ matrix.build_type }}
- name: Upload cross-compiled binaries
if: ${{ matrix.build_only }}
uses: actions/upload-artifact@v4
with:
name: cross-build-${{ matrix.target_triple }}
path: |
build/${{ matrix.build_type }}/bin/
build/${{ matrix.build_type }}/lib/
if-no-files-found: error
retention-days: 1
- name: Unit tests
if: ${{ !matrix.build_only }}
timeout-minutes: 5
run: pixi run unit-test ${{ matrix.build_type }}
- name: Integration tests
if: ${{ !matrix.build_only }}
timeout-minutes: 20
run: pixi run integration-test ${{ matrix.build_type }}
- name: Smoke tests
if: ${{ !matrix.build_only }}
timeout-minutes: 15
run: pixi run smoke-test ${{ matrix.build_type }}
- name: Print cache stats and stop server
if: always()
run: |
ENV="${{ matrix.pixi_env || 'default' }}"
if [ "$RUNNER_OS" = "Windows" ]; then
pixi run -e "$ENV" -- sccache --show-stats
pixi run -e "$ENV" -- sccache --stop-server || true
else
pixi run -e "$ENV" -- ccache --show-stats
fi
shell: bash
test-cross:
needs: build
strategy:
fail-fast: false
matrix:
include:
- os: macos-15-intel
build_type: RelWithDebInfo
target_triple: x86_64-apple-darwin
- os: ubuntu-24.04-arm
build_type: RelWithDebInfo
target_triple: aarch64-linux-gnu
- os: windows-11-arm
build_type: RelWithDebInfo
target_triple: aarch64-pc-windows-msvc
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: test-run
- name: Download cross-compiled binaries
uses: actions/download-artifact@v4
with:
name: cross-build-${{ matrix.target_triple }}
path: build/${{ matrix.build_type }}/
- name: Make binaries executable
if: runner.os != 'Windows'
run: chmod +x build/${{ matrix.build_type }}/bin/*
- name: Unit tests
timeout-minutes: 5
run: pixi run -e test-run unit-test ${{ matrix.build_type }}
- name: Integration tests
timeout-minutes: 20
run: pixi run -e test-run integration-test ${{ matrix.build_type }}
- name: Smoke tests
timeout-minutes: 10
run: pixi run -e test-run smoke-test ${{ matrix.build_type }}

View File

@@ -0,0 +1,38 @@
name: upload-llvm
permissions:
contents: write
on:
pull_request:
# if you want to run this workflow, change the branch name to main,
# if you want to turn off it, change it to non existent branch.
branches: [main-turn-off]
workflow_dispatch:
inputs:
workflow_id:
description: "Workflow run ID to pull artifacts from"
required: true
type: string
version:
description: "Release version/tag to publish (e.g., v1.2.3)"
required: true
type: string
jobs:
upload:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- name: Download artifacts from workflow
env:
GH_TOKEN: ${{ github.token }}
run: scripts/download-llvm.sh "${{ inputs.workflow_id }}"
- name: Recreate release with artifacts
env:
GH_TOKEN: ${{ secrets.UPLOAD_LLVM }}
TARGET_REPO: clice-io/clice-llvm
run: python3 scripts/upload-llvm.py "${{ inputs.version }}" "${TARGET_REPO}" "${{ inputs.workflow_id }}"

View File

@@ -7,6 +7,10 @@ on:
pull_request:
branches: [main]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
jobs:
changes:
if: ${{ !startsWith(github.ref, 'refs/tags/') }}

View File

@@ -96,9 +96,20 @@ jobs:
if-no-files-found: error
retention-days: 1
- name: Run tests
- name: Unit tests
if: ${{ !matrix.build_only }}
run: pixi run test ${{ matrix.build_type }}
timeout-minutes: 5
run: pixi run unit-test ${{ matrix.build_type }}
- name: Integration tests
if: ${{ !matrix.build_only }}
timeout-minutes: 20
run: pixi run integration-test ${{ matrix.build_type }}
- name: Smoke tests
if: ${{ !matrix.build_only }}
timeout-minutes: 15
run: pixi run smoke-test ${{ matrix.build_type }}
- name: Print cache stats and stop server
if: always()
@@ -146,5 +157,14 @@ jobs:
if: runner.os != 'Windows'
run: chmod +x build/${{ matrix.build_type }}/bin/*
- name: Run tests
run: pixi run -e test-run test ${{ matrix.build_type }}
- name: Unit tests
timeout-minutes: 5
run: pixi run -e test-run unit-test ${{ matrix.build_type }}
- name: Integration tests
timeout-minutes: 20
run: pixi run -e test-run integration-test ${{ matrix.build_type }}
- name: Smoke tests
timeout-minutes: 10
run: pixi run -e test-run smoke-test ${{ matrix.build_type }}

3
.gitignore vendored
View File

@@ -68,8 +68,7 @@ tests/unit/Local/
.pixi/*
!.pixi/config.toml
.codex/
.codex
.claude/*
!.claude/CLAUDE.md
!.claude/commands/
openspec/

View File

@@ -124,21 +124,42 @@ if(CLICE_CI_ENVIRONMENT)
target_compile_definitions(clice_options INTERFACE CLICE_CI_ENVIRONMENT=1)
endif()
set(FBS_SCHEMA_FILE "${PROJECT_SOURCE_DIR}/src/index/schema.fbs")
set(GENERATED_HEADER "${PROJECT_BINARY_DIR}/generated/schema_generated.h")
if(CMAKE_CROSSCOMPILING)
find_program(FLATC_EXECUTABLE flatc REQUIRED)
set(FLATC_CMD "${FLATC_EXECUTABLE}")
else()
set(FLATC_CMD "$<TARGET_FILE:flatc>")
endif()
add_custom_command(
OUTPUT "${GENERATED_HEADER}"
COMMAND ${FLATC_CMD} --cpp -o "${PROJECT_BINARY_DIR}/generated" "${FBS_SCHEMA_FILE}"
DEPENDS "${FBS_SCHEMA_FILE}"
COMMENT "Generating C++ header from ${FBS_SCHEMA_FILE}"
)
add_custom_target(generate_flatbuffers_schema DEPENDS "${GENERATED_HEADER}")
file(GLOB_RECURSE CLICE_CORE_SOURCES CONFIGURE_DEPENDS "${PROJECT_SOURCE_DIR}/src/*.cpp")
add_library(clice-core STATIC ${CLICE_CORE_SOURCES})
add_library(clice::core ALIAS clice-core)
add_dependencies(clice-core generate_flatbuffers_schema)
target_include_directories(clice-core PUBLIC
"${PROJECT_SOURCE_DIR}/src"
"${PROJECT_BINARY_DIR}/generated"
)
target_link_libraries(clice-core PUBLIC
clice_options
llvm-libs
spdlog::spdlog
roaring::roaring
flatbuffers
kota::ipc::lsp
kota::codec::toml
kota::codec::flatbuffers
simdjson::simdjson
)

View File

@@ -26,7 +26,7 @@
#include "support/path_pool.h"
#include "syntax/dependency_graph.h"
#include "kota/codec/json/serializer.h"
#include "kota/codec/json/json.h"
#include "kota/deco/deco.h"
#include "llvm/Support/FileSystem.h"

View File

@@ -27,11 +27,21 @@ FetchContent_Declare(
set(ENABLE_ROARING_TESTS OFF CACHE INTERNAL "" FORCE)
set(ENABLE_ROARING_MICROBENCHMARKS OFF CACHE INTERNAL "" FORCE)
# flatbuffers
FetchContent_Declare(
flatbuffers
GIT_REPOSITORY https://github.com/google/flatbuffers.git
GIT_TAG v25.9.23
GIT_SHALLOW TRUE
)
set(FLATBUFFERS_BUILD_GRPC OFF CACHE BOOL "" FORCE)
set(FLATBUFFERS_BUILD_TESTS OFF CACHE BOOL "" FORCE)
set(FLATBUFFERS_BUILD_FLATHASH OFF CACHE BOOL "" FORCE)
FetchContent_Declare(
kotatsu
GIT_REPOSITORY https://github.com/clice-io/kotatsu
GIT_TAG refactor/flatbuffers-schema-driven
GIT_SHALLOW TRUE
GIT_TAG 73814044ce8142f4438a3028f44668675fc09fff
)
set(KOTA_ENABLE_ZEST ON)
@@ -39,8 +49,7 @@ set(KOTA_ENABLE_TEST OFF)
set(KOTA_CODEC_ENABLE_SIMDJSON ON)
set(KOTA_CODEC_ENABLE_YYJSON ON)
set(KOTA_CODEC_ENABLE_TOML ON)
set(KOTA_CODEC_ENABLE_FLATBUFFERS ON)
set(KOTA_ENABLE_EXCEPTIONS OFF)
set(KOTA_ENABLE_RTTI OFF)
FetchContent_MakeAvailable(kotatsu spdlog croaring)
FetchContent_MakeAvailable(kotatsu spdlog croaring flatbuffers)

View File

@@ -153,7 +153,7 @@ String values support `${workspace}` substitution.
## IPC Protocol
The master and workers communicate using custom RPC messages defined in `src/server/protocol.h`. Each message type has a `RequestTraits` or `NotificationTraits` specialization that defines the method name and result type.
The master and workers communicate using custom RPC messages defined in `src/server/protocol/`. Each message type has a `RequestTraits` or `NotificationTraits` specialization that defines the method name and result type.
### Stateful Worker Messages

23
pixi.lock generated
View File

@@ -1078,6 +1078,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
linux-aarch64:
- conda: https://conda.anaconda.org/conda-forge/linux-aarch64/_openmp_mutex-4.5-20_gnu.conda
@@ -1152,6 +1153,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-64:
- conda: https://conda.anaconda.org/conda-forge/osx-64/bzip2-1.0.8-h500dc9f_9.conda
@@ -1224,6 +1226,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-arm64:
- conda: https://conda.anaconda.org/conda-forge/osx-arm64/bzip2-1.0.8-hd037594_8.conda
@@ -1289,6 +1292,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
win-64:
- conda: https://conda.anaconda.org/conda-forge/win-64/bzip2-1.0.8-h0ad9c76_8.conda
@@ -1343,6 +1347,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
format:
channels:
@@ -1704,6 +1709,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
linux-aarch64:
- conda: https://conda.anaconda.org/conda-forge/linux-aarch64/_openmp_mutex-4.5-20_gnu.conda
@@ -1782,6 +1788,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-64:
- conda: https://conda.anaconda.org/conda-forge/osx-64/bzip2-1.0.8-h500dc9f_9.conda
@@ -1858,6 +1865,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-arm64:
- conda: https://conda.anaconda.org/conda-forge/osx-arm64/bzip2-1.0.8-hd037594_8.conda
@@ -1926,6 +1934,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
win-64:
- conda: https://conda.anaconda.org/conda-forge/win-64/bzip2-1.0.8-h0ad9c76_8.conda
@@ -1982,6 +1991,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
test-run:
channels:
@@ -2025,6 +2035,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
linux-aarch64:
- conda: https://conda.anaconda.org/conda-forge/linux-aarch64/_openmp_mutex-4.5-20_gnu.conda
@@ -2058,6 +2069,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-64:
- conda: https://conda.anaconda.org/conda-forge/osx-64/bzip2-1.0.8-h500dc9f_9.conda
@@ -2113,6 +2125,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-arm64:
- conda: https://conda.anaconda.org/conda-forge/osx-arm64/bzip2-1.0.8-hd037594_9.conda
@@ -2168,6 +2181,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
win-64:
- conda: https://conda.anaconda.org/conda-forge/win-64/bzip2-1.0.8-h0ad9c76_9.conda
@@ -2199,6 +2213,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
win-arm64:
- conda: https://conda.anaconda.org/conda-forge/win-arm64/bzip2-1.0.8-h50b96f5_9.conda
@@ -2229,6 +2244,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
packages:
- conda: https://conda.anaconda.org/conda-forge/linux-64/_libgcc_mutex-0.1-conda_forge.tar.bz2
@@ -7795,6 +7811,13 @@ packages:
- coverage>=6.2 ; extra == 'testing'
- hypothesis>=5.7.1 ; extra == 'testing'
requires_python: '>=3.10'
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
name: pytest-timeout
version: 2.4.0
sha256: c42667e5cdadb151aeb5b26d114aff6bdf5a907f176a007a30b940d3d865b5c2
requires_dist:
- pytest>=7.0.0
requires_python: '>=3.7'
- conda: https://conda.anaconda.org/conda-forge/linux-64/python-3.14.2-h32b2ec7_100_cp314.conda
build_number: 100
sha256: a120fb2da4e4d51dd32918c149b04a08815fd2bd52099dad1334647984bb07f1

View File

@@ -102,6 +102,7 @@ lld = "==20.1.8"
[feature.test.pypi-dependencies]
pytest = "*"
pytest-asyncio = ">=1.1.0"
pytest-timeout = "*"
pygls = ">=2.0.0"
lsprotocol = ">=2024.0.0"
@@ -160,13 +161,13 @@ depends-on = [{ task = "lint-cpp", args = ["{{ type }}"] }]
[feature.test.tasks.unit-test]
args = [{ arg = "type", default = "RelWithDebInfo" }]
cmd = './build/{{ type }}/bin/unit_tests --test-dir="./tests/data"'
cmd = './build/{{ type }}/bin/unit_tests --test-dir="./tests/data" --snapshot-dir="./tests/snapshots" --corpus-dir="./tests/corpus" --verbose'
[feature.test.tasks.integration-test]
args = [{ arg = "type", default = "RelWithDebInfo" }]
cmd = """
pytest -s --log-cli-level=INFO tests/integration \
--executable=./build/{{ type }}/bin/clice
pytest -s --log-cli-level=INFO --timeout=300 --timeout-method=thread \
tests/integration --executable=./build/{{ type }}/bin/clice
"""
[feature.test.tasks.smoke-test]
@@ -256,7 +257,7 @@ format-markdown = "fd -H -e md -x prettier --write"
format-json = "fd -H -e json -E package-lock.json -x prettier --write"
format-toml = "fd -H -e toml -x tombi format"
format-yaml = """
fd -H -e yaml -e yml -E pnpm-lock.yaml -x prettier --write && \
fd -H -e yaml -e yml -E pnpm-lock.yaml -E '*.snap.yml' -x prettier --write && \
fd -H "^\\.clang-(format|tidy)$" -x prettier --write --parser yaml
"""
format = { depends-on = [

View File

@@ -4,33 +4,33 @@
#include <print>
#include <string>
#include "server/master_server.h"
#include "server/stateful_worker.h"
#include "server/stateless_worker.h"
#include "server/service/agentic.h"
#include "server/service/master_server.h"
#include "server/worker/stateful_worker.h"
#include "server/worker/stateless_worker.h"
#include "support/logging.h"
#include "kota/async/async.h"
#include "kota/deco/deco.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/peer.h"
#include "kota/ipc/recording_transport.h"
#include "kota/ipc/transport.h"
namespace clice {
using kota::deco::decl::KVStyle;
struct Options {
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Running mode: pipe, socket, stateless-worker, stateful-worker",
required = false)
DecoKV(
style = KVStyle::JoinedOrSeparate,
help =
"Running mode: pipe, socket, daemon, relay, agentic, stateless-worker, stateful-worker",
required = false)
<std::string> mode;
DecoKV(style = KVStyle::JoinedOrSeparate, help = "Socket mode address", required = false)
<std::string> host = "127.0.0.1";
DecoKV(style = KVStyle::JoinedOrSeparate, help = "Socket mode port", required = false)
<int> port = 50051;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Agentic TCP port (0 = disabled)",
required = false)
<int> port = 0;
DecoKV(style = KVStyle::JoinedOrSeparate,
names = {"--log-level", "--log-level="},
@@ -43,6 +43,50 @@ struct Options {
required = false)
<std::string> record;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "File path for agentic queries",
required = false)
<std::string> path;
DecoKV(
style = KVStyle::JoinedOrSeparate,
help =
"Agentic method (compileCommand, symbolSearch, definition, references, "
"documentSymbols, readSymbol, callGraph, typeHierarchy, projectFiles, "
"fileDeps, impactAnalysis, status, shutdown)",
required = false)
<std::string> method;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Symbol name for agentic queries",
required = false)
<std::string> name;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Search query for symbolSearch",
required = false)
<std::string> query;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Line number for position-based lookup",
required = false)
<int> line;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Direction: callers/callees or supertypes/subtypes",
required = false)
<std::string> direction;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Unix domain socket path for daemon mode",
required = false)
<std::string> socket;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Workspace root directory for daemon mode",
required = false)
<std::string> workspace;
// Internal options (passed from master to worker processes)
DecoKV(style = KVStyle::JoinedOrSeparate,
names = {"--worker-memory-limit", "--worker-memory-limit="},
@@ -68,9 +112,6 @@ struct Options {
int main(int argc, const char** argv) {
#ifndef _WIN32
// On POSIX systems, ignore SIGPIPE so that writing to a closed pipe
// (e.g. when the LSP client disconnects) returns EPIPE instead of
// killing the process. This is standard practice for pipe-based servers.
signal(SIGPIPE, SIG_IGN);
#endif
@@ -110,8 +151,6 @@ int main(int argc, const char** argv) {
return 1;
}
std::string self_path = argv[0];
auto& mode = *opts.mode;
auto worker_name = opts.worker_name.value_or("");
@@ -131,77 +170,51 @@ int main(int argc, const char** argv) {
log_dir);
}
if(mode == "pipe") {
clice::logging::stderr_logger("master", clice::logging::options);
kota::event_loop loop;
auto transport = kota::ipc::StreamTransport::open_stdio(loop);
if(!transport) {
LOG_ERROR("failed to open stdio transport");
return 1;
}
std::unique_ptr<kota::ipc::Transport> final_transport = std::move(*transport);
if(opts.record.has_value()) {
final_transport =
std::make_unique<kota::ipc::RecordingTransport>(std::move(final_transport),
*opts.record);
}
kota::ipc::JsonPeer peer(loop, std::move(final_transport));
clice::MasterServer server(loop, peer, std::move(self_path));
server.register_handlers();
loop.schedule(peer.run());
loop.run();
return 0;
if(mode == "pipe" || mode == "socket") {
clice::ServerOptions server_opts;
server_opts.mode = mode;
server_opts.host = opts.host.value_or("127.0.0.1");
server_opts.port = opts.port.value_or(0);
server_opts.self_path = argv[0];
server_opts.record = opts.record.value_or("");
return clice::run_server_mode(server_opts);
}
if(mode == "socket") {
clice::logging::stderr_logger("master", clice::logging::options);
kota::event_loop loop;
auto host = opts.host.value_or("127.0.0.1");
auto port = opts.port.value_or(50051);
auto acceptor = kota::tcp::listen(host, port, {}, loop);
if(!acceptor) {
LOG_ERROR("failed to listen on {}:{}", host, port);
if(mode == "daemon") {
auto workspace = opts.workspace.value_or("");
if(workspace.empty()) {
LOG_ERROR("--workspace is required for daemon mode");
return 1;
}
LOG_INFO("Listening on {}:{} ...", host, port);
clice::DaemonOptions daemon_opts;
daemon_opts.socket_path = opts.socket.value_or("");
daemon_opts.workspace = std::move(workspace);
daemon_opts.self_path = argv[0];
return clice::run_daemon_mode(daemon_opts);
}
auto task = [&]() -> kota::task<> {
auto client = co_await acceptor->accept();
if(!client.has_value()) {
LOG_ERROR("failed to accept connection");
loop.stop();
co_return;
}
if(mode == "agentic") {
auto port = opts.port.value_or(0);
if(port <= 0) {
LOG_ERROR("--port is required for agentic mode");
return 1;
}
clice::AgenticQueryOptions aq;
aq.host = opts.host.value_or("127.0.0.1");
aq.port = port;
aq.method = opts.method.value_or("compileCommand");
aq.path = opts.path.value_or("");
aq.name = opts.name.value_or("");
aq.query = opts.query.value_or("");
aq.line = opts.line.value_or(0);
aq.direction = opts.direction.value_or("");
return clice::run_agentic_mode(aq);
}
LOG_INFO("Client connected");
std::unique_ptr<kota::ipc::Transport> transport =
std::make_unique<kota::ipc::StreamTransport>(std::move(client.value()));
if(opts.record.has_value()) {
transport = std::make_unique<kota::ipc::RecordingTransport>(std::move(transport),
*opts.record);
}
kota::ipc::JsonPeer peer(loop, std::move(transport));
clice::MasterServer server(loop, peer, std::string(self_path));
server.register_handlers();
co_await peer.run();
peer.close();
loop.stop();
};
loop.schedule(task());
loop.run();
return 0;
if(mode == "relay") {
auto socket = opts.socket.value_or("");
return clice::run_relay_mode(socket);
}
LOG_ERROR("unknown mode '{}'", mode);

View File

@@ -9,6 +9,7 @@
#include "support/logging.h"
#include "kota/meta/enum.h"
#include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/ScopeExit.h"
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/FileSystem.h"
@@ -17,6 +18,7 @@
#include "llvm/TargetParser/Host.h"
#include "clang/Driver/Compilation.h"
#include "clang/Driver/Driver.h"
#include "clang/Driver/Options.h"
#include "clang/Driver/Tool.h"
#ifndef _WIN32
@@ -470,11 +472,32 @@ std::vector<const char*> query_clang_toolchain(const QueryParams& params) {
continue;
}
for(auto arg: args) {
if(arg == "-###"sv) {
// FIXME: the system compiler may be newer than our embedded LLVM,
// producing cc1 flags we don't recognize. Filter them out here.
// Long-term we should unify the command pipeline so the driver
// version always matches the embedded LLVM.
auto& table = clang::driver::getDriverOptTable();
auto cc1_args = llvm::ArrayRef(args).drop_front(2);
unsigned missing_index = 0, missing_count = 0;
auto parsed = table.ParseArgs(cc1_args, missing_index, missing_count);
llvm::DenseSet<unsigned> unknown_indices;
for(auto* a: parsed) {
if(a->getOption().getKind() == llvm::opt::Option::UnknownClass) {
unknown_indices.insert(a->getIndex());
}
}
result.emplace_back(params.callback(args[0]));
result.emplace_back(params.callback(args[1]));
for(unsigned i = 0; i < cc1_args.size(); ++i) {
if(unknown_indices.contains(i)) {
continue;
}
result.emplace_back(params.callback(arg));
if(cc1_args[i] == "-###"sv) {
continue;
}
result.emplace_back(params.callback(cc1_args[i]));
}
}
}

View File

@@ -219,9 +219,10 @@ public:
auto CreateASTConsumer(clang::CompilerInstance& instance, llvm::StringRef file)
-> std::unique_ptr<clang::ASTConsumer> final {
return std::make_unique<ProxyASTConsumer>(
WrapperFrontendAction::CreateASTConsumer(instance, file),
unit);
auto consumer = WrapperFrontendAction::CreateASTConsumer(instance, file);
if(!consumer)
return nullptr;
return std::make_unique<ProxyASTConsumer>(std::move(consumer), unit);
}
/// Make this public.

View File

@@ -81,7 +81,8 @@ auto CompilationUnitRef::file_offset(clang::SourceLocation location) -> std::uin
}
auto CompilationUnitRef::file_path(clang::FileID fid) -> llvm::StringRef {
assert(fid.isValid() && "Invalid fid");
if(!fid.isValid())
return {};
if(auto it = self->path_cache.find(fid); it != self->path_cache.end()) {
return it->second;
}

View File

@@ -92,15 +92,11 @@ tidy::ClangTidyOptions create_options() {
// include-cleaner is directly integrated in IncludeCleaner.cpp
"-misc-include-cleaner",
// ----- False Positives -----
// Check relies on seeing ifndef/define/endif directives,
// clangd doesn't replay those when using a preamble.
"-llvm-header-guard",
"-modernize-macro-to-enum",
// ----- Crashing Checks -----
// Check can choke on invalid (intermediate) c++
// code, which is often the case when clangd
// tries to build an AST.

View File

@@ -93,18 +93,9 @@ auto symbol_detail(clang::ASTContext& context, const clang::NamedDecl& decl) ->
return detail;
}
struct InternalSymbol {
std::string name;
std::string detail;
SymbolKind kind = SymbolKind::Invalid;
LocalSourceRange range;
LocalSourceRange selection_range;
std::vector<InternalSymbol> children;
};
struct SymbolFrame {
std::vector<InternalSymbol> symbols;
std::vector<InternalSymbol>* cursor = &symbols;
std::vector<DocumentSymbol> symbols;
std::vector<DocumentSymbol>* cursor = &symbols;
};
class DocumentSymbolCollector : public FilteredASTVisitor<DocumentSymbolCollector> {
@@ -143,7 +134,7 @@ public:
return ok;
}
auto collect() -> std::vector<InternalSymbol> {
auto collect() -> std::vector<DocumentSymbol> {
TraverseDecl(unit.tu());
return std::move(result.symbols);
}
@@ -174,8 +165,8 @@ private:
SymbolFrame result;
};
void sort_symbols(std::vector<InternalSymbol>& symbols) {
std::ranges::sort(symbols, [](const InternalSymbol& lhs, const InternalSymbol& rhs) {
void sort_symbols(std::vector<DocumentSymbol>& symbols) {
std::ranges::sort(symbols, [](const DocumentSymbol& lhs, const DocumentSymbol& rhs) {
if(lhs.range.begin != rhs.range.begin) {
return lhs.range.begin < rhs.range.begin;
}
@@ -187,7 +178,7 @@ void sort_symbols(std::vector<InternalSymbol>& symbols) {
}
}
auto to_protocol_symbol(const InternalSymbol& symbol, const PositionMapper& converter)
auto to_protocol_symbol(const DocumentSymbol& symbol, const PositionMapper& converter)
-> protocol::DocumentSymbol {
protocol::DocumentSymbol result{
.name = symbol.name,
@@ -215,10 +206,15 @@ auto to_protocol_symbol(const InternalSymbol& symbol, const PositionMapper& conv
} // namespace
auto document_symbols(CompilationUnitRef unit) -> std::vector<DocumentSymbol> {
auto result = DocumentSymbolCollector(unit).collect();
sort_symbols(result);
return result;
}
auto document_symbols(CompilationUnitRef unit, PositionEncoding encoding)
-> std::vector<protocol::DocumentSymbol> {
auto internal = DocumentSymbolCollector(unit).collect();
sort_symbols(internal);
auto internal = document_symbols(unit);
PositionMapper converter(unit.interested_content(), encoding);
std::vector<protocol::DocumentSymbol> symbols;

View File

@@ -7,6 +7,7 @@
#include "compile/compilation.h"
#include "compile/compilation_unit.h"
#include "semantic/symbol_kind.h"
#include "kota/ipc/lsp/position.h"
#include "kota/ipc/lsp/protocol.h"
@@ -59,18 +60,66 @@ struct InlayHintsOptions {
struct SignatureHelpOptions {};
auto semantic_tokens(CompilationUnitRef unit, PositionEncoding encoding = PositionEncoding::UTF16)
struct SemanticToken {
LocalSourceRange range;
SymbolKind kind = SymbolKind::Invalid;
std::uint32_t modifiers = 0;
};
struct FoldingRange {
LocalSourceRange range;
std::optional<protocol::FoldingRangeKind> kind;
std::string collapsed_text;
};
struct DocumentSymbol {
std::string name;
std::string detail;
SymbolKind kind = SymbolKind::Invalid;
LocalSourceRange range;
LocalSourceRange selection_range;
std::vector<DocumentSymbol> children;
};
enum class HintCategory : std::uint8_t {
Parameter,
DefaultArgument,
Type,
Designator,
BlockEnd,
};
struct InlayHint {
std::uint32_t offset = 0;
HintCategory kind = HintCategory::Type;
std::string label;
bool padding_left = false;
bool padding_right = false;
};
auto semantic_tokens(CompilationUnitRef unit) -> std::vector<SemanticToken>;
auto semantic_tokens(CompilationUnitRef unit, PositionEncoding encoding)
-> protocol::SemanticTokens;
auto folding_ranges(CompilationUnitRef unit) -> std::vector<FoldingRange>;
auto folding_ranges(CompilationUnitRef unit, PositionEncoding encoding)
-> std::vector<protocol::FoldingRange>;
auto document_symbols(CompilationUnitRef unit) -> std::vector<DocumentSymbol>;
auto document_symbols(CompilationUnitRef unit, PositionEncoding encoding)
-> std::vector<protocol::DocumentSymbol>;
auto inlay_hints(CompilationUnitRef unit,
LocalSourceRange target,
const InlayHintsOptions& options = {}) -> std::vector<InlayHint>;
auto inlay_hints(CompilationUnitRef unit,
LocalSourceRange target,
const InlayHintsOptions& options,
PositionEncoding encoding) -> std::vector<protocol::InlayHint>;
auto document_links(CompilationUnitRef unit, PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::DocumentLink>;
auto document_symbols(CompilationUnitRef unit, PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::DocumentSymbol>;
auto folding_ranges(CompilationUnitRef unit, PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::FoldingRange>;
auto diagnostics(CompilationUnitRef unit, PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::Diagnostic>;
@@ -89,12 +138,6 @@ auto hover(CompilationUnitRef unit,
const HoverOptions& options = {},
PositionEncoding encoding = PositionEncoding::UTF16) -> std::optional<protocol::Hover>;
auto inlay_hints(CompilationUnitRef unit,
LocalSourceRange target,
const InlayHintsOptions& options = {},
PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::InlayHint>;
auto signature_help(CompilationParams& params, const SignatureHelpOptions& options = {})
-> protocol::SignatureHelp;

View File

@@ -53,12 +53,6 @@ auto to_kind(FoldingKind kind) -> protocol::FoldingRangeKind {
return protocol::FoldingRangeKind(protocol::FoldingRangeKind::region);
}
struct RawFoldingRange {
LocalSourceRange range;
std::optional<protocol::FoldingRangeKind> kind;
std::string collapsed_text;
};
class FoldingRangeCollector : public FilteredASTVisitor<FoldingRangeCollector> {
public:
explicit FoldingRangeCollector(CompilationUnitRef unit) : FilteredASTVisitor(unit, true) {}
@@ -185,7 +179,7 @@ public:
return true;
}
auto collect() -> std::vector<RawFoldingRange> {
auto collect() -> std::vector<FoldingRange> {
TraverseDecl(unit.tu());
auto directives_it = unit.directives().find(unit.interested_file());
@@ -193,7 +187,7 @@ public:
collect_directives(directives_it->second);
}
std::ranges::sort(ranges, [](const RawFoldingRange& lhs, const RawFoldingRange& rhs) {
std::ranges::sort(ranges, [](const FoldingRange& lhs, const FoldingRange& rhs) {
if(lhs.range.begin != rhs.range.begin) {
return lhs.range.begin < rhs.range.begin;
}
@@ -343,14 +337,18 @@ private:
}
private:
std::vector<RawFoldingRange> ranges;
std::vector<FoldingRange> ranges;
};
} // namespace
auto folding_ranges(CompilationUnitRef unit) -> std::vector<FoldingRange> {
return FoldingRangeCollector(unit).collect();
}
auto folding_ranges(CompilationUnitRef unit, PositionEncoding encoding)
-> std::vector<protocol::FoldingRange> {
auto collected = FoldingRangeCollector(unit).collect();
auto collected = folding_ranges(unit);
PositionMapper converter(unit.interested_content(), encoding);
std::vector<protocol::FoldingRange> result;

View File

@@ -49,7 +49,7 @@ auto document_format(llvm::StringRef file,
range ? tooling::Range(range->begin, range->length()) : tooling::Range(0, content.size());
auto replacements = format_content(file, content, selection);
if(!replacements) {
LOG_INFO("Fail to format for {}\n{}", file, replacements.error());
LOG_WARN("Failed to format {}: {}", file, replacements.error());
return edits;
}

View File

@@ -26,22 +26,6 @@ using llvm::dyn_cast_or_null;
// For now, inlay hints are always anchored at the left or right of their range.
enum class HintSide { Left, Right };
enum class HintCategory : std::uint8_t {
Parameter,
DefaultArgument,
Type,
Designator,
BlockEnd,
};
struct RawInlayHint {
std::uint32_t offset = 0;
HintCategory kind = HintCategory::Type;
std::string label;
bool padding_left = false;
bool padding_right = false;
};
bool is_expanded_from_param_pack(const clang::ParmVarDecl* param) {
return ast::underlying_pack_type(param) != nullptr;
}
@@ -123,7 +107,7 @@ struct Callee {
class Builder {
public:
Builder(std::vector<RawInlayHint>& result,
Builder(std::vector<InlayHint>& result,
CompilationUnitRef unit,
LocalSourceRange restrict_range,
const InlayHintsOptions& options) :
@@ -499,7 +483,7 @@ public:
bool pad_left = prefix.consume_front(" ");
bool pad_right = suffix.consume_back(" ");
RawInlayHint hint{
InlayHint hint{
.offset = offset,
.kind = kind,
.label = (prefix + label + suffix).str(),
@@ -554,7 +538,7 @@ public:
}
private:
std::vector<RawInlayHint>& result;
std::vector<InlayHint>& result;
CompilationUnitRef unit;
LocalSourceRange restrict_range;
const InlayHintsOptions& options;
@@ -913,36 +897,43 @@ private:
} // namespace
auto inlay_hints(CompilationUnitRef unit,
LocalSourceRange target,
const InlayHintsOptions& options,
PositionEncoding encoding) -> std::vector<protocol::InlayHint> {
auto inlay_hints(CompilationUnitRef unit, LocalSourceRange target, const InlayHintsOptions& options)
-> std::vector<InlayHint> {
if(!options.enabled) {
return {};
}
std::vector<RawInlayHint> raw_hints;
std::vector<InlayHint> raw_hints;
Builder builder(raw_hints, unit, target, options);
Visitor visitor(builder, unit, target, options);
visitor.TraverseDecl(unit.tu());
std::ranges::sort(raw_hints, [](const RawInlayHint& lhs, const RawInlayHint& rhs) {
std::ranges::sort(raw_hints, [](const InlayHint& lhs, const InlayHint& rhs) {
return std::tie(lhs.offset, lhs.label, lhs.kind, lhs.padding_left, lhs.padding_right) <
std::tie(rhs.offset, rhs.label, rhs.kind, rhs.padding_left, rhs.padding_right);
});
auto unique_begin =
std::ranges::unique(raw_hints, [](const RawInlayHint& lhs, const RawInlayHint& rhs) {
std::ranges::unique(raw_hints, [](const InlayHint& lhs, const InlayHint& rhs) {
return lhs.offset == rhs.offset && lhs.kind == rhs.kind && lhs.label == rhs.label &&
lhs.padding_left == rhs.padding_left && lhs.padding_right == rhs.padding_right;
});
raw_hints.erase(unique_begin.begin(), unique_begin.end());
return raw_hints;
}
auto inlay_hints(CompilationUnitRef unit,
LocalSourceRange target,
const InlayHintsOptions& options,
PositionEncoding encoding) -> std::vector<protocol::InlayHint> {
auto collected = inlay_hints(unit, target, options);
PositionMapper converter(unit.interested_content(), encoding);
std::vector<protocol::InlayHint> hints;
hints.reserve(raw_hints.size());
hints.reserve(collected.size());
for(const auto& hint: raw_hints) {
for(const auto& hint: collected) {
protocol::InlayHint out{
.position = *converter.to_position(hint.offset),
.label = hint.label,

View File

@@ -18,12 +18,6 @@ namespace clice::feature {
namespace {
struct RawToken {
LocalSourceRange range;
SymbolKind kind = SymbolKind::Invalid;
std::uint32_t modifiers = 0;
};
void add_modifier(std::uint32_t& modifiers, SymbolModifiers::Kind kind) {
modifiers |= SymbolModifiers::to_mask(kind);
}
@@ -40,6 +34,34 @@ bool is_dependent(const clang::Decl* D) {
return isa<clang::UnresolvedUsingValueDecl>(D);
}
/// Whether a declaration name is backed by source text that should be highlighted.
bool can_highlight_name(clang::DeclarationName name) {
switch(name.getNameKind()) {
case clang::DeclarationName::Identifier: {
auto* info = name.getAsIdentifierInfo();
return info && !info->getName().empty();
}
case clang::DeclarationName::CXXConstructorName:
case clang::DeclarationName::CXXDestructorName: {
return true;
}
case clang::DeclarationName::CXXConversionFunctionName:
case clang::DeclarationName::CXXOperatorName:
case clang::DeclarationName::CXXDeductionGuideName:
case clang::DeclarationName::CXXLiteralOperatorName:
case clang::DeclarationName::CXXUsingDirective:
case clang::DeclarationName::ObjCZeroArgSelector:
case clang::DeclarationName::ObjCOneArgSelector:
case clang::DeclarationName::ObjCMultiArgSelector: {
return false;
}
}
std::unreachable();
}
/// Returns true if `decl` is considered to be from a default/system library.
/// This currently checks the systemness of the file by include type, although
/// different heuristics may be used in the future (e.g. sysroot paths).
@@ -166,7 +188,7 @@ class SemanticTokensCollector : public SemanticVisitor<SemanticTokensCollector>
public:
explicit SemanticTokensCollector(CompilationUnitRef unit) : SemanticVisitor(unit, true) {}
auto collect() -> std::vector<RawToken> {
auto collect() -> std::vector<SemanticToken> {
highlight_lexical(unit.interested_file());
run();
highlight_modules();
@@ -177,6 +199,10 @@ public:
void handleDeclOccurrence(const clang::NamedDecl* decl,
RelationKind relation,
clang::SourceLocation location) {
if(relation.isReference() && !can_highlight_name(decl->getDeclName())) {
return;
}
std::uint32_t modifiers = 0;
if(relation.is_one_of(RelationKind::Definition)) {
// todo: clangd add both Declaration and Definition modifiers for definitions.
@@ -398,7 +424,7 @@ private:
}
}
static void resolve_conflict(RawToken& last, const RawToken& current) {
static void resolve_conflict(SemanticToken& last, const SemanticToken& current) {
if(last.kind == SymbolKind::Conflict) {
return;
}
@@ -414,14 +440,14 @@ private:
}
void merge_tokens() {
std::ranges::sort(tokens, [](const RawToken& lhs, const RawToken& rhs) {
std::ranges::sort(tokens, [](const SemanticToken& lhs, const SemanticToken& rhs) {
if(lhs.range.begin != rhs.range.begin) {
return lhs.range.begin < rhs.range.begin;
}
return lhs.range.end < rhs.range.end;
});
std::vector<RawToken> merged;
std::vector<SemanticToken> merged;
merged.reserve(tokens.size());
for(const auto& token: tokens) {
@@ -448,7 +474,7 @@ private:
}
public:
std::vector<RawToken> tokens;
std::vector<SemanticToken> tokens;
};
class SemanticTokenEncoder {
@@ -458,7 +484,7 @@ public:
protocol::SemanticTokens& output) :
content(content), converter(content, encoding), output(output) {}
void append(const RawToken& token) {
void append(const SemanticToken& token) {
if(!token.range.valid() || token.range.end <= token.range.begin ||
token.range.end > content.size()) {
return;
@@ -542,10 +568,14 @@ private:
} // namespace
auto semantic_tokens(CompilationUnitRef unit) -> std::vector<SemanticToken> {
SemanticTokensCollector collector(unit);
return collector.collect();
}
auto semantic_tokens(CompilationUnitRef unit, PositionEncoding encoding)
-> protocol::SemanticTokens {
SemanticTokensCollector collector(unit);
auto tokens = collector.collect();
auto tokens = semantic_tokens(unit);
protocol::SemanticTokens result;
result.data.reserve(tokens.size() * 5);

View File

@@ -7,7 +7,6 @@
#include "syntax/token.h"
#include "kota/meta/annotation.h"
#include "llvm/ADT/DenseMap.h"
namespace clice {
@@ -43,10 +42,7 @@ struct IncludeGraph {
/// Each `FileID` represents a new header context and is introduced
/// by a new include directive. So a include directive is a new header
/// context. A map between FileID and its include location.
///
/// Runtime-only: `clang::FileID` is an AST-scoped handle; on-disk the
/// include graph is fully described by `paths` + `locations`.
kota::meta::skip<llvm::DenseMap<clang::FileID, std::uint32_t>> file_table;
llvm::DenseMap<clang::FileID, std::uint32_t> file_table;
static IncludeGraph from(CompilationUnitRef unit);

View File

@@ -1,121 +0,0 @@
#pragma once
#include <chrono>
#include <cstddef>
#include <cstdint>
#include <utility>
#include <vector>
#include "semantic/relation_kind.h"
#include "semantic/symbol_kind.h"
#include "support/bitmap.h"
#include "kota/codec/arena/traits.h"
#include "kota/codec/detail/fwd.h"
/// Type-level wire traits for clice index types.
///
/// These partially specialize the primary
/// `kota::codec::serialize_traits<S, T>` / `deserialize_traits<D, T>`
/// templates, constrained so only arena backends pick them up. They
/// declare the wire representation for `T` and propagate through map
/// values, sequence elements, and nested containers — no per-field
/// `annotation<T, with<...>>` required.
namespace kota::codec {
/// `std::chrono::milliseconds` ⇄ `int64` tick count.
template <typename S>
requires arena::arena_serializer_like<S>
struct serialize_traits<S, std::chrono::milliseconds> {
using wire_type = std::int64_t;
static std::int64_t serialize(S&, std::chrono::milliseconds value) noexcept {
return value.count();
}
};
template <typename D>
requires arena::arena_deserializer_like<D>
struct deserialize_traits<D, std::chrono::milliseconds> {
using wire_type = std::int64_t;
static std::chrono::milliseconds deserialize(const D&, std::int64_t value) noexcept {
return std::chrono::milliseconds(value);
}
};
/// `RelationKind` ⇄ underlying `uint32` bitflags.
template <typename S>
requires arena::arena_serializer_like<S>
struct serialize_traits<S, clice::RelationKind> {
using wire_type = std::uint32_t;
static std::uint32_t serialize(S&, const clice::RelationKind& k) noexcept {
return k.value();
}
};
template <typename D>
requires arena::arena_deserializer_like<D>
struct deserialize_traits<D, clice::RelationKind> {
using wire_type = std::uint32_t;
static clice::RelationKind deserialize(const D&, std::uint32_t v) noexcept {
return clice::RelationKind(static_cast<clice::RelationKind::Kind>(v));
}
};
/// `SymbolKind` ⇄ underlying `uint8`.
template <typename S>
requires arena::arena_serializer_like<S>
struct serialize_traits<S, clice::SymbolKind> {
using wire_type = std::uint8_t;
static std::uint8_t serialize(S&, const clice::SymbolKind& k) noexcept {
return k.value();
}
};
template <typename D>
requires arena::arena_deserializer_like<D>
struct deserialize_traits<D, clice::SymbolKind> {
using wire_type = std::uint8_t;
static clice::SymbolKind deserialize(const D&, std::uint8_t v) noexcept {
return clice::SymbolKind(v);
}
};
/// `clice::Bitmap` (= `roaring::Roaring`) ⇄ opaque byte blob produced by
/// Roaring's non-portable serialization (matches the legacy wire format).
template <typename S>
requires arena::arena_serializer_like<S>
struct serialize_traits<S, clice::Bitmap> {
using wire_type = std::vector<std::byte>;
static std::vector<std::byte> serialize(S&, const clice::Bitmap& bitmap) {
std::vector<std::byte> buffer;
if(bitmap.isEmpty()) {
return buffer;
}
buffer.resize(bitmap.getSizeInBytes(false));
bitmap.write(reinterpret_cast<char*>(buffer.data()), false);
return buffer;
}
};
template <typename D>
requires arena::arena_deserializer_like<D>
struct deserialize_traits<D, clice::Bitmap> {
using wire_type = std::vector<std::byte>;
static clice::Bitmap deserialize(const D&, std::vector<std::byte> bytes) {
if(bytes.empty()) {
return clice::Bitmap();
}
return clice::Bitmap::read(reinterpret_cast<const char*>(bytes.data()), false);
}
};
} // namespace kota::codec

View File

@@ -1,18 +1,11 @@
#include "index/merged_index.h"
#include <cassert>
#include <cstdint>
#include <ranges>
#include <span>
#include <tuple>
#include "index/kotatsu_adapters.h" // type_adapter specializations
#include "index/serialization.h"
#include "support/filesystem.h"
#include "kota/codec/flatbuffers/deserializer.h"
#include "kota/codec/flatbuffers/proxy.h"
#include "kota/codec/flatbuffers/serializer.h"
#include "kota/meta/annotation.h"
#include "llvm/ADT/DenseSet.h"
#include "llvm/Support/raw_os_ostream.h"
@@ -104,7 +97,7 @@ struct CompilationContext {
std::uint32_t canonical_id = 0;
std::uint64_t build_at = 0;
std::uint64_t build_at;
std::vector<IncludeLocation> include_locations;
@@ -132,9 +125,8 @@ struct MergedIndex::Impl {
/// The max canonical id we have allocated.
std::uint32_t max_canonical_id = 0;
/// Reference counts per canonical id — derivable from header/compilation
/// contexts at load time, so it doesn't need to live on the wire.
kota::meta::skip<std::vector<std::uint32_t>> canonical_ref_counts;
/// The reference count of each canonical id.
std::vector<std::uint32_t> canonical_ref_counts;
/// The canonical id set of removed index.
roaring::Roaring removed;
@@ -145,8 +137,8 @@ struct MergedIndex::Impl {
/// All merged symbol relations.
llvm::DenseMap<SymbolHash, llvm::DenseMap<Relation, roaring::Roaring>> relations;
/// Sorted occurrences cache for fast lookup — rebuilt on demand.
kota::meta::skip<std::vector<Occurrence>> occurrences_cache;
/// Sorted occurrences cache for fast lookup.
std::vector<Occurrence> occurrences_cache;
void merge(this Impl& self, std::uint32_t path_id, FileIndex& index, auto&& add_context) {
auto hash = index.hash();
@@ -180,18 +172,6 @@ struct MergedIndex::Impl {
friend bool operator==(const Impl&, const Impl&) = default;
};
namespace {
namespace kfb = kota::codec::flatbuffers;
std::span<const std::uint8_t> buffer_bytes(const llvm::MemoryBuffer& buffer) {
return std::span<const std::uint8_t>(
reinterpret_cast<const std::uint8_t*>(buffer.getBufferStart()),
buffer.getBufferSize());
}
} // namespace
MergedIndex::MergedIndex(std::unique_ptr<llvm::MemoryBuffer> buffer, std::unique_ptr<Impl> impl) :
buffer(std::move(buffer)), impl(std::move(impl)) {}
@@ -216,24 +196,65 @@ void MergedIndex::load_in_memory(this Self& self) {
return;
}
auto bytes = buffer_bytes(*self.buffer);
auto result = kfb::from_flatbuffer(bytes, *self.impl);
if(!result) {
self.buffer.reset();
return;
auto& index = *self.impl;
auto root = fbs::GetRoot<binary::MergedIndex>(self.buffer->getBufferStart());
index.max_canonical_id = root->max_canonical_id();
for(auto entry: *root->canonical_cache()) {
index.canonical_cache.try_emplace(entry->sha256()->string_view(), entry->canonical_id());
}
// Rebuild the ref count table from the already-loaded contexts.
auto& index = *self.impl;
index.canonical_ref_counts.clear();
index.canonical_ref_counts.resize(index.max_canonical_id, 0);
for(auto& [_, ctx]: index.header_contexts) {
for(auto& inc: ctx.includes) {
index.canonical_ref_counts[inc.canonical_id] += 1;
for(auto entry: *root->header_contexts()) {
HeaderContext context;
auto path = entry->path_id();
context.version = entry->version();
for(auto include: *entry->includes()) {
index.canonical_ref_counts[include->canonical_id()] += 1;
context.includes.emplace_back(*safe_cast<IncludeContext>(include));
}
index.header_contexts.try_emplace(path, std::move(context));
}
for(auto entry: *root->compilation_contexts()) {
CompilationContext context;
auto path = entry->path_id();
context.version = entry->version();
context.canonical_id = entry->canonical_id();
context.build_at = entry->build_at();
for(auto include: *entry->include_locations()) {
context.include_locations.emplace_back(*safe_cast<IncludeLocation>(include));
}
index.compilation_contexts.try_emplace(path, std::move(context));
}
// Count ref counts from compilation contexts.
for(auto entry: *root->compilation_contexts()) {
index.canonical_ref_counts[entry->canonical_id()] += 1;
}
// Deserialize removed bitmap.
if(root->removed() && root->removed()->size() > 0) {
index.removed = read_bitmap(root->removed());
}
for(auto entry: *root->occurrences()) {
index.occurrences.try_emplace(*safe_cast<Occurrence>(entry->occurrence()),
read_bitmap(entry->context()));
}
for(auto entry: *root->relations()) {
auto& relations = index.relations[entry->symbol()];
for(auto relation_entry: *entry->relations()) {
relations.try_emplace(*safe_cast<Relation>(relation_entry->relation()),
read_bitmap(relation_entry->context()));
}
}
for(auto& [_, ctx]: index.compilation_contexts) {
index.canonical_ref_counts[ctx.canonical_id] += 1;
if(root->content()) {
index.content = root->content()->str();
}
self.buffer.reset();
@@ -258,9 +279,100 @@ void MergedIndex::serialize(this const Self& self, llvm::raw_ostream& out) {
return;
}
auto bytes = kfb::to_flatbuffer(*self.impl);
assert(bytes && "MergedIndex flatbuffer serialization failed");
out.write(reinterpret_cast<const char*>(bytes->data()), bytes->size());
auto& index = self.impl;
fbs::FlatBufferBuilder builder(1024);
llvm::SmallVector<char, 1024> buffer;
auto canonical_cache = transform(index->canonical_cache, [&](auto&& value) {
auto&& [hash, canonical_id] = value;
return binary::CreateCacheEntry(builder, CreateString(builder, hash), canonical_id);
});
auto header_contexts = transform(index->header_contexts, [&](auto&& value) {
auto& [path_id, context] = value;
return binary::CreateHeaderContextEntry(
builder,
path_id,
context.version,
CreateStructVector<binary::IncludeContext>(builder, context.includes));
});
auto compilation_contexts = transform(index->compilation_contexts, [&](auto&& value) {
auto& [path_id, context] = value;
return binary::CreateCompilationContextEntry(
builder,
path_id,
context.version,
context.canonical_id,
context.build_at,
CreateStructVector<binary::IncludeLocation>(builder, context.include_locations));
});
llvm::SmallVector<const Occurrence*> occurrence_keys;
occurrence_keys.reserve(index->occurrences.size());
auto occurrences = transform(index->occurrences, [&](auto&& value) {
auto&& [occurrence, bitmap] = value;
buffer.clear();
buffer.resize_for_overwrite(bitmap.getSizeInBytes(false));
bitmap.write(buffer.data(), false);
occurrence_keys.emplace_back(&occurrence);
return binary::CreateOccurrenceEntry(builder,
safe_cast<binary::Occurrence>(&occurrence),
CreateVector(builder, buffer));
});
std::ranges::sort(std::views::zip(occurrence_keys, occurrences), [](auto lhs, auto rhs) {
const auto& lo = *std::get<0>(lhs);
const auto& ro = *std::get<0>(rhs);
return std::tuple(lo.range.begin, lo.range.end, lo.target) <
std::tuple(ro.range.begin, ro.range.end, ro.target);
});
llvm::SmallVector<std::uint64_t> relation_keys;
relation_keys.reserve(index->relations.size());
auto relations = transform(index->relations, [&](auto&& value) {
auto&& [symbol_id, symbol_relations] = value;
auto relations = transform(symbol_relations, [&](auto&& value) {
auto&& [relation, bitmap] = value;
buffer.clear();
buffer.resize_for_overwrite(bitmap.getSizeInBytes(false));
bitmap.write(buffer.data(), false);
return binary::CreateRelationEntry(builder,
safe_cast<binary::Relation>(&relation),
CreateVector(builder, buffer));
});
relation_keys.emplace_back(symbol_id);
return binary::CreateSymbolRelationsEntry(builder,
symbol_id,
CreateVector(builder, relations));
});
std::ranges::sort(std::views::zip(relation_keys, relations), {}, [](auto e) {
return std::get<0>(e);
});
// Serialize removed bitmap.
buffer.clear();
if(!index->removed.isEmpty()) {
buffer.resize_for_overwrite(index->removed.getSizeInBytes(false));
index->removed.write(buffer.data(), false);
}
auto removed = CreateVector(builder, buffer);
auto content_offset = CreateString(builder, index->content);
auto merged_index = binary::CreateMergedIndex(builder,
index->max_canonical_id,
CreateVector(builder, canonical_cache),
CreateVector(builder, header_contexts),
CreateVector(builder, compilation_contexts),
CreateVector(builder, occurrences),
CreateVector(builder, relations),
removed,
content_offset);
builder.Finish(merged_index);
out.write(safe_cast<char>(builder.GetBufferPointer()), builder.GetSize());
}
void MergedIndex::lookup(this const Self& self,
@@ -308,43 +420,25 @@ void MergedIndex::lookup(this const Self& self,
break;
}
} else if(self.buffer) {
// Lazy path: binary-search the sorted occurrences array directly in
// the flatbuffer without materializing the in-memory Impl.
auto root = kfb::table_view<Impl>::from_bytes(buffer_bytes(*self.buffer));
auto entries = root[&Impl::occurrences];
auto index = fbs::GetRoot<binary::MergedIndex>(self.buffer->getBufferStart());
auto& occurrences = *index->occurrences();
auto read_occurrence = [](auto occ_view) -> Occurrence {
auto range_view = occ_view[&Occurrence::range];
return Occurrence{
LocalSourceRange{range_view[&LocalSourceRange::begin],
range_view[&LocalSourceRange::end]},
occ_view[&Occurrence::target],
};
};
auto it = std::ranges::lower_bound(occurrences, offset, {}, [](auto o) {
return o->occurrence()->range().end();
});
const std::size_t count = entries.size();
std::size_t lo = 0;
std::size_t hi = count;
while(lo < hi) {
auto mid = lo + (hi - lo) / 2;
auto entry = entries.at(mid);
auto range_view = entry.template get<0>()[&Occurrence::range];
if(range_view[&LocalSourceRange::end] < offset) {
lo = mid + 1;
} else {
hi = mid;
}
}
while(it != occurrences.end()) {
auto o = safe_cast<Occurrence>(it->occurrence());
if(o->range.contains(offset)) {
if(!callback(*o)) {
break;
}
for(; lo < count; ++lo) {
auto entry = entries.at(lo);
auto occurrence = read_occurrence(entry.template get<0>());
if(!occurrence.range.contains(offset)) {
break;
}
if(!callback(occurrence)) {
break;
it++;
continue;
}
break;
}
}
}
@@ -376,31 +470,18 @@ void MergedIndex::lookup(this const Self& self,
}
}
} else if(self.buffer) {
// Lazy path: binary-search the outer relations map and iterate the
// inner map without materializing Impl.
auto root = kfb::table_view<Impl>::from_bytes(buffer_bytes(*self.buffer));
auto outer = root[&Impl::relations];
auto entry = outer.find(symbol);
if(!entry) {
auto index = fbs::GetRoot<binary::MergedIndex>(self.buffer->getBufferStart());
auto& entries = *index->relations();
auto it = std::ranges::lower_bound(entries, symbol, {}, [](auto e) { return e->symbol(); });
if(it == entries.end() || it->symbol() != symbol) [[unlikely]] {
return;
}
auto inner = entry->template get<1>();
const std::size_t count = inner.size();
for(std::size_t i = 0; i < count; ++i) {
auto rel_view = inner.at(i).template get<0>();
// Kind comes back as the wire uint32 via the type_adapter; rewrap it.
auto relation_kind =
RelationKind(static_cast<RelationKind::Kind>(rel_view[&Relation::kind]));
if(relation_kind & kind) {
auto range_view = rel_view[&Relation::range];
Relation relation{
.kind = relation_kind,
.padding = rel_view[&Relation::padding],
.range = LocalSourceRange{range_view[&LocalSourceRange::begin],
range_view[&LocalSourceRange::end]},
.target_symbol = rel_view[&Relation::target_symbol],
};
if(!callback(relation)) {
for(auto entry: *it->relations()) {
auto r = safe_cast<Relation>(entry->relation());
if(r->kind & kind) {
if(!callback(*r)) {
break;
}
}
@@ -435,31 +516,25 @@ bool MergedIndex::need_update(this const Self& self, llvm::ArrayRef<llvm::String
return false;
} else if(self.buffer) {
auto root = kfb::table_view<Impl>::from_bytes(buffer_bytes(*self.buffer));
auto contexts = root[&Impl::compilation_contexts];
if(contexts.empty()) {
auto index = fbs::GetRoot<binary::MergedIndex>(self.buffer->getBufferStart());
if(index->compilation_contexts()->empty()) {
return true;
}
auto context = contexts.at(0).template get<1>();
auto build_at = context[&CompilationContext::build_at];
auto include_locations = context[&CompilationContext::include_locations];
auto context = *index->compilation_contexts()->begin();
llvm::DenseSet<std::uint32_t> deps;
const std::size_t count = include_locations.size();
for(std::size_t i = 0; i < count; ++i) {
auto location = include_locations.at(i);
auto path_id = location[&IncludeLocation::path_id];
auto [_, success] = deps.insert(path_id);
for(auto location: *context->include_locations()) {
auto [_, success] = deps.insert(location->path_id());
if(success) {
fs::file_status status;
if(auto err = fs::status(path_mapping[path_id], status)) {
if(auto err = fs::status(path_mapping[location->path_id()], status)) {
return true;
}
auto time = std::chrono::duration_cast<std::chrono::milliseconds>(
status.getLastModificationTime().time_since_epoch());
if(time.count() > build_at) {
if(time.count() > context->build_at()) {
return true;
}
}
@@ -541,9 +616,10 @@ llvm::StringRef MergedIndex::content(this const Self& self) {
if(self.impl) {
return self.impl->content;
} else if(self.buffer) {
auto root = kfb::table_view<Impl>::from_bytes(buffer_bytes(*self.buffer));
auto view = root[&Impl::content];
return llvm::StringRef(view.data(), view.size());
auto root = fbs::GetRoot<binary::MergedIndex>(self.buffer->getBufferStart());
if(root->content()) {
return root->content()->string_view();
}
}
return {};
}

View File

@@ -1,22 +1,9 @@
#include "index/project_index.h"
#include <cassert>
#include <cstdint>
#include <span>
#include "index/kotatsu_adapters.h" // type_adapter specializations
#include "kota/codec/flatbuffers/deserializer.h"
#include "kota/codec/flatbuffers/serializer.h"
#include "index/serialization.h"
namespace clice::index {
namespace {
namespace kfb = kota::codec::flatbuffers;
} // namespace
llvm::SmallVector<std::uint32_t> ProjectIndex::merge(this ProjectIndex& self, TUIndex& index) {
auto& paths = index.graph.paths;
llvm::SmallVector<std::uint32_t> file_ids_map;
@@ -41,22 +28,79 @@ llvm::SmallVector<std::uint32_t> ProjectIndex::merge(this ProjectIndex& self, TU
}
void ProjectIndex::serialize(this ProjectIndex& self, llvm::raw_ostream& os) {
auto bytes = kfb::to_flatbuffer(self);
assert(bytes && "ProjectIndex flatbuffer serialization failed");
os.write(reinterpret_cast<const char*>(bytes->data()), bytes->size());
fbs::FlatBufferBuilder builder(1024);
llvm::SmallVector<char, 1024> buffer;
auto i = 0;
auto paths = transform(self.path_pool.paths, [&](llvm::StringRef path) {
auto entry =
binary::CreatePathEntry(builder, CreateString(builder, self.path_pool.paths[i]), i);
i += 1;
return entry;
});
auto indices = transform(self.indices, [&](auto&& value) {
auto&& [source, index] = value;
return binary::PathMapEntry(source, index);
});
auto symbols = transform(self.symbols, [&](auto&& value) {
auto& [symbol_id, symbol] = value;
buffer.clear();
buffer.resize_for_overwrite(symbol.reference_files.getSizeInBytes(false));
symbol.reference_files.write(buffer.data(), false);
return binary::CreateSymbolEntry(builder,
symbol_id,
binary::CreateSymbol(builder,
CreateString(builder, symbol.name),
symbol.kind.value(),
CreateVector(builder, buffer)));
});
auto project_index =
binary::CreateProjectIndex(builder,
CreateVector(builder, paths),
CreateStructVector<binary::PathMapEntry>(builder, indices),
CreateVector(builder, symbols));
builder.Finish(project_index);
os.write(safe_cast<const char>(builder.GetBufferPointer()), builder.GetSize());
}
ProjectIndex ProjectIndex::from(const void* data, std::size_t size) {
ProjectIndex ProjectIndex::from(const void* data) {
auto root = fbs::GetRoot<binary::ProjectIndex>(data);
ProjectIndex index;
if(data == nullptr || size == 0) {
return index;
auto& pool = index.path_pool;
pool.paths.resize(root->paths()->size());
for(auto entry: *root->paths()) {
// Normalize backslashes to forward slashes for cross-platform consistency
// (persisted index may contain native-separator paths from Windows).
llvm::SmallString<256> normalized(entry->path()->string_view());
std::replace(normalized.begin(), normalized.end(), '\\', '/');
auto k = pool.save(normalized.str());
pool.paths[entry->id()] = k;
pool.cache.try_emplace(k, entry->id());
}
std::span<const std::uint8_t> bytes(static_cast<const std::uint8_t*>(data), size);
auto result = kfb::from_flatbuffer(bytes, index);
if(!result) {
return ProjectIndex();
for(auto entry: *root->indices()) {
index.indices.try_emplace(entry->source(), entry->index());
}
for(auto entry: *root->symbols()) {
auto& symbol = index.symbols[entry->symbol_id()];
auto* fb_symbol = entry->symbol();
if(auto* name = fb_symbol->name()) {
symbol.name = name->str();
}
symbol.kind = SymbolKind(static_cast<std::uint8_t>(fb_symbol->kind()));
symbol.reference_files = read_bitmap(fb_symbol->refs());
}
return index;
}

View File

@@ -2,14 +2,10 @@
#include <algorithm>
#include <cstdint>
#include <string>
#include <vector>
#include "index/tu_index.h"
#include "kota/codec/arena/traits.h"
#include "kota/codec/detail/fwd.h"
#include "kota/support/expected_try.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/SmallString.h"
#include "llvm/ADT/SmallVector.h"
@@ -88,71 +84,7 @@ struct ProjectIndex {
void serialize(this ProjectIndex& self, llvm::raw_ostream& os);
static ProjectIndex from(const void* data, std::size_t size);
static ProjectIndex from(const void* data);
};
} // namespace clice::index
namespace kota::codec {
/// `PathPool` on the wire is a flat list of absolute paths; `id` is the
/// position in the vector. The allocator and reverse cache are runtime-only.
///
/// Streaming serialize: iterate `pool.paths` and allocate strings directly
/// into the builder, avoiding the double-copy that a value-mode
/// `wire_type = std::vector<std::string>` conversion would introduce.
template <typename S>
requires arena::arena_serializer_like<S>
struct serialize_traits<S, clice::index::PathPool> {
// Structural wire shape — declared so the flatbuffers proxy views
// a `PathPool` field as an `array_view<std::string>`.
using wire_type = std::vector<std::string>;
static auto serialize(S& s, const clice::index::PathPool& pool)
-> std::expected<typename S::vector_ref, typename S::error_type> {
std::vector<typename S::string_ref> offsets;
offsets.reserve(pool.paths.size());
for(const auto& path: pool.paths) {
auto r = s.alloc_string(std::string_view(path.data(), path.size()));
if(!r) {
return std::unexpected(r.error());
}
offsets.push_back(*r);
}
return s.alloc_string_vector(
std::span<const typename S::string_ref>(offsets.data(), offsets.size()));
}
};
/// Streaming deserialize: read each path out of the flatbuffer's
/// string-vector view directly, interning it into the pool's allocator
/// in-place. Avoids the transient `std::vector<std::string>` the
/// value-mode form would materialize.
template <typename D>
requires arena::arena_deserializer_like<D>
struct deserialize_traits<D, clice::index::PathPool> {
using wire_type = std::vector<std::string>;
static auto deserialize(const D& d,
typename D::TableView view,
typename D::slot_id sid,
clice::index::PathPool& out)
-> std::expected<void, typename D::error_type> {
if(!view.has(sid)) {
return {};
}
KOTA_EXPECTED_TRY_V(auto vec, d.get_string_vector(view, sid));
out.paths.resize(vec.size());
for(std::size_t i = 0; i < vec.size(); ++i) {
auto sv = vec[i];
llvm::SmallString<256> normalized(llvm::StringRef(sv.data(), sv.size()));
std::replace(normalized.begin(), normalized.end(), '\\', '/');
auto interned = out.save(normalized.str());
out.paths[i] = interned;
out.cache.try_emplace(interned, static_cast<std::uint32_t>(i));
}
return {};
}
};
} // namespace kota::codec

173
src/index/schema.fbs Normal file
View File

@@ -0,0 +1,173 @@
namespace clice.index.binary;
struct Range {
begin : uint;
end : uint;
}
struct Occurrence {
range : Range;
target : ulong;
}
struct Relation {
kind : uint;
padding : uint;
range : Range;
target_symbol : ulong;
}
table CacheEntry {
sha256:
string;
canonical_id:
uint;
}
struct IncludeContext {
include_id : uint;
canonical_id : uint;
}
table HeaderContextEntry {
path_id:
uint;
version:
uint;
includes:
[IncludeContext];
}
struct IncludeLocation {
path_id : uint;
line : uint;
include_id : uint;
}
table CompilationContextEntry {
path_id:
uint;
version:
uint;
canonical_id:
uint;
build_at:
ulong;
include_locations:
[IncludeLocation];
}
table OccurrenceEntry {
occurrence:
Occurrence;
context:
[ubyte];
}
table RelationEntry {
relation:
Relation;
context:
[ubyte];
}
table SymbolRelationsEntry {
symbol:
ulong;
relations:
[RelationEntry];
}
table Symbol {
name:
string;
kind:
ubyte;
refs:
[ubyte];
}
table SymbolEntry {
symbol_id:
ulong;
symbol:
Symbol;
}
table MergedIndex {
max_canonical_id:
uint;
canonical_cache:
[CacheEntry];
header_contexts:
[HeaderContextEntry];
compilation_contexts:
[CompilationContextEntry];
occurrences:
[OccurrenceEntry];
relations:
[SymbolRelationsEntry];
removed:
[ubyte];
content:
string;
}
table TUFileRelationsEntry {
symbol:
ulong;
relations:
[Relation];
}
table TUFileIndexEntry {
file_id:
uint;
occurrences:
[Occurrence];
relations:
[TUFileRelationsEntry];
}
table TUIndex {
built_at:
ulong;
paths:
[string];
locations:
[IncludeLocation];
symbols:
[SymbolEntry];
file_indices:
[TUFileIndexEntry];
main_file_index:
TUFileIndexEntry;
}
table PathEntry {
path:
string;
id:
uint;
}
struct PathMapEntry {
source : uint;
index : uint;
}
table ProjectIndex {
paths:
[PathEntry];
indices:
[PathMapEntry];
symbols:
[SymbolEntry];
}

79
src/index/serialization.h Normal file
View File

@@ -0,0 +1,79 @@
#include <cstdint>
#include <ranges>
#include <type_traits>
#include "schema_generated.h"
#include "support/bitmap.h"
#include "llvm/ADT/SmallVector.h"
#include "llvm/ADT/StringRef.h"
namespace clice::index {
namespace fbs = flatbuffers;
namespace {
template <typename Range>
concept sequence_range = std::ranges::input_range<Range> &&
!requires { typename Range::key_type; } && requires(const Range& r) {
r.data();
r.size();
};
template <typename T>
using Offsets = llvm::SmallVector<fbs::Offset<T>, 0>;
template <typename U, typename V>
const U* safe_cast(const V* v) {
static_assert(sizeof(U) == sizeof(V), "size mismatch");
static_assert(alignof(U) == alignof(V), "alignment mismatch");
static_assert(std::is_trivially_copyable_v<U> && std::is_trivially_copyable_v<V>,
"requires trivially copyable");
/// If aliasing issues arise, prefer copying into a temporary SmallVector<U>.
return reinterpret_cast<const U*>(v);
}
auto CreateString(fbs::FlatBufferBuilder& builder, llvm::StringRef string) {
return builder.CreateString(string.data(), string.size());
}
template <sequence_range Range>
auto CreateVector(fbs::FlatBufferBuilder& builder, const Range& range) {
return builder.CreateVector(range.data(), range.size());
}
auto CreateVector(fbs::FlatBufferBuilder& builder, const llvm::SmallVector<char, 1024>& range) {
return builder.CreateVector(reinterpret_cast<const std::uint8_t*>(range.data()), range.size());
}
template <typename U, sequence_range Range>
auto CreateStructVector(fbs::FlatBufferBuilder& builder, const Range& range) {
using V = std::ranges::range_value_t<Range>;
(void)sizeof(V);
return builder.CreateVectorOfStructs(safe_cast<U>(range.data()), range.size());
}
template <typename Range, typename Functor>
auto transform(const Range& range, const Functor& functor) {
using V = std::ranges::range_value_t<Range>;
using R = std::invoke_result_t<Functor, V>;
llvm::SmallVector<R, 0> result;
result.resize_for_overwrite(std::ranges::size(range));
auto i = 0;
for(auto&& v: range) {
result[i] = functor(v);
i += 1;
}
return result;
}
Bitmap read_bitmap(const fbs::Vector<uint8_t>* buffer) {
return Bitmap::read(reinterpret_cast<const char*>(buffer->data()), false);
}
} // namespace
} // namespace clice::index

View File

@@ -1,24 +1,17 @@
#include "index/tu_index.h"
#include <cassert>
#include <cstdint>
#include <span>
#include <tuple>
#include "index/kotatsu_adapters.h" // type_adapter specializations
#include "index/serialization.h"
#include "semantic/ast_utility.h"
#include "semantic/semantic_visitor.h"
#include "kota/codec/flatbuffers/deserializer.h"
#include "kota/codec/flatbuffers/serializer.h"
#include "llvm/Support/SHA256.h"
namespace clice::index {
namespace {
namespace kfb = kota::codec::flatbuffers;
class Builder : public SemanticVisitor<Builder> {
public:
Builder(TUIndex& result, CompilationUnitRef unit, bool interested_only) :
@@ -121,8 +114,6 @@ public:
void build() {
run();
auto interested = unit.interested_file();
for(auto& [fid, index]: result.file_indices) {
for(auto& [symbol_id, relations]: index.relations) {
std::ranges::sort(relations, [](const Relation& lhs, const Relation& rhs) {
@@ -153,19 +144,13 @@ public:
return lhs.range == rhs.range && lhs.target == rhs.target;
});
index.occurrences.erase(range.begin(), range.end());
}
// Populate main_file_index (interested file) and path_file_indices
// (keyed by path_id) for serialization. `file_indices` itself is
// `skip`-marked (runtime-only, keyed by clang::FileID) and retained
// for in-memory consumers/tests that need FileID access.
for(auto& [fid, index]: result.file_indices) {
if(fid == interested) {
result.main_file_index = index;
} else {
result.path_file_indices[result.graph.path_id(fid)] = index;
if(fid == unit.interested_file()) {
result.main_file_index = std::move(index);
}
}
result.file_indices.erase(unit.interested_file());
}
private:
@@ -213,23 +198,119 @@ TUIndex TUIndex::build(CompilationUnitRef unit, bool interested_only) {
return index;
}
void TUIndex::serialize(llvm::raw_ostream& os) {
auto bytes = kfb::to_flatbuffer(*this);
assert(bytes && "TUIndex flatbuffer serialization failed");
os.write(reinterpret_cast<const char*>(bytes->data()), bytes->size());
void TUIndex::serialize(llvm::raw_ostream& os) const {
fbs::FlatBufferBuilder builder(4096);
llvm::SmallVector<char, 1024> buffer;
auto paths =
transform(graph.paths, [&](const std::string& p) { return builder.CreateString(p); });
auto syms = transform(symbols, [&](auto&& value) {
auto& [symbol_id, symbol] = value;
buffer.clear();
buffer.resize_for_overwrite(symbol.reference_files.getSizeInBytes(false));
symbol.reference_files.write(buffer.data(), false);
return binary::CreateSymbolEntry(builder,
symbol_id,
binary::CreateSymbol(builder,
CreateString(builder, symbol.name),
symbol.kind.value(),
CreateVector(builder, buffer)));
});
/// Serialize a single FileIndex into a TUFileIndexEntry.
auto serialize_file_index = [&](std::uint32_t fid, const FileIndex& index) {
auto occs = CreateStructVector<binary::Occurrence>(builder, index.occurrences);
auto rels = transform(index.relations, [&](auto&& value) {
auto& [symbol_id, relations] = value;
return binary::CreateTUFileRelationsEntry(
builder,
symbol_id,
CreateStructVector<binary::Relation>(builder, relations));
});
return binary::CreateTUFileIndexEntry(builder, fid, occs, CreateVector(builder, rels));
};
/// Convert FileID-keyed file_indices to path_id-keyed entries.
llvm::SmallVector<fbs::Offset<binary::TUFileIndexEntry>> file_idx_vec;
for(auto& [fid, index]: file_indices) {
auto pid = graph.path_id(fid);
file_idx_vec.push_back(serialize_file_index(pid, index));
}
/// Main file is the last path in graph.paths (convention from IncludeGraph).
auto main_idx =
serialize_file_index(static_cast<std::uint32_t>(graph.paths.size() - 1), main_file_index);
auto tu_index =
binary::CreateTUIndex(builder,
static_cast<std::uint64_t>(built_at.count()),
CreateVector(builder, paths),
CreateStructVector<binary::IncludeLocation>(builder, graph.locations),
CreateVector(builder, syms),
builder.CreateVector(file_idx_vec.data(), file_idx_vec.size()),
main_idx);
builder.Finish(tu_index);
os.write(safe_cast<const char>(builder.GetBufferPointer()), builder.GetSize());
}
TUIndex TUIndex::from(const void* data, std::size_t size) {
TUIndex TUIndex::from(const void* data) {
auto root = fbs::GetRoot<binary::TUIndex>(data);
TUIndex index;
if(data == nullptr || size == 0) {
return index;
index.built_at = std::chrono::milliseconds(root->built_at());
for(auto p: *root->paths()) {
index.graph.paths.emplace_back(p->str());
}
std::span<const std::uint8_t> bytes(static_cast<const std::uint8_t*>(data), size);
auto result = kfb::from_flatbuffer(bytes, index);
if(!result) {
return TUIndex();
for(auto loc: *root->locations()) {
index.graph.locations.emplace_back(*safe_cast<IncludeLocation>(loc));
}
for(auto entry: *root->symbols()) {
auto& symbol = index.symbols[entry->symbol_id()];
symbol.name = entry->symbol()->name()->str();
symbol.kind = SymbolKind(static_cast<std::uint8_t>(entry->symbol()->kind()));
symbol.reference_files = read_bitmap(entry->symbol()->refs());
}
/// Helper to deserialize a TUFileIndexEntry into a FileIndex.
auto deserialize_file_index = [](const binary::TUFileIndexEntry* entry) -> FileIndex {
FileIndex fi;
if(entry->occurrences()) {
fi.occurrences.reserve(entry->occurrences()->size());
for(auto o: *entry->occurrences()) {
fi.occurrences.emplace_back(*safe_cast<Occurrence>(o));
}
}
if(entry->relations()) {
for(auto rel_entry: *entry->relations()) {
auto& rels = fi.relations[rel_entry->symbol()];
if(rel_entry->relations()) {
rels.reserve(rel_entry->relations()->size());
for(auto r: *rel_entry->relations()) {
rels.emplace_back(*safe_cast<Relation>(r));
}
}
}
}
return fi;
};
/// Populate path_file_indices keyed by path_id (no clang::FileID needed).
if(root->file_indices()) {
for(auto entry: *root->file_indices()) {
index.path_file_indices[entry->file_id()] = deserialize_file_index(entry);
}
}
if(root->main_file_index()) {
index.main_file_index = deserialize_file_index(root->main_file_index());
}
return index;
}

View File

@@ -12,7 +12,6 @@
#include "semantic/symbol_kind.h"
#include "support/bitmap.h"
#include "kota/meta/annotation.h"
#include "llvm/Support/raw_ostream.h"
namespace clice::index {
@@ -36,10 +35,6 @@ struct Relation {
constexpr auto definition_range() {
return std::bit_cast<LocalSourceRange>(target_symbol);
}
friend bool operator==(const Relation&, const Relation&) = default;
friend auto operator<=>(const Relation&, const Relation&) = default;
};
struct Occurrence {
@@ -50,8 +45,6 @@ struct Occurrence {
SymbolHash target;
friend bool operator==(const Occurrence&, const Occurrence&) = default;
friend auto operator<=>(const Occurrence&, const Occurrence&) = default;
};
struct FileIndex {
@@ -84,21 +77,19 @@ struct TUIndex {
SymbolTable symbols;
/// Runtime-only: keyed by AST-scoped `clang::FileID` during build; flushed
/// into `path_file_indices` (keyed by path id) before serialization.
kota::meta::skip<llvm::DenseMap<clang::FileID, FileIndex>> file_indices;
llvm::DenseMap<clang::FileID, FileIndex> file_indices;
/// File indices keyed by path_id. Populated from `file_indices` at
/// serialize time, and directly from the wire on deserialize.
/// File indices keyed by path_id, populated by from() for deserialized data.
/// When built from AST, this is empty and file_indices (keyed by FileID) is used.
llvm::DenseMap<std::uint32_t, FileIndex> path_file_indices;
FileIndex main_file_index;
static TUIndex build(CompilationUnitRef unit, bool interested_only = false);
void serialize(llvm::raw_ostream& os);
void serialize(llvm::raw_ostream& os) const;
static TUIndex from(const void* data, std::size_t size);
static TUIndex from(const void* data);
};
} // namespace clice::index

View File

@@ -308,6 +308,10 @@ const clang::NamedDecl* decl_of_impl(const void* T) {
}
auto decl_of(clang::QualType type) -> const clang::NamedDecl* {
if(type.isNull()) {
return nullptr;
}
// Strip type-sugar that wraps the underlying type without adding a decl
// (e.g. ElaboratedType for "struct Foo" vs plain "Foo").
if(auto ET = type->getAs<clang::ElaboratedType>()) {

View File

@@ -71,10 +71,6 @@ constexpr bool operator==(RelationKind lhs, RelationKind rhs) {
return lhs.value() == rhs.value();
}
constexpr auto operator<=>(RelationKind lhs, RelationKind rhs) {
return lhs.value() <=> rhs.value();
}
constexpr bool operator&(RelationKind lhs, RelationKind rhs) {
return lhs.value() == rhs.value();
}

View File

@@ -1111,8 +1111,6 @@ public:
return Base::TransformDecltypeType(TLB, TL);
}
// --- State ---
private:
clang::Sema& sema;
clang::ASTContext& context;

View File

@@ -1,4 +1,4 @@
#include "server/compile_graph.h"
#include "server/compiler/compile_graph.h"
#include <algorithm>

View File

@@ -1,4 +1,4 @@
#include "server/compiler.h"
#include "server/compiler/compiler.h"
#include <format>
#include <ranges>
@@ -6,7 +6,7 @@
#include "command/search_config.h"
#include "index/tu_index.h"
#include "server/protocol.h"
#include "server/protocol/worker.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "syntax/include_resolver.h"
@@ -28,16 +28,20 @@ using serde_raw = kota::codec::RawValue;
/// Detect whether the cursor is inside a preamble directive (include/import).
Compiler::Compiler(kota::event_loop& loop,
kota::ipc::JsonPeer& peer,
Workspace& workspace,
WorkerPool& pool,
llvm::DenseMap<std::uint32_t, Session>& sessions) :
loop(loop), peer(peer), workspace(workspace), pool(pool), sessions(sessions) {}
loop(loop), workspace(workspace), pool(pool), sessions(sessions) {}
Compiler::~Compiler() {
workspace.cancel_all();
}
kota::task<> Compiler::stop() {
compile_tasks.cancel();
co_await compile_tasks.join();
}
void Compiler::init_compile_graph() {
if(workspace.path_to_module.empty()) {
LOG_INFO("No C++20 modules detected, skipping CompileGraph");
@@ -410,6 +414,8 @@ std::string uri_to_path(const std::string& uri) {
void Compiler::publish_diagnostics(const std::string& uri,
int version,
const kota::codec::RawValue& diagnostics_json) {
if(!peer)
return;
std::vector<protocol::Diagnostic> diagnostics;
if(!diagnostics_json.empty()) {
auto status = kota::codec::json::from_json(diagnostics_json.data, diagnostics);
@@ -421,14 +427,16 @@ void Compiler::publish_diagnostics(const std::string& uri,
params.uri = uri;
params.version = version;
params.diagnostics = std::move(diagnostics);
peer.send_notification(params);
peer->send_notification(params);
}
void Compiler::clear_diagnostics(const std::string& uri) {
if(!peer)
return;
protocol::PublishDiagnosticsParams params;
params.uri = uri;
params.diagnostics = {};
peer.send_notification(params);
peer->send_notification(params);
}
kota::task<bool> Compiler::ensure_pch(Session& session,
@@ -490,6 +498,22 @@ kota::task<bool> Compiler::ensure_pch(Session& session,
auto completion = std::make_shared<kota::event>();
workspace.pch_cache[path_id].building = completion;
if(workspace.config.project.cache_dir.empty()) {
LOG_WARN("PCH build skipped: cache_dir is not configured");
workspace.pch_cache[path_id].building.reset();
completion->set();
co_return false;
}
// Ensure the PCH cache directory exists.
auto pch_dir = path::join(workspace.config.project.cache_dir, "cache", "pch");
if(auto ec = llvm::sys::fs::create_directories(pch_dir)) {
LOG_WARN("Cannot create PCH cache dir {}: {}", pch_dir, ec.message());
workspace.pch_cache[path_id].building.reset();
completion->set();
co_return false;
}
// Build a new PCH via stateless worker.
worker::BuildParams bp;
bp.kind = worker::BuildKind::BuildPCH;
@@ -613,6 +637,101 @@ void Compiler::record_deps(Session& session, llvm::ArrayRef<std::string> deps) {
/// Called lazily by forward_query() / forward_build() before every
/// feature request (hover, semantic tokens, etc.). Guarantees that when it
/// returns true the stateful worker assigned to `path_id` holds an up-to-date
kota::task<> Compiler::run_compile(std::uint32_t pid, std::shared_ptr<Session::PendingCompile> pc) {
auto find_session = [&]() -> Session* {
auto it = sessions.find(pid);
return it != sessions.end() ? &it->second : nullptr;
};
auto* sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
auto finish_compile = [&]() {
auto* s = find_session();
if(s && s->compiling == pc) {
s->compiling.reset();
}
LOG_INFO("ensure_compiled: finish path_id={}", pid);
pc->done.set();
};
auto gen = sess->generation;
LOG_INFO("ensure_compiled: starting compile path_id={} gen={}", pid, gen);
auto file_path = std::string(workspace.path_pool.resolve(pid));
auto uri = lsp::URI::from_file_path(file_path);
std::string uri_str = uri.has_value() ? uri->str() : file_path;
worker::CompileParams params;
params.path = file_path;
params.version = sess->version;
params.text = sess->text;
if(!fill_compile_args(file_path, params.directory, params.arguments, sess)) {
finish_compile();
co_return;
}
if(!co_await ensure_deps(*sess, params.directory, params.arguments, params.pch, params.pcms)) {
LOG_WARN("Dependency preparation failed for {}, skipping compile", uri_str);
finish_compile();
co_return;
}
sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
auto result = co_await pool.send_stateful(pid, params);
sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
if(sess->generation != gen) {
LOG_INFO("ensure_compiled: generation mismatch ({} vs {}) for {}",
sess->generation,
gen,
uri_str);
finish_compile();
co_return;
}
if(!result.has_value()) {
LOG_WARN("Compile failed for {}: {}", uri_str, result.error().message);
clear_diagnostics(uri_str);
finish_compile();
co_return;
}
sess->ast_dirty = false;
pc->succeeded = true;
record_deps(*sess, result.value().deps);
if(!result.value().tu_index_data.empty()) {
auto tu_index = index::TUIndex::from(result.value().tu_index_data.data());
OpenFileIndex ofi;
ofi.file_index = std::move(tu_index.main_file_index);
ofi.symbols = std::move(tu_index.symbols);
ofi.content = sess->text;
ofi.mapper.emplace(ofi.content, lsp::PositionEncoding::UTF16);
sess->file_index = std::move(ofi);
}
auto version = sess->version;
finish_compile();
publish_diagnostics(uri_str, version, result.value().diagnostics);
if(on_indexing_needed)
on_indexing_needed();
}
/// AST and diagnostics have been published to the client.
///
/// Lifecycle overview (pull-based model):
@@ -632,9 +751,9 @@ void Compiler::record_deps(Session& session, llvm::ArrayRef<std::string> deps) {
/// worker); every other file is read from disk by the compiler.
///
/// Concurrency: multiple concurrent feature requests for the same file will
/// each call ensure_compiled(). The first one launches a detached compile
/// task via loop.schedule(); subsequent ones wait on the shared event.
/// The detached task cannot be cancelled by LSP $/cancelRequest, preventing
/// each call ensure_compiled(). The first one spawns a compile task into the
/// Compiler's task_group; subsequent ones wait on the shared event.
/// The spawned task is not cancelled by LSP $/cancelRequest, preventing
/// the race where cancellation wakes all waiters and they all start compiles.
kota::task<bool> Compiler::ensure_compiled(Session& session) {
auto path_id = session.path_id;
@@ -663,125 +782,12 @@ kota::task<bool> Compiler::ensure_compiled(Session& session) {
co_return true;
}
// No compile in flight and AST is dirty — launch a detached compile task.
// The detached task is scheduled via loop.schedule() so it is NOT subject
// to LSP $/cancelRequest cancellation. This eliminates the race where
// cancellation fires the RAII guard, waking all waiters simultaneously
// and causing them all to start new compiles.
auto pending_compile = std::make_shared<Session::PendingCompile>();
session.compiling = pending_compile;
LOG_INFO("ensure_compiled: launching detached compile path_id={} gen={}",
path_id,
session.generation);
LOG_INFO("ensure_compiled: launching compile path_id={} gen={}", path_id, session.generation);
// Capture path_id by value so the detached lambda can re-lookup the session
// from the sessions map after co_await (DenseMap may invalidate pointers).
loop.schedule([](Compiler* self,
std::uint32_t pid,
std::shared_ptr<Session::PendingCompile> pc) -> kota::task<> {
// Re-lookup session from the sessions map (pointer may have been
// invalidated by DenseMap growth during co_await).
auto find_session = [&]() -> Session* {
auto it = self->sessions.find(pid);
return it != self->sessions.end() ? &it->second : nullptr;
};
auto* sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
auto finish_compile = [&]() {
auto* s = find_session();
if(s && s->compiling == pc) {
s->compiling.reset();
}
LOG_INFO("ensure_compiled: finish_compile (detached) path_id={}", pid);
pc->done.set();
};
auto gen = sess->generation;
LOG_INFO("ensure_compiled: starting compile (detached) path_id={} gen={}", pid, gen);
auto file_path = std::string(self->workspace.path_pool.resolve(pid));
auto uri = lsp::URI::from_file_path(file_path);
std::string uri_str = uri.has_value() ? uri->str() : file_path;
worker::CompileParams params;
params.path = file_path;
params.version = sess->version;
params.text = sess->text;
if(!self->fill_compile_args(file_path, params.directory, params.arguments, sess)) {
finish_compile();
co_return;
}
if(!co_await self
->ensure_deps(*sess, params.directory, params.arguments, params.pch, params.pcms)) {
LOG_WARN("Dependency preparation failed for {}, skipping compile", uri_str);
finish_compile();
co_return;
}
// Re-lookup after co_await (DenseMap may have grown).
sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
auto result = co_await self->pool.send_stateful(pid, params);
// Re-lookup after co_await.
sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
if(sess->generation != gen) {
LOG_INFO("ensure_compiled: generation mismatch ({} vs {}) for {}",
sess->generation,
gen,
uri_str);
finish_compile();
co_return;
}
if(!result.has_value()) {
LOG_WARN("Compile failed for {}: {}", uri_str, result.error().message);
self->clear_diagnostics(uri_str);
finish_compile();
co_return;
}
sess->ast_dirty = false;
pc->succeeded = true;
self->record_deps(*sess, result.value().deps);
// Store open file index from the stateful worker's TUIndex.
if(!result.value().tu_index_data.empty()) {
auto tu_index = index::TUIndex::from(result.value().tu_index_data.data(),
result.value().tu_index_data.size());
OpenFileIndex ofi;
ofi.file_index = std::move(tu_index.main_file_index);
ofi.symbols = std::move(tu_index.symbols);
ofi.content = sess->text;
ofi.mapper.emplace(ofi.content, lsp::PositionEncoding::UTF16);
sess->file_index = std::move(ofi);
}
auto version = sess->version;
finish_compile();
// Publish diagnostics AFTER marking compile as done, so that concurrent
// forward_query() calls can proceed immediately.
self->publish_diagnostics(uri_str, version, result.value().diagnostics);
if(self->on_indexing_needed)
self->on_indexing_needed();
}(this, path_id, pending_compile));
compile_tasks.spawn(run_compile(path_id, pending_compile));
// Wait for the detached compile to finish. If this wait is cancelled
// by LSP $/cancelRequest, the detached task continues unaffected.
@@ -876,6 +882,32 @@ Compiler::RawResult Compiler::forward_build(worker::BuildKind kind,
co_return std::move(result.value().result_json);
}
Compiler::RawResult Compiler::forward_format(Session& session,
std::optional<protocol::Range> range) {
auto path_id = session.path_id;
auto path = std::string(workspace.path_pool.resolve(path_id));
worker::BuildParams wp;
wp.kind = worker::BuildKind::Format;
wp.file = path;
wp.text = session.text;
if(range) {
lsp::PositionMapper mapper(wp.text, lsp::PositionEncoding::UTF16);
auto begin = mapper.to_offset(range->start);
auto end = mapper.to_offset(range->end);
if(!begin || !end)
co_return serde_raw{"null"};
wp.format_range = {*begin, *end};
}
auto result = co_await pool.send_stateless(wp);
if(!result.has_value()) {
co_return serde_raw{"null"};
}
co_return std::move(result.value().result_json);
}
Compiler::RawResult Compiler::handle_completion(const protocol::Position& position,
Session& session) {
auto path_id = session.path_id;

View File

@@ -8,13 +8,13 @@
#include <vector>
#include "command/command.h"
#include "server/session.h"
#include "server/worker_pool.h"
#include "server/workspace.h"
#include "server/service/session.h"
#include "server/worker/worker_pool.h"
#include "server/workspace/workspace.h"
#include "syntax/completion.h"
#include "kota/async/async.h"
#include "kota/codec/raw_value.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/peer.h"
@@ -50,10 +50,14 @@ std::string uri_to_path(const std::string& uri);
class Compiler {
public:
Compiler(kota::event_loop& loop,
kota::ipc::JsonPeer& peer,
Workspace& workspace,
WorkerPool& pool,
llvm::DenseMap<std::uint32_t, Session>& sessions);
void set_peer(kota::ipc::JsonPeer* p) {
peer = p;
}
~Compiler();
void init_compile_graph();
@@ -86,6 +90,9 @@ public:
const protocol::Position& position,
Session& session);
/// Forward a formatting request to a stateless worker.
RawResult forward_format(Session& session, std::optional<protocol::Range> range = {});
/// Handle completion requests. Detects preamble context (include/import)
/// and serves those locally; delegates code completion to a stateless worker.
RawResult handle_completion(const protocol::Position& position, Session& session);
@@ -96,7 +103,12 @@ public:
/// Callback invoked when indexing should be scheduled.
std::function<void()> on_indexing_needed;
/// Cancel in-flight compile tasks and wait for them to finish.
kota::task<> stop();
private:
kota::task<> run_compile(std::uint32_t path_id, std::shared_ptr<Session::PendingCompile> pc);
kota::task<bool> ensure_deps(Session& session,
const std::string& directory,
const std::vector<std::string>& arguments,
@@ -125,10 +137,11 @@ private:
private:
kota::event_loop& loop;
kota::ipc::JsonPeer& peer;
kota::ipc::JsonPeer* peer = nullptr;
Workspace& workspace;
WorkerPool& pool;
llvm::DenseMap<std::uint32_t, Session>& sessions;
kota::task_group<> compile_tasks{loop};
};
} // namespace clice

View File

@@ -1,14 +1,15 @@
#include "server/indexer.h"
#include "server/compiler/indexer.h"
#include <algorithm>
#include <string>
#include <variant>
#include <vector>
#include "index/tu_index.h"
#include "server/compiler.h"
#include "server/protocol.h"
#include "server/session.h"
#include "server/worker_pool.h"
#include "server/compiler/compiler.h"
#include "server/protocol/worker.h"
#include "server/service/session.h"
#include "server/worker/worker_pool.h"
#include "support/filesystem.h"
#include "support/logging.h"
@@ -25,7 +26,7 @@ namespace clice {
namespace lsp = kota::ipc::lsp;
void Indexer::merge(const void* tu_index_data, std::size_t size) {
auto tu_index = index::TUIndex::from(tu_index_data, size);
auto tu_index = index::TUIndex::from(tu_index_data);
if(tu_index.graph.paths.empty()) {
LOG_WARN("Ignoring TUIndex with empty path graph");
return;
@@ -144,8 +145,7 @@ void Indexer::load(llvm::StringRef index_dir) {
auto project_path = path::join(index_dir, "project.idx");
auto buf = llvm::MemoryBuffer::getFile(project_path);
if(buf) {
workspace.project_index =
index::ProjectIndex::from((*buf)->getBufferStart(), (*buf)->getBufferSize());
workspace.project_index = index::ProjectIndex::from((*buf)->getBufferStart());
LOG_INFO("Loaded ProjectIndex: {} symbols", workspace.project_index.symbols.size());
}
@@ -447,6 +447,152 @@ std::optional<SymbolInfo> Indexer::resolve_symbol(index::SymbolHash hash) {
return SymbolInfo{hash, std::move(name), kind, def_loc->uri, def_loc->range};
}
static std::string extract_line(llvm::StringRef content, std::uint32_t offset) {
if(content.empty() || offset >= content.size())
return {};
std::size_t line_start = 0;
if(offset > 0) {
auto pos = content.rfind('\n', offset - 1);
if(pos != llvm::StringRef::npos)
line_start = pos + 1;
}
auto line_end = content.find('\n', offset);
if(line_end == llvm::StringRef::npos)
line_end = content.size();
return content.slice(line_start, line_end).str();
}
std::optional<Indexer::DefinitionText> Indexer::get_definition_text(index::SymbolHash hash) {
for(auto& [id, sess]: sessions) {
if(!sess.file_index || !sess.file_index->mapper)
continue;
auto it = sess.file_index->file_index.relations.find(hash);
if(it == sess.file_index->file_index.relations.end())
continue;
for(auto& rel: it->second) {
if(rel.kind.value() != RelationKind::Definition)
continue;
auto def_range = std::bit_cast<LocalSourceRange>(rel.target_symbol);
if(def_range.begin >= def_range.end)
continue;
llvm::StringRef content = sess.file_index->content;
if(def_range.end > content.size())
continue;
auto start = sess.file_index->mapper->to_position(def_range.begin);
auto end = sess.file_index->mapper->to_position(def_range.end);
if(!start || !end)
continue;
return DefinitionText{
.file = std::string(workspace.path_pool.resolve(id)),
.start_line = static_cast<int>(start->line) + 1,
.end_line = static_cast<int>(end->line) + 1,
.text =
std::string(content.substr(def_range.begin, def_range.end - def_range.begin)),
};
}
}
auto sym_it = workspace.project_index.symbols.find(hash);
if(sym_it == workspace.project_index.symbols.end())
return std::nullopt;
for(auto file_id: sym_it->second.reference_files) {
if(is_proj_path_open(file_id))
continue;
auto shard_it = workspace.merged_indices.find(file_id);
if(shard_it == workspace.merged_indices.end())
continue;
auto* m = shard_it->second.mapper();
if(!m)
continue;
auto content = shard_it->second.index.content();
std::optional<DefinitionText> result;
shard_it->second.index.lookup(
hash,
RelationKind::Definition,
[&](const index::Relation& r) {
auto def_range = std::bit_cast<LocalSourceRange>(r.target_symbol);
if(def_range.begin >= def_range.end || def_range.end > content.size())
return true;
auto start = m->to_position(def_range.begin);
auto end = m->to_position(def_range.end);
if(!start || !end)
return true;
result = DefinitionText{
.file = workspace.project_index.path_pool.path(file_id).str(),
.start_line = static_cast<int>(start->line) + 1,
.end_line = static_cast<int>(end->line) + 1,
.text = std::string(
content.substr(def_range.begin, def_range.end - def_range.begin)),
};
return false;
});
if(result)
return result;
}
return std::nullopt;
}
std::vector<Indexer::ReferenceWithContext> Indexer::collect_references(index::SymbolHash hash,
RelationKind kind) {
std::vector<ReferenceWithContext> results;
auto sym_it = workspace.project_index.symbols.find(hash);
if(sym_it != workspace.project_index.symbols.end()) {
for(auto file_id: sym_it->second.reference_files) {
if(is_proj_path_open(file_id))
continue;
auto shard_it = workspace.merged_indices.find(file_id);
if(shard_it == workspace.merged_indices.end())
continue;
auto* m = shard_it->second.mapper();
if(!m)
continue;
auto content = shard_it->second.index.content();
auto file_path = workspace.project_index.path_pool.path(file_id);
shard_it->second.index.lookup(hash, kind, [&](const index::Relation& r) {
auto start = m->to_position(r.range.begin);
if(!start)
return true;
results.push_back(ReferenceWithContext{
.file = file_path.str(),
.line = static_cast<int>(start->line) + 1,
.context = extract_line(content, r.range.begin),
});
return true;
});
}
}
for(auto& [id, sess]: sessions) {
if(!sess.file_index || !sess.file_index->mapper)
continue;
auto it = sess.file_index->file_index.relations.find(hash);
if(it == sess.file_index->file_index.relations.end())
continue;
auto file_path = workspace.path_pool.resolve(id);
llvm::StringRef content = sess.file_index->content;
for(auto& rel: it->second) {
if(rel.kind != kind)
continue;
auto start = sess.file_index->mapper->to_position(rel.range.begin);
if(!start)
continue;
results.push_back(ReferenceWithContext{
.file = file_path.str(),
.line = static_cast<int>(start->line) + 1,
.context = extract_line(content, rel.range.begin),
});
}
}
return results;
}
std::vector<protocol::CallHierarchyIncomingCall>
Indexer::find_incoming_calls(index::SymbolHash hash) {
llvm::DenseMap<index::SymbolHash, std::vector<protocol::Range>> caller_ranges;
@@ -625,6 +771,28 @@ void Indexer::enqueue(std::uint32_t server_path_id) {
index_queue.push_back(server_path_id);
}
void Indexer::pause_indexing() {
++pause_depth;
if(pause_depth == 1) {
resume_event.reset();
LOG_DEBUG("Background indexing paused");
}
}
void Indexer::resume_indexing() {
if(pause_depth > 0)
--pause_depth;
if(pause_depth == 0) {
resume_event.set();
LOG_DEBUG("Background indexing resumed");
}
}
kota::task<> Indexer::stop() {
bg_tasks.cancel();
co_await bg_tasks.join();
}
void Indexer::schedule() {
if(!*workspace.config.project.enable_indexing || indexing_active || indexing_scheduled)
return;
@@ -634,7 +802,77 @@ void Indexer::schedule() {
index_idle_timer = std::make_shared<kota::timer>(kota::timer::create(loop));
}
index_idle_timer->start(std::chrono::milliseconds(*workspace.config.project.idle_timeout_ms));
loop.schedule(run_background_indexing());
if(!bg_tasks.spawn(run_background_indexing())) {
indexing_scheduled = false;
LOG_WARN("Failed to spawn background indexing task (task group stopped)");
}
}
kota::task<> Indexer::index_one(std::uint32_t server_path_id) {
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
if(sessions.contains(server_path_id))
co_return;
if(!need_update(file_path))
co_return;
// For module interface units, compile their PCM (and transitive deps)
// first so the stateless worker has the artifacts it needs.
if(workspace.compile_graph && workspace.path_to_module.contains(server_path_id)) {
co_await workspace.compile_graph->compile(server_path_id);
}
worker::BuildParams params;
params.kind = worker::BuildKind::Index;
params.file = file_path;
if(!compiler.fill_compile_args(file_path, params.directory, params.arguments, nullptr))
co_return;
workspace.fill_pcm_deps(params.pcms);
LOG_INFO("Background indexing: {}", file_path);
auto result = co_await pool.send_stateless(params);
if(result.has_value() && result.value().success && !result.value().tu_index_data.empty()) {
LOG_INFO("Background indexing got TUIndex for {}: {} bytes",
file_path,
result.value().tu_index_data.size());
merge(result.value().tu_index_data.data(), result.value().tu_index_data.size());
} else if(result.has_value() && !result.value().success) {
LOG_WARN("Background index failed for {}: {}", file_path, result.value().error);
} else if(result.has_value() && result.value().tu_index_data.empty()) {
LOG_WARN("Background index returned empty TUIndex for {}", file_path);
} else {
LOG_WARN("Background index IPC error for {}: {}", file_path, result.error().message);
}
}
kota::task<> Indexer::monitor_resources() {
while(true) {
co_await kota::sleep(std::chrono::milliseconds(3000));
auto mem = kota::sys::memory();
if(mem.total == 0)
continue;
auto effective_total =
(mem.constrained > 0 && mem.constrained < mem.total) ? mem.constrained : mem.total;
auto ratio = static_cast<double>(mem.available) / static_cast<double>(effective_total);
if(ratio < 0.15 && max_concurrent > 1) {
--max_concurrent;
LOG_INFO("Index concurrency -> {} (memory pressure: {:.0f}% available)",
max_concurrent,
ratio * 100);
} else if(ratio > 0.30 && max_concurrent < baseline_concurrent) {
++max_concurrent;
LOG_DEBUG("Index concurrency -> {} (memory OK: {:.0f}% available)",
max_concurrent,
ratio * 100);
}
}
}
kota::task<> Indexer::run_background_indexing() {
@@ -649,48 +887,74 @@ kota::task<> Indexer::run_background_indexing() {
}
indexing_active = true;
std::size_t processed = 0;
while(index_queue_pos < index_queue.size()) {
auto server_path_id = index_queue[index_queue_pos];
index_queue_pos++;
kota::cancellation_source monitor_cancel;
bg_tasks.spawn(kota::with_token(monitor_resources(), monitor_cancel.token()));
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
std::stable_partition(
index_queue.begin() + index_queue_pos,
index_queue.end(),
[this](std::uint32_t id) { return workspace.path_to_module.contains(id); });
if(sessions.contains(server_path_id))
continue;
auto total = index_queue.size() - index_queue_pos;
std::size_t dispatched = 0;
std::size_t completed = 0;
if(!need_update(file_path))
continue;
worker::BuildParams params;
params.kind = worker::BuildKind::Index;
params.file = file_path;
if(!compiler.fill_compile_args(file_path, params.directory, params.arguments, nullptr))
continue;
workspace.fill_pcm_deps(params.pcms);
LOG_INFO("Background indexing: {}", file_path);
auto result = co_await pool.send_stateless(params);
if(result.has_value() && result.value().success && !result.value().tu_index_data.empty()) {
LOG_INFO("Background indexing got TUIndex for {}: {} bytes",
file_path,
result.value().tu_index_data.size());
merge(result.value().tu_index_data.data(), result.value().tu_index_data.size());
++processed;
} else if(result.has_value() && !result.value().success) {
LOG_WARN("Background index failed for {}: {}", file_path, result.value().error);
} else if(result.has_value() && result.value().tu_index_data.empty()) {
LOG_WARN("Background index returned empty TUIndex for {}", file_path);
std::optional<lsp::ProgressReporter<kota::ipc::JsonPeer>> progress;
if(peer) {
progress.emplace(*peer, protocol::ProgressToken(std::string("clice/backgroundIndex")));
auto create_result = co_await progress->create();
if(!create_result.has_error()) {
progress->begin("Indexing", std::format("0/{} files", total), 0);
} else {
LOG_WARN("Background index IPC error for {}: {}", file_path, result.error().message);
progress.reset();
}
}
kota::task_group<> workers(loop);
std::size_t in_flight = 0;
kota::event slot_available;
while(index_queue_pos < index_queue.size()) {
if(pause_depth > 0)
co_await resume_event.wait();
auto server_path_id = index_queue[index_queue_pos++];
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
if(sessions.contains(server_path_id) || !need_update(file_path)) {
++completed;
continue;
}
while(in_flight >= max_concurrent) {
slot_available.reset();
co_await slot_available.wait();
}
++in_flight;
++dispatched;
workers.spawn([&, server_path_id]() -> kota::task<> {
co_await index_one(server_path_id);
--in_flight;
++completed;
if(progress) {
auto pct = total > 0 ? static_cast<std::uint32_t>(completed * 100 / total) : 100;
progress->report(std::format("{}/{} files", completed, total), pct);
}
slot_available.set();
}());
}
co_await workers.join();
if(progress) {
progress->end(std::format("Indexed {} files", dispatched));
}
monitor_cancel.cancel();
indexing_active = false;
LOG_INFO("Background indexing complete: {} files processed", processed);
LOG_INFO("Background indexing complete: {} files dispatched", dispatched);
save(workspace.config.project.index_dir);
}

View File

@@ -9,10 +9,12 @@
#include "semantic/relation_kind.h"
#include "semantic/symbol_kind.h"
#include "server/workspace.h"
#include "server/workspace/workspace.h"
#include "kota/async/async.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/lsp/position.h"
#include "kota/ipc/lsp/progress.h"
#include "kota/ipc/lsp/protocol.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/SmallVector.h"
@@ -59,8 +61,49 @@ public:
WorkerPool& pool,
Compiler& compiler,
std::function<bool(std::uint32_t)> is_file_open = {}) :
loop(loop), workspace(workspace), sessions(sessions), pool(pool), compiler(compiler),
is_file_open(std::move(is_file_open)) {}
loop(loop), bg_tasks(loop), workspace(workspace), sessions(sessions), pool(pool),
compiler(compiler), is_file_open(std::move(is_file_open)) {}
/// Set the LSP peer for progress reporting. Must be called before
/// schedule() if progress notifications are desired.
void set_peer(kota::ipc::JsonPeer* p) {
peer = p;
}
/// Temporarily pause background indexing to give priority to user
/// requests. Indexing tasks already dispatched to workers continue,
/// but no new tasks will be sent until resume_indexing() is called.
void pause_indexing();
/// Resume background indexing after a pause.
void resume_indexing();
/// RAII guard that pauses indexing for its lifetime.
struct [[nodiscard]] ScopedPause {
Indexer& indexer;
explicit ScopedPause(Indexer& idx) : indexer(idx) {
indexer.pause_indexing();
}
~ScopedPause() {
indexer.resume_indexing();
}
ScopedPause(const ScopedPause&) = delete;
ScopedPause& operator=(const ScopedPause&) = delete;
};
ScopedPause scoped_pause() {
return ScopedPause{*this};
}
/// Set the maximum number of concurrent index tasks.
/// Also sets the baseline that dynamic adjustment will restore to.
void set_max_concurrency(std::size_t n) {
max_concurrent = std::max<std::size_t>(n, 1);
baseline_concurrent = max_concurrent;
}
/// Add a file to the background indexing queue.
void enqueue(std::uint32_t server_path_id);
@@ -124,6 +167,43 @@ public:
std::vector<protocol::SymbolInformation> search_symbols(llvm::StringRef query,
std::size_t max_results = 100);
struct DefinitionText {
std::string file;
int start_line;
int end_line;
std::string text;
};
/// Get full definition text for a symbol, using stored index ranges and content.
std::optional<DefinitionText> get_definition_text(index::SymbolHash hash);
struct ReferenceWithContext {
std::string file;
int line;
std::string context;
};
/// Collect references (or definitions) with context lines from stored content.
std::vector<ReferenceWithContext> collect_references(index::SymbolHash hash, RelationKind kind);
/// Cancel background indexing and wait for all tasks to settle.
kota::task<> stop();
/// Whether background indexing is currently idle (no active or queued work).
bool is_idle() const {
return !indexing_active && index_queue_pos >= index_queue.size();
}
/// Number of files remaining in the indexing queue.
std::size_t pending_files() const {
return index_queue_pos < index_queue.size() ? index_queue.size() - index_queue_pos : 0;
}
/// Total files that were enqueued in the current (or last) indexing round.
std::size_t total_queued() const {
return index_queue.size();
}
/// Convert internal SymbolKind to LSP SymbolKind.
static protocol::SymbolKind to_lsp_symbol_kind(SymbolKind kind);
@@ -165,6 +245,7 @@ private:
private:
kota::event_loop& loop;
kota::task_group<> bg_tasks;
Workspace& workspace;
llvm::DenseMap<std::uint32_t, Session>& sessions;
WorkerPool& pool;
@@ -175,6 +256,9 @@ private:
/// server-path-id-keyed sessions map to project-level path_ids.
std::function<bool(std::uint32_t)> is_file_open;
/// LSP peer for progress reporting (optional, not owned).
kota::ipc::JsonPeer* peer = nullptr;
/// Background indexing queue and scheduling state.
std::vector<std::uint32_t> index_queue;
std::size_t index_queue_pos = 0;
@@ -182,7 +266,18 @@ private:
bool indexing_scheduled = false;
std::shared_ptr<kota::timer> index_idle_timer;
/// Concurrency control for background indexing.
std::size_t max_concurrent = 2;
std::size_t baseline_concurrent = 2;
/// Pause/resume: when paused, new index tasks wait on this event.
/// Uses a counter so nested pause/resume pairs work correctly.
std::size_t pause_depth = 0;
kota::event resume_event{true};
kota::task<> run_background_indexing();
kota::task<> index_one(std::uint32_t server_path_id);
kota::task<> monitor_resources();
};
} // namespace clice

View File

@@ -1,81 +0,0 @@
#pragma once
#include <cstdint>
#include <memory>
#include <string>
#include <vector>
#include "server/compiler.h"
#include "server/indexer.h"
#include "server/session.h"
#include "server/worker_pool.h"
#include "server/workspace.h"
#include "kota/async/async.h"
#include "kota/codec/raw_value.h"
#include "kota/ipc/peer.h"
#include "llvm/ADT/DenseMap.h"
namespace clice {
enum class ServerLifecycle : std::uint8_t {
Uninitialized,
Initialized,
Ready,
ShuttingDown,
Exited,
};
/// Top-level LSP server — the single orchestration point for the language
/// server process.
///
/// Responsibilities:
/// - Owns the two-layer state model: Workspace (disk truth) and Sessions
/// (per-open-file volatile state).
/// - Manages Session lifecycle directly: didOpen creates, didChange mutates,
/// didSave syncs to Workspace, didClose destroys.
/// - Dispatches compilation and feature queries to Compiler.
/// - Dispatches index lookups and background indexing to Indexer.
///
/// Design principle:
/// Open files are never depended upon by other files. Dependencies always
/// point to disk files. The only path from Session to Workspace is didSave.
class MasterServer {
public:
MasterServer(kota::event_loop& loop, kota::ipc::JsonPeer& peer, std::string self_path);
~MasterServer();
void register_handlers();
private:
kota::event_loop& loop;
kota::ipc::JsonPeer& peer;
/// Persistent project-wide state (config, CDB, path pool, dependency
/// graphs, compilation caches, symbol index).
Workspace workspace;
/// Per-file editing sessions, keyed by server-level path_id.
llvm::DenseMap<std::uint32_t, Session> sessions;
/// Worker process pool for offloading compilation and queries.
WorkerPool pool;
/// Compilation lifecycle manager (reads/writes workspace and sessions).
Compiler compiler;
/// Index query and background scheduling (reads from workspace and sessions).
Indexer indexer;
ServerLifecycle lifecycle = ServerLifecycle::Uninitialized;
std::string self_path;
std::string workspace_root;
std::string session_log_dir;
std::string init_options_json; ///< Raw JSON from initializationOptions, consumed once.
kota::task<> load_workspace();
using RawResult = kota::task<kota::codec::RawValue, kota::ipc::Error>;
};
} // namespace clice

View File

@@ -0,0 +1,297 @@
#pragma once
#include <cstdint>
#include <optional>
#include <string>
#include <vector>
#include "kota/ipc/protocol.h"
namespace clice::agentic {
struct CompileCommandParams {
std::string path;
};
struct CompileCommandResult {
std::string file;
std::string directory;
std::vector<std::string> arguments;
};
struct FileInfo {
std::string path;
std::string kind;
std::optional<std::string> module_name;
};
struct ProjectFilesParams {
std::optional<std::string> filter;
};
struct ProjectFilesResult {
std::vector<FileInfo> files;
int total = 0;
};
struct DepEntry {
std::string path;
int depth = 0;
};
struct FileDepsParams {
std::string path;
std::optional<std::string> direction;
std::optional<int> depth;
};
struct FileDepsResult {
std::string file;
std::vector<DepEntry> includes;
std::vector<DepEntry> includers;
};
struct ImpactAnalysisParams {
std::string path;
};
struct ImpactAnalysisResult {
std::vector<std::string> direct_dependents;
std::vector<std::string> transitive_dependents;
std::vector<std::string> affected_modules;
};
struct SymbolEntry {
std::string name;
std::string kind;
std::string file;
int line = 0;
std::optional<std::string> container;
std::uint64_t symbol_id = 0;
};
struct SymbolSearchParams {
std::string query;
std::optional<std::vector<std::string>> kind_filter;
std::optional<int> max_results;
};
struct SymbolSearchResult {
std::vector<SymbolEntry> symbols;
};
struct ReadSymbolParams {
std::optional<std::string> name;
std::optional<std::string> path;
std::optional<int> line;
std::optional<std::uint64_t> symbol_id;
};
struct ReadSymbolResult {
std::string name;
std::string kind;
std::string file;
int start_line = 0;
int end_line = 0;
std::string text;
std::optional<std::string> signature;
std::uint64_t symbol_id = 0;
};
struct DocumentSymbolEntry {
std::string name;
std::string kind;
int start_line = 0;
int end_line = 0;
std::uint64_t symbol_id = 0;
};
struct DocumentSymbolsParams {
std::string path;
};
struct DocumentSymbolsResult {
std::vector<DocumentSymbolEntry> symbols;
};
struct DefinitionParams {
std::optional<std::string> name;
std::optional<std::string> path;
std::optional<int> line;
std::optional<std::uint64_t> symbol_id;
};
struct LocationEntry {
std::string file;
int start_line = 0;
int end_line = 0;
std::string text;
};
struct DefinitionResult {
std::string name;
std::string kind;
std::uint64_t symbol_id = 0;
std::optional<LocationEntry> definition;
};
struct ReferenceEntry {
std::string file;
int line = 0;
std::string context;
};
struct ReferencesParams {
std::optional<std::string> name;
std::optional<std::string> path;
std::optional<int> line;
std::optional<std::uint64_t> symbol_id;
std::optional<bool> include_declaration;
};
struct ReferencesResult {
std::string name;
std::string kind;
std::uint64_t symbol_id = 0;
std::vector<ReferenceEntry> references;
int total = 0;
};
struct CallGraphEntry {
std::string name;
std::string kind;
std::string file;
int line = 0;
std::uint64_t symbol_id = 0;
};
struct CallGraphParams {
std::optional<std::string> name;
std::optional<std::string> path;
std::optional<int> line;
std::optional<std::uint64_t> symbol_id;
std::optional<std::string> direction;
std::optional<int> depth;
};
struct CallGraphResult {
CallGraphEntry root;
std::vector<CallGraphEntry> callers;
std::vector<CallGraphEntry> callees;
};
struct TypeHierarchyEntry {
std::string name;
std::string kind;
std::string file;
int line = 0;
std::uint64_t symbol_id = 0;
};
struct TypeHierarchyParams {
std::optional<std::string> name;
std::optional<std::string> path;
std::optional<int> line;
std::optional<std::uint64_t> symbol_id;
std::optional<std::string> direction;
};
struct TypeHierarchyResult {
TypeHierarchyEntry root;
std::vector<TypeHierarchyEntry> supertypes;
std::vector<TypeHierarchyEntry> subtypes;
};
struct StatusParams {};
struct StatusResult {
bool idle = true;
int pending = 0;
int total = 0;
int indexed = 0;
};
struct ShutdownParams {};
} // namespace clice::agentic
namespace kota::ipc::protocol {
template <>
struct RequestTraits<clice::agentic::CompileCommandParams> {
using Result = clice::agentic::CompileCommandResult;
constexpr inline static std::string_view method = "agentic/compileCommand";
};
template <>
struct RequestTraits<clice::agentic::ProjectFilesParams> {
using Result = clice::agentic::ProjectFilesResult;
constexpr inline static std::string_view method = "agentic/projectFiles";
};
template <>
struct RequestTraits<clice::agentic::FileDepsParams> {
using Result = clice::agentic::FileDepsResult;
constexpr inline static std::string_view method = "agentic/fileDeps";
};
template <>
struct RequestTraits<clice::agentic::ImpactAnalysisParams> {
using Result = clice::agentic::ImpactAnalysisResult;
constexpr inline static std::string_view method = "agentic/impactAnalysis";
};
template <>
struct RequestTraits<clice::agentic::SymbolSearchParams> {
using Result = clice::agentic::SymbolSearchResult;
constexpr inline static std::string_view method = "agentic/symbolSearch";
};
template <>
struct RequestTraits<clice::agentic::ReadSymbolParams> {
using Result = clice::agentic::ReadSymbolResult;
constexpr inline static std::string_view method = "agentic/readSymbol";
};
template <>
struct RequestTraits<clice::agentic::DocumentSymbolsParams> {
using Result = clice::agentic::DocumentSymbolsResult;
constexpr inline static std::string_view method = "agentic/documentSymbols";
};
template <>
struct RequestTraits<clice::agentic::DefinitionParams> {
using Result = clice::agentic::DefinitionResult;
constexpr inline static std::string_view method = "agentic/definition";
};
template <>
struct RequestTraits<clice::agentic::ReferencesParams> {
using Result = clice::agentic::ReferencesResult;
constexpr inline static std::string_view method = "agentic/references";
};
template <>
struct RequestTraits<clice::agentic::CallGraphParams> {
using Result = clice::agentic::CallGraphResult;
constexpr inline static std::string_view method = "agentic/callGraph";
};
template <>
struct RequestTraits<clice::agentic::TypeHierarchyParams> {
using Result = clice::agentic::TypeHierarchyResult;
constexpr inline static std::string_view method = "agentic/typeHierarchy";
};
template <>
struct RequestTraits<clice::agentic::StatusParams> {
using Result = clice::agentic::StatusResult;
constexpr inline static std::string_view method = "agentic/status";
};
template <>
struct NotificationTraits<clice::agentic::ShutdownParams> {
constexpr inline static std::string_view method = "agentic/shutdown";
};
} // namespace kota::ipc::protocol

View File

@@ -0,0 +1,42 @@
#pragma once
#include <optional>
#include <string>
#include <vector>
namespace clice::ext {
struct ContextItem {
std::string label;
std::string description;
std::string uri;
};
struct QueryContextParams {
std::string uri;
std::optional<int> offset;
};
struct QueryContextResult {
std::vector<ContextItem> contexts;
int total = 0;
};
struct CurrentContextParams {
std::string uri;
};
struct CurrentContextResult {
std::optional<ContextItem> context;
};
struct SwitchContextParams {
std::string uri;
std::string context_uri;
};
struct SwitchContextResult {
bool success = false;
};
} // namespace clice::ext

View File

@@ -1,7 +1,6 @@
#pragma once
#include <cstdint>
#include <optional>
#include <string>
#include <unordered_map>
#include <utility>
@@ -9,8 +8,7 @@
#include "syntax/token.h"
#include "kota/codec/raw_value.h"
#include "kota/ipc/lsp/protocol.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/protocol.h"
namespace clice::worker {
@@ -66,6 +64,7 @@ enum class BuildKind : uint8_t {
Index,
Completion,
SignatureHelp,
Format,
};
/// Unified parameters for all stateless build/compilation tasks.
@@ -76,6 +75,7 @@ enum class BuildKind : uint8_t {
/// - Index: + pcms
/// - Completion: + text, version, offset, pch, pcms
/// - SignatureHelp: + text, version, offset, pch, pcms
/// - Format: + text, format_range (optional)
struct BuildParams {
BuildKind kind;
std::string file;
@@ -92,6 +92,7 @@ struct BuildParams {
std::string output_path; ///< BuildPCH, BuildPCM
std::string module_name; ///< BuildPCM
uint32_t preamble_bound = UINT32_MAX; ///< BuildPCH
LocalSourceRange format_range; ///< Format (default = full document)
};
/// Unified result for stateless build tasks.
@@ -122,43 +123,6 @@ struct EvictedParams {
} // namespace clice::worker
namespace clice::ext {
struct ContextItem {
std::string label;
std::string description;
std::string uri;
};
struct QueryContextParams {
std::string uri;
std::optional<int> offset;
};
struct QueryContextResult {
std::vector<ContextItem> contexts;
int total;
};
struct CurrentContextParams {
std::string uri;
};
struct CurrentContextResult {
std::optional<ContextItem> context;
};
struct SwitchContextParams {
std::string uri;
std::string context_uri;
};
struct SwitchContextResult {
bool success;
};
} // namespace clice::ext
namespace kota::ipc::protocol {
template <>

View File

@@ -0,0 +1,787 @@
#include "server/service/agent_client.h"
#include <algorithm>
#include <format>
#include <ranges>
#include <string>
#include <vector>
#include "server/protocol/agentic.h"
#include "server/service/master_server.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "kota/ipc/lsp/uri.h"
#include "kota/meta/enum.h"
#include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/SmallVector.h"
namespace clice {
using kota::ipc::RequestResult;
using RequestContext = kota::ipc::JsonPeer::RequestContext;
namespace lsp = kota::ipc::lsp;
namespace protocol = kota::ipc::protocol;
static std::string_view symbol_kind_name(SymbolKind kind) {
constexpr auto names = kota::meta::reflection<SymbolKind::Kind>::member_names;
auto idx = static_cast<std::size_t>(kind.value());
if(idx < names.size())
return names[idx];
return "Unknown";
}
struct ResolvedSymbol {
index::SymbolHash hash = 0;
std::string name;
SymbolKind kind;
std::string file;
int line = 0;
};
static std::vector<ResolvedSymbol> resolve_locator(const agentic::ReadSymbolParams& loc,
Workspace& workspace,
llvm::DenseMap<std::uint32_t, Session>& sessions,
Indexer& indexer) {
if(loc.symbol_id.has_value() && *loc.symbol_id != 0) {
auto hash = static_cast<index::SymbolHash>(*loc.symbol_id);
std::string name;
SymbolKind kind;
if(!indexer.find_symbol_info(hash, name, kind))
return {};
auto def_loc = indexer.find_definition_location(hash);
if(!def_loc)
return {};
auto file = uri_to_path(def_loc->uri);
int line_num = static_cast<int>(def_loc->range.start.line) + 1;
return {
{hash, std::move(name), kind, std::move(file), line_num}
};
}
if(loc.name.has_value() && !loc.name->empty()) {
std::string query_lower = llvm::StringRef(*loc.name).lower();
std::vector<ResolvedSymbol> candidates;
std::vector<ResolvedSymbol> exact_matches;
llvm::DenseSet<index::SymbolHash> seen;
auto try_symbol = [&](index::SymbolHash hash, const index::Symbol& symbol) {
if(symbol.name.empty())
return;
if(llvm::StringRef(symbol.name).lower().find(query_lower) == std::string::npos)
return;
auto def_loc = indexer.find_definition_location(hash);
if(!def_loc)
return;
if(!seen.insert(hash).second)
return;
auto file = uri_to_path(def_loc->uri);
int line_num = static_cast<int>(def_loc->range.start.line) + 1;
if(loc.path.has_value() && !loc.path->empty()) {
llvm::StringRef wanted(*loc.path);
bool basename_only = wanted.find_last_of("/\\") == llvm::StringRef::npos;
if(basename_only) {
if(llvm::sys::path::filename(file) != wanted)
return;
} else if(!llvm::StringRef(file).ends_with(wanted)) {
return;
}
}
bool is_exact = llvm::StringRef(symbol.name).lower() == query_lower ||
llvm::StringRef(symbol.name).ends_with("::" + *loc.name);
ResolvedSymbol rs{hash, symbol.name, symbol.kind, std::move(file), line_num};
if(is_exact)
exact_matches.push_back(std::move(rs));
else
candidates.push_back(std::move(rs));
};
for(auto& [hash, symbol]: workspace.project_index.symbols)
try_symbol(hash, symbol);
for(auto& [_, sess]: sessions) {
if(!sess.file_index)
continue;
for(auto& [hash, symbol]: sess.file_index->symbols)
try_symbol(hash, symbol);
}
if(!exact_matches.empty())
return exact_matches;
return candidates;
}
if(loc.path.has_value() && loc.line.has_value()) {
auto path_str = *loc.path;
auto target_line = static_cast<protocol::uinteger>(*loc.line - 1);
auto pool_it = workspace.path_pool.cache.find(path_str);
auto server_id = pool_it != workspace.path_pool.cache.end() ? pool_it->second : ~0u;
auto* sess =
server_id != ~0u && sessions.contains(server_id) ? &sessions[server_id] : nullptr;
if(sess && sess->file_index) {
auto& fi = *sess->file_index;
if(fi.mapper) {
for(auto& [hash, rels]: fi.file_index.relations) {
for(auto& rel: rels) {
if(rel.kind.value() != RelationKind::Definition)
continue;
auto start = fi.mapper->to_position(rel.range.begin);
if(start && start->line == target_line) {
std::string name;
SymbolKind kind;
if(indexer.find_symbol_info(hash, name, kind))
return {
{hash, std::move(name), kind, path_str, *loc.line}
};
}
}
}
}
}
auto it = workspace.project_index.path_pool.find(path_str);
if(it == workspace.project_index.path_pool.cache.end())
return {};
auto proj_id = it->second;
auto shard_it = workspace.merged_indices.find(proj_id);
if(shard_it == workspace.merged_indices.end())
return {};
for(auto& [hash, symbol]: workspace.project_index.symbols) {
if(!symbol.reference_files.contains(proj_id))
continue;
bool found = false;
shard_it->second.find_relations(hash,
RelationKind::Definition,
[&](const index::Relation&, protocol::Range range) {
if(range.start.line == target_line) {
found = true;
return false;
}
return true;
});
if(found)
return {
{hash, symbol.name, symbol.kind, path_str, *loc.line}
};
}
return {};
}
return {};
}
static std::uint64_t extract_symbol_id(const std::optional<protocol::LSPAny>& data) {
if(!data.has_value())
return 0;
if(auto* val = std::get_if<std::int64_t>(&static_cast<const protocol::LSPVariant&>(*data)))
return static_cast<std::uint64_t>(*val);
LOG_WARN("extract_symbol_id: unexpected LSPAny variant type");
return 0;
}
AgentClient::AgentClient(MasterServer& server, kota::ipc::JsonPeer& peer) :
server(server), peer(peer) {
using namespace agentic;
auto& srv = this->server;
peer.on_request(
[&srv](RequestContext&,
const CompileCommandParams& params) -> RequestResult<CompileCommandParams> {
std::string directory;
std::vector<std::string> arguments;
if(!srv.compiler.fill_compile_args(params.path, directory, arguments)) {
co_return kota::outcome_error(
kota::ipc::Error{std::format("no compile command found for {}", params.path)});
}
co_return CompileCommandResult{
.file = params.path,
.directory = std::move(directory),
.arguments = std::move(arguments),
};
});
peer.on_request([&srv](RequestContext&,
const ProjectFilesParams& params) -> RequestResult<ProjectFilesParams> {
auto& ws = srv.workspace;
auto filter = params.filter.value_or("all");
ProjectFilesResult result;
llvm::DenseSet<std::uint32_t> seen;
for(auto& entry: ws.cdb.get_entries()) {
auto file_path = ws.cdb.resolve_path(entry.file);
if(file_path.empty())
continue;
auto proj_it = ws.project_index.path_pool.find(file_path);
if(proj_it != ws.project_index.path_pool.cache.end()) {
if(!seen.insert(proj_it->second).second)
continue;
}
std::string kind_str;
auto mod_it = ws.path_to_module.find(ws.path_pool.intern(file_path));
if(mod_it != ws.path_to_module.end()) {
kind_str = "module";
} else {
auto ext = llvm::sys::path::extension(file_path);
if(ext == ".h" || ext == ".hpp" || ext == ".hxx" || ext == ".hh")
kind_str = "header";
else
kind_str = "source";
}
if(filter != "all" && filter != kind_str)
continue;
FileInfo fi;
fi.path = file_path.str();
fi.kind = std::move(kind_str);
if(mod_it != ws.path_to_module.end())
fi.module_name = mod_it->second;
result.files.push_back(std::move(fi));
}
if(filter == "all" || filter == "header") {
for(auto& [path_id, shard]: ws.merged_indices) {
if(seen.contains(path_id))
continue;
auto path_str = ws.project_index.path_pool.path(path_id);
auto ext = llvm::sys::path::extension(path_str);
if(ext == ".h" || ext == ".hpp" || ext == ".hxx" || ext == ".hh") {
seen.insert(path_id);
result.files.push_back(FileInfo{
.path = path_str.str(),
.kind = "header",
});
}
}
}
result.total = static_cast<int>(result.files.size());
co_return result;
});
peer.on_request(
[&srv](RequestContext&, const FileDepsParams& params) -> RequestResult<FileDepsParams> {
auto& ws = srv.workspace;
auto pool_it = ws.path_pool.cache.find(params.path);
if(pool_it == ws.path_pool.cache.end())
co_return FileDepsResult{.file = params.path};
auto path_id = pool_it->second;
auto direction = params.direction.value_or("both");
auto max_depth = params.depth.value_or(1);
FileDepsResult result;
result.file = params.path;
if(direction == "includes" || direction == "both") {
auto includes = ws.dep_graph.get_all_includes(path_id);
for(auto inc_id: includes) {
auto real_id = inc_id & DependencyGraph::PATH_ID_MASK;
auto inc_path = ws.path_pool.resolve(real_id);
result.includes.push_back(DepEntry{.path = inc_path.str(), .depth = 1});
}
if(max_depth == 0 || max_depth > 1) {
llvm::DenseSet<std::uint32_t> visited;
visited.insert(path_id);
for(auto& dep: result.includes)
visited.insert(ws.path_pool.intern(dep.path));
for(std::size_t i = 0; i < result.includes.size(); ++i) {
if(max_depth > 0 && result.includes[i].depth >= max_depth)
continue;
auto dep_id = ws.path_pool.intern(result.includes[i].path);
auto sub = ws.dep_graph.get_all_includes(dep_id);
for(auto sub_id: sub) {
auto real_id = sub_id & DependencyGraph::PATH_ID_MASK;
if(!visited.insert(real_id).second)
continue;
auto sub_path = ws.path_pool.resolve(real_id);
result.includes.push_back(DepEntry{
.path = sub_path.str(),
.depth = result.includes[i].depth + 1,
});
}
}
}
}
if(direction == "includers" || direction == "both") {
auto includers = ws.dep_graph.get_includers(path_id);
for(auto inc_id: includers) {
auto inc_path = ws.path_pool.resolve(inc_id);
result.includers.push_back(DepEntry{.path = inc_path.str(), .depth = 1});
}
if(max_depth == 0 || max_depth > 1) {
llvm::DenseSet<std::uint32_t> visited;
visited.insert(path_id);
for(auto& dep: result.includers) {
auto it = ws.path_pool.cache.find(dep.path);
if(it != ws.path_pool.cache.end())
visited.insert(it->second);
}
for(std::size_t i = 0; i < result.includers.size(); ++i) {
if(max_depth > 0 && result.includers[i].depth >= max_depth)
continue;
auto dep_it = ws.path_pool.cache.find(result.includers[i].path);
if(dep_it == ws.path_pool.cache.end())
continue;
auto sub = ws.dep_graph.get_includers(dep_it->second);
for(auto sub_id: sub) {
if(!visited.insert(sub_id).second)
continue;
auto sub_path = ws.path_pool.resolve(sub_id);
result.includers.push_back(DepEntry{
.path = sub_path.str(),
.depth = result.includers[i].depth + 1,
});
}
}
}
}
co_return result;
});
peer.on_request(
[&srv](RequestContext&,
const ImpactAnalysisParams& params) -> RequestResult<ImpactAnalysisParams> {
auto& ws = srv.workspace;
auto pool_it = ws.path_pool.cache.find(params.path);
if(pool_it == ws.path_pool.cache.end())
co_return ImpactAnalysisResult{};
auto path_id = pool_it->second;
ImpactAnalysisResult result;
auto direct_includers = ws.dep_graph.get_includers(path_id);
for(auto inc_id: direct_includers) {
result.direct_dependents.push_back(ws.path_pool.resolve(inc_id).str());
}
auto hosts = ws.dep_graph.find_host_sources(path_id);
llvm::DenseSet<std::uint32_t> seen;
seen.insert(path_id);
for(auto inc_id: direct_includers)
seen.insert(inc_id);
for(auto host_id: hosts) {
if(seen.insert(host_id).second)
result.transitive_dependents.push_back(ws.path_pool.resolve(host_id).str());
}
for(auto host_id: hosts) {
auto it = ws.path_to_module.find(host_id);
if(it != ws.path_to_module.end())
result.affected_modules.push_back(it->second);
}
auto mod_it = ws.path_to_module.find(path_id);
if(mod_it != ws.path_to_module.end())
result.affected_modules.push_back(mod_it->second);
co_return result;
});
peer.on_request([&srv](RequestContext&,
const SymbolSearchParams& params) -> RequestResult<SymbolSearchParams> {
auto max = params.max_results.value_or(100);
std::string query_lower = llvm::StringRef(params.query).lower();
SymbolSearchResult result;
llvm::DenseSet<index::SymbolHash> seen;
auto try_symbol = [&](index::SymbolHash hash, const index::Symbol& symbol) {
if(static_cast<int>(result.symbols.size()) >= max)
return;
if(symbol.name.empty())
return;
if(!query_lower.empty() &&
llvm::StringRef(symbol.name).lower().find(query_lower) == std::string::npos)
return;
if(params.kind_filter.has_value()) {
auto kind_name = std::string(symbol_kind_name(symbol.kind));
auto& filter = *params.kind_filter;
if(std::ranges::find(filter, kind_name) == filter.end())
return;
}
auto def_loc = srv.indexer.find_definition_location(hash);
if(!def_loc)
return;
if(!seen.insert(hash).second)
return;
auto file = uri_to_path(def_loc->uri);
result.symbols.push_back(SymbolEntry{
.name = symbol.name,
.kind = std::string(symbol_kind_name(symbol.kind)),
.file = std::move(file),
.line = static_cast<int>(def_loc->range.start.line) + 1,
.symbol_id = hash,
});
};
for(auto& [hash, symbol]: srv.workspace.project_index.symbols)
try_symbol(hash, symbol);
for(auto& [_, sess]: srv.sessions) {
if(!sess.file_index)
continue;
for(auto& [hash, symbol]: sess.file_index->symbols)
try_symbol(hash, symbol);
}
co_return result;
});
peer.on_request(
[&srv](RequestContext&, const ReadSymbolParams& params) -> RequestResult<ReadSymbolParams> {
auto candidates = resolve_locator(params, srv.workspace, srv.sessions, srv.indexer);
if(candidates.empty())
co_return kota::outcome_error(kota::ipc::Error{"symbol not found"});
if(candidates.size() > 1) {
co_return kota::outcome_error(kota::ipc::Error{
std::format("ambiguous: {} candidates, use symbolId to disambiguate",
candidates.size())});
}
auto& rs = candidates[0];
auto def_text = srv.indexer.get_definition_text(rs.hash);
if(!def_text)
co_return kota::outcome_error(kota::ipc::Error{"definition not found"});
co_return ReadSymbolResult{
.name = rs.name,
.kind = std::string(symbol_kind_name(rs.kind)),
.file = std::move(def_text->file),
.start_line = def_text->start_line,
.end_line = def_text->end_line,
.text = std::move(def_text->text),
.symbol_id = rs.hash,
};
});
peer.on_request(
[&srv](RequestContext&,
const DocumentSymbolsParams& params) -> RequestResult<DocumentSymbolsParams> {
auto is_document_level = [](SymbolKind kind) {
return kind == SymbolKind::Namespace || kind == SymbolKind::Class ||
kind == SymbolKind::Struct || kind == SymbolKind::Union ||
kind == SymbolKind::Enum || kind == SymbolKind::Type ||
kind == SymbolKind::Field || kind == SymbolKind::EnumMember ||
kind == SymbolKind::Function || kind == SymbolKind::Method ||
kind == SymbolKind::Variable || kind == SymbolKind::Macro ||
kind == SymbolKind::Concept || kind == SymbolKind::Module ||
kind == SymbolKind::Operator || kind == SymbolKind::Attribute;
};
DocumentSymbolsResult result;
auto pool_it = srv.workspace.path_pool.cache.find(params.path);
if(pool_it == srv.workspace.path_pool.cache.end())
co_return result;
auto server_id = pool_it->second;
auto sess_it = srv.sessions.find(server_id);
if(sess_it != srv.sessions.end() && sess_it->second.file_index) {
auto& fi = *sess_it->second.file_index;
for(auto& [hash, rels]: fi.file_index.relations) {
for(auto& rel: rels) {
if(rel.kind.value() != RelationKind::Definition)
continue;
std::string name;
SymbolKind kind;
if(!srv.indexer.find_symbol_info(hash, name, kind))
continue;
if(!is_document_level(kind))
continue;
if(fi.mapper) {
auto start = fi.mapper->to_position(rel.range.begin);
auto end = fi.mapper->to_position(rel.range.end);
if(start && end) {
result.symbols.push_back(DocumentSymbolEntry{
.name = std::move(name),
.kind = std::string(symbol_kind_name(kind)),
.start_line = static_cast<int>(start->line) + 1,
.end_line = static_cast<int>(end->line) + 1,
.symbol_id = hash,
});
break;
}
}
}
}
co_return result;
}
auto it = srv.workspace.project_index.path_pool.find(params.path);
if(it == srv.workspace.project_index.path_pool.cache.end())
co_return result;
auto proj_id = it->second;
auto shard_it = srv.workspace.merged_indices.find(proj_id);
if(shard_it == srv.workspace.merged_indices.end())
co_return result;
for(auto& [hash, symbol]: srv.workspace.project_index.symbols) {
if(symbol.name.empty())
continue;
if(!is_document_level(symbol.kind))
continue;
if(!symbol.reference_files.contains(proj_id))
continue;
shard_it->second.find_relations(
hash,
RelationKind::Definition,
[&](const index::Relation&, protocol::Range range) {
result.symbols.push_back(DocumentSymbolEntry{
.name = symbol.name,
.kind = std::string(symbol_kind_name(symbol.kind)),
.start_line = static_cast<int>(range.start.line) + 1,
.end_line = static_cast<int>(range.end.line) + 1,
.symbol_id = hash,
});
return true;
});
}
co_return result;
});
peer.on_request(
[&srv](RequestContext&, const DefinitionParams& params) -> RequestResult<DefinitionParams> {
auto candidates = resolve_locator(
ReadSymbolParams{params.name, params.path, params.line, params.symbol_id},
srv.workspace,
srv.sessions,
srv.indexer);
if(candidates.empty())
co_return kota::outcome_error(kota::ipc::Error{"symbol not found"});
if(candidates.size() > 1) {
co_return kota::outcome_error(kota::ipc::Error{
std::format("ambiguous: {} candidates, use symbolId to disambiguate",
candidates.size())});
}
auto& rs = candidates[0];
DefinitionResult result;
result.name = rs.name;
result.kind = std::string(symbol_kind_name(rs.kind));
result.symbol_id = rs.hash;
if(auto def_text = srv.indexer.get_definition_text(rs.hash)) {
result.definition = LocationEntry{
.file = std::move(def_text->file),
.start_line = def_text->start_line,
.end_line = def_text->end_line,
.text = std::move(def_text->text),
};
}
co_return result;
});
peer.on_request(
[&srv](RequestContext&, const ReferencesParams& params) -> RequestResult<ReferencesParams> {
auto candidates = resolve_locator(
ReadSymbolParams{params.name, params.path, params.line, params.symbol_id},
srv.workspace,
srv.sessions,
srv.indexer);
if(candidates.empty())
co_return kota::outcome_error(kota::ipc::Error{"symbol not found"});
if(candidates.size() > 1) {
co_return kota::outcome_error(kota::ipc::Error{
std::format("ambiguous: {} candidates, use symbolId to disambiguate",
candidates.size())});
}
auto& rs = candidates[0];
ReferencesResult result;
result.name = rs.name;
result.kind = std::string(symbol_kind_name(rs.kind));
result.symbol_id = rs.hash;
for(auto& ref: srv.indexer.collect_references(rs.hash, RelationKind::Reference)) {
result.references.push_back(ReferenceEntry{
.file = std::move(ref.file),
.line = ref.line,
.context = std::move(ref.context),
});
}
if(params.include_declaration.value_or(false)) {
for(auto& ref: srv.indexer.collect_references(rs.hash, RelationKind::Definition)) {
result.references.push_back(ReferenceEntry{
.file = std::move(ref.file),
.line = ref.line,
.context = std::move(ref.context),
});
}
}
result.total = static_cast<int>(result.references.size());
co_return result;
});
peer.on_request(
[&srv](RequestContext&, const CallGraphParams& params) -> RequestResult<CallGraphParams> {
auto candidates = resolve_locator(
ReadSymbolParams{params.name, params.path, params.line, params.symbol_id},
srv.workspace,
srv.sessions,
srv.indexer);
if(candidates.empty())
co_return kota::outcome_error(kota::ipc::Error{"symbol not found"});
if(candidates.size() > 1) {
co_return kota::outcome_error(kota::ipc::Error{
std::format("ambiguous: {} candidates, use symbolId to disambiguate",
candidates.size())});
}
auto& rs = candidates[0];
auto direction = params.direction.value_or("both");
CallGraphResult result;
result.root = CallGraphEntry{
.name = rs.name,
.kind = std::string(symbol_kind_name(rs.kind)),
.file = rs.file,
.line = rs.line,
.symbol_id = rs.hash,
};
auto resolve_kind = [&](std::uint64_t sym_id) -> std::string {
if(sym_id == 0)
return "Function";
std::string name;
SymbolKind kind;
if(srv.indexer.find_symbol_info(sym_id, name, kind))
return std::string(symbol_kind_name(kind));
return "Function";
};
if(direction == "callers" || direction == "both") {
auto incoming = srv.indexer.find_incoming_calls(rs.hash);
for(auto& call: incoming) {
auto sid = extract_symbol_id(call.from.data);
result.callers.push_back(CallGraphEntry{
.name = call.from.name,
.kind = resolve_kind(sid),
.file = uri_to_path(call.from.uri),
.line = static_cast<int>(call.from.range.start.line) + 1,
.symbol_id = sid,
});
}
}
if(direction == "callees" || direction == "both") {
auto outgoing = srv.indexer.find_outgoing_calls(rs.hash);
for(auto& call: outgoing) {
auto sid = extract_symbol_id(call.to.data);
result.callees.push_back(CallGraphEntry{
.name = call.to.name,
.kind = resolve_kind(sid),
.file = uri_to_path(call.to.uri),
.line = static_cast<int>(call.to.range.start.line) + 1,
.symbol_id = sid,
});
}
}
co_return result;
});
peer.on_request(
[&srv](RequestContext&,
const TypeHierarchyParams& params) -> RequestResult<TypeHierarchyParams> {
auto candidates = resolve_locator(
ReadSymbolParams{params.name, params.path, params.line, params.symbol_id},
srv.workspace,
srv.sessions,
srv.indexer);
if(candidates.empty())
co_return kota::outcome_error(kota::ipc::Error{"symbol not found"});
if(candidates.size() > 1) {
co_return kota::outcome_error(kota::ipc::Error{
std::format("ambiguous: {} candidates, use symbolId to disambiguate",
candidates.size())});
}
auto& rs = candidates[0];
auto direction = params.direction.value_or("both");
TypeHierarchyResult result;
result.root = TypeHierarchyEntry{
.name = rs.name,
.kind = std::string(symbol_kind_name(rs.kind)),
.file = rs.file,
.line = rs.line,
.symbol_id = rs.hash,
};
auto resolve_kind = [&](std::uint64_t sym_id) -> std::string {
if(sym_id == 0)
return "Class";
std::string name;
SymbolKind kind;
if(srv.indexer.find_symbol_info(sym_id, name, kind))
return std::string(symbol_kind_name(kind));
return "Class";
};
if(direction == "supertypes" || direction == "both") {
for(auto& item: srv.indexer.find_supertypes(rs.hash)) {
auto sid = extract_symbol_id(item.data);
result.supertypes.push_back(TypeHierarchyEntry{
.name = item.name,
.kind = resolve_kind(sid),
.file = uri_to_path(item.uri),
.line = static_cast<int>(item.range.start.line) + 1,
.symbol_id = sid,
});
}
}
if(direction == "subtypes" || direction == "both") {
for(auto& item: srv.indexer.find_subtypes(rs.hash)) {
auto sid = extract_symbol_id(item.data);
result.subtypes.push_back(TypeHierarchyEntry{
.name = item.name,
.kind = resolve_kind(sid),
.file = uri_to_path(item.uri),
.line = static_cast<int>(item.range.start.line) + 1,
.symbol_id = sid,
});
}
}
co_return result;
});
peer.on_request([&srv](RequestContext&, const StatusParams&) -> RequestResult<StatusParams> {
StatusResult result;
result.idle = srv.indexer.is_idle();
result.pending = static_cast<int>(srv.indexer.pending_files());
result.total = static_cast<int>(srv.indexer.total_queued());
result.indexed = std::max(0, result.total - result.pending);
co_return result;
});
peer.on_notification([&srv](const ShutdownParams&) {
LOG_INFO("agentic/shutdown received, shutting down");
srv.schedule_shutdown();
});
}
} // namespace clice

View File

@@ -0,0 +1,18 @@
#pragma once
#include "kota/ipc/codec/json.h"
namespace clice {
class MasterServer;
class AgentClient {
public:
AgentClient(MasterServer& server, kota::ipc::JsonPeer& peer);
private:
MasterServer& server;
kota::ipc::JsonPeer& peer;
};
} // namespace clice

View File

@@ -0,0 +1,177 @@
#include "server/service/agentic.h"
#include <memory>
#include <print>
#include <string>
#include "server/protocol/agentic.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "kota/async/async.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/transport.h"
namespace clice {
template <typename Params>
static kota::task<bool> send_and_print(kota::ipc::JsonPeer& peer, Params params) {
auto result = co_await peer.send_request(std::move(params));
if(!result) {
LOG_ERROR("request failed: {}", result.error().message);
co_return false;
}
auto json = kota::codec::json::to_string<kota::ipc::lsp_config>(*result);
std::println("{}", json ? *json : "null");
co_return true;
}
static kota::task<> agentic_request(kota::ipc::JsonPeer& peer,
int& exit_code,
const AgenticQueryOptions& opts) {
bool ok = false;
if(opts.method == "compileCommand") {
ok = co_await send_and_print(peer, agentic::CompileCommandParams{.path = opts.path});
} else if(opts.method == "projectFiles") {
auto filter = opts.query.empty() ? std::nullopt : std::optional(opts.query);
ok = co_await send_and_print(peer, agentic::ProjectFilesParams{.filter = filter});
} else if(opts.method == "symbolSearch") {
ok = co_await send_and_print(peer, agentic::SymbolSearchParams{.query = opts.query});
} else if(opts.method == "definition") {
auto name = opts.name.empty() ? std::nullopt : std::optional(opts.name);
auto path = opts.path.empty() ? std::nullopt : std::optional(opts.path);
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
ok = co_await send_and_print(
peer,
agentic::DefinitionParams{.name = name, .path = path, .line = line});
} else if(opts.method == "references") {
auto name = opts.name.empty() ? std::nullopt : std::optional(opts.name);
auto path = opts.path.empty() ? std::nullopt : std::optional(opts.path);
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
ok = co_await send_and_print(
peer,
agentic::ReferencesParams{.name = name, .path = path, .line = line});
} else if(opts.method == "readSymbol") {
auto name = opts.name.empty() ? std::nullopt : std::optional(opts.name);
auto path = opts.path.empty() ? std::nullopt : std::optional(opts.path);
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
ok = co_await send_and_print(
peer,
agentic::ReadSymbolParams{.name = name, .path = path, .line = line});
} else if(opts.method == "documentSymbols") {
ok = co_await send_and_print(peer, agentic::DocumentSymbolsParams{.path = opts.path});
} else if(opts.method == "callGraph") {
auto name = opts.name.empty() ? std::nullopt : std::optional(opts.name);
auto path = opts.path.empty() ? std::nullopt : std::optional(opts.path);
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
auto dir = opts.direction.empty() ? std::nullopt : std::optional(opts.direction);
ok = co_await send_and_print(peer,
agentic::CallGraphParams{
.name = name,
.path = path,
.line = line,
.direction = dir,
});
} else if(opts.method == "typeHierarchy") {
auto name = opts.name.empty() ? std::nullopt : std::optional(opts.name);
auto path = opts.path.empty() ? std::nullopt : std::optional(opts.path);
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
auto dir = opts.direction.empty() ? std::nullopt : std::optional(opts.direction);
ok = co_await send_and_print(peer,
agentic::TypeHierarchyParams{
.name = name,
.path = path,
.line = line,
.direction = dir,
});
} else if(opts.method == "fileDeps") {
auto dir = opts.direction.empty() ? std::nullopt : std::optional(opts.direction);
ok = co_await send_and_print(peer,
agentic::FileDepsParams{.path = opts.path, .direction = dir});
} else if(opts.method == "impactAnalysis") {
ok = co_await send_and_print(peer, agentic::ImpactAnalysisParams{.path = opts.path});
} else if(opts.method == "status") {
ok = co_await send_and_print(peer, agentic::StatusParams{});
} else if(opts.method == "shutdown") {
peer.send_notification(agentic::ShutdownParams{});
ok = true;
} else {
LOG_ERROR("unknown agentic method '{}'", opts.method);
}
if(ok)
exit_code = 0;
peer.close();
}
static kota::task<> agentic_client(int& exit_code,
std::unique_ptr<kota::ipc::JsonPeer>& peer_out,
const AgenticQueryOptions& opts) {
auto& loop = kota::event_loop::current();
auto transport = co_await kota::ipc::StreamTransport::connect_tcp(opts.host, opts.port, loop);
if(!transport) {
LOG_ERROR("failed to connect to {}:{}", opts.host, opts.port);
co_return;
}
peer_out = std::make_unique<kota::ipc::JsonPeer>(loop, std::move(*transport));
co_await kota::when_all(peer_out->run(), agentic_request(*peer_out, exit_code, opts));
}
int run_agentic_mode(const AgenticQueryOptions& opts) {
logging::stderr_logger("agentic", logging::options);
kota::event_loop loop;
int exit_code = 1;
std::unique_ptr<kota::ipc::JsonPeer> peer;
loop.schedule(agentic_client(exit_code, peer, opts));
loop.run();
return exit_code;
}
static kota::task<> relay_forward(kota::ipc::Transport& from, kota::ipc::Transport& to) {
while(true) {
auto msg = co_await from.read_message();
if(!msg)
break;
co_await to.write_message(*msg);
}
to.close();
}
static kota::task<> relay_main(kota::event_loop& loop, int& exit_code, std::string socket_path) {
auto stdio = kota::ipc::StreamTransport::open_stdio(loop);
if(!stdio) {
LOG_ERROR("failed to open stdio transport");
loop.stop();
co_return;
}
auto conn = co_await kota::pipe::connect(socket_path, {}, loop);
if(!conn) {
LOG_ERROR("failed to connect to {}", socket_path);
loop.stop();
co_return;
}
auto socket = std::make_unique<kota::ipc::StreamTransport>(std::move(*conn));
co_await kota::when_all(relay_forward(**stdio, *socket), relay_forward(*socket, **stdio));
exit_code = 0;
loop.stop();
}
int run_relay_mode(llvm::StringRef socket_path) {
logging::stderr_logger("relay", logging::options);
auto path = socket_path.empty() ? path::default_socket_path() : socket_path.str();
kota::event_loop loop;
int exit_code = 1;
loop.schedule(relay_main(loop, exit_code, std::move(path)));
loop.run();
return exit_code;
}
} // namespace clice

View File

@@ -0,0 +1,24 @@
#pragma once
#include <string>
#include "llvm/ADT/StringRef.h"
namespace clice {
struct AgenticQueryOptions {
std::string host;
int port = 0;
std::string method;
std::string path;
std::string name;
std::string query;
int line = 0;
std::string direction;
};
int run_agentic_mode(const AgenticQueryOptions& opts);
int run_relay_mode(llvm::StringRef socket_path);
} // namespace clice

View File

@@ -1,4 +1,4 @@
#include "server/master_server.h"
#include "server/service/lsp_client.h"
#include <algorithm>
#include <format>
@@ -7,7 +7,9 @@
#include <variant>
#include "semantic/symbol_kind.h"
#include "server/protocol.h"
#include "server/protocol/extension.h"
#include "server/protocol/worker.h"
#include "server/service/master_server.h"
#include "support/filesystem.h"
#include "support/logging.h"
@@ -16,7 +18,6 @@
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/lsp/uri.h"
#include "kota/meta/enum.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/Process.h"
@@ -29,177 +30,39 @@ using kota::ipc::RequestResult;
using RequestContext = kota::ipc::JsonPeer::RequestContext;
using serde_raw = kota::codec::RawValue;
/// Serialize a value to a JSON RawValue using LSP config.
template <typename T>
static serde_raw to_raw(const T& value) {
auto json = kota::codec::json::to_json<kota::ipc::lsp_config>(value);
return serde_raw{json ? std::move(*json) : "null"};
}
MasterServer::MasterServer(kota::event_loop& loop,
kota::ipc::JsonPeer& peer,
std::string self_path) :
loop(loop), peer(peer), pool(loop), compiler(loop, peer, workspace, pool, sessions),
indexer(loop,
workspace,
sessions,
pool,
compiler,
[this](uint32_t proj_path_id) {
// Bridge project-level path_id to server-level path_id.
// The two PathPools may assign different IDs to the same path.
auto path = workspace.project_index.path_pool.path(proj_path_id);
auto server_id = workspace.path_pool.intern(path);
return sessions.contains(server_id);
}),
self_path(std::move(self_path)) {}
LSPClient::LSPClient(MasterServer& server, kota::ipc::JsonPeer& peer) : server(server), peer(peer) {
server.compiler.set_peer(&peer);
server.indexer.set_peer(&peer);
MasterServer::~MasterServer() = default;
kota::task<> MasterServer::load_workspace() {
if(workspace_root.empty())
co_return;
auto& cfg = workspace.config.project;
if(!cfg.cache_dir.empty()) {
auto ec = llvm::sys::fs::create_directories(cfg.cache_dir);
if(ec) {
LOG_WARN("Failed to create cache directory {}: {}",
std::string_view(cfg.cache_dir),
ec.message());
} else {
LOG_INFO("Cache directory: {}", std::string_view(cfg.cache_dir));
}
for(auto* subdir: {"cache/pch", "cache/pcm"}) {
auto dir = path::join(cfg.cache_dir, subdir);
if(auto ec2 = llvm::sys::fs::create_directories(dir))
LOG_WARN("Failed to create {}: {}", dir, ec2.message());
}
workspace.cleanup_cache();
workspace.load_cache();
}
// Discover compile_commands.json: configured paths first, then auto-scan.
std::string cdb_path;
for(auto& configured: cfg.compile_commands_paths) {
// Each entry can be a file or a directory containing compile_commands.json.
if(llvm::sys::fs::is_directory(configured)) {
auto candidate = path::join(configured, "compile_commands.json");
if(llvm::sys::fs::exists(candidate)) {
cdb_path = std::move(candidate);
break;
}
} else if(llvm::sys::fs::exists(configured)) {
cdb_path = configured;
break;
} else {
LOG_WARN("Configured compile_commands_path not found: {}", configured);
}
}
// Auto-scan: workspace root + all immediate subdirectories.
if(cdb_path.empty()) {
auto try_candidate = [&](llvm::StringRef dir) -> bool {
auto candidate = path::join(dir, "compile_commands.json");
if(llvm::sys::fs::exists(candidate)) {
cdb_path = std::move(candidate);
return true;
}
return false;
};
if(!try_candidate(workspace_root)) {
std::error_code ec;
for(llvm::sys::fs::directory_iterator it(workspace_root, ec), end; it != end && !ec;
it.increment(ec)) {
if(it->type() == llvm::sys::fs::file_type::directory_file) {
if(try_candidate(it->path()))
break;
}
}
}
}
if(cdb_path.empty()) {
LOG_WARN("No compile_commands.json found in workspace {}", workspace_root);
co_return;
}
auto count = workspace.cdb.load(cdb_path);
LOG_INFO("Loaded CDB from {} with {} entries", cdb_path, count);
auto report = scan_dependency_graph(workspace.cdb,
workspace.path_pool,
workspace.dep_graph,
/*cache=*/nullptr,
[this](llvm::StringRef path,
std::vector<std::string>& append,
std::vector<std::string>& remove) {
workspace.config.match_rules(path, append, remove);
});
workspace.dep_graph.build_reverse_map();
auto unresolved = report.includes_found - report.includes_resolved;
double accuracy =
report.includes_found > 0
? 100.0 * static_cast<double>(report.includes_resolved) / report.includes_found
: 100.0;
LOG_INFO(
"Dependency scan: {}ms, {} files ({} source + {} header), " "{} edges, {}/{} resolved ({:.1f}%), {} waves",
report.elapsed_ms,
report.total_files,
report.source_files,
report.header_files,
report.total_edges,
report.includes_resolved,
report.includes_found,
accuracy,
report.waves);
if(unresolved > 0)
LOG_WARN("{} unresolved includes", unresolved);
workspace.build_module_map();
indexer.load(cfg.index_dir);
if(*cfg.enable_indexing) {
for(auto& entry: workspace.cdb.get_entries()) {
auto file = workspace.cdb.resolve_path(entry.file);
auto server_id = workspace.path_pool.intern(file);
indexer.enqueue(server_id);
}
indexer.schedule();
}
compiler.init_compile_graph();
}
void MasterServer::register_handlers() {
using StringVec = std::vector<std::string>;
peer.on_request([this](RequestContext& ctx, const protocol::InitializeParams& params)
-> RequestResult<protocol::InitializeParams> {
if(lifecycle != ServerLifecycle::Uninitialized) {
auto& srv = this->server;
if(srv.lifecycle != ServerLifecycle::Uninitialized) {
co_return kota::outcome_error(protocol::Error{"Server already initialized"});
}
auto& init = params.lsp__initialize_params;
if(init.root_uri.has_value()) {
workspace_root = uri_to_path(*init.root_uri);
srv.workspace_root = uri_to_path(*init.root_uri);
}
// Capture initializationOptions as raw JSON for config loading.
if(init.initialization_options.has_value()) {
auto json =
kota::codec::json::to_json<kota::ipc::lsp_config>(*init.initialization_options);
if(json)
init_options_json = std::move(*json);
srv.init_options_json = std::move(*json);
}
lifecycle = ServerLifecycle::Initialized;
LOG_INFO("Initialized with workspace: {}", workspace_root);
srv.lifecycle = ServerLifecycle::Initialized;
LOG_INFO("Initialized with workspace: {}", srv.workspace_root);
protocol::InitializeResult result;
auto& caps = result.capabilities;
@@ -222,7 +85,6 @@ void MasterServer::register_handlers() {
caps.signature_help_provider = protocol::SignatureHelpOptions{
.trigger_characters = StringVec{"(", ")", "{", "}", "<", ">", ","},
};
/// FIXME: In the future, we would support work done progress.
caps.declaration_provider = protocol::DeclarationOptions{
.work_done_progress = false,
};
@@ -246,6 +108,8 @@ void MasterServer::register_handlers() {
caps.call_hierarchy_provider = true;
caps.type_hierarchy_provider = true;
caps.workspace_symbol_provider = true;
caps.document_formatting_provider = true;
caps.document_range_formatting_provider = true;
protocol::SemanticTokensOptions sem_opts;
{
@@ -277,100 +141,32 @@ void MasterServer::register_handlers() {
co_return result;
});
peer.on_notification([this](const protocol::InitializedParams& params) {
// Config priority: initializationOptions > clice.toml > defaults.
// Load the workspace config (with defaults applied) first, then overlay
// any initializationOptions on top so fields not mentioned in the JSON
// keep the values from clice.toml — kotatsu's deserializer only touches
// fields that are present in the input.
workspace.config = Config::load_from_workspace(workspace_root);
if(!init_options_json.empty()) {
if(auto ov = kota::codec::json::parse(init_options_json, workspace.config); !ov) {
LOG_WARN("Failed to apply initializationOptions: {}", ov.error().to_string());
} else {
// Re-run apply_defaults so overridden strings get workspace
// substitution and `compiled_rules` is rebuilt if `rules`
// changed. Defaults are gated on zero/empty sentinels, so
// existing values from the overlay are preserved.
workspace.config.apply_defaults(workspace_root);
LOG_INFO("Applied initializationOptions overlay");
}
init_options_json.clear();
}
auto& cfg = workspace.config.project;
if(!cfg.logging_dir.empty()) {
auto now = std::chrono::system_clock::now();
auto pid = llvm::sys::Process::getProcessId();
auto session_dir =
path::join(cfg.logging_dir, std::format("{:%Y-%m-%d_%H-%M-%S}_{}", now, pid));
logging::file_logger("master", session_dir, logging::options);
session_log_dir = session_dir;
}
LOG_INFO("Server ready (stateful={}, stateless={}, idle={}ms)",
cfg.stateful_worker_count.value,
cfg.stateless_worker_count.value,
*cfg.idle_timeout_ms);
WorkerPoolOptions pool_opts;
pool_opts.self_path = self_path;
pool_opts.stateful_count = cfg.stateful_worker_count;
pool_opts.stateless_count = cfg.stateless_worker_count;
pool_opts.worker_memory_limit = cfg.worker_memory_limit;
pool_opts.log_dir = session_log_dir;
if(!pool.start(pool_opts)) {
LOG_ERROR("Failed to start worker pool");
return;
}
lifecycle = ServerLifecycle::Ready;
compiler.on_indexing_needed = [this]() {
indexer.schedule();
};
loop.schedule(load_workspace());
peer.on_notification([this]([[maybe_unused]] const protocol::InitializedParams& params) {
this->server.initialize();
});
peer.on_request(
[this](RequestContext& ctx,
const protocol::ShutdownParams& params) -> RequestResult<protocol::ShutdownParams> {
lifecycle = ServerLifecycle::ShuttingDown;
this->server.lifecycle = ServerLifecycle::ShuttingDown;
LOG_INFO("Shutdown requested");
co_return nullptr;
});
peer.on_notification([this](const protocol::ExitParams& params) {
lifecycle = ServerLifecycle::Exited;
peer.on_notification([this]([[maybe_unused]] const protocol::ExitParams& params) {
LOG_INFO("Exit notification received");
indexer.save(workspace.config.project.index_dir);
workspace.save_cache();
loop.schedule([this]() -> kota::task<> {
co_await pool.stop();
loop.stop();
}());
this->server.schedule_shutdown();
});
/// Document lifecycle — handled directly by MasterServer.
peer.on_notification([this](const protocol::DidOpenTextDocumentParams& params) {
if(lifecycle != ServerLifecycle::Ready)
auto& srv = this->server;
if(srv.lifecycle != ServerLifecycle::Ready)
return;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto path_id = srv.workspace.path_pool.intern(path);
auto [it, inserted] = sessions.try_emplace(path_id);
auto& session = it->second;
if(!inserted) {
// DenseMap tombstone may retain stale data — reset to a fresh Session.
session = Session{};
}
session.path_id = path_id;
auto& session = srv.open_session(path_id);
session.version = params.text_document.version;
session.text = params.text_document.text;
session.generation++;
@@ -379,18 +175,18 @@ void MasterServer::register_handlers() {
});
peer.on_notification([this](const protocol::DidChangeTextDocumentParams& params) {
if(lifecycle != ServerLifecycle::Ready)
auto& srv = this->server;
if(srv.lifecycle != ServerLifecycle::Ready)
return;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto path_id = srv.workspace.path_pool.intern(path);
auto it = sessions.find(path_id);
if(it == sessions.end())
auto* session = srv.find_session(path_id);
if(!session)
return;
auto& session = it->second;
session.version = params.text_document.version;
session->version = params.text_document.version;
for(auto& change: params.content_changes) {
std::visit(
@@ -398,186 +194,157 @@ void MasterServer::register_handlers() {
using T = std::remove_cvref_t<decltype(c)>;
if constexpr(std::is_same_v<T,
protocol::TextDocumentContentChangeWholeDocument>) {
session.text = c.text;
session->text = c.text;
} else {
auto& range = c.range;
lsp::PositionMapper mapper(session.text, lsp::PositionEncoding::UTF16);
lsp::PositionMapper mapper(session->text, lsp::PositionEncoding::UTF16);
auto start = mapper.to_offset(range.start);
auto end = mapper.to_offset(range.end);
if(start && end && *start <= *end) {
session.text.replace(*start, *end - *start, c.text);
session->text.replace(*start, *end - *start, c.text);
}
}
},
change);
}
session.generation++;
session.ast_dirty = true;
session->generation++;
session->ast_dirty = true;
LOG_DEBUG("didChange: path={} version={} gen={}",
path,
session.version,
session.generation);
session->version,
session->generation);
worker::DocumentUpdateParams update;
update.path = path;
update.version = session.version;
pool.notify_stateful(path_id, update);
update.version = session->version;
srv.pool.notify_stateful(path_id, update);
});
peer.on_notification([this](const protocol::DidCloseTextDocumentParams& params) {
if(lifecycle != ServerLifecycle::Ready)
auto& srv = this->server;
if(srv.lifecycle != ServerLifecycle::Ready)
return;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
workspace.on_file_closed(path_id);
pool.notify_stateful(path_id, worker::EvictParams{path});
// Clear diagnostics for the closed file.
protocol::PublishDiagnosticsParams diag_params;
diag_params.uri = params.text_document.uri;
peer.send_notification(diag_params);
sessions.erase(path_id);
indexer.enqueue(path_id);
indexer.schedule();
LOG_DEBUG("didClose: {}", path);
auto path_id = srv.workspace.path_pool.intern(uri_to_path(params.text_document.uri));
srv.close_session(path_id, this->peer);
});
peer.on_notification([this](const protocol::DidSaveTextDocumentParams& params) {
if(lifecycle != ServerLifecycle::Ready)
auto& srv = this->server;
if(srv.lifecycle != ServerLifecycle::Ready)
return;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto dirtied = workspace.on_file_saved(path_id);
for(auto dirty_id: dirtied) {
if(auto sit = sessions.find(dirty_id); sit != sessions.end()) {
sit->second.ast_dirty = true;
} else {
indexer.enqueue(dirty_id);
}
}
// Invalidate header contexts for sessions whose host is this file.
for(auto& [hdr_id, session]: sessions) {
if(session.header_context && session.header_context->host_path_id == path_id) {
session.header_context.reset();
session.ast_dirty = true;
}
}
indexer.schedule();
auto path_id = srv.workspace.path_pool.intern(path);
srv.on_file_saved(path_id);
LOG_DEBUG("didSave: {}", path);
});
/// Feature requests — stateful forwarding.
peer.on_request([this](RequestContext& ctx, const protocol::HoverParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document_position_params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::Hover,
sit->second,
params.text_document_position_params.position);
co_return co_await srv.compiler.forward_query(
worker::QueryKind::Hover,
*session,
params.text_document_position_params.position);
});
peer.on_request([this](RequestContext& ctx,
const protocol::SemanticTokensParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::SemanticTokens, sit->second);
co_return co_await srv.compiler.forward_query(worker::QueryKind::SemanticTokens, *session);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::InlayHintParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::InlayHints,
sit->second,
{},
params.range);
co_return co_await srv.compiler.forward_query(worker::QueryKind::InlayHints,
*session,
{},
params.range);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::FoldingRangeParams& params) -> RawResult {
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::FoldingRange, sit->second);
});
peer.on_request([this](RequestContext& ctx,
const protocol::FoldingRangeParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await srv.compiler.forward_query(worker::QueryKind::FoldingRange, *session);
});
peer.on_request([this](RequestContext& ctx,
const protocol::DocumentSymbolParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::DocumentSymbol, sit->second);
});
peer.on_request([this](RequestContext& ctx,
const protocol::DocumentLinkParams& params) -> RawResult {
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
co_return serde_raw{"null"};
auto& session = sit->second;
auto result = co_await compiler.forward_query(worker::QueryKind::DocumentLink, session);
if(!result.has_value())
co_return serde_raw{"null"};
// Merge document links from PCH if available.
auto& links = result.value();
// Re-lookup session after co_await since iterators may be invalidated.
auto sit2 = sessions.find(path_id);
if(sit2 != sessions.end() && sit2->second.pch_ref) {
auto pch_it = workspace.pch_cache.find(sit2->second.pch_ref->path_id);
if(pch_it != workspace.pch_cache.end() && !pch_it->second.document_links_json.empty()) {
auto& pch_json = pch_it->second.document_links_json;
// Merge two JSON arrays.
if(!links.data.empty() && links.data != "null" && links.data.size() > 2) {
// "[a,b]" + "[c,d]" -> "[a,b,c,d]"
links.data.pop_back(); // remove trailing ']'
links.data += ',';
links.data.append(pch_json.begin() + 1, pch_json.end()); // skip '['
} else {
links.data = pch_json;
}
}
}
co_return std::move(links);
co_return co_await srv.compiler.forward_query(worker::QueryKind::DocumentSymbol, *session);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::CodeActionParams& params) -> RawResult {
[this](RequestContext& ctx, const protocol::DocumentLinkParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::CodeAction, sit->second);
auto result =
co_await srv.compiler.forward_query(worker::QueryKind::DocumentLink, *session);
if(!result.has_value())
co_return serde_raw{"null"};
auto& links = result.value();
auto* session2 = srv.find_session(path_id);
if(session2 && session2->pch_ref) {
auto& pch_cache = srv.workspace.pch_cache;
auto pch_it = pch_cache.find(session2->pch_ref->path_id);
if(pch_it != pch_cache.end() && !pch_it->second.document_links_json.empty()) {
auto& pch_json = pch_it->second.document_links_json;
if(!links.data.empty() && links.data != "null" && links.data.size() > 2) {
links.data.pop_back();
links.data += ',';
links.data.append(pch_json.begin() + 1, pch_json.end());
} else {
links.data = pch_json;
}
}
}
co_return std::move(links);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::CodeActionParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await srv.compiler.forward_query(worker::QueryKind::CodeAction, *session);
});
/// Helper: resolve URI to path, path_id, and Session pointer.
auto resolve_uri = [this](const std::string& uri) {
struct Result {
std::string path;
@@ -585,22 +352,21 @@ void MasterServer::register_handlers() {
Session* session;
};
auto path = uri_to_path(uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
Session* session = (sit != sessions.end()) ? &sit->second : nullptr;
auto path_id = this->server.workspace.path_pool.intern(path);
auto* session = this->server.find_session(path_id);
return Result{std::move(path), path_id, session};
};
auto lookup_at = [this, resolve_uri](const std::string& uri, const protocol::Position& pos) {
auto [path, path_id, session] = resolve_uri(uri);
return indexer.lookup_symbol(uri, path, pos, session);
return this->server.indexer.lookup_symbol(uri, path, pos, session);
};
auto query_at = [this, resolve_uri](const std::string& uri,
const protocol::Position& pos,
RelationKind kind) -> std::vector<protocol::Location> {
auto [path, path_id, session] = resolve_uri(uri);
return indexer.query_relations(path, pos, kind, session);
return this->server.indexer.query_relations(path, pos, kind, session);
};
auto resolve_item =
@@ -609,11 +375,9 @@ void MasterServer::register_handlers() {
const protocol::Range& range,
const std::optional<protocol::LSPAny>& data) -> std::optional<SymbolInfo> {
auto [path, path_id, session] = resolve_uri(uri);
return indexer.resolve_hierarchy_item(uri, path, range, data, session);
return this->server.indexer.resolve_hierarchy_item(uri, path, range, data, session);
};
/// Feature requests — index-based with AST fallback.
peer.on_request([this, query_at](RequestContext& ctx,
const protocol::DefinitionParams& params) -> RawResult {
auto& uri = params.text_document_position_params.text_document.uri;
@@ -624,14 +388,15 @@ void MasterServer::register_handlers() {
co_return to_raw(result);
}
auto& srv = this->server;
auto path = uri_to_path(uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::GoToDefinition,
sit->second,
pos);
co_return co_await srv.compiler.forward_query(worker::QueryKind::GoToDefinition,
*session,
pos);
});
peer.on_request([this, query_at](RequestContext& ctx,
@@ -668,32 +433,60 @@ void MasterServer::register_handlers() {
co_return serde_raw{"null"};
});
/// Feature requests — stateless forwarding.
peer.on_request([this](RequestContext& ctx,
const protocol::CompletionParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document_position_params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.handle_completion(params.text_document_position_params.position,
sit->second);
auto pause = srv.indexer.scoped_pause();
auto result =
co_await srv.compiler.handle_completion(params.text_document_position_params.position,
*session);
co_return std::move(result);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::SignatureHelpParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document_position_params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_build(worker::BuildKind::SignatureHelp,
params.text_document_position_params.position,
sit->second);
auto pause = srv.indexer.scoped_pause();
auto result =
co_await srv.compiler.forward_build(worker::BuildKind::SignatureHelp,
params.text_document_position_params.position,
*session);
co_return std::move(result);
});
/// Hierarchy queries — index-based.
peer.on_request(
[this](RequestContext& ctx, const protocol::DocumentFormattingParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
auto pause = srv.indexer.scoped_pause();
co_return co_await srv.compiler.forward_format(*session);
});
peer.on_request([this](RequestContext& ctx,
const protocol::DocumentRangeFormattingParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
auto pause = srv.indexer.scoped_pause();
co_return co_await srv.compiler.forward_format(*session, params.range);
});
peer.on_request(
[this, lookup_at](RequestContext& ctx,
@@ -718,7 +511,7 @@ void MasterServer::register_handlers() {
auto info = resolve_item(params.item.uri, params.item.range, params.item.data);
if(!info)
co_return serde_raw{"null"};
auto results = indexer.find_incoming_calls(info->hash);
auto results = this->server.indexer.find_incoming_calls(info->hash);
if(results.empty())
co_return serde_raw{"null"};
co_return to_raw(results);
@@ -730,7 +523,7 @@ void MasterServer::register_handlers() {
auto info = resolve_item(params.item.uri, params.item.range, params.item.data);
if(!info)
co_return serde_raw{"null"};
auto results = indexer.find_outgoing_calls(info->hash);
auto results = this->server.indexer.find_outgoing_calls(info->hash);
if(results.empty())
co_return serde_raw{"null"};
co_return to_raw(results);
@@ -760,7 +553,7 @@ void MasterServer::register_handlers() {
auto info = resolve_item(params.item.uri, params.item.range, params.item.data);
if(!info)
co_return serde_raw{"null"};
auto results = indexer.find_supertypes(info->hash);
auto results = this->server.indexer.find_supertypes(info->hash);
if(results.empty())
co_return serde_raw{"null"};
co_return to_raw(results);
@@ -772,7 +565,7 @@ void MasterServer::register_handlers() {
auto info = resolve_item(params.item.uri, params.item.range, params.item.data);
if(!info)
co_return serde_raw{"null"};
auto results = indexer.find_subtypes(info->hash);
auto results = this->server.indexer.find_subtypes(info->hash);
if(results.empty())
co_return serde_raw{"null"};
co_return to_raw(results);
@@ -780,29 +573,29 @@ void MasterServer::register_handlers() {
peer.on_request(
[this](RequestContext& ctx, const protocol::WorkspaceSymbolParams& params) -> RawResult {
auto results = indexer.search_symbols(params.query);
auto results = this->server.indexer.search_symbols(params.query);
if(results.empty())
co_return serde_raw{"null"};
co_return to_raw(results);
});
/// clice/ extension commands.
peer.on_request(
"clice/queryContext",
[this](RequestContext& ctx, const ext::QueryContextParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.uri);
auto path_id = workspace.path_pool.intern(path);
auto path_id = srv.workspace.path_pool.intern(path);
int offset_val = std::max(0, params.offset.value_or(0));
constexpr int page_size = 10;
ext::QueryContextResult result;
std::vector<ext::ContextItem> all_items;
auto hosts = workspace.dep_graph.find_host_sources(path_id);
auto& ws = srv.workspace;
auto hosts = ws.dep_graph.find_host_sources(path_id);
for(auto host_id: hosts) {
auto host_path = workspace.path_pool.resolve(host_id);
auto host_cdb = workspace.cdb.lookup(host_path, {.suppress_logging = true});
auto host_path = ws.path_pool.resolve(host_id);
auto host_cdb = ws.cdb.lookup(host_path, {.suppress_logging = true});
if(host_cdb.empty())
continue;
auto host_uri_opt = lsp::URI::from_file_path(std::string(host_path));
@@ -816,7 +609,7 @@ void MasterServer::register_handlers() {
}
if(hosts.empty()) {
auto entries = workspace.cdb.lookup(path, {.suppress_logging = true});
auto entries = ws.cdb.lookup(path, {.suppress_logging = true});
for(std::size_t i = 0; i < entries.size(); ++i) {
auto& cmd = entries[i];
auto argv = cmd.to_argv();
@@ -858,13 +651,14 @@ void MasterServer::register_handlers() {
peer.on_request(
"clice/currentContext",
[this](RequestContext& ctx, const ext::CurrentContextParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.uri);
auto path_id = workspace.path_pool.intern(path);
auto path_id = srv.workspace.path_pool.intern(path);
ext::CurrentContextResult result;
auto sit = sessions.find(path_id);
if(sit != sessions.end() && sit->second.active_context) {
auto ctx_path = workspace.path_pool.resolve(*sit->second.active_context);
auto* session = srv.find_session(path_id);
if(session && session->active_context) {
auto ctx_path = srv.workspace.path_pool.resolve(*session->active_context);
auto ctx_uri_opt = lsp::URI::from_file_path(std::string(ctx_path));
if(ctx_uri_opt) {
ext::ContextItem item;
@@ -880,34 +674,41 @@ void MasterServer::register_handlers() {
peer.on_request(
"clice/switchContext",
[this](RequestContext& ctx, const ext::SwitchContextParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.uri);
auto path_id = workspace.path_pool.intern(path);
auto path_id = srv.workspace.path_pool.intern(path);
auto context_path = uri_to_path(params.context_uri);
auto context_path_id = workspace.path_pool.intern(context_path);
auto context_path_id = srv.workspace.path_pool.intern(context_path);
ext::SwitchContextResult result;
auto context_cdb = workspace.cdb.lookup(context_path, {.suppress_logging = true});
auto& ws = srv.workspace;
auto context_cdb = ws.cdb.lookup(context_path, {.suppress_logging = true});
if(context_cdb.empty()) {
result.success = false;
co_return to_raw(result);
}
auto sit = sessions.find(path_id);
if(sit == sessions.end()) {
auto* session = srv.find_session(path_id);
if(!session) {
result.success = false;
co_return to_raw(result);
}
sit->second.active_context = context_path_id;
sit->second.header_context.reset();
sit->second.pch_ref.reset();
sit->second.ast_deps.reset();
sit->second.ast_dirty = true;
session->active_context = context_path_id;
session->header_context.reset();
session->pch_ref.reset();
session->ast_deps.reset();
session->ast_dirty = true;
result.success = true;
co_return to_raw(result);
});
}
LSPClient::~LSPClient() {
server.compiler.set_peer(nullptr);
server.indexer.set_peer(nullptr);
}
} // namespace clice

View File

@@ -0,0 +1,23 @@
#pragma once
#include "kota/async/async.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
namespace clice {
class MasterServer;
class LSPClient {
public:
LSPClient(MasterServer& server, kota::ipc::JsonPeer& peer);
~LSPClient();
private:
using RawResult = kota::task<kota::codec::RawValue, kota::ipc::Error>;
MasterServer& server;
kota::ipc::JsonPeer& peer;
};
} // namespace clice

View File

@@ -0,0 +1,551 @@
#include "server/service/master_server.h"
#include <cerrno>
#include <cstring>
#include <list>
#include <memory>
#include <string>
#include <vector>
#ifndef _WIN32
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#endif
#include "server/protocol/worker.h"
#include "server/service/agent_client.h"
#include "server/service/lsp_client.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "kota/async/async.h"
#include "kota/async/io/fs_event.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/lsp/uri.h"
#include "kota/ipc/recording_transport.h"
#include "kota/ipc/transport.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/Process.h"
namespace clice {
namespace lsp = kota::ipc::lsp;
namespace protocol = kota::ipc::protocol;
MasterServer::MasterServer(kota::event_loop& loop, std::string self_path) :
loop(loop), pool(loop), compiler(loop, workspace, pool, sessions),
indexer(loop,
workspace,
sessions,
pool,
compiler,
[this](uint32_t proj_path_id) {
auto path = workspace.project_index.path_pool.path(proj_path_id);
auto server_id = workspace.path_pool.intern(path);
return sessions.contains(server_id);
}),
self_path(std::move(self_path)) {}
MasterServer::~MasterServer() = default;
void MasterServer::initialize() {
workspace.config = Config::load_from_workspace(workspace_root);
if(!init_options_json.empty()) {
if(auto ov = kota::codec::json::parse(init_options_json, workspace.config); !ov) {
LOG_WARN("Failed to apply initializationOptions: {}", ov.error().to_string());
} else {
workspace.config.apply_defaults(workspace_root);
LOG_INFO("Applied initializationOptions overlay");
}
init_options_json.clear();
}
auto& cfg = workspace.config.project;
if(!cfg.logging_dir.empty()) {
auto now = std::chrono::system_clock::now();
auto pid = llvm::sys::Process::getProcessId();
session_log_dir =
path::join(cfg.logging_dir, std::format("{:%Y-%m-%d_%H-%M-%S}_{}", now, pid));
logging::file_logger("master", session_log_dir, logging::options);
}
LOG_INFO("Server ready (stateful={}, stateless={}, idle={}ms)",
cfg.stateful_worker_count.value,
cfg.stateless_worker_count.value,
*cfg.idle_timeout_ms);
WorkerPoolOptions pool_opts;
pool_opts.self_path = self_path;
pool_opts.stateful_count = cfg.stateful_worker_count;
pool_opts.stateless_count = cfg.stateless_worker_count;
pool_opts.worker_memory_limit = cfg.worker_memory_limit;
pool_opts.log_dir = session_log_dir;
if(!pool.start(pool_opts)) {
LOG_ERROR("Failed to start worker pool");
return;
}
lifecycle = ServerLifecycle::Ready;
compiler.on_indexing_needed = [this]() {
indexer.schedule();
};
indexer.set_max_concurrency(cfg.stateless_worker_count.value);
load_workspace();
}
void MasterServer::initialize(llvm::StringRef root) {
workspace_root = root.str();
initialize();
}
void MasterServer::start_file_watcher() {
if(workspace_root.empty())
return;
loop.schedule([this]() -> kota::task<> {
auto watcher = kota::fs_event::create(workspace_root, {}, loop);
if(!watcher) {
LOG_WARN("Failed to start file watcher for {}", workspace_root);
co_return;
}
LOG_INFO("File watcher started for {}", workspace_root);
while(true) {
auto changes = co_await watcher->next();
if(!changes)
break;
for(auto& change: *changes) {
if(change.type != kota::fs_event::effect::modify &&
change.type != kota::fs_event::effect::create)
continue;
llvm::StringRef file(change.path);
if(file.ends_with("compile_commands.json")) {
LOG_INFO("CDB changed, reloading workspace");
load_workspace();
continue;
}
if(file.ends_with(".cpp") || file.ends_with(".cc") || file.ends_with(".cxx") ||
file.ends_with(".c") || file.ends_with(".h") || file.ends_with(".hpp") ||
file.ends_with(".hxx") || file.ends_with(".cppm") || file.ends_with(".ixx")) {
auto path_id = workspace.path_pool.intern(file);
on_file_saved(path_id);
}
}
}
}());
}
Session* MasterServer::find_session(std::uint32_t path_id) {
auto it = sessions.find(path_id);
return it != sessions.end() ? &it->second : nullptr;
}
Session& MasterServer::open_session(std::uint32_t path_id) {
auto [it, inserted] = sessions.try_emplace(path_id);
auto& session = it->second;
if(!inserted)
session = Session{};
session.path_id = path_id;
return session;
}
void MasterServer::close_session(std::uint32_t path_id, kota::ipc::JsonPeer& peer) {
namespace protocol = kota::ipc::protocol;
auto path = workspace.path_pool.resolve(path_id);
workspace.on_file_closed(path_id);
pool.notify_stateful(path_id, worker::EvictParams{std::string(path)});
protocol::PublishDiagnosticsParams diag_params;
auto uri = lsp::URI::from_file_path(std::string(path));
if(uri)
diag_params.uri = uri->str();
diag_params.diagnostics = {};
peer.send_notification(diag_params);
sessions.erase(path_id);
indexer.enqueue(path_id);
indexer.schedule();
LOG_DEBUG("didClose: {}", path);
}
void MasterServer::on_file_saved(std::uint32_t path_id) {
auto dirtied = workspace.on_file_saved(path_id);
for(auto dirty_id: dirtied) {
if(auto* session = find_session(dirty_id)) {
session->ast_dirty = true;
} else {
indexer.enqueue(dirty_id);
}
}
for(auto& [hdr_id, session]: sessions) {
if(session.header_context && session.header_context->host_path_id == path_id) {
session.header_context.reset();
session.ast_dirty = true;
}
}
indexer.schedule();
}
void MasterServer::schedule_shutdown() {
if(lifecycle == ServerLifecycle::Exited)
return;
lifecycle = ServerLifecycle::Exited;
indexer.save(workspace.config.project.index_dir);
workspace.save_cache();
shutdown_event.set();
loop.schedule([this]() -> kota::task<> {
co_await kota::when_all(indexer.stop(), compiler.stop(), pool.stop());
loop.stop();
}());
}
void MasterServer::load_workspace() {
if(workspace_root.empty())
return;
auto& cfg = workspace.config.project;
if(!cfg.cache_dir.empty()) {
auto ec = llvm::sys::fs::create_directories(cfg.cache_dir);
if(ec) {
LOG_WARN("Failed to create cache directory {}: {}",
std::string_view(cfg.cache_dir),
ec.message());
} else {
LOG_INFO("Cache directory: {}", std::string_view(cfg.cache_dir));
}
for(auto* subdir: {"cache/pch", "cache/pcm"}) {
auto dir = path::join(cfg.cache_dir, subdir);
if(auto ec2 = llvm::sys::fs::create_directories(dir))
LOG_WARN("Failed to create {}: {}", dir, ec2.message());
}
workspace.cleanup_cache();
workspace.load_cache();
}
std::string cdb_path;
for(auto& configured: cfg.compile_commands_paths) {
if(llvm::sys::fs::is_directory(configured)) {
auto candidate = path::join(configured, "compile_commands.json");
if(llvm::sys::fs::exists(candidate)) {
cdb_path = std::move(candidate);
break;
}
} else if(llvm::sys::fs::exists(configured)) {
cdb_path = configured;
break;
} else {
LOG_WARN("Configured compile_commands_path not found: {}", configured);
}
}
if(cdb_path.empty()) {
auto try_candidate = [&](llvm::StringRef dir) -> bool {
auto candidate = path::join(dir, "compile_commands.json");
if(llvm::sys::fs::exists(candidate)) {
cdb_path = std::move(candidate);
return true;
}
return false;
};
if(!try_candidate(workspace_root)) {
std::error_code ec;
for(llvm::sys::fs::directory_iterator it(workspace_root, ec), end; it != end && !ec;
it.increment(ec)) {
if(it->type() == llvm::sys::fs::file_type::directory_file) {
if(try_candidate(it->path()))
break;
}
}
}
}
if(cdb_path.empty()) {
LOG_WARN("No compile_commands.json found in workspace {}", workspace_root);
return;
}
auto count = workspace.cdb.load(cdb_path);
LOG_INFO("Loaded CDB from {} with {} entries", cdb_path, count);
auto report = scan_dependency_graph(workspace.cdb,
workspace.path_pool,
workspace.dep_graph,
/*cache=*/nullptr,
[this](llvm::StringRef path,
std::vector<std::string>& append,
std::vector<std::string>& remove) {
workspace.config.match_rules(path, append, remove);
});
workspace.dep_graph.build_reverse_map();
auto unresolved = report.includes_found - report.includes_resolved;
double accuracy =
report.includes_found > 0
? 100.0 * static_cast<double>(report.includes_resolved) / report.includes_found
: 100.0;
LOG_INFO(
"Dependency scan: {}ms, {} files ({} source + {} header), " "{} edges, {}/{} resolved ({:.1f}%), {} waves",
report.elapsed_ms,
report.total_files,
report.source_files,
report.header_files,
report.total_edges,
report.includes_resolved,
report.includes_found,
accuracy,
report.waves);
if(unresolved > 0)
LOG_WARN("{} unresolved includes", unresolved);
workspace.build_module_map();
indexer.load(cfg.index_dir);
if(*cfg.enable_indexing) {
for(auto& entry: workspace.cdb.get_entries()) {
auto file = workspace.cdb.resolve_path(entry.file);
auto server_id = workspace.path_pool.intern(file);
indexer.enqueue(server_id);
}
indexer.schedule();
}
compiler.init_compile_graph();
}
struct Connection {
std::unique_ptr<kota::ipc::JsonPeer> peer;
std::unique_ptr<LSPClient> lsp_client;
std::unique_ptr<AgentClient> agent_client;
};
static kota::task<> run_connection(kota::ipc::JsonPeer* peer,
std::list<Connection>& connections,
std::list<Connection>::iterator pos) {
co_await peer->run();
LOG_INFO("Client disconnected");
connections.erase(pos);
}
static kota::task<> accept_connections(MasterServer& server,
kota::tcp::acceptor acceptor,
bool register_lsp,
std::list<Connection>& connections) {
auto& loop = kota::event_loop::current();
kota::task_group<> connection_group(loop);
bool lsp_registered = false;
while(true) {
auto conn = co_await acceptor.accept();
if(!conn.has_value())
break;
LOG_INFO("Client connected");
auto transport = std::make_unique<kota::ipc::StreamTransport>(std::move(*conn));
auto peer = std::make_unique<kota::ipc::JsonPeer>(loop, std::move(transport));
std::unique_ptr<LSPClient> lsp;
if(register_lsp && !lsp_registered) {
lsp = std::make_unique<LSPClient>(server, *peer);
lsp_registered = true;
}
auto agent = std::make_unique<AgentClient>(server, *peer);
auto* peer_ptr = peer.get();
auto it = connections.emplace(connections.end(),
Connection{
.peer = std::move(peer),
.lsp_client = std::move(lsp),
.agent_client = std::move(agent),
});
connection_group.spawn(run_connection(peer_ptr, connections, it));
}
co_await connection_group.join();
}
int run_server_mode(const ServerOptions& opts) {
logging::stderr_logger("master", logging::options);
kota::event_loop loop;
MasterServer server(loop, opts.self_path);
std::list<Connection> connections;
if(opts.mode == "pipe") {
auto transport = kota::ipc::StreamTransport::open_stdio(loop);
if(!transport) {
LOG_ERROR("failed to open stdio transport");
return 1;
}
std::unique_ptr<kota::ipc::Transport> final_transport = std::move(*transport);
if(!opts.record.empty()) {
final_transport =
std::make_unique<kota::ipc::RecordingTransport>(std::move(final_transport),
opts.record);
}
kota::ipc::JsonPeer lsp_peer(loop, std::move(final_transport));
LSPClient lsp_client(server, lsp_peer);
if(opts.port > 0) {
auto acceptor = kota::tcp::listen(opts.host, opts.port, {}, loop);
if(acceptor) {
LOG_INFO("Agentic protocol listening on {}:{}", opts.host, opts.port);
loop.schedule(accept_connections(server, std::move(*acceptor), false, connections));
} else {
LOG_WARN("Failed to start agentic listener on {}:{}", opts.host, opts.port);
}
}
loop.schedule(lsp_peer.run());
loop.run();
return 0;
}
if(opts.mode == "socket") {
auto acceptor = kota::tcp::listen(opts.host, opts.port, {}, loop);
if(!acceptor) {
LOG_ERROR("failed to listen on {}:{}", opts.host, opts.port);
return 1;
}
LOG_INFO("Listening on {}:{} ...", opts.host, opts.port);
loop.schedule(accept_connections(server, std::move(*acceptor), true, connections));
loop.run();
return 0;
}
LOG_ERROR("unknown server mode '{}'", opts.mode);
return 1;
}
struct DaemonConnection {
std::unique_ptr<kota::ipc::JsonPeer> peer;
std::unique_ptr<AgentClient> agent_client;
};
static kota::task<> run_daemon_connection(kota::ipc::JsonPeer* peer,
std::list<DaemonConnection>& connections,
std::list<DaemonConnection>::iterator pos) {
co_await peer->run();
LOG_INFO("Daemon client disconnected");
connections.erase(pos);
}
static kota::task<> daemon_main(MasterServer& server, kota::pipe::acceptor acceptor) {
auto& loop = kota::event_loop::current();
std::list<DaemonConnection> connections;
kota::task_group<> connection_group(loop);
co_await kota::when_all(
[&]() -> kota::task<> {
while(true) {
auto conn = co_await acceptor.accept();
if(!conn.has_value())
break;
LOG_INFO("Daemon client connected");
auto transport = std::make_unique<kota::ipc::StreamTransport>(std::move(*conn));
auto peer = std::make_unique<kota::ipc::JsonPeer>(loop, std::move(transport));
auto agent = std::make_unique<AgentClient>(server, *peer);
auto* peer_ptr = peer.get();
auto it = connections.emplace(connections.end(),
DaemonConnection{
.peer = std::move(peer),
.agent_client = std::move(agent),
});
connection_group.spawn(run_daemon_connection(peer_ptr, connections, it));
}
}(),
[&]() -> kota::task<> {
co_await server.get_shutdown_event().wait();
acceptor.stop();
for(auto& conn: connections) {
conn.peer->close();
}
}());
co_await connection_group.join();
}
int run_daemon_mode(const DaemonOptions& opts) {
logging::stderr_logger("daemon", logging::options);
auto socket_path = opts.socket_path.empty() ? path::default_socket_path() : opts.socket_path;
auto socket_dir = llvm::sys::path::parent_path(socket_path);
if(auto ec = llvm::sys::fs::create_directories(socket_dir)) {
LOG_ERROR("Failed to create socket directory {}: {}", socket_dir, ec.message());
return 1;
}
if(llvm::sys::fs::exists(socket_path)) {
#ifndef _WIN32
int fd = ::socket(AF_UNIX, SOCK_STREAM, 0);
if(fd >= 0) {
struct sockaddr_un addr{};
addr.sun_family = AF_UNIX;
auto len = std::min(socket_path.size(), sizeof(addr.sun_path) - 1);
std::memcpy(addr.sun_path, socket_path.data(), len);
bool live = ::connect(fd, reinterpret_cast<struct sockaddr*>(&addr), sizeof(addr)) == 0;
::close(fd);
if(live) {
LOG_ERROR("Another daemon is already running on {}", socket_path);
return 1;
}
}
#endif
llvm::sys::fs::remove(socket_path);
}
kota::event_loop loop;
MasterServer server(loop, opts.self_path);
if(!opts.workspace.empty()) {
server.initialize(opts.workspace);
server.start_file_watcher();
}
auto acceptor = kota::pipe::listen(socket_path, {}, loop);
if(!acceptor) {
LOG_ERROR("Failed to listen on {}", socket_path);
return 1;
}
LOG_INFO("Daemon listening on {}", socket_path);
loop.schedule(daemon_main(server, std::move(*acceptor)));
loop.run();
llvm::sys::fs::remove(socket_path);
return 0;
}
} // namespace clice

View File

@@ -0,0 +1,93 @@
#pragma once
#include <cstdint>
#include <string>
#include "server/compiler/compiler.h"
#include "server/compiler/indexer.h"
#include "server/service/session.h"
#include "server/worker/worker_pool.h"
#include "server/workspace/workspace.h"
#include "kota/async/async.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/StringRef.h"
namespace clice {
enum class ServerLifecycle : std::uint8_t {
Uninitialized,
Initialized,
Ready,
ShuttingDown,
Exited,
};
/// Core server state — owns the two-layer state model (Workspace + Sessions),
/// the worker pool, compilation engine, and indexer.
///
/// Does NOT own any transport or peer. Protocol-specific handler registration
/// is done by LSPClient and AgentClient, which access private members directly.
class MasterServer {
friend class LSPClient;
friend class AgentClient;
public:
MasterServer(kota::event_loop& loop, std::string self_path);
~MasterServer();
void initialize();
void initialize(llvm::StringRef root);
void start_file_watcher();
Session* find_session(std::uint32_t path_id);
Session& open_session(std::uint32_t path_id);
void close_session(std::uint32_t path_id, kota::ipc::JsonPeer& peer);
void on_file_saved(std::uint32_t path_id);
void schedule_shutdown();
kota::event& get_shutdown_event() {
return shutdown_event;
}
private:
kota::event shutdown_event;
void load_workspace();
kota::event_loop& loop;
Workspace workspace;
llvm::DenseMap<std::uint32_t, Session> sessions;
WorkerPool pool;
Compiler compiler;
Indexer indexer;
ServerLifecycle lifecycle = ServerLifecycle::Uninitialized;
std::string self_path;
std::string workspace_root;
std::string session_log_dir;
std::string init_options_json;
};
struct ServerOptions {
std::string mode;
std::string host = "127.0.0.1";
int port = 0;
std::string self_path;
std::string record;
};
int run_server_mode(const ServerOptions& opts);
struct DaemonOptions {
std::string socket_path;
std::string workspace;
std::string self_path;
};
int run_daemon_mode(const DaemonOptions& opts);
} // namespace clice

View File

@@ -5,7 +5,7 @@
#include <optional>
#include <string>
#include "server/workspace.h"
#include "server/workspace/workspace.h"
#include "kota/async/async.h"
#include "llvm/ADT/SmallVector.h"

View File

@@ -1,4 +1,4 @@
#include "server/stateful_worker.h"
#include "server/worker/stateful_worker.h"
#include <atomic>
#include <cstdint>
@@ -10,8 +10,8 @@
#include "compile/compilation.h"
#include "feature/feature.h"
#include "index/tu_index.h"
#include "server/protocol.h"
#include "server/worker_common.h"
#include "server/protocol/worker.h"
#include "server/worker/worker_common.h"
#include "support/logging.h"
#include "kota/async/async.h"
@@ -245,26 +245,33 @@ void StatefulWorker::register_handlers() {
co_return kota::codec::RawValue{"[]"};
case K::SemanticTokens:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::semantic_tokens(doc.unit));
return to_raw(
feature::semantic_tokens(doc.unit, feature::PositionEncoding::UTF16));
});
case K::InlayHints:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
auto range = params.range;
if(range.begin == static_cast<uint32_t>(-1))
range = LocalSourceRange{0, static_cast<uint32_t>(doc.text.size())};
return to_raw(feature::inlay_hints(doc.unit, range));
return to_raw(feature::inlay_hints(doc.unit,
range,
{},
feature::PositionEncoding::UTF16));
});
case K::FoldingRange:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::folding_ranges(doc.unit));
return to_raw(
feature::folding_ranges(doc.unit, feature::PositionEncoding::UTF16));
});
case K::DocumentSymbol:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::document_symbols(doc.unit));
return to_raw(
feature::document_symbols(doc.unit, feature::PositionEncoding::UTF16));
});
case K::DocumentLink:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::document_links(doc.unit));
return to_raw(
feature::document_links(doc.unit, feature::PositionEncoding::UTF16));
});
case K::CodeAction:
// TODO: Implement code actions

View File

@@ -1,10 +1,10 @@
#include "server/stateless_worker.h"
#include "server/worker/stateless_worker.h"
#include "compile/compilation.h"
#include "feature/feature.h"
#include "index/tu_index.h"
#include "server/protocol.h"
#include "server/worker_common.h"
#include "server/protocol/worker.h"
#include "server/worker/worker_common.h"
#include "support/logging.h"
#include "kota/async/async.h"
@@ -15,6 +15,22 @@
namespace clice {
/// RAII guard that lowers the current process's scheduling priority and
/// restores it on destruction.
struct ScopedNice {
int saved;
explicit ScopedNice(int increment = 10) {
auto p = kota::sys::priority();
saved = p ? *p : 0;
kota::sys::set_priority(saved + increment);
}
~ScopedNice() {
kota::sys::set_priority(saved);
}
};
using kota::ipc::RequestResult;
using RequestContext = kota::ipc::BincodePeer::RequestContext;
@@ -258,6 +274,22 @@ static worker::BuildResult handle_signature_help(const worker::BuildParams& para
return result;
}
static worker::BuildResult handle_format(const worker::BuildParams& params) {
ScopedTimer timer;
std::optional<LocalSourceRange> range;
if(params.format_range.valid()) {
range = params.format_range;
}
auto edits = feature::document_format(params.file, params.text, range);
LOG_DEBUG("Format done: {} edits, {}ms", edits.size(), timer.ms());
worker::BuildResult result;
result.result_json = to_raw(edits);
return result;
}
int run_stateless_worker_mode(const std::string& worker_name, const std::string& log_dir) {
logging::stderr_logger(worker_name, logging::options);
if(!log_dir.empty()) {
@@ -283,9 +315,13 @@ int run_stateless_worker_mode(const std::string& worker_name, const std::string&
switch(params.kind) {
case K::BuildPCH: return handle_build_pch(params);
case K::BuildPCM: return handle_build_pcm(params);
case K::Index: return handle_index(params);
case K::Index: {
ScopedNice guard;
return handle_index(params);
}
case K::Completion: return handle_completion(params);
case K::SignatureHelp: return handle_signature_help(params);
case K::Format: return handle_format(params);
}
return {false, "Unknown build kind"};
});

View File

@@ -8,8 +8,7 @@
#include "compile/compilation.h"
#include "kota/codec/json/serializer.h"
#include "kota/codec/raw_value.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
namespace clice {

View File

@@ -1,4 +1,4 @@
#include "server/worker_pool.h"
#include "server/worker/worker_pool.h"
#include <csignal>
#include <string>
@@ -13,14 +13,13 @@ namespace {
/// Coroutine that drains a worker's stderr pipe.
/// Workers write their own log files, so this only captures unexpected output
/// (crash stacktraces, assertion failures, etc.) that bypasses spdlog.
/// (crash stacktraces, assertion failures, sanitizer reports, etc.).
kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
std::string buffer;
while(true) {
auto result = co_await stderr_pipe.read();
if(!result.has_value()) {
if(!result.has_value())
break;
}
auto& chunk = result.value();
if(chunk.empty())
break;
@@ -34,7 +33,7 @@ kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
break;
auto line = buffer.substr(pos, nl - pos);
if(!line.empty()) {
LOG_DEBUG("{} {}", prefix, line);
LOG_WARN("{} {}", prefix, line);
}
pos = nl + 1;
}
@@ -42,7 +41,7 @@ kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
}
if(!buffer.empty()) {
LOG_DEBUG("{} {}", prefix, buffer);
LOG_WARN("{} {}", prefix, buffer);
}
}
@@ -97,9 +96,8 @@ bool WorkerPool::spawn_worker(const std::string& self_path,
std::move(spawn.stdin_pipe));
auto peer = std::make_unique<kota::ipc::BincodePeer>(loop, std::move(transport));
// Schedule stderr log collection
std::string prefix = "[" + worker_name + "]";
loop.schedule(drain_stderr(std::move(spawn.stderr_pipe), prefix));
io_group.spawn(drain_stderr(std::move(spawn.stderr_pipe), prefix));
workers.push_back(WorkerProcess{
.proc = std::move(spawn.proc),
@@ -108,24 +106,28 @@ bool WorkerPool::spawn_worker(const std::string& self_path,
});
auto& w = workers.back();
loop.schedule(w.peer->run());
w.alive = true;
io_group.spawn(w.peer->run());
return true;
}
bool WorkerPool::start(const WorkerPoolOptions& options) {
options_ = options;
log_dir_ = options.log_dir;
for(std::uint32_t i = 0; i < options.stateless_count; ++i) {
if(!spawn_worker(options.self_path, false, 0)) {
return false;
}
monitor_group.spawn(monitor_worker(stateless_workers.size() - 1, false));
}
for(std::uint32_t i = 0; i < options.stateful_count; ++i) {
if(!spawn_worker(options.self_path, true, options.worker_memory_limit)) {
return false;
}
monitor_group.spawn(monitor_worker(stateful_workers.size() - 1, true));
}
// Register evicted notification handler for each stateful worker
@@ -145,30 +147,19 @@ bool WorkerPool::start(const WorkerPoolOptions& options) {
kota::task<> WorkerPool::stop() {
LOG_INFO("WorkerPool stopping...");
shutting_down_ = true;
// Close output pipes to signal workers to exit gracefully
for(auto& w: stateless_workers) {
for(auto& w: stateless_workers)
w.peer->close_output();
}
for(auto& w: stateful_workers) {
for(auto& w: stateful_workers)
w.peer->close_output();
}
// Send SIGTERM to all workers
for(auto& w: stateless_workers) {
for(auto& w: stateless_workers)
w.proc.kill(SIGTERM);
}
for(auto& w: stateful_workers) {
for(auto& w: stateful_workers)
w.proc.kill(SIGTERM);
}
// Wait for all worker processes to exit
for(auto& w: stateless_workers) {
co_await w.proc.wait();
}
for(auto& w: stateful_workers) {
co_await w.proc.wait();
}
co_await kota::when_all(monitor_group.join(), io_group.join());
LOG_INFO("WorkerPool stopped");
}
@@ -198,7 +189,10 @@ std::size_t WorkerPool::assign_worker(std::uint32_t path_id) {
std::size_t WorkerPool::pick_least_loaded() {
std::size_t best = 0;
for(std::size_t i = 1; i < stateful_workers.size(); ++i) {
if(stateful_workers[i].owned_documents < stateful_workers[best].owned_documents) {
if(!stateful_workers[i].alive)
continue;
if(!stateful_workers[best].alive ||
stateful_workers[i].owned_documents < stateful_workers[best].owned_documents) {
best = i;
}
}
@@ -233,4 +227,122 @@ void WorkerPool::clear_owner(std::size_t worker_index) {
}
}
kota::task<> WorkerPool::monitor_worker(std::size_t index, bool stateful) {
auto& workers = stateful ? stateful_workers : stateless_workers;
auto& w = workers[index];
auto name = std::string(stateful ? "SF-" : "SL-") + std::to_string(index);
auto result = co_await w.proc.wait();
w.alive = false;
if(shutting_down_)
co_return;
if(result.has_value()) {
auto& exit = result.value();
if(exit.term_signal != 0) {
LOG_ERROR("Worker {} killed by signal {} (restarts: {})",
name,
exit.term_signal,
w.restart_count);
} else {
LOG_ERROR("Worker {} exited with code {} (restarts: {})",
name,
exit.status,
w.restart_count);
}
} else {
LOG_ERROR("Worker {} lost: {} (restarts: {})",
name,
result.error().message(),
w.restart_count);
}
if(stateful)
clear_owner(index);
constexpr unsigned max_restarts = 5;
if(w.restart_count >= max_restarts) {
LOG_ERROR("Worker {} exceeded max restarts ({}), giving up", name, max_restarts);
co_return;
}
if(!respawn_worker(index, stateful)) {
LOG_ERROR("Worker {} respawn failed", name);
}
}
bool WorkerPool::respawn_worker(std::size_t index, bool stateful) {
auto& workers = stateful ? stateful_workers : stateless_workers;
auto old_restart_count = workers[index].restart_count + 1;
auto worker_name = std::string(stateful ? "SF-" : "SL-") + std::to_string(index);
// Close the old peer and retire it so its coroutines (run/write_loop)
// can finish naturally before the object is destroyed.
if(workers[index].peer) {
workers[index].peer->close();
retired_peers.push_back(std::move(workers[index].peer));
}
kota::process::options opts;
opts.file = options_.self_path;
if(stateful) {
opts.args = {options_.self_path,
"--mode",
"stateful-worker",
"--worker-memory-limit",
std::to_string(options_.worker_memory_limit)};
} else {
opts.args = {options_.self_path, "--mode", "stateless-worker"};
}
opts.args.push_back("--worker-name");
opts.args.push_back(worker_name);
if(!log_dir_.empty()) {
opts.args.push_back("--log-dir");
opts.args.push_back(log_dir_);
}
opts.streams = {
kota::process::stdio::pipe(true, false),
kota::process::stdio::pipe(false, true),
kota::process::stdio::pipe(false, true),
};
auto result = kota::process::spawn(opts, loop);
if(!result) {
LOG_ERROR("Failed to respawn worker {}: {}", worker_name, result.error().message());
return false;
}
auto& spawn = *result;
auto transport = std::make_unique<kota::ipc::StreamTransport>(std::move(spawn.stdout_pipe),
std::move(spawn.stdin_pipe));
auto peer = std::make_unique<kota::ipc::BincodePeer>(loop, std::move(transport));
std::string prefix = "[" + worker_name + "]";
io_group.spawn(drain_stderr(std::move(spawn.stderr_pipe), prefix));
workers[index] = WorkerProcess{
.proc = std::move(spawn.proc),
.peer = std::move(peer),
.owned_documents = 0,
.alive = true,
.restart_count = old_restart_count,
};
auto& w = workers[index];
io_group.spawn(w.peer->run());
if(stateful) {
w.peer->on_notification([this](const worker::EvictedParams& params) {
if(on_evicted)
on_evicted(params.path);
});
}
monitor_group.spawn(monitor_worker(index, stateful));
LOG_INFO("Worker {} restarted (attempt {})", worker_name, old_restart_count);
return true;
}
} // namespace clice

View File

@@ -6,7 +6,7 @@
#include <list>
#include <memory>
#include "server/protocol.h"
#include "server/protocol/worker.h"
#include "kota/async/async.h"
#include "kota/ipc/codec/bincode.h"
@@ -64,6 +64,8 @@ private:
kota::process proc;
std::unique_ptr<kota::ipc::BincodePeer> peer;
std::size_t owned_documents = 0;
bool alive = true;
unsigned restart_count = 0;
};
kota::event_loop& loop;
@@ -80,8 +82,19 @@ private:
void clear_owner(std::size_t worker_index);
std::size_t pick_least_loaded();
bool shutting_down_ = false;
kota::task_group<> monitor_group{loop};
kota::task_group<> io_group{loop};
WorkerPoolOptions options_;
std::string log_dir_;
/// Peers moved here during respawn so their coroutines can finish
/// before the object is destroyed.
llvm::SmallVector<std::unique_ptr<kota::ipc::BincodePeer>> retired_peers;
bool spawn_worker(const std::string& self_path, bool stateful, std::uint64_t memory_limit);
bool respawn_worker(std::size_t index, bool stateful);
kota::task<> monitor_worker(std::size_t index, bool stateful);
};
template <typename Params>
@@ -91,11 +104,10 @@ RequestResult<Params> WorkerPool::send_stateful(std::uint32_t path_id,
if(stateful_workers.empty()) {
co_return kota::outcome_error(kota::ipc::Error{"No stateful workers available"});
}
// No timeout: compile tasks run as detached tasks (loop.schedule) that
// are immune to LSP $/cancelRequest. Adding a timeout here would use
// kotatsu's with_token/when_any which has a spurious-cancellation bug
// that kills requests within milliseconds instead of the configured period.
auto idx = assign_worker(path_id);
if(!stateful_workers[idx].alive) {
co_return kota::outcome_error(kota::ipc::Error{"Assigned stateful worker is down"});
}
co_return co_await stateful_workers[idx].peer->send_request(params, opts);
}
@@ -105,9 +117,16 @@ RequestResult<Params> WorkerPool::send_stateless(const Params& params,
if(stateless_workers.empty()) {
co_return kota::outcome_error(kota::ipc::Error{"No stateless workers available"});
}
auto idx = next_stateless;
next_stateless = (next_stateless + 1) % stateless_workers.size();
co_return co_await stateless_workers[idx].peer->send_request(params, opts);
// Round-robin, skipping dead workers.
auto start = next_stateless;
for(std::size_t i = 0; i < stateless_workers.size(); ++i) {
auto idx = (start + i) % stateless_workers.size();
if(stateless_workers[idx].alive) {
next_stateless = (idx + 1) % stateless_workers.size();
co_return co_await stateless_workers[idx].peer->send_request(params, opts);
}
}
co_return kota::outcome_error(kota::ipc::Error{"All stateless workers are down"});
}
template <typename Params>
@@ -115,6 +134,8 @@ void WorkerPool::notify_stateful(std::uint32_t path_id, const Params& params) {
auto it = owner.find(path_id);
if(it == owner.end())
return;
if(!stateful_workers[it->second].alive)
return;
stateful_workers[it->second].peer->send_notification(params);
}

View File

@@ -1,4 +1,4 @@
#include "server/config.h"
#include "server/workspace/config.h"
#include <algorithm>
@@ -6,8 +6,9 @@
#include "support/glob_pattern.h"
#include "support/logging.h"
#include "kota/async/io/system.h"
#include "kota/codec/json/json.h"
#include "kota/codec/toml.h"
#include "kota/codec/toml/toml.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/Process.h"
@@ -65,8 +66,10 @@ void Config::apply_defaults(llvm::StringRef workspace_root) {
if(p.stateful_worker_count == 0)
p.stateful_worker_count = 2;
if(p.stateless_worker_count == 0)
p.stateless_worker_count = 3;
if(p.stateless_worker_count == 0) {
auto cores = kota::sys::parallelism();
p.stateless_worker_count = std::max(cores / 2, 2u);
}
if(p.worker_memory_limit == 0)
p.worker_memory_limit = 4ULL * 1024 * 1024 * 1024; // 4GB
@@ -153,13 +156,13 @@ std::optional<Config> Config::load(llvm::StringRef path, llvm::StringRef workspa
}
std::optional<Config> Config::load_from_json(llvm::StringRef json, llvm::StringRef workspace_root) {
auto result = kota::codec::json::from_json<Config>(json);
Config config{};
auto result = kota::codec::json::from_json(json, config);
if(!result) {
LOG_WARN("Failed to parse initializationOptions JSON: {}", result.error().message());
LOG_WARN("Failed to parse initializationOptions JSON: {}", result.error().message);
return std::nullopt;
}
auto config = std::move(*result);
config.apply_defaults(workspace_root);
LOG_INFO("Loaded config from initializationOptions");
return config;

View File

@@ -1,4 +1,4 @@
#include "server/workspace.h"
#include "server/workspace/workspace.h"
#include <algorithm>
#include <chrono>

View File

@@ -11,8 +11,8 @@
#include "index/merged_index.h"
#include "index/project_index.h"
#include "semantic/relation_kind.h"
#include "server/compile_graph.h"
#include "server/config.h"
#include "server/compiler/compile_graph.h"
#include "server/workspace/config.h"
#include "support/path_pool.h"
#include "syntax/dependency_graph.h"

View File

@@ -37,6 +37,14 @@ inline std::string real_path(llvm::StringRef file) {
return path.str().str();
}
inline std::string default_socket_path() {
llvm::SmallString<128> home;
if(!llvm::sys::path::home_directory(home))
return "/tmp/clice.sock";
llvm::sys::path::append(home, ".clice", "clice.sock");
return home.str().str();
}
} // namespace path
namespace fs {

View File

@@ -46,8 +46,6 @@ struct LocalSourceRange {
constexpr bool operator==(const LocalSourceRange& other) const = default;
constexpr auto operator<=>(const LocalSourceRange& other) const = default;
constexpr std::uint32_t length() const {
return end - begin;
}

View File

@@ -1,6 +1,7 @@
import asyncio
import json
import shutil
import socket
import subprocess
import sys
from pathlib import Path
@@ -93,24 +94,27 @@ def workspace(request: pytest.FixtureRequest, test_data_dir: Path) -> Path | Non
@pytest.fixture
async def client(
request: pytest.FixtureRequest, executable: Path, workspace: Path | None
request: pytest.FixtureRequest,
executable: Path,
workspace: Path | None,
):
"""Spawn clice server, auto-initialize if @pytest.mark.workspace is present."""
config = request.config
mode = config.getoption("--mode")
host = config.getoption("--host")
cmd = [str(executable), "--mode", mode]
if mode == "socket":
host = config.getoption("--host")
port = config.getoption("--port")
cmd += ["--host", host, "--port", str(port)]
cmd = [str(executable), "--mode", mode, "--host", host]
c = CliceClient()
await c.start_io(*cmd)
if workspace is not None:
init_options_marker = request.node.get_closest_marker("init_options")
init_options = init_options_marker.args[0] if init_options_marker else None
init_options = dict(init_options_marker.args[0]) if init_options_marker else {}
# Force cache_dir into the workspace so .clice/ cleanup prevents stale PCH.
project = dict(init_options.get("project", {}))
project.setdefault("cache_dir", str(workspace / ".clice"))
init_options["project"] = project
await c.initialize(workspace, initialization_options=init_options)
yield c
@@ -118,6 +122,39 @@ async def client(
await _shutdown_client(c)
def _find_free_port() -> int:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(("127.0.0.1", 0))
return s.getsockname()[1]
@pytest.fixture
async def agentic(
request: pytest.FixtureRequest,
executable: Path,
workspace: Path | None,
):
"""Start a server with agentic TCP port, yield (executable, host, port)."""
host = "127.0.0.1"
port = _find_free_port()
cmd = [str(executable), "--mode", "pipe", "--host", host, "--port", str(port)]
c = CliceClient()
await c.start_io(*cmd)
if workspace is not None:
init_options_marker = request.node.get_closest_marker("init_options")
init_options = dict(init_options_marker.args[0]) if init_options_marker else {}
project = dict(init_options.get("project", {}))
project.setdefault("cache_dir", str(workspace / ".clice"))
init_options["project"] = project
await c.initialize(workspace, initialization_options=init_options)
yield executable, host, port
await _shutdown_client(c)
def generate_cdb(workspace: Path) -> None:
"""Generate compile_commands.json using CMake with Ninja backend."""
cmake = shutil.which("cmake")
@@ -165,12 +202,17 @@ async def _shutdown_client(c: CliceClient) -> None:
try:
server = getattr(c, "_server", None)
if server and server.stderr:
stderr_data = await asyncio.wait_for(server.stderr.read(), timeout=2.0)
if stderr_data:
for line in stderr_data.decode("utf-8", errors="replace").splitlines():
if "[warn]" in line or "[error]" in line:
print(f"[server] {line}", flush=True)
if server:
if server.returncode is not None:
print(f"[server] exit code: {server.returncode}", flush=True)
if server.stderr:
stderr_data = await asyncio.wait_for(server.stderr.read(), timeout=2.0)
if stderr_data:
for line in stderr_data.decode(
"utf-8", errors="replace"
).splitlines():
if "[warn]" in line or "[error]" in line or "Sanitizer" in line:
print(f"[server] {line}", flush=True)
except Exception:
pass
@@ -250,6 +292,12 @@ def _generate_test_data_cdbs(data_dir: Path) -> None:
if cr_main.exists():
_write(cr_dir, [_entry(cr_dir, cr_main)])
# formatting
fmt_dir = data_dir / "formatting"
fmt_main = fmt_dir / "main.cpp"
if fmt_main.exists():
_write(fmt_dir, [_entry(fmt_dir, fmt_main)])
# pch_test
pt_dir = data_dir / "pch_test"
if pt_dir.exists():

View File

@@ -0,0 +1,36 @@
// basic if and if-else
namespace basic_if {
int abs_val(int x) {
if(x < 0)
return -x;
return x;
}
const char* sign(int x) {
if(x > 0) {
return "positive";
} else if(x < 0) {
return "negative";
} else {
return "zero";
}
}
// dangling else: else binds to nearest if
int nested_if(int a, int b) {
if(a > 0)
if(b > 0)
return 1;
else
return 2;
return 0;
}
void test() {
[[maybe_unused]] int r1 = abs_val(-3);
[[maybe_unused]] auto r2 = sign(5);
[[maybe_unused]] int r3 = nested_if(1, -1);
}
} // namespace basic_if

View File

@@ -0,0 +1,3 @@
BasedOnStyle: LLVM
IndentWidth: 4
ColumnLimit: 80

View File

@@ -0,0 +1 @@
int add(int a, int b) { return a + b; }

View File

View File

@@ -0,0 +1,592 @@
"""Tests for the agentic protocol handlers."""
import asyncio
import json
import socket
import subprocess
from concurrent.futures import ThreadPoolExecutor
import pytest
from tests.integration.utils.wait import wait_for_index
class AgenticRpcClient:
"""Minimal JSON-RPC client that speaks Content-Length framing over TCP."""
def __init__(self, host: str, port: int):
self.sock = socket.create_connection((host, port), timeout=10)
self.request_id = 0
self.buffer = b""
def request(self, method: str, params: dict):
self.request_id += 1
body = json.dumps(
{
"jsonrpc": "2.0",
"id": self.request_id,
"method": method,
"params": params,
}
)
payload = f"Content-Length: {len(body)}\r\n\r\n{body}".encode("utf-8")
self.sock.sendall(payload)
return self._read_response()
def _read_response(self):
while b"\r\n\r\n" not in self.buffer:
data = self.sock.recv(4096)
if not data:
raise ConnectionError("connection closed")
self.buffer += data
header_end = self.buffer.index(b"\r\n\r\n")
headers = self.buffer[:header_end].decode("utf-8")
self.buffer = self.buffer[header_end + 4 :]
content_length = 0
for line in headers.split("\r\n"):
if line.lower().startswith("content-length:"):
content_length = int(line.split(":")[1].strip())
while len(self.buffer) < content_length:
data = self.sock.recv(4096)
if not data:
raise ConnectionError("connection closed")
self.buffer += data
body = self.buffer[:content_length].decode("utf-8")
self.buffer = self.buffer[content_length:]
return json.loads(body)
def close(self):
self.sock.close()
def run_agentic(executable, host, port, path, timeout=10):
result = subprocess.run(
[
str(executable),
"--mode",
"agentic",
"--host",
host,
"--port",
str(port),
"--path",
path,
],
capture_output=True,
text=True,
timeout=timeout,
)
return result
@pytest.mark.workspace("hello_world")
async def test_compile_command(agentic, workspace):
executable, host, port = agentic
main_cpp = (workspace / "main.cpp").as_posix()
result = run_agentic(executable, host, port, main_cpp)
assert result.returncode == 0, f"stderr: {result.stderr}"
data = json.loads(result.stdout)
assert data["file"] == main_cpp
assert data["directory"] == workspace.as_posix()
assert len(data["arguments"]) > 0
@pytest.mark.workspace("hello_world")
async def test_compile_command_fallback(agentic, workspace):
executable, host, port = agentic
result = run_agentic(executable, host, port, "/nonexistent/file.cpp")
assert result.returncode == 0, f"stderr: {result.stderr}"
data = json.loads(result.stdout)
assert data["file"] == "/nonexistent/file.cpp"
@pytest.mark.workspace("hello_world")
async def test_multiple_requests(agentic, workspace):
executable, host, port = agentic
main_cpp = (workspace / "main.cpp").as_posix()
for _ in range(3):
result = run_agentic(executable, host, port, main_cpp)
assert result.returncode == 0, f"stderr: {result.stderr}"
data = json.loads(result.stdout)
assert data["file"] == main_cpp
async def test_connection_refused(executable):
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(("127.0.0.1", 0))
free_port = s.getsockname()[1]
result = run_agentic(executable, "127.0.0.1", free_port, "/some/file.cpp")
assert result.returncode != 0
@pytest.mark.workspace("hello_world")
async def test_concurrent_connections(agentic, workspace):
executable, host, port = agentic
main_cpp = (workspace / "main.cpp").as_posix()
def do_request(_):
return run_agentic(executable, host, port, main_cpp)
with ThreadPoolExecutor(max_workers=4) as pool:
results = list(pool.map(do_request, range(4)))
for r in results:
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["file"] == main_cpp
@pytest.fixture
async def indexed_agentic(request, executable, workspace):
"""Start server with LSP+agentic, compile a file, wait for indexing."""
from tests.integration.utils.client import CliceClient
from tests.conftest import _shutdown_client, _find_free_port
host = "127.0.0.1"
port = _find_free_port()
cmd = [str(executable), "--mode", "pipe", "--host", host, "--port", str(port)]
c = CliceClient()
await c.start_io(*cmd)
init_options = {"project": {"cache_dir": str(workspace / ".clice")}}
await c.initialize(workspace, initialization_options=init_options)
uri, _ = await c.open_and_wait(workspace / "main.cpp")
assert await wait_for_index(c, uri, "add"), "Index not ready"
rpc = AgenticRpcClient(host, port)
for _ in range(30):
resp = rpc.request("agentic/symbolSearch", {"query": "add"})
if "result" in resp and resp["result"]["symbols"]:
break
await asyncio.sleep(1)
else:
pytest.fail("agentic/symbolSearch never returned indexed symbols")
yield rpc, workspace
rpc.close()
c.close(uri)
await _shutdown_client(c)
@pytest.mark.workspace("index_features")
async def test_rpc_compile_command(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/compileCommand", {"path": path})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["file"] == path
assert len(result["arguments"]) > 0
@pytest.mark.workspace("index_features")
async def test_rpc_project_files(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/projectFiles", {})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["total"] > 0
paths = [f["path"] for f in result["files"]]
assert any("main.cpp" in p for p in paths)
@pytest.mark.workspace("index_features")
async def test_rpc_project_files_filter(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/projectFiles", {"filter": "source"})
assert "result" in resp
for f in resp["result"]["files"]:
assert f["kind"] == "source"
@pytest.mark.workspace("index_features")
async def test_rpc_symbol_search(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/symbolSearch", {"query": "add"})
assert "result" in resp, f"unexpected response: {resp}"
symbols = resp["result"]["symbols"]
add_sym = next((s for s in symbols if s["name"] == "add"), None)
assert add_sym is not None, f"'add' not found in {[s['name'] for s in symbols]}"
assert add_sym["kind"] == "Function"
assert add_sym["line"] == 19
assert add_sym["symbolId"] != 0
assert "main.cpp" in add_sym["file"]
@pytest.mark.workspace("index_features")
async def test_rpc_symbol_search_kind(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request(
"agentic/symbolSearch", {"query": "Animal", "kindFilter": ["Struct"]}
)
assert "result" in resp
for s in resp["result"]["symbols"]:
assert s["kind"] == "Struct"
@pytest.mark.workspace("index_features")
async def test_rpc_symbol_search_max(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/symbolSearch", {"query": "", "maxResults": 3})
assert "result" in resp
assert len(resp["result"]["symbols"]) <= 3
@pytest.mark.workspace("index_features")
async def test_rpc_read_symbol(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/readSymbol", {"name": "add"})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["name"] == "add"
assert result["symbolId"] != 0
assert result["startLine"] == 19
assert result["endLine"] == 21
assert "int add(int a, int b)" in result["text"]
assert "return a + b;" in result["text"]
@pytest.mark.workspace("index_features")
async def test_rpc_read_symbol_by_id(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp1 = rpc.request("agentic/readSymbol", {"name": "add"})
assert "result" in resp1
sid = resp1["result"]["symbolId"]
resp2 = rpc.request("agentic/readSymbol", {"symbolId": sid})
assert "result" in resp2
assert resp2["result"]["name"] == "add"
assert resp2["result"]["symbolId"] == sid
@pytest.mark.workspace("index_features")
async def test_rpc_document_symbols(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/documentSymbols", {"path": path})
assert "result" in resp, f"unexpected response: {resp}"
symbols = resp["result"]["symbols"]
names = [s["name"] for s in symbols]
kinds = [s["kind"] for s in symbols]
assert "add" in names, f"expected 'add' in {names}"
assert "main" in names, f"expected 'main' in {names}"
assert "global_var" in names, f"expected 'global_var' in {names}"
assert "Parameter" not in kinds, (
f"Parameters should be filtered: {list(zip(names, kinds))}"
)
@pytest.mark.workspace("index_features")
async def test_rpc_definition(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/definition", {"name": "add"})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["name"] == "add"
assert result["definition"] is not None
defn = result["definition"]
assert "main.cpp" in defn["file"]
assert defn["startLine"] == 19
assert defn["endLine"] == 21
assert "int add(int a, int b)" in defn["text"]
assert "return a + b;" in defn["text"]
@pytest.mark.workspace("index_features")
async def test_rpc_definition_by_position(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/definition", {"path": path, "line": 19})
assert "result" in resp, f"unexpected response: {resp}"
assert resp["result"]["name"] == "add"
@pytest.mark.workspace("index_features")
async def test_rpc_references(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/references", {"name": "global_var"})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["name"] == "global_var"
assert result["total"] == 2
lines = sorted(r["line"] for r in result["references"])
assert lines == [34, 38]
contexts = [r["context"] for r in result["references"]]
assert any("global_var + 1" in c for c in contexts)
assert any("global_var * 2" in c for c in contexts)
@pytest.mark.workspace("index_features")
async def test_rpc_references_include_decl(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request(
"agentic/references", {"name": "global_var", "includeDeclaration": True}
)
assert "result" in resp
result = resp["result"]
assert result["total"] == 3
lines = sorted(r["line"] for r in result["references"])
assert 31 in lines, f"expected declaration line 31 in {lines}"
@pytest.mark.workspace("index_features")
async def test_rpc_call_graph_incoming(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/callGraph", {"name": "add", "direction": "callers"})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["root"]["name"] == "add"
assert result["root"]["line"] == 19
assert result["root"]["symbolId"] != 0
callers = result["callers"]
caller_names = [c["name"] for c in callers]
assert "compute" in caller_names, f"expected 'compute' in {caller_names}"
compute = next(c for c in callers if c["name"] == "compute")
assert compute["line"] == 24
assert compute["symbolId"] != 0
assert result["callees"] == []
@pytest.mark.workspace("index_features")
async def test_rpc_call_graph_outgoing(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/callGraph", {"name": "compute", "direction": "callees"})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["root"]["name"] == "compute"
callees = result["callees"]
callee_names = [c["name"] for c in callees]
assert "add" in callee_names, f"expected 'add' in {callee_names}"
add_entry = next(c for c in callees if c["name"] == "add")
assert add_entry["line"] == 19
assert result["callers"] == []
@pytest.mark.workspace("index_features")
async def test_rpc_type_hierarchy_supertypes(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request(
"agentic/typeHierarchy", {"name": "Dog", "direction": "supertypes"}
)
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["root"]["name"] == "Dog"
assert result["root"]["line"] == 9
supertypes = result["supertypes"]
supertype_names = [t["name"] for t in supertypes]
assert "Animal" in supertype_names, f"expected 'Animal' in {supertype_names}"
animal = next(t for t in supertypes if t["name"] == "Animal")
assert animal["line"] == 2
assert animal["symbolId"] != 0
@pytest.mark.workspace("index_features")
async def test_rpc_type_hierarchy_subtypes(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request(
"agentic/typeHierarchy", {"name": "Animal", "direction": "subtypes"}
)
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["root"]["name"] == "Animal"
assert result["root"]["line"] == 2
subtypes = result["subtypes"]
subtype_names = [t["name"] for t in subtypes]
assert "Dog" in subtype_names, f"expected 'Dog' in {subtype_names}"
assert "Cat" in subtype_names, f"expected 'Cat' in {subtype_names}"
dog = next(t for t in subtypes if t["name"] == "Dog")
assert dog["line"] == 9
cat = next(t for t in subtypes if t["name"] == "Cat")
assert cat["line"] == 14
@pytest.mark.workspace("index_features")
async def test_rpc_status(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/status", {})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert isinstance(result["idle"], bool)
assert result["total"] > 0
assert isinstance(result["pending"], int)
assert isinstance(result["indexed"], int)
@pytest.mark.workspace("hello_world")
async def test_rpc_shutdown(executable, workspace):
"""Shutdown notification should cause the server to exit."""
from tests.integration.utils.client import CliceClient
from tests.conftest import _shutdown_client, _find_free_port
host = "127.0.0.1"
port = _find_free_port()
cmd = [str(executable), "--mode", "pipe", "--host", host, "--port", str(port)]
c = CliceClient()
await c.start_io(*cmd)
init_options = {"project": {"cache_dir": str(workspace / ".clice")}}
await c.initialize(workspace, initialization_options=init_options)
rpc = AgenticRpcClient(host, port)
body = json.dumps({"jsonrpc": "2.0", "method": "agentic/shutdown", "params": {}})
rpc.sock.sendall(f"Content-Length: {len(body)}\r\n\r\n{body}".encode())
rpc.sock.settimeout(5)
try:
rpc.sock.recv(4096)
except (socket.timeout, OSError):
pass
rpc.sock.close()
import asyncio
for _ in range(20):
if c._server.returncode is not None:
break
await asyncio.sleep(0.5)
assert c._server.returncode is not None, "Server did not exit after shutdown"
@pytest.mark.workspace("index_features")
async def test_rpc_symbol_not_found(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/definition", {"name": "nonexistent_symbol_xyz"})
assert "error" in resp
@pytest.mark.workspace("index_features")
async def test_rpc_symbol_id_roundtrip(indexed_agentic, workspace):
"""Search -> get symbolId -> definition -> verify consistency."""
rpc, _ = indexed_agentic
search = rpc.request("agentic/symbolSearch", {"query": "compute"})
assert "result" in search
symbols = search["result"]["symbols"]
compute = next((s for s in symbols if s["name"] == "compute"), None)
assert compute is not None, f"'compute' not found in {[s['name'] for s in symbols]}"
defn = rpc.request("agentic/definition", {"symbolId": compute["symbolId"]})
assert "result" in defn
assert defn["result"]["name"] == "compute"
assert defn["result"]["symbolId"] == compute["symbolId"]
@pytest.mark.workspace("index_features")
async def test_rpc_file_deps(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/fileDeps", {"path": path})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["file"] == path
assert isinstance(result["includes"], list)
assert isinstance(result["includers"], list)
@pytest.mark.workspace("index_features")
async def test_rpc_file_deps_direction(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/fileDeps", {"path": path, "direction": "includes"})
assert "result" in resp
assert resp["result"]["includers"] == []
@pytest.mark.workspace("index_features")
async def test_rpc_file_deps_unknown(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/fileDeps", {"path": "/nonexistent/file.cpp"})
assert "result" in resp
assert resp["result"]["includes"] == []
assert resp["result"]["includers"] == []
@pytest.mark.workspace("index_features")
async def test_rpc_impact_analysis(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/impactAnalysis", {"path": path})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert isinstance(result["directDependents"], list)
assert isinstance(result["transitiveDependents"], list)
assert isinstance(result["affectedModules"], list)
@pytest.mark.workspace("index_features")
async def test_rpc_impact_analysis_unknown(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/impactAnalysis", {"path": "/nonexistent/file.cpp"})
assert "result" in resp
assert resp["result"]["directDependents"] == []
async def test_shutdown_during_indexing(executable, tmp_path):
"""Shutdown during active background indexing must exit cleanly."""
from tests.integration.utils.client import CliceClient
from tests.conftest import _find_free_port
workspace = tmp_path / "ws"
workspace.mkdir()
entries = []
for i in range(20):
src = workspace / f"file_{i}.cpp"
src.write_text(
f"struct Type_{i} {{ int v = {i}; void m() {{}} }};\n"
f"int func_{i}(int x) {{ return x + {i}; }}\n"
f"int caller_{i}() {{ return func_{i}({i}); }}\n"
)
entries.append(
{
"directory": workspace.as_posix(),
"file": src.as_posix(),
"arguments": ["clang++", "-std=c++17", "-fsyntax-only", src.as_posix()],
}
)
(workspace / "compile_commands.json").write_text(json.dumps(entries))
host = "127.0.0.1"
port = _find_free_port()
cmd = [str(executable), "--mode", "pipe", "--host", host, "--port", str(port)]
c = CliceClient()
await c.start_io(*cmd)
init_options = {
"project": {
"cache_dir": str(workspace / ".clice"),
"idle_timeout_ms": 0,
}
}
await c.initialize(workspace, initialization_options=init_options)
# Give indexing a moment to start, then send shutdown
await asyncio.sleep(0.5)
rpc = AgenticRpcClient(host, port)
body = json.dumps({"jsonrpc": "2.0", "method": "agentic/shutdown", "params": {}})
rpc.sock.sendall(f"Content-Length: {len(body)}\r\n\r\n{body}".encode())
rpc.sock.settimeout(5)
try:
rpc.sock.recv(4096)
except (socket.timeout, OSError):
pass
rpc.sock.close()
for _ in range(30):
if c._server.returncode is not None:
break
await asyncio.sleep(0.5)
assert c._server.returncode is not None, "Server did not exit after shutdown"
assert c._server.returncode >= 0, (
f"Server crashed with signal {-c._server.returncode}"
)

View File

@@ -0,0 +1,189 @@
"""CLI-based tests for agentic mode — run clice --mode agentic as a subprocess."""
import json
import subprocess
import pytest
from tests.integration.utils.wait import wait_for_index
def run_cli(executable, host, port, method, **kwargs):
cmd = [
str(executable),
"--mode",
"agentic",
"--host",
host,
"--port",
str(port),
"--method",
method,
]
for k, v in kwargs.items():
cmd.extend([f"--{k}", str(v)])
result = subprocess.run(cmd, capture_output=True, text=True, timeout=10)
return result
@pytest.fixture
async def indexed_server(request, executable, workspace):
"""Start server with LSP+agentic, compile a file, wait for indexing."""
import asyncio
from tests.integration.utils.client import CliceClient
from tests.conftest import _shutdown_client, _find_free_port
host = "127.0.0.1"
port = _find_free_port()
cmd = [str(executable), "--mode", "pipe", "--host", host, "--port", str(port)]
c = CliceClient()
await c.start_io(*cmd)
init_options = {"project": {"cache_dir": str(workspace / ".clice")}}
await c.initialize(workspace, initialization_options=init_options)
uri, _ = await c.open_and_wait(workspace / "main.cpp")
assert await wait_for_index(c, uri, "add"), "Index not ready"
from tests.integration.agentic.test_agentic import AgenticRpcClient
rpc = AgenticRpcClient(host, port)
for _ in range(30):
resp = rpc.request("agentic/symbolSearch", {"query": "add"})
if "result" in resp and resp["result"]["symbols"]:
break
await asyncio.sleep(1)
rpc.close()
yield executable, host, port, workspace
c.close(uri)
await _shutdown_client(c)
@pytest.mark.workspace("index_features")
async def test_cli_compile_command(indexed_server, workspace):
exe, host, port, _ = indexed_server
path = (workspace / "main.cpp").as_posix()
r = run_cli(exe, host, port, "compileCommand", path=path)
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["file"] == path
assert len(data["arguments"]) > 0
@pytest.mark.workspace("index_features")
async def test_cli_symbol_search(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "symbolSearch", query="add")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
names = [s["name"] for s in data["symbols"]]
assert "add" in names
add_sym = next(s for s in data["symbols"] if s["name"] == "add")
assert add_sym["kind"] == "Function"
assert add_sym["line"] == 19
@pytest.mark.workspace("index_features")
async def test_cli_definition(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "definition", name="add")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["name"] == "add"
defn = data["definition"]
assert defn["startLine"] == 19
assert defn["endLine"] == 21
assert "return a + b;" in defn["text"]
@pytest.mark.workspace("index_features")
async def test_cli_definition_by_position(indexed_server, workspace):
exe, host, port, _ = indexed_server
path = (workspace / "main.cpp").as_posix()
r = run_cli(exe, host, port, "definition", path=path, line=19)
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["name"] == "add"
@pytest.mark.workspace("index_features")
async def test_cli_references(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "references", name="global_var")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["name"] == "global_var"
assert data["total"] == 2
lines = sorted(ref["line"] for ref in data["references"])
assert lines == [34, 38]
@pytest.mark.workspace("index_features")
async def test_cli_read_symbol(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "readSymbol", name="compute")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["name"] == "compute"
assert "add(1, 2)" in data["text"]
@pytest.mark.workspace("index_features")
async def test_cli_document_symbols(indexed_server, workspace):
exe, host, port, _ = indexed_server
path = (workspace / "main.cpp").as_posix()
r = run_cli(exe, host, port, "documentSymbols", path=path)
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
names = [s["name"] for s in data["symbols"]]
assert "add" in names
assert "main" in names
assert "global_var" in names
@pytest.mark.workspace("index_features")
async def test_cli_call_graph(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "callGraph", name="add", direction="callers")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["root"]["name"] == "add"
caller_names = [c["name"] for c in data["callers"]]
assert "compute" in caller_names
@pytest.mark.workspace("index_features")
async def test_cli_type_hierarchy(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "typeHierarchy", name="Dog", direction="supertypes")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["root"]["name"] == "Dog"
supertype_names = [t["name"] for t in data["supertypes"]]
assert "Animal" in supertype_names
@pytest.mark.workspace("index_features")
async def test_cli_project_files(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "projectFiles")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["total"] > 0
paths = [f["path"] for f in data["files"]]
assert any("main.cpp" in p for p in paths)
@pytest.mark.workspace("index_features")
async def test_cli_status(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "status")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert isinstance(data["idle"], bool)
assert data["total"] > 0
assert isinstance(data["pending"], int)
assert isinstance(data["indexed"], int)

View File

@@ -0,0 +1,74 @@
import pytest
from lsprotocol.types import Position, Range
from tests.integration.utils.workspace import did_change
UNFORMATTED = "int add( int a , int b ) {\nreturn a+b ;\n}\n"
FORMATTED = "int add(int a, int b) { return a + b; }\n"
def apply_edits(text, edits):
"""Apply LSP TextEdits to a string, processing from end to start."""
lines = text.split("\n")
for edit in sorted(
edits, key=lambda e: (e.range.start.line, e.range.start.character), reverse=True
):
start = edit.range.start
end = edit.range.end
before = (
"\n".join(lines[: start.line])
+ ("\n" if start.line > 0 else "")
+ lines[start.line][: start.character]
)
after = (
lines[end.line][end.character :]
+ ("\n" if end.line < len(lines) - 1 else "")
+ "\n".join(lines[end.line + 1 :])
)
text = before + edit.new_text + after
lines = text.split("\n")
return text
@pytest.mark.workspace("formatting")
async def test_format_document(client, workspace):
uri, _ = await client.open_and_wait(workspace / "main.cpp")
did_change(client, uri, 1, UNFORMATTED)
edits = await client.format_document(uri)
assert edits is not None
assert len(edits) > 0
result = apply_edits(UNFORMATTED, edits)
assert result == FORMATTED
client.close(uri)
@pytest.mark.workspace("formatting")
async def test_format_range(client, workspace):
uri, _ = await client.open_and_wait(workspace / "main.cpp")
did_change(client, uri, 1, UNFORMATTED)
edits = await client.format_range(
uri,
Range(start=Position(line=1, character=0), end=Position(line=2, character=0)),
)
assert edits is not None
assert len(edits) > 0
client.close(uri)
@pytest.mark.workspace("formatting")
async def test_format_already_formatted(client, workspace):
uri, _ = await client.open_and_wait(workspace / "main.cpp")
did_change(client, uri, 1, FORMATTED)
edits = await client.format_document(uri)
assert edits is not None
assert len(edits) == 0
client.close(uri)

View File

@@ -34,6 +34,8 @@ async def test_capabilities(client, workspace):
assert capability_enabled(caps.folding_range_provider)
assert capability_enabled(caps.inlay_hint_provider)
assert capability_enabled(caps.code_action_provider)
assert caps.document_formatting_provider is True
assert caps.document_range_formatting_provider is True
assert caps.semantic_tokens_provider is not None

View File

@@ -16,9 +16,12 @@ from lsprotocol.types import (
Diagnostic,
DidCloseTextDocumentParams,
DidOpenTextDocumentParams,
DocumentFormattingParams,
DocumentLinkParams,
DocumentRangeFormattingParams,
DocumentSymbolParams,
FoldingRangeParams,
FormattingOptions,
HoverParams,
InlayHintParams,
InitializeParams,
@@ -92,13 +95,18 @@ class CliceClient(BaseLanguageClient):
*,
initialization_options: dict | None = None,
) -> InitializeResult:
if initialization_options is None:
initialization_options = {}
project = dict(initialization_options.get("project", {}))
project.setdefault("cache_dir", str(workspace / ".clice"))
initialization_options["project"] = project
params = InitializeParams(
capabilities=ClientCapabilities(),
root_uri=workspace.as_uri(),
workspace_folders=[WorkspaceFolder(uri=workspace.as_uri(), name="test")],
)
if initialization_options is not None:
params.initialization_options = initialization_options
params.initialization_options = initialization_options
result = await self.initialize_async(params)
self.initialized(InitializedParams())
self.init_result = result
@@ -307,6 +315,29 @@ class CliceClient(BaseLanguageClient):
timeout=timeout,
)
async def format_document(self, uri: str, *, timeout: float = 30.0):
return await asyncio.wait_for(
self.text_document_formatting_async(
DocumentFormattingParams(
text_document=TextDocumentIdentifier(uri=uri),
options=FormattingOptions(tab_size=4, insert_spaces=True),
)
),
timeout=timeout,
)
async def format_range(self, uri: str, range_: Range, *, timeout: float = 30.0):
return await asyncio.wait_for(
self.text_document_range_formatting_async(
DocumentRangeFormattingParams(
text_document=TextDocumentIdentifier(uri=uri),
range=range_,
options=FormattingOptions(tab_size=4, insert_spaces=True),
)
),
timeout=timeout,
)
# ── Extension protocol ───────────────────────────────────────────
async def query_context(self, uri: str, *, timeout: float = 30.0):

View File

@@ -13,6 +13,9 @@ import re
import signal
import sys
import time
# Force line-buffered stdout so CI sees output immediately.
sys.stdout.reconfigure(line_buffering=True)
from pathlib import Path
from urllib.parse import quote, unquote
@@ -109,7 +112,9 @@ async def write_lsp_message(writer: asyncio.StreamWriter, payload: str):
await writer.drain()
async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool | None:
async def replay_one(
trace_path: Path, clice_bin: Path, timeout: int, wall_timeout: int = 300
) -> bool | None:
"""Replay a single trace. Returns True=PASS, False=FAIL, None=SKIP."""
records = load_trace(trace_path)
if not records:
@@ -179,8 +184,21 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
last_method = None
sent_count = 0
wall_deadline = wall_start + wall_timeout
def remaining_wall():
return max(0, wall_deadline - time.monotonic())
try:
for i, rec in enumerate(records):
if remaining_wall() <= 0:
elapsed = time.monotonic() - wall_start
print(
f" result: TIMEOUT (wall-clock {wall_timeout}s exceeded, {elapsed:.1f}s)"
)
success = False
break
if i > 0:
delay = rec["ts"] - records[i - 1]["ts"]
if delay > 0:
@@ -196,7 +214,7 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
try:
await asyncio.wait_for(
asyncio.gather(*pending.values(), return_exceptions=True),
timeout=timeout,
timeout=min(timeout, remaining_wall()),
)
except asyncio.TimeoutError:
elapsed = time.monotonic() - wall_start
@@ -210,7 +228,19 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
if msg_id is not None and method is not None:
pending[msg_id] = asyncio.get_event_loop().create_future()
await write_lsp_message(proc.stdin, rec["msg"])
try:
await asyncio.wait_for(
write_lsp_message(proc.stdin, rec["msg"]),
timeout=min(30, remaining_wall()),
)
except asyncio.TimeoutError:
elapsed = time.monotonic() - wall_start
print(
f" result: HANG (write blocked at {last_method},"
f" sent={sent_count}/{len(records)}, {elapsed:.1f}s)"
)
success = False
break
sent_count = i + 1
except (ConnectionError, BrokenPipeError):
@@ -231,7 +261,7 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
try:
await asyncio.wait_for(
asyncio.gather(*pending.values(), return_exceptions=True),
timeout=timeout,
timeout=min(timeout, remaining_wall()),
)
except asyncio.TimeoutError:
elapsed = time.monotonic() - wall_start
@@ -294,7 +324,7 @@ async def async_main(args):
print(f"SKIP: {trace} (not found)")
skipped += 1
continue
result = await replay_one(trace, args.clice, args.timeout)
result = await replay_one(trace, args.clice, args.timeout, args.wall_timeout)
if result is None:
skipped += 1
elif result:
@@ -317,7 +347,16 @@ def main():
p.add_argument("traces", nargs="+", type=Path, help="JSONL trace files")
p.add_argument("--clice", required=True, type=Path, help="Path to clice binary")
p.add_argument(
"--timeout", type=int, default=120, help="Timeout in seconds (default: 120)"
"--timeout",
type=int,
default=120,
help="Per-request timeout in seconds (default: 120)",
)
p.add_argument(
"--wall-timeout",
type=int,
default=300,
help="Max wall-clock time per trace in seconds (default: 300)",
)
args = p.parse_args()
sys.exit(asyncio.run(async_main(args)))

View File

@@ -0,0 +1,14 @@
---
source: document_symbol_tests.cpp
created_at: 2026-05-20
input_file: statements/if/basic_if.cpp
---
- { name: "basic_if", kind: Namespace, range: "1:0-35:1", selection_range: "1:10-1:18" }
- { name: "abs_val", kind: Function, range: "3:0-7:1", selection_range: "3:4-3:11", detail: "int (int)" }
- { name: "sign", kind: Function, range: "9:0-17:1", selection_range: "9:12-9:16", detail: "const char *(int)" }
- { name: "nested_if", kind: Function, range: "20:0-27:1", selection_range: "20:4-20:13", detail: "int (int, int)" }
- { name: "test", kind: Function, range: "29:0-33:1", selection_range: "29:5-29:9", detail: "void ()" }
- { name: "r1", kind: Variable, range: "30:21-30:41", selection_range: "30:25-30:27", detail: "int" }
- { name: "r2", kind: Variable, range: "31:21-31:38", selection_range: "31:26-31:28", detail: "const char *" }
- { name: "r3", kind: Variable, range: "32:21-32:46", selection_range: "32:25-32:27", detail: "int" }

View File

@@ -0,0 +1,11 @@
---
source: folding_range_tests.cpp
created_at: 2026-05-20
input_file: statements/if/basic_if.cpp
---
- { range: "1:19-35:1", kind: namespace, collapsed_text: "{...}" }
- { range: "3:19-7:1", kind: functionBody, collapsed_text: "{...}" }
- { range: "9:24-17:1", kind: functionBody, collapsed_text: "{...}" }
- { range: "20:28-27:1", kind: functionBody, collapsed_text: "{...}" }
- { range: "29:12-33:1", kind: functionBody, collapsed_text: "{...}" }

View File

@@ -0,0 +1,11 @@
---
source: inlay_hint_tests.cpp
created_at: 2026-05-20
input_file: statements/if/basic_if.cpp
---
- { pos: "30:38", kind: Parameter, label: "x:", padding_right: true }
- { pos: "31:28", kind: Type, label: ": const char *" }
- { pos: "31:36", kind: Parameter, label: "x:", padding_right: true }
- { pos: "32:40", kind: Parameter, label: "a:", padding_right: true }
- { pos: "32:43", kind: Parameter, label: "b:", padding_right: true }

View File

@@ -0,0 +1,75 @@
---
source: semantic_tokens_tests.cpp
created_at: 2026-05-20
input_file: statements/if/basic_if.cpp
---
- { loc: "0:0", text: "// basic if and if-else", kind: Comment }
- { loc: "1:0", text: "namespace", kind: Keyword }
- { loc: "1:10", text: "basic_if", kind: Namespace, modifiers: [Definition] }
- { loc: "3:0", text: "int", kind: Keyword }
- { loc: "3:4", text: "abs_val", kind: Function, modifiers: [Definition] }
- { loc: "3:12", text: "int", kind: Keyword }
- { loc: "3:16", text: "x", kind: Parameter, modifiers: [Definition] }
- { loc: "4:4", text: "if", kind: Keyword }
- { loc: "4:7", text: "x", kind: Parameter }
- { loc: "4:11", text: "0", kind: Number }
- { loc: "5:8", text: "return", kind: Keyword }
- { loc: "5:16", text: "x", kind: Parameter }
- { loc: "6:4", text: "return", kind: Keyword }
- { loc: "6:11", text: "x", kind: Parameter }
- { loc: "9:0", text: "const", kind: Keyword }
- { loc: "9:6", text: "char", kind: Keyword }
- { loc: "9:12", text: "sign", kind: Function, modifiers: [Definition, Readonly] }
- { loc: "9:17", text: "int", kind: Keyword }
- { loc: "9:21", text: "x", kind: Parameter, modifiers: [Definition] }
- { loc: "10:4", text: "if", kind: Keyword }
- { loc: "10:7", text: "x", kind: Parameter }
- { loc: "10:11", text: "0", kind: Number }
- { loc: "11:8", text: "return", kind: Keyword }
- { loc: "11:15", text: "\"positive\"", kind: String }
- { loc: "12:6", text: "else", kind: Keyword }
- { loc: "12:11", text: "if", kind: Keyword }
- { loc: "12:14", text: "x", kind: Parameter }
- { loc: "12:18", text: "0", kind: Number }
- { loc: "13:8", text: "return", kind: Keyword }
- { loc: "13:15", text: "\"negative\"", kind: String }
- { loc: "14:6", text: "else", kind: Keyword }
- { loc: "15:8", text: "return", kind: Keyword }
- { loc: "15:15", text: "\"zero\"", kind: String }
- { loc: "19:0", text: "// dangling else: else binds to nearest if", kind: Comment }
- { loc: "20:0", text: "int", kind: Keyword }
- { loc: "20:4", text: "nested_if", kind: Function, modifiers: [Definition] }
- { loc: "20:14", text: "int", kind: Keyword }
- { loc: "20:18", text: "a", kind: Parameter, modifiers: [Definition] }
- { loc: "20:21", text: "int", kind: Keyword }
- { loc: "20:25", text: "b", kind: Parameter, modifiers: [Definition] }
- { loc: "21:4", text: "if", kind: Keyword }
- { loc: "21:7", text: "a", kind: Parameter }
- { loc: "21:11", text: "0", kind: Number }
- { loc: "22:8", text: "if", kind: Keyword }
- { loc: "22:11", text: "b", kind: Parameter }
- { loc: "22:15", text: "0", kind: Number }
- { loc: "23:12", text: "return", kind: Keyword }
- { loc: "23:19", text: "1", kind: Number }
- { loc: "24:8", text: "else", kind: Keyword }
- { loc: "25:12", text: "return", kind: Keyword }
- { loc: "25:19", text: "2", kind: Number }
- { loc: "26:4", text: "return", kind: Keyword }
- { loc: "26:11", text: "0", kind: Number }
- { loc: "29:0", text: "void", kind: Keyword }
- { loc: "29:5", text: "test", kind: Function, modifiers: [Definition] }
- { loc: "30:21", text: "int", kind: Keyword }
- { loc: "30:25", text: "r1", kind: Variable, modifiers: [Definition] }
- { loc: "30:30", text: "abs_val", kind: Function }
- { loc: "30:39", text: "3", kind: Number }
- { loc: "31:21", text: "auto", kind: Keyword }
- { loc: "31:26", text: "r2", kind: Variable, modifiers: [Definition, Readonly] }
- { loc: "31:31", text: "sign", kind: Function, modifiers: [Readonly] }
- { loc: "31:36", text: "5", kind: Number }
- { loc: "32:21", text: "int", kind: Keyword }
- { loc: "32:25", text: "r3", kind: Variable, modifiers: [Definition] }
- { loc: "32:30", text: "nested_if", kind: Function }
- { loc: "32:40", text: "1", kind: Number }
- { loc: "32:44", text: "1", kind: Number }
- { loc: "35:3", text: "// namespace basic_if", kind: Comment }

View File

@@ -0,0 +1,28 @@
---
source: tu_index_tests.cpp
created_at: 2026-05-20
input_file: statements/if/basic_if.cpp
---
- { loc: "1:10", kind: Namespace, text: "basic_if", relations: [Definition] }
- { loc: "3:4", kind: Function, text: "abs_val", relations: [Definition] }
- { loc: "3:16", kind: Parameter, text: "x", relations: [Definition] }
- { loc: "4:7", kind: Parameter, text: "x", relations: [Reference] }
- { loc: "5:16", kind: Parameter, text: "x", relations: [Reference] }
- { loc: "6:11", kind: Parameter, text: "x", relations: [Reference] }
- { loc: "9:12", kind: Function, text: "sign", relations: [Definition] }
- { loc: "9:21", kind: Parameter, text: "x", relations: [Definition] }
- { loc: "10:7", kind: Parameter, text: "x", relations: [Reference] }
- { loc: "12:14", kind: Parameter, text: "x", relations: [Reference] }
- { loc: "20:4", kind: Function, text: "nested_if", relations: [Definition] }
- { loc: "20:18", kind: Parameter, text: "a", relations: [Definition] }
- { loc: "20:25", kind: Parameter, text: "b", relations: [Definition] }
- { loc: "21:7", kind: Parameter, text: "a", relations: [Reference] }
- { loc: "22:11", kind: Parameter, text: "b", relations: [Reference] }
- { loc: "29:5", kind: Function, text: "test", relations: [Definition] }
- { loc: "30:25", kind: Variable, text: "r1", relations: [Definition] }
- { loc: "30:30", kind: Function, text: "abs_val", relations: [Reference] }
- { loc: "31:26", kind: Variable, text: "r2", relations: [Definition] }
- { loc: "31:31", kind: Function, text: "sign", relations: [Reference] }
- { loc: "32:25", kind: Variable, text: "r3", relations: [Definition] }
- { loc: "32:30", kind: Function, text: "nested_if", relations: [Reference] }

View File

@@ -11,7 +11,7 @@ namespace {
namespace protocol = kota::ipc::protocol;
TEST_SUITE(DocumentLink, Tester) {
TEST_SUITE(document_link, Tester) {
std::vector<protocol::DocumentLink> links;
@@ -136,7 +136,7 @@ ABCDE
EXPECT_LINK(0, "0", TestVFS::path("data.bin"));
}
}; // TEST_SUITE(DocumentLink)
}; // TEST_SUITE(document_link)
} // namespace

View File

@@ -1,4 +1,5 @@
#include <cstddef>
#include <format>
#include <functional>
#include <memory>
#include <vector>
@@ -7,13 +8,15 @@
#include "test/tester.h"
#include "feature/feature.h"
#include "kota/meta/enum.h"
namespace clice::testing {
namespace {
namespace protocol = kota::ipc::protocol;
TEST_SUITE(DocumentSymbol, Tester) {
TEST_SUITE(document_symbol, Tester) {
std::vector<protocol::DocumentSymbol> symbols;
@@ -180,7 +183,57 @@ VAR(test)
ASSERT_EQ(total_size(symbols), 3U);
}
}; // TEST_SUITE(DocumentSymbol)
void format_document_symbols(std::string& out,
const feature::PositionMapper& mapper,
llvm::ArrayRef<feature::DocumentSymbol> nodes,
int depth) {
auto pad = std::string(depth * 2, ' ');
for(auto& node: nodes) {
auto kind = kota::meta::enum_name(static_cast<SymbolKind::Kind>(node.kind), "Unknown");
auto start = mapper.to_position(node.range.begin);
auto end = mapper.to_position(node.range.end);
if(!start || !end)
continue;
auto sel_start = mapper.to_position(node.selection_range.begin);
auto sel_end = mapper.to_position(node.selection_range.end);
out += std::format("- {}{{ name: {}, kind: {}, range: \"{}:{}-{}:{}\"",
pad,
yaml_str(node.name),
kind,
start->line,
start->character,
end->line,
end->character);
if(sel_start && sel_end) {
out += std::format(", selection_range: \"{}:{}-{}:{}\"",
sel_start->line,
sel_start->character,
sel_end->line,
sel_end->character);
}
if(!node.detail.empty()) {
out += std::format(", detail: {}", yaml_str(node.detail));
}
out += " }\n";
if(!node.children.empty()) {
format_document_symbols(out, mapper, node.children, depth + 1);
}
}
}
TEST_CASE(snapshot) {
ASSERT_SNAPSHOT_GLOB(corpus_dir, "**/*.cpp", [&](std::string_view path) -> std::string {
if(!compile_file(path))
return "COMPILE_ERROR";
auto content = unit->interested_content();
feature::PositionMapper mapper(content, feature::PositionEncoding::UTF8);
std::string result;
format_document_symbols(result, mapper, feature::document_symbols(*unit), 0);
return result;
});
}
}; // TEST_SUITE(document_symbol)
} // namespace

View File

@@ -11,7 +11,7 @@ namespace {
namespace protocol = kota::ipc::protocol;
TEST_SUITE(FoldingRange, Tester) {
TEST_SUITE(folding_range, Tester) {
std::vector<protocol::FoldingRange> ranges;
@@ -429,7 +429,36 @@ $(1)#pragma region level1
)cpp");
}
}; // TEST_SUITE(FoldingRange)
TEST_CASE(snapshot) {
ASSERT_SNAPSHOT_GLOB(corpus_dir, "**/*.cpp", [&](std::string_view path) -> std::string {
if(!compile_file(path))
return "COMPILE_ERROR";
auto ranges = feature::folding_ranges(*unit);
feature::PositionMapper mapper(unit->interested_content(), feature::PositionEncoding::UTF8);
std::string result;
for(auto& r: ranges) {
auto start = mapper.to_position(r.range.begin);
auto end = mapper.to_position(r.range.end);
if(!start || !end)
continue;
result += std::format("- {{ range: \"{}:{}-{}:{}\"",
start->line,
start->character,
end->line,
end->character);
if(r.kind.has_value()) {
result += std::format(", kind: {}", static_cast<const std::string&>(*r.kind));
}
if(!r.collapsed_text.empty()) {
result += std::format(", collapsed_text: {}", yaml_str(r.collapsed_text));
}
result += " }\n";
}
return result;
});
}
}; // TEST_SUITE(folding_range)
} // namespace

View File

@@ -12,6 +12,29 @@ TEST_CASE(Simple) {
ASSERT_NE(edits.size(), 0U);
}
TEST_CASE(RangeFormat) {
llvm::StringRef code = "int x=1;\nint y = 2 ;\nint z=3;\n";
LocalSourceRange range;
range.begin = static_cast<std::uint32_t>(code.find("int y"));
range.end = static_cast<std::uint32_t>(code.find("\nint z") + 1);
auto range_edits = feature::document_format("main.cpp", code, range);
auto full_edits = feature::document_format("main.cpp", code, std::nullopt);
ASSERT_NE(range_edits.size(), 0U);
EXPECT_LE(range_edits.size(), full_edits.size());
}
TEST_CASE(Idempotent) {
llvm::StringRef code = "int main() {\n return 0;\n}\n";
auto edits = feature::document_format("main.cpp", code, std::nullopt);
EXPECT_EQ(edits.size(), 0U);
}
TEST_CASE(IncludeSort) {
llvm::StringRef code = "#include <vector>\n#include <algorithm>\n\nint main() {}\n";
auto edits = feature::document_format("main.cpp", code, std::nullopt);
ASSERT_NE(edits.size(), 0U);
}
}; // TEST_SUITE(Formatting)
} // namespace

View File

@@ -1,16 +1,19 @@
#include <format>
#include <string>
#include "test/test.h"
#include "test/tester.h"
#include "feature/feature.h"
#include "kota/meta/enum.h"
namespace clice::testing {
namespace {
namespace protocol = kota::ipc::protocol;
TEST_SUITE(InlayHint, Tester) {
TEST_SUITE(inlay_hint, Tester) {
std::vector<protocol::InlayHint> hints;
llvm::DenseMap<std::uint32_t, protocol::InlayHint> hints_map;
@@ -1529,6 +1532,38 @@ TEST_CASE(Dependent, skip = true) {
EXPECT_HINT("2", "par3:");
}
}; // TEST_SUITE(InlayHint)
TEST_CASE(snapshot) {
ASSERT_SNAPSHOT_GLOB(corpus_dir, "**/*.cpp", [&](std::string_view path) -> std::string {
if(!compile_file(path))
return "COMPILE_ERROR";
auto content = unit->interested_content();
LocalSourceRange range(0, content.size());
auto hints = feature::inlay_hints(*unit, range);
feature::PositionMapper mapper(content, feature::PositionEncoding::UTF8);
std::string result;
for(auto& hint: hints) {
auto pos = mapper.to_position(hint.offset);
if(!pos)
continue;
auto kind = kota::meta::enum_name(hint.kind, "Unknown");
result += std::format("- {{ pos: \"{}:{}\", kind: {}, label: {}",
pos->line,
pos->character,
kind,
yaml_str(hint.label));
if(hint.padding_left) {
result += ", padding_left: true";
}
if(hint.padding_right) {
result += ", padding_right: true";
}
result += " }\n";
}
return result;
});
}
}; // TEST_SUITE(inlay_hint)
} // namespace
} // namespace clice::testing

View File

@@ -9,6 +9,8 @@
#include "feature/feature.h"
#include "semantic/symbol_kind.h"
#include "kota/meta/enum.h"
namespace clice::testing {
namespace {
@@ -99,7 +101,7 @@ auto decode_relative_tokens(const protocol::SemanticTokens& tokens) -> std::vect
return result;
}
TEST_SUITE(SemanticTokens, Tester) {
TEST_SUITE(semantic_tokens, Tester) {
protocol::SemanticTokens tokens;
std::vector<DecodedToken> decoded;
@@ -138,6 +140,10 @@ void EXPECT_TOKEN(llvm::StringRef name,
ASSERT_EQ(token->modifiers, expected_modifiers);
}
void EXPECT_NO_TOKEN(llvm::StringRef name) {
ASSERT_TRUE(find_by_range(name) == nullptr);
}
TEST_CASE(BasicLexicalKinds) {
run_utf8(R"cpp(
@d1[#define] @m0[FOO]
@@ -264,6 +270,44 @@ int main() {
EXPECT_TOKEN("x3", SymbolKind::Variable, 0);
}
TEST_CASE(IneligibleOperatorReferenceIsSuppressed) {
run_utf8(R"cpp(
struct S {};
S operator+(S lhs, S rhs);
void use(S lhs, S rhs) {
(void)(lhs @plus[+] rhs);
}
)cpp");
EXPECT_NO_TOKEN("plus");
}
TEST_CASE(ConstructorAndDestructorNamesRemainHighlighted) {
run_utf8(R"cpp(
struct S {
@ctor_decl[S]();
@dtor_decl[~]S();
};
S::@ctor_def[S]() {}
void use(S* value) {
value->@dtor_ref[~]S();
}
)cpp");
auto declaration = modifier_mask({SymbolModifiers::Declaration});
auto definition = modifier_mask({SymbolModifiers::Definition});
auto special_member = modifier_mask({SymbolModifiers::ConstructorOrDestructor});
EXPECT_TOKEN("ctor_decl", SymbolKind::Method, declaration | special_member);
EXPECT_TOKEN("dtor_decl", SymbolKind::Method, declaration | special_member);
EXPECT_TOKEN("ctor_def", SymbolKind::Method, definition | special_member);
EXPECT_TOKEN("dtor_ref", SymbolKind::Method, special_member);
}
TEST_CASE(LegacyVarDeclTemplates) {
run_utf8(R"cpp(
extern int @x1[x];
@@ -539,7 +583,53 @@ void f() {
EXPECT_TOKEN("v2", SymbolKind::Variable, definition);
}
}; // TEST_SUITE(SemanticTokens)
TEST_CASE(snapshot) {
ASSERT_SNAPSHOT_GLOB(corpus_dir, "**/*.cpp", [&](std::string_view path) -> std::string {
if(!compile_file(path))
return "COMPILE_ERROR";
auto content = unit->interested_content();
auto tokens = feature::semantic_tokens(*unit);
feature::PositionMapper mapper(content, feature::PositionEncoding::UTF8);
std::string result;
for(auto& token: tokens) {
if(!token.range.valid() || token.range.end <= token.range.begin ||
token.range.end > content.size())
continue;
auto pos = mapper.to_position(token.range.begin);
if(!pos)
continue;
auto text = content.substr(token.range.begin, token.range.length());
auto kind = kota::meta::enum_name(static_cast<SymbolKind::Kind>(token.kind), "Unknown");
result += std::format("- {{ loc: \"{}:{}\", text: {}, kind: {}",
pos->line,
pos->character,
yaml_str(text),
kind);
std::string mods;
for(std::uint32_t i = 0; i < 32; ++i) {
if(token.modifiers & (1u << i)) {
auto name = kota::meta::enum_name(static_cast<SymbolModifiers::Kind>(i));
if(!name.empty()) {
if(!mods.empty())
mods += ", ";
mods += name;
}
}
}
if(!mods.empty()) {
result += std::format(", modifiers: [{}]", mods);
}
result += " }\n";
}
return result;
});
}
}; // TEST_SUITE(semantic_tokens)
} // namespace

View File

@@ -128,7 +128,7 @@ TEST_CASE(SerializationRoundTrip) {
project.serialize(os);
// Deserialize.
auto restored = index::ProjectIndex::from(buf.data(), buf.size());
auto restored = index::ProjectIndex::from(buf.data());
// Path pools should match.
ASSERT_EQ(project.path_pool.paths.size(), restored.path_pool.paths.size());
@@ -190,7 +190,7 @@ TEST_CASE(NameSurvivesRoundTrip) {
llvm::SmallString<4096> buf;
llvm::raw_svector_ostream os(buf);
project.serialize(os);
auto restored = index::ProjectIndex::from(buf.data(), buf.size());
auto restored = index::ProjectIndex::from(buf.data());
// Verify names survive round-trip.
for(auto& [hash, symbol]: project.symbols) {

View File

@@ -1,13 +1,18 @@
#include <algorithm>
#include <format>
#include <set>
#include "test/test.h"
#include "test/tester.h"
#include "feature/feature.h"
#include "index/tu_index.h"
#include "kota/meta/enum.h"
namespace clice::testing {
namespace {
TEST_SUITE(TUIndex, Tester) {
TEST_SUITE(tu_index, Tester) {
index::TUIndex tu_index;
@@ -500,6 +505,64 @@ TEST_CASE(SymbolKinds) {
check_kind("ns", SymbolKind::Namespace);
}
}; // TEST_SUITE(TUIndex)
TEST_CASE(snapshot) {
ASSERT_SNAPSHOT_GLOB(corpus_dir, "**/*.cpp", [&](std::string_view path) -> std::string {
if(!compile_file(path))
return "COMPILE_ERROR";
auto idx = index::TUIndex::build(*unit);
auto content = unit->interested_content();
feature::PositionMapper mapper(content, feature::PositionEncoding::UTF8);
std::string result;
auto sorted = idx.main_file_index.occurrences;
std::ranges::sort(sorted, [](auto& lhs, auto& rhs) {
return std::tuple(lhs.range.begin, lhs.range.end, lhs.target) <
std::tuple(rhs.range.begin, rhs.range.end, rhs.target);
});
for(auto& occ: sorted) {
auto text = content.substr(occ.range.begin, occ.range.end - occ.range.begin);
auto pos = mapper.to_position(occ.range.begin);
if(!pos)
continue;
auto sym_it = idx.symbols.find(occ.target);
std::string_view kind_name = "?";
if(sym_it != idx.symbols.end()) {
kind_name =
kota::meta::enum_name(static_cast<SymbolKind::Kind>(sym_it->second.kind),
"Unknown");
}
result += std::format("- {{ loc: \"{}:{}\", kind: {}, text: {}",
pos->line,
pos->character,
kind_name,
yaml_str(text));
auto rel_it = idx.main_file_index.relations.find(occ.target);
if(rel_it != idx.main_file_index.relations.end()) {
std::string rels;
for(auto& rel: rel_it->second) {
if(rel.range != occ.range)
continue;
if(!rels.empty())
rels += ", ";
rels += kota::meta::enum_name(static_cast<RelationKind::Kind>(rel.kind), "?");
}
if(!rels.empty()) {
result += std::format(", relations: [{}]", rels);
}
}
result += " }\n";
}
return result;
});
}
}; // TEST_SUITE(tu_index)
} // namespace
} // namespace clice::testing

Some files were not shown because too many files have changed in this diff Show More