14 Commits

Author SHA1 Message Date
Myriad-Dreamin
04af0e51f8 test(agentic): remove lint integration test 2026-06-05 09:46:38 +08:00
Myriad-Dreamin
7a37222fd3 fix(prune): keep clang-tidy module libraries 2026-06-05 09:11:07 +08:00
Myriad-Dreamin
8e49d7b9e9 Force link clang-tidy check modules 2026-06-05 08:58:52 +08:00
Myriad-Dreamin
98e258d50c Format agentic lint test 2026-06-05 02:29:55 +08:00
Myriad-Dreamin
cb190d3d99 Test agentic clang-tidy lint diagnostics 2026-06-05 02:24:58 +08:00
Myriad-Dreamin
2baf947bff Run clang-tidy for agentic lint 2026-06-05 02:05:38 +08:00
Myriad-Dreamin
a17b3f6b7b Add agentic lint request 2026-06-05 01:26:54 +08:00
Myriad-Dreamin
3b45888622 fix(semantic-tokens): filter ineligible highlight references (#434)
Add a reusable declaration-name eligibility helper that mirrors clangd's
`canHighlightName`, use it to suppress unsupported reference tokens in
the semantic-token collector, and cover the change with focused
semantic-token regression tests plus a constructor/destructor positive
case.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved semantic token highlighting to suppress ineligible operator
references lacking meaningful source text.
* Ensured constructor and destructor names remain properly highlighted
with correct visual modifiers.

* **Tests**
* Added test coverage for semantic token highlighting behavior across
various declaration types.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/clice-io/clice/pull/434?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-05 00:03:18 +08:00
ykiko
cc5b25d5c3 refactor: public feature types and snapshot testing infrastructure (#442)
## Summary

- **Public feature types**: Move `SemanticToken`, `FoldingRange`,
`DocumentSymbol`, `InlayHint`, and `HintCategory` from internal `.cpp`
files to `feature.h` as public API types. Each feature now exposes two
overloads: a raw overload returning offset-based types and a protocol
overload that converts to LSP wire-format with explicit
`PositionEncoding`.
- **Snapshot testing**: Add corpus-driven snapshot tests using
`ASSERT_SNAPSHOT_GLOB` for semantic tokens, folding ranges, inlay hints,
document symbols, and TU index. Tests compile real C++ corpus files,
format output as YAML flow mappings, and diff against `.snap.yml`
baselines.
- **Test infrastructure**: Add `compile_file()` to `Tester`,
`yaml_str()` utility, `--corpus-dir` / `--snapshot-dir` CLI options, and
`--verbose` flag for unit tests. Migrate to kotatsu's unified
`kota::zest::Options` API.
- **Toolchain robustness**: Filter unknown cc1 args via
`clang::driver::getDriverOptTable()` to handle system compilers newer
than embedded LLVM.
- **Dependency bump**: Update kotatsu to 7381404 (unified zest Options,
out-param `from_json` API).

## Details

### Feature type changes
All five feature modules (`semantic_tokens`, `folding_ranges`,
`document_symbols`, `inlay_hints`, `document_links`) now follow the same
two-overload pattern. The raw overload returns offset-based structs
suitable for indexing and testing; the protocol overload adds
`PositionEncoding` conversion for LSP responses. `stateful_worker.cpp`
explicitly passes `PositionEncoding::UTF16` at every call site.

### Snapshot tests
Corpus files live in `tests/corpus/` (organized by language construct).
Snapshot baselines live in `tests/snapshots/<feature>/`. Format lambdas
are inlined directly in test bodies — no separate format functions for
single-use formatters. YAML output uses flow mappings (`- { key: value
}`) for compact, diffable baselines.

### cc1 arg filtering
`src/command/toolchain.cpp` now parses the cc1 argument list through
LLVM's driver option table and drops any args classified as
`UnknownClass`. This prevents compilation failures when the system
compiler emits flags that the embedded LLVM version doesn't recognize.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-24 19:36:27 +08:00
ykiko
3305465d1f feat(formatting): wire up textDocument/formatting and rangeFormatting (#441)
## Summary

- Wire up the existing `document_format` feature to LSP via stateless
workers
- Add `Format` kind to stateless worker dispatch, with a lightweight
`forward_format` path in `Compiler` (no compilation/deps needed — just
file path + content)
- Register `textDocument/formatting` and `textDocument/rangeFormatting`
handlers with `scoped_pause`
- Style lookup uses `clang::format::getStyle` which walks parent
directories for `.clang-format`, matching clangd's behavior

## Test plan

- [x] 4 unit tests: simple format, range format, idempotent (no edits),
include sort
- [x] 3 integration tests: full document format (verifies applied edits
match expected output), range format, already-formatted no-op
- [x] Capability assertions added to `test_capabilities`
- [x] All existing tests pass (554 unit, 170 integration, 2 smoke)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

* **New Features**
* Added document formatting and range formatting capabilities to the LSP
server
* Formatting can target the entire document or a specific range of code
  * Server now advertises formatting support to LSP clients

* **Tests**
  * Added comprehensive test coverage for formatting functionality

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-04 19:15:07 +08:00
ykiko
47ad905f5b feat(server): agentic query API, daemon mode, and relay mode (#438)
## Summary
- **Agentic query API**: 11 JSON-RPC handlers over TCP for AI agent
integration — compileCommand, projectFiles, fileDeps, impactAnalysis,
symbolSearch, readSymbol, documentSymbols, definition, references,
callGraph, typeHierarchy
- **Daemon mode**: MasterServer listens on a unix domain socket,
accepting multiple agent connections (`--mode daemon`)
- **Relay mode**: Bidirectional stdin/stdout ↔ unix socket proxy for
editor integration (`--mode relay`)
- **Full-body definition text**: readSymbol/definition return complete
function/class bodies via brace matching, not just the declaration line
- **24 integration tests** with concrete value assertions covering all
handlers

## Test plan
- [x] All 551 unit tests pass
- [x] All 2 smoke tests pass
- [x] All 148 integration tests pass (including 24 new agentic tests)
- [x] Raw JSON responses manually inspected for correctness (line
numbers, text content, field names, structural completeness)
- [x] Formatted with `pixi run format`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Daemon mode: background service with workspace watching and socket
support
  * Relay mode: stdio ↔ Unix-socket message relay
* Expanded agentic RPC/CLI: project files, deps/impact, symbol
search/read, document symbols, definitions, references, call graphs,
type hierarchy, status, shutdown; richer query options

* **Behavioral**
* Improved indexing lifecycle and status reporting; safer shutdown
coordination and client handling

* **Tests**
* New integration tests covering agentic RPC endpoints, CLI status,
shutdown, and error cases
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-04 00:15:37 +08:00
ykiko
75b9ea05b8 refactor(server): split into service layer, add agentic protocol, adopt task_group (#437)
## Summary

- **Restructure `src/server/` into subdirectories** (`service/`,
`compiler/`, `worker/`, `workspace/`, `protocol/`) to separate concerns:
transport/session management, compilation, worker orchestration, and
persistent workspace state.
- **Decouple MasterServer from transport**: MasterServer no longer holds
a `JsonPeer&` reference or registers handlers itself. New `LSPClient`
and `AgentClient` classes own their peer references and register
protocol handlers, accessing MasterServer internals via `friend class`.
- **Add agentic protocol**: A TCP-based side channel
(`agentic/compileCommand`) that lets external tools (AI agents, build
systems) query compile commands from a running clice server. Includes a
CLI client mode (`--mode agentic --port N --path FILE`), server-side
listener when `--port` is specified in pipe mode, and integration tests
for happy path, fallback, concurrency, and connection-refused.
- **Replace fire-and-forget `loop.schedule()` with `kota::task_group`**:
Compiler compile tasks, Indexer background indexing + resource monitor,
WorkerPool worker monitors, and socket accept loops now use structured
concurrency. This eliminates manual `alive_count_`/generation counters
and ensures all spawned tasks are joined on shutdown.
- **Fix flaky integration test**: `CliceClient.initialize()` now always
sets `cache_dir` to a workspace-local `.clice/` directory, preventing
stale PCH artifacts from the global `~/.cache/clice/` from polluting
test runs.

## Details

**Compiler peer lifetime**: `Compiler` and `Indexer` previously took
`JsonPeer&` in their constructors, coupling them to a single connection.
They now store a `JsonPeer*` set via `set_peer()`, with null checks
before sending diagnostics/progress. This supports the multi-connection
model where agentic clients don't need diagnostics.

**Socket mode single-LSP enforcement**: `accept_connections()` takes a
`register_lsp` flag; when true, only the first connection gets an
`LSPClient`. All connections get an `AgentClient`. This prevents
multiple LSP sessions from racing on shared server state.

**Structured shutdown**: `Compiler::stop()` cancels in-flight compile
tasks and joins them. `WorkerPool::stop()` signals workers and joins the
monitor task group. `Indexer` uses a `cancellation_source` to stop its
resource monitor when a background indexing run completes.

**Pin kotatsu**: Changed from `GIT_TAG main` + `GIT_SHALLOW TRUE` to an
exact commit hash for reproducible builds.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-02 01:06:18 +08:00
ykiko
939ab6d0d4 feat(server): concurrent background indexing with priority control (#432)
## Summary

- Rewrite serial background indexing to concurrent dispatch (up to
`stateless_worker_count / 2` parallel tasks)
- Add depth-counted pause/resume mechanism: completion and
signature-help handlers pause new index dispatches to prioritize user
requests
- Report indexing progress via LSP `$/progress` notifications
(percentage + file count)
- Lower thread scheduling priority (`nice +10`) for index tasks in
stateless workers via RAII `ScopedNice` guard

## Test plan

- [x] `pixi run format` — no changes
- [x] `pixi run unit-test Debug` — 551 passed, 9 skipped (pre-existing)
- [x] `pixi run smoke-test Debug` — 2/2 passed
- [x] `pixi run integration-test Debug` — 121 passed, 3 failed (all
pre-existing on main: header_context x2, staleness x1)
- [ ] Manual test: open a large project (e.g. LLVM), verify progress bar
appears and completion remains responsive during indexing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Pause/resume controls for background indexing
* Concurrent, adaptive background indexing with configurable concurrency
* LSP progress reporting (create/begin/report/end) and updated
completion metrics

* **Behavior Change**
* Code completion and signature help temporarily pause indexing for
responsiveness
* Background indexing runs with reduced scheduling priority on
non-Windows and logs "files dispatched" at finish

* **Tests**
* Test client fixture defaults init options and sets workspace cache dir
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-23 13:28:59 +08:00
ykiko
e1202d2fa5 fix: prevent worker crashes from null ASTConsumer, invalid FileID, and missing PCH cache dir (#435)
## Summary

Three pre-existing bugs cause worker processes to crash with SEGV or
SIGABRT. On the main branch these crashes are silent (workers die,
requests fail fast with "transport closed", tests still pass because
null responses are accepted). However when combined with #432's worker
respawn mechanism, the crash-respawn-crash cycle on low-core CI machines
causes request timeouts and smoke test hangs.

### Fixes

- **compilation.cpp**: `ProxyAction::CreateASTConsumer` now checks for
null before passing to `MultiplexConsumer`. When the wrapped action's
`CreateASTConsumer` fails (e.g. missing system headers during PCH
generation), this previously caused a null pointer dereference, SEGV,
ASAN kills the stateless worker.
- **compilation_unit.cpp**: `file_path()` returns empty `StringRef` on
invalid `FileID` instead of asserting. The assert fired when
`IncludeGraph::from()` called `file_path(interested_file())` on an AST
compiled with synthesized default commands (no compile_commands.json,
clang++ -std=c++20 fallback, no system headers, invalid main file ID),
SIGABRT, stateful worker crash.
- **compiler.cpp**: `ensure_pch` now creates the PCH cache directory
before sending the build request. Previously, when `load_workspace()`
exited early (no compile_commands.json), the cache subdirectories were
never created, causing every PCH write to fail with "No such file or
directory".
- **master_server.cpp/h**: `load_workspace()` changed from
`kota::task<>` to plain `void` -- it contains only synchronous
filesystem operations and no co_await, so the coroutine wrapper was
unnecessary. Called directly instead of via `loop.schedule()`.

## Test plan

- [x] Verified zero SEGV/SIGABRT/assertion crashes in worker stderr
after fix
- [x] rapid_edit.jsonl smoke test passes 3/3 runs consistently (34s
each)
- [x] Behavior matches main branch (both return 134 responses, 0
pending)
- [x] Debug build with ASAN (detect_leaks=0) -- clean run, no sanitizer
reports

<!-- codesmith:footer -->
---
<a
href="https://app.blacksmith.sh/clice-io/codesmith/clice/pr/435"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-in-codesmith-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-in-codesmith-light.svg"><img
alt="View in Codesmith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/view-in-codesmith-dark.svg"></picture></a>
<sup>Codesmith can help with this PR — just tag <code>@codesmith</code>
or enable autofix.</sup>

- [ ] Autofix CI and bot reviews
<!-- /codesmith:footer -->

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved error handling for AST consumer creation with null checks and
a clear failure path.
* Safer file-path access that returns empty for invalid identifiers
instead of asserting.
* PCH cache handling now validates cache configuration, attempts
directory creation, logs warnings, and aborts PCH builds on failure.

* **Refactor**
* Workspace loading changed from asynchronous to synchronous execution.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-23 10:36:03 +08:00
109 changed files with 5529 additions and 1373 deletions

View File

@@ -7,6 +7,11 @@ on:
description: "LLVM version to build (e.g., 21.1.8)"
required: true
type: string
skip_upload:
description: "Skip upload and PR creation (build-only mode)"
required: false
type: boolean
default: false
skip_pr:
description: "Skip PR creation (upload only, no PR)"
required: false
@@ -18,7 +23,7 @@ on:
branches: [main-turn-off]
jobs:
build-llvm:
build:
strategy:
fail-fast: false
matrix:
@@ -145,17 +150,117 @@ jobs:
--build-dir=build \
${EXTRA_ARGS}
# Upload raw install directory for validation (non-LTO only, one per platform/config).
# Artifact name matches the key used by test-cmake.yml: llvm-install-{target_triple|os}-{mode}
- name: Upload LLVM install for validation
if: ${{ matrix.lto == 'OFF' }}
- name: Build clice using installed LLVM
if: ${{ !matrix.target_triple }}
shell: bash
run: |
pixi run cmake-config ${{ matrix.llvm_mode }} ON -- \
"-DCLICE_ENABLE_LTO=${{ matrix.lto }}" \
"-DLLVM_INSTALL_PATH=.llvm/build-install"
pixi run cmake-build ${{ matrix.llvm_mode }}
- name: Build clice using installed LLVM (cross-compile)
if: ${{ matrix.target_triple }}
shell: bash
run: |
ENV="${{ matrix.pixi_env || 'package' }}"
pixi run -e "$ENV" cmake-config ${{ matrix.llvm_mode }} ON -- \
"-DCLICE_ENABLE_LTO=${{ matrix.lto }}" \
"-DCLICE_TARGET_TRIPLE=${{ matrix.target_triple }}" \
"-DLLVM_INSTALL_PATH=.llvm/build-install"
pixi run -e "$ENV" cmake-build ${{ matrix.llvm_mode }}
- name: Verify cross-compiled binary architecture
if: ${{ matrix.target_triple && runner.os != 'Windows' }}
shell: bash
run: |
BINARY="build/${{ matrix.llvm_mode }}/bin/clice"
echo "Binary info:"
file "$BINARY"
case "${{ matrix.target_triple }}" in
aarch64-linux-gnu) file "$BINARY" | grep -q "aarch64" ;;
x86_64-apple-darwin) file "$BINARY" | grep -q "x86_64" ;;
esac
- name: Upload cross-compiled clice for functional test
if: ${{ matrix.target_triple && matrix.lto == 'OFF' }}
uses: actions/upload-artifact@v4
with:
name: llvm-install-${{ matrix.target_triple || matrix.os }}-${{ matrix.llvm_mode }}
path: .llvm/build-install/
name: cross-clice-${{ matrix.target_triple }}-${{ matrix.llvm_mode }}
path: |
build/${{ matrix.llvm_mode }}/bin/
build/${{ matrix.llvm_mode }}/lib/
if-no-files-found: error
retention-days: 1
# Package as .tar.xz for the final clice-llvm release (all entries).
- name: Run tests
if: ${{ !matrix.target_triple }}
shell: bash
run: pixi run test ${{ matrix.llvm_mode }}
# Prune is only supported for native builds (requires linking clice to test).
# Cross-compiled targets reuse the native prune manifest of the same OS.
- name: Prune LLVM static libraries (Debug/RelWithDebInfo no LTO)
if: (!matrix.target_triple) && (matrix.llvm_mode == 'Debug' || (matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF'))
shell: bash
run: |
MANIFEST="pruned-libs-${{ matrix.os }}.json"
echo "LLVM_PRUNED_MANIFEST=${MANIFEST}" >> "${GITHUB_ENV}"
python3 scripts/prune-llvm-bin.py \
--action discover \
--install-dir ".llvm/build-install/lib" \
--build-dir "build/${{ matrix.llvm_mode }}" \
--max-attempts 60 \
--sleep-seconds 60 \
--manifest "${MANIFEST}"
- name: Upload pruned-libs manifest
if: (!matrix.target_triple) && matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF'
uses: actions/upload-artifact@v4
with:
name: llvm-pruned-libs-${{ matrix.os }}
path: ${{ env.LLVM_PRUNED_MANIFEST }}
if-no-files-found: error
compression-level: 0
- name: Apply pruned-libs manifest (RelWithDebInfo + LTO, native only)
if: (!matrix.target_triple) && matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'ON'
shell: bash
env:
GH_TOKEN: ${{ github.token }}
run: |
MANIFEST="pruned-libs-${{ matrix.os }}.json"
python3 scripts/prune-llvm-bin.py \
--action apply \
--manifest "${MANIFEST}" \
--install-dir ".llvm/build-install/lib" \
--build-dir "build/${{ matrix.llvm_mode }}" \
--gh-run-id "${{ github.run_id }}" \
--gh-artifact "llvm-pruned-libs-${{ matrix.os }}" \
--gh-download-dir "artifacts" \
--max-attempts 60 \
--sleep-seconds 60
# For cross-compiled LTO builds, apply the native prune manifest.
# The unused library set is arch-independent (same API surface).
- name: Apply pruned-libs manifest (cross-compile + LTO)
if: matrix.target_triple && matrix.lto == 'ON'
shell: bash
env:
GH_TOKEN: ${{ github.token }}
run: |
MANIFEST="pruned-libs-${{ matrix.os }}.json"
python3 scripts/prune-llvm-bin.py \
--action apply \
--manifest "${MANIFEST}" \
--install-dir ".llvm/build-install/lib" \
--build-dir "build/${{ matrix.llvm_mode }}" \
--gh-run-id "${{ github.run_id }}" \
--gh-artifact "llvm-pruned-libs-${{ matrix.os }}" \
--gh-download-dir "artifacts" \
--max-attempts 60 \
--sleep-seconds 60
- name: Package LLVM install directory
shell: bash
run: |
@@ -164,6 +269,7 @@ jobs:
MODE_TAG="debug"
fi
# Determine arch/platform/toolchain from target triple or runner OS
if [[ -n "${{ matrix.target_triple }}" ]]; then
case "${{ matrix.target_triple }}" in
x86_64-apple-darwin)
@@ -198,25 +304,56 @@ jobs:
ARCHIVE="${ARCH}-${PLATFORM}-${TOOLCHAIN}-${MODE_TAG}${SUFFIX}.tar.xz"
set -eo pipefail
tar -C .llvm/build-install -cf - . | xz -T0 -9 -c > "${ARCHIVE}"
tar -C .llvm -cf - build-install | xz -T0 -9 -c > "${ARCHIVE}"
echo "LLVM_INSTALL_ARCHIVE=${ARCHIVE}" >> "${GITHUB_ENV}"
- name: Upload LLVM release archive
- name: Upload LLVM install artifact
uses: actions/upload-artifact@v4
with:
name: ${{ env.LLVM_INSTALL_ARCHIVE }}
path: ${{ env.LLVM_INSTALL_ARCHIVE }}
if-no-files-found: error
validate:
needs: build-llvm
uses: ./.github/workflows/test-cmake.yml
with:
llvm_from_artifact: true
test-cross:
needs: build
strategy:
fail-fast: false
matrix:
include:
- os: macos-15-intel
llvm_mode: RelWithDebInfo
target_triple: x86_64-apple-darwin
- os: ubuntu-24.04-arm
llvm_mode: RelWithDebInfo
target_triple: aarch64-linux-gnu
- os: windows-11-arm
llvm_mode: RelWithDebInfo
target_triple: aarch64-pc-windows-msvc
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: test-run
- name: Download cross-compiled clice
uses: actions/download-artifact@v4
with:
name: cross-clice-${{ matrix.target_triple }}-${{ matrix.llvm_mode }}
path: build/${{ matrix.llvm_mode }}/
- name: Make binaries executable
if: runner.os != 'Windows'
run: chmod +x build/${{ matrix.llvm_mode }}/bin/*
- name: Run tests
run: pixi run -e test-run test ${{ matrix.llvm_mode }}
upload:
needs: [build-llvm, validate]
if: ${{ inputs.llvm_version }}
needs: build
if: ${{ !cancelled() && inputs.llvm_version && !inputs.skip_upload }}
runs-on: ubuntu-24.04
permissions:
contents: read

View File

@@ -7,6 +7,10 @@ on:
pull_request:
branches: [main]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
jobs:
changes:
if: ${{ !startsWith(github.ref, 'refs/tags/') }}

View File

@@ -2,11 +2,6 @@ name: cmake
on:
workflow_call:
inputs:
llvm_from_artifact:
description: "Download LLVM from workflow artifacts instead of release"
type: boolean
default: false
env:
CCACHE_DIR: ${{ github.workspace }}/.cache/ccache
@@ -53,22 +48,12 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
ref: ${{ inputs.llvm_from_artifact && github.ref_name || '' }}
- uses: ./.github/actions/setup-pixi
with:
environments: ${{ matrix.pixi_env || 'default' }}
- name: Download LLVM install
if: ${{ inputs.llvm_from_artifact }}
uses: actions/download-artifact@v4
with:
name: llvm-install-${{ matrix.target_triple || matrix.os }}-${{ matrix.build_type }}
path: .llvm-install/
- name: Restore compiler cache
if: ${{ !inputs.llvm_from_artifact }}
uses: actions/cache@v4
with:
path: ${{ runner.os == 'Windows' && '.cache/sccache' || '.cache/ccache' }}
@@ -77,7 +62,6 @@ jobs:
${{ runner.os }}-${{ matrix.build_type }}-${{ matrix.target_triple || 'native' }}-ccache-
- name: Zero cache stats
if: ${{ !inputs.llvm_from_artifact }}
run: |
ENV="${{ matrix.pixi_env || 'default' }}"
if [ "$RUNNER_OS" = "Windows" ]; then
@@ -90,28 +74,15 @@ jobs:
- name: Build (native)
if: ${{ !matrix.target_triple }}
shell: bash
run: |
if [ "${{ inputs.llvm_from_artifact }}" = "true" ]; then
pixi run cmake-config ${{ matrix.build_type }} ON -- \
"-DLLVM_INSTALL_PATH=.llvm-install"
pixi run cmake-build ${{ matrix.build_type }}
else
pixi run build ${{ matrix.build_type }} ON
fi
run: pixi run build ${{ matrix.build_type }} ON
- name: Build (cross-compile)
if: ${{ matrix.target_triple }}
shell: bash
run: |
ENV="${{ matrix.pixi_env || 'default' }}"
EXTRA_ARGS=""
if [ "${{ inputs.llvm_from_artifact }}" = "true" ]; then
EXTRA_ARGS="-DLLVM_INSTALL_PATH=.llvm-install"
fi
pixi run -e "$ENV" cmake-config ${{ matrix.build_type }} OFF -- \
"-DCLICE_TARGET_TRIPLE=${{ matrix.target_triple }}" \
$EXTRA_ARGS
"-DCLICE_TARGET_TRIPLE=${{ matrix.target_triple }}"
pixi run -e "$ENV" cmake-build ${{ matrix.build_type }}
- name: Upload cross-compiled binaries
@@ -125,12 +96,23 @@ jobs:
if-no-files-found: error
retention-days: 1
- name: Run tests
- name: Unit tests
if: ${{ !matrix.build_only }}
run: pixi run test ${{ matrix.build_type }}
timeout-minutes: 5
run: pixi run unit-test ${{ matrix.build_type }}
- name: Integration tests
if: ${{ !matrix.build_only }}
timeout-minutes: 20
run: pixi run integration-test ${{ matrix.build_type }}
- name: Smoke tests
if: ${{ !matrix.build_only }}
timeout-minutes: 15
run: pixi run smoke-test ${{ matrix.build_type }}
- name: Print cache stats and stop server
if: ${{ always() && !inputs.llvm_from_artifact }}
if: always()
run: |
ENV="${{ matrix.pixi_env || 'default' }}"
if [ "$RUNNER_OS" = "Windows" ]; then
@@ -175,5 +157,14 @@ jobs:
if: runner.os != 'Windows'
run: chmod +x build/${{ matrix.build_type }}/bin/*
- name: Run tests
run: pixi run -e test-run test ${{ matrix.build_type }}
- name: Unit tests
timeout-minutes: 5
run: pixi run -e test-run unit-test ${{ matrix.build_type }}
- name: Integration tests
timeout-minutes: 20
run: pixi run -e test-run integration-test ${{ matrix.build_type }}
- name: Smoke tests
timeout-minutes: 10
run: pixi run -e test-run smoke-test ${{ matrix.build_type }}

3
.gitignore vendored
View File

@@ -68,8 +68,7 @@ tests/unit/Local/
.pixi/*
!.pixi/config.toml
.codex/
.codex
.claude/*
!.claude/CLAUDE.md
!.claude/commands/
openspec/

View File

@@ -124,8 +124,31 @@ if(CLICE_CI_ENVIRONMENT)
target_compile_definitions(clice_options INTERFACE CLICE_CI_ENVIRONMENT=1)
endif()
set(CLICE_CLANG_TIDY_MODULE_LIBRARIES)
set(CLICE_MISSING_CLANG_TIDY_MODULES)
foreach(module IN LISTS CLICE_CLANG_TIDY_MODULE_COMPONENTS)
find_library(CLICE_${module}_LIBRARY
NAMES "${module}"
PATHS "${LLVM_INSTALL_PATH}/lib"
NO_DEFAULT_PATH
)
if(CLICE_${module}_LIBRARY)
list(APPEND CLICE_CLANG_TIDY_MODULE_LIBRARIES "${CLICE_${module}_LIBRARY}")
else()
list(APPEND CLICE_MISSING_CLANG_TIDY_MODULES "${module}")
endif()
endforeach()
if(CLICE_MISSING_CLANG_TIDY_MODULES)
message(STATUS "Clang-tidy module libraries not available: ${CLICE_MISSING_CLANG_TIDY_MODULES}")
else()
target_compile_definitions(clice_options INTERFACE CLICE_HAS_CLANG_TIDY_MODULES=1)
endif()
set(FBS_SCHEMA_FILE "${PROJECT_SOURCE_DIR}/src/index/schema.fbs")
set(GENERATED_HEADER "${PROJECT_BINARY_DIR}/generated/schema_generated.h")
set(CLANG_TIDY_CONFIG_SOURCE_FILE "${PROJECT_SOURCE_DIR}/config/clang-tidy-config.h")
set(CLANG_TIDY_CONFIG_GENERATED_FILE "${PROJECT_BINARY_DIR}/generated/clang-tidy-config.h")
if(CMAKE_CROSSCOMPILING)
find_program(FLATC_EXECUTABLE flatc REQUIRED)
@@ -143,10 +166,21 @@ add_custom_command(
add_custom_target(generate_flatbuffers_schema DEPENDS "${GENERATED_HEADER}")
add_custom_command(
OUTPUT "${CLANG_TIDY_CONFIG_GENERATED_FILE}"
COMMAND ${CMAKE_COMMAND} -E copy_if_different
"${CLANG_TIDY_CONFIG_SOURCE_FILE}"
"${CLANG_TIDY_CONFIG_GENERATED_FILE}"
DEPENDS "${CLANG_TIDY_CONFIG_SOURCE_FILE}"
COMMENT "Generating C++ header from ${CLANG_TIDY_CONFIG_SOURCE_FILE}"
)
add_custom_target(generate_clang_tidy_config DEPENDS "${CLANG_TIDY_CONFIG_GENERATED_FILE}")
file(GLOB_RECURSE CLICE_CORE_SOURCES CONFIGURE_DEPENDS "${PROJECT_SOURCE_DIR}/src/*.cpp")
add_library(clice-core STATIC ${CLICE_CORE_SOURCES})
add_library(clice::core ALIAS clice-core)
add_dependencies(clice-core generate_flatbuffers_schema)
add_dependencies(clice-core generate_flatbuffers_schema generate_clang_tidy_config)
target_include_directories(clice-core PUBLIC
"${PROJECT_SOURCE_DIR}/src"
@@ -162,6 +196,9 @@ target_link_libraries(clice-core PUBLIC
kota::codec::toml
simdjson::simdjson
)
if(CLICE_CLANG_TIDY_MODULE_LIBRARIES)
target_link_libraries(clice-core PUBLIC ${CLICE_CLANG_TIDY_MODULE_LIBRARIES})
endif()
add_executable(clice "${PROJECT_SOURCE_DIR}/src/clice.cc")
target_link_libraries(clice PRIVATE clice::core kota::deco)

View File

@@ -26,7 +26,7 @@
#include "support/path_pool.h"
#include "syntax/dependency_graph.h"
#include "kota/codec/json/serializer.h"
#include "kota/codec/json/json.h"
#include "kota/deco/deco.h"
#include "llvm/Support/FileSystem.h"

View File

@@ -1,5 +1,34 @@
include_guard()
set(CLICE_CLANG_TIDY_MODULE_COMPONENTS
# Keep this in sync with scripts/llvm-components.json and the old
# ALL_CLANG_TIDY_CHECKS list. MPIModule is intentionally excluded because
# clice disables static analyzer checks in ClangTidyForceLinker.h.
clangTidyAndroidModule
clangTidyAbseilModule
clangTidyAlteraModule
clangTidyBoostModule
clangTidyBugproneModule
clangTidyCERTModule
clangTidyConcurrencyModule
clangTidyCppCoreGuidelinesModule
clangTidyDarwinModule
clangTidyFuchsiaModule
clangTidyGoogleModule
clangTidyHICPPModule
clangTidyLinuxKernelModule
clangTidyLLVMModule
clangTidyLLVMLibcModule
clangTidyMiscModule
clangTidyModernizeModule
clangTidyObjCModule
clangTidyOpenMPModule
clangTidyPerformanceModule
clangTidyPortabilityModule
clangTidyReadabilityModule
clangTidyZirconModule
)
function(setup_llvm LLVM_VERSION)
find_package(Python3 COMPONENTS Interpreter REQUIRED)
@@ -81,36 +110,12 @@ function(setup_llvm LLVM_VERSION)
clangBasic
clangDriver
clangFormat
clangOptions
clangFrontend
clangLex
clangSema
clangSerialization
clangTidy
clangTidyUtils
clangTidyAndroidModule
clangTidyAbseilModule
clangTidyAlteraModule
clangTidyBoostModule
clangTidyBugproneModule
clangTidyCERTModule
clangTidyConcurrencyModule
clangTidyCppCoreGuidelinesModule
clangTidyDarwinModule
clangTidyFuchsiaModule
clangTidyGoogleModule
clangTidyHICPPModule
clangTidyLinuxKernelModule
clangTidyLLVMModule
clangTidyLLVMLibcModule
clangTidyMiscModule
clangTidyModernizeModule
clangTidyObjCModule
clangTidyOpenMPModule
clangTidyPerformanceModule
clangTidyPortabilityModule
clangTidyReadabilityModule
clangTidyZirconModule
clangTooling
clangToolingCore
clangToolingInclusions

View File

@@ -41,7 +41,7 @@ set(FLATBUFFERS_BUILD_FLATHASH OFF CACHE BOOL "" FORCE)
FetchContent_Declare(
kotatsu
GIT_REPOSITORY https://github.com/clice-io/kotatsu
GIT_TAG 048f23f0d786
GIT_TAG 73814044ce8142f4438a3028f44668675fc09fff
)
set(KOTA_ENABLE_ZEST ON)

View File

@@ -153,7 +153,7 @@ String values support `${workspace}` substitution.
## IPC Protocol
The master and workers communicate using custom RPC messages defined in `src/server/protocol.h`. Each message type has a `RequestTraits` or `NotificationTraits` specialization that defines the method name and result type.
The master and workers communicate using custom RPC messages defined in `src/server/protocol/`. Each message type has a `RequestTraits` or `NotificationTraits` specialization that defines the method name and result type.
### Stateful Worker Messages

23
pixi.lock generated
View File

@@ -1078,6 +1078,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
linux-aarch64:
- conda: https://conda.anaconda.org/conda-forge/linux-aarch64/_openmp_mutex-4.5-20_gnu.conda
@@ -1152,6 +1153,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-64:
- conda: https://conda.anaconda.org/conda-forge/osx-64/bzip2-1.0.8-h500dc9f_9.conda
@@ -1224,6 +1226,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-arm64:
- conda: https://conda.anaconda.org/conda-forge/osx-arm64/bzip2-1.0.8-hd037594_8.conda
@@ -1289,6 +1292,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
win-64:
- conda: https://conda.anaconda.org/conda-forge/win-64/bzip2-1.0.8-h0ad9c76_8.conda
@@ -1343,6 +1347,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
format:
channels:
@@ -1704,6 +1709,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
linux-aarch64:
- conda: https://conda.anaconda.org/conda-forge/linux-aarch64/_openmp_mutex-4.5-20_gnu.conda
@@ -1782,6 +1788,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-64:
- conda: https://conda.anaconda.org/conda-forge/osx-64/bzip2-1.0.8-h500dc9f_9.conda
@@ -1858,6 +1865,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-arm64:
- conda: https://conda.anaconda.org/conda-forge/osx-arm64/bzip2-1.0.8-hd037594_8.conda
@@ -1926,6 +1934,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
win-64:
- conda: https://conda.anaconda.org/conda-forge/win-64/bzip2-1.0.8-h0ad9c76_8.conda
@@ -1982,6 +1991,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
test-run:
channels:
@@ -2025,6 +2035,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
linux-aarch64:
- conda: https://conda.anaconda.org/conda-forge/linux-aarch64/_openmp_mutex-4.5-20_gnu.conda
@@ -2058,6 +2069,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-64:
- conda: https://conda.anaconda.org/conda-forge/osx-64/bzip2-1.0.8-h500dc9f_9.conda
@@ -2113,6 +2125,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-arm64:
- conda: https://conda.anaconda.org/conda-forge/osx-arm64/bzip2-1.0.8-hd037594_9.conda
@@ -2168,6 +2181,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
win-64:
- conda: https://conda.anaconda.org/conda-forge/win-64/bzip2-1.0.8-h0ad9c76_9.conda
@@ -2199,6 +2213,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
win-arm64:
- conda: https://conda.anaconda.org/conda-forge/win-arm64/bzip2-1.0.8-h50b96f5_9.conda
@@ -2229,6 +2244,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
packages:
- conda: https://conda.anaconda.org/conda-forge/linux-64/_libgcc_mutex-0.1-conda_forge.tar.bz2
@@ -7795,6 +7811,13 @@ packages:
- coverage>=6.2 ; extra == 'testing'
- hypothesis>=5.7.1 ; extra == 'testing'
requires_python: '>=3.10'
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
name: pytest-timeout
version: 2.4.0
sha256: c42667e5cdadb151aeb5b26d114aff6bdf5a907f176a007a30b940d3d865b5c2
requires_dist:
- pytest>=7.0.0
requires_python: '>=3.7'
- conda: https://conda.anaconda.org/conda-forge/linux-64/python-3.14.2-h32b2ec7_100_cp314.conda
build_number: 100
sha256: a120fb2da4e4d51dd32918c149b04a08815fd2bd52099dad1334647984bb07f1

View File

@@ -102,6 +102,7 @@ lld = "==20.1.8"
[feature.test.pypi-dependencies]
pytest = "*"
pytest-asyncio = ">=1.1.0"
pytest-timeout = "*"
pygls = ">=2.0.0"
lsprotocol = ">=2024.0.0"
@@ -160,13 +161,13 @@ depends-on = [{ task = "lint-cpp", args = ["{{ type }}"] }]
[feature.test.tasks.unit-test]
args = [{ arg = "type", default = "RelWithDebInfo" }]
cmd = './build/{{ type }}/bin/unit_tests --test-dir="./tests/data"'
cmd = './build/{{ type }}/bin/unit_tests --test-dir="./tests/data" --snapshot-dir="./tests/snapshots" --corpus-dir="./tests/corpus" --verbose'
[feature.test.tasks.integration-test]
args = [{ arg = "type", default = "RelWithDebInfo" }]
cmd = """
pytest -s --log-cli-level=INFO tests/integration \
--executable=./build/{{ type }}/bin/clice
pytest -s --log-cli-level=INFO --timeout=300 --timeout-method=thread \
tests/integration --executable=./build/{{ type }}/bin/clice
"""
[feature.test.tasks.smoke-test]
@@ -256,7 +257,7 @@ format-markdown = "fd -H -e md -x prettier --write"
format-json = "fd -H -e json -E package-lock.json -x prettier --write"
format-toml = "fd -H -e toml -x tombi format"
format-yaml = """
fd -H -e yaml -e yml -E pnpm-lock.yaml -x prettier --write && \
fd -H -e yaml -e yml -E pnpm-lock.yaml -E '*.snap.yml' -x prettier --write && \
fd -H "^\\.clang-(format|tidy)$" -x prettier --write --parser yaml
"""
format = { depends-on = [

View File

@@ -10,7 +10,7 @@ fi
WORKFLOW_ID="$1"
mkdir -p artifacts
gh run download "${WORKFLOW_ID}" --dir artifacts --pattern "*.tar.xz"
gh run download "${WORKFLOW_ID}" --dir artifacts
echo "Downloaded artifacts:"
find artifacts -maxdepth 4 -type f -print

View File

@@ -35,19 +35,16 @@
"LLVMFrontendOffloading",
"LLVMFrontendAtomic",
"LLVMFrontendDirective",
"LLVMPlugins",
"LLVMWindowsDriver",
"clangIndex",
"clangAPINotes",
"clangAST",
"clangASTMatchers",
"clangBasic",
"clangDependencyScanning",
"clangDriver",
"clangFormat",
"clangFrontend",
"clangLex",
"clangOptions",
"clangParse",
"clangSema",
"clangSerialization",
@@ -90,7 +87,6 @@
"clangToolingRefactoring",
"clangTransformer",
"clangCrossTU",
"clangAnalysisLifetimeSafety",
"clangAnalysisFlowSensitive",
"clangAnalysisFlowSensitiveModels",
"clangStaticAnalyzerCheckers",

View File

@@ -16,7 +16,10 @@ import subprocess
import time
from datetime import datetime, timezone
from pathlib import Path
from typing import Iterable, List, Optional
from typing import Iterable, List, Optional, Set
LLVM_COMPONENTS_FILE = Path(__file__).with_name("llvm-components.json")
def parse_args() -> argparse.Namespace:
@@ -102,12 +105,33 @@ def run_build(build_dir: Path) -> bool:
return False
def protected_library_names() -> Set[str]:
data = json.loads(LLVM_COMPONENTS_FILE.read_text())
components = data.get("components", [])
if not isinstance(components, list):
raise ValueError(f"{LLVM_COMPONENTS_FILE} missing 'components' list")
names: Set[str] = set()
for component in components:
if not isinstance(component, str):
continue
if not (component.startswith("clangTidy") and component.endswith("Module")):
continue
names.add(f"lib{component}.a")
names.add(f"{component}.lib")
return names
def candidate_files(install_dir: Path) -> Iterable[Path]:
if not install_dir.is_dir():
raise FileNotFoundError(f"lib dir not found: {install_dir}")
protected = protected_library_names()
for path in sorted(install_dir.iterdir()):
if not path.is_file():
continue
if path.name in protected:
print(f"Keeping protected clang-tidy module library: {path.name}")
continue
if path.suffix.lower() in {".a", ".lib"}:
yield path
else:
@@ -156,7 +180,11 @@ def apply_manifest(manifest: Path, install_dir: Path) -> None:
removed = data.get("removed", [])
if not isinstance(removed, list):
raise ValueError("Manifest missing 'removed' list")
protected = protected_library_names()
for name in removed:
if name in protected:
print(f"Keeping protected clang-tidy module library from manifest: {name}")
continue
target = install_dir / name
if target.exists():
print(f"Deleting {target}")

View File

@@ -4,33 +4,33 @@
#include <print>
#include <string>
#include "server/master_server.h"
#include "server/stateful_worker.h"
#include "server/stateless_worker.h"
#include "server/service/agentic.h"
#include "server/service/master_server.h"
#include "server/worker/stateful_worker.h"
#include "server/worker/stateless_worker.h"
#include "support/logging.h"
#include "kota/async/async.h"
#include "kota/deco/deco.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/peer.h"
#include "kota/ipc/recording_transport.h"
#include "kota/ipc/transport.h"
namespace clice {
using kota::deco::decl::KVStyle;
struct Options {
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Running mode: pipe, socket, stateless-worker, stateful-worker",
required = false)
DecoKV(
style = KVStyle::JoinedOrSeparate,
help =
"Running mode: pipe, socket, daemon, relay, agentic, stateless-worker, stateful-worker",
required = false)
<std::string> mode;
DecoKV(style = KVStyle::JoinedOrSeparate, help = "Socket mode address", required = false)
<std::string> host = "127.0.0.1";
DecoKV(style = KVStyle::JoinedOrSeparate, help = "Socket mode port", required = false)
<int> port = 50051;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Agentic TCP port (0 = disabled)",
required = false)
<int> port = 0;
DecoKV(style = KVStyle::JoinedOrSeparate,
names = {"--log-level", "--log-level="},
@@ -43,6 +43,50 @@ struct Options {
required = false)
<std::string> record;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "File path for agentic queries",
required = false)
<std::string> path;
DecoKV(
style = KVStyle::JoinedOrSeparate,
help =
"Agentic method (compileCommand, symbolSearch, definition, references, "
"documentSymbols, readSymbol, callGraph, typeHierarchy, projectFiles, "
"lint, fileDeps, impactAnalysis, status, shutdown)",
required = false)
<std::string> method;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Symbol name for agentic queries",
required = false)
<std::string> name;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Search query for symbolSearch",
required = false)
<std::string> query;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Line number for position-based lookup",
required = false)
<int> line;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Direction: callers/callees or supertypes/subtypes",
required = false)
<std::string> direction;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Unix domain socket path for daemon mode",
required = false)
<std::string> socket;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Workspace root directory for daemon mode",
required = false)
<std::string> workspace;
// Internal options (passed from master to worker processes)
DecoKV(style = KVStyle::JoinedOrSeparate,
names = {"--worker-memory-limit", "--worker-memory-limit="},
@@ -68,9 +112,6 @@ struct Options {
int main(int argc, const char** argv) {
#ifndef _WIN32
// On POSIX systems, ignore SIGPIPE so that writing to a closed pipe
// (e.g. when the LSP client disconnects) returns EPIPE instead of
// killing the process. This is standard practice for pipe-based servers.
signal(SIGPIPE, SIG_IGN);
#endif
@@ -110,8 +151,6 @@ int main(int argc, const char** argv) {
return 1;
}
std::string self_path = argv[0];
auto& mode = *opts.mode;
auto worker_name = opts.worker_name.value_or("");
@@ -131,77 +170,51 @@ int main(int argc, const char** argv) {
log_dir);
}
if(mode == "pipe") {
clice::logging::stderr_logger("master", clice::logging::options);
kota::event_loop loop;
auto transport = kota::ipc::StreamTransport::open_stdio(loop);
if(!transport) {
LOG_ERROR("failed to open stdio transport");
return 1;
}
std::unique_ptr<kota::ipc::Transport> final_transport = std::move(*transport);
if(opts.record.has_value()) {
final_transport =
std::make_unique<kota::ipc::RecordingTransport>(std::move(final_transport),
*opts.record);
}
kota::ipc::JsonPeer peer(loop, std::move(final_transport));
clice::MasterServer server(loop, peer, std::move(self_path));
server.register_handlers();
loop.schedule(peer.run());
loop.run();
return 0;
if(mode == "pipe" || mode == "socket") {
clice::ServerOptions server_opts;
server_opts.mode = mode;
server_opts.host = opts.host.value_or("127.0.0.1");
server_opts.port = opts.port.value_or(0);
server_opts.self_path = argv[0];
server_opts.record = opts.record.value_or("");
return clice::run_server_mode(server_opts);
}
if(mode == "socket") {
clice::logging::stderr_logger("master", clice::logging::options);
kota::event_loop loop;
auto host = opts.host.value_or("127.0.0.1");
auto port = opts.port.value_or(50051);
auto acceptor = kota::tcp::listen(host, port, {}, loop);
if(!acceptor) {
LOG_ERROR("failed to listen on {}:{}", host, port);
if(mode == "daemon") {
auto workspace = opts.workspace.value_or("");
if(workspace.empty()) {
LOG_ERROR("--workspace is required for daemon mode");
return 1;
}
LOG_INFO("Listening on {}:{} ...", host, port);
clice::DaemonOptions daemon_opts;
daemon_opts.socket_path = opts.socket.value_or("");
daemon_opts.workspace = std::move(workspace);
daemon_opts.self_path = argv[0];
return clice::run_daemon_mode(daemon_opts);
}
auto task = [&]() -> kota::task<> {
auto client = co_await acceptor->accept();
if(!client.has_value()) {
LOG_ERROR("failed to accept connection");
loop.stop();
co_return;
}
if(mode == "agentic") {
auto port = opts.port.value_or(0);
if(port <= 0) {
LOG_ERROR("--port is required for agentic mode");
return 1;
}
clice::AgenticQueryOptions aq;
aq.host = opts.host.value_or("127.0.0.1");
aq.port = port;
aq.method = opts.method.value_or("compileCommand");
aq.path = opts.path.value_or("");
aq.name = opts.name.value_or("");
aq.query = opts.query.value_or("");
aq.line = opts.line.value_or(0);
aq.direction = opts.direction.value_or("");
return clice::run_agentic_mode(aq);
}
LOG_INFO("Client connected");
std::unique_ptr<kota::ipc::Transport> transport =
std::make_unique<kota::ipc::StreamTransport>(std::move(client.value()));
if(opts.record.has_value()) {
transport = std::make_unique<kota::ipc::RecordingTransport>(std::move(transport),
*opts.record);
}
kota::ipc::JsonPeer peer(loop, std::move(transport));
clice::MasterServer server(loop, peer, std::string(self_path));
server.register_handlers();
co_await peer.run();
peer.close();
loop.stop();
};
loop.schedule(task());
loop.run();
return 0;
if(mode == "relay") {
auto socket = opts.socket.value_or("");
return clice::run_relay_mode(socket);
}
LOG_ERROR("unknown mode '{}'", mode);

View File

@@ -7,9 +7,9 @@
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/raw_ostream.h"
#include "clang/Driver/Driver.h"
#include "clang/Driver/Options.h"
#include "clang/Driver/Types.h"
#include "clang/Options/OptionUtils.h"
#include "clang/Options/Options.h"
namespace clice {
@@ -35,7 +35,7 @@ struct Thief {
template struct Thief<&opt::OptTable::DashDashParsing, &opt::OptTable::GroupedShortOptions>;
auto& option_table = clang::getDriverOptTable();
auto& option_table = driver::getDriverOptTable();
} // namespace
@@ -45,7 +45,7 @@ std::unique_ptr<llvm::opt::Arg> ArgumentParser::parse_one(unsigned& index) {
return option_table.ParseOneArg(*this, index, opt::Visibility(visibility_mask));
}
using ID = clang::options::ID;
using ID = clang::driver::options::ID;
bool is_discarded_option(unsigned id) {
switch(id) {
@@ -165,7 +165,7 @@ llvm::StringRef resource_dir() {
if(exe.empty()) {
return std::string{};
}
return clang::GetResourcesPath(exe);
return clang::driver::Driver::GetResourcesPath(exe);
}();
return dir;
}
@@ -246,7 +246,7 @@ std::string print_argv(llvm::ArrayRef<const char*> args) {
}
unsigned default_visibility(llvm::StringRef driver) {
namespace options = clang::options;
namespace options = clang::driver::options;
auto name = llvm::sys::path::filename(driver);
name.consume_back(".exe");

View File

@@ -6,11 +6,11 @@
#include "llvm/ADT/StringSet.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Path.h"
#include "clang/Options/Options.h"
#include "clang/Driver/Options.h"
namespace clice {
using ID = clang::options::ID;
using ID = clang::driver::options::ID;
SearchConfig extract_search_config(llvm::ArrayRef<const char*> arguments,
llvm::StringRef directory) {
@@ -24,12 +24,9 @@ SearchConfig extract_search_config(llvm::ArrayRef<const char*> arguments,
std::vector<SearchDir> after;
auto make_absolute = [&](llvm::StringRef path) -> std::string {
llvm::SmallString<256> abs_path;
if(llvm::sys::path::is_absolute(path)) {
abs_path = path;
} else {
abs_path = directory;
llvm::sys::path::append(abs_path, path);
llvm::SmallString<256> abs_path(path);
if(!llvm::sys::path::is_absolute(abs_path)) {
llvm::sys::fs::make_absolute(directory, abs_path);
}
llvm::sys::path::remove_dots(abs_path, true);
return abs_path.str().str();

View File

@@ -9,6 +9,7 @@
#include "support/logging.h"
#include "kota/meta/enum.h"
#include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/ScopeExit.h"
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/FileSystem.h"
@@ -17,6 +18,7 @@
#include "llvm/TargetParser/Host.h"
#include "clang/Driver/Compilation.h"
#include "clang/Driver/Driver.h"
#include "clang/Driver/Options.h"
#include "clang/Driver/Tool.h"
#ifndef _WIN32
@@ -470,11 +472,32 @@ std::vector<const char*> query_clang_toolchain(const QueryParams& params) {
continue;
}
for(auto arg: args) {
if(arg == "-###"sv) {
// FIXME: the system compiler may be newer than our embedded LLVM,
// producing cc1 flags we don't recognize. Filter them out here.
// Long-term we should unify the command pipeline so the driver
// version always matches the embedded LLVM.
auto& table = clang::driver::getDriverOptTable();
auto cc1_args = llvm::ArrayRef(args).drop_front(2);
unsigned missing_index = 0, missing_count = 0;
auto parsed = table.ParseArgs(cc1_args, missing_index, missing_count);
llvm::DenseSet<unsigned> unknown_indices;
for(auto* a: parsed) {
if(a->getOption().getKind() == llvm::opt::Option::UnknownClass) {
unknown_indices.insert(a->getIndex());
}
}
result.emplace_back(params.callback(args[0]));
result.emplace_back(params.callback(args[1]));
for(unsigned i = 0; i < cc1_args.size(); ++i) {
if(unknown_indices.contains(i)) {
continue;
}
result.emplace_back(params.callback(arg));
if(cc1_args[i] == "-###"sv) {
continue;
}
result.emplace_back(params.callback(cc1_args[i]));
}
}
}

View File

@@ -7,7 +7,6 @@
#include "support/logging.h"
#include "llvm/Support/Error.h"
#include "clang/Driver/CreateInvocationFromArgs.h"
#include "clang/Frontend/MultiplexConsumer.h"
#include "clang/Frontend/TextDiagnosticPrinter.h"
#include "clang/Lex/PreprocessorOptions.h"
@@ -220,9 +219,10 @@ public:
auto CreateASTConsumer(clang::CompilerInstance& instance, llvm::StringRef file)
-> std::unique_ptr<clang::ASTConsumer> final {
return std::make_unique<ProxyASTConsumer>(
WrapperFrontendAction::CreateASTConsumer(instance, file),
unit);
auto consumer = WrapperFrontendAction::CreateASTConsumer(instance, file);
if(!consumer)
return nullptr;
return std::make_unique<ProxyASTConsumer>(std::move(consumer), unit);
}
/// Make this public.
@@ -247,14 +247,13 @@ CompilationStatus CompilationUnitRef::Self::run_clang(
self.instance = std::make_unique<clang::CompilerInstance>(std::move(invocation));
auto& instance = *self.instance;
instance.createDiagnostics(diagnostic_consumer.release(), true);
instance.createDiagnostics(*params.vfs, diagnostic_consumer.release(), true);
if(auto remapping = clang::createVFSFromCompilerInvocation(instance.getInvocation(),
instance.getDiagnostics(),
params.vfs)) {
instance.setVirtualFileSystem(std::move(remapping));
instance.createFileManager(std::move(remapping));
}
instance.createFileManager();
if(!instance.createTarget()) {
return CompilationStatus::SetupFail;
@@ -419,6 +418,8 @@ CompilationUnit compile(CompilationParams& params, PCMInfo& out) {
}
CompilationUnit complete(CompilationParams& params, clang::CodeCompleteConsumer* consumer) {
params.kind = CompilationKind::Completion;
auto& [file, offset] = params.completion;
/// The location of clang is 1-1 based.

View File

@@ -65,7 +65,7 @@ struct PCMInfo : ModuleInfo {
struct CompilationParams {
/// The kind of this compilation.
CompilationKind kind;
CompilationKind kind = CompilationKind::Content;
/// Whether to run clang-tidy.
bool clang_tidy = false;

View File

@@ -81,7 +81,8 @@ auto CompilationUnitRef::file_offset(clang::SourceLocation location) -> std::uin
}
auto CompilationUnitRef::file_path(clang::FileID fid) -> llvm::StringRef {
assert(fid.isValid() && "Invalid fid");
if(!fid.isValid())
return {};
if(auto it = self->path_cache.find(fid); it != self->path_cache.end()) {
return it->second;
}

View File

@@ -16,7 +16,17 @@ namespace clice {
llvm::StringRef DiagnosticID::diagnostic_code() const {
switch(value) {
#define DIAG(ENUM, ...) \
#define DIAG(ENUM, \
CLASS, \
DEFAULT_MAPPING, \
DESC, \
GROPU, \
SFINAE, \
NOWERROR, \
SHOWINSYSHEADER, \
SHOWINSYSMACRO, \
DEFERRABLE, \
CATEGORY) \
case clang::diag::ENUM: return #ENUM;
#include "clang/Basic/DiagnosticASTKinds.inc"
#include "clang/Basic/DiagnosticAnalysisKinds.inc"

View File

@@ -6,7 +6,7 @@
#include "clang/Frontend/CompilerInstance.h"
#include "clang/Frontend/FrontendActions.h"
#include "clang-tidy/ClangTidyCheck.h"
#include "clang-tidy/ClangTidyModule.h"
#include "clang-tidy/ClangTidyModuleRegistry.h"
#include "clang-tidy/ClangTidyOptions.h"
namespace clice::tidy {

View File

@@ -10,8 +10,12 @@
#include "clang/Frontend/CompilerInstance.h"
#include "clang-tidy/ClangTidyCheck.h"
#include "clang-tidy/ClangTidyDiagnosticConsumer.h"
#include "clang-tidy/ClangTidyModule.h"
#include "clang-tidy/ClangTidyModuleRegistry.h"
#include "clang-tidy/ClangTidyOptions.h"
#ifdef CLICE_HAS_CLANG_TIDY_MODULES
#define CLANG_TIDY_DISABLE_STATIC_ANALYZER_CHECKS
#include "clang-tidy/ClangTidyForceLinker.h"
#endif
namespace clice::tidy {
@@ -92,15 +96,11 @@ tidy::ClangTidyOptions create_options() {
// include-cleaner is directly integrated in IncludeCleaner.cpp
"-misc-include-cleaner",
// ----- False Positives -----
// Check relies on seeing ifndef/define/endif directives,
// clangd doesn't replay those when using a preamble.
"-llvm-header-guard",
"-modernize-macro-to-enum",
// ----- Crashing Checks -----
// Check can choke on invalid (intermediate) c++
// code, which is often the case when clangd
// tries to build an AST.

View File

@@ -93,18 +93,9 @@ auto symbol_detail(clang::ASTContext& context, const clang::NamedDecl& decl) ->
return detail;
}
struct InternalSymbol {
std::string name;
std::string detail;
SymbolKind kind = SymbolKind::Invalid;
LocalSourceRange range;
LocalSourceRange selection_range;
std::vector<InternalSymbol> children;
};
struct SymbolFrame {
std::vector<InternalSymbol> symbols;
std::vector<InternalSymbol>* cursor = &symbols;
std::vector<DocumentSymbol> symbols;
std::vector<DocumentSymbol>* cursor = &symbols;
};
class DocumentSymbolCollector : public FilteredASTVisitor<DocumentSymbolCollector> {
@@ -143,7 +134,7 @@ public:
return ok;
}
auto collect() -> std::vector<InternalSymbol> {
auto collect() -> std::vector<DocumentSymbol> {
TraverseDecl(unit.tu());
return std::move(result.symbols);
}
@@ -174,8 +165,8 @@ private:
SymbolFrame result;
};
void sort_symbols(std::vector<InternalSymbol>& symbols) {
std::ranges::sort(symbols, [](const InternalSymbol& lhs, const InternalSymbol& rhs) {
void sort_symbols(std::vector<DocumentSymbol>& symbols) {
std::ranges::sort(symbols, [](const DocumentSymbol& lhs, const DocumentSymbol& rhs) {
if(lhs.range.begin != rhs.range.begin) {
return lhs.range.begin < rhs.range.begin;
}
@@ -187,7 +178,7 @@ void sort_symbols(std::vector<InternalSymbol>& symbols) {
}
}
auto to_protocol_symbol(const InternalSymbol& symbol, const PositionMapper& converter)
auto to_protocol_symbol(const DocumentSymbol& symbol, const PositionMapper& converter)
-> protocol::DocumentSymbol {
protocol::DocumentSymbol result{
.name = symbol.name,
@@ -215,10 +206,15 @@ auto to_protocol_symbol(const InternalSymbol& symbol, const PositionMapper& conv
} // namespace
auto document_symbols(CompilationUnitRef unit) -> std::vector<DocumentSymbol> {
auto result = DocumentSymbolCollector(unit).collect();
sort_symbols(result);
return result;
}
auto document_symbols(CompilationUnitRef unit, PositionEncoding encoding)
-> std::vector<protocol::DocumentSymbol> {
auto internal = DocumentSymbolCollector(unit).collect();
sort_symbols(internal);
auto internal = document_symbols(unit);
PositionMapper converter(unit.interested_content(), encoding);
std::vector<protocol::DocumentSymbol> symbols;

View File

@@ -7,6 +7,7 @@
#include "compile/compilation.h"
#include "compile/compilation_unit.h"
#include "semantic/symbol_kind.h"
#include "kota/ipc/lsp/position.h"
#include "kota/ipc/lsp/protocol.h"
@@ -59,18 +60,66 @@ struct InlayHintsOptions {
struct SignatureHelpOptions {};
auto semantic_tokens(CompilationUnitRef unit, PositionEncoding encoding = PositionEncoding::UTF16)
struct SemanticToken {
LocalSourceRange range;
SymbolKind kind = SymbolKind::Invalid;
std::uint32_t modifiers = 0;
};
struct FoldingRange {
LocalSourceRange range;
std::optional<protocol::FoldingRangeKind> kind;
std::string collapsed_text;
};
struct DocumentSymbol {
std::string name;
std::string detail;
SymbolKind kind = SymbolKind::Invalid;
LocalSourceRange range;
LocalSourceRange selection_range;
std::vector<DocumentSymbol> children;
};
enum class HintCategory : std::uint8_t {
Parameter,
DefaultArgument,
Type,
Designator,
BlockEnd,
};
struct InlayHint {
std::uint32_t offset = 0;
HintCategory kind = HintCategory::Type;
std::string label;
bool padding_left = false;
bool padding_right = false;
};
auto semantic_tokens(CompilationUnitRef unit) -> std::vector<SemanticToken>;
auto semantic_tokens(CompilationUnitRef unit, PositionEncoding encoding)
-> protocol::SemanticTokens;
auto folding_ranges(CompilationUnitRef unit) -> std::vector<FoldingRange>;
auto folding_ranges(CompilationUnitRef unit, PositionEncoding encoding)
-> std::vector<protocol::FoldingRange>;
auto document_symbols(CompilationUnitRef unit) -> std::vector<DocumentSymbol>;
auto document_symbols(CompilationUnitRef unit, PositionEncoding encoding)
-> std::vector<protocol::DocumentSymbol>;
auto inlay_hints(CompilationUnitRef unit,
LocalSourceRange target,
const InlayHintsOptions& options = {}) -> std::vector<InlayHint>;
auto inlay_hints(CompilationUnitRef unit,
LocalSourceRange target,
const InlayHintsOptions& options,
PositionEncoding encoding) -> std::vector<protocol::InlayHint>;
auto document_links(CompilationUnitRef unit, PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::DocumentLink>;
auto document_symbols(CompilationUnitRef unit, PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::DocumentSymbol>;
auto folding_ranges(CompilationUnitRef unit, PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::FoldingRange>;
auto diagnostics(CompilationUnitRef unit, PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::Diagnostic>;
@@ -89,12 +138,6 @@ auto hover(CompilationUnitRef unit,
const HoverOptions& options = {},
PositionEncoding encoding = PositionEncoding::UTF16) -> std::optional<protocol::Hover>;
auto inlay_hints(CompilationUnitRef unit,
LocalSourceRange target,
const InlayHintsOptions& options = {},
PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::InlayHint>;
auto signature_help(CompilationParams& params, const SignatureHelpOptions& options = {})
-> protocol::SignatureHelp;

View File

@@ -53,12 +53,6 @@ auto to_kind(FoldingKind kind) -> protocol::FoldingRangeKind {
return protocol::FoldingRangeKind(protocol::FoldingRangeKind::region);
}
struct RawFoldingRange {
LocalSourceRange range;
std::optional<protocol::FoldingRangeKind> kind;
std::string collapsed_text;
};
class FoldingRangeCollector : public FilteredASTVisitor<FoldingRangeCollector> {
public:
explicit FoldingRangeCollector(CompilationUnitRef unit) : FilteredASTVisitor(unit, true) {}
@@ -185,7 +179,7 @@ public:
return true;
}
auto collect() -> std::vector<RawFoldingRange> {
auto collect() -> std::vector<FoldingRange> {
TraverseDecl(unit.tu());
auto directives_it = unit.directives().find(unit.interested_file());
@@ -193,7 +187,7 @@ public:
collect_directives(directives_it->second);
}
std::ranges::sort(ranges, [](const RawFoldingRange& lhs, const RawFoldingRange& rhs) {
std::ranges::sort(ranges, [](const FoldingRange& lhs, const FoldingRange& rhs) {
if(lhs.range.begin != rhs.range.begin) {
return lhs.range.begin < rhs.range.begin;
}
@@ -343,14 +337,18 @@ private:
}
private:
std::vector<RawFoldingRange> ranges;
std::vector<FoldingRange> ranges;
};
} // namespace
auto folding_ranges(CompilationUnitRef unit) -> std::vector<FoldingRange> {
return FoldingRangeCollector(unit).collect();
}
auto folding_ranges(CompilationUnitRef unit, PositionEncoding encoding)
-> std::vector<protocol::FoldingRange> {
auto collected = FoldingRangeCollector(unit).collect();
auto collected = folding_ranges(unit);
PositionMapper converter(unit.interested_content(), encoding);
std::vector<protocol::FoldingRange> result;

View File

@@ -49,7 +49,7 @@ auto document_format(llvm::StringRef file,
range ? tooling::Range(range->begin, range->length()) : tooling::Range(0, content.size());
auto replacements = format_content(file, content, selection);
if(!replacements) {
LOG_INFO("Fail to format for {}\n{}", file, replacements.error());
LOG_WARN("Failed to format {}: {}", file, replacements.error());
return edits;
}

View File

@@ -26,22 +26,6 @@ using llvm::dyn_cast_or_null;
// For now, inlay hints are always anchored at the left or right of their range.
enum class HintSide { Left, Right };
enum class HintCategory : std::uint8_t {
Parameter,
DefaultArgument,
Type,
Designator,
BlockEnd,
};
struct RawInlayHint {
std::uint32_t offset = 0;
HintCategory kind = HintCategory::Type;
std::string label;
bool padding_left = false;
bool padding_right = false;
};
bool is_expanded_from_param_pack(const clang::ParmVarDecl* param) {
return ast::underlying_pack_type(param) != nullptr;
}
@@ -123,7 +107,7 @@ struct Callee {
class Builder {
public:
Builder(std::vector<RawInlayHint>& result,
Builder(std::vector<InlayHint>& result,
CompilationUnitRef unit,
LocalSourceRange restrict_range,
const InlayHintsOptions& options) :
@@ -131,8 +115,8 @@ public:
policy(unit.context().getPrintingPolicy()) {
// The sugared type is more useful in some cases, and the canonical
// type in other cases.
policy.SuppressScope = true; // keep type names short
policy.AnonymousTagLocations = false;
policy.SuppressScope = true; // keep type names short
policy.AnonymousTagLocations = false; // do not print lambda locations
// Not setting PrintCanonicalTypes for "auto" allows
// SuppressDefaultTemplateArgs (set by default) to have an effect.
}
@@ -499,7 +483,7 @@ public:
bool pad_left = prefix.consume_front(" ");
bool pad_right = suffix.consume_back(" ");
RawInlayHint hint{
InlayHint hint{
.offset = offset,
.kind = kind,
.label = (prefix + label + suffix).str(),
@@ -554,7 +538,7 @@ public:
}
private:
std::vector<RawInlayHint>& result;
std::vector<InlayHint>& result;
CompilationUnitRef unit;
LocalSourceRange restrict_range;
const InlayHintsOptions& options;
@@ -913,36 +897,43 @@ private:
} // namespace
auto inlay_hints(CompilationUnitRef unit,
LocalSourceRange target,
const InlayHintsOptions& options,
PositionEncoding encoding) -> std::vector<protocol::InlayHint> {
auto inlay_hints(CompilationUnitRef unit, LocalSourceRange target, const InlayHintsOptions& options)
-> std::vector<InlayHint> {
if(!options.enabled) {
return {};
}
std::vector<RawInlayHint> raw_hints;
std::vector<InlayHint> raw_hints;
Builder builder(raw_hints, unit, target, options);
Visitor visitor(builder, unit, target, options);
visitor.TraverseDecl(unit.tu());
std::ranges::sort(raw_hints, [](const RawInlayHint& lhs, const RawInlayHint& rhs) {
std::ranges::sort(raw_hints, [](const InlayHint& lhs, const InlayHint& rhs) {
return std::tie(lhs.offset, lhs.label, lhs.kind, lhs.padding_left, lhs.padding_right) <
std::tie(rhs.offset, rhs.label, rhs.kind, rhs.padding_left, rhs.padding_right);
});
auto unique_begin =
std::ranges::unique(raw_hints, [](const RawInlayHint& lhs, const RawInlayHint& rhs) {
std::ranges::unique(raw_hints, [](const InlayHint& lhs, const InlayHint& rhs) {
return lhs.offset == rhs.offset && lhs.kind == rhs.kind && lhs.label == rhs.label &&
lhs.padding_left == rhs.padding_left && lhs.padding_right == rhs.padding_right;
});
raw_hints.erase(unique_begin.begin(), unique_begin.end());
return raw_hints;
}
auto inlay_hints(CompilationUnitRef unit,
LocalSourceRange target,
const InlayHintsOptions& options,
PositionEncoding encoding) -> std::vector<protocol::InlayHint> {
auto collected = inlay_hints(unit, target, options);
PositionMapper converter(unit.interested_content(), encoding);
std::vector<protocol::InlayHint> hints;
hints.reserve(raw_hints.size());
hints.reserve(collected.size());
for(const auto& hint: raw_hints) {
for(const auto& hint: collected) {
protocol::InlayHint out{
.position = *converter.to_position(hint.offset),
.label = hint.label,

View File

@@ -18,12 +18,6 @@ namespace clice::feature {
namespace {
struct RawToken {
LocalSourceRange range;
SymbolKind kind = SymbolKind::Invalid;
std::uint32_t modifiers = 0;
};
void add_modifier(std::uint32_t& modifiers, SymbolModifiers::Kind kind) {
modifiers |= SymbolModifiers::to_mask(kind);
}
@@ -40,6 +34,34 @@ bool is_dependent(const clang::Decl* D) {
return isa<clang::UnresolvedUsingValueDecl>(D);
}
/// Whether a declaration name is backed by source text that should be highlighted.
bool can_highlight_name(clang::DeclarationName name) {
switch(name.getNameKind()) {
case clang::DeclarationName::Identifier: {
auto* info = name.getAsIdentifierInfo();
return info && !info->getName().empty();
}
case clang::DeclarationName::CXXConstructorName:
case clang::DeclarationName::CXXDestructorName: {
return true;
}
case clang::DeclarationName::CXXConversionFunctionName:
case clang::DeclarationName::CXXOperatorName:
case clang::DeclarationName::CXXDeductionGuideName:
case clang::DeclarationName::CXXLiteralOperatorName:
case clang::DeclarationName::CXXUsingDirective:
case clang::DeclarationName::ObjCZeroArgSelector:
case clang::DeclarationName::ObjCOneArgSelector:
case clang::DeclarationName::ObjCMultiArgSelector: {
return false;
}
}
std::unreachable();
}
/// Returns true if `decl` is considered to be from a default/system library.
/// This currently checks the systemness of the file by include type, although
/// different heuristics may be used in the future (e.g. sysroot paths).
@@ -166,7 +188,7 @@ class SemanticTokensCollector : public SemanticVisitor<SemanticTokensCollector>
public:
explicit SemanticTokensCollector(CompilationUnitRef unit) : SemanticVisitor(unit, true) {}
auto collect() -> std::vector<RawToken> {
auto collect() -> std::vector<SemanticToken> {
highlight_lexical(unit.interested_file());
run();
highlight_modules();
@@ -177,6 +199,10 @@ public:
void handleDeclOccurrence(const clang::NamedDecl* decl,
RelationKind relation,
clang::SourceLocation location) {
if(relation.isReference() && !can_highlight_name(decl->getDeclName())) {
return;
}
std::uint32_t modifiers = 0;
if(relation.is_one_of(RelationKind::Definition)) {
// todo: clangd add both Declaration and Definition modifiers for definitions.
@@ -398,7 +424,7 @@ private:
}
}
static void resolve_conflict(RawToken& last, const RawToken& current) {
static void resolve_conflict(SemanticToken& last, const SemanticToken& current) {
if(last.kind == SymbolKind::Conflict) {
return;
}
@@ -414,14 +440,14 @@ private:
}
void merge_tokens() {
std::ranges::sort(tokens, [](const RawToken& lhs, const RawToken& rhs) {
std::ranges::sort(tokens, [](const SemanticToken& lhs, const SemanticToken& rhs) {
if(lhs.range.begin != rhs.range.begin) {
return lhs.range.begin < rhs.range.begin;
}
return lhs.range.end < rhs.range.end;
});
std::vector<RawToken> merged;
std::vector<SemanticToken> merged;
merged.reserve(tokens.size());
for(const auto& token: tokens) {
@@ -448,7 +474,7 @@ private:
}
public:
std::vector<RawToken> tokens;
std::vector<SemanticToken> tokens;
};
class SemanticTokenEncoder {
@@ -458,7 +484,7 @@ public:
protocol::SemanticTokens& output) :
content(content), converter(content, encoding), output(output) {}
void append(const RawToken& token) {
void append(const SemanticToken& token) {
if(!token.range.valid() || token.range.end <= token.range.begin ||
token.range.end > content.size()) {
return;
@@ -542,10 +568,14 @@ private:
} // namespace
auto semantic_tokens(CompilationUnitRef unit) -> std::vector<SemanticToken> {
SemanticTokensCollector collector(unit);
return collector.collect();
}
auto semantic_tokens(CompilationUnitRef unit, PositionEncoding encoding)
-> protocol::SemanticTokens {
SemanticTokensCollector collector(unit);
auto tokens = collector.collect();
auto tokens = semantic_tokens(unit);
protocol::SemanticTokens result;
result.data.reserve(tokens.size() * 5);

View File

@@ -504,14 +504,14 @@ bool USRGenerator::GenLoc(const Decl* D, bool IncludeOffset) {
static void printQualifier(llvm::raw_ostream& Out,
const LangOptions& LangOpts,
NestedNameSpecifier NNS) {
NestedNameSpecifier* NNS) {
// FIXME: Encode the qualifier, don't just print it.
PrintingPolicy PO(LangOpts);
PO.SuppressTagKeyword = true;
PO.SuppressUnwrittenScope = true;
PO.ConstantArraySizeAsWritten = false;
PO.AnonymousTagLocations = false;
NNS.print(Out, PO);
NNS->print(Out, PO);
}
void USRGenerator::VisitType(QualType T) {
@@ -740,7 +740,7 @@ void USRGenerator::VisitType(QualType T) {
return;
}
if(const InjectedClassNameType* InjT = T->getAs<InjectedClassNameType>()) {
T = InjT->desugar();
T = InjT->getInjectedSpecializationType();
continue;
}
if(const auto* VT = T->getAs<VectorType>()) {

View File

@@ -200,6 +200,10 @@ llvm::StringRef identifier_of(const clang::NamedDecl& D) {
}
llvm::StringRef identifier_of(clang::QualType type) {
if(const auto* ET = llvm::dyn_cast<clang::ElaboratedType>(type)) {
return identifier_of(ET->getNamedType());
}
if(const auto* BT = llvm::dyn_cast<clang::BuiltinType>(type)) {
clang::PrintingPolicy PP(clang::LangOptions{});
PP.adjustForCPlusPlus();
@@ -304,6 +308,16 @@ const clang::NamedDecl* decl_of_impl(const void* T) {
}
auto decl_of(clang::QualType type) -> const clang::NamedDecl* {
if(type.isNull()) {
return nullptr;
}
// Strip type-sugar that wraps the underlying type without adding a decl
// (e.g. ElaboratedType for "struct Foo" vs plain "Foo").
if(auto ET = type->getAs<clang::ElaboratedType>()) {
type = ET->getNamedType();
}
if(auto TST = type->getAs<clang::TemplateSpecializationType>()) {
auto decl = TST->getTemplateName().getAsTemplateDecl();
if(type->isDependentType()) {
@@ -395,8 +409,8 @@ std::string display_name_of(const clang::NamedDecl* decl) {
// Handle 'using namespace'. They all have the same name - <using-directive>.
if(auto* UD = llvm::dyn_cast<clang::UsingDirectiveDecl>(decl)) {
out << "using namespace ";
if(auto Qual = UD->getQualifier())
Qual.print(out, policy);
if(auto* Qual = UD->getQualifier())
Qual->print(out, policy);
UD->getNominatedNamespaceAsWritten()->printName(out);
return out.str();
}
@@ -423,8 +437,8 @@ std::string display_name_of(const clang::NamedDecl* decl) {
}
// Print nested name qualifier if it was written in the source code.
if(auto qualifier = get_qualifier_loc(decl).getNestedNameSpecifier()) {
qualifier.print(out, policy);
if(auto* qualifier = get_qualifier_loc(decl).getNestedNameSpecifier()) {
qualifier->print(out, policy);
}
// Print the name itself.

View File

@@ -107,7 +107,7 @@ public:
return true;
}
bool TraverseTypeLoc(clang::TypeLoc loc, bool TraverseQualifier = true) {
bool TraverseTypeLoc(clang::TypeLoc loc) {
CHECK_DERIVED_IMPL(TraverseTypeLoc);
if(!loc) {
@@ -116,10 +116,10 @@ public:
/// FIXME: Workaround for `QualifiedTypeLoc`.
if(auto QL = loc.getAs<clang::QualifiedTypeLoc>()) {
return Base::TraverseTypeLoc(QL.getUnqualifiedLoc(), TraverseQualifier);
return Base::TraverseTypeLoc(QL.getUnqualifiedLoc());
}
return Base::TraverseTypeLoc(loc, TraverseQualifier);
return Base::TraverseTypeLoc(loc);
}
bool TraverseAttr(clang::Attr* attr) {
@@ -132,8 +132,9 @@ public:
return Base::TraverseAttr(attr);
}
/// We don't want to node without location information.
constexpr bool TraverseNestedNameSpecifier [[gnu::always_inline]] (clang::NestedNameSpecifier) {
/// We don't want to node withou location information.
constexpr bool TraverseNestedNameSpecifier
[[gnu::always_inline]] (clang::NestedNameSpecifier*) {
CHECK_DERIVED_IMPL(TraverseNestedNameSpecifier);
return true;
}

View File

@@ -126,18 +126,17 @@ struct InstantiationStack {
return data;
}
/// Look up a template type parameter in the stack by matching its depth and
/// ownership against each frame. Searches from innermost (top) to outermost
/// (bottom). Returns nullptr if no matching frame or index out of range.
/// Look up a template type parameter in the stack by matching its depth against
/// each frame's template parameter list depth. Searches from innermost (top) to
/// outermost (bottom). Returns nullptr if no matching frame or index out of range.
///
/// When the TTP has a declaration, ownership is verified: the frame's template
/// parameter at the same index must be the exact same TemplateTypeParmDecl.
/// This prevents depth collisions between unrelated templates at the same
/// nesting level (e.g. __alloc_traits and vector both at depth 0).
/// IMPORTANT: depth alone identifies the template "level", not the specific template.
/// Different templates at the same depth (e.g. vector and test both at depth 0) will
/// match the FIRST frame found. Callers must ensure the stack only contains relevant
/// frames when calling this.
const clang::TemplateArgument* find_argument(const clang::TemplateTypeParmType* T) const {
auto depth = T->getDepth();
auto index = T->getIndex();
auto* decl = T->getDecl();
for(auto it = data.rbegin(); it != data.rend(); ++it) {
clang::TemplateParameterList* params = nullptr;
if(auto* CTD = llvm::dyn_cast<clang::ClassTemplateDecl>(it->first)) {
@@ -151,16 +150,10 @@ struct InstantiationStack {
params = FTD->getTemplateParameters();
}
if(params && params->getDepth() == depth) {
if(index >= it->second.size()) {
return nullptr;
if(index < it->second.size()) {
return &it->second[index];
}
// If the TTP has a declaration, verify it actually belongs to this
// frame's template by checking the parameter list. Skip frames from
// unrelated templates that happen to share the same depth.
if(decl && index < params->size() && params->getParam(index) != decl) {
continue;
}
return &it->second[index];
return nullptr;
}
}
return nullptr;
@@ -174,10 +167,7 @@ static clang::QualType get_decl_type(clang::Decl* decl) {
if(auto* TND = llvm::dyn_cast<clang::TypedefNameDecl>(decl))
return TND->getUnderlyingType();
if(auto* RD = llvm::dyn_cast<clang::RecordDecl>(decl))
return decl->getASTContext().getTagType(clang::ElaboratedTypeKeyword::None,
std::nullopt,
RD,
false);
return clang::QualType(RD->getTypeForDecl(), 0);
return clang::QualType();
}
@@ -215,11 +205,14 @@ public:
/// Desugar dependent typedefs to expose template parameters for substitution.
clang::QualType TransformTypedefType(clang::TypeLocBuilder& TLB, clang::TypedefTypeLoc TL) {
if(auto* TND = TL.getDecl()) {
if(auto* TND = TL.getTypedefNameDecl()) {
auto underlying = TND->getUnderlyingType();
if(underlying->isDependentType()) {
auto type = TransformType(underlying);
if(!type.isNull()) {
if(auto ET = llvm::dyn_cast<clang::ElaboratedType>(type)) {
type = ET->getNamedType();
}
TLB.pushTrivial(context, type, {});
return type;
}
@@ -228,10 +221,20 @@ public:
return Base::TransformTypedefType(TLB, TL);
}
clang::QualType TransformElaboratedType(clang::TypeLocBuilder& TLB,
clang::ElaboratedTypeLoc TL) {
clang::QualType type = TransformType(TL.getNamedTypeLoc().getType());
if(type.isNull()) {
return Base::TransformElaboratedType(TLB, TL);
}
TLB.pushTrivial(context, type, {});
return type;
}
clang::QualType TransformInjectedClassNameType(clang::TypeLocBuilder& TLB,
clang::InjectedClassNameTypeLoc TL) {
auto ICT = TL.getTypePtr();
clang::QualType type = TransformType(ICT->desugar());
clang::QualType type = TransformType(ICT->getInjectedSpecializationType());
if(type.isNull()) {
return Base::TransformInjectedClassNameType(TLB, TL);
}
@@ -424,8 +427,7 @@ public:
visit_template_decl_contexts(
llvm::dyn_cast<clang::Decl>(decl->getDeclContext()),
[&](clang::Decl* decl, clang::TemplateParameterList* params) {
auto args = params->getInjectedTemplateArgs(context);
stack.push(decl, args);
stack.push(decl, params->getInjectedTemplateArgs(context));
});
std::ranges::reverse(stack.frames());
}
@@ -487,46 +489,31 @@ public:
lookup_result lookup(clang::QualType type, clang::DeclarationName name) {
clang::Decl* TD = nullptr;
llvm::ArrayRef<clang::TemplateArgument> args;
type = TransformType(type);
// For concrete TSTs (non-dependent template name), extract template info directly
// without calling TransformType. This avoids corrupting template arguments when
// unrelated stack frames match by depth (e.g. __alloc_traits at depth 0 would
// incorrectly substitute test's T which is also at depth 0).
if(auto TST = type->getAs<clang::TemplateSpecializationType>()) {
if(!TST->getTemplateName().getAsDependentTemplateName()) {
TD = TST->getTemplateName().getAsTemplateDecl();
args = TST->template_arguments();
}
if(type.isNull()) {
return lookup_result();
}
if(!TD) {
type = TransformType(type);
if(type.isNull()) {
if(auto TST = type->getAs<clang::TemplateSpecializationType>()) {
TD = TST->getTemplateName().getAsTemplateDecl();
args = TST->template_arguments();
} else if(auto DTST = type->getAs<clang::DependentTemplateSpecializationType>()) {
// If this DTST was already resolved (possibly to itself when unresolvable),
// skip the redundant lookup.
if(resolved.count(DTST)) {
return lookup_result();
}
if(auto TST = type->getAs<clang::TemplateSpecializationType>()) {
if(auto* DTN = TST->getTemplateName().getAsDependentTemplateName()) {
// If this dependent TST was already resolved (possibly to itself when
// unresolvable), skip the redundant lookup.
if(resolved.count(TST)) {
return lookup_result();
}
auto& template_name = DTST->getDependentTemplateName();
auto name = template_name.getName().getIdentifier();
if(!name) {
return {};
}
auto name = DTN->getName().getIdentifier();
if(!name) {
return {};
}
if(auto decl = preferred(lookup(DTN->getQualifier(), name))) {
TD = decl;
args = TST->template_arguments();
}
} else {
TD = TST->getTemplateName().getAsTemplateDecl();
args = TST->template_arguments();
}
if(auto decl = preferred(lookup(template_name.getQualifier(), name))) {
TD = decl;
args = DTST->template_arguments();
}
}
@@ -549,23 +536,45 @@ public:
return lookup_result();
}
lookup_result lookup(clang::NestedNameSpecifier NNS, clang::DeclarationName name) {
lookup_result lookup(const clang::NestedNameSpecifier* NNS, clang::DeclarationName name) {
if(!NNS) {
return lookup_result();
}
// In LLVM 22+, the old Identifier NNS kind is represented as a Type
// (DependentNameType), so the Type case handles both old TypeSpec and
// Identifier cases via lookup(QualType, name) → TransformDependentNameType.
switch(NNS.getKind()) {
case clang::NestedNameSpecifier::Kind::Type: {
return lookup(clang::QualType(NNS.getAsType(), 0), name);
if(auto iter = resolved.find(NNS); iter != resolved.end()) {
return lookup(iter->second, name);
}
// Handle each NestedNameSpecifier kind:
// - Identifier: dependent name in NNS chain (e.g. `base::type::inner`), resolve recursively
// - TypeSpec: concrete or dependent type used as qualifier (e.g. `vector<T>::`)
// - Global/Namespace/NamespaceAlias/Super: not dependent, cannot resolve further
switch(NNS->getKind()) {
case clang::NestedNameSpecifier::Identifier: {
auto stack_size = stack.data.size();
auto* decl = preferred(lookup(NNS->getPrefix(), NNS->getAsIdentifier()));
auto type = get_decl_type(decl);
if(!type.isNull()) {
type = substitute(type);
}
while(stack.data.size() > stack_size) {
stack.pop();
}
if(!type.isNull()) {
resolved.try_emplace(NNS, type);
return lookup(type, name);
}
return {};
}
case clang::NestedNameSpecifier::Kind::Global:
case clang::NestedNameSpecifier::Kind::Namespace:
case clang::NestedNameSpecifier::Kind::MicrosoftSuper:
case clang::NestedNameSpecifier::Kind::Null: {
case clang::NestedNameSpecifier::TypeSpec: {
return lookup(clang::QualType(NNS->getAsType(), 0), name);
}
case clang::NestedNameSpecifier::Global:
case clang::NestedNameSpecifier::Namespace:
case clang::NestedNameSpecifier::NamespaceAlias:
case clang::NestedNameSpecifier::Super: {
return {};
}
}
@@ -699,14 +708,14 @@ public:
///
/// TODO: Replace with a general mechanism for resolving well-known standard
/// library patterns, or improve the resolver to handle these chains naturally.
clang::QualType hole(clang::NestedNameSpecifier NNS,
clang::QualType hole(clang::NestedNameSpecifier* NNS,
const clang::IdentifierInfo* member,
TemplateArguments arguments) {
if(NNS.getKind() != clang::NestedNameSpecifier::Kind::Type) {
if(NNS->getKind() != clang::NestedNameSpecifier::TypeSpec) {
return clang::QualType();
}
auto TST = NNS.getAsType()->getAs<clang::TemplateSpecializationType>();
auto TST = NNS->getAsType()->getAs<clang::TemplateSpecializationType>();
if(!TST) {
return clang::QualType();
}
@@ -729,18 +738,17 @@ public:
return clang::QualType();
auto T = arguments[0].getAsType();
clang::NestedNameSpecifier prefix(Alloc.getTypePtr());
auto prefix =
clang::NestedNameSpecifier::Create(context, nullptr, Alloc.getTypePtr());
auto rebind = sema.getPreprocessor().getIdentifierInfo("rebind");
auto DTN = context.getDependentTemplateName(
clang::DependentTemplateStorage(prefix, rebind, false));
auto TST = context.getTemplateSpecializationType(clang::ElaboratedTypeKeyword::None,
DTN,
arguments,
{});
auto DTST = context.getDependentTemplateSpecializationType(
clang::ElaboratedTypeKeyword::None,
clang::DependentTemplateStorage(prefix, rebind, false),
arguments);
prefix = clang::NestedNameSpecifier(TST.getTypePtr());
prefix = clang::NestedNameSpecifier::Create(context, prefix, DTST.getTypePtr());
auto other = sema.getPreprocessor().getIdentifierInfo("other");
auto DNT = context.getDependentNameType(clang::ElaboratedTypeKeyword::Typename,
@@ -762,11 +770,9 @@ public:
for(auto& arg: replaceArguments) {
canonicalArguments.emplace_back(context.getCanonicalTemplateArgument(arg));
}
auto result =
context.getTemplateSpecializationType(clang::ElaboratedTypeKeyword::None,
TST->getTemplateName(),
replaceArguments,
canonicalArguments);
auto result = context.getTemplateSpecializationType(TST->getTemplateName(),
replaceArguments,
canonicalArguments);
LOG_DEBUG(
"{}" "hole: 'allocator_traits::rebind_alloc' → '{}'",
pad(),
@@ -851,39 +857,9 @@ public:
return TL.getType();
}
/// Push frames for outer templates in a DependentTemplateName qualifier chain.
/// For `A<X>::template B<Y>::template C<Z>::type`, walking the chain from the
/// DependentNameType's NNS pushes frames for A (T1→X) and B (T2→Y) so that
/// when "type" is found in C<Z>, the substitute step correctly maps outer
/// template parameters to the actual arguments from the NNS chain.
void push_nns_qualifier_frames(clang::NestedNameSpecifier NNS) {
if(!NNS || NNS.getKind() != clang::NestedNameSpecifier::Kind::Type)
return;
auto* TST = NNS.getAsType()->getAs<clang::TemplateSpecializationType>();
if(!TST)
return;
if(auto* DTN = TST->getTemplateName().getAsDependentTemplateName()) {
push_nns_qualifier_frames(DTN->getQualifier());
if(auto it = resolved.find(TST); it != resolved.end()) {
if(auto* rTST = it->second->getAs<clang::TemplateSpecializationType>()) {
if(auto* TD = rTST->getTemplateName().getAsTemplateDecl()) {
stack.push(TD, TST->template_arguments());
}
}
}
} else if(auto* TD = TST->getTemplateName().getAsTemplateDecl()) {
stack.push(TD, TST->template_arguments());
}
}
clang::QualType TransformDependentNameType(clang::TypeLocBuilder& TLB,
clang::DependentNameTypeLoc TL,
bool DeducedTSTContext = false,
clang::QualType ObjectType = {},
clang::NamedDecl* UnqualLookup = nullptr) {
bool DeducedTSTContext = false) {
auto* DNT = TL.getTypePtr();
LOG_DEBUG("{}" "resolve '{}'", pad(), clang::QualType(DNT, 0).getAsString());
++indent;
@@ -908,14 +884,8 @@ public:
return original;
}
// Save entry stack size to clean up any frames leaked by nested operations
// (NNS transform, push_nns_qualifier_frames, lookup, TransformType).
auto entry_stack_size = stack.data.size();
auto NNSLoc = TransformNestedNameSpecifierLoc(TL.getQualifierLoc());
if(!NNSLoc) {
while(stack.data.size() > entry_stack_size)
stack.pop();
active_resolutions.erase(DNT);
LOG_DEBUG("{}→ <unresolved>", pad());
--indent;
@@ -927,21 +897,7 @@ public:
return original;
}
auto NNS = NNSLoc.getNestedNameSpecifier();
// For nested dependent template specializations like A<X>::B<Y>::C<Z>::type,
// push frames for outer templates (A, B) using args from the original NNS
// chain. Without this, the lookup fabricates self-referential injected args
// for outer templates, leaving their parameters unresolved.
auto origNNS = DNT->getQualifier();
if(origNNS.getKind() == clang::NestedNameSpecifier::Kind::Type) {
if(auto* TST = origNNS.getAsType()->getAs<clang::TemplateSpecializationType>()) {
if(auto* DTN = TST->getTemplateName().getAsDependentTemplateName()) {
push_nns_qualifier_frames(DTN->getQualifier());
}
}
}
auto* NNS = NNSLoc.getNestedNameSpecifier();
auto stack_size = stack.data.size();
auto* decl = preferred(lookup(NNS, DNT->getIdentifier()));
auto type = get_decl_type(decl);
@@ -987,11 +943,6 @@ public:
}
}
// Clean up all frames pushed during this resolution (push_nns_qualifier_frames,
// nested lookups via TransformType, etc.) to prevent leaking into sibling calls.
while(stack.data.size() > entry_stack_size)
stack.pop();
active_resolutions.erase(DNT);
if(!result.isNull()) {
@@ -1012,57 +963,43 @@ public:
return original;
}
clang::QualType rebuild_tst(clang::TypeLocBuilder& TLB,
clang::TemplateSpecializationTypeLoc TL) {
auto* TST = TL.getTypePtr();
return TLB.push<clang::TemplateSpecializationTypeLoc>(clang::QualType(TST, 0)).getType();
using Base::TransformDependentTemplateSpecializationType;
clang::QualType rebuild_dtst(clang::TypeLocBuilder& TLB,
clang::DependentTemplateSpecializationTypeLoc TL) {
auto* DTST = TL.getTypePtr();
return TLB.push<clang::DependentTemplateSpecializationTypeLoc>(clang::QualType(DTST, 0))
.getType();
}
clang::QualType
TransformTemplateSpecializationType(clang::TypeLocBuilder& TLB,
clang::TemplateSpecializationTypeLoc TL,
clang::QualType ObjectType = {},
clang::NamedDecl* FirstQualifierInScope = nullptr,
bool AllowInjectedClassName = false) {
auto* TST = TL.getTypePtr();
auto* DTN = TST->getTemplateName().getAsDependentTemplateName();
if(!DTN) {
return Base::TransformTemplateSpecializationType(TLB,
TL,
ObjectType,
FirstQualifierInScope,
AllowInjectedClassName);
}
LOG_DEBUG("{}" "resolve dependent TST '{}'", pad(), clang::QualType(TST, 0).getAsString());
clang::QualType TransformDependentTemplateSpecializationType(
clang::TypeLocBuilder& TLB,
clang::DependentTemplateSpecializationTypeLoc TL) {
auto* DTST = TL.getTypePtr();
LOG_DEBUG("{}" "resolve DTST '{}'", pad(), clang::QualType(DTST, 0).getAsString());
++indent;
if(auto iter = resolved.find(TST); iter != resolved.end()) {
if(auto iter = resolved.find(DTST); iter != resolved.end()) {
--indent;
TLB.pushTrivial(context, iter->second, {});
return iter->second;
}
// Save stack state to clean up frames pushed by NNS/lookup side effects.
auto stack_size = stack.data.size();
auto NNSLoc = TransformNestedNameSpecifierLoc(TL.getQualifierLoc());
if(!NNSLoc) {
while(stack.data.size() > stack_size)
stack.pop();
LOG_DEBUG("{}→ <unresolved dependent TST>", pad());
LOG_DEBUG("{}→ <unresolved DTST>", pad());
--indent;
return rebuild_tst(TLB, TL);
return rebuild_dtst(TLB, TL);
}
auto NNS = NNSLoc.getNestedNameSpecifier();
auto* NNS = NNSLoc.getNestedNameSpecifier();
clang::TemplateArgumentListInfo info;
using iterator =
clang::TemplateArgumentLocContainerIterator<clang::TemplateSpecializationTypeLoc>;
using iterator = clang::TemplateArgumentLocContainerIterator<
clang::DependentTemplateSpecializationTypeLoc>;
if(TransformTemplateArguments(iterator(TL, 0), iterator(TL, TL.getNumArgs()), info)) {
LOG_DEBUG("{}→ <unresolved dependent TST>", pad());
LOG_DEBUG("{}→ <unresolved DTST>", pad());
--indent;
return rebuild_tst(TLB, TL);
return rebuild_dtst(TLB, TL);
}
llvm::SmallVector<clang::TemplateArgument, 4> arguments;
@@ -1070,21 +1007,22 @@ public:
arguments.push_back(arg.getArgument());
}
auto* name = DTN->getName().getIdentifier();
auto* name = DTST->getDependentTemplateName().getName().getIdentifier();
if(!name) {
LOG_DEBUG("{}→ <unresolved dependent TST>", pad());
LOG_DEBUG("{}→ <unresolved DTST>", pad());
--indent;
return rebuild_tst(TLB, TL);
return rebuild_dtst(TLB, TL);
}
if(auto result = hole(NNS, name, arguments); !result.isNull()) {
LOG_DEBUG("{}" "hole: '{}' → '{}'", pad(), name->getName().str(), result.getAsString());
--indent;
resolved.try_emplace(TST, result);
resolved.try_emplace(DTST, result);
TLB.pushTrivial(context, result, {});
return result;
}
auto stack_size = stack.data.size();
if(auto* decl = preferred(lookup(NNS, name))) {
if(auto* TATD = llvm::dyn_cast<clang::TypeAliasTemplateDecl>(decl)) {
if(deduce_template_arguments(TATD, arguments)) {
@@ -1099,29 +1037,26 @@ public:
if(!type.isNull()) {
LOG_DEBUG("{}" "→ '{}' (alias)", pad(), type.getAsString());
--indent;
resolved.try_emplace(TST, type);
resolved.try_emplace(DTST, type);
TLB.pushTrivial(context, type, {});
return type;
}
}
} else if(auto* CTD = llvm::dyn_cast<clang::ClassTemplateDecl>(decl)) {
// Pop lookup frames — we only needed them to find the CTD.
while(stack.data.size() > stack_size) {
stack.pop();
}
// Resolve DTST to a concrete TemplateSpecializationType.
// e.g. __alloc_traits<allocator<T>>::rebind<T> → rebind<T> (a TST)
// This allows subsequent lookup of members (like "other") to work.
// Keep lookup frames on stack — the caller (e.g. TransformNestedNameSpecifierLoc
// processing A<X>::B<Y>::C<Z>) needs them for parameter substitution.
clang::TemplateName TN(CTD);
llvm::SmallVector<clang::TemplateArgument> canonArgs;
for(auto& arg: arguments) {
canonArgs.push_back(context.getCanonicalTemplateArgument(arg));
}
auto result =
context.getTemplateSpecializationType(clang::ElaboratedTypeKeyword::None,
TN,
arguments,
canonArgs);
auto result = context.getTemplateSpecializationType(TN, arguments, canonArgs);
LOG_DEBUG("{}" "→ TST '{}' (class)", pad(), result.getAsString());
--indent;
resolved.try_emplace(TST, result);
resolved.try_emplace(DTST, result);
TLB.pushTrivial(context, result, {});
return result;
}
@@ -1130,10 +1065,10 @@ public:
stack.pop();
}
LOG_DEBUG("{}→ <unresolved dependent TST>", pad());
LOG_DEBUG("{}→ <unresolved DTST>", pad());
--indent;
auto fallback = rebuild_tst(TLB, TL);
resolved.try_emplace(TST, fallback);
auto fallback = rebuild_dtst(TLB, TL);
resolved.try_emplace(DTST, fallback);
return fallback;
}
@@ -1142,11 +1077,14 @@ public:
/// its own TransformTypedefType). Using substitute() here ensures that typedef
/// expansion does NOT trigger heuristic lookup, preventing the typedef ↔ lookup cycle.
clang::QualType TransformTypedefType(clang::TypeLocBuilder& TLB, clang::TypedefTypeLoc TL) {
if(auto* TND = TL.getDecl()) {
if(auto* TND = TL.getTypedefNameDecl()) {
auto underlying = TND->getUnderlyingType();
if(underlying->isDependentType()) {
auto type = substitute(underlying);
if(!type.isNull()) {
if(auto ET = llvm::dyn_cast<clang::ElaboratedType>(type)) {
type = ET->getNamedType();
}
TLB.pushTrivial(context, type, {});
return type;
}
@@ -1173,8 +1111,6 @@ public:
return Base::TransformDecltypeType(TLB, TL);
}
// --- State ---
private:
clang::Sema& sema;
clang::ASTContext& context;
@@ -1202,7 +1138,7 @@ clang::QualType TemplateResolver::resugar(clang::QualType type, clang::Decl* dec
return resugar.TransformType(type);
}
TemplateResolver::lookup_result TemplateResolver::lookup(clang::NestedNameSpecifier NNS,
TemplateResolver::lookup_result TemplateResolver::lookup(const clang::NestedNameSpecifier* NNS,
clang::DeclarationName name) {
PseudoInstantiator instantiator(sema, resolved);
return instantiator.lookup(NNS, name);

View File

@@ -41,17 +41,17 @@ public:
using lookup_result = clang::DeclContext::lookup_result;
/// Look up the name in the given nested name specifier.
lookup_result lookup(clang::NestedNameSpecifier NNS, clang::DeclarationName name);
lookup_result lookup(const clang::NestedNameSpecifier* NNS, clang::DeclarationName name);
lookup_result lookup(const clang::DependentNameType* type) {
return lookup(type->getQualifier(), type->getIdentifier());
}
lookup_result lookup(const clang::TemplateSpecializationType* type,
const clang::DependentTemplateName* DTN) {
auto identifier = DTN->getName().getIdentifier();
lookup_result lookup(const clang::DependentTemplateSpecializationType* type) {
auto& template_name = type->getDependentTemplateName();
auto identifier = template_name.getName().getIdentifier();
if(identifier) {
return lookup(DTN->getQualifier(), identifier);
return lookup(template_name.getQualifier(), identifier);
} else {
/// TODO: Operators don't have an IdentifierInfo; need DeclarationName-based lookup.
return {};

View File

@@ -756,8 +756,8 @@ public:
return traverse_node(X, [&] { return Base::TraverseDecl(X); });
}
bool TraverseTypeLoc(clang::TypeLoc X, bool TraverseQualifier = true) {
return traverse_node(&X, [&] { return Base::TraverseTypeLoc(X, TraverseQualifier); });
bool TraverseTypeLoc(clang::TypeLoc X) {
return traverse_node(&X, [&] { return Base::TraverseTypeLoc(X); });
}
bool TraverseTemplateArgumentLoc(const clang::TemplateArgumentLoc& X) {
@@ -814,9 +814,9 @@ public:
// This means we'd never see 'int' in 'const int'! Work around that here.
// (The reason for the behavior is to avoid traversing the nested Type twice,
// but we ignore TraverseType anyway).
bool TraverseQualifiedTypeLoc(clang::QualifiedTypeLoc QX, bool TraverseQualifier = true) {
bool TraverseQualifiedTypeLoc(clang::QualifiedTypeLoc QX) {
return traverse_node<clang::TypeLoc>(&QX, [&] {
return TraverseTypeLoc(QX.getUnqualifiedLoc(), TraverseQualifier);
return TraverseTypeLoc(QX.getUnqualifiedLoc());
});
}
@@ -825,7 +825,7 @@ public:
}
// Uninteresting parts of the AST that don't have locations within them.
bool TraverseNestedNameSpecifier(clang::NestedNameSpecifier) {
bool TraverseNestedNameSpecifier(clang::NestedNameSpecifier*) {
return true;
}

View File

@@ -515,7 +515,7 @@ public:
/// using Foo = int; Foo foo;
/// ^~~~ reference
VISIT_TYPELOC(TypedefTypeLoc) {
auto decl = loc.getDecl();
auto decl = loc.getTypedefNameDecl();
auto location = loc.getNameLoc();
handleDeclOccurrence(decl, RelationKind::Reference, location);
handleRelation(decl, RelationKind::Reference, decl, location);
@@ -561,9 +561,14 @@ public:
/// std::allocator<T>::rebind<U>
/// ^~~~ reference
/// Note: In LLVM 22+, DependentTemplateSpecializationTypeLoc was merged
/// into TemplateSpecializationTypeLoc. Dependent template cases are now
/// handled by the TemplateSpecializationTypeLoc visitor above.
VISIT_TYPELOC(DependentTemplateSpecializationTypeLoc) {
auto location = loc.getTemplateNameLoc();
// for(auto decl: resolver.lookup(loc.getTypePtr())) {
// handleDeclOccurrence(decl, RelationKind::WeakReference, location);
// handleRelation(decl, RelationKind::WeakReference, decl, location);
// }
return true;
}
/// ============================================================================
/// Specifier
@@ -571,19 +576,32 @@ public:
bool VisitNestedNameSpecifierLoc(clang::NestedNameSpecifierLoc loc) {
auto NNS = loc.getNestedNameSpecifier();
switch(NNS.getKind()) {
case clang::NestedNameSpecifier::Kind::Namespace: {
auto [ns, prefix] = NNS.getAsNamespaceAndPrefix();
switch(NNS->getKind()) {
case clang::NestedNameSpecifier::Namespace: {
auto decl = NNS->getAsNamespace();
auto location = loc.getLocalBeginLoc();
handleDeclOccurrence(ns, RelationKind::Reference, location);
handleRelation(ns, RelationKind::Reference, ns, location);
handleDeclOccurrence(decl, RelationKind::Reference, location);
handleRelation(decl, RelationKind::Reference, decl, location);
break;
}
case clang::NestedNameSpecifier::Kind::Type:
case clang::NestedNameSpecifier::Kind::Global:
case clang::NestedNameSpecifier::Kind::MicrosoftSuper:
case clang::NestedNameSpecifier::Kind::Null: {
case clang::NestedNameSpecifier::NamespaceAlias: {
auto decl = NNS->getAsNamespaceAlias();
auto location = loc.getLocalBeginLoc();
handleDeclOccurrence(decl, RelationKind::Reference, location);
handleRelation(decl, RelationKind::Reference, decl, location);
break;
}
case clang::NestedNameSpecifier::Identifier: {
assert(NNS->isDependent() && "Identifier NNS should be dependent");
// FIXME: use TemplateResolver here.
break;
}
case clang::NestedNameSpecifier::TypeSpec:
case clang::NestedNameSpecifier::Global:
case clang::NestedNameSpecifier::Super: {
break;
};
}

View File

@@ -1,4 +1,4 @@
#include "server/compile_graph.h"
#include "server/compiler/compile_graph.h"
#include <algorithm>

View File

@@ -1,4 +1,4 @@
#include "server/compiler.h"
#include "server/compiler/compiler.h"
#include <format>
#include <ranges>
@@ -6,7 +6,7 @@
#include "command/search_config.h"
#include "index/tu_index.h"
#include "server/protocol.h"
#include "server/protocol/worker.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "syntax/include_resolver.h"
@@ -28,16 +28,20 @@ using serde_raw = kota::codec::RawValue;
/// Detect whether the cursor is inside a preamble directive (include/import).
Compiler::Compiler(kota::event_loop& loop,
kota::ipc::JsonPeer& peer,
Workspace& workspace,
WorkerPool& pool,
llvm::DenseMap<std::uint32_t, Session>& sessions) :
loop(loop), peer(peer), workspace(workspace), pool(pool), sessions(sessions) {}
loop(loop), workspace(workspace), pool(pool), sessions(sessions) {}
Compiler::~Compiler() {
workspace.cancel_all();
}
kota::task<> Compiler::stop() {
compile_tasks.cancel();
co_await compile_tasks.join();
}
void Compiler::init_compile_graph() {
if(workspace.path_to_module.empty()) {
LOG_INFO("No C++20 modules detected, skipping CompileGraph");
@@ -410,6 +414,8 @@ std::string uri_to_path(const std::string& uri) {
void Compiler::publish_diagnostics(const std::string& uri,
int version,
const kota::codec::RawValue& diagnostics_json) {
if(!peer)
return;
std::vector<protocol::Diagnostic> diagnostics;
if(!diagnostics_json.empty()) {
auto status = kota::codec::json::from_json(diagnostics_json.data, diagnostics);
@@ -421,14 +427,16 @@ void Compiler::publish_diagnostics(const std::string& uri,
params.uri = uri;
params.version = version;
params.diagnostics = std::move(diagnostics);
peer.send_notification(params);
peer->send_notification(params);
}
void Compiler::clear_diagnostics(const std::string& uri) {
if(!peer)
return;
protocol::PublishDiagnosticsParams params;
params.uri = uri;
params.diagnostics = {};
peer.send_notification(params);
peer->send_notification(params);
}
kota::task<bool> Compiler::ensure_pch(Session& session,
@@ -490,6 +498,22 @@ kota::task<bool> Compiler::ensure_pch(Session& session,
auto completion = std::make_shared<kota::event>();
workspace.pch_cache[path_id].building = completion;
if(workspace.config.project.cache_dir.empty()) {
LOG_WARN("PCH build skipped: cache_dir is not configured");
workspace.pch_cache[path_id].building.reset();
completion->set();
co_return false;
}
// Ensure the PCH cache directory exists.
auto pch_dir = path::join(workspace.config.project.cache_dir, "cache", "pch");
if(auto ec = llvm::sys::fs::create_directories(pch_dir)) {
LOG_WARN("Cannot create PCH cache dir {}: {}", pch_dir, ec.message());
workspace.pch_cache[path_id].building.reset();
completion->set();
co_return false;
}
// Build a new PCH via stateless worker.
worker::BuildParams bp;
bp.kind = worker::BuildKind::BuildPCH;
@@ -613,6 +637,102 @@ void Compiler::record_deps(Session& session, llvm::ArrayRef<std::string> deps) {
/// Called lazily by forward_query() / forward_build() before every
/// feature request (hover, semantic tokens, etc.). Guarantees that when it
/// returns true the stateful worker assigned to `path_id` holds an up-to-date
kota::task<> Compiler::run_compile(std::uint32_t pid, std::shared_ptr<Session::PendingCompile> pc) {
auto find_session = [&]() -> Session* {
auto it = sessions.find(pid);
return it != sessions.end() ? &it->second : nullptr;
};
auto* sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
auto finish_compile = [&]() {
auto* s = find_session();
if(s && s->compiling == pc) {
s->compiling.reset();
}
LOG_INFO("ensure_compiled: finish path_id={}", pid);
pc->done.set();
};
auto gen = sess->generation;
LOG_INFO("ensure_compiled: starting compile path_id={} gen={}", pid, gen);
auto file_path = std::string(workspace.path_pool.resolve(pid));
auto uri = lsp::URI::from_file_path(file_path);
std::string uri_str = uri.has_value() ? uri->str() : file_path;
worker::CompileParams params;
params.path = file_path;
params.version = sess->version;
params.text = sess->text;
params.clang_tidy = workspace.config.project.clang_tidy.value;
if(!fill_compile_args(file_path, params.directory, params.arguments, sess)) {
finish_compile();
co_return;
}
if(!co_await ensure_deps(*sess, params.directory, params.arguments, params.pch, params.pcms)) {
LOG_WARN("Dependency preparation failed for {}, skipping compile", uri_str);
finish_compile();
co_return;
}
sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
auto result = co_await pool.send_stateful(pid, params);
sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
if(sess->generation != gen) {
LOG_INFO("ensure_compiled: generation mismatch ({} vs {}) for {}",
sess->generation,
gen,
uri_str);
finish_compile();
co_return;
}
if(!result.has_value()) {
LOG_WARN("Compile failed for {}: {}", uri_str, result.error().message);
clear_diagnostics(uri_str);
finish_compile();
co_return;
}
sess->ast_dirty = false;
pc->succeeded = true;
record_deps(*sess, result.value().deps);
if(!result.value().tu_index_data.empty()) {
auto tu_index = index::TUIndex::from(result.value().tu_index_data.data());
OpenFileIndex ofi;
ofi.file_index = std::move(tu_index.main_file_index);
ofi.symbols = std::move(tu_index.symbols);
ofi.content = sess->text;
ofi.mapper.emplace(ofi.content, lsp::PositionEncoding::UTF16);
sess->file_index = std::move(ofi);
}
auto version = sess->version;
finish_compile();
publish_diagnostics(uri_str, version, result.value().diagnostics);
if(on_indexing_needed)
on_indexing_needed();
}
/// AST and diagnostics have been published to the client.
///
/// Lifecycle overview (pull-based model):
@@ -632,9 +752,9 @@ void Compiler::record_deps(Session& session, llvm::ArrayRef<std::string> deps) {
/// worker); every other file is read from disk by the compiler.
///
/// Concurrency: multiple concurrent feature requests for the same file will
/// each call ensure_compiled(). The first one launches a detached compile
/// task via loop.schedule(); subsequent ones wait on the shared event.
/// The detached task cannot be cancelled by LSP $/cancelRequest, preventing
/// each call ensure_compiled(). The first one spawns a compile task into the
/// Compiler's task_group; subsequent ones wait on the shared event.
/// The spawned task is not cancelled by LSP $/cancelRequest, preventing
/// the race where cancellation wakes all waiters and they all start compiles.
kota::task<bool> Compiler::ensure_compiled(Session& session) {
auto path_id = session.path_id;
@@ -663,124 +783,12 @@ kota::task<bool> Compiler::ensure_compiled(Session& session) {
co_return true;
}
// No compile in flight and AST is dirty — launch a detached compile task.
// The detached task is scheduled via loop.schedule() so it is NOT subject
// to LSP $/cancelRequest cancellation. This eliminates the race where
// cancellation fires the RAII guard, waking all waiters simultaneously
// and causing them all to start new compiles.
auto pending_compile = std::make_shared<Session::PendingCompile>();
session.compiling = pending_compile;
LOG_INFO("ensure_compiled: launching detached compile path_id={} gen={}",
path_id,
session.generation);
LOG_INFO("ensure_compiled: launching compile path_id={} gen={}", path_id, session.generation);
// Capture path_id by value so the detached lambda can re-lookup the session
// from the sessions map after co_await (DenseMap may invalidate pointers).
loop.schedule([](Compiler* self,
std::uint32_t pid,
std::shared_ptr<Session::PendingCompile> pc) -> kota::task<> {
// Re-lookup session from the sessions map (pointer may have been
// invalidated by DenseMap growth during co_await).
auto find_session = [&]() -> Session* {
auto it = self->sessions.find(pid);
return it != self->sessions.end() ? &it->second : nullptr;
};
auto* sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
auto finish_compile = [&]() {
auto* s = find_session();
if(s && s->compiling == pc) {
s->compiling.reset();
}
LOG_INFO("ensure_compiled: finish_compile (detached) path_id={}", pid);
pc->done.set();
};
auto gen = sess->generation;
LOG_INFO("ensure_compiled: starting compile (detached) path_id={} gen={}", pid, gen);
auto file_path = std::string(self->workspace.path_pool.resolve(pid));
auto uri = lsp::URI::from_file_path(file_path);
std::string uri_str = uri.has_value() ? uri->str() : file_path;
worker::CompileParams params;
params.path = file_path;
params.version = sess->version;
params.text = sess->text;
if(!self->fill_compile_args(file_path, params.directory, params.arguments, sess)) {
finish_compile();
co_return;
}
if(!co_await self
->ensure_deps(*sess, params.directory, params.arguments, params.pch, params.pcms)) {
LOG_WARN("Dependency preparation failed for {}, skipping compile", uri_str);
finish_compile();
co_return;
}
// Re-lookup after co_await (DenseMap may have grown).
sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
auto result = co_await self->pool.send_stateful(pid, params);
// Re-lookup after co_await.
sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
if(sess->generation != gen) {
LOG_INFO("ensure_compiled: generation mismatch ({} vs {}) for {}",
sess->generation,
gen,
uri_str);
finish_compile();
co_return;
}
if(!result.has_value()) {
LOG_WARN("Compile failed for {}: {}", uri_str, result.error().message);
self->clear_diagnostics(uri_str);
finish_compile();
co_return;
}
sess->ast_dirty = false;
pc->succeeded = true;
self->record_deps(*sess, result.value().deps);
// Store open file index from the stateful worker's TUIndex.
if(!result.value().tu_index_data.empty()) {
auto tu_index = index::TUIndex::from(result.value().tu_index_data.data());
OpenFileIndex ofi;
ofi.file_index = std::move(tu_index.main_file_index);
ofi.symbols = std::move(tu_index.symbols);
ofi.content = sess->text;
ofi.mapper.emplace(ofi.content, lsp::PositionEncoding::UTF16);
sess->file_index = std::move(ofi);
}
auto version = sess->version;
finish_compile();
// Publish diagnostics AFTER marking compile as done, so that concurrent
// forward_query() calls can proceed immediately.
self->publish_diagnostics(uri_str, version, result.value().diagnostics);
if(self->on_indexing_needed)
self->on_indexing_needed();
}(this, path_id, pending_compile));
compile_tasks.spawn(run_compile(path_id, pending_compile));
// Wait for the detached compile to finish. If this wait is cancelled
// by LSP $/cancelRequest, the detached task continues unaffected.
@@ -875,6 +883,32 @@ Compiler::RawResult Compiler::forward_build(worker::BuildKind kind,
co_return std::move(result.value().result_json);
}
Compiler::RawResult Compiler::forward_format(Session& session,
std::optional<protocol::Range> range) {
auto path_id = session.path_id;
auto path = std::string(workspace.path_pool.resolve(path_id));
worker::BuildParams wp;
wp.kind = worker::BuildKind::Format;
wp.file = path;
wp.text = session.text;
if(range) {
lsp::PositionMapper mapper(wp.text, lsp::PositionEncoding::UTF16);
auto begin = mapper.to_offset(range->start);
auto end = mapper.to_offset(range->end);
if(!begin || !end)
co_return serde_raw{"null"};
wp.format_range = {*begin, *end};
}
auto result = co_await pool.send_stateless(wp);
if(!result.has_value()) {
co_return serde_raw{"null"};
}
co_return std::move(result.value().result_json);
}
Compiler::RawResult Compiler::handle_completion(const protocol::Position& position,
Session& session) {
auto path_id = session.path_id;

View File

@@ -8,13 +8,13 @@
#include <vector>
#include "command/command.h"
#include "server/session.h"
#include "server/worker_pool.h"
#include "server/workspace.h"
#include "server/service/session.h"
#include "server/worker/worker_pool.h"
#include "server/workspace/workspace.h"
#include "syntax/completion.h"
#include "kota/async/async.h"
#include "kota/codec/raw_value.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/peer.h"
@@ -50,10 +50,14 @@ std::string uri_to_path(const std::string& uri);
class Compiler {
public:
Compiler(kota::event_loop& loop,
kota::ipc::JsonPeer& peer,
Workspace& workspace,
WorkerPool& pool,
llvm::DenseMap<std::uint32_t, Session>& sessions);
void set_peer(kota::ipc::JsonPeer* p) {
peer = p;
}
~Compiler();
void init_compile_graph();
@@ -86,6 +90,9 @@ public:
const protocol::Position& position,
Session& session);
/// Forward a formatting request to a stateless worker.
RawResult forward_format(Session& session, std::optional<protocol::Range> range = {});
/// Handle completion requests. Detects preamble context (include/import)
/// and serves those locally; delegates code completion to a stateless worker.
RawResult handle_completion(const protocol::Position& position, Session& session);
@@ -96,7 +103,12 @@ public:
/// Callback invoked when indexing should be scheduled.
std::function<void()> on_indexing_needed;
/// Cancel in-flight compile tasks and wait for them to finish.
kota::task<> stop();
private:
kota::task<> run_compile(std::uint32_t path_id, std::shared_ptr<Session::PendingCompile> pc);
kota::task<bool> ensure_deps(Session& session,
const std::string& directory,
const std::vector<std::string>& arguments,
@@ -125,10 +137,11 @@ private:
private:
kota::event_loop& loop;
kota::ipc::JsonPeer& peer;
kota::ipc::JsonPeer* peer = nullptr;
Workspace& workspace;
WorkerPool& pool;
llvm::DenseMap<std::uint32_t, Session>& sessions;
kota::task_group<> compile_tasks{loop};
};
} // namespace clice

View File

@@ -1,14 +1,15 @@
#include "server/indexer.h"
#include "server/compiler/indexer.h"
#include <algorithm>
#include <string>
#include <variant>
#include <vector>
#include "index/tu_index.h"
#include "server/compiler.h"
#include "server/protocol.h"
#include "server/session.h"
#include "server/worker_pool.h"
#include "server/compiler/compiler.h"
#include "server/protocol/worker.h"
#include "server/service/session.h"
#include "server/worker/worker_pool.h"
#include "support/filesystem.h"
#include "support/logging.h"
@@ -446,6 +447,152 @@ std::optional<SymbolInfo> Indexer::resolve_symbol(index::SymbolHash hash) {
return SymbolInfo{hash, std::move(name), kind, def_loc->uri, def_loc->range};
}
static std::string extract_line(llvm::StringRef content, std::uint32_t offset) {
if(content.empty() || offset >= content.size())
return {};
std::size_t line_start = 0;
if(offset > 0) {
auto pos = content.rfind('\n', offset - 1);
if(pos != llvm::StringRef::npos)
line_start = pos + 1;
}
auto line_end = content.find('\n', offset);
if(line_end == llvm::StringRef::npos)
line_end = content.size();
return content.slice(line_start, line_end).str();
}
std::optional<Indexer::DefinitionText> Indexer::get_definition_text(index::SymbolHash hash) {
for(auto& [id, sess]: sessions) {
if(!sess.file_index || !sess.file_index->mapper)
continue;
auto it = sess.file_index->file_index.relations.find(hash);
if(it == sess.file_index->file_index.relations.end())
continue;
for(auto& rel: it->second) {
if(rel.kind.value() != RelationKind::Definition)
continue;
auto def_range = std::bit_cast<LocalSourceRange>(rel.target_symbol);
if(def_range.begin >= def_range.end)
continue;
llvm::StringRef content = sess.file_index->content;
if(def_range.end > content.size())
continue;
auto start = sess.file_index->mapper->to_position(def_range.begin);
auto end = sess.file_index->mapper->to_position(def_range.end);
if(!start || !end)
continue;
return DefinitionText{
.file = std::string(workspace.path_pool.resolve(id)),
.start_line = static_cast<int>(start->line) + 1,
.end_line = static_cast<int>(end->line) + 1,
.text =
std::string(content.substr(def_range.begin, def_range.end - def_range.begin)),
};
}
}
auto sym_it = workspace.project_index.symbols.find(hash);
if(sym_it == workspace.project_index.symbols.end())
return std::nullopt;
for(auto file_id: sym_it->second.reference_files) {
if(is_proj_path_open(file_id))
continue;
auto shard_it = workspace.merged_indices.find(file_id);
if(shard_it == workspace.merged_indices.end())
continue;
auto* m = shard_it->second.mapper();
if(!m)
continue;
auto content = shard_it->second.index.content();
std::optional<DefinitionText> result;
shard_it->second.index.lookup(
hash,
RelationKind::Definition,
[&](const index::Relation& r) {
auto def_range = std::bit_cast<LocalSourceRange>(r.target_symbol);
if(def_range.begin >= def_range.end || def_range.end > content.size())
return true;
auto start = m->to_position(def_range.begin);
auto end = m->to_position(def_range.end);
if(!start || !end)
return true;
result = DefinitionText{
.file = workspace.project_index.path_pool.path(file_id).str(),
.start_line = static_cast<int>(start->line) + 1,
.end_line = static_cast<int>(end->line) + 1,
.text = std::string(
content.substr(def_range.begin, def_range.end - def_range.begin)),
};
return false;
});
if(result)
return result;
}
return std::nullopt;
}
std::vector<Indexer::ReferenceWithContext> Indexer::collect_references(index::SymbolHash hash,
RelationKind kind) {
std::vector<ReferenceWithContext> results;
auto sym_it = workspace.project_index.symbols.find(hash);
if(sym_it != workspace.project_index.symbols.end()) {
for(auto file_id: sym_it->second.reference_files) {
if(is_proj_path_open(file_id))
continue;
auto shard_it = workspace.merged_indices.find(file_id);
if(shard_it == workspace.merged_indices.end())
continue;
auto* m = shard_it->second.mapper();
if(!m)
continue;
auto content = shard_it->second.index.content();
auto file_path = workspace.project_index.path_pool.path(file_id);
shard_it->second.index.lookup(hash, kind, [&](const index::Relation& r) {
auto start = m->to_position(r.range.begin);
if(!start)
return true;
results.push_back(ReferenceWithContext{
.file = file_path.str(),
.line = static_cast<int>(start->line) + 1,
.context = extract_line(content, r.range.begin),
});
return true;
});
}
}
for(auto& [id, sess]: sessions) {
if(!sess.file_index || !sess.file_index->mapper)
continue;
auto it = sess.file_index->file_index.relations.find(hash);
if(it == sess.file_index->file_index.relations.end())
continue;
auto file_path = workspace.path_pool.resolve(id);
llvm::StringRef content = sess.file_index->content;
for(auto& rel: it->second) {
if(rel.kind != kind)
continue;
auto start = sess.file_index->mapper->to_position(rel.range.begin);
if(!start)
continue;
results.push_back(ReferenceWithContext{
.file = file_path.str(),
.line = static_cast<int>(start->line) + 1,
.context = extract_line(content, rel.range.begin),
});
}
}
return results;
}
std::vector<protocol::CallHierarchyIncomingCall>
Indexer::find_incoming_calls(index::SymbolHash hash) {
llvm::DenseMap<index::SymbolHash, std::vector<protocol::Range>> caller_ranges;
@@ -624,6 +771,28 @@ void Indexer::enqueue(std::uint32_t server_path_id) {
index_queue.push_back(server_path_id);
}
void Indexer::pause_indexing() {
++pause_depth;
if(pause_depth == 1) {
resume_event.reset();
LOG_DEBUG("Background indexing paused");
}
}
void Indexer::resume_indexing() {
if(pause_depth > 0)
--pause_depth;
if(pause_depth == 0) {
resume_event.set();
LOG_DEBUG("Background indexing resumed");
}
}
kota::task<> Indexer::stop() {
bg_tasks.cancel();
co_await bg_tasks.join();
}
void Indexer::schedule() {
if(!*workspace.config.project.enable_indexing || indexing_active || indexing_scheduled)
return;
@@ -633,7 +802,77 @@ void Indexer::schedule() {
index_idle_timer = std::make_shared<kota::timer>(kota::timer::create(loop));
}
index_idle_timer->start(std::chrono::milliseconds(*workspace.config.project.idle_timeout_ms));
loop.schedule(run_background_indexing());
if(!bg_tasks.spawn(run_background_indexing())) {
indexing_scheduled = false;
LOG_WARN("Failed to spawn background indexing task (task group stopped)");
}
}
kota::task<> Indexer::index_one(std::uint32_t server_path_id) {
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
if(sessions.contains(server_path_id))
co_return;
if(!need_update(file_path))
co_return;
// For module interface units, compile their PCM (and transitive deps)
// first so the stateless worker has the artifacts it needs.
if(workspace.compile_graph && workspace.path_to_module.contains(server_path_id)) {
co_await workspace.compile_graph->compile(server_path_id);
}
worker::BuildParams params;
params.kind = worker::BuildKind::Index;
params.file = file_path;
if(!compiler.fill_compile_args(file_path, params.directory, params.arguments, nullptr))
co_return;
workspace.fill_pcm_deps(params.pcms);
LOG_INFO("Background indexing: {}", file_path);
auto result = co_await pool.send_stateless(params);
if(result.has_value() && result.value().success && !result.value().tu_index_data.empty()) {
LOG_INFO("Background indexing got TUIndex for {}: {} bytes",
file_path,
result.value().tu_index_data.size());
merge(result.value().tu_index_data.data(), result.value().tu_index_data.size());
} else if(result.has_value() && !result.value().success) {
LOG_WARN("Background index failed for {}: {}", file_path, result.value().error);
} else if(result.has_value() && result.value().tu_index_data.empty()) {
LOG_WARN("Background index returned empty TUIndex for {}", file_path);
} else {
LOG_WARN("Background index IPC error for {}: {}", file_path, result.error().message);
}
}
kota::task<> Indexer::monitor_resources() {
while(true) {
co_await kota::sleep(std::chrono::milliseconds(3000));
auto mem = kota::sys::memory();
if(mem.total == 0)
continue;
auto effective_total =
(mem.constrained > 0 && mem.constrained < mem.total) ? mem.constrained : mem.total;
auto ratio = static_cast<double>(mem.available) / static_cast<double>(effective_total);
if(ratio < 0.15 && max_concurrent > 1) {
--max_concurrent;
LOG_INFO("Index concurrency -> {} (memory pressure: {:.0f}% available)",
max_concurrent,
ratio * 100);
} else if(ratio > 0.30 && max_concurrent < baseline_concurrent) {
++max_concurrent;
LOG_DEBUG("Index concurrency -> {} (memory OK: {:.0f}% available)",
max_concurrent,
ratio * 100);
}
}
}
kota::task<> Indexer::run_background_indexing() {
@@ -648,48 +887,74 @@ kota::task<> Indexer::run_background_indexing() {
}
indexing_active = true;
std::size_t processed = 0;
while(index_queue_pos < index_queue.size()) {
auto server_path_id = index_queue[index_queue_pos];
index_queue_pos++;
kota::cancellation_source monitor_cancel;
bg_tasks.spawn(kota::with_token(monitor_resources(), monitor_cancel.token()));
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
std::stable_partition(
index_queue.begin() + index_queue_pos,
index_queue.end(),
[this](std::uint32_t id) { return workspace.path_to_module.contains(id); });
if(sessions.contains(server_path_id))
continue;
auto total = index_queue.size() - index_queue_pos;
std::size_t dispatched = 0;
std::size_t completed = 0;
if(!need_update(file_path))
continue;
worker::BuildParams params;
params.kind = worker::BuildKind::Index;
params.file = file_path;
if(!compiler.fill_compile_args(file_path, params.directory, params.arguments, nullptr))
continue;
workspace.fill_pcm_deps(params.pcms);
LOG_INFO("Background indexing: {}", file_path);
auto result = co_await pool.send_stateless(params);
if(result.has_value() && result.value().success && !result.value().tu_index_data.empty()) {
LOG_INFO("Background indexing got TUIndex for {}: {} bytes",
file_path,
result.value().tu_index_data.size());
merge(result.value().tu_index_data.data(), result.value().tu_index_data.size());
++processed;
} else if(result.has_value() && !result.value().success) {
LOG_WARN("Background index failed for {}: {}", file_path, result.value().error);
} else if(result.has_value() && result.value().tu_index_data.empty()) {
LOG_WARN("Background index returned empty TUIndex for {}", file_path);
std::optional<lsp::ProgressReporter<kota::ipc::JsonPeer>> progress;
if(peer) {
progress.emplace(*peer, protocol::ProgressToken(std::string("clice/backgroundIndex")));
auto create_result = co_await progress->create();
if(!create_result.has_error()) {
progress->begin("Indexing", std::format("0/{} files", total), 0);
} else {
LOG_WARN("Background index IPC error for {}: {}", file_path, result.error().message);
progress.reset();
}
}
kota::task_group<> workers(loop);
std::size_t in_flight = 0;
kota::event slot_available;
while(index_queue_pos < index_queue.size()) {
if(pause_depth > 0)
co_await resume_event.wait();
auto server_path_id = index_queue[index_queue_pos++];
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
if(sessions.contains(server_path_id) || !need_update(file_path)) {
++completed;
continue;
}
while(in_flight >= max_concurrent) {
slot_available.reset();
co_await slot_available.wait();
}
++in_flight;
++dispatched;
workers.spawn([&, server_path_id]() -> kota::task<> {
co_await index_one(server_path_id);
--in_flight;
++completed;
if(progress) {
auto pct = total > 0 ? static_cast<std::uint32_t>(completed * 100 / total) : 100;
progress->report(std::format("{}/{} files", completed, total), pct);
}
slot_available.set();
}());
}
co_await workers.join();
if(progress) {
progress->end(std::format("Indexed {} files", dispatched));
}
monitor_cancel.cancel();
indexing_active = false;
LOG_INFO("Background indexing complete: {} files processed", processed);
LOG_INFO("Background indexing complete: {} files dispatched", dispatched);
save(workspace.config.project.index_dir);
}

View File

@@ -9,10 +9,12 @@
#include "semantic/relation_kind.h"
#include "semantic/symbol_kind.h"
#include "server/workspace.h"
#include "server/workspace/workspace.h"
#include "kota/async/async.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/lsp/position.h"
#include "kota/ipc/lsp/progress.h"
#include "kota/ipc/lsp/protocol.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/SmallVector.h"
@@ -59,8 +61,49 @@ public:
WorkerPool& pool,
Compiler& compiler,
std::function<bool(std::uint32_t)> is_file_open = {}) :
loop(loop), workspace(workspace), sessions(sessions), pool(pool), compiler(compiler),
is_file_open(std::move(is_file_open)) {}
loop(loop), bg_tasks(loop), workspace(workspace), sessions(sessions), pool(pool),
compiler(compiler), is_file_open(std::move(is_file_open)) {}
/// Set the LSP peer for progress reporting. Must be called before
/// schedule() if progress notifications are desired.
void set_peer(kota::ipc::JsonPeer* p) {
peer = p;
}
/// Temporarily pause background indexing to give priority to user
/// requests. Indexing tasks already dispatched to workers continue,
/// but no new tasks will be sent until resume_indexing() is called.
void pause_indexing();
/// Resume background indexing after a pause.
void resume_indexing();
/// RAII guard that pauses indexing for its lifetime.
struct [[nodiscard]] ScopedPause {
Indexer& indexer;
explicit ScopedPause(Indexer& idx) : indexer(idx) {
indexer.pause_indexing();
}
~ScopedPause() {
indexer.resume_indexing();
}
ScopedPause(const ScopedPause&) = delete;
ScopedPause& operator=(const ScopedPause&) = delete;
};
ScopedPause scoped_pause() {
return ScopedPause{*this};
}
/// Set the maximum number of concurrent index tasks.
/// Also sets the baseline that dynamic adjustment will restore to.
void set_max_concurrency(std::size_t n) {
max_concurrent = std::max<std::size_t>(n, 1);
baseline_concurrent = max_concurrent;
}
/// Add a file to the background indexing queue.
void enqueue(std::uint32_t server_path_id);
@@ -124,6 +167,43 @@ public:
std::vector<protocol::SymbolInformation> search_symbols(llvm::StringRef query,
std::size_t max_results = 100);
struct DefinitionText {
std::string file;
int start_line;
int end_line;
std::string text;
};
/// Get full definition text for a symbol, using stored index ranges and content.
std::optional<DefinitionText> get_definition_text(index::SymbolHash hash);
struct ReferenceWithContext {
std::string file;
int line;
std::string context;
};
/// Collect references (or definitions) with context lines from stored content.
std::vector<ReferenceWithContext> collect_references(index::SymbolHash hash, RelationKind kind);
/// Cancel background indexing and wait for all tasks to settle.
kota::task<> stop();
/// Whether background indexing is currently idle (no active or queued work).
bool is_idle() const {
return !indexing_active && index_queue_pos >= index_queue.size();
}
/// Number of files remaining in the indexing queue.
std::size_t pending_files() const {
return index_queue_pos < index_queue.size() ? index_queue.size() - index_queue_pos : 0;
}
/// Total files that were enqueued in the current (or last) indexing round.
std::size_t total_queued() const {
return index_queue.size();
}
/// Convert internal SymbolKind to LSP SymbolKind.
static protocol::SymbolKind to_lsp_symbol_kind(SymbolKind kind);
@@ -165,6 +245,7 @@ private:
private:
kota::event_loop& loop;
kota::task_group<> bg_tasks;
Workspace& workspace;
llvm::DenseMap<std::uint32_t, Session>& sessions;
WorkerPool& pool;
@@ -175,6 +256,9 @@ private:
/// server-path-id-keyed sessions map to project-level path_ids.
std::function<bool(std::uint32_t)> is_file_open;
/// LSP peer for progress reporting (optional, not owned).
kota::ipc::JsonPeer* peer = nullptr;
/// Background indexing queue and scheduling state.
std::vector<std::uint32_t> index_queue;
std::size_t index_queue_pos = 0;
@@ -182,7 +266,18 @@ private:
bool indexing_scheduled = false;
std::shared_ptr<kota::timer> index_idle_timer;
/// Concurrency control for background indexing.
std::size_t max_concurrent = 2;
std::size_t baseline_concurrent = 2;
/// Pause/resume: when paused, new index tasks wait on this event.
/// Uses a counter so nested pause/resume pairs work correctly.
std::size_t pause_depth = 0;
kota::event resume_event{true};
kota::task<> run_background_indexing();
kota::task<> index_one(std::uint32_t server_path_id);
kota::task<> monitor_resources();
};
} // namespace clice

View File

@@ -1,81 +0,0 @@
#pragma once
#include <cstdint>
#include <memory>
#include <string>
#include <vector>
#include "server/compiler.h"
#include "server/indexer.h"
#include "server/session.h"
#include "server/worker_pool.h"
#include "server/workspace.h"
#include "kota/async/async.h"
#include "kota/codec/raw_value.h"
#include "kota/ipc/peer.h"
#include "llvm/ADT/DenseMap.h"
namespace clice {
enum class ServerLifecycle : std::uint8_t {
Uninitialized,
Initialized,
Ready,
ShuttingDown,
Exited,
};
/// Top-level LSP server — the single orchestration point for the language
/// server process.
///
/// Responsibilities:
/// - Owns the two-layer state model: Workspace (disk truth) and Sessions
/// (per-open-file volatile state).
/// - Manages Session lifecycle directly: didOpen creates, didChange mutates,
/// didSave syncs to Workspace, didClose destroys.
/// - Dispatches compilation and feature queries to Compiler.
/// - Dispatches index lookups and background indexing to Indexer.
///
/// Design principle:
/// Open files are never depended upon by other files. Dependencies always
/// point to disk files. The only path from Session to Workspace is didSave.
class MasterServer {
public:
MasterServer(kota::event_loop& loop, kota::ipc::JsonPeer& peer, std::string self_path);
~MasterServer();
void register_handlers();
private:
kota::event_loop& loop;
kota::ipc::JsonPeer& peer;
/// Persistent project-wide state (config, CDB, path pool, dependency
/// graphs, compilation caches, symbol index).
Workspace workspace;
/// Per-file editing sessions, keyed by server-level path_id.
llvm::DenseMap<std::uint32_t, Session> sessions;
/// Worker process pool for offloading compilation and queries.
WorkerPool pool;
/// Compilation lifecycle manager (reads/writes workspace and sessions).
Compiler compiler;
/// Index query and background scheduling (reads from workspace and sessions).
Indexer indexer;
ServerLifecycle lifecycle = ServerLifecycle::Uninitialized;
std::string self_path;
std::string workspace_root;
std::string session_log_dir;
std::string init_options_json; ///< Raw JSON from initializationOptions, consumed once.
kota::task<> load_workspace();
using RawResult = kota::task<kota::codec::RawValue, kota::ipc::Error>;
};
} // namespace clice

View File

@@ -0,0 +1,311 @@
#pragma once
#include <cstdint>
#include <optional>
#include <string>
#include <vector>
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/protocol.h"
namespace clice::agentic {
struct CompileCommandParams {
std::string path;
};
struct CompileCommandResult {
std::string file;
std::string directory;
std::vector<std::string> arguments;
};
struct FileInfo {
std::string path;
std::string kind;
std::optional<std::string> module_name;
};
struct ProjectFilesParams {
std::optional<std::string> filter;
};
struct ProjectFilesResult {
std::vector<FileInfo> files;
int total = 0;
};
struct DepEntry {
std::string path;
int depth = 0;
};
struct FileDepsParams {
std::string path;
std::optional<std::string> direction;
std::optional<int> depth;
};
struct FileDepsResult {
std::string file;
std::vector<DepEntry> includes;
std::vector<DepEntry> includers;
};
struct ImpactAnalysisParams {
std::string path;
};
struct ImpactAnalysisResult {
std::vector<std::string> direct_dependents;
std::vector<std::string> transitive_dependents;
std::vector<std::string> affected_modules;
};
struct SymbolEntry {
std::string name;
std::string kind;
std::string file;
int line = 0;
std::optional<std::string> container;
std::uint64_t symbol_id = 0;
};
struct SymbolSearchParams {
std::string query;
std::optional<std::vector<std::string>> kind_filter;
std::optional<int> max_results;
};
struct SymbolSearchResult {
std::vector<SymbolEntry> symbols;
};
struct ReadSymbolParams {
std::optional<std::string> name;
std::optional<std::string> path;
std::optional<int> line;
std::optional<std::uint64_t> symbol_id;
};
struct ReadSymbolResult {
std::string name;
std::string kind;
std::string file;
int start_line = 0;
int end_line = 0;
std::string text;
std::optional<std::string> signature;
std::uint64_t symbol_id = 0;
};
struct DocumentSymbolEntry {
std::string name;
std::string kind;
int start_line = 0;
int end_line = 0;
std::uint64_t symbol_id = 0;
};
struct DocumentSymbolsParams {
std::string path;
};
struct DocumentSymbolsResult {
std::vector<DocumentSymbolEntry> symbols;
};
struct DefinitionParams {
std::optional<std::string> name;
std::optional<std::string> path;
std::optional<int> line;
std::optional<std::uint64_t> symbol_id;
};
struct LocationEntry {
std::string file;
int start_line = 0;
int end_line = 0;
std::string text;
};
struct DefinitionResult {
std::string name;
std::string kind;
std::uint64_t symbol_id = 0;
std::optional<LocationEntry> definition;
};
struct ReferenceEntry {
std::string file;
int line = 0;
std::string context;
};
struct ReferencesParams {
std::optional<std::string> name;
std::optional<std::string> path;
std::optional<int> line;
std::optional<std::uint64_t> symbol_id;
std::optional<bool> include_declaration;
};
struct ReferencesResult {
std::string name;
std::string kind;
std::uint64_t symbol_id = 0;
std::vector<ReferenceEntry> references;
int total = 0;
};
struct CallGraphEntry {
std::string name;
std::string kind;
std::string file;
int line = 0;
std::uint64_t symbol_id = 0;
};
struct CallGraphParams {
std::optional<std::string> name;
std::optional<std::string> path;
std::optional<int> line;
std::optional<std::uint64_t> symbol_id;
std::optional<std::string> direction;
std::optional<int> depth;
};
struct CallGraphResult {
CallGraphEntry root;
std::vector<CallGraphEntry> callers;
std::vector<CallGraphEntry> callees;
};
struct TypeHierarchyEntry {
std::string name;
std::string kind;
std::string file;
int line = 0;
std::uint64_t symbol_id = 0;
};
struct TypeHierarchyParams {
std::optional<std::string> name;
std::optional<std::string> path;
std::optional<int> line;
std::optional<std::uint64_t> symbol_id;
std::optional<std::string> direction;
};
struct TypeHierarchyResult {
TypeHierarchyEntry root;
std::vector<TypeHierarchyEntry> supertypes;
std::vector<TypeHierarchyEntry> subtypes;
};
struct LintParams {
std::string path;
std::optional<int> line;
};
using LintResult = std::vector<kota::ipc::protocol::Diagnostic>;
struct StatusParams {};
struct StatusResult {
bool idle = true;
int pending = 0;
int total = 0;
int indexed = 0;
};
struct ShutdownParams {};
} // namespace clice::agentic
namespace kota::ipc::protocol {
template <>
struct RequestTraits<clice::agentic::CompileCommandParams> {
using Result = clice::agentic::CompileCommandResult;
constexpr inline static std::string_view method = "agentic/compileCommand";
};
template <>
struct RequestTraits<clice::agentic::ProjectFilesParams> {
using Result = clice::agentic::ProjectFilesResult;
constexpr inline static std::string_view method = "agentic/projectFiles";
};
template <>
struct RequestTraits<clice::agentic::FileDepsParams> {
using Result = clice::agentic::FileDepsResult;
constexpr inline static std::string_view method = "agentic/fileDeps";
};
template <>
struct RequestTraits<clice::agentic::ImpactAnalysisParams> {
using Result = clice::agentic::ImpactAnalysisResult;
constexpr inline static std::string_view method = "agentic/impactAnalysis";
};
template <>
struct RequestTraits<clice::agentic::SymbolSearchParams> {
using Result = clice::agentic::SymbolSearchResult;
constexpr inline static std::string_view method = "agentic/symbolSearch";
};
template <>
struct RequestTraits<clice::agentic::ReadSymbolParams> {
using Result = clice::agentic::ReadSymbolResult;
constexpr inline static std::string_view method = "agentic/readSymbol";
};
template <>
struct RequestTraits<clice::agentic::DocumentSymbolsParams> {
using Result = clice::agentic::DocumentSymbolsResult;
constexpr inline static std::string_view method = "agentic/documentSymbols";
};
template <>
struct RequestTraits<clice::agentic::DefinitionParams> {
using Result = clice::agentic::DefinitionResult;
constexpr inline static std::string_view method = "agentic/definition";
};
template <>
struct RequestTraits<clice::agentic::ReferencesParams> {
using Result = clice::agentic::ReferencesResult;
constexpr inline static std::string_view method = "agentic/references";
};
template <>
struct RequestTraits<clice::agentic::CallGraphParams> {
using Result = clice::agentic::CallGraphResult;
constexpr inline static std::string_view method = "agentic/callGraph";
};
template <>
struct RequestTraits<clice::agentic::TypeHierarchyParams> {
using Result = clice::agentic::TypeHierarchyResult;
constexpr inline static std::string_view method = "agentic/typeHierarchy";
};
template <>
struct RequestTraits<clice::agentic::LintParams> {
using Result = clice::agentic::LintResult;
constexpr inline static std::string_view method = "agentic/lint";
};
template <>
struct RequestTraits<clice::agentic::StatusParams> {
using Result = clice::agentic::StatusResult;
constexpr inline static std::string_view method = "agentic/status";
};
template <>
struct NotificationTraits<clice::agentic::ShutdownParams> {
constexpr inline static std::string_view method = "agentic/shutdown";
};
} // namespace kota::ipc::protocol

View File

@@ -0,0 +1,42 @@
#pragma once
#include <optional>
#include <string>
#include <vector>
namespace clice::ext {
struct ContextItem {
std::string label;
std::string description;
std::string uri;
};
struct QueryContextParams {
std::string uri;
std::optional<int> offset;
};
struct QueryContextResult {
std::vector<ContextItem> contexts;
int total = 0;
};
struct CurrentContextParams {
std::string uri;
};
struct CurrentContextResult {
std::optional<ContextItem> context;
};
struct SwitchContextParams {
std::string uri;
std::string context_uri;
};
struct SwitchContextResult {
bool success = false;
};
} // namespace clice::ext

View File

@@ -1,7 +1,6 @@
#pragma once
#include <cstdint>
#include <optional>
#include <string>
#include <unordered_map>
#include <utility>
@@ -9,8 +8,7 @@
#include "syntax/token.h"
#include "kota/codec/raw_value.h"
#include "kota/ipc/lsp/protocol.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/protocol.h"
namespace clice::worker {
@@ -45,6 +43,7 @@ struct CompileParams {
std::string text;
std::string directory;
std::vector<std::string> arguments;
bool clang_tidy = false;
std::pair<std::string, uint32_t> pch;
std::unordered_map<std::string, std::string> pcms;
};
@@ -66,6 +65,7 @@ enum class BuildKind : uint8_t {
Index,
Completion,
SignatureHelp,
Format,
};
/// Unified parameters for all stateless build/compilation tasks.
@@ -76,6 +76,7 @@ enum class BuildKind : uint8_t {
/// - Index: + pcms
/// - Completion: + text, version, offset, pch, pcms
/// - SignatureHelp: + text, version, offset, pch, pcms
/// - Format: + text, format_range (optional)
struct BuildParams {
BuildKind kind;
std::string file;
@@ -92,6 +93,7 @@ struct BuildParams {
std::string output_path; ///< BuildPCH, BuildPCM
std::string module_name; ///< BuildPCM
uint32_t preamble_bound = UINT32_MAX; ///< BuildPCH
LocalSourceRange format_range; ///< Format (default = full document)
};
/// Unified result for stateless build tasks.
@@ -122,43 +124,6 @@ struct EvictedParams {
} // namespace clice::worker
namespace clice::ext {
struct ContextItem {
std::string label;
std::string description;
std::string uri;
};
struct QueryContextParams {
std::string uri;
std::optional<int> offset;
};
struct QueryContextResult {
std::vector<ContextItem> contexts;
int total;
};
struct CurrentContextParams {
std::string uri;
};
struct CurrentContextResult {
std::optional<ContextItem> context;
};
struct SwitchContextParams {
std::string uri;
std::string context_uri;
};
struct SwitchContextResult {
bool success;
};
} // namespace clice::ext
namespace kota::ipc::protocol {
template <>

View File

@@ -0,0 +1,820 @@
#include "server/service/agent_client.h"
#include <algorithm>
#include <format>
#include <ranges>
#include <string>
#include <vector>
#include "compile/compilation.h"
#include "feature/feature.h"
#include "server/protocol/agentic.h"
#include "server/service/master_server.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "kota/async/async.h"
#include "kota/ipc/lsp/uri.h"
#include "kota/meta/enum.h"
#include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/SmallVector.h"
namespace clice {
using kota::ipc::RequestResult;
using RequestContext = kota::ipc::JsonPeer::RequestContext;
namespace lsp = kota::ipc::lsp;
namespace protocol = kota::ipc::protocol;
static std::string_view symbol_kind_name(SymbolKind kind) {
constexpr auto names = kota::meta::reflection<SymbolKind::Kind>::member_names;
auto idx = static_cast<std::size_t>(kind.value());
if(idx < names.size())
return names[idx];
return "Unknown";
}
struct ResolvedSymbol {
index::SymbolHash hash = 0;
std::string name;
SymbolKind kind;
std::string file;
int line = 0;
};
static std::vector<ResolvedSymbol> resolve_locator(const agentic::ReadSymbolParams& loc,
Workspace& workspace,
llvm::DenseMap<std::uint32_t, Session>& sessions,
Indexer& indexer) {
if(loc.symbol_id.has_value() && *loc.symbol_id != 0) {
auto hash = static_cast<index::SymbolHash>(*loc.symbol_id);
std::string name;
SymbolKind kind;
if(!indexer.find_symbol_info(hash, name, kind))
return {};
auto def_loc = indexer.find_definition_location(hash);
if(!def_loc)
return {};
auto file = uri_to_path(def_loc->uri);
int line_num = static_cast<int>(def_loc->range.start.line) + 1;
return {
{hash, std::move(name), kind, std::move(file), line_num}
};
}
if(loc.name.has_value() && !loc.name->empty()) {
std::string query_lower = llvm::StringRef(*loc.name).lower();
std::vector<ResolvedSymbol> candidates;
std::vector<ResolvedSymbol> exact_matches;
llvm::DenseSet<index::SymbolHash> seen;
auto try_symbol = [&](index::SymbolHash hash, const index::Symbol& symbol) {
if(symbol.name.empty())
return;
if(llvm::StringRef(symbol.name).lower().find(query_lower) == std::string::npos)
return;
auto def_loc = indexer.find_definition_location(hash);
if(!def_loc)
return;
if(!seen.insert(hash).second)
return;
auto file = uri_to_path(def_loc->uri);
int line_num = static_cast<int>(def_loc->range.start.line) + 1;
if(loc.path.has_value() && !loc.path->empty()) {
llvm::StringRef wanted(*loc.path);
bool basename_only = wanted.find_last_of("/\\") == llvm::StringRef::npos;
if(basename_only) {
if(llvm::sys::path::filename(file) != wanted)
return;
} else if(!llvm::StringRef(file).ends_with(wanted)) {
return;
}
}
bool is_exact = llvm::StringRef(symbol.name).lower() == query_lower ||
llvm::StringRef(symbol.name).ends_with("::" + *loc.name);
ResolvedSymbol rs{hash, symbol.name, symbol.kind, std::move(file), line_num};
if(is_exact)
exact_matches.push_back(std::move(rs));
else
candidates.push_back(std::move(rs));
};
for(auto& [hash, symbol]: workspace.project_index.symbols)
try_symbol(hash, symbol);
for(auto& [_, sess]: sessions) {
if(!sess.file_index)
continue;
for(auto& [hash, symbol]: sess.file_index->symbols)
try_symbol(hash, symbol);
}
if(!exact_matches.empty())
return exact_matches;
return candidates;
}
if(loc.path.has_value() && loc.line.has_value()) {
auto path_str = *loc.path;
auto target_line = static_cast<protocol::uinteger>(*loc.line - 1);
auto pool_it = workspace.path_pool.cache.find(path_str);
auto server_id = pool_it != workspace.path_pool.cache.end() ? pool_it->second : ~0u;
auto* sess =
server_id != ~0u && sessions.contains(server_id) ? &sessions[server_id] : nullptr;
if(sess && sess->file_index) {
auto& fi = *sess->file_index;
if(fi.mapper) {
for(auto& [hash, rels]: fi.file_index.relations) {
for(auto& rel: rels) {
if(rel.kind.value() != RelationKind::Definition)
continue;
auto start = fi.mapper->to_position(rel.range.begin);
if(start && start->line == target_line) {
std::string name;
SymbolKind kind;
if(indexer.find_symbol_info(hash, name, kind))
return {
{hash, std::move(name), kind, path_str, *loc.line}
};
}
}
}
}
}
auto it = workspace.project_index.path_pool.find(path_str);
if(it == workspace.project_index.path_pool.cache.end())
return {};
auto proj_id = it->second;
auto shard_it = workspace.merged_indices.find(proj_id);
if(shard_it == workspace.merged_indices.end())
return {};
for(auto& [hash, symbol]: workspace.project_index.symbols) {
if(!symbol.reference_files.contains(proj_id))
continue;
bool found = false;
shard_it->second.find_relations(hash,
RelationKind::Definition,
[&](const index::Relation&, protocol::Range range) {
if(range.start.line == target_line) {
found = true;
return false;
}
return true;
});
if(found)
return {
{hash, symbol.name, symbol.kind, path_str, *loc.line}
};
}
return {};
}
return {};
}
static std::uint64_t extract_symbol_id(const std::optional<protocol::LSPAny>& data) {
if(!data.has_value())
return 0;
if(auto* val = std::get_if<std::int64_t>(&static_cast<const protocol::LSPVariant&>(*data)))
return static_cast<std::uint64_t>(*val);
LOG_WARN("extract_symbol_id: unexpected LSPAny variant type");
return 0;
}
AgentClient::AgentClient(MasterServer& server, kota::ipc::JsonPeer& peer) :
server(server), peer(peer) {
using namespace agentic;
auto& srv = this->server;
peer.on_request(
[&srv](RequestContext&,
const CompileCommandParams& params) -> RequestResult<CompileCommandParams> {
std::string directory;
std::vector<std::string> arguments;
if(!srv.compiler.fill_compile_args(params.path, directory, arguments)) {
co_return kota::outcome_error(
kota::ipc::Error{std::format("no compile command found for {}", params.path)});
}
co_return CompileCommandResult{
.file = params.path,
.directory = std::move(directory),
.arguments = std::move(arguments),
};
});
peer.on_request([&srv](RequestContext&,
const ProjectFilesParams& params) -> RequestResult<ProjectFilesParams> {
auto& ws = srv.workspace;
auto filter = params.filter.value_or("all");
ProjectFilesResult result;
llvm::DenseSet<std::uint32_t> seen;
for(auto& entry: ws.cdb.get_entries()) {
auto file_path = ws.cdb.resolve_path(entry.file);
if(file_path.empty())
continue;
auto proj_it = ws.project_index.path_pool.find(file_path);
if(proj_it != ws.project_index.path_pool.cache.end()) {
if(!seen.insert(proj_it->second).second)
continue;
}
std::string kind_str;
auto mod_it = ws.path_to_module.find(ws.path_pool.intern(file_path));
if(mod_it != ws.path_to_module.end()) {
kind_str = "module";
} else {
auto ext = llvm::sys::path::extension(file_path);
if(ext == ".h" || ext == ".hpp" || ext == ".hxx" || ext == ".hh")
kind_str = "header";
else
kind_str = "source";
}
if(filter != "all" && filter != kind_str)
continue;
FileInfo fi;
fi.path = file_path.str();
fi.kind = std::move(kind_str);
if(mod_it != ws.path_to_module.end())
fi.module_name = mod_it->second;
result.files.push_back(std::move(fi));
}
if(filter == "all" || filter == "header") {
for(auto& [path_id, shard]: ws.merged_indices) {
if(seen.contains(path_id))
continue;
auto path_str = ws.project_index.path_pool.path(path_id);
auto ext = llvm::sys::path::extension(path_str);
if(ext == ".h" || ext == ".hpp" || ext == ".hxx" || ext == ".hh") {
seen.insert(path_id);
result.files.push_back(FileInfo{
.path = path_str.str(),
.kind = "header",
});
}
}
}
result.total = static_cast<int>(result.files.size());
co_return result;
});
peer.on_request(
[&srv](RequestContext&, const FileDepsParams& params) -> RequestResult<FileDepsParams> {
auto& ws = srv.workspace;
auto pool_it = ws.path_pool.cache.find(params.path);
if(pool_it == ws.path_pool.cache.end())
co_return FileDepsResult{.file = params.path};
auto path_id = pool_it->second;
auto direction = params.direction.value_or("both");
auto max_depth = params.depth.value_or(1);
FileDepsResult result;
result.file = params.path;
if(direction == "includes" || direction == "both") {
auto includes = ws.dep_graph.get_all_includes(path_id);
for(auto inc_id: includes) {
auto real_id = inc_id & DependencyGraph::PATH_ID_MASK;
auto inc_path = ws.path_pool.resolve(real_id);
result.includes.push_back(DepEntry{.path = inc_path.str(), .depth = 1});
}
if(max_depth == 0 || max_depth > 1) {
llvm::DenseSet<std::uint32_t> visited;
visited.insert(path_id);
for(auto& dep: result.includes)
visited.insert(ws.path_pool.intern(dep.path));
for(std::size_t i = 0; i < result.includes.size(); ++i) {
if(max_depth > 0 && result.includes[i].depth >= max_depth)
continue;
auto dep_id = ws.path_pool.intern(result.includes[i].path);
auto sub = ws.dep_graph.get_all_includes(dep_id);
for(auto sub_id: sub) {
auto real_id = sub_id & DependencyGraph::PATH_ID_MASK;
if(!visited.insert(real_id).second)
continue;
auto sub_path = ws.path_pool.resolve(real_id);
result.includes.push_back(DepEntry{
.path = sub_path.str(),
.depth = result.includes[i].depth + 1,
});
}
}
}
}
if(direction == "includers" || direction == "both") {
auto includers = ws.dep_graph.get_includers(path_id);
for(auto inc_id: includers) {
auto inc_path = ws.path_pool.resolve(inc_id);
result.includers.push_back(DepEntry{.path = inc_path.str(), .depth = 1});
}
if(max_depth == 0 || max_depth > 1) {
llvm::DenseSet<std::uint32_t> visited;
visited.insert(path_id);
for(auto& dep: result.includers) {
auto it = ws.path_pool.cache.find(dep.path);
if(it != ws.path_pool.cache.end())
visited.insert(it->second);
}
for(std::size_t i = 0; i < result.includers.size(); ++i) {
if(max_depth > 0 && result.includers[i].depth >= max_depth)
continue;
auto dep_it = ws.path_pool.cache.find(result.includers[i].path);
if(dep_it == ws.path_pool.cache.end())
continue;
auto sub = ws.dep_graph.get_includers(dep_it->second);
for(auto sub_id: sub) {
if(!visited.insert(sub_id).second)
continue;
auto sub_path = ws.path_pool.resolve(sub_id);
result.includers.push_back(DepEntry{
.path = sub_path.str(),
.depth = result.includers[i].depth + 1,
});
}
}
}
}
co_return result;
});
peer.on_request(
[&srv](RequestContext&,
const ImpactAnalysisParams& params) -> RequestResult<ImpactAnalysisParams> {
auto& ws = srv.workspace;
auto pool_it = ws.path_pool.cache.find(params.path);
if(pool_it == ws.path_pool.cache.end())
co_return ImpactAnalysisResult{};
auto path_id = pool_it->second;
ImpactAnalysisResult result;
auto direct_includers = ws.dep_graph.get_includers(path_id);
for(auto inc_id: direct_includers) {
result.direct_dependents.push_back(ws.path_pool.resolve(inc_id).str());
}
auto hosts = ws.dep_graph.find_host_sources(path_id);
llvm::DenseSet<std::uint32_t> seen;
seen.insert(path_id);
for(auto inc_id: direct_includers)
seen.insert(inc_id);
for(auto host_id: hosts) {
if(seen.insert(host_id).second)
result.transitive_dependents.push_back(ws.path_pool.resolve(host_id).str());
}
for(auto host_id: hosts) {
auto it = ws.path_to_module.find(host_id);
if(it != ws.path_to_module.end())
result.affected_modules.push_back(it->second);
}
auto mod_it = ws.path_to_module.find(path_id);
if(mod_it != ws.path_to_module.end())
result.affected_modules.push_back(mod_it->second);
co_return result;
});
peer.on_request([&srv](RequestContext&,
const SymbolSearchParams& params) -> RequestResult<SymbolSearchParams> {
auto max = params.max_results.value_or(100);
std::string query_lower = llvm::StringRef(params.query).lower();
SymbolSearchResult result;
llvm::DenseSet<index::SymbolHash> seen;
auto try_symbol = [&](index::SymbolHash hash, const index::Symbol& symbol) {
if(static_cast<int>(result.symbols.size()) >= max)
return;
if(symbol.name.empty())
return;
if(!query_lower.empty() &&
llvm::StringRef(symbol.name).lower().find(query_lower) == std::string::npos)
return;
if(params.kind_filter.has_value()) {
auto kind_name = std::string(symbol_kind_name(symbol.kind));
auto& filter = *params.kind_filter;
if(std::ranges::find(filter, kind_name) == filter.end())
return;
}
auto def_loc = srv.indexer.find_definition_location(hash);
if(!def_loc)
return;
if(!seen.insert(hash).second)
return;
auto file = uri_to_path(def_loc->uri);
result.symbols.push_back(SymbolEntry{
.name = symbol.name,
.kind = std::string(symbol_kind_name(symbol.kind)),
.file = std::move(file),
.line = static_cast<int>(def_loc->range.start.line) + 1,
.symbol_id = hash,
});
};
for(auto& [hash, symbol]: srv.workspace.project_index.symbols)
try_symbol(hash, symbol);
for(auto& [_, sess]: srv.sessions) {
if(!sess.file_index)
continue;
for(auto& [hash, symbol]: sess.file_index->symbols)
try_symbol(hash, symbol);
}
co_return result;
});
peer.on_request(
[&srv](RequestContext&, const ReadSymbolParams& params) -> RequestResult<ReadSymbolParams> {
auto candidates = resolve_locator(params, srv.workspace, srv.sessions, srv.indexer);
if(candidates.empty())
co_return kota::outcome_error(kota::ipc::Error{"symbol not found"});
if(candidates.size() > 1) {
co_return kota::outcome_error(kota::ipc::Error{
std::format("ambiguous: {} candidates, use symbolId to disambiguate",
candidates.size())});
}
auto& rs = candidates[0];
auto def_text = srv.indexer.get_definition_text(rs.hash);
if(!def_text)
co_return kota::outcome_error(kota::ipc::Error{"definition not found"});
co_return ReadSymbolResult{
.name = rs.name,
.kind = std::string(symbol_kind_name(rs.kind)),
.file = std::move(def_text->file),
.start_line = def_text->start_line,
.end_line = def_text->end_line,
.text = std::move(def_text->text),
.symbol_id = rs.hash,
};
});
peer.on_request(
[&srv](RequestContext&,
const DocumentSymbolsParams& params) -> RequestResult<DocumentSymbolsParams> {
auto is_document_level = [](SymbolKind kind) {
return kind == SymbolKind::Namespace || kind == SymbolKind::Class ||
kind == SymbolKind::Struct || kind == SymbolKind::Union ||
kind == SymbolKind::Enum || kind == SymbolKind::Type ||
kind == SymbolKind::Field || kind == SymbolKind::EnumMember ||
kind == SymbolKind::Function || kind == SymbolKind::Method ||
kind == SymbolKind::Variable || kind == SymbolKind::Macro ||
kind == SymbolKind::Concept || kind == SymbolKind::Module ||
kind == SymbolKind::Operator || kind == SymbolKind::Attribute;
};
DocumentSymbolsResult result;
auto pool_it = srv.workspace.path_pool.cache.find(params.path);
if(pool_it == srv.workspace.path_pool.cache.end())
co_return result;
auto server_id = pool_it->second;
auto sess_it = srv.sessions.find(server_id);
if(sess_it != srv.sessions.end() && sess_it->second.file_index) {
auto& fi = *sess_it->second.file_index;
for(auto& [hash, rels]: fi.file_index.relations) {
for(auto& rel: rels) {
if(rel.kind.value() != RelationKind::Definition)
continue;
std::string name;
SymbolKind kind;
if(!srv.indexer.find_symbol_info(hash, name, kind))
continue;
if(!is_document_level(kind))
continue;
if(fi.mapper) {
auto start = fi.mapper->to_position(rel.range.begin);
auto end = fi.mapper->to_position(rel.range.end);
if(start && end) {
result.symbols.push_back(DocumentSymbolEntry{
.name = std::move(name),
.kind = std::string(symbol_kind_name(kind)),
.start_line = static_cast<int>(start->line) + 1,
.end_line = static_cast<int>(end->line) + 1,
.symbol_id = hash,
});
break;
}
}
}
}
co_return result;
}
auto it = srv.workspace.project_index.path_pool.find(params.path);
if(it == srv.workspace.project_index.path_pool.cache.end())
co_return result;
auto proj_id = it->second;
auto shard_it = srv.workspace.merged_indices.find(proj_id);
if(shard_it == srv.workspace.merged_indices.end())
co_return result;
for(auto& [hash, symbol]: srv.workspace.project_index.symbols) {
if(symbol.name.empty())
continue;
if(!is_document_level(symbol.kind))
continue;
if(!symbol.reference_files.contains(proj_id))
continue;
shard_it->second.find_relations(
hash,
RelationKind::Definition,
[&](const index::Relation&, protocol::Range range) {
result.symbols.push_back(DocumentSymbolEntry{
.name = symbol.name,
.kind = std::string(symbol_kind_name(symbol.kind)),
.start_line = static_cast<int>(range.start.line) + 1,
.end_line = static_cast<int>(range.end.line) + 1,
.symbol_id = hash,
});
return true;
});
}
co_return result;
});
peer.on_request(
[&srv](RequestContext&, const DefinitionParams& params) -> RequestResult<DefinitionParams> {
auto candidates = resolve_locator(
ReadSymbolParams{params.name, params.path, params.line, params.symbol_id},
srv.workspace,
srv.sessions,
srv.indexer);
if(candidates.empty())
co_return kota::outcome_error(kota::ipc::Error{"symbol not found"});
if(candidates.size() > 1) {
co_return kota::outcome_error(kota::ipc::Error{
std::format("ambiguous: {} candidates, use symbolId to disambiguate",
candidates.size())});
}
auto& rs = candidates[0];
DefinitionResult result;
result.name = rs.name;
result.kind = std::string(symbol_kind_name(rs.kind));
result.symbol_id = rs.hash;
if(auto def_text = srv.indexer.get_definition_text(rs.hash)) {
result.definition = LocationEntry{
.file = std::move(def_text->file),
.start_line = def_text->start_line,
.end_line = def_text->end_line,
.text = std::move(def_text->text),
};
}
co_return result;
});
peer.on_request(
[&srv](RequestContext&, const ReferencesParams& params) -> RequestResult<ReferencesParams> {
auto candidates = resolve_locator(
ReadSymbolParams{params.name, params.path, params.line, params.symbol_id},
srv.workspace,
srv.sessions,
srv.indexer);
if(candidates.empty())
co_return kota::outcome_error(kota::ipc::Error{"symbol not found"});
if(candidates.size() > 1) {
co_return kota::outcome_error(kota::ipc::Error{
std::format("ambiguous: {} candidates, use symbolId to disambiguate",
candidates.size())});
}
auto& rs = candidates[0];
ReferencesResult result;
result.name = rs.name;
result.kind = std::string(symbol_kind_name(rs.kind));
result.symbol_id = rs.hash;
for(auto& ref: srv.indexer.collect_references(rs.hash, RelationKind::Reference)) {
result.references.push_back(ReferenceEntry{
.file = std::move(ref.file),
.line = ref.line,
.context = std::move(ref.context),
});
}
if(params.include_declaration.value_or(false)) {
for(auto& ref: srv.indexer.collect_references(rs.hash, RelationKind::Definition)) {
result.references.push_back(ReferenceEntry{
.file = std::move(ref.file),
.line = ref.line,
.context = std::move(ref.context),
});
}
}
result.total = static_cast<int>(result.references.size());
co_return result;
});
peer.on_request(
[&srv](RequestContext&, const CallGraphParams& params) -> RequestResult<CallGraphParams> {
auto candidates = resolve_locator(
ReadSymbolParams{params.name, params.path, params.line, params.symbol_id},
srv.workspace,
srv.sessions,
srv.indexer);
if(candidates.empty())
co_return kota::outcome_error(kota::ipc::Error{"symbol not found"});
if(candidates.size() > 1) {
co_return kota::outcome_error(kota::ipc::Error{
std::format("ambiguous: {} candidates, use symbolId to disambiguate",
candidates.size())});
}
auto& rs = candidates[0];
auto direction = params.direction.value_or("both");
CallGraphResult result;
result.root = CallGraphEntry{
.name = rs.name,
.kind = std::string(symbol_kind_name(rs.kind)),
.file = rs.file,
.line = rs.line,
.symbol_id = rs.hash,
};
auto resolve_kind = [&](std::uint64_t sym_id) -> std::string {
if(sym_id == 0)
return "Function";
std::string name;
SymbolKind kind;
if(srv.indexer.find_symbol_info(sym_id, name, kind))
return std::string(symbol_kind_name(kind));
return "Function";
};
if(direction == "callers" || direction == "both") {
auto incoming = srv.indexer.find_incoming_calls(rs.hash);
for(auto& call: incoming) {
auto sid = extract_symbol_id(call.from.data);
result.callers.push_back(CallGraphEntry{
.name = call.from.name,
.kind = resolve_kind(sid),
.file = uri_to_path(call.from.uri),
.line = static_cast<int>(call.from.range.start.line) + 1,
.symbol_id = sid,
});
}
}
if(direction == "callees" || direction == "both") {
auto outgoing = srv.indexer.find_outgoing_calls(rs.hash);
for(auto& call: outgoing) {
auto sid = extract_symbol_id(call.to.data);
result.callees.push_back(CallGraphEntry{
.name = call.to.name,
.kind = resolve_kind(sid),
.file = uri_to_path(call.to.uri),
.line = static_cast<int>(call.to.range.start.line) + 1,
.symbol_id = sid,
});
}
}
co_return result;
});
peer.on_request(
[&srv](RequestContext&,
const TypeHierarchyParams& params) -> RequestResult<TypeHierarchyParams> {
auto candidates = resolve_locator(
ReadSymbolParams{params.name, params.path, params.line, params.symbol_id},
srv.workspace,
srv.sessions,
srv.indexer);
if(candidates.empty())
co_return kota::outcome_error(kota::ipc::Error{"symbol not found"});
if(candidates.size() > 1) {
co_return kota::outcome_error(kota::ipc::Error{
std::format("ambiguous: {} candidates, use symbolId to disambiguate",
candidates.size())});
}
auto& rs = candidates[0];
auto direction = params.direction.value_or("both");
TypeHierarchyResult result;
result.root = TypeHierarchyEntry{
.name = rs.name,
.kind = std::string(symbol_kind_name(rs.kind)),
.file = rs.file,
.line = rs.line,
.symbol_id = rs.hash,
};
auto resolve_kind = [&](std::uint64_t sym_id) -> std::string {
if(sym_id == 0)
return "Class";
std::string name;
SymbolKind kind;
if(srv.indexer.find_symbol_info(sym_id, name, kind))
return std::string(symbol_kind_name(kind));
return "Class";
};
if(direction == "supertypes" || direction == "both") {
for(auto& item: srv.indexer.find_supertypes(rs.hash)) {
auto sid = extract_symbol_id(item.data);
result.supertypes.push_back(TypeHierarchyEntry{
.name = item.name,
.kind = resolve_kind(sid),
.file = uri_to_path(item.uri),
.line = static_cast<int>(item.range.start.line) + 1,
.symbol_id = sid,
});
}
}
if(direction == "subtypes" || direction == "both") {
for(auto& item: srv.indexer.find_subtypes(rs.hash)) {
auto sid = extract_symbol_id(item.data);
result.subtypes.push_back(TypeHierarchyEntry{
.name = item.name,
.kind = resolve_kind(sid),
.file = uri_to_path(item.uri),
.line = static_cast<int>(item.range.start.line) + 1,
.symbol_id = sid,
});
}
}
co_return result;
});
peer.on_request([&srv](RequestContext&, const LintParams& params) -> RequestResult<LintParams> {
std::string directory;
std::vector<std::string> arguments;
if(!srv.compiler.fill_compile_args(params.path, directory, arguments)) {
co_return kota::outcome_error(
kota::ipc::Error{std::format("no compile command found for {}", params.path)});
}
auto result = co_await kota::queue([path = params.path,
directory = std::move(directory),
arguments = std::move(arguments)]() mutable {
CompilationParams cp;
cp.kind = CompilationKind::Content;
cp.clang_tidy = true;
cp.directory = std::move(directory);
for(auto& arg: arguments) {
cp.arguments.push_back(arg.c_str());
}
auto unit = compile(cp);
if(!unit.completed() && !unit.fatal_error()) {
LOG_WARN("Lint compilation failed: {}", path);
return LintResult{};
}
return feature::diagnostics(unit);
});
co_return result.value();
});
peer.on_request([&srv](RequestContext&, const StatusParams&) -> RequestResult<StatusParams> {
StatusResult result;
result.idle = srv.indexer.is_idle();
result.pending = static_cast<int>(srv.indexer.pending_files());
result.total = static_cast<int>(srv.indexer.total_queued());
result.indexed = std::max(0, result.total - result.pending);
co_return result;
});
peer.on_notification([&srv](const ShutdownParams&) {
LOG_INFO("agentic/shutdown received, shutting down");
srv.schedule_shutdown();
});
}
} // namespace clice

View File

@@ -0,0 +1,18 @@
#pragma once
#include "kota/ipc/codec/json.h"
namespace clice {
class MasterServer;
class AgentClient {
public:
AgentClient(MasterServer& server, kota::ipc::JsonPeer& peer);
private:
MasterServer& server;
kota::ipc::JsonPeer& peer;
};
} // namespace clice

View File

@@ -0,0 +1,180 @@
#include "server/service/agentic.h"
#include <memory>
#include <print>
#include <string>
#include "server/protocol/agentic.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "kota/async/async.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/transport.h"
namespace clice {
template <typename Params>
static kota::task<bool> send_and_print(kota::ipc::JsonPeer& peer, Params params) {
auto result = co_await peer.send_request(std::move(params));
if(!result) {
LOG_ERROR("request failed: {}", result.error().message);
co_return false;
}
auto json = kota::codec::json::to_string<kota::ipc::lsp_config>(*result);
std::println("{}", json ? *json : "null");
co_return true;
}
static kota::task<> agentic_request(kota::ipc::JsonPeer& peer,
int& exit_code,
const AgenticQueryOptions& opts) {
bool ok = false;
if(opts.method == "compileCommand") {
ok = co_await send_and_print(peer, agentic::CompileCommandParams{.path = opts.path});
} else if(opts.method == "projectFiles") {
auto filter = opts.query.empty() ? std::nullopt : std::optional(opts.query);
ok = co_await send_and_print(peer, agentic::ProjectFilesParams{.filter = filter});
} else if(opts.method == "symbolSearch") {
ok = co_await send_and_print(peer, agentic::SymbolSearchParams{.query = opts.query});
} else if(opts.method == "definition") {
auto name = opts.name.empty() ? std::nullopt : std::optional(opts.name);
auto path = opts.path.empty() ? std::nullopt : std::optional(opts.path);
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
ok = co_await send_and_print(
peer,
agentic::DefinitionParams{.name = name, .path = path, .line = line});
} else if(opts.method == "references") {
auto name = opts.name.empty() ? std::nullopt : std::optional(opts.name);
auto path = opts.path.empty() ? std::nullopt : std::optional(opts.path);
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
ok = co_await send_and_print(
peer,
agentic::ReferencesParams{.name = name, .path = path, .line = line});
} else if(opts.method == "readSymbol") {
auto name = opts.name.empty() ? std::nullopt : std::optional(opts.name);
auto path = opts.path.empty() ? std::nullopt : std::optional(opts.path);
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
ok = co_await send_and_print(
peer,
agentic::ReadSymbolParams{.name = name, .path = path, .line = line});
} else if(opts.method == "documentSymbols") {
ok = co_await send_and_print(peer, agentic::DocumentSymbolsParams{.path = opts.path});
} else if(opts.method == "callGraph") {
auto name = opts.name.empty() ? std::nullopt : std::optional(opts.name);
auto path = opts.path.empty() ? std::nullopt : std::optional(opts.path);
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
auto dir = opts.direction.empty() ? std::nullopt : std::optional(opts.direction);
ok = co_await send_and_print(peer,
agentic::CallGraphParams{
.name = name,
.path = path,
.line = line,
.direction = dir,
});
} else if(opts.method == "typeHierarchy") {
auto name = opts.name.empty() ? std::nullopt : std::optional(opts.name);
auto path = opts.path.empty() ? std::nullopt : std::optional(opts.path);
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
auto dir = opts.direction.empty() ? std::nullopt : std::optional(opts.direction);
ok = co_await send_and_print(peer,
agentic::TypeHierarchyParams{
.name = name,
.path = path,
.line = line,
.direction = dir,
});
} else if(opts.method == "lint") {
auto line = opts.line > 0 ? std::optional(opts.line) : std::nullopt;
ok = co_await send_and_print(peer, agentic::LintParams{.path = opts.path, .line = line});
} else if(opts.method == "fileDeps") {
auto dir = opts.direction.empty() ? std::nullopt : std::optional(opts.direction);
ok = co_await send_and_print(peer,
agentic::FileDepsParams{.path = opts.path, .direction = dir});
} else if(opts.method == "impactAnalysis") {
ok = co_await send_and_print(peer, agentic::ImpactAnalysisParams{.path = opts.path});
} else if(opts.method == "status") {
ok = co_await send_and_print(peer, agentic::StatusParams{});
} else if(opts.method == "shutdown") {
peer.send_notification(agentic::ShutdownParams{});
ok = true;
} else {
LOG_ERROR("unknown agentic method '{}'", opts.method);
}
if(ok)
exit_code = 0;
peer.close();
}
static kota::task<> agentic_client(int& exit_code,
std::unique_ptr<kota::ipc::JsonPeer>& peer_out,
const AgenticQueryOptions& opts) {
auto& loop = kota::event_loop::current();
auto transport = co_await kota::ipc::StreamTransport::connect_tcp(opts.host, opts.port, loop);
if(!transport) {
LOG_ERROR("failed to connect to {}:{}", opts.host, opts.port);
co_return;
}
peer_out = std::make_unique<kota::ipc::JsonPeer>(loop, std::move(*transport));
co_await kota::when_all(peer_out->run(), agentic_request(*peer_out, exit_code, opts));
}
int run_agentic_mode(const AgenticQueryOptions& opts) {
logging::stderr_logger("agentic", logging::options);
kota::event_loop loop;
int exit_code = 1;
std::unique_ptr<kota::ipc::JsonPeer> peer;
loop.schedule(agentic_client(exit_code, peer, opts));
loop.run();
return exit_code;
}
static kota::task<> relay_forward(kota::ipc::Transport& from, kota::ipc::Transport& to) {
while(true) {
auto msg = co_await from.read_message();
if(!msg)
break;
co_await to.write_message(*msg);
}
to.close();
}
static kota::task<> relay_main(kota::event_loop& loop, int& exit_code, std::string socket_path) {
auto stdio = kota::ipc::StreamTransport::open_stdio(loop);
if(!stdio) {
LOG_ERROR("failed to open stdio transport");
loop.stop();
co_return;
}
auto conn = co_await kota::pipe::connect(socket_path, {}, loop);
if(!conn) {
LOG_ERROR("failed to connect to {}", socket_path);
loop.stop();
co_return;
}
auto socket = std::make_unique<kota::ipc::StreamTransport>(std::move(*conn));
co_await kota::when_all(relay_forward(**stdio, *socket), relay_forward(*socket, **stdio));
exit_code = 0;
loop.stop();
}
int run_relay_mode(llvm::StringRef socket_path) {
logging::stderr_logger("relay", logging::options);
auto path = socket_path.empty() ? path::default_socket_path() : socket_path.str();
kota::event_loop loop;
int exit_code = 1;
loop.schedule(relay_main(loop, exit_code, std::move(path)));
loop.run();
return exit_code;
}
} // namespace clice

View File

@@ -0,0 +1,24 @@
#pragma once
#include <string>
#include "llvm/ADT/StringRef.h"
namespace clice {
struct AgenticQueryOptions {
std::string host;
int port = 0;
std::string method;
std::string path;
std::string name;
std::string query;
int line = 0;
std::string direction;
};
int run_agentic_mode(const AgenticQueryOptions& opts);
int run_relay_mode(llvm::StringRef socket_path);
} // namespace clice

View File

@@ -1,4 +1,4 @@
#include "server/master_server.h"
#include "server/service/lsp_client.h"
#include <algorithm>
#include <format>
@@ -7,7 +7,9 @@
#include <variant>
#include "semantic/symbol_kind.h"
#include "server/protocol.h"
#include "server/protocol/extension.h"
#include "server/protocol/worker.h"
#include "server/service/master_server.h"
#include "support/filesystem.h"
#include "support/logging.h"
@@ -16,7 +18,6 @@
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/lsp/uri.h"
#include "kota/meta/enum.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/Process.h"
@@ -29,177 +30,39 @@ using kota::ipc::RequestResult;
using RequestContext = kota::ipc::JsonPeer::RequestContext;
using serde_raw = kota::codec::RawValue;
/// Serialize a value to a JSON RawValue using LSP config.
template <typename T>
static serde_raw to_raw(const T& value) {
auto json = kota::codec::json::to_json<kota::ipc::lsp_config>(value);
return serde_raw{json ? std::move(*json) : "null"};
}
MasterServer::MasterServer(kota::event_loop& loop,
kota::ipc::JsonPeer& peer,
std::string self_path) :
loop(loop), peer(peer), pool(loop), compiler(loop, peer, workspace, pool, sessions),
indexer(loop,
workspace,
sessions,
pool,
compiler,
[this](uint32_t proj_path_id) {
// Bridge project-level path_id to server-level path_id.
// The two PathPools may assign different IDs to the same path.
auto path = workspace.project_index.path_pool.path(proj_path_id);
auto server_id = workspace.path_pool.intern(path);
return sessions.contains(server_id);
}),
self_path(std::move(self_path)) {}
LSPClient::LSPClient(MasterServer& server, kota::ipc::JsonPeer& peer) : server(server), peer(peer) {
server.compiler.set_peer(&peer);
server.indexer.set_peer(&peer);
MasterServer::~MasterServer() = default;
kota::task<> MasterServer::load_workspace() {
if(workspace_root.empty())
co_return;
auto& cfg = workspace.config.project;
if(!cfg.cache_dir.empty()) {
auto ec = llvm::sys::fs::create_directories(cfg.cache_dir);
if(ec) {
LOG_WARN("Failed to create cache directory {}: {}",
std::string_view(cfg.cache_dir),
ec.message());
} else {
LOG_INFO("Cache directory: {}", std::string_view(cfg.cache_dir));
}
for(auto* subdir: {"cache/pch", "cache/pcm"}) {
auto dir = path::join(cfg.cache_dir, subdir);
if(auto ec2 = llvm::sys::fs::create_directories(dir))
LOG_WARN("Failed to create {}: {}", dir, ec2.message());
}
workspace.cleanup_cache();
workspace.load_cache();
}
// Discover compile_commands.json: configured paths first, then auto-scan.
std::string cdb_path;
for(auto& configured: cfg.compile_commands_paths) {
// Each entry can be a file or a directory containing compile_commands.json.
if(llvm::sys::fs::is_directory(configured)) {
auto candidate = path::join(configured, "compile_commands.json");
if(llvm::sys::fs::exists(candidate)) {
cdb_path = std::move(candidate);
break;
}
} else if(llvm::sys::fs::exists(configured)) {
cdb_path = configured;
break;
} else {
LOG_WARN("Configured compile_commands_path not found: {}", configured);
}
}
// Auto-scan: workspace root + all immediate subdirectories.
if(cdb_path.empty()) {
auto try_candidate = [&](llvm::StringRef dir) -> bool {
auto candidate = path::join(dir, "compile_commands.json");
if(llvm::sys::fs::exists(candidate)) {
cdb_path = std::move(candidate);
return true;
}
return false;
};
if(!try_candidate(workspace_root)) {
std::error_code ec;
for(llvm::sys::fs::directory_iterator it(workspace_root, ec), end; it != end && !ec;
it.increment(ec)) {
if(it->type() == llvm::sys::fs::file_type::directory_file) {
if(try_candidate(it->path()))
break;
}
}
}
}
if(cdb_path.empty()) {
LOG_WARN("No compile_commands.json found in workspace {}", workspace_root);
co_return;
}
auto count = workspace.cdb.load(cdb_path);
LOG_INFO("Loaded CDB from {} with {} entries", cdb_path, count);
auto report = scan_dependency_graph(workspace.cdb,
workspace.path_pool,
workspace.dep_graph,
/*cache=*/nullptr,
[this](llvm::StringRef path,
std::vector<std::string>& append,
std::vector<std::string>& remove) {
workspace.config.match_rules(path, append, remove);
});
workspace.dep_graph.build_reverse_map();
auto unresolved = report.includes_found - report.includes_resolved;
double accuracy =
report.includes_found > 0
? 100.0 * static_cast<double>(report.includes_resolved) / report.includes_found
: 100.0;
LOG_INFO(
"Dependency scan: {}ms, {} files ({} source + {} header), " "{} edges, {}/{} resolved ({:.1f}%), {} waves",
report.elapsed_ms,
report.total_files,
report.source_files,
report.header_files,
report.total_edges,
report.includes_resolved,
report.includes_found,
accuracy,
report.waves);
if(unresolved > 0)
LOG_WARN("{} unresolved includes", unresolved);
workspace.build_module_map();
indexer.load(cfg.index_dir);
if(*cfg.enable_indexing) {
for(auto& entry: workspace.cdb.get_entries()) {
auto file = workspace.cdb.resolve_path(entry.file);
auto server_id = workspace.path_pool.intern(file);
indexer.enqueue(server_id);
}
indexer.schedule();
}
compiler.init_compile_graph();
}
void MasterServer::register_handlers() {
using StringVec = std::vector<std::string>;
peer.on_request([this](RequestContext& ctx, const protocol::InitializeParams& params)
-> RequestResult<protocol::InitializeParams> {
if(lifecycle != ServerLifecycle::Uninitialized) {
auto& srv = this->server;
if(srv.lifecycle != ServerLifecycle::Uninitialized) {
co_return kota::outcome_error(protocol::Error{"Server already initialized"});
}
auto& init = params.lsp__initialize_params;
if(init.root_uri.has_value()) {
workspace_root = uri_to_path(*init.root_uri);
srv.workspace_root = uri_to_path(*init.root_uri);
}
// Capture initializationOptions as raw JSON for config loading.
if(init.initialization_options.has_value()) {
auto json =
kota::codec::json::to_json<kota::ipc::lsp_config>(*init.initialization_options);
if(json)
init_options_json = std::move(*json);
srv.init_options_json = std::move(*json);
}
lifecycle = ServerLifecycle::Initialized;
LOG_INFO("Initialized with workspace: {}", workspace_root);
srv.lifecycle = ServerLifecycle::Initialized;
LOG_INFO("Initialized with workspace: {}", srv.workspace_root);
protocol::InitializeResult result;
auto& caps = result.capabilities;
@@ -222,7 +85,6 @@ void MasterServer::register_handlers() {
caps.signature_help_provider = protocol::SignatureHelpOptions{
.trigger_characters = StringVec{"(", ")", "{", "}", "<", ">", ","},
};
/// FIXME: In the future, we would support work done progress.
caps.declaration_provider = protocol::DeclarationOptions{
.work_done_progress = false,
};
@@ -246,6 +108,8 @@ void MasterServer::register_handlers() {
caps.call_hierarchy_provider = true;
caps.type_hierarchy_provider = true;
caps.workspace_symbol_provider = true;
caps.document_formatting_provider = true;
caps.document_range_formatting_provider = true;
protocol::SemanticTokensOptions sem_opts;
{
@@ -277,100 +141,32 @@ void MasterServer::register_handlers() {
co_return result;
});
peer.on_notification([this](const protocol::InitializedParams& params) {
// Config priority: initializationOptions > clice.toml > defaults.
// Load the workspace config (with defaults applied) first, then overlay
// any initializationOptions on top so fields not mentioned in the JSON
// keep the values from clice.toml — kotatsu's deserializer only touches
// fields that are present in the input.
workspace.config = Config::load_from_workspace(workspace_root);
if(!init_options_json.empty()) {
if(auto ov = kota::codec::json::parse(init_options_json, workspace.config); !ov) {
LOG_WARN("Failed to apply initializationOptions: {}", ov.error().to_string());
} else {
// Re-run apply_defaults so overridden strings get workspace
// substitution and `compiled_rules` is rebuilt if `rules`
// changed. Defaults are gated on zero/empty sentinels, so
// existing values from the overlay are preserved.
workspace.config.apply_defaults(workspace_root);
LOG_INFO("Applied initializationOptions overlay");
}
init_options_json.clear();
}
auto& cfg = workspace.config.project;
if(!cfg.logging_dir.empty()) {
auto now = std::chrono::system_clock::now();
auto pid = llvm::sys::Process::getProcessId();
auto session_dir =
path::join(cfg.logging_dir, std::format("{:%Y-%m-%d_%H-%M-%S}_{}", now, pid));
logging::file_logger("master", session_dir, logging::options);
session_log_dir = session_dir;
}
LOG_INFO("Server ready (stateful={}, stateless={}, idle={}ms)",
cfg.stateful_worker_count.value,
cfg.stateless_worker_count.value,
*cfg.idle_timeout_ms);
WorkerPoolOptions pool_opts;
pool_opts.self_path = self_path;
pool_opts.stateful_count = cfg.stateful_worker_count;
pool_opts.stateless_count = cfg.stateless_worker_count;
pool_opts.worker_memory_limit = cfg.worker_memory_limit;
pool_opts.log_dir = session_log_dir;
if(!pool.start(pool_opts)) {
LOG_ERROR("Failed to start worker pool");
return;
}
lifecycle = ServerLifecycle::Ready;
compiler.on_indexing_needed = [this]() {
indexer.schedule();
};
loop.schedule(load_workspace());
peer.on_notification([this]([[maybe_unused]] const protocol::InitializedParams& params) {
this->server.initialize();
});
peer.on_request(
[this](RequestContext& ctx,
const protocol::ShutdownParams& params) -> RequestResult<protocol::ShutdownParams> {
lifecycle = ServerLifecycle::ShuttingDown;
this->server.lifecycle = ServerLifecycle::ShuttingDown;
LOG_INFO("Shutdown requested");
co_return nullptr;
});
peer.on_notification([this](const protocol::ExitParams& params) {
lifecycle = ServerLifecycle::Exited;
peer.on_notification([this]([[maybe_unused]] const protocol::ExitParams& params) {
LOG_INFO("Exit notification received");
indexer.save(workspace.config.project.index_dir);
workspace.save_cache();
loop.schedule([this]() -> kota::task<> {
co_await pool.stop();
loop.stop();
}());
this->server.schedule_shutdown();
});
/// Document lifecycle — handled directly by MasterServer.
peer.on_notification([this](const protocol::DidOpenTextDocumentParams& params) {
if(lifecycle != ServerLifecycle::Ready)
auto& srv = this->server;
if(srv.lifecycle != ServerLifecycle::Ready)
return;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto path_id = srv.workspace.path_pool.intern(path);
auto [it, inserted] = sessions.try_emplace(path_id);
auto& session = it->second;
if(!inserted) {
// DenseMap tombstone may retain stale data — reset to a fresh Session.
session = Session{};
}
session.path_id = path_id;
auto& session = srv.open_session(path_id);
session.version = params.text_document.version;
session.text = params.text_document.text;
session.generation++;
@@ -379,18 +175,18 @@ void MasterServer::register_handlers() {
});
peer.on_notification([this](const protocol::DidChangeTextDocumentParams& params) {
if(lifecycle != ServerLifecycle::Ready)
auto& srv = this->server;
if(srv.lifecycle != ServerLifecycle::Ready)
return;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto path_id = srv.workspace.path_pool.intern(path);
auto it = sessions.find(path_id);
if(it == sessions.end())
auto* session = srv.find_session(path_id);
if(!session)
return;
auto& session = it->second;
session.version = params.text_document.version;
session->version = params.text_document.version;
for(auto& change: params.content_changes) {
std::visit(
@@ -398,186 +194,157 @@ void MasterServer::register_handlers() {
using T = std::remove_cvref_t<decltype(c)>;
if constexpr(std::is_same_v<T,
protocol::TextDocumentContentChangeWholeDocument>) {
session.text = c.text;
session->text = c.text;
} else {
auto& range = c.range;
lsp::PositionMapper mapper(session.text, lsp::PositionEncoding::UTF16);
lsp::PositionMapper mapper(session->text, lsp::PositionEncoding::UTF16);
auto start = mapper.to_offset(range.start);
auto end = mapper.to_offset(range.end);
if(start && end && *start <= *end) {
session.text.replace(*start, *end - *start, c.text);
session->text.replace(*start, *end - *start, c.text);
}
}
},
change);
}
session.generation++;
session.ast_dirty = true;
session->generation++;
session->ast_dirty = true;
LOG_DEBUG("didChange: path={} version={} gen={}",
path,
session.version,
session.generation);
session->version,
session->generation);
worker::DocumentUpdateParams update;
update.path = path;
update.version = session.version;
pool.notify_stateful(path_id, update);
update.version = session->version;
srv.pool.notify_stateful(path_id, update);
});
peer.on_notification([this](const protocol::DidCloseTextDocumentParams& params) {
if(lifecycle != ServerLifecycle::Ready)
auto& srv = this->server;
if(srv.lifecycle != ServerLifecycle::Ready)
return;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
workspace.on_file_closed(path_id);
pool.notify_stateful(path_id, worker::EvictParams{path});
// Clear diagnostics for the closed file.
protocol::PublishDiagnosticsParams diag_params;
diag_params.uri = params.text_document.uri;
peer.send_notification(diag_params);
sessions.erase(path_id);
indexer.enqueue(path_id);
indexer.schedule();
LOG_DEBUG("didClose: {}", path);
auto path_id = srv.workspace.path_pool.intern(uri_to_path(params.text_document.uri));
srv.close_session(path_id, this->peer);
});
peer.on_notification([this](const protocol::DidSaveTextDocumentParams& params) {
if(lifecycle != ServerLifecycle::Ready)
auto& srv = this->server;
if(srv.lifecycle != ServerLifecycle::Ready)
return;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto dirtied = workspace.on_file_saved(path_id);
for(auto dirty_id: dirtied) {
if(auto sit = sessions.find(dirty_id); sit != sessions.end()) {
sit->second.ast_dirty = true;
} else {
indexer.enqueue(dirty_id);
}
}
// Invalidate header contexts for sessions whose host is this file.
for(auto& [hdr_id, session]: sessions) {
if(session.header_context && session.header_context->host_path_id == path_id) {
session.header_context.reset();
session.ast_dirty = true;
}
}
indexer.schedule();
auto path_id = srv.workspace.path_pool.intern(path);
srv.on_file_saved(path_id);
LOG_DEBUG("didSave: {}", path);
});
/// Feature requests — stateful forwarding.
peer.on_request([this](RequestContext& ctx, const protocol::HoverParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document_position_params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::Hover,
sit->second,
params.text_document_position_params.position);
co_return co_await srv.compiler.forward_query(
worker::QueryKind::Hover,
*session,
params.text_document_position_params.position);
});
peer.on_request([this](RequestContext& ctx,
const protocol::SemanticTokensParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::SemanticTokens, sit->second);
co_return co_await srv.compiler.forward_query(worker::QueryKind::SemanticTokens, *session);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::InlayHintParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::InlayHints,
sit->second,
{},
params.range);
co_return co_await srv.compiler.forward_query(worker::QueryKind::InlayHints,
*session,
{},
params.range);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::FoldingRangeParams& params) -> RawResult {
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::FoldingRange, sit->second);
});
peer.on_request([this](RequestContext& ctx,
const protocol::FoldingRangeParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await srv.compiler.forward_query(worker::QueryKind::FoldingRange, *session);
});
peer.on_request([this](RequestContext& ctx,
const protocol::DocumentSymbolParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::DocumentSymbol, sit->second);
});
peer.on_request([this](RequestContext& ctx,
const protocol::DocumentLinkParams& params) -> RawResult {
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
co_return serde_raw{"null"};
auto& session = sit->second;
auto result = co_await compiler.forward_query(worker::QueryKind::DocumentLink, session);
if(!result.has_value())
co_return serde_raw{"null"};
// Merge document links from PCH if available.
auto& links = result.value();
// Re-lookup session after co_await since iterators may be invalidated.
auto sit2 = sessions.find(path_id);
if(sit2 != sessions.end() && sit2->second.pch_ref) {
auto pch_it = workspace.pch_cache.find(sit2->second.pch_ref->path_id);
if(pch_it != workspace.pch_cache.end() && !pch_it->second.document_links_json.empty()) {
auto& pch_json = pch_it->second.document_links_json;
// Merge two JSON arrays.
if(!links.data.empty() && links.data != "null" && links.data.size() > 2) {
// "[a,b]" + "[c,d]" -> "[a,b,c,d]"
links.data.pop_back(); // remove trailing ']'
links.data += ',';
links.data.append(pch_json.begin() + 1, pch_json.end()); // skip '['
} else {
links.data = pch_json;
}
}
}
co_return std::move(links);
co_return co_await srv.compiler.forward_query(worker::QueryKind::DocumentSymbol, *session);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::CodeActionParams& params) -> RawResult {
[this](RequestContext& ctx, const protocol::DocumentLinkParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::CodeAction, sit->second);
auto result =
co_await srv.compiler.forward_query(worker::QueryKind::DocumentLink, *session);
if(!result.has_value())
co_return serde_raw{"null"};
auto& links = result.value();
auto* session2 = srv.find_session(path_id);
if(session2 && session2->pch_ref) {
auto& pch_cache = srv.workspace.pch_cache;
auto pch_it = pch_cache.find(session2->pch_ref->path_id);
if(pch_it != pch_cache.end() && !pch_it->second.document_links_json.empty()) {
auto& pch_json = pch_it->second.document_links_json;
if(!links.data.empty() && links.data != "null" && links.data.size() > 2) {
links.data.pop_back();
links.data += ',';
links.data.append(pch_json.begin() + 1, pch_json.end());
} else {
links.data = pch_json;
}
}
}
co_return std::move(links);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::CodeActionParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await srv.compiler.forward_query(worker::QueryKind::CodeAction, *session);
});
/// Helper: resolve URI to path, path_id, and Session pointer.
auto resolve_uri = [this](const std::string& uri) {
struct Result {
std::string path;
@@ -585,22 +352,21 @@ void MasterServer::register_handlers() {
Session* session;
};
auto path = uri_to_path(uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
Session* session = (sit != sessions.end()) ? &sit->second : nullptr;
auto path_id = this->server.workspace.path_pool.intern(path);
auto* session = this->server.find_session(path_id);
return Result{std::move(path), path_id, session};
};
auto lookup_at = [this, resolve_uri](const std::string& uri, const protocol::Position& pos) {
auto [path, path_id, session] = resolve_uri(uri);
return indexer.lookup_symbol(uri, path, pos, session);
return this->server.indexer.lookup_symbol(uri, path, pos, session);
};
auto query_at = [this, resolve_uri](const std::string& uri,
const protocol::Position& pos,
RelationKind kind) -> std::vector<protocol::Location> {
auto [path, path_id, session] = resolve_uri(uri);
return indexer.query_relations(path, pos, kind, session);
return this->server.indexer.query_relations(path, pos, kind, session);
};
auto resolve_item =
@@ -609,11 +375,9 @@ void MasterServer::register_handlers() {
const protocol::Range& range,
const std::optional<protocol::LSPAny>& data) -> std::optional<SymbolInfo> {
auto [path, path_id, session] = resolve_uri(uri);
return indexer.resolve_hierarchy_item(uri, path, range, data, session);
return this->server.indexer.resolve_hierarchy_item(uri, path, range, data, session);
};
/// Feature requests — index-based with AST fallback.
peer.on_request([this, query_at](RequestContext& ctx,
const protocol::DefinitionParams& params) -> RawResult {
auto& uri = params.text_document_position_params.text_document.uri;
@@ -624,14 +388,15 @@ void MasterServer::register_handlers() {
co_return to_raw(result);
}
auto& srv = this->server;
auto path = uri_to_path(uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_query(worker::QueryKind::GoToDefinition,
sit->second,
pos);
co_return co_await srv.compiler.forward_query(worker::QueryKind::GoToDefinition,
*session,
pos);
});
peer.on_request([this, query_at](RequestContext& ctx,
@@ -668,32 +433,60 @@ void MasterServer::register_handlers() {
co_return serde_raw{"null"};
});
/// Feature requests — stateless forwarding.
peer.on_request([this](RequestContext& ctx,
const protocol::CompletionParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document_position_params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.handle_completion(params.text_document_position_params.position,
sit->second);
auto pause = srv.indexer.scoped_pause();
auto result =
co_await srv.compiler.handle_completion(params.text_document_position_params.position,
*session);
co_return std::move(result);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::SignatureHelpParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document_position_params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
co_return co_await compiler.forward_build(worker::BuildKind::SignatureHelp,
params.text_document_position_params.position,
sit->second);
auto pause = srv.indexer.scoped_pause();
auto result =
co_await srv.compiler.forward_build(worker::BuildKind::SignatureHelp,
params.text_document_position_params.position,
*session);
co_return std::move(result);
});
/// Hierarchy queries — index-based.
peer.on_request(
[this](RequestContext& ctx, const protocol::DocumentFormattingParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
auto pause = srv.indexer.scoped_pause();
co_return co_await srv.compiler.forward_format(*session);
});
peer.on_request([this](RequestContext& ctx,
const protocol::DocumentRangeFormattingParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.text_document.uri);
auto path_id = srv.workspace.path_pool.intern(path);
auto* session = srv.find_session(path_id);
if(!session)
co_return serde_raw{"null"};
auto pause = srv.indexer.scoped_pause();
co_return co_await srv.compiler.forward_format(*session, params.range);
});
peer.on_request(
[this, lookup_at](RequestContext& ctx,
@@ -718,7 +511,7 @@ void MasterServer::register_handlers() {
auto info = resolve_item(params.item.uri, params.item.range, params.item.data);
if(!info)
co_return serde_raw{"null"};
auto results = indexer.find_incoming_calls(info->hash);
auto results = this->server.indexer.find_incoming_calls(info->hash);
if(results.empty())
co_return serde_raw{"null"};
co_return to_raw(results);
@@ -730,7 +523,7 @@ void MasterServer::register_handlers() {
auto info = resolve_item(params.item.uri, params.item.range, params.item.data);
if(!info)
co_return serde_raw{"null"};
auto results = indexer.find_outgoing_calls(info->hash);
auto results = this->server.indexer.find_outgoing_calls(info->hash);
if(results.empty())
co_return serde_raw{"null"};
co_return to_raw(results);
@@ -760,7 +553,7 @@ void MasterServer::register_handlers() {
auto info = resolve_item(params.item.uri, params.item.range, params.item.data);
if(!info)
co_return serde_raw{"null"};
auto results = indexer.find_supertypes(info->hash);
auto results = this->server.indexer.find_supertypes(info->hash);
if(results.empty())
co_return serde_raw{"null"};
co_return to_raw(results);
@@ -772,7 +565,7 @@ void MasterServer::register_handlers() {
auto info = resolve_item(params.item.uri, params.item.range, params.item.data);
if(!info)
co_return serde_raw{"null"};
auto results = indexer.find_subtypes(info->hash);
auto results = this->server.indexer.find_subtypes(info->hash);
if(results.empty())
co_return serde_raw{"null"};
co_return to_raw(results);
@@ -780,29 +573,29 @@ void MasterServer::register_handlers() {
peer.on_request(
[this](RequestContext& ctx, const protocol::WorkspaceSymbolParams& params) -> RawResult {
auto results = indexer.search_symbols(params.query);
auto results = this->server.indexer.search_symbols(params.query);
if(results.empty())
co_return serde_raw{"null"};
co_return to_raw(results);
});
/// clice/ extension commands.
peer.on_request(
"clice/queryContext",
[this](RequestContext& ctx, const ext::QueryContextParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.uri);
auto path_id = workspace.path_pool.intern(path);
auto path_id = srv.workspace.path_pool.intern(path);
int offset_val = std::max(0, params.offset.value_or(0));
constexpr int page_size = 10;
ext::QueryContextResult result;
std::vector<ext::ContextItem> all_items;
auto hosts = workspace.dep_graph.find_host_sources(path_id);
auto& ws = srv.workspace;
auto hosts = ws.dep_graph.find_host_sources(path_id);
for(auto host_id: hosts) {
auto host_path = workspace.path_pool.resolve(host_id);
auto host_cdb = workspace.cdb.lookup(host_path, {.suppress_logging = true});
auto host_path = ws.path_pool.resolve(host_id);
auto host_cdb = ws.cdb.lookup(host_path, {.suppress_logging = true});
if(host_cdb.empty())
continue;
auto host_uri_opt = lsp::URI::from_file_path(std::string(host_path));
@@ -816,7 +609,7 @@ void MasterServer::register_handlers() {
}
if(hosts.empty()) {
auto entries = workspace.cdb.lookup(path, {.suppress_logging = true});
auto entries = ws.cdb.lookup(path, {.suppress_logging = true});
for(std::size_t i = 0; i < entries.size(); ++i) {
auto& cmd = entries[i];
auto argv = cmd.to_argv();
@@ -858,13 +651,14 @@ void MasterServer::register_handlers() {
peer.on_request(
"clice/currentContext",
[this](RequestContext& ctx, const ext::CurrentContextParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.uri);
auto path_id = workspace.path_pool.intern(path);
auto path_id = srv.workspace.path_pool.intern(path);
ext::CurrentContextResult result;
auto sit = sessions.find(path_id);
if(sit != sessions.end() && sit->second.active_context) {
auto ctx_path = workspace.path_pool.resolve(*sit->second.active_context);
auto* session = srv.find_session(path_id);
if(session && session->active_context) {
auto ctx_path = srv.workspace.path_pool.resolve(*session->active_context);
auto ctx_uri_opt = lsp::URI::from_file_path(std::string(ctx_path));
if(ctx_uri_opt) {
ext::ContextItem item;
@@ -880,34 +674,41 @@ void MasterServer::register_handlers() {
peer.on_request(
"clice/switchContext",
[this](RequestContext& ctx, const ext::SwitchContextParams& params) -> RawResult {
auto& srv = this->server;
auto path = uri_to_path(params.uri);
auto path_id = workspace.path_pool.intern(path);
auto path_id = srv.workspace.path_pool.intern(path);
auto context_path = uri_to_path(params.context_uri);
auto context_path_id = workspace.path_pool.intern(context_path);
auto context_path_id = srv.workspace.path_pool.intern(context_path);
ext::SwitchContextResult result;
auto context_cdb = workspace.cdb.lookup(context_path, {.suppress_logging = true});
auto& ws = srv.workspace;
auto context_cdb = ws.cdb.lookup(context_path, {.suppress_logging = true});
if(context_cdb.empty()) {
result.success = false;
co_return to_raw(result);
}
auto sit = sessions.find(path_id);
if(sit == sessions.end()) {
auto* session = srv.find_session(path_id);
if(!session) {
result.success = false;
co_return to_raw(result);
}
sit->second.active_context = context_path_id;
sit->second.header_context.reset();
sit->second.pch_ref.reset();
sit->second.ast_deps.reset();
sit->second.ast_dirty = true;
session->active_context = context_path_id;
session->header_context.reset();
session->pch_ref.reset();
session->ast_deps.reset();
session->ast_dirty = true;
result.success = true;
co_return to_raw(result);
});
}
LSPClient::~LSPClient() {
server.compiler.set_peer(nullptr);
server.indexer.set_peer(nullptr);
}
} // namespace clice

View File

@@ -0,0 +1,23 @@
#pragma once
#include "kota/async/async.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
namespace clice {
class MasterServer;
class LSPClient {
public:
LSPClient(MasterServer& server, kota::ipc::JsonPeer& peer);
~LSPClient();
private:
using RawResult = kota::task<kota::codec::RawValue, kota::ipc::Error>;
MasterServer& server;
kota::ipc::JsonPeer& peer;
};
} // namespace clice

View File

@@ -0,0 +1,551 @@
#include "server/service/master_server.h"
#include <cerrno>
#include <cstring>
#include <list>
#include <memory>
#include <string>
#include <vector>
#ifndef _WIN32
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#endif
#include "server/protocol/worker.h"
#include "server/service/agent_client.h"
#include "server/service/lsp_client.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "kota/async/async.h"
#include "kota/async/io/fs_event.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/lsp/uri.h"
#include "kota/ipc/recording_transport.h"
#include "kota/ipc/transport.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/Process.h"
namespace clice {
namespace lsp = kota::ipc::lsp;
namespace protocol = kota::ipc::protocol;
MasterServer::MasterServer(kota::event_loop& loop, std::string self_path) :
loop(loop), pool(loop), compiler(loop, workspace, pool, sessions),
indexer(loop,
workspace,
sessions,
pool,
compiler,
[this](uint32_t proj_path_id) {
auto path = workspace.project_index.path_pool.path(proj_path_id);
auto server_id = workspace.path_pool.intern(path);
return sessions.contains(server_id);
}),
self_path(std::move(self_path)) {}
MasterServer::~MasterServer() = default;
void MasterServer::initialize() {
workspace.config = Config::load_from_workspace(workspace_root);
if(!init_options_json.empty()) {
if(auto ov = kota::codec::json::parse(init_options_json, workspace.config); !ov) {
LOG_WARN("Failed to apply initializationOptions: {}", ov.error().to_string());
} else {
workspace.config.apply_defaults(workspace_root);
LOG_INFO("Applied initializationOptions overlay");
}
init_options_json.clear();
}
auto& cfg = workspace.config.project;
if(!cfg.logging_dir.empty()) {
auto now = std::chrono::system_clock::now();
auto pid = llvm::sys::Process::getProcessId();
session_log_dir =
path::join(cfg.logging_dir, std::format("{:%Y-%m-%d_%H-%M-%S}_{}", now, pid));
logging::file_logger("master", session_log_dir, logging::options);
}
LOG_INFO("Server ready (stateful={}, stateless={}, idle={}ms)",
cfg.stateful_worker_count.value,
cfg.stateless_worker_count.value,
*cfg.idle_timeout_ms);
WorkerPoolOptions pool_opts;
pool_opts.self_path = self_path;
pool_opts.stateful_count = cfg.stateful_worker_count;
pool_opts.stateless_count = cfg.stateless_worker_count;
pool_opts.worker_memory_limit = cfg.worker_memory_limit;
pool_opts.log_dir = session_log_dir;
if(!pool.start(pool_opts)) {
LOG_ERROR("Failed to start worker pool");
return;
}
lifecycle = ServerLifecycle::Ready;
compiler.on_indexing_needed = [this]() {
indexer.schedule();
};
indexer.set_max_concurrency(cfg.stateless_worker_count.value);
load_workspace();
}
void MasterServer::initialize(llvm::StringRef root) {
workspace_root = root.str();
initialize();
}
void MasterServer::start_file_watcher() {
if(workspace_root.empty())
return;
loop.schedule([this]() -> kota::task<> {
auto watcher = kota::fs_event::create(workspace_root, {}, loop);
if(!watcher) {
LOG_WARN("Failed to start file watcher for {}", workspace_root);
co_return;
}
LOG_INFO("File watcher started for {}", workspace_root);
while(true) {
auto changes = co_await watcher->next();
if(!changes)
break;
for(auto& change: *changes) {
if(change.type != kota::fs_event::effect::modify &&
change.type != kota::fs_event::effect::create)
continue;
llvm::StringRef file(change.path);
if(file.ends_with("compile_commands.json")) {
LOG_INFO("CDB changed, reloading workspace");
load_workspace();
continue;
}
if(file.ends_with(".cpp") || file.ends_with(".cc") || file.ends_with(".cxx") ||
file.ends_with(".c") || file.ends_with(".h") || file.ends_with(".hpp") ||
file.ends_with(".hxx") || file.ends_with(".cppm") || file.ends_with(".ixx")) {
auto path_id = workspace.path_pool.intern(file);
on_file_saved(path_id);
}
}
}
}());
}
Session* MasterServer::find_session(std::uint32_t path_id) {
auto it = sessions.find(path_id);
return it != sessions.end() ? &it->second : nullptr;
}
Session& MasterServer::open_session(std::uint32_t path_id) {
auto [it, inserted] = sessions.try_emplace(path_id);
auto& session = it->second;
if(!inserted)
session = Session{};
session.path_id = path_id;
return session;
}
void MasterServer::close_session(std::uint32_t path_id, kota::ipc::JsonPeer& peer) {
namespace protocol = kota::ipc::protocol;
auto path = workspace.path_pool.resolve(path_id);
workspace.on_file_closed(path_id);
pool.notify_stateful(path_id, worker::EvictParams{std::string(path)});
protocol::PublishDiagnosticsParams diag_params;
auto uri = lsp::URI::from_file_path(std::string(path));
if(uri)
diag_params.uri = uri->str();
diag_params.diagnostics = {};
peer.send_notification(diag_params);
sessions.erase(path_id);
indexer.enqueue(path_id);
indexer.schedule();
LOG_DEBUG("didClose: {}", path);
}
void MasterServer::on_file_saved(std::uint32_t path_id) {
auto dirtied = workspace.on_file_saved(path_id);
for(auto dirty_id: dirtied) {
if(auto* session = find_session(dirty_id)) {
session->ast_dirty = true;
} else {
indexer.enqueue(dirty_id);
}
}
for(auto& [hdr_id, session]: sessions) {
if(session.header_context && session.header_context->host_path_id == path_id) {
session.header_context.reset();
session.ast_dirty = true;
}
}
indexer.schedule();
}
void MasterServer::schedule_shutdown() {
if(lifecycle == ServerLifecycle::Exited)
return;
lifecycle = ServerLifecycle::Exited;
indexer.save(workspace.config.project.index_dir);
workspace.save_cache();
shutdown_event.set();
loop.schedule([this]() -> kota::task<> {
co_await kota::when_all(indexer.stop(), compiler.stop(), pool.stop());
loop.stop();
}());
}
void MasterServer::load_workspace() {
if(workspace_root.empty())
return;
auto& cfg = workspace.config.project;
if(!cfg.cache_dir.empty()) {
auto ec = llvm::sys::fs::create_directories(cfg.cache_dir);
if(ec) {
LOG_WARN("Failed to create cache directory {}: {}",
std::string_view(cfg.cache_dir),
ec.message());
} else {
LOG_INFO("Cache directory: {}", std::string_view(cfg.cache_dir));
}
for(auto* subdir: {"cache/pch", "cache/pcm"}) {
auto dir = path::join(cfg.cache_dir, subdir);
if(auto ec2 = llvm::sys::fs::create_directories(dir))
LOG_WARN("Failed to create {}: {}", dir, ec2.message());
}
workspace.cleanup_cache();
workspace.load_cache();
}
std::string cdb_path;
for(auto& configured: cfg.compile_commands_paths) {
if(llvm::sys::fs::is_directory(configured)) {
auto candidate = path::join(configured, "compile_commands.json");
if(llvm::sys::fs::exists(candidate)) {
cdb_path = std::move(candidate);
break;
}
} else if(llvm::sys::fs::exists(configured)) {
cdb_path = configured;
break;
} else {
LOG_WARN("Configured compile_commands_path not found: {}", configured);
}
}
if(cdb_path.empty()) {
auto try_candidate = [&](llvm::StringRef dir) -> bool {
auto candidate = path::join(dir, "compile_commands.json");
if(llvm::sys::fs::exists(candidate)) {
cdb_path = std::move(candidate);
return true;
}
return false;
};
if(!try_candidate(workspace_root)) {
std::error_code ec;
for(llvm::sys::fs::directory_iterator it(workspace_root, ec), end; it != end && !ec;
it.increment(ec)) {
if(it->type() == llvm::sys::fs::file_type::directory_file) {
if(try_candidate(it->path()))
break;
}
}
}
}
if(cdb_path.empty()) {
LOG_WARN("No compile_commands.json found in workspace {}", workspace_root);
return;
}
auto count = workspace.cdb.load(cdb_path);
LOG_INFO("Loaded CDB from {} with {} entries", cdb_path, count);
auto report = scan_dependency_graph(workspace.cdb,
workspace.path_pool,
workspace.dep_graph,
/*cache=*/nullptr,
[this](llvm::StringRef path,
std::vector<std::string>& append,
std::vector<std::string>& remove) {
workspace.config.match_rules(path, append, remove);
});
workspace.dep_graph.build_reverse_map();
auto unresolved = report.includes_found - report.includes_resolved;
double accuracy =
report.includes_found > 0
? 100.0 * static_cast<double>(report.includes_resolved) / report.includes_found
: 100.0;
LOG_INFO(
"Dependency scan: {}ms, {} files ({} source + {} header), " "{} edges, {}/{} resolved ({:.1f}%), {} waves",
report.elapsed_ms,
report.total_files,
report.source_files,
report.header_files,
report.total_edges,
report.includes_resolved,
report.includes_found,
accuracy,
report.waves);
if(unresolved > 0)
LOG_WARN("{} unresolved includes", unresolved);
workspace.build_module_map();
indexer.load(cfg.index_dir);
if(*cfg.enable_indexing) {
for(auto& entry: workspace.cdb.get_entries()) {
auto file = workspace.cdb.resolve_path(entry.file);
auto server_id = workspace.path_pool.intern(file);
indexer.enqueue(server_id);
}
indexer.schedule();
}
compiler.init_compile_graph();
}
struct Connection {
std::unique_ptr<kota::ipc::JsonPeer> peer;
std::unique_ptr<LSPClient> lsp_client;
std::unique_ptr<AgentClient> agent_client;
};
static kota::task<> run_connection(kota::ipc::JsonPeer* peer,
std::list<Connection>& connections,
std::list<Connection>::iterator pos) {
co_await peer->run();
LOG_INFO("Client disconnected");
connections.erase(pos);
}
static kota::task<> accept_connections(MasterServer& server,
kota::tcp::acceptor acceptor,
bool register_lsp,
std::list<Connection>& connections) {
auto& loop = kota::event_loop::current();
kota::task_group<> connection_group(loop);
bool lsp_registered = false;
while(true) {
auto conn = co_await acceptor.accept();
if(!conn.has_value())
break;
LOG_INFO("Client connected");
auto transport = std::make_unique<kota::ipc::StreamTransport>(std::move(*conn));
auto peer = std::make_unique<kota::ipc::JsonPeer>(loop, std::move(transport));
std::unique_ptr<LSPClient> lsp;
if(register_lsp && !lsp_registered) {
lsp = std::make_unique<LSPClient>(server, *peer);
lsp_registered = true;
}
auto agent = std::make_unique<AgentClient>(server, *peer);
auto* peer_ptr = peer.get();
auto it = connections.emplace(connections.end(),
Connection{
.peer = std::move(peer),
.lsp_client = std::move(lsp),
.agent_client = std::move(agent),
});
connection_group.spawn(run_connection(peer_ptr, connections, it));
}
co_await connection_group.join();
}
int run_server_mode(const ServerOptions& opts) {
logging::stderr_logger("master", logging::options);
kota::event_loop loop;
MasterServer server(loop, opts.self_path);
std::list<Connection> connections;
if(opts.mode == "pipe") {
auto transport = kota::ipc::StreamTransport::open_stdio(loop);
if(!transport) {
LOG_ERROR("failed to open stdio transport");
return 1;
}
std::unique_ptr<kota::ipc::Transport> final_transport = std::move(*transport);
if(!opts.record.empty()) {
final_transport =
std::make_unique<kota::ipc::RecordingTransport>(std::move(final_transport),
opts.record);
}
kota::ipc::JsonPeer lsp_peer(loop, std::move(final_transport));
LSPClient lsp_client(server, lsp_peer);
if(opts.port > 0) {
auto acceptor = kota::tcp::listen(opts.host, opts.port, {}, loop);
if(acceptor) {
LOG_INFO("Agentic protocol listening on {}:{}", opts.host, opts.port);
loop.schedule(accept_connections(server, std::move(*acceptor), false, connections));
} else {
LOG_WARN("Failed to start agentic listener on {}:{}", opts.host, opts.port);
}
}
loop.schedule(lsp_peer.run());
loop.run();
return 0;
}
if(opts.mode == "socket") {
auto acceptor = kota::tcp::listen(opts.host, opts.port, {}, loop);
if(!acceptor) {
LOG_ERROR("failed to listen on {}:{}", opts.host, opts.port);
return 1;
}
LOG_INFO("Listening on {}:{} ...", opts.host, opts.port);
loop.schedule(accept_connections(server, std::move(*acceptor), true, connections));
loop.run();
return 0;
}
LOG_ERROR("unknown server mode '{}'", opts.mode);
return 1;
}
struct DaemonConnection {
std::unique_ptr<kota::ipc::JsonPeer> peer;
std::unique_ptr<AgentClient> agent_client;
};
static kota::task<> run_daemon_connection(kota::ipc::JsonPeer* peer,
std::list<DaemonConnection>& connections,
std::list<DaemonConnection>::iterator pos) {
co_await peer->run();
LOG_INFO("Daemon client disconnected");
connections.erase(pos);
}
static kota::task<> daemon_main(MasterServer& server, kota::pipe::acceptor acceptor) {
auto& loop = kota::event_loop::current();
std::list<DaemonConnection> connections;
kota::task_group<> connection_group(loop);
co_await kota::when_all(
[&]() -> kota::task<> {
while(true) {
auto conn = co_await acceptor.accept();
if(!conn.has_value())
break;
LOG_INFO("Daemon client connected");
auto transport = std::make_unique<kota::ipc::StreamTransport>(std::move(*conn));
auto peer = std::make_unique<kota::ipc::JsonPeer>(loop, std::move(transport));
auto agent = std::make_unique<AgentClient>(server, *peer);
auto* peer_ptr = peer.get();
auto it = connections.emplace(connections.end(),
DaemonConnection{
.peer = std::move(peer),
.agent_client = std::move(agent),
});
connection_group.spawn(run_daemon_connection(peer_ptr, connections, it));
}
}(),
[&]() -> kota::task<> {
co_await server.get_shutdown_event().wait();
acceptor.stop();
for(auto& conn: connections) {
conn.peer->close();
}
}());
co_await connection_group.join();
}
int run_daemon_mode(const DaemonOptions& opts) {
logging::stderr_logger("daemon", logging::options);
auto socket_path = opts.socket_path.empty() ? path::default_socket_path() : opts.socket_path;
auto socket_dir = llvm::sys::path::parent_path(socket_path);
if(auto ec = llvm::sys::fs::create_directories(socket_dir)) {
LOG_ERROR("Failed to create socket directory {}: {}", socket_dir, ec.message());
return 1;
}
if(llvm::sys::fs::exists(socket_path)) {
#ifndef _WIN32
int fd = ::socket(AF_UNIX, SOCK_STREAM, 0);
if(fd >= 0) {
struct sockaddr_un addr{};
addr.sun_family = AF_UNIX;
auto len = std::min(socket_path.size(), sizeof(addr.sun_path) - 1);
std::memcpy(addr.sun_path, socket_path.data(), len);
bool live = ::connect(fd, reinterpret_cast<struct sockaddr*>(&addr), sizeof(addr)) == 0;
::close(fd);
if(live) {
LOG_ERROR("Another daemon is already running on {}", socket_path);
return 1;
}
}
#endif
llvm::sys::fs::remove(socket_path);
}
kota::event_loop loop;
MasterServer server(loop, opts.self_path);
if(!opts.workspace.empty()) {
server.initialize(opts.workspace);
server.start_file_watcher();
}
auto acceptor = kota::pipe::listen(socket_path, {}, loop);
if(!acceptor) {
LOG_ERROR("Failed to listen on {}", socket_path);
return 1;
}
LOG_INFO("Daemon listening on {}", socket_path);
loop.schedule(daemon_main(server, std::move(*acceptor)));
loop.run();
llvm::sys::fs::remove(socket_path);
return 0;
}
} // namespace clice

View File

@@ -0,0 +1,93 @@
#pragma once
#include <cstdint>
#include <string>
#include "server/compiler/compiler.h"
#include "server/compiler/indexer.h"
#include "server/service/session.h"
#include "server/worker/worker_pool.h"
#include "server/workspace/workspace.h"
#include "kota/async/async.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/StringRef.h"
namespace clice {
enum class ServerLifecycle : std::uint8_t {
Uninitialized,
Initialized,
Ready,
ShuttingDown,
Exited,
};
/// Core server state — owns the two-layer state model (Workspace + Sessions),
/// the worker pool, compilation engine, and indexer.
///
/// Does NOT own any transport or peer. Protocol-specific handler registration
/// is done by LSPClient and AgentClient, which access private members directly.
class MasterServer {
friend class LSPClient;
friend class AgentClient;
public:
MasterServer(kota::event_loop& loop, std::string self_path);
~MasterServer();
void initialize();
void initialize(llvm::StringRef root);
void start_file_watcher();
Session* find_session(std::uint32_t path_id);
Session& open_session(std::uint32_t path_id);
void close_session(std::uint32_t path_id, kota::ipc::JsonPeer& peer);
void on_file_saved(std::uint32_t path_id);
void schedule_shutdown();
kota::event& get_shutdown_event() {
return shutdown_event;
}
private:
kota::event shutdown_event;
void load_workspace();
kota::event_loop& loop;
Workspace workspace;
llvm::DenseMap<std::uint32_t, Session> sessions;
WorkerPool pool;
Compiler compiler;
Indexer indexer;
ServerLifecycle lifecycle = ServerLifecycle::Uninitialized;
std::string self_path;
std::string workspace_root;
std::string session_log_dir;
std::string init_options_json;
};
struct ServerOptions {
std::string mode;
std::string host = "127.0.0.1";
int port = 0;
std::string self_path;
std::string record;
};
int run_server_mode(const ServerOptions& opts);
struct DaemonOptions {
std::string socket_path;
std::string workspace;
std::string self_path;
};
int run_daemon_mode(const DaemonOptions& opts);
} // namespace clice

View File

@@ -5,7 +5,7 @@
#include <optional>
#include <string>
#include "server/workspace.h"
#include "server/workspace/workspace.h"
#include "kota/async/async.h"
#include "llvm/ADT/SmallVector.h"

View File

@@ -1,4 +1,4 @@
#include "server/stateful_worker.h"
#include "server/worker/stateful_worker.h"
#include <atomic>
#include <cstdint>
@@ -10,8 +10,8 @@
#include "compile/compilation.h"
#include "feature/feature.h"
#include "index/tu_index.h"
#include "server/protocol.h"
#include "server/worker_common.h"
#include "server/protocol/worker.h"
#include "server/worker/worker_common.h"
#include "support/logging.h"
#include "kota/async/async.h"
@@ -152,6 +152,7 @@ void StatefulWorker::register_handlers() {
CompilationParams cp;
cp.kind = CompilationKind::Content;
cp.clang_tidy = params.clang_tidy;
fill_args(cp, doc->directory, doc->arguments);
if(!doc->pch.first.empty()) {
cp.pch = doc->pch;
@@ -245,26 +246,33 @@ void StatefulWorker::register_handlers() {
co_return kota::codec::RawValue{"[]"};
case K::SemanticTokens:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::semantic_tokens(doc.unit));
return to_raw(
feature::semantic_tokens(doc.unit, feature::PositionEncoding::UTF16));
});
case K::InlayHints:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
auto range = params.range;
if(range.begin == static_cast<uint32_t>(-1))
range = LocalSourceRange{0, static_cast<uint32_t>(doc.text.size())};
return to_raw(feature::inlay_hints(doc.unit, range));
return to_raw(feature::inlay_hints(doc.unit,
range,
{},
feature::PositionEncoding::UTF16));
});
case K::FoldingRange:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::folding_ranges(doc.unit));
return to_raw(
feature::folding_ranges(doc.unit, feature::PositionEncoding::UTF16));
});
case K::DocumentSymbol:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::document_symbols(doc.unit));
return to_raw(
feature::document_symbols(doc.unit, feature::PositionEncoding::UTF16));
});
case K::DocumentLink:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::document_links(doc.unit));
return to_raw(
feature::document_links(doc.unit, feature::PositionEncoding::UTF16));
});
case K::CodeAction:
// TODO: Implement code actions

View File

@@ -1,10 +1,10 @@
#include "server/stateless_worker.h"
#include "server/worker/stateless_worker.h"
#include "compile/compilation.h"
#include "feature/feature.h"
#include "index/tu_index.h"
#include "server/protocol.h"
#include "server/worker_common.h"
#include "server/protocol/worker.h"
#include "server/worker/worker_common.h"
#include "support/logging.h"
#include "kota/async/async.h"
@@ -15,6 +15,22 @@
namespace clice {
/// RAII guard that lowers the current process's scheduling priority and
/// restores it on destruction.
struct ScopedNice {
int saved;
explicit ScopedNice(int increment = 10) {
auto p = kota::sys::priority();
saved = p ? *p : 0;
kota::sys::set_priority(saved + increment);
}
~ScopedNice() {
kota::sys::set_priority(saved);
}
};
using kota::ipc::RequestResult;
using RequestContext = kota::ipc::BincodePeer::RequestContext;
@@ -258,6 +274,22 @@ static worker::BuildResult handle_signature_help(const worker::BuildParams& para
return result;
}
static worker::BuildResult handle_format(const worker::BuildParams& params) {
ScopedTimer timer;
std::optional<LocalSourceRange> range;
if(params.format_range.valid()) {
range = params.format_range;
}
auto edits = feature::document_format(params.file, params.text, range);
LOG_DEBUG("Format done: {} edits, {}ms", edits.size(), timer.ms());
worker::BuildResult result;
result.result_json = to_raw(edits);
return result;
}
int run_stateless_worker_mode(const std::string& worker_name, const std::string& log_dir) {
logging::stderr_logger(worker_name, logging::options);
if(!log_dir.empty()) {
@@ -283,9 +315,13 @@ int run_stateless_worker_mode(const std::string& worker_name, const std::string&
switch(params.kind) {
case K::BuildPCH: return handle_build_pch(params);
case K::BuildPCM: return handle_build_pcm(params);
case K::Index: return handle_index(params);
case K::Index: {
ScopedNice guard;
return handle_index(params);
}
case K::Completion: return handle_completion(params);
case K::SignatureHelp: return handle_signature_help(params);
case K::Format: return handle_format(params);
}
return {false, "Unknown build kind"};
});

View File

@@ -8,8 +8,7 @@
#include "compile/compilation.h"
#include "kota/codec/json/serializer.h"
#include "kota/codec/raw_value.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
namespace clice {

View File

@@ -1,4 +1,4 @@
#include "server/worker_pool.h"
#include "server/worker/worker_pool.h"
#include <csignal>
#include <string>
@@ -13,14 +13,13 @@ namespace {
/// Coroutine that drains a worker's stderr pipe.
/// Workers write their own log files, so this only captures unexpected output
/// (crash stacktraces, assertion failures, etc.) that bypasses spdlog.
/// (crash stacktraces, assertion failures, sanitizer reports, etc.).
kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
std::string buffer;
while(true) {
auto result = co_await stderr_pipe.read();
if(!result.has_value()) {
if(!result.has_value())
break;
}
auto& chunk = result.value();
if(chunk.empty())
break;
@@ -34,7 +33,7 @@ kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
break;
auto line = buffer.substr(pos, nl - pos);
if(!line.empty()) {
LOG_DEBUG("{} {}", prefix, line);
LOG_WARN("{} {}", prefix, line);
}
pos = nl + 1;
}
@@ -42,7 +41,7 @@ kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
}
if(!buffer.empty()) {
LOG_DEBUG("{} {}", prefix, buffer);
LOG_WARN("{} {}", prefix, buffer);
}
}
@@ -97,9 +96,8 @@ bool WorkerPool::spawn_worker(const std::string& self_path,
std::move(spawn.stdin_pipe));
auto peer = std::make_unique<kota::ipc::BincodePeer>(loop, std::move(transport));
// Schedule stderr log collection
std::string prefix = "[" + worker_name + "]";
loop.schedule(drain_stderr(std::move(spawn.stderr_pipe), prefix));
io_group.spawn(drain_stderr(std::move(spawn.stderr_pipe), prefix));
workers.push_back(WorkerProcess{
.proc = std::move(spawn.proc),
@@ -108,24 +106,28 @@ bool WorkerPool::spawn_worker(const std::string& self_path,
});
auto& w = workers.back();
loop.schedule(w.peer->run());
w.alive = true;
io_group.spawn(w.peer->run());
return true;
}
bool WorkerPool::start(const WorkerPoolOptions& options) {
options_ = options;
log_dir_ = options.log_dir;
for(std::uint32_t i = 0; i < options.stateless_count; ++i) {
if(!spawn_worker(options.self_path, false, 0)) {
return false;
}
monitor_group.spawn(monitor_worker(stateless_workers.size() - 1, false));
}
for(std::uint32_t i = 0; i < options.stateful_count; ++i) {
if(!spawn_worker(options.self_path, true, options.worker_memory_limit)) {
return false;
}
monitor_group.spawn(monitor_worker(stateful_workers.size() - 1, true));
}
// Register evicted notification handler for each stateful worker
@@ -145,30 +147,19 @@ bool WorkerPool::start(const WorkerPoolOptions& options) {
kota::task<> WorkerPool::stop() {
LOG_INFO("WorkerPool stopping...");
shutting_down_ = true;
// Close output pipes to signal workers to exit gracefully
for(auto& w: stateless_workers) {
for(auto& w: stateless_workers)
w.peer->close_output();
}
for(auto& w: stateful_workers) {
for(auto& w: stateful_workers)
w.peer->close_output();
}
// Send SIGTERM to all workers
for(auto& w: stateless_workers) {
for(auto& w: stateless_workers)
w.proc.kill(SIGTERM);
}
for(auto& w: stateful_workers) {
for(auto& w: stateful_workers)
w.proc.kill(SIGTERM);
}
// Wait for all worker processes to exit
for(auto& w: stateless_workers) {
co_await w.proc.wait();
}
for(auto& w: stateful_workers) {
co_await w.proc.wait();
}
co_await kota::when_all(monitor_group.join(), io_group.join());
LOG_INFO("WorkerPool stopped");
}
@@ -198,7 +189,10 @@ std::size_t WorkerPool::assign_worker(std::uint32_t path_id) {
std::size_t WorkerPool::pick_least_loaded() {
std::size_t best = 0;
for(std::size_t i = 1; i < stateful_workers.size(); ++i) {
if(stateful_workers[i].owned_documents < stateful_workers[best].owned_documents) {
if(!stateful_workers[i].alive)
continue;
if(!stateful_workers[best].alive ||
stateful_workers[i].owned_documents < stateful_workers[best].owned_documents) {
best = i;
}
}
@@ -233,4 +227,122 @@ void WorkerPool::clear_owner(std::size_t worker_index) {
}
}
kota::task<> WorkerPool::monitor_worker(std::size_t index, bool stateful) {
auto& workers = stateful ? stateful_workers : stateless_workers;
auto& w = workers[index];
auto name = std::string(stateful ? "SF-" : "SL-") + std::to_string(index);
auto result = co_await w.proc.wait();
w.alive = false;
if(shutting_down_)
co_return;
if(result.has_value()) {
auto& exit = result.value();
if(exit.term_signal != 0) {
LOG_ERROR("Worker {} killed by signal {} (restarts: {})",
name,
exit.term_signal,
w.restart_count);
} else {
LOG_ERROR("Worker {} exited with code {} (restarts: {})",
name,
exit.status,
w.restart_count);
}
} else {
LOG_ERROR("Worker {} lost: {} (restarts: {})",
name,
result.error().message(),
w.restart_count);
}
if(stateful)
clear_owner(index);
constexpr unsigned max_restarts = 5;
if(w.restart_count >= max_restarts) {
LOG_ERROR("Worker {} exceeded max restarts ({}), giving up", name, max_restarts);
co_return;
}
if(!respawn_worker(index, stateful)) {
LOG_ERROR("Worker {} respawn failed", name);
}
}
bool WorkerPool::respawn_worker(std::size_t index, bool stateful) {
auto& workers = stateful ? stateful_workers : stateless_workers;
auto old_restart_count = workers[index].restart_count + 1;
auto worker_name = std::string(stateful ? "SF-" : "SL-") + std::to_string(index);
// Close the old peer and retire it so its coroutines (run/write_loop)
// can finish naturally before the object is destroyed.
if(workers[index].peer) {
workers[index].peer->close();
retired_peers.push_back(std::move(workers[index].peer));
}
kota::process::options opts;
opts.file = options_.self_path;
if(stateful) {
opts.args = {options_.self_path,
"--mode",
"stateful-worker",
"--worker-memory-limit",
std::to_string(options_.worker_memory_limit)};
} else {
opts.args = {options_.self_path, "--mode", "stateless-worker"};
}
opts.args.push_back("--worker-name");
opts.args.push_back(worker_name);
if(!log_dir_.empty()) {
opts.args.push_back("--log-dir");
opts.args.push_back(log_dir_);
}
opts.streams = {
kota::process::stdio::pipe(true, false),
kota::process::stdio::pipe(false, true),
kota::process::stdio::pipe(false, true),
};
auto result = kota::process::spawn(opts, loop);
if(!result) {
LOG_ERROR("Failed to respawn worker {}: {}", worker_name, result.error().message());
return false;
}
auto& spawn = *result;
auto transport = std::make_unique<kota::ipc::StreamTransport>(std::move(spawn.stdout_pipe),
std::move(spawn.stdin_pipe));
auto peer = std::make_unique<kota::ipc::BincodePeer>(loop, std::move(transport));
std::string prefix = "[" + worker_name + "]";
io_group.spawn(drain_stderr(std::move(spawn.stderr_pipe), prefix));
workers[index] = WorkerProcess{
.proc = std::move(spawn.proc),
.peer = std::move(peer),
.owned_documents = 0,
.alive = true,
.restart_count = old_restart_count,
};
auto& w = workers[index];
io_group.spawn(w.peer->run());
if(stateful) {
w.peer->on_notification([this](const worker::EvictedParams& params) {
if(on_evicted)
on_evicted(params.path);
});
}
monitor_group.spawn(monitor_worker(index, stateful));
LOG_INFO("Worker {} restarted (attempt {})", worker_name, old_restart_count);
return true;
}
} // namespace clice

View File

@@ -6,7 +6,7 @@
#include <list>
#include <memory>
#include "server/protocol.h"
#include "server/protocol/worker.h"
#include "kota/async/async.h"
#include "kota/ipc/codec/bincode.h"
@@ -64,6 +64,8 @@ private:
kota::process proc;
std::unique_ptr<kota::ipc::BincodePeer> peer;
std::size_t owned_documents = 0;
bool alive = true;
unsigned restart_count = 0;
};
kota::event_loop& loop;
@@ -80,8 +82,19 @@ private:
void clear_owner(std::size_t worker_index);
std::size_t pick_least_loaded();
bool shutting_down_ = false;
kota::task_group<> monitor_group{loop};
kota::task_group<> io_group{loop};
WorkerPoolOptions options_;
std::string log_dir_;
/// Peers moved here during respawn so their coroutines can finish
/// before the object is destroyed.
llvm::SmallVector<std::unique_ptr<kota::ipc::BincodePeer>> retired_peers;
bool spawn_worker(const std::string& self_path, bool stateful, std::uint64_t memory_limit);
bool respawn_worker(std::size_t index, bool stateful);
kota::task<> monitor_worker(std::size_t index, bool stateful);
};
template <typename Params>
@@ -91,11 +104,10 @@ RequestResult<Params> WorkerPool::send_stateful(std::uint32_t path_id,
if(stateful_workers.empty()) {
co_return kota::outcome_error(kota::ipc::Error{"No stateful workers available"});
}
// No timeout: compile tasks run as detached tasks (loop.schedule) that
// are immune to LSP $/cancelRequest. Adding a timeout here would use
// kotatsu's with_token/when_any which has a spurious-cancellation bug
// that kills requests within milliseconds instead of the configured period.
auto idx = assign_worker(path_id);
if(!stateful_workers[idx].alive) {
co_return kota::outcome_error(kota::ipc::Error{"Assigned stateful worker is down"});
}
co_return co_await stateful_workers[idx].peer->send_request(params, opts);
}
@@ -105,9 +117,16 @@ RequestResult<Params> WorkerPool::send_stateless(const Params& params,
if(stateless_workers.empty()) {
co_return kota::outcome_error(kota::ipc::Error{"No stateless workers available"});
}
auto idx = next_stateless;
next_stateless = (next_stateless + 1) % stateless_workers.size();
co_return co_await stateless_workers[idx].peer->send_request(params, opts);
// Round-robin, skipping dead workers.
auto start = next_stateless;
for(std::size_t i = 0; i < stateless_workers.size(); ++i) {
auto idx = (start + i) % stateless_workers.size();
if(stateless_workers[idx].alive) {
next_stateless = (idx + 1) % stateless_workers.size();
co_return co_await stateless_workers[idx].peer->send_request(params, opts);
}
}
co_return kota::outcome_error(kota::ipc::Error{"All stateless workers are down"});
}
template <typename Params>
@@ -115,6 +134,8 @@ void WorkerPool::notify_stateful(std::uint32_t path_id, const Params& params) {
auto it = owner.find(path_id);
if(it == owner.end())
return;
if(!stateful_workers[it->second].alive)
return;
stateful_workers[it->second].peer->send_notification(params);
}

View File

@@ -1,4 +1,4 @@
#include "server/config.h"
#include "server/workspace/config.h"
#include <algorithm>
@@ -6,8 +6,9 @@
#include "support/glob_pattern.h"
#include "support/logging.h"
#include "kota/async/io/system.h"
#include "kota/codec/json/json.h"
#include "kota/codec/toml.h"
#include "kota/codec/toml/toml.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/Process.h"
@@ -65,8 +66,10 @@ void Config::apply_defaults(llvm::StringRef workspace_root) {
if(p.stateful_worker_count == 0)
p.stateful_worker_count = 2;
if(p.stateless_worker_count == 0)
p.stateless_worker_count = 3;
if(p.stateless_worker_count == 0) {
auto cores = kota::sys::parallelism();
p.stateless_worker_count = std::max(cores / 2, 2u);
}
if(p.worker_memory_limit == 0)
p.worker_memory_limit = 4ULL * 1024 * 1024 * 1024; // 4GB
@@ -153,13 +156,13 @@ std::optional<Config> Config::load(llvm::StringRef path, llvm::StringRef workspa
}
std::optional<Config> Config::load_from_json(llvm::StringRef json, llvm::StringRef workspace_root) {
auto result = kota::codec::json::from_json<Config>(json);
Config config{};
auto result = kota::codec::json::from_json(json, config);
if(!result) {
LOG_WARN("Failed to parse initializationOptions JSON: {}", result.error().message());
LOG_WARN("Failed to parse initializationOptions JSON: {}", result.error().message);
return std::nullopt;
}
auto config = std::move(*result);
config.apply_defaults(workspace_root);
LOG_INFO("Loaded config from initializationOptions");
return config;

View File

@@ -1,4 +1,4 @@
#include "server/workspace.h"
#include "server/workspace/workspace.h"
#include <algorithm>
#include <chrono>

View File

@@ -11,8 +11,8 @@
#include "index/merged_index.h"
#include "index/project_index.h"
#include "semantic/relation_kind.h"
#include "server/compile_graph.h"
#include "server/config.h"
#include "server/compiler/compile_graph.h"
#include "server/workspace/config.h"
#include "support/path_pool.h"
#include "syntax/dependency_graph.h"

View File

@@ -37,6 +37,14 @@ inline std::string real_path(llvm::StringRef file) {
return path.str().str();
}
inline std::string default_socket_path() {
llvm::SmallString<128> home;
if(!llvm::sys::path::home_directory(home))
return "/tmp/clice.sock";
llvm::sys::path::append(home, ".clice", "clice.sock");
return home.str().str();
}
} // namespace path
namespace fs {

View File

@@ -10,7 +10,6 @@
#include "clang/Basic/FileEntry.h"
#include "clang/Basic/FileManager.h"
#include "clang/Basic/SourceManager.h"
#include "clang/Driver/CreateInvocationFromArgs.h"
#include "clang/Frontend/CompilerInstance.h"
#include "clang/Frontend/FrontendActions.h"
#include "clang/Lex/PPCallbacks.h"
@@ -320,10 +319,9 @@ std::unique_ptr<clang::CompilerInstance>
}
auto instance = std::make_unique<clang::CompilerInstance>(std::move(invocation));
instance->createDiagnostics(new clang::IgnoringDiagConsumer(), true);
instance->createDiagnostics(*vfs, new clang::IgnoringDiagConsumer(), true);
instance->getDiagnostics().setSuppressAllDiagnostics(true);
instance->setVirtualFileSystem(vfs);
instance->createFileManager();
instance->createFileManager(vfs);
return instance;
}

View File

@@ -1,6 +1,7 @@
import asyncio
import json
import shutil
import socket
import subprocess
import sys
from pathlib import Path
@@ -93,24 +94,27 @@ def workspace(request: pytest.FixtureRequest, test_data_dir: Path) -> Path | Non
@pytest.fixture
async def client(
request: pytest.FixtureRequest, executable: Path, workspace: Path | None
request: pytest.FixtureRequest,
executable: Path,
workspace: Path | None,
):
"""Spawn clice server, auto-initialize if @pytest.mark.workspace is present."""
config = request.config
mode = config.getoption("--mode")
host = config.getoption("--host")
cmd = [str(executable), "--mode", mode]
if mode == "socket":
host = config.getoption("--host")
port = config.getoption("--port")
cmd += ["--host", host, "--port", str(port)]
cmd = [str(executable), "--mode", mode, "--host", host]
c = CliceClient()
await c.start_io(*cmd)
if workspace is not None:
init_options_marker = request.node.get_closest_marker("init_options")
init_options = init_options_marker.args[0] if init_options_marker else None
init_options = dict(init_options_marker.args[0]) if init_options_marker else {}
# Force cache_dir into the workspace so .clice/ cleanup prevents stale PCH.
project = dict(init_options.get("project", {}))
project.setdefault("cache_dir", str(workspace / ".clice"))
init_options["project"] = project
await c.initialize(workspace, initialization_options=init_options)
yield c
@@ -118,6 +122,39 @@ async def client(
await _shutdown_client(c)
def _find_free_port() -> int:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(("127.0.0.1", 0))
return s.getsockname()[1]
@pytest.fixture
async def agentic(
request: pytest.FixtureRequest,
executable: Path,
workspace: Path | None,
):
"""Start a server with agentic TCP port, yield (executable, host, port)."""
host = "127.0.0.1"
port = _find_free_port()
cmd = [str(executable), "--mode", "pipe", "--host", host, "--port", str(port)]
c = CliceClient()
await c.start_io(*cmd)
if workspace is not None:
init_options_marker = request.node.get_closest_marker("init_options")
init_options = dict(init_options_marker.args[0]) if init_options_marker else {}
project = dict(init_options.get("project", {}))
project.setdefault("cache_dir", str(workspace / ".clice"))
init_options["project"] = project
await c.initialize(workspace, initialization_options=init_options)
yield executable, host, port
await _shutdown_client(c)
def generate_cdb(workspace: Path) -> None:
"""Generate compile_commands.json using CMake with Ninja backend."""
cmake = shutil.which("cmake")
@@ -165,12 +202,17 @@ async def _shutdown_client(c: CliceClient) -> None:
try:
server = getattr(c, "_server", None)
if server and server.stderr:
stderr_data = await asyncio.wait_for(server.stderr.read(), timeout=2.0)
if stderr_data:
for line in stderr_data.decode("utf-8", errors="replace").splitlines():
if "[warn]" in line or "[error]" in line:
print(f"[server] {line}", flush=True)
if server:
if server.returncode is not None:
print(f"[server] exit code: {server.returncode}", flush=True)
if server.stderr:
stderr_data = await asyncio.wait_for(server.stderr.read(), timeout=2.0)
if stderr_data:
for line in stderr_data.decode(
"utf-8", errors="replace"
).splitlines():
if "[warn]" in line or "[error]" in line or "Sanitizer" in line:
print(f"[server] {line}", flush=True)
except Exception:
pass
@@ -250,6 +292,12 @@ def _generate_test_data_cdbs(data_dir: Path) -> None:
if cr_main.exists():
_write(cr_dir, [_entry(cr_dir, cr_main)])
# formatting
fmt_dir = data_dir / "formatting"
fmt_main = fmt_dir / "main.cpp"
if fmt_main.exists():
_write(fmt_dir, [_entry(fmt_dir, fmt_main)])
# pch_test
pt_dir = data_dir / "pch_test"
if pt_dir.exists():

View File

@@ -0,0 +1,36 @@
// basic if and if-else
namespace basic_if {
int abs_val(int x) {
if(x < 0)
return -x;
return x;
}
const char* sign(int x) {
if(x > 0) {
return "positive";
} else if(x < 0) {
return "negative";
} else {
return "zero";
}
}
// dangling else: else binds to nearest if
int nested_if(int a, int b) {
if(a > 0)
if(b > 0)
return 1;
else
return 2;
return 0;
}
void test() {
[[maybe_unused]] int r1 = abs_val(-3);
[[maybe_unused]] auto r2 = sign(5);
[[maybe_unused]] int r3 = nested_if(1, -1);
}
} // namespace basic_if

View File

@@ -0,0 +1,3 @@
BasedOnStyle: LLVM
IndentWidth: 4
ColumnLimit: 80

View File

@@ -0,0 +1 @@
int add(int a, int b) { return a + b; }

View File

View File

@@ -0,0 +1,592 @@
"""Tests for the agentic protocol handlers."""
import asyncio
import json
import socket
import subprocess
from concurrent.futures import ThreadPoolExecutor
import pytest
from tests.integration.utils.wait import wait_for_index
class AgenticRpcClient:
"""Minimal JSON-RPC client that speaks Content-Length framing over TCP."""
def __init__(self, host: str, port: int):
self.sock = socket.create_connection((host, port), timeout=10)
self.request_id = 0
self.buffer = b""
def request(self, method: str, params: dict):
self.request_id += 1
body = json.dumps(
{
"jsonrpc": "2.0",
"id": self.request_id,
"method": method,
"params": params,
}
)
payload = f"Content-Length: {len(body)}\r\n\r\n{body}".encode("utf-8")
self.sock.sendall(payload)
return self._read_response()
def _read_response(self):
while b"\r\n\r\n" not in self.buffer:
data = self.sock.recv(4096)
if not data:
raise ConnectionError("connection closed")
self.buffer += data
header_end = self.buffer.index(b"\r\n\r\n")
headers = self.buffer[:header_end].decode("utf-8")
self.buffer = self.buffer[header_end + 4 :]
content_length = 0
for line in headers.split("\r\n"):
if line.lower().startswith("content-length:"):
content_length = int(line.split(":")[1].strip())
while len(self.buffer) < content_length:
data = self.sock.recv(4096)
if not data:
raise ConnectionError("connection closed")
self.buffer += data
body = self.buffer[:content_length].decode("utf-8")
self.buffer = self.buffer[content_length:]
return json.loads(body)
def close(self):
self.sock.close()
def run_agentic(executable, host, port, path, timeout=10):
result = subprocess.run(
[
str(executable),
"--mode",
"agentic",
"--host",
host,
"--port",
str(port),
"--path",
path,
],
capture_output=True,
text=True,
timeout=timeout,
)
return result
@pytest.mark.workspace("hello_world")
async def test_compile_command(agentic, workspace):
executable, host, port = agentic
main_cpp = (workspace / "main.cpp").as_posix()
result = run_agentic(executable, host, port, main_cpp)
assert result.returncode == 0, f"stderr: {result.stderr}"
data = json.loads(result.stdout)
assert data["file"] == main_cpp
assert data["directory"] == workspace.as_posix()
assert len(data["arguments"]) > 0
@pytest.mark.workspace("hello_world")
async def test_compile_command_fallback(agentic, workspace):
executable, host, port = agentic
result = run_agentic(executable, host, port, "/nonexistent/file.cpp")
assert result.returncode == 0, f"stderr: {result.stderr}"
data = json.loads(result.stdout)
assert data["file"] == "/nonexistent/file.cpp"
@pytest.mark.workspace("hello_world")
async def test_multiple_requests(agentic, workspace):
executable, host, port = agentic
main_cpp = (workspace / "main.cpp").as_posix()
for _ in range(3):
result = run_agentic(executable, host, port, main_cpp)
assert result.returncode == 0, f"stderr: {result.stderr}"
data = json.loads(result.stdout)
assert data["file"] == main_cpp
async def test_connection_refused(executable):
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(("127.0.0.1", 0))
free_port = s.getsockname()[1]
result = run_agentic(executable, "127.0.0.1", free_port, "/some/file.cpp")
assert result.returncode != 0
@pytest.mark.workspace("hello_world")
async def test_concurrent_connections(agentic, workspace):
executable, host, port = agentic
main_cpp = (workspace / "main.cpp").as_posix()
def do_request(_):
return run_agentic(executable, host, port, main_cpp)
with ThreadPoolExecutor(max_workers=4) as pool:
results = list(pool.map(do_request, range(4)))
for r in results:
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["file"] == main_cpp
@pytest.fixture
async def indexed_agentic(request, executable, workspace):
"""Start server with LSP+agentic, compile a file, wait for indexing."""
from tests.integration.utils.client import CliceClient
from tests.conftest import _shutdown_client, _find_free_port
host = "127.0.0.1"
port = _find_free_port()
cmd = [str(executable), "--mode", "pipe", "--host", host, "--port", str(port)]
c = CliceClient()
await c.start_io(*cmd)
init_options = {"project": {"cache_dir": str(workspace / ".clice")}}
await c.initialize(workspace, initialization_options=init_options)
uri, _ = await c.open_and_wait(workspace / "main.cpp")
assert await wait_for_index(c, uri, "add"), "Index not ready"
rpc = AgenticRpcClient(host, port)
for _ in range(30):
resp = rpc.request("agentic/symbolSearch", {"query": "add"})
if "result" in resp and resp["result"]["symbols"]:
break
await asyncio.sleep(1)
else:
pytest.fail("agentic/symbolSearch never returned indexed symbols")
yield rpc, workspace
rpc.close()
c.close(uri)
await _shutdown_client(c)
@pytest.mark.workspace("index_features")
async def test_rpc_compile_command(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/compileCommand", {"path": path})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["file"] == path
assert len(result["arguments"]) > 0
@pytest.mark.workspace("index_features")
async def test_rpc_project_files(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/projectFiles", {})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["total"] > 0
paths = [f["path"] for f in result["files"]]
assert any("main.cpp" in p for p in paths)
@pytest.mark.workspace("index_features")
async def test_rpc_project_files_filter(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/projectFiles", {"filter": "source"})
assert "result" in resp
for f in resp["result"]["files"]:
assert f["kind"] == "source"
@pytest.mark.workspace("index_features")
async def test_rpc_symbol_search(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/symbolSearch", {"query": "add"})
assert "result" in resp, f"unexpected response: {resp}"
symbols = resp["result"]["symbols"]
add_sym = next((s for s in symbols if s["name"] == "add"), None)
assert add_sym is not None, f"'add' not found in {[s['name'] for s in symbols]}"
assert add_sym["kind"] == "Function"
assert add_sym["line"] == 19
assert add_sym["symbolId"] != 0
assert "main.cpp" in add_sym["file"]
@pytest.mark.workspace("index_features")
async def test_rpc_symbol_search_kind(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request(
"agentic/symbolSearch", {"query": "Animal", "kindFilter": ["Struct"]}
)
assert "result" in resp
for s in resp["result"]["symbols"]:
assert s["kind"] == "Struct"
@pytest.mark.workspace("index_features")
async def test_rpc_symbol_search_max(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/symbolSearch", {"query": "", "maxResults": 3})
assert "result" in resp
assert len(resp["result"]["symbols"]) <= 3
@pytest.mark.workspace("index_features")
async def test_rpc_read_symbol(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/readSymbol", {"name": "add"})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["name"] == "add"
assert result["symbolId"] != 0
assert result["startLine"] == 19
assert result["endLine"] == 21
assert "int add(int a, int b)" in result["text"]
assert "return a + b;" in result["text"]
@pytest.mark.workspace("index_features")
async def test_rpc_read_symbol_by_id(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp1 = rpc.request("agentic/readSymbol", {"name": "add"})
assert "result" in resp1
sid = resp1["result"]["symbolId"]
resp2 = rpc.request("agentic/readSymbol", {"symbolId": sid})
assert "result" in resp2
assert resp2["result"]["name"] == "add"
assert resp2["result"]["symbolId"] == sid
@pytest.mark.workspace("index_features")
async def test_rpc_document_symbols(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/documentSymbols", {"path": path})
assert "result" in resp, f"unexpected response: {resp}"
symbols = resp["result"]["symbols"]
names = [s["name"] for s in symbols]
kinds = [s["kind"] for s in symbols]
assert "add" in names, f"expected 'add' in {names}"
assert "main" in names, f"expected 'main' in {names}"
assert "global_var" in names, f"expected 'global_var' in {names}"
assert "Parameter" not in kinds, (
f"Parameters should be filtered: {list(zip(names, kinds))}"
)
@pytest.mark.workspace("index_features")
async def test_rpc_definition(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/definition", {"name": "add"})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["name"] == "add"
assert result["definition"] is not None
defn = result["definition"]
assert "main.cpp" in defn["file"]
assert defn["startLine"] == 19
assert defn["endLine"] == 21
assert "int add(int a, int b)" in defn["text"]
assert "return a + b;" in defn["text"]
@pytest.mark.workspace("index_features")
async def test_rpc_definition_by_position(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/definition", {"path": path, "line": 19})
assert "result" in resp, f"unexpected response: {resp}"
assert resp["result"]["name"] == "add"
@pytest.mark.workspace("index_features")
async def test_rpc_references(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/references", {"name": "global_var"})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["name"] == "global_var"
assert result["total"] == 2
lines = sorted(r["line"] for r in result["references"])
assert lines == [34, 38]
contexts = [r["context"] for r in result["references"]]
assert any("global_var + 1" in c for c in contexts)
assert any("global_var * 2" in c for c in contexts)
@pytest.mark.workspace("index_features")
async def test_rpc_references_include_decl(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request(
"agentic/references", {"name": "global_var", "includeDeclaration": True}
)
assert "result" in resp
result = resp["result"]
assert result["total"] == 3
lines = sorted(r["line"] for r in result["references"])
assert 31 in lines, f"expected declaration line 31 in {lines}"
@pytest.mark.workspace("index_features")
async def test_rpc_call_graph_incoming(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/callGraph", {"name": "add", "direction": "callers"})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["root"]["name"] == "add"
assert result["root"]["line"] == 19
assert result["root"]["symbolId"] != 0
callers = result["callers"]
caller_names = [c["name"] for c in callers]
assert "compute" in caller_names, f"expected 'compute' in {caller_names}"
compute = next(c for c in callers if c["name"] == "compute")
assert compute["line"] == 24
assert compute["symbolId"] != 0
assert result["callees"] == []
@pytest.mark.workspace("index_features")
async def test_rpc_call_graph_outgoing(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/callGraph", {"name": "compute", "direction": "callees"})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["root"]["name"] == "compute"
callees = result["callees"]
callee_names = [c["name"] for c in callees]
assert "add" in callee_names, f"expected 'add' in {callee_names}"
add_entry = next(c for c in callees if c["name"] == "add")
assert add_entry["line"] == 19
assert result["callers"] == []
@pytest.mark.workspace("index_features")
async def test_rpc_type_hierarchy_supertypes(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request(
"agentic/typeHierarchy", {"name": "Dog", "direction": "supertypes"}
)
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["root"]["name"] == "Dog"
assert result["root"]["line"] == 9
supertypes = result["supertypes"]
supertype_names = [t["name"] for t in supertypes]
assert "Animal" in supertype_names, f"expected 'Animal' in {supertype_names}"
animal = next(t for t in supertypes if t["name"] == "Animal")
assert animal["line"] == 2
assert animal["symbolId"] != 0
@pytest.mark.workspace("index_features")
async def test_rpc_type_hierarchy_subtypes(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request(
"agentic/typeHierarchy", {"name": "Animal", "direction": "subtypes"}
)
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["root"]["name"] == "Animal"
assert result["root"]["line"] == 2
subtypes = result["subtypes"]
subtype_names = [t["name"] for t in subtypes]
assert "Dog" in subtype_names, f"expected 'Dog' in {subtype_names}"
assert "Cat" in subtype_names, f"expected 'Cat' in {subtype_names}"
dog = next(t for t in subtypes if t["name"] == "Dog")
assert dog["line"] == 9
cat = next(t for t in subtypes if t["name"] == "Cat")
assert cat["line"] == 14
@pytest.mark.workspace("index_features")
async def test_rpc_status(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/status", {})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert isinstance(result["idle"], bool)
assert result["total"] > 0
assert isinstance(result["pending"], int)
assert isinstance(result["indexed"], int)
@pytest.mark.workspace("hello_world")
async def test_rpc_shutdown(executable, workspace):
"""Shutdown notification should cause the server to exit."""
from tests.integration.utils.client import CliceClient
from tests.conftest import _shutdown_client, _find_free_port
host = "127.0.0.1"
port = _find_free_port()
cmd = [str(executable), "--mode", "pipe", "--host", host, "--port", str(port)]
c = CliceClient()
await c.start_io(*cmd)
init_options = {"project": {"cache_dir": str(workspace / ".clice")}}
await c.initialize(workspace, initialization_options=init_options)
rpc = AgenticRpcClient(host, port)
body = json.dumps({"jsonrpc": "2.0", "method": "agentic/shutdown", "params": {}})
rpc.sock.sendall(f"Content-Length: {len(body)}\r\n\r\n{body}".encode())
rpc.sock.settimeout(5)
try:
rpc.sock.recv(4096)
except (socket.timeout, OSError):
pass
rpc.sock.close()
import asyncio
for _ in range(20):
if c._server.returncode is not None:
break
await asyncio.sleep(0.5)
assert c._server.returncode is not None, "Server did not exit after shutdown"
@pytest.mark.workspace("index_features")
async def test_rpc_symbol_not_found(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/definition", {"name": "nonexistent_symbol_xyz"})
assert "error" in resp
@pytest.mark.workspace("index_features")
async def test_rpc_symbol_id_roundtrip(indexed_agentic, workspace):
"""Search -> get symbolId -> definition -> verify consistency."""
rpc, _ = indexed_agentic
search = rpc.request("agentic/symbolSearch", {"query": "compute"})
assert "result" in search
symbols = search["result"]["symbols"]
compute = next((s for s in symbols if s["name"] == "compute"), None)
assert compute is not None, f"'compute' not found in {[s['name'] for s in symbols]}"
defn = rpc.request("agentic/definition", {"symbolId": compute["symbolId"]})
assert "result" in defn
assert defn["result"]["name"] == "compute"
assert defn["result"]["symbolId"] == compute["symbolId"]
@pytest.mark.workspace("index_features")
async def test_rpc_file_deps(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/fileDeps", {"path": path})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert result["file"] == path
assert isinstance(result["includes"], list)
assert isinstance(result["includers"], list)
@pytest.mark.workspace("index_features")
async def test_rpc_file_deps_direction(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/fileDeps", {"path": path, "direction": "includes"})
assert "result" in resp
assert resp["result"]["includers"] == []
@pytest.mark.workspace("index_features")
async def test_rpc_file_deps_unknown(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/fileDeps", {"path": "/nonexistent/file.cpp"})
assert "result" in resp
assert resp["result"]["includes"] == []
assert resp["result"]["includers"] == []
@pytest.mark.workspace("index_features")
async def test_rpc_impact_analysis(indexed_agentic, workspace):
rpc, _ = indexed_agentic
path = (workspace / "main.cpp").as_posix()
resp = rpc.request("agentic/impactAnalysis", {"path": path})
assert "result" in resp, f"unexpected response: {resp}"
result = resp["result"]
assert isinstance(result["directDependents"], list)
assert isinstance(result["transitiveDependents"], list)
assert isinstance(result["affectedModules"], list)
@pytest.mark.workspace("index_features")
async def test_rpc_impact_analysis_unknown(indexed_agentic, workspace):
rpc, _ = indexed_agentic
resp = rpc.request("agentic/impactAnalysis", {"path": "/nonexistent/file.cpp"})
assert "result" in resp
assert resp["result"]["directDependents"] == []
async def test_shutdown_during_indexing(executable, tmp_path):
"""Shutdown during active background indexing must exit cleanly."""
from tests.integration.utils.client import CliceClient
from tests.conftest import _find_free_port
workspace = tmp_path / "ws"
workspace.mkdir()
entries = []
for i in range(20):
src = workspace / f"file_{i}.cpp"
src.write_text(
f"struct Type_{i} {{ int v = {i}; void m() {{}} }};\n"
f"int func_{i}(int x) {{ return x + {i}; }}\n"
f"int caller_{i}() {{ return func_{i}({i}); }}\n"
)
entries.append(
{
"directory": workspace.as_posix(),
"file": src.as_posix(),
"arguments": ["clang++", "-std=c++17", "-fsyntax-only", src.as_posix()],
}
)
(workspace / "compile_commands.json").write_text(json.dumps(entries))
host = "127.0.0.1"
port = _find_free_port()
cmd = [str(executable), "--mode", "pipe", "--host", host, "--port", str(port)]
c = CliceClient()
await c.start_io(*cmd)
init_options = {
"project": {
"cache_dir": str(workspace / ".clice"),
"idle_timeout_ms": 0,
}
}
await c.initialize(workspace, initialization_options=init_options)
# Give indexing a moment to start, then send shutdown
await asyncio.sleep(0.5)
rpc = AgenticRpcClient(host, port)
body = json.dumps({"jsonrpc": "2.0", "method": "agentic/shutdown", "params": {}})
rpc.sock.sendall(f"Content-Length: {len(body)}\r\n\r\n{body}".encode())
rpc.sock.settimeout(5)
try:
rpc.sock.recv(4096)
except (socket.timeout, OSError):
pass
rpc.sock.close()
for _ in range(30):
if c._server.returncode is not None:
break
await asyncio.sleep(0.5)
assert c._server.returncode is not None, "Server did not exit after shutdown"
assert c._server.returncode >= 0, (
f"Server crashed with signal {-c._server.returncode}"
)

View File

@@ -0,0 +1,189 @@
"""CLI-based tests for agentic mode — run clice --mode agentic as a subprocess."""
import json
import subprocess
import pytest
from tests.integration.utils.wait import wait_for_index
def run_cli(executable, host, port, method, **kwargs):
cmd = [
str(executable),
"--mode",
"agentic",
"--host",
host,
"--port",
str(port),
"--method",
method,
]
for k, v in kwargs.items():
cmd.extend([f"--{k}", str(v)])
result = subprocess.run(cmd, capture_output=True, text=True, timeout=10)
return result
@pytest.fixture
async def indexed_server(request, executable, workspace):
"""Start server with LSP+agentic, compile a file, wait for indexing."""
import asyncio
from tests.integration.utils.client import CliceClient
from tests.conftest import _shutdown_client, _find_free_port
host = "127.0.0.1"
port = _find_free_port()
cmd = [str(executable), "--mode", "pipe", "--host", host, "--port", str(port)]
c = CliceClient()
await c.start_io(*cmd)
init_options = {"project": {"cache_dir": str(workspace / ".clice")}}
await c.initialize(workspace, initialization_options=init_options)
uri, _ = await c.open_and_wait(workspace / "main.cpp")
assert await wait_for_index(c, uri, "add"), "Index not ready"
from tests.integration.agentic.test_agentic import AgenticRpcClient
rpc = AgenticRpcClient(host, port)
for _ in range(30):
resp = rpc.request("agentic/symbolSearch", {"query": "add"})
if "result" in resp and resp["result"]["symbols"]:
break
await asyncio.sleep(1)
rpc.close()
yield executable, host, port, workspace
c.close(uri)
await _shutdown_client(c)
@pytest.mark.workspace("index_features")
async def test_cli_compile_command(indexed_server, workspace):
exe, host, port, _ = indexed_server
path = (workspace / "main.cpp").as_posix()
r = run_cli(exe, host, port, "compileCommand", path=path)
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["file"] == path
assert len(data["arguments"]) > 0
@pytest.mark.workspace("index_features")
async def test_cli_symbol_search(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "symbolSearch", query="add")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
names = [s["name"] for s in data["symbols"]]
assert "add" in names
add_sym = next(s for s in data["symbols"] if s["name"] == "add")
assert add_sym["kind"] == "Function"
assert add_sym["line"] == 19
@pytest.mark.workspace("index_features")
async def test_cli_definition(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "definition", name="add")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["name"] == "add"
defn = data["definition"]
assert defn["startLine"] == 19
assert defn["endLine"] == 21
assert "return a + b;" in defn["text"]
@pytest.mark.workspace("index_features")
async def test_cli_definition_by_position(indexed_server, workspace):
exe, host, port, _ = indexed_server
path = (workspace / "main.cpp").as_posix()
r = run_cli(exe, host, port, "definition", path=path, line=19)
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["name"] == "add"
@pytest.mark.workspace("index_features")
async def test_cli_references(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "references", name="global_var")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["name"] == "global_var"
assert data["total"] == 2
lines = sorted(ref["line"] for ref in data["references"])
assert lines == [34, 38]
@pytest.mark.workspace("index_features")
async def test_cli_read_symbol(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "readSymbol", name="compute")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["name"] == "compute"
assert "add(1, 2)" in data["text"]
@pytest.mark.workspace("index_features")
async def test_cli_document_symbols(indexed_server, workspace):
exe, host, port, _ = indexed_server
path = (workspace / "main.cpp").as_posix()
r = run_cli(exe, host, port, "documentSymbols", path=path)
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
names = [s["name"] for s in data["symbols"]]
assert "add" in names
assert "main" in names
assert "global_var" in names
@pytest.mark.workspace("index_features")
async def test_cli_call_graph(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "callGraph", name="add", direction="callers")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["root"]["name"] == "add"
caller_names = [c["name"] for c in data["callers"]]
assert "compute" in caller_names
@pytest.mark.workspace("index_features")
async def test_cli_type_hierarchy(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "typeHierarchy", name="Dog", direction="supertypes")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["root"]["name"] == "Dog"
supertype_names = [t["name"] for t in data["supertypes"]]
assert "Animal" in supertype_names
@pytest.mark.workspace("index_features")
async def test_cli_project_files(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "projectFiles")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert data["total"] > 0
paths = [f["path"] for f in data["files"]]
assert any("main.cpp" in p for p in paths)
@pytest.mark.workspace("index_features")
async def test_cli_status(indexed_server, workspace):
exe, host, port, _ = indexed_server
r = run_cli(exe, host, port, "status")
assert r.returncode == 0, f"stderr: {r.stderr}"
data = json.loads(r.stdout)
assert isinstance(data["idle"], bool)
assert data["total"] > 0
assert isinstance(data["pending"], int)
assert isinstance(data["indexed"], int)

View File

@@ -0,0 +1,74 @@
import pytest
from lsprotocol.types import Position, Range
from tests.integration.utils.workspace import did_change
UNFORMATTED = "int add( int a , int b ) {\nreturn a+b ;\n}\n"
FORMATTED = "int add(int a, int b) { return a + b; }\n"
def apply_edits(text, edits):
"""Apply LSP TextEdits to a string, processing from end to start."""
lines = text.split("\n")
for edit in sorted(
edits, key=lambda e: (e.range.start.line, e.range.start.character), reverse=True
):
start = edit.range.start
end = edit.range.end
before = (
"\n".join(lines[: start.line])
+ ("\n" if start.line > 0 else "")
+ lines[start.line][: start.character]
)
after = (
lines[end.line][end.character :]
+ ("\n" if end.line < len(lines) - 1 else "")
+ "\n".join(lines[end.line + 1 :])
)
text = before + edit.new_text + after
lines = text.split("\n")
return text
@pytest.mark.workspace("formatting")
async def test_format_document(client, workspace):
uri, _ = await client.open_and_wait(workspace / "main.cpp")
did_change(client, uri, 1, UNFORMATTED)
edits = await client.format_document(uri)
assert edits is not None
assert len(edits) > 0
result = apply_edits(UNFORMATTED, edits)
assert result == FORMATTED
client.close(uri)
@pytest.mark.workspace("formatting")
async def test_format_range(client, workspace):
uri, _ = await client.open_and_wait(workspace / "main.cpp")
did_change(client, uri, 1, UNFORMATTED)
edits = await client.format_range(
uri,
Range(start=Position(line=1, character=0), end=Position(line=2, character=0)),
)
assert edits is not None
assert len(edits) > 0
client.close(uri)
@pytest.mark.workspace("formatting")
async def test_format_already_formatted(client, workspace):
uri, _ = await client.open_and_wait(workspace / "main.cpp")
did_change(client, uri, 1, FORMATTED)
edits = await client.format_document(uri)
assert edits is not None
assert len(edits) == 0
client.close(uri)

View File

@@ -34,6 +34,8 @@ async def test_capabilities(client, workspace):
assert capability_enabled(caps.folding_range_provider)
assert capability_enabled(caps.inlay_hint_provider)
assert capability_enabled(caps.code_action_provider)
assert caps.document_formatting_provider is True
assert caps.document_range_formatting_provider is True
assert caps.semantic_tokens_provider is not None

View File

@@ -16,9 +16,12 @@ from lsprotocol.types import (
Diagnostic,
DidCloseTextDocumentParams,
DidOpenTextDocumentParams,
DocumentFormattingParams,
DocumentLinkParams,
DocumentRangeFormattingParams,
DocumentSymbolParams,
FoldingRangeParams,
FormattingOptions,
HoverParams,
InlayHintParams,
InitializeParams,
@@ -92,13 +95,18 @@ class CliceClient(BaseLanguageClient):
*,
initialization_options: dict | None = None,
) -> InitializeResult:
if initialization_options is None:
initialization_options = {}
project = dict(initialization_options.get("project", {}))
project.setdefault("cache_dir", str(workspace / ".clice"))
initialization_options["project"] = project
params = InitializeParams(
capabilities=ClientCapabilities(),
root_uri=workspace.as_uri(),
workspace_folders=[WorkspaceFolder(uri=workspace.as_uri(), name="test")],
)
if initialization_options is not None:
params.initialization_options = initialization_options
params.initialization_options = initialization_options
result = await self.initialize_async(params)
self.initialized(InitializedParams())
self.init_result = result
@@ -307,6 +315,29 @@ class CliceClient(BaseLanguageClient):
timeout=timeout,
)
async def format_document(self, uri: str, *, timeout: float = 30.0):
return await asyncio.wait_for(
self.text_document_formatting_async(
DocumentFormattingParams(
text_document=TextDocumentIdentifier(uri=uri),
options=FormattingOptions(tab_size=4, insert_spaces=True),
)
),
timeout=timeout,
)
async def format_range(self, uri: str, range_: Range, *, timeout: float = 30.0):
return await asyncio.wait_for(
self.text_document_range_formatting_async(
DocumentRangeFormattingParams(
text_document=TextDocumentIdentifier(uri=uri),
range=range_,
options=FormattingOptions(tab_size=4, insert_spaces=True),
)
),
timeout=timeout,
)
# ── Extension protocol ───────────────────────────────────────────
async def query_context(self, uri: str, *, timeout: float = 30.0):

View File

@@ -13,6 +13,9 @@ import re
import signal
import sys
import time
# Force line-buffered stdout so CI sees output immediately.
sys.stdout.reconfigure(line_buffering=True)
from pathlib import Path
from urllib.parse import quote, unquote
@@ -109,7 +112,9 @@ async def write_lsp_message(writer: asyncio.StreamWriter, payload: str):
await writer.drain()
async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool | None:
async def replay_one(
trace_path: Path, clice_bin: Path, timeout: int, wall_timeout: int = 300
) -> bool | None:
"""Replay a single trace. Returns True=PASS, False=FAIL, None=SKIP."""
records = load_trace(trace_path)
if not records:
@@ -179,8 +184,21 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
last_method = None
sent_count = 0
wall_deadline = wall_start + wall_timeout
def remaining_wall():
return max(0, wall_deadline - time.monotonic())
try:
for i, rec in enumerate(records):
if remaining_wall() <= 0:
elapsed = time.monotonic() - wall_start
print(
f" result: TIMEOUT (wall-clock {wall_timeout}s exceeded, {elapsed:.1f}s)"
)
success = False
break
if i > 0:
delay = rec["ts"] - records[i - 1]["ts"]
if delay > 0:
@@ -196,7 +214,7 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
try:
await asyncio.wait_for(
asyncio.gather(*pending.values(), return_exceptions=True),
timeout=timeout,
timeout=min(timeout, remaining_wall()),
)
except asyncio.TimeoutError:
elapsed = time.monotonic() - wall_start
@@ -210,7 +228,19 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
if msg_id is not None and method is not None:
pending[msg_id] = asyncio.get_event_loop().create_future()
await write_lsp_message(proc.stdin, rec["msg"])
try:
await asyncio.wait_for(
write_lsp_message(proc.stdin, rec["msg"]),
timeout=min(30, remaining_wall()),
)
except asyncio.TimeoutError:
elapsed = time.monotonic() - wall_start
print(
f" result: HANG (write blocked at {last_method},"
f" sent={sent_count}/{len(records)}, {elapsed:.1f}s)"
)
success = False
break
sent_count = i + 1
except (ConnectionError, BrokenPipeError):
@@ -231,7 +261,7 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
try:
await asyncio.wait_for(
asyncio.gather(*pending.values(), return_exceptions=True),
timeout=timeout,
timeout=min(timeout, remaining_wall()),
)
except asyncio.TimeoutError:
elapsed = time.monotonic() - wall_start
@@ -294,7 +324,7 @@ async def async_main(args):
print(f"SKIP: {trace} (not found)")
skipped += 1
continue
result = await replay_one(trace, args.clice, args.timeout)
result = await replay_one(trace, args.clice, args.timeout, args.wall_timeout)
if result is None:
skipped += 1
elif result:
@@ -317,7 +347,16 @@ def main():
p.add_argument("traces", nargs="+", type=Path, help="JSONL trace files")
p.add_argument("--clice", required=True, type=Path, help="Path to clice binary")
p.add_argument(
"--timeout", type=int, default=120, help="Timeout in seconds (default: 120)"
"--timeout",
type=int,
default=120,
help="Per-request timeout in seconds (default: 120)",
)
p.add_argument(
"--wall-timeout",
type=int,
default=300,
help="Max wall-clock time per trace in seconds (default: 300)",
)
args = p.parse_args()
sys.exit(asyncio.run(async_main(args)))

View File

@@ -0,0 +1,14 @@
---
source: document_symbol_tests.cpp
created_at: 2026-05-20
input_file: statements/if/basic_if.cpp
---
- { name: "basic_if", kind: Namespace, range: "1:0-35:1", selection_range: "1:10-1:18" }
- { name: "abs_val", kind: Function, range: "3:0-7:1", selection_range: "3:4-3:11", detail: "int (int)" }
- { name: "sign", kind: Function, range: "9:0-17:1", selection_range: "9:12-9:16", detail: "const char *(int)" }
- { name: "nested_if", kind: Function, range: "20:0-27:1", selection_range: "20:4-20:13", detail: "int (int, int)" }
- { name: "test", kind: Function, range: "29:0-33:1", selection_range: "29:5-29:9", detail: "void ()" }
- { name: "r1", kind: Variable, range: "30:21-30:41", selection_range: "30:25-30:27", detail: "int" }
- { name: "r2", kind: Variable, range: "31:21-31:38", selection_range: "31:26-31:28", detail: "const char *" }
- { name: "r3", kind: Variable, range: "32:21-32:46", selection_range: "32:25-32:27", detail: "int" }

View File

@@ -0,0 +1,11 @@
---
source: folding_range_tests.cpp
created_at: 2026-05-20
input_file: statements/if/basic_if.cpp
---
- { range: "1:19-35:1", kind: namespace, collapsed_text: "{...}" }
- { range: "3:19-7:1", kind: functionBody, collapsed_text: "{...}" }
- { range: "9:24-17:1", kind: functionBody, collapsed_text: "{...}" }
- { range: "20:28-27:1", kind: functionBody, collapsed_text: "{...}" }
- { range: "29:12-33:1", kind: functionBody, collapsed_text: "{...}" }

View File

@@ -0,0 +1,11 @@
---
source: inlay_hint_tests.cpp
created_at: 2026-05-20
input_file: statements/if/basic_if.cpp
---
- { pos: "30:38", kind: Parameter, label: "x:", padding_right: true }
- { pos: "31:28", kind: Type, label: ": const char *" }
- { pos: "31:36", kind: Parameter, label: "x:", padding_right: true }
- { pos: "32:40", kind: Parameter, label: "a:", padding_right: true }
- { pos: "32:43", kind: Parameter, label: "b:", padding_right: true }

View File

@@ -0,0 +1,75 @@
---
source: semantic_tokens_tests.cpp
created_at: 2026-05-20
input_file: statements/if/basic_if.cpp
---
- { loc: "0:0", text: "// basic if and if-else", kind: Comment }
- { loc: "1:0", text: "namespace", kind: Keyword }
- { loc: "1:10", text: "basic_if", kind: Namespace, modifiers: [Definition] }
- { loc: "3:0", text: "int", kind: Keyword }
- { loc: "3:4", text: "abs_val", kind: Function, modifiers: [Definition] }
- { loc: "3:12", text: "int", kind: Keyword }
- { loc: "3:16", text: "x", kind: Parameter, modifiers: [Definition] }
- { loc: "4:4", text: "if", kind: Keyword }
- { loc: "4:7", text: "x", kind: Parameter }
- { loc: "4:11", text: "0", kind: Number }
- { loc: "5:8", text: "return", kind: Keyword }
- { loc: "5:16", text: "x", kind: Parameter }
- { loc: "6:4", text: "return", kind: Keyword }
- { loc: "6:11", text: "x", kind: Parameter }
- { loc: "9:0", text: "const", kind: Keyword }
- { loc: "9:6", text: "char", kind: Keyword }
- { loc: "9:12", text: "sign", kind: Function, modifiers: [Definition, Readonly] }
- { loc: "9:17", text: "int", kind: Keyword }
- { loc: "9:21", text: "x", kind: Parameter, modifiers: [Definition] }
- { loc: "10:4", text: "if", kind: Keyword }
- { loc: "10:7", text: "x", kind: Parameter }
- { loc: "10:11", text: "0", kind: Number }
- { loc: "11:8", text: "return", kind: Keyword }
- { loc: "11:15", text: "\"positive\"", kind: String }
- { loc: "12:6", text: "else", kind: Keyword }
- { loc: "12:11", text: "if", kind: Keyword }
- { loc: "12:14", text: "x", kind: Parameter }
- { loc: "12:18", text: "0", kind: Number }
- { loc: "13:8", text: "return", kind: Keyword }
- { loc: "13:15", text: "\"negative\"", kind: String }
- { loc: "14:6", text: "else", kind: Keyword }
- { loc: "15:8", text: "return", kind: Keyword }
- { loc: "15:15", text: "\"zero\"", kind: String }
- { loc: "19:0", text: "// dangling else: else binds to nearest if", kind: Comment }
- { loc: "20:0", text: "int", kind: Keyword }
- { loc: "20:4", text: "nested_if", kind: Function, modifiers: [Definition] }
- { loc: "20:14", text: "int", kind: Keyword }
- { loc: "20:18", text: "a", kind: Parameter, modifiers: [Definition] }
- { loc: "20:21", text: "int", kind: Keyword }
- { loc: "20:25", text: "b", kind: Parameter, modifiers: [Definition] }
- { loc: "21:4", text: "if", kind: Keyword }
- { loc: "21:7", text: "a", kind: Parameter }
- { loc: "21:11", text: "0", kind: Number }
- { loc: "22:8", text: "if", kind: Keyword }
- { loc: "22:11", text: "b", kind: Parameter }
- { loc: "22:15", text: "0", kind: Number }
- { loc: "23:12", text: "return", kind: Keyword }
- { loc: "23:19", text: "1", kind: Number }
- { loc: "24:8", text: "else", kind: Keyword }
- { loc: "25:12", text: "return", kind: Keyword }
- { loc: "25:19", text: "2", kind: Number }
- { loc: "26:4", text: "return", kind: Keyword }
- { loc: "26:11", text: "0", kind: Number }
- { loc: "29:0", text: "void", kind: Keyword }
- { loc: "29:5", text: "test", kind: Function, modifiers: [Definition] }
- { loc: "30:21", text: "int", kind: Keyword }
- { loc: "30:25", text: "r1", kind: Variable, modifiers: [Definition] }
- { loc: "30:30", text: "abs_val", kind: Function }
- { loc: "30:39", text: "3", kind: Number }
- { loc: "31:21", text: "auto", kind: Keyword }
- { loc: "31:26", text: "r2", kind: Variable, modifiers: [Definition, Readonly] }
- { loc: "31:31", text: "sign", kind: Function, modifiers: [Readonly] }
- { loc: "31:36", text: "5", kind: Number }
- { loc: "32:21", text: "int", kind: Keyword }
- { loc: "32:25", text: "r3", kind: Variable, modifiers: [Definition] }
- { loc: "32:30", text: "nested_if", kind: Function }
- { loc: "32:40", text: "1", kind: Number }
- { loc: "32:44", text: "1", kind: Number }
- { loc: "35:3", text: "// namespace basic_if", kind: Comment }

View File

@@ -0,0 +1,28 @@
---
source: tu_index_tests.cpp
created_at: 2026-05-20
input_file: statements/if/basic_if.cpp
---
- { loc: "1:10", kind: Namespace, text: "basic_if", relations: [Definition] }
- { loc: "3:4", kind: Function, text: "abs_val", relations: [Definition] }
- { loc: "3:16", kind: Parameter, text: "x", relations: [Definition] }
- { loc: "4:7", kind: Parameter, text: "x", relations: [Reference] }
- { loc: "5:16", kind: Parameter, text: "x", relations: [Reference] }
- { loc: "6:11", kind: Parameter, text: "x", relations: [Reference] }
- { loc: "9:12", kind: Function, text: "sign", relations: [Definition] }
- { loc: "9:21", kind: Parameter, text: "x", relations: [Definition] }
- { loc: "10:7", kind: Parameter, text: "x", relations: [Reference] }
- { loc: "12:14", kind: Parameter, text: "x", relations: [Reference] }
- { loc: "20:4", kind: Function, text: "nested_if", relations: [Definition] }
- { loc: "20:18", kind: Parameter, text: "a", relations: [Definition] }
- { loc: "20:25", kind: Parameter, text: "b", relations: [Definition] }
- { loc: "21:7", kind: Parameter, text: "a", relations: [Reference] }
- { loc: "22:11", kind: Parameter, text: "b", relations: [Reference] }
- { loc: "29:5", kind: Function, text: "test", relations: [Definition] }
- { loc: "30:25", kind: Variable, text: "r1", relations: [Definition] }
- { loc: "30:30", kind: Function, text: "abs_val", relations: [Reference] }
- { loc: "31:26", kind: Variable, text: "r2", relations: [Definition] }
- { loc: "31:31", kind: Function, text: "sign", relations: [Reference] }
- { loc: "32:25", kind: Variable, text: "r3", relations: [Definition] }
- { loc: "32:30", kind: Function, text: "nested_if", relations: [Reference] }

View File

@@ -1,7 +1,7 @@
#include "test/test.h"
#include "command/argument_parser.h"
#include "clang/Options/Options.h"
#include "clang/Driver/Options.h"
namespace clice::testing {
@@ -9,7 +9,7 @@ namespace {
TEST_SUITE(ArgumentParser) {
using option = clang::options::ID;
using option = clang::driver::options::ID;
void EXPECT_ID(llvm::StringRef command, option opt) {
auto id = get_option_id(command);

View File

@@ -1,5 +1,6 @@
#include "test/test.h"
#include "compile/compilation.h"
#include "compile/implement.h"
namespace clice::testing {
namespace {
@@ -7,6 +8,10 @@ namespace {
TEST_SUITE(ClangTidy) {
TEST_CASE(FastCheck) {
#ifdef CLICE_HAS_CLANG_TIDY_MODULES
ASSERT_TRUE(tidy::is_registered_tidy_check("bugprone-integer-division"));
#endif
// ASSERT_TRUE(tidy::is_fast_tidy_check("readability-misleading-indentation"));
// ASSERT_TRUE(tidy::is_fast_tidy_check("bugprone-unused-return-value"));
//
@@ -22,6 +27,7 @@ TEST_CASE(Tidy) {
std::string main_path = TestVFS::path("main.cpp");
CompilationParams params;
params.kind = CompilationKind::Content;
params.clang_tidy = true;
params.vfs = vfs;
params.arguments = {"clang++", "-ffreestanding", "-Xclang", "-undef", main_path.c_str()};
@@ -30,6 +36,37 @@ TEST_CASE(Tidy) {
ASSERT_FALSE(unit.diagnostics().empty());
}
#ifdef CLICE_HAS_CLANG_TIDY_MODULES
TEST_CASE(BugproneIntegerDivision) {
auto vfs = llvm::makeIntrusiveRefCnt<TestVFS>();
vfs->add("main.cpp",
"int main() {"
" double d;"
" int i = 42;"
" d = 32 * 8 / (2 + i);"
" return static_cast<int>(d);"
"}");
std::string main_path = TestVFS::path("main.cpp");
CompilationParams params;
params.kind = CompilationKind::Content;
params.clang_tidy = true;
params.vfs = vfs;
params.arguments = {"clang++", "-ffreestanding", "-Xclang", "-undef", main_path.c_str()};
auto unit = compile(params);
ASSERT_TRUE(unit.completed());
bool found = false;
for(auto& diagnostic: unit.diagnostics()) {
if(llvm::StringRef(diagnostic.message).contains("integer division")) {
found = true;
break;
}
}
ASSERT_TRUE(found);
}
#endif
}; // TEST_SUITE(ClangTidy)
} // namespace
} // namespace clice::testing

View File

@@ -11,7 +11,7 @@ namespace {
namespace protocol = kota::ipc::protocol;
TEST_SUITE(DocumentLink, Tester) {
TEST_SUITE(document_link, Tester) {
std::vector<protocol::DocumentLink> links;
@@ -136,7 +136,7 @@ ABCDE
EXPECT_LINK(0, "0", TestVFS::path("data.bin"));
}
}; // TEST_SUITE(DocumentLink)
}; // TEST_SUITE(document_link)
} // namespace

View File

@@ -1,4 +1,5 @@
#include <cstddef>
#include <format>
#include <functional>
#include <memory>
#include <vector>
@@ -7,13 +8,15 @@
#include "test/tester.h"
#include "feature/feature.h"
#include "kota/meta/enum.h"
namespace clice::testing {
namespace {
namespace protocol = kota::ipc::protocol;
TEST_SUITE(DocumentSymbol, Tester) {
TEST_SUITE(document_symbol, Tester) {
std::vector<protocol::DocumentSymbol> symbols;
@@ -180,7 +183,57 @@ VAR(test)
ASSERT_EQ(total_size(symbols), 3U);
}
}; // TEST_SUITE(DocumentSymbol)
void format_document_symbols(std::string& out,
const feature::PositionMapper& mapper,
llvm::ArrayRef<feature::DocumentSymbol> nodes,
int depth) {
auto pad = std::string(depth * 2, ' ');
for(auto& node: nodes) {
auto kind = kota::meta::enum_name(static_cast<SymbolKind::Kind>(node.kind), "Unknown");
auto start = mapper.to_position(node.range.begin);
auto end = mapper.to_position(node.range.end);
if(!start || !end)
continue;
auto sel_start = mapper.to_position(node.selection_range.begin);
auto sel_end = mapper.to_position(node.selection_range.end);
out += std::format("- {}{{ name: {}, kind: {}, range: \"{}:{}-{}:{}\"",
pad,
yaml_str(node.name),
kind,
start->line,
start->character,
end->line,
end->character);
if(sel_start && sel_end) {
out += std::format(", selection_range: \"{}:{}-{}:{}\"",
sel_start->line,
sel_start->character,
sel_end->line,
sel_end->character);
}
if(!node.detail.empty()) {
out += std::format(", detail: {}", yaml_str(node.detail));
}
out += " }\n";
if(!node.children.empty()) {
format_document_symbols(out, mapper, node.children, depth + 1);
}
}
}
TEST_CASE(snapshot) {
ASSERT_SNAPSHOT_GLOB(corpus_dir, "**/*.cpp", [&](std::string_view path) -> std::string {
if(!compile_file(path))
return "COMPILE_ERROR";
auto content = unit->interested_content();
feature::PositionMapper mapper(content, feature::PositionEncoding::UTF8);
std::string result;
format_document_symbols(result, mapper, feature::document_symbols(*unit), 0);
return result;
});
}
}; // TEST_SUITE(document_symbol)
} // namespace

View File

@@ -11,7 +11,7 @@ namespace {
namespace protocol = kota::ipc::protocol;
TEST_SUITE(FoldingRange, Tester) {
TEST_SUITE(folding_range, Tester) {
std::vector<protocol::FoldingRange> ranges;
@@ -429,7 +429,36 @@ $(1)#pragma region level1
)cpp");
}
}; // TEST_SUITE(FoldingRange)
TEST_CASE(snapshot) {
ASSERT_SNAPSHOT_GLOB(corpus_dir, "**/*.cpp", [&](std::string_view path) -> std::string {
if(!compile_file(path))
return "COMPILE_ERROR";
auto ranges = feature::folding_ranges(*unit);
feature::PositionMapper mapper(unit->interested_content(), feature::PositionEncoding::UTF8);
std::string result;
for(auto& r: ranges) {
auto start = mapper.to_position(r.range.begin);
auto end = mapper.to_position(r.range.end);
if(!start || !end)
continue;
result += std::format("- {{ range: \"{}:{}-{}:{}\"",
start->line,
start->character,
end->line,
end->character);
if(r.kind.has_value()) {
result += std::format(", kind: {}", static_cast<const std::string&>(*r.kind));
}
if(!r.collapsed_text.empty()) {
result += std::format(", collapsed_text: {}", yaml_str(r.collapsed_text));
}
result += " }\n";
}
return result;
});
}
}; // TEST_SUITE(folding_range)
} // namespace

View File

@@ -12,6 +12,29 @@ TEST_CASE(Simple) {
ASSERT_NE(edits.size(), 0U);
}
TEST_CASE(RangeFormat) {
llvm::StringRef code = "int x=1;\nint y = 2 ;\nint z=3;\n";
LocalSourceRange range;
range.begin = static_cast<std::uint32_t>(code.find("int y"));
range.end = static_cast<std::uint32_t>(code.find("\nint z") + 1);
auto range_edits = feature::document_format("main.cpp", code, range);
auto full_edits = feature::document_format("main.cpp", code, std::nullopt);
ASSERT_NE(range_edits.size(), 0U);
EXPECT_LE(range_edits.size(), full_edits.size());
}
TEST_CASE(Idempotent) {
llvm::StringRef code = "int main() {\n return 0;\n}\n";
auto edits = feature::document_format("main.cpp", code, std::nullopt);
EXPECT_EQ(edits.size(), 0U);
}
TEST_CASE(IncludeSort) {
llvm::StringRef code = "#include <vector>\n#include <algorithm>\n\nint main() {}\n";
auto edits = feature::document_format("main.cpp", code, std::nullopt);
ASSERT_NE(edits.size(), 0U);
}
}; // TEST_SUITE(Formatting)
} // namespace

View File

@@ -1,16 +1,19 @@
#include <format>
#include <string>
#include "test/test.h"
#include "test/tester.h"
#include "feature/feature.h"
#include "kota/meta/enum.h"
namespace clice::testing {
namespace {
namespace protocol = kota::ipc::protocol;
TEST_SUITE(InlayHint, Tester) {
TEST_SUITE(inlay_hint, Tester) {
std::vector<protocol::InlayHint> hints;
llvm::DenseMap<std::uint32_t, protocol::InlayHint> hints_map;
@@ -621,7 +624,7 @@ TEST_CASE(Types) {
)c");
EXPECT_SIZE(2);
EXPECT_HINT("0", ": S1");
EXPECT_HINT("1", ": Inner<int>");
EXPECT_HINT("1", ": S2::Inner<int>");
// Lambda
run(R"c(
@@ -1529,6 +1532,38 @@ TEST_CASE(Dependent, skip = true) {
EXPECT_HINT("2", "par3:");
}
}; // TEST_SUITE(InlayHint)
TEST_CASE(snapshot) {
ASSERT_SNAPSHOT_GLOB(corpus_dir, "**/*.cpp", [&](std::string_view path) -> std::string {
if(!compile_file(path))
return "COMPILE_ERROR";
auto content = unit->interested_content();
LocalSourceRange range(0, content.size());
auto hints = feature::inlay_hints(*unit, range);
feature::PositionMapper mapper(content, feature::PositionEncoding::UTF8);
std::string result;
for(auto& hint: hints) {
auto pos = mapper.to_position(hint.offset);
if(!pos)
continue;
auto kind = kota::meta::enum_name(hint.kind, "Unknown");
result += std::format("- {{ pos: \"{}:{}\", kind: {}, label: {}",
pos->line,
pos->character,
kind,
yaml_str(hint.label));
if(hint.padding_left) {
result += ", padding_left: true";
}
if(hint.padding_right) {
result += ", padding_right: true";
}
result += " }\n";
}
return result;
});
}
}; // TEST_SUITE(inlay_hint)
} // namespace
} // namespace clice::testing

View File

@@ -9,6 +9,8 @@
#include "feature/feature.h"
#include "semantic/symbol_kind.h"
#include "kota/meta/enum.h"
namespace clice::testing {
namespace {
@@ -99,7 +101,7 @@ auto decode_relative_tokens(const protocol::SemanticTokens& tokens) -> std::vect
return result;
}
TEST_SUITE(SemanticTokens, Tester) {
TEST_SUITE(semantic_tokens, Tester) {
protocol::SemanticTokens tokens;
std::vector<DecodedToken> decoded;
@@ -138,6 +140,10 @@ void EXPECT_TOKEN(llvm::StringRef name,
ASSERT_EQ(token->modifiers, expected_modifiers);
}
void EXPECT_NO_TOKEN(llvm::StringRef name) {
ASSERT_TRUE(find_by_range(name) == nullptr);
}
TEST_CASE(BasicLexicalKinds) {
run_utf8(R"cpp(
@d1[#define] @m0[FOO]
@@ -264,6 +270,44 @@ int main() {
EXPECT_TOKEN("x3", SymbolKind::Variable, 0);
}
TEST_CASE(IneligibleOperatorReferenceIsSuppressed) {
run_utf8(R"cpp(
struct S {};
S operator+(S lhs, S rhs);
void use(S lhs, S rhs) {
(void)(lhs @plus[+] rhs);
}
)cpp");
EXPECT_NO_TOKEN("plus");
}
TEST_CASE(ConstructorAndDestructorNamesRemainHighlighted) {
run_utf8(R"cpp(
struct S {
@ctor_decl[S]();
@dtor_decl[~]S();
};
S::@ctor_def[S]() {}
void use(S* value) {
value->@dtor_ref[~]S();
}
)cpp");
auto declaration = modifier_mask({SymbolModifiers::Declaration});
auto definition = modifier_mask({SymbolModifiers::Definition});
auto special_member = modifier_mask({SymbolModifiers::ConstructorOrDestructor});
EXPECT_TOKEN("ctor_decl", SymbolKind::Method, declaration | special_member);
EXPECT_TOKEN("dtor_decl", SymbolKind::Method, declaration | special_member);
EXPECT_TOKEN("ctor_def", SymbolKind::Method, definition | special_member);
EXPECT_TOKEN("dtor_ref", SymbolKind::Method, special_member);
}
TEST_CASE(LegacyVarDeclTemplates) {
run_utf8(R"cpp(
extern int @x1[x];
@@ -539,7 +583,53 @@ void f() {
EXPECT_TOKEN("v2", SymbolKind::Variable, definition);
}
}; // TEST_SUITE(SemanticTokens)
TEST_CASE(snapshot) {
ASSERT_SNAPSHOT_GLOB(corpus_dir, "**/*.cpp", [&](std::string_view path) -> std::string {
if(!compile_file(path))
return "COMPILE_ERROR";
auto content = unit->interested_content();
auto tokens = feature::semantic_tokens(*unit);
feature::PositionMapper mapper(content, feature::PositionEncoding::UTF8);
std::string result;
for(auto& token: tokens) {
if(!token.range.valid() || token.range.end <= token.range.begin ||
token.range.end > content.size())
continue;
auto pos = mapper.to_position(token.range.begin);
if(!pos)
continue;
auto text = content.substr(token.range.begin, token.range.length());
auto kind = kota::meta::enum_name(static_cast<SymbolKind::Kind>(token.kind), "Unknown");
result += std::format("- {{ loc: \"{}:{}\", text: {}, kind: {}",
pos->line,
pos->character,
yaml_str(text),
kind);
std::string mods;
for(std::uint32_t i = 0; i < 32; ++i) {
if(token.modifiers & (1u << i)) {
auto name = kota::meta::enum_name(static_cast<SymbolModifiers::Kind>(i));
if(!name.empty()) {
if(!mods.empty())
mods += ", ";
mods += name;
}
}
}
if(!mods.empty()) {
result += std::format(", modifiers: [{}]", mods);
}
result += " }\n";
}
return result;
});
}
}; // TEST_SUITE(semantic_tokens)
} // namespace

View File

@@ -1,13 +1,18 @@
#include <algorithm>
#include <format>
#include <set>
#include "test/test.h"
#include "test/tester.h"
#include "feature/feature.h"
#include "index/tu_index.h"
#include "kota/meta/enum.h"
namespace clice::testing {
namespace {
TEST_SUITE(TUIndex, Tester) {
TEST_SUITE(tu_index, Tester) {
index::TUIndex tu_index;
@@ -500,6 +505,64 @@ TEST_CASE(SymbolKinds) {
check_kind("ns", SymbolKind::Namespace);
}
}; // TEST_SUITE(TUIndex)
TEST_CASE(snapshot) {
ASSERT_SNAPSHOT_GLOB(corpus_dir, "**/*.cpp", [&](std::string_view path) -> std::string {
if(!compile_file(path))
return "COMPILE_ERROR";
auto idx = index::TUIndex::build(*unit);
auto content = unit->interested_content();
feature::PositionMapper mapper(content, feature::PositionEncoding::UTF8);
std::string result;
auto sorted = idx.main_file_index.occurrences;
std::ranges::sort(sorted, [](auto& lhs, auto& rhs) {
return std::tuple(lhs.range.begin, lhs.range.end, lhs.target) <
std::tuple(rhs.range.begin, rhs.range.end, rhs.target);
});
for(auto& occ: sorted) {
auto text = content.substr(occ.range.begin, occ.range.end - occ.range.begin);
auto pos = mapper.to_position(occ.range.begin);
if(!pos)
continue;
auto sym_it = idx.symbols.find(occ.target);
std::string_view kind_name = "?";
if(sym_it != idx.symbols.end()) {
kind_name =
kota::meta::enum_name(static_cast<SymbolKind::Kind>(sym_it->second.kind),
"Unknown");
}
result += std::format("- {{ loc: \"{}:{}\", kind: {}, text: {}",
pos->line,
pos->character,
kind_name,
yaml_str(text));
auto rel_it = idx.main_file_index.relations.find(occ.target);
if(rel_it != idx.main_file_index.relations.end()) {
std::string rels;
for(auto& rel: rel_it->second) {
if(rel.range != occ.range)
continue;
if(!rels.empty())
rels += ", ";
rels += kota::meta::enum_name(static_cast<RelationKind::Kind>(rel.kind), "?");
}
if(!rels.empty()) {
result += std::format(", relations: [{}]", rels);
}
}
result += " }\n";
}
return result;
});
}
}; // TEST_SUITE(tu_index)
} // namespace
} // namespace clice::testing

View File

@@ -442,9 +442,9 @@ TEST_CASE(Types) {
TEST_CASE(CXXFeatures) {
EXPECT_SELECT(R"(
template <typename T>
int x = T::@[$U]::ccc();
int x = @[T::$U::]ccc();
)",
"DependentNameTypeLoc");
"NestedNameSpecifierLoc");
EXPECT_SELECT(R"(
struct Foo {};
struct Bar : @[v$ir$tual private Foo] {};
@@ -486,9 +486,9 @@ TEST_CASE(UsingEnum) {
"TypedefTypeLoc");
EXPECT_SELECT(R"(
namespace ns { enum class A {}; };
@[using enum $ns::A];
using enum @[$ns::]A;
)",
"UsingEnumDecl");
"NestedNameSpecifierLoc");
EXPECT_SELECT(R"(
namespace ns { enum class A {}; };
@[using $enum ns::A];

View File

@@ -456,8 +456,6 @@ TEST_CASE(BasePackExpansion) {
)code");
}
// --- Robustness tests for edge cases found during stress testing ---
TEST_CASE(RecursiveBaseClass) {
// Regression test: callback_traits<F> inherits callback_traits<decltype(&F::operator())>,
// creating infinite recursion through lookupInBases. CTD cycle detection must bail out.

View File

@@ -3,7 +3,7 @@
#include "test/test.h"
#include "command/command.h"
#include "compile/compilation.h"
#include "server/compile_graph.h"
#include "server/compiler/compile_graph.h"
#include "support/path_pool.h"
#include "syntax/dependency_graph.h"
#include "syntax/scan.h"

View File

@@ -1,7 +1,7 @@
#include <optional>
#include "test/test.h"
#include "server/compile_graph.h"
#include "server/compiler/compile_graph.h"
namespace clice::testing {
namespace {

View File

@@ -2,11 +2,11 @@
#include "test/temp_dir.h"
#include "test/test.h"
#include "server/config.h"
#include "server/workspace/config.h"
#include "support/filesystem.h"
#include "kota/codec/json/json.h"
#include "kota/codec/toml.h"
#include "kota/codec/toml/toml.h"
namespace clice::testing {
@@ -148,7 +148,7 @@ TEST_CASE(ApplyDefaults) {
EXPECT_EQ(*config.project.idle_timeout_ms, 3000);
EXPECT_EQ(config.project.max_active_file.value, 8);
EXPECT_EQ(config.project.stateful_worker_count.value, 2u);
EXPECT_EQ(config.project.stateless_worker_count.value, 3u);
EXPECT_GE(config.project.stateless_worker_count.value, 2u);
EXPECT_FALSE(config.project.cache_dir.empty());
EXPECT_FALSE(config.project.index_dir.empty());
EXPECT_FALSE(config.project.logging_dir.empty());

View File

@@ -2,7 +2,7 @@
#include <vector>
#include "test/test.h"
#include "server/protocol.h"
#include "server/protocol/worker.h"
#include "server/worker_test_helpers.h"
namespace clice::testing {
@@ -29,7 +29,6 @@ TEST_CASE(BuildPCMThenCompileWithImport) {
tmp.touch("consumer.cpp", "import Hello;\n" "int main() { return hello()[0]; }\n");
auto consumer = tmp.path("consumer.cpp");
// --- Phase 1: Build PCM via stateless worker ---
WorkerHandle sl;
ASSERT_TRUE(sl.spawn("stateless-worker"));
@@ -63,7 +62,6 @@ TEST_CASE(BuildPCMThenCompileWithImport) {
ASSERT_TRUE(phase1_done);
ASSERT_FALSE(pcm_path.empty());
// --- Phase 2: Compile consumer with the PCM via stateful worker ---
WorkerHandle sf;
ASSERT_TRUE(sf.spawn("stateful-worker"));

Some files were not shown because too many files have changed in this diff Show More