7 Commits

Author SHA1 Message Date
Myriad-Dreamin
9d75659fb1 feat: implement explicit_reference_targets 2026-04-04 03:18:45 +08:00
Myriad-Dreamin
09e95bbc7e feat: explore gap between clice and clangd 2026-04-04 01:51:56 +08:00
Myriad-Dreamin
69454812bf feat: explicit specify lexical tokens to handle 2026-04-04 00:38:22 +08:00
Myriad-Dreamin
511b71f19a dev: add module example 2026-04-04 00:04:55 +08:00
Myriad-Dreamin
ed8b8b7745 dev: add single-file sample 2026-04-03 23:49:03 +08:00
Myriad-Dreamin
a303e13f58 dev: add cmake-workspace sample 2026-04-03 23:48:08 +08:00
Myriad-Dreamin
72c0a74609 dev: watch socket 2026-04-03 23:47:58 +08:00
196 changed files with 9275 additions and 19912 deletions

View File

@@ -100,7 +100,7 @@ SortIncludes: true
SortUsingDeclarations: Never
IncludeBlocks: Regroup
IncludeCategories:
- Regex: '^["<](spdlog|toml\+\+|coraing|cpptrace|flatbuffers|kota)/'
- Regex: '^["<](spdlog|toml\+\+|coraing|cpptrace|flatbuffers)/'
Priority: 30
SortPriority: 31

View File

@@ -1,50 +0,0 @@
---
Checks: >
-*,
bugprone-*,
modernize-*,
performance-*,
readability-*,
-modernize-use-trailing-return-type,
-readability-magic-numbers,
-readability-else-after-return,
-readability-braces-around-statements,
-readability-avoid-const-params-in-decls,
-readability-named-parameter,
-readability-implicit-bool-conversion,
-readability-use-anyofallof,
-bugprone-easily-swappable-parameters,
-bugprone-exception-escape,
-bugprone-narrowing-conversions,
-modernize-use-nodiscard,
WarningsAsErrors: ""
HeaderFilterRegex: "(src|tests)/.*"
CheckOptions:
# Naming conventions matching project style
- key: readability-identifier-naming.ClassCase
value: CamelCase
- key: readability-identifier-naming.StructCase
value: CamelCase
- key: readability-identifier-naming.EnumCase
value: CamelCase
- key: readability-identifier-naming.EnumConstantCase
value: CamelCase
- key: readability-identifier-naming.TemplateParameterCase
value: CamelCase
- key: readability-identifier-naming.TypeAliasCase
value: CamelCase
- key: readability-identifier-naming.FunctionCase
value: lower_case
- key: readability-identifier-naming.MethodCase
value: lower_case
- key: readability-identifier-naming.VariableCase
value: lower_case
- key: readability-identifier-naming.ParameterCase
value: lower_case
- key: readability-identifier-naming.MemberCase
value: lower_case
- key: readability-identifier-naming.NamespaceCase
value: lower_case

View File

@@ -1,268 +0,0 @@
# clice — Project Guide
## Project Overview
clice is a next-generation C++ language server (LSP) built on LLVM/Clang, targeting modern C++ (C++20/23). It uses a multi-process architecture with a master server coordinating stateless and stateful workers.
## Core Correction Patterns — Lessons from Past Interactions
The following patterns were extracted from extensive real-world collaboration. These are recurring mistakes that MUST be avoided. Read them carefully — they represent hard-won lessons, not hypothetical concerns.
### Pattern 1: Misjudging Real-World Priorities
AI tends to optimize whatever metric looks most impressive, rather than what actually matters in the user's real scenario.
**Example**: During performance optimization, the AI proudly reported "hot cache is 4-7x faster!" — but the function in question runs at LSP server startup, which is ALWAYS a cold start. Optimizing hot cache was completely meaningless.
**Rule**: Before optimizing or analyzing anything, first understand the REAL usage scenario. Ask yourself: "When does this code actually run? What does the user actually experience?" Do not chase metrics that look good on paper but are irrelevant in practice.
### Pattern 2: Pushing Without Local Verification
The most common and most damaging pattern. AI proposes a fix, pushes it immediately, CI fails, then another fix, push, fail again — wasting CI cycles and the user's time.
**Rule**: NEVER push code that you haven't verified locally. Before every push:
- Build locally with the same configuration CI uses.
- Run the relevant tests locally and confirm they pass.
- If you cannot reproduce the CI environment locally, say so — do not just "try and see."
- "It compiles" is NOT sufficient. Tests must pass.
### Pattern 3: Superficial Refactoring
When asked to refactor, AI tends to do mechanical code movement (copy functions from A to B) without understanding the deeper design intent (ownership, responsibility boundaries, API cleanliness).
**Example**: When splitting `MasterServer` into `Workspace` and `Session`, the AI moved functions but kept ugly APIs like `f(path_id, sessions_map)` instead of the clean `f(Session&)` that the refactoring was meant to achieve.
**Rule**: When refactoring, understand the WHY. Ask: "What design problem is this refactoring solving?" If you're just moving code around without improving the abstractions, you're not refactoring — you're rearranging deck chairs.
### Pattern 4: Fixing Only the Immediate Instance, Not the Pattern
When given a cleanup instruction, AI applies it to the single file or function currently being discussed, ignoring all other occurrences in the project.
**Example**: User says "remove decorative `===` comment separators." AI removes them from `workspace.cpp` only. User has to say: "The other files too!"
**Rule**: When given a cleanup or style instruction, apply it project-wide. Use `Grep` to find ALL occurrences and fix them all in one pass. Think: "Where else does this pattern appear?"
### Pattern 5: Never Skip, Disable, or Work Around Failing Tests
When stuck on a difficult bug (especially flaky CI, race conditions, platform-specific issues), AI may propose marking tests as `continue-on-error`, skipping them, or adding `expected-failure` annotations to make CI green.
**Rule**: This is ABSOLUTELY FORBIDDEN. If a test fails, fix the root cause. There are ZERO exceptions. Skipping a test to make CI green is not "fixing" — it is hiding a bug. If you ever find yourself thinking "maybe we should just skip this test," stop and reconsider your approach entirely.
### Pattern 6: Excessive Confirmation Seeking vs. Premature Execution
AI oscillates between two extremes: asking "should I do X?" for every trivial decision, or silently executing major changes without confirmation.
**Rule**: Calibrate based on reversibility and impact:
- **Small, reversible changes** (formatting, renaming a local variable, adding a test): just do it.
- **Architecture decisions, API changes, large refactors**: propose the plan first, wait for confirmation.
- **Pushing to remote, creating PRs, modifying CI**: always confirm.
- When the user says "go ahead" or "do it," execute fully without asking again mid-way.
## Code Reuse & Understanding Before Implementation
**This is the single most important rule in this project.** Before writing ANY new code, you MUST thoroughly read and understand the existing codebase first. This project has a rich set of utilities, abstractions, and patterns already in place — duplicating them wastes effort and creates maintenance burden.
Concrete requirements:
1. **Read before you write.** Before implementing a feature or fix, explore the relevant modules in `src/`. Search for existing helpers, utilities, and patterns that solve the same or similar problems. Use `Grep`, `Glob`, and `Agent` tools to investigate thoroughly — do not assume something doesn't exist just because you haven't seen it yet.
2. **Reuse existing infrastructure.** This project already has:
- A `Lexer` class (`src/syntax/lexer.h`) — do not hand-write token scanning logic.
- A `PositionMapper` for source location conversion — do not reimplement offset-to-line/column math.
- `CompilationUnitRef` methods (`decompose_location`, `decompose_range`, `file_path`, `directives`, etc.) — use them instead of raw Clang APIs.
- `SemanticVisitor` for AST traversal — extend it, do not write custom recursive AST walkers.
- `Tester` framework for unit tests with VFS, annotation support, and multi-phase compilation — use it, do not create ad-hoc test setups.
- Utility functions in `src/support/` — check there before writing new helpers.
3. **Follow established patterns.** When adding a new feature (e.g., a new LSP request handler), look at how 2-3 existing features of the same kind are implemented. Match their structure: same file organization, same function signatures, same error handling patterns. If every other feature in `src/feature/` follows a certain pattern, yours should too.
4. **Do not reinvent what the project already has.** If you find yourself writing a helper function that feels generic (string manipulation, path handling, JSON serialization, source range conversion), STOP and search the codebase first. There is a high probability it already exists. Creating duplicates leads to inconsistencies and bugs when one copy gets updated but the other doesn't.
5. **When in doubt, ask.** If you're unsure whether an existing utility covers your use case or whether to extend an existing abstraction vs. create a new one, ask the user rather than guessing.
## Source Layout
- `src/server/` — LSP server core: master server, compiler, indexer, stateful/stateless workers
- `src/feature/` — LSP feature implementations: hover, completion, document links, semantic tokens, etc.
- `src/compile/` — Compilation orchestration: compilation unit, directives, diagnostics
- `src/index/` — Symbol indexing: TUIndex, ProjectIndex, MergedIndex, include graph
- `src/semantic/` — Semantic analysis: symbol kinds, relations, AST visitor, template resolver
- `src/syntax/` — Lexer, scanner, token types, dependency graph
- `src/command/` — CLI parsing, compilation database, toolchain detection
- `src/support/` — Utilities: logging, filesystem, JSON, string helpers
## Build System
- Uses **pixi** for environment management and **CMake + Ninja** for building.
- Two build types: `Debug` and `RelWithDebInfo` (default).
- Build output goes to `build/[type]/`.
- See `/build`, `/test`, `/format` commands for common operations.
## Commit Message Format
Use **conventional commits** — enforced by CI:
```
<type>(<scope>): <short description>
```
- **Types**: `feat`, `fix`, `refactor`, `chore`, `docs`, `ci`, `test`
- **Scopes**: match `src/` subdirectories or feature names, e.g. `completion`, `server`, `index`, `tests`, `document links`
- Keep the subject line under 70 characters.
## Tests
Three types of tests, all must pass before committing:
- **Unit tests** (`tests/unit/`): C++ tests using the project's own test framework. Test names should be at most 4 words.
- **Integration tests** (`tests/integration/`): Python pytest tests that start a real clice server and communicate via LSP.
- **Smoke tests** (`tests/smoke/`): Replay recorded LSP sessions via `tests/replay.py`.
### Integration Test Style
- Keep tests concise. Do NOT write large comment blocks explaining the test layout or expected behavior.
- Use descriptive test function names and short inline comments only where logic is non-obvious.
## Pre-PR Review
Before opening a PR, launch **3 parallel subagents** to review the diff independently:
1. **Correctness reviewer**: Check for logic errors, edge cases, undefined behavior, and off-by-one mistakes.
2. **Style reviewer**: Verify the code follows this project's naming conventions, coding style, and CLAUDE.md rules.
3. **Test reviewer**: Confirm test coverage is adequate — new functionality has tests, edge cases are covered, and no existing tests were broken or weakened.
Each agent should read the full diff (`git diff main...HEAD`) and report issues. Fix all reported issues before opening the PR.
## Pre-commit Checklist
Before committing code, you MUST:
1. **Run `pixi run format`** to format all source files.
2. **Pass all three types of tests:**
- Unit tests: `pixi run unit-test [type]`
- Integration tests: `pixi run integration-test [type]`
- Smoke tests: `pixi run smoke-test [type]`
3. **All test failures must be fixed before committing.** This is a HARD REQUIREMENT with NO exceptions:
- If a test fails, it MUST be fixed before you commit. Do NOT commit with known failures.
- Do NOT skip, disable, or mark tests as expected-failure to work around breakage.
- Do NOT argue "this test was already broken before my changes" — if it fails on your branch, it is YOUR responsibility to fix it before committing. The main branch CI is green; any failure on your branch is caused by your changes, period.
- Do NOT defer fixing to a follow-up PR. Fix it NOW, in this branch, before committing.
---
## C++ Coding Style
### Template & Type Traits
- Do NOT blindly add `std::remove_cvref_t` on every template parameter. Understand C++ template argument deduction rules:
- `template<typename T> void f(T x)``T` is always deduced as a non-reference, non-cv-qualified type. No need for `remove_cvref_t`.
- `template<typename T> void f(T& x)``T` is deduced as the referred-to type (possibly cv-qualified, but never a reference). No need for `remove_cvref_t` to strip references.
- `template<typename T> void f(const T& x)``T` is deduced as a non-const, non-reference type. No need for `remove_cvref_t`.
- `template<typename T> void f(T&& x)`**forwarding reference**: `T` CAN be deduced as an lvalue reference (e.g., `int&`). This is the ONLY case where `std::remove_cvref_t<T>` is needed to get the bare type.
- Class template parameters and return types are also never deduced as references; don't add `remove_cvref_t` on them either.
### Type Traits & Concepts (C++20/23)
- This project targets C++20/23. Use variable templates directly for type traits — do NOT use the old pattern of wrapping a class template static member in a variable template. Prefer:
```cpp
// Good: directly specialize a variable template
template<typename T>
inline constexpr bool is_my_type_v = false;
template<>
inline constexpr bool is_my_type_v<MyType> = true;
```
```cpp
// Bad: unnecessary class template wrapper
template<typename T>
struct is_my_type : std::false_type {};
template<>
struct is_my_type<MyType> : std::true_type {};
template<typename T>
inline constexpr bool is_my_type_v = is_my_type<T>::value;
```
- When defining a concept that checks a type trait, do NOT add `std::remove_cvref_t` unless you specifically intend the concept to see through references/cv-qualifiers. If the concept is meant for a bare type, just use `T` directly — the caller is responsible for passing the right type.
```cpp
// Good
template<typename T>
concept MyTrait = is_my_type_v<T>;
// Bad: unnecessary remove_cvref_t
template<typename T>
concept MyTrait = is_my_type_v<std::remove_cvref_t<T>>;
```
### Naming Conventions
- **Variables, member fields, function names**: `snake_case`. Class member fields do NOT use any special suffix/prefix (no trailing `_`, no `m_` prefix).
- **Class names, template parameter names, enum names**: `PascalCase`. Exception: some class names also use `snake_case` — follow the existing style in the project.
- **Enum values**: `PascalCase`.
### String Literals
- Prefer C++11 raw string literals `R"(...)"` over escaped strings. Avoid `\"`, `\\`, `\n` in string literals when a raw literal is cleaner.
### Error Handling
- **Prefer `if` with init-statements to tightly scope error variables**, but avoid them when they compromise code readability or flatten control flow.
- **Omit redundant conditions:** If the error type provides an `operator bool` or evaluates implicitly (e.g., standard error codes, custom error wrappers), omit the redundant condition check.
- **Avoid forced `else` branches:** If scoping the variable inside the `if` requires you to introduce an `else` block for the success path (especially when returning early on error), declare the variable in the local scope instead to keep the control flow flat.
```cpp
// Good: Omit redundant condition when the type has operator bool
if (auto err = foo()) {
/* handle error */
}
// Bad: Redundant condition check
if (auto err = foo(); err) {
/* handle error */
}
// Good: Use init-statement when a custom condition is required,
// AND the variable isn't needed outside the if-statement
if (auto result = foo(); !result.has_value()) {
/* handle error */
}
// --- Scope and Control Flow Considerations ---
// Bad: Using init-statement forces an 'else' block because 'result'
// goes out of scope, leading to nested/redundant code.
if (auto result = get_data(); !result.has_value()) {
return result.error();
} else {
process(result.value()); // Success path is forced into a nested block
}
// Good: Declare as a regular local variable to allow early exit
// and keep the success path un-nested (flat control flow).
auto result = get_data();
if (!result.has_value()) {
return result.error();
}
process(result.value());
```
### Style
- Prefer `[[maybe_unused]]` over `(void)` for intentionally unused variables or parameters.
### Modern C++ Usage
- Use C++20/23 APIs whenever possible. Do NOT use `<iostream>` facilities (`std::cout`, `std::cin`, `std::cerr`, etc.). Also do NOT use C-style I/O (`printf`, `fprintf`, etc.).
- Prefer `std::ranges` / `std::views` APIs over raw loops and traditional `<algorithm>` calls.
- If the project depends on LLVM, prefer LLVM's efficient data structures (e.g., `llvm::SmallVector`, `llvm::DenseMap`, `llvm::StringMap`, `llvm::StringRef`) over their `std` counterparts when appropriate.
### Parameter Passing Preferences
- For string parameters, prefer `llvm::StringRef` > `std::string_view` > `const std::string&`.
- For array/span parameters, prefer `llvm::ArrayRef` > `std::span` > `const std::vector&`.

View File

@@ -1,15 +0,0 @@
Build the project. Accepts an optional argument for build type: `Debug` or `RelWithDebInfo` (default).
Available build commands:
- CMake configure only: `pixi run cmake-config [type]`
- CMake build only (skip configure): `pixi run cmake-build [type]`
- Full build (configure + build): `pixi run build [type]`
- Build a specific target: `pixi run cmake-build [type]` then `cmake --build build/[type] --target [target]`
Common targets: `clice`, `unit_tests`
Example usage:
- `/build` — full build RelWithDebInfo
- `/build Debug` — full build Debug

View File

@@ -1,5 +0,0 @@
Format all project source files.
Run: `pixi run format`
Formats C++, Python, Lua, JS/TS, Markdown, JSON, TOML, and YAML files.

View File

@@ -1,19 +0,0 @@
Run tests. Accepts an optional argument for build type: `Debug` or `RelWithDebInfo` (default).
Available test commands:
- Unit tests: `pixi run unit-test [type]`
- Integration tests: `pixi run integration-test [type]`
- Smoke tests: `pixi run smoke-test [type]`
- All tests (unit + integration): `pixi run test [type]`
Filtering specific tests:
- Unit tests: `pixi run unit-test [type] --test-filter=SuiteName.CaseName`
- Integration tests: `pixi run pytest tests/integration -k "test_name" --executable=./build/[type]/bin/clice`
- Smoke tests: `pixi run python tests/replay.py tests/smoke/specific.jsonl --clice=./build/[type]/bin/clice`
Example usage:
- `/test` — run all tests (RelWithDebInfo)
- `/test Debug` — run all tests (Debug)

View File

@@ -1,7 +1,2 @@
chat:
auto_reply: false
reviews:
auto_review:
enabled: true
summary:
enabled: false

View File

@@ -13,7 +13,7 @@ runs:
- name: Setup Pixi
uses: prefix-dev/setup-pixi@v0.9.3
with:
pixi-version: v0.67.0
pixi-version: v0.62.0
environments: ${{ inputs.environments }}
activate-environment: true
cache: true

View File

@@ -1,7 +1,8 @@
name: benchmark
on:
workflow_dispatch:
pull_request:
branches: [main]
jobs:
benchmark:
@@ -21,7 +22,7 @@ jobs:
- name: Build scan_benchmark
run: |
pixi run cmake-config RelWithDebInfo ON -- -DCLICE_ENABLE_BENCHMARK=ON
pixi run cmake-config RelWithDebInfo ON "-DCLICE_ENABLE_BENCHMARK=ON"
cmake --build build/RelWithDebInfo --target scan_benchmark
- name: Clone LLVM

View File

@@ -1,22 +1,6 @@
name: build llvm
on:
workflow_dispatch:
inputs:
llvm_version:
description: "LLVM version to build (e.g., 21.1.8)"
required: true
type: string
skip_upload:
description: "Skip upload and PR creation (build-only mode)"
required: false
type: boolean
default: false
skip_pr:
description: "Skip PR creation (upload only, no PR)"
required: false
type: boolean
default: false
pull_request:
# if you want to run this workflow, change the branch name to main,
# if you want to turn off it, change it to non existent branch.
@@ -28,7 +12,9 @@ jobs:
fail-fast: false
matrix:
include:
# Native builds
- os: windows-2025
llvm_mode: Debug
lto: OFF
- os: windows-2025
llvm_mode: RelWithDebInfo
lto: OFF
@@ -53,42 +39,6 @@ jobs:
- os: macos-15
llvm_mode: RelWithDebInfo
lto: ON
# Cross-compilation builds
# macOS x64 (from arm64 macos-15)
- os: macos-15
llvm_mode: RelWithDebInfo
lto: OFF
target_triple: x86_64-apple-darwin
- os: macos-15
llvm_mode: RelWithDebInfo
lto: ON
target_triple: x86_64-apple-darwin
# Linux aarch64 (from x64 ubuntu-24.04)
- os: ubuntu-24.04
llvm_mode: RelWithDebInfo
lto: OFF
target_triple: aarch64-linux-gnu
pixi_env: cross-linux-aarch64
- os: ubuntu-24.04
llvm_mode: RelWithDebInfo
lto: ON
target_triple: aarch64-linux-gnu
pixi_env: cross-linux-aarch64
# Windows arm64 (from x64 windows-2025)
- os: windows-2025
llvm_mode: RelWithDebInfo
lto: OFF
target_triple: aarch64-pc-windows-msvc
pixi_env: cross-windows-arm64
- os: windows-2025
llvm_mode: RelWithDebInfo
lto: ON
target_triple: aarch64-pc-windows-msvc
pixi_env: cross-windows-arm64
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
@@ -117,91 +67,49 @@ jobs:
free -h
df -h
- uses: ./.github/actions/setup-pixi
- name: Setup Pixi
uses: prefix-dev/setup-pixi@v0.9.3
with:
environments: ${{ matrix.pixi_env || 'package' }}
pixi-version: v0.59.0
environments: package
activate-environment: true
cache: true
locked: true
- name: Clone llvm-project
- name: Clone llvm-project (21.1.4)
shell: bash
run: |
VERSION="${{ inputs.llvm_version || '21.1.8' }}"
echo "Cloning LLVM ${VERSION}..."
git clone --branch "llvmorg-${VERSION}" --depth 1 https://github.com/llvm/llvm-project.git .llvm
- name: Validate distribution components
shell: bash
run: |
python3 scripts/validate-llvm-components.py \
--llvm-src=.llvm \
--components-file=scripts/llvm-components.json
git clone --branch llvmorg-21.1.4 --depth 1 https://github.com/llvm/llvm-project.git .llvm
- name: Build LLVM (install-distribution)
shell: bash
run: |
ENV="${{ matrix.pixi_env || 'package' }}"
EXTRA_ARGS=""
if [[ -n "${{ matrix.target_triple }}" ]]; then
EXTRA_ARGS="--target-triple=${{ matrix.target_triple }}"
fi
pixi run -e "$ENV" build-llvm \
--llvm-src=.llvm \
--mode="${{ matrix.llvm_mode }}" \
--lto="${{ matrix.lto }}" \
--build-dir=build \
${EXTRA_ARGS}
pixi run build-llvm --llvm-src=.llvm --mode="${{ matrix.llvm_mode }}" --lto="${{ matrix.lto }}" --build-dir=build
- name: Build clice using installed LLVM
if: ${{ !matrix.target_triple }}
shell: bash
run: |
pixi run cmake-config ${{ matrix.llvm_mode }} ON -- \
"-DCLICE_ENABLE_LTO=${{ matrix.lto }}" \
"-DLLVM_INSTALL_PATH=.llvm/build-install"
pixi run cmake-build ${{ matrix.llvm_mode }}
- name: Build clice using installed LLVM (cross-compile)
if: ${{ matrix.target_triple }}
shell: bash
run: |
ENV="${{ matrix.pixi_env || 'package' }}"
pixi run -e "$ENV" cmake-config ${{ matrix.llvm_mode }} ON -- \
"-DCLICE_ENABLE_LTO=${{ matrix.lto }}" \
"-DCLICE_TARGET_TRIPLE=${{ matrix.target_triple }}" \
"-DLLVM_INSTALL_PATH=.llvm/build-install"
pixi run -e "$ENV" cmake-build ${{ matrix.llvm_mode }}
- name: Verify cross-compiled binary architecture
if: ${{ matrix.target_triple && runner.os != 'Windows' }}
shell: bash
run: |
BINARY="build/${{ matrix.llvm_mode }}/bin/clice"
echo "Binary info:"
file "$BINARY"
case "${{ matrix.target_triple }}" in
aarch64-linux-gnu) file "$BINARY" | grep -q "aarch64" ;;
x86_64-apple-darwin) file "$BINARY" | grep -q "x86_64" ;;
esac
- name: Upload cross-compiled clice for functional test
if: ${{ matrix.target_triple && matrix.lto == 'OFF' }}
uses: actions/upload-artifact@v4
with:
name: cross-clice-${{ matrix.target_triple }}-${{ matrix.llvm_mode }}
path: |
build/${{ matrix.llvm_mode }}/bin/
build/${{ matrix.llvm_mode }}/lib/
if-no-files-found: error
retention-days: 1
cmake -B build -G Ninja \
-DCMAKE_BUILD_TYPE=${{ matrix.llvm_mode }} \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchain.cmake \
-DCLICE_ENABLE_TEST=ON \
-DCLICE_CI_ENVIRONMENT=ON \
-DCLICE_ENABLE_LTO=${{ matrix.lto }} \
-DLLVM_INSTALL_PATH=".llvm/build-install"
cmake --build build
- name: Run tests
if: ${{ !matrix.target_triple }}
shell: bash
run: pixi run test ${{ matrix.llvm_mode }}
run: |
EXE_EXT=""
if [[ "${{ runner.os }}" == "Windows" ]]; then
EXE_EXT=".exe"
fi
./build/bin/unit_tests${EXE_EXT} --test-dir="./tests/data"
uv run --project tests pytest -s --log-cli-level=INFO tests/integration --executable=./build/bin/clice${EXE_EXT}
# Prune is only supported for native builds (requires linking clice to test).
# Cross-compiled targets reuse the native prune manifest of the same OS.
- name: Prune LLVM static libraries (Debug/RelWithDebInfo no LTO)
if: (!matrix.target_triple) && (matrix.llvm_mode == 'Debug' || (matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF'))
if: matrix.llvm_mode == 'Debug' || (matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF')
shell: bash
run: |
MANIFEST="pruned-libs-${{ matrix.os }}.json"
@@ -209,13 +117,13 @@ jobs:
python3 scripts/prune-llvm-bin.py \
--action discover \
--install-dir ".llvm/build-install/lib" \
--build-dir "build/${{ matrix.llvm_mode }}" \
--build-dir "build" \
--max-attempts 60 \
--sleep-seconds 60 \
--manifest "${MANIFEST}"
- name: Upload pruned-libs manifest
if: (!matrix.target_triple) && matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF'
if: matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF'
uses: actions/upload-artifact@v4
with:
name: llvm-pruned-libs-${{ matrix.os }}
@@ -223,8 +131,8 @@ jobs:
if-no-files-found: error
compression-level: 0
- name: Apply pruned-libs manifest (RelWithDebInfo + LTO, native only)
if: (!matrix.target_triple) && matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'ON'
- name: Apply pruned-libs manifest (RelWithDebInfo + LTO)
if: matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'ON'
shell: bash
env:
GH_TOKEN: ${{ github.token }}
@@ -234,27 +142,7 @@ jobs:
--action apply \
--manifest "${MANIFEST}" \
--install-dir ".llvm/build-install/lib" \
--build-dir "build/${{ matrix.llvm_mode }}" \
--gh-run-id "${{ github.run_id }}" \
--gh-artifact "llvm-pruned-libs-${{ matrix.os }}" \
--gh-download-dir "artifacts" \
--max-attempts 60 \
--sleep-seconds 60
# For cross-compiled LTO builds, apply the native prune manifest.
# The unused library set is arch-independent (same API surface).
- name: Apply pruned-libs manifest (cross-compile + LTO)
if: matrix.target_triple && matrix.lto == 'ON'
shell: bash
env:
GH_TOKEN: ${{ github.token }}
run: |
MANIFEST="pruned-libs-${{ matrix.os }}.json"
python3 scripts/prune-llvm-bin.py \
--action apply \
--manifest "${MANIFEST}" \
--install-dir ".llvm/build-install/lib" \
--build-dir "build/${{ matrix.llvm_mode }}" \
--build-dir "build" \
--gh-run-id "${{ github.run_id }}" \
--gh-artifact "llvm-pruned-libs-${{ matrix.os }}" \
--gh-download-dir "artifacts" \
@@ -269,35 +157,23 @@ jobs:
MODE_TAG="debug"
fi
# Determine arch/platform/toolchain from target triple or runner OS
if [[ -n "${{ matrix.target_triple }}" ]]; then
case "${{ matrix.target_triple }}" in
x86_64-apple-darwin)
ARCH="x64"; PLATFORM="macos"; TOOLCHAIN="clang" ;;
aarch64-linux-gnu)
ARCH="aarch64"; PLATFORM="linux"; TOOLCHAIN="gnu" ;;
aarch64-pc-windows-msvc)
ARCH="aarch64"; PLATFORM="windows"; TOOLCHAIN="msvc" ;;
esac
else
ARCH="x64"
PLATFORM="linux"
TOOLCHAIN="gnu"
if [[ "${{ matrix.os }}" == windows-* ]]; then
PLATFORM="windows"
TOOLCHAIN="msvc"
elif [[ "${{ matrix.os }}" == macos-* ]]; then
ARCH="arm64"
PLATFORM="macos"
TOOLCHAIN="clang"
fi
ARCH="x64"
PLATFORM="linux"
TOOLCHAIN="gnu"
if [[ "${{ matrix.os }}" == windows-* ]]; then
PLATFORM="windows"
TOOLCHAIN="msvc"
elif [[ "${{ matrix.os }}" == macos-* ]]; then
ARCH="arm64"
PLATFORM="macos"
TOOLCHAIN="clang"
fi
SUFFIX=""
if [[ "${{ matrix.lto }}" == "ON" ]]; then
SUFFIX="-lto"
fi
if [[ "${{ matrix.llvm_mode }}" == "Debug" && "${{ matrix.os }}" != windows-* ]]; then
if [[ "${{ matrix.llvm_mode }}" == "Debug" ]]; then
SUFFIX="${SUFFIX}-asan"
fi
@@ -313,134 +189,3 @@ jobs:
name: ${{ env.LLVM_INSTALL_ARCHIVE }}
path: ${{ env.LLVM_INSTALL_ARCHIVE }}
if-no-files-found: error
test-cross:
needs: build
strategy:
fail-fast: false
matrix:
include:
- os: macos-15-intel
llvm_mode: RelWithDebInfo
target_triple: x86_64-apple-darwin
- os: ubuntu-24.04-arm
llvm_mode: RelWithDebInfo
target_triple: aarch64-linux-gnu
- os: windows-11-arm
llvm_mode: RelWithDebInfo
target_triple: aarch64-pc-windows-msvc
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: test-run
- name: Download cross-compiled clice
uses: actions/download-artifact@v4
with:
name: cross-clice-${{ matrix.target_triple }}-${{ matrix.llvm_mode }}
path: build/${{ matrix.llvm_mode }}/
- name: Make binaries executable
if: runner.os != 'Windows'
run: chmod +x build/${{ matrix.llvm_mode }}/bin/*
- name: Run tests
run: pixi run -e test-run test ${{ matrix.llvm_mode }}
upload:
needs: build
if: ${{ !cancelled() && inputs.llvm_version && !inputs.skip_upload }}
runs-on: ubuntu-24.04
permissions:
contents: read
steps:
- uses: actions/checkout@v4
- name: Download all build artifacts
env:
GH_TOKEN: ${{ github.token }}
run: scripts/download-llvm.sh "${{ github.run_id }}"
- name: Upload to clice-llvm
env:
GH_TOKEN: ${{ secrets.UPLOAD_LLVM }}
TARGET_REPO: clice-io/clice-llvm
run: python3 scripts/upload-llvm.py "${{ inputs.llvm_version }}" "${TARGET_REPO}" "${{ github.run_id }}"
- name: Save manifest for update-clice job
uses: actions/upload-artifact@v4
with:
name: llvm-manifest-final
path: artifacts/llvm-manifest.json
if-no-files-found: error
compression-level: 0
update-clice:
needs: upload
if: ${{ !inputs.skip_pr }}
runs-on: ubuntu-24.04
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v4
- name: Download manifest
uses: actions/download-artifact@v4
with:
name: llvm-manifest-final
path: .
- name: Update manifest and version
run: |
python3 scripts/update-llvm-version.py \
--version "${{ inputs.llvm_version }}" \
--manifest-src llvm-manifest.json \
--manifest-dest config/llvm-manifest.json \
--package-cmake cmake/package.cmake
- name: Create or update PR
env:
GH_TOKEN: ${{ github.token }}
run: |
VERSION="${{ inputs.llvm_version }}"
BRANCH="chore/update-llvm-${VERSION}"
RUN_URL="https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}"
RELEASE_URL="https://github.com/clice-io/clice-llvm/releases/tag/${VERSION}"
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git checkout -b "${BRANCH}"
git add config/llvm-manifest.json cmake/package.cmake
git commit -m "chore: update LLVM to ${VERSION}"
git push --force-with-lease origin "${BRANCH}"
# Check if PR already exists for this branch
EXISTING_PR=$(gh pr list --head "${BRANCH}" --json number --jq '.[0].number // empty')
BODY="$(cat <<EOF
## Summary
- Update LLVM prebuilt binaries to version ${VERSION}
- Updated \`config/llvm-manifest.json\` with new SHA256 hashes
- Updated \`cmake/package.cmake\` version string
**Artifacts:** [clice-llvm release](${RELEASE_URL})
**Build:** [workflow run](${RUN_URL})
> Auto-generated by build-llvm workflow
EOF
)"
if [[ -n "${EXISTING_PR}" ]]; then
echo "Updating existing PR #${EXISTING_PR}"
gh pr edit "${EXISTING_PR}" --body "${BODY}"
else
gh pr create \
--title "chore: update LLVM to ${VERSION}" \
--body "${BODY}" \
--base main
fi

View File

@@ -14,12 +14,6 @@ jobs:
with:
environments: format
- name: Validate update-llvm-version.py can still patch package.cmake
run: |
python3 scripts/update-llvm-version.py --check \
--manifest-dest config/llvm-manifest.json \
--package-cmake cmake/package.cmake
- name: Run formatter
run: pixi run format
continue-on-error: true

View File

@@ -19,7 +19,7 @@ jobs:
clice: ${{ steps.filter.outputs.clice }}
vscode: ${{ steps.filter.outputs.vscode }}
cmake: ${{ steps.filter.outputs.cmake }}
xmake: ${{ steps.filter.outputs.xmake }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
@@ -46,31 +46,13 @@ jobs:
- 'tests/**'
- 'config/**'
- '.github/workflows/test-cmake.yml'
conventional-commit:
if: ${{ !startsWith(github.ref, 'refs/tags/') }}
runs-on: ubuntu-latest
steps:
- name: Check conventional commit format
env:
IS_PR: ${{ github.event_name == 'pull_request' }}
PR_TITLE: ${{ github.event.pull_request.title }}
COMMIT_MSG: ${{ github.event.head_commit.message }}
run: |
pattern='^(feat|fix|refactor|chore|build|ci|docs|test|perf|style|revert)(\(.+\))?: .+'
if [[ "$IS_PR" == "true" ]]; then
subject="$PR_TITLE"
label="PR title"
else
subject=$(echo "$COMMIT_MSG" | head -n1)
label="Commit message"
fi
if [[ ! "$subject" =~ $pattern ]]; then
echo "::error::$label must follow conventional commit format: type(scope)?: description"
echo " Valid types: feat, fix, refactor, chore, build, ci, docs, test, perf, style, revert"
echo " Got: '$subject'"
exit 1
fi
xmake:
- 'xmake.lua'
- 'src/**'
- 'include/**'
- 'tests/**'
- 'config/**'
- '.github/workflows/test-xmake.yml'
format:
needs: changes
@@ -100,6 +82,11 @@ jobs:
if: ${{ needs.changes.outputs.cmake == 'true' }}
uses: ./.github/workflows/test-cmake.yml
# xmake:
# needs: changes
# if: ${{ needs.changes.outputs.xmake == 'true' }}
# uses: ./.github/workflows/test-xmake.yml
release-clice:
permissions:
contents: write
@@ -117,17 +104,16 @@ jobs:
checks-passed:
if: ${{ always() && !startsWith(github.ref, 'refs/tags/') }}
needs:
- conventional-commit
- format
- deploy
# - clice
- vscode
- cmake
# - xmake
runs-on: ubuntu-latest
steps:
- name: Check results
uses: re-actors/alls-green@release/v1
with:
allowed-skips: conventional-commit,format,deploy,clice,vscode,cmake
allowed-skips: format,deploy,clice,vscode,cmake,xmake
jobs: ${{ toJSON(needs) }}

View File

@@ -9,7 +9,6 @@ jobs:
fail-fast: false
matrix:
include:
# Native builds
- os: windows-2025
artifact_name: clice.zip
asset_name: clice-x64-windows-msvc.zip
@@ -28,63 +27,43 @@ jobs:
symbol_artifact_name: clice-symbol.tar.gz
symbol_asset_name: clice-arm64-macos-darwin-symbol.tar.gz
# Cross-compilation builds
- os: macos-15
target_triple: x86_64-apple-darwin
pixi_env: cross-macos-x64
artifact_name: clice.tar.gz
asset_name: clice-x86_64-macos-darwin.tar.gz
symbol_artifact_name: clice-symbol.tar.gz
symbol_asset_name: clice-x86_64-macos-darwin-symbol.tar.gz
- os: ubuntu-24.04
target_triple: aarch64-linux-gnu
pixi_env: cross-linux-aarch64
artifact_name: clice.tar.gz
asset_name: clice-aarch64-linux-gnu.tar.gz
symbol_artifact_name: clice-symbol.tar.gz
symbol_asset_name: clice-aarch64-linux-gnu-symbol.tar.gz
- os: windows-2025
target_triple: aarch64-pc-windows-msvc
pixi_env: cross-windows-arm64
artifact_name: clice.zip
asset_name: clice-aarch64-windows-msvc.zip
symbol_artifact_name: clice-symbol.zip
symbol_asset_name: clice-aarch64-windows-msvc-symbol.zip
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup xmake
uses: xmake-io/github-action-setup-xmake@v1
with:
xmake-version: 3.0.5
actions-cache-folder: ".xmake-cache"
actions-cache-key: ${{ matrix.os }}
package-cache: true
package-cache-key: ${{ matrix.os }}-pkg-release-v1
build-cache: true
build-cache-key: ${{ matrix.os }}-build-release-v1
- uses: ./.github/actions/setup-pixi
with:
environments: ${{ matrix.pixi_env || 'package' }}
environments: package
- name: Package (native)
if: ${{ !matrix.target_triple }}
run: pixi run package
- name: Package (cross-compile)
if: ${{ matrix.target_triple }}
- name: Remove ci llvm toolchain on Windows
if: runner.os == 'Windows'
run: |
ENV="${{ matrix.pixi_env }}"
pixi run -e "$ENV" package-config -- \
"-DCLICE_TARGET_TRIPLE=${{ matrix.target_triple }}"
pixi run -e "$ENV" cmake-build
# @see https://github.com/xmake-io/xmake/issues/7158
xmake lua os.rmdir "C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/Llvm"
xmake lua os.rmdir "C:/Program Files/LLVM"
- name: Package
run: pixi run package
- name: Upload Main Package to Release
if: startsWith(github.ref, 'refs/tags/v')
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: build/RelWithDebInfo/${{ matrix.artifact_name }}
file: build/xpack/clice/${{ matrix.artifact_name }}
asset_name: ${{ matrix.asset_name }}
tag: ${{ github.ref }}
overwrite: true
@@ -94,7 +73,7 @@ jobs:
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: build/RelWithDebInfo/${{ matrix.symbol_artifact_name }}
file: build/xpack/clice/${{ matrix.symbol_artifact_name }}
asset_name: ${{ matrix.symbol_asset_name }}
tag: ${{ github.ref }}
overwrite: true

View File

@@ -17,134 +17,53 @@ jobs:
strategy:
fail-fast: false
matrix:
include:
# Native builds
- os: windows-2025
build_type: RelWithDebInfo
- os: ubuntu-24.04
build_type: Debug
- os: ubuntu-24.04
build_type: RelWithDebInfo
- os: macos-15
build_type: Debug
- os: macos-15
build_type: RelWithDebInfo
# Cross-compile (build only; tests run on native runners)
- os: macos-15
build_type: RelWithDebInfo
target_triple: x86_64-apple-darwin
build_only: true
- os: ubuntu-24.04
build_type: RelWithDebInfo
target_triple: aarch64-linux-gnu
build_only: true
pixi_env: cross-linux-aarch64
- os: windows-2025
build_type: RelWithDebInfo
target_triple: aarch64-pc-windows-msvc
build_only: true
pixi_env: cross-windows-arm64
os: [windows-2025, ubuntu-24.04, macos-15]
build_type: [Debug, RelWithDebInfo]
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: ${{ matrix.pixi_env || 'default' }}
- name: Restore compiler cache
uses: actions/cache@v4
with:
path: ${{ runner.os == 'Windows' && '.cache/sccache' || '.cache/ccache' }}
key: ${{ runner.os }}-${{ matrix.build_type }}-${{ matrix.target_triple || 'native' }}-ccache-${{ github.sha }}
key: ${{ runner.os }}-${{ matrix.build_type }}-ccache-${{ github.sha }}
restore-keys: |
${{ runner.os }}-${{ matrix.build_type }}-${{ matrix.target_triple || 'native' }}-ccache-
${{ runner.os }}-${{ matrix.build_type }}-ccache-
- name: Zero cache stats
run: |
ENV="${{ matrix.pixi_env || 'default' }}"
if [ "$RUNNER_OS" = "Windows" ]; then
pixi run -e "$ENV" -- sccache --stop-server || true
pixi run -e "$ENV" -- sccache --zero-stats || true
pixi run -- sccache --stop-server || true
pixi run -- sccache --zero-stats || true
else
pixi run -e "$ENV" -- ccache --zero-stats || true
pixi run -- ccache --zero-stats || true
fi
shell: bash
- name: Build (native)
if: ${{ !matrix.target_triple }}
- name: Build
run: pixi run build ${{ matrix.build_type }} ON
- name: Build (cross-compile)
if: ${{ matrix.target_triple }}
shell: bash
run: |
ENV="${{ matrix.pixi_env || 'default' }}"
pixi run -e "$ENV" cmake-config ${{ matrix.build_type }} OFF -- \
"-DCLICE_TARGET_TRIPLE=${{ matrix.target_triple }}"
pixi run -e "$ENV" cmake-build ${{ matrix.build_type }}
- name: Unit Test
run: pixi run unit-test ${{ matrix.build_type }}
- name: Upload cross-compiled binaries
if: ${{ matrix.build_only }}
uses: actions/upload-artifact@v4
with:
name: cross-build-${{ matrix.target_triple }}
path: |
build/${{ matrix.build_type }}/bin/
build/${{ matrix.build_type }}/lib/
if-no-files-found: error
retention-days: 1
- name: Integration Test
run: pixi run integration-test ${{ matrix.build_type }}
- name: Run tests
if: ${{ !matrix.build_only }}
run: pixi run test ${{ matrix.build_type }}
- name: Smoke Test
if: success() || failure()
run: pixi run smoke-test ${{ matrix.build_type }}
- name: Print cache stats and stop server
if: always()
run: |
ENV="${{ matrix.pixi_env || 'default' }}"
if [ "$RUNNER_OS" = "Windows" ]; then
pixi run -e "$ENV" -- sccache --show-stats
pixi run -e "$ENV" -- sccache --stop-server || true
pixi run -- sccache --show-stats
pixi run -- sccache --stop-server || true
else
pixi run -e "$ENV" -- ccache --show-stats
pixi run -- ccache --show-stats
fi
shell: bash
test-cross:
needs: build
strategy:
fail-fast: false
matrix:
include:
- os: macos-15-intel
build_type: RelWithDebInfo
target_triple: x86_64-apple-darwin
- os: ubuntu-24.04-arm
build_type: RelWithDebInfo
target_triple: aarch64-linux-gnu
- os: windows-11-arm
build_type: RelWithDebInfo
target_triple: aarch64-pc-windows-msvc
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: test-run
- name: Download cross-compiled binaries
uses: actions/download-artifact@v4
with:
name: cross-build-${{ matrix.target_triple }}
path: build/${{ matrix.build_type }}/
- name: Make binaries executable
if: runner.os != 'Windows'
run: chmod +x build/${{ matrix.build_type }}/bin/*
- name: Run tests
run: pixi run -e test-run test ${{ matrix.build_type }}

42
.github/workflows/test-xmake.yml vendored Normal file
View File

@@ -0,0 +1,42 @@
name: xmake
on:
workflow_call:
jobs:
build:
strategy:
fail-fast: false
matrix:
os: [windows-2025, ubuntu-24.04, macos-15]
build_type: [debug, releasedbg]
exclude:
- os: windows-2025
build_type: debug
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup xmake
uses: xmake-io/github-action-setup-xmake@v1
with:
xmake-version: 3.0.5
actions-cache-folder: ".xmake-cache"
actions-cache-key: ${{ matrix.os }}
package-cache: true
package-cache-key: ${{ matrix.os }}-pixi
build-cache: true
build-cache-key: ${{ matrix.os }}-${{ matrix.build_type }}
- uses: ./.github/actions/setup-pixi
- name: Build
run: pixi run xmake ${{ matrix.build_type }}
- name: Test
run: pixi run xmake-test
- name: Remove llvm package (Linux)
if: runner.os == 'Linux'
run: xmake require --uninstall clice-llvm

13
.gitignore vendored
View File

@@ -35,7 +35,7 @@
*build*/
temp/
.cache/
.xmake/
.llvm*/
.clice/
compile_commands.json
@@ -56,11 +56,10 @@ __pycache__/
tests/unit/Local/
# IDEs & Editors
/.vscode/*
!/.vscode/launch.json
!/.vscode/tasks.json
/.vscode/
.vs/
.idea/
.claude
.clangd
# pixi environments
@@ -69,7 +68,5 @@ tests/unit/Local/
!.pixi/config.toml
.codex/
.claude/*
!.claude/CLAUDE.md
!.claude/commands/
openspec/
.claude/
openspec/

83
.vscode/launch.json vendored
View File

@@ -1,83 +0,0 @@
{
"version": "0.2.0",
"configurations": [
{
"type": "lldb",
"request": "launch",
"name": "Debug clice",
"program": "${workspaceFolder}/build/Debug/bin/clice",
"args": ["--mode=socket", "--port=50051"],
"cwd": "${workspaceFolder}"
},
{
"type": "lldb",
"request": "launch",
"name": "Debug clice (socket, RelWithDebInfo)",
"program": "${workspaceFolder}/build/RelWithDebInfo/bin/clice",
"args": ["--mode=socket", "--port=50051"],
"cwd": "${workspaceFolder}"
},
{
"name": "VSCode Extension (pipe)",
"type": "extensionHost",
"request": "launch",
"args": [
"--disable-extension=llvm-vs-code-extensions.vscode-clangd",
"--disable-extension=ms-vscode.cpptools",
"--disable-extension=ms-vscode.cpptools-extension-pack",
"--extensionDevelopmentPath=${workspaceFolder}/editors/vscode"
],
"env": {
"CLICE_MODE": "pipe"
},
"outFiles": ["${workspaceFolder}/editors/vscode/dist/**/*.js"],
"preLaunchTask": "npm: watch vscode ext"
},
{
"name": "VSCode Extension (socket)",
"type": "extensionHost",
"request": "launch",
"args": [
"--disable-extension=llvm-vs-code-extensions.vscode-clangd",
"--disable-extension=ms-vscode.cpptools",
"--disable-extension=ms-vscode.cpptools-extension-pack",
"--extensionDevelopmentPath=${workspaceFolder}/editors/vscode"
],
"env": {
"CLICE_MODE": "socket"
},
"outFiles": ["${workspaceFolder}/editors/vscode/dist/**/*.js"],
"preLaunchTask": "npm: watch vscode ext"
},
{
"type": "lldb",
"request": "launch",
"name": "Unit Test",
"program": "${workspaceFolder}/build/Debug/bin/unit_tests",
"args": ["--test-dir=./tests/data", "--test-filter=${input:filter}"],
"cwd": "${workspaceFolder}"
},
{
"type": "lldb",
"request": "launch",
"name": "Release Unit Test",
"program": "${workspaceFolder}/build/RelWithDebInfo/bin/unit_tests",
"args": ["--test-dir=./tests/data", "--test-filter=${input:filter}"],
"cwd": "${workspaceFolder}"
}
],
"compounds": [
{
"name": "clice + VSCode Extension (socket)",
"configurations": ["Debug clice", "VSCode Extension (socket)"],
"stopAll": true
}
],
"inputs": [
{
"id": "filter",
"type": "promptString",
"description": "Unit Test Filter"
}
]
}

42
.vscode/tasks.json vendored
View File

@@ -1,42 +0,0 @@
{
"version": "2.0.0",
"tasks": [
{
"label": "npm: install vscode deps",
"type": "shell",
"command": "pnpm",
"args": ["install"],
"options": {
"cwd": "${workspaceFolder}/editors/vscode"
},
"problemMatcher": []
},
{
"label": "npm: watch vscode ext",
"type": "shell",
"command": "pnpm",
"args": ["run", "watch"],
"options": {
"cwd": "${workspaceFolder}/editors/vscode"
},
"dependsOn": "npm: install vscode deps",
"problemMatcher": "$ts-webpack-watch",
"isBackground": true,
"presentation": {
"reveal": "never",
"group": "watchers"
}
},
{
"label": "npm: package vscode ext",
"type": "shell",
"command": "pnpm",
"args": ["run", "package"],
"options": {
"cwd": "${workspaceFolder}/editors/vscode"
},
"dependsOn": "npm: install vscode deps",
"problemMatcher": []
}
]
}

View File

@@ -127,16 +127,9 @@ endif()
set(FBS_SCHEMA_FILE "${PROJECT_SOURCE_DIR}/src/index/schema.fbs")
set(GENERATED_HEADER "${PROJECT_BINARY_DIR}/generated/schema_generated.h")
if(CMAKE_CROSSCOMPILING)
find_program(FLATC_EXECUTABLE flatc REQUIRED)
set(FLATC_CMD "${FLATC_EXECUTABLE}")
else()
set(FLATC_CMD "$<TARGET_FILE:flatc>")
endif()
add_custom_command(
OUTPUT "${GENERATED_HEADER}"
COMMAND ${FLATC_CMD} --cpp -o "${PROJECT_BINARY_DIR}/generated" "${FBS_SCHEMA_FILE}"
COMMAND $<TARGET_FILE:flatc> --cpp -o "${PROJECT_BINARY_DIR}/generated" "${FBS_SCHEMA_FILE}"
DEPENDS "${FBS_SCHEMA_FILE}"
COMMENT "Generating C++ header from ${FBS_SCHEMA_FILE}"
)
@@ -158,13 +151,13 @@ target_link_libraries(clice-core PUBLIC
spdlog::spdlog
roaring::roaring
flatbuffers
kota::ipc::lsp
kota::codec::toml
eventide::ipc::lsp
eventide::serde::toml
simdjson::simdjson
)
add_executable(clice "${PROJECT_SOURCE_DIR}/src/clice.cc")
target_link_libraries(clice PRIVATE clice::core kota::deco)
target_link_libraries(clice PRIVATE clice::core eventide::deco)
install(TARGETS clice RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
add_custom_target(copy_clang_resource ALL
@@ -196,7 +189,7 @@ if(CLICE_ENABLE_TEST)
"${PROJECT_SOURCE_DIR}/src"
"${PROJECT_SOURCE_DIR}/tests/unit"
)
target_link_libraries(unit_tests PRIVATE clice::core kota::zest kota::deco)
target_link_libraries(unit_tests PRIVATE clice::core eventide::zest eventide::deco)
endif()
if(CLICE_ENABLE_BENCHMARK)
@@ -206,7 +199,7 @@ if(CLICE_ENABLE_BENCHMARK)
target_include_directories(scan_benchmark PRIVATE
"${PROJECT_SOURCE_DIR}/src"
)
target_link_libraries(scan_benchmark PRIVATE clice::core kota::deco)
target_link_libraries(scan_benchmark PRIVATE clice::core eventide::deco)
endif()
if(CLICE_RELEASE)

View File

@@ -21,15 +21,17 @@
#include <thread>
#include "command/command.h"
#include "eventide/deco/deco.h"
#include "eventide/serde/json/serializer.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "support/path_pool.h"
#include "syntax/dependency_graph.h"
#include "kota/codec/json/json.h"
#include "kota/deco/deco.h"
#include "llvm/Support/FileSystem.h"
namespace et = eventide;
using namespace clice;
struct BenchmarkOptions {
@@ -95,7 +97,7 @@ void export_graph_json(const PathPool& path_pool,
export_data.files.push_back(std::move(node));
}
auto json = kota::codec::json::to_json(export_data);
auto json = et::serde::json::to_json(export_data);
if(!json) {
std::println(stderr, "Failed to serialize dependency graph");
return;
@@ -219,8 +221,8 @@ void print_report(const ScanReport& report) {
}
int main(int argc, const char** argv) {
auto args = kota::deco::util::argvify(argc, argv);
auto result = kota::deco::cli::parse<BenchmarkOptions>(args);
auto args = deco::util::argvify(argc, argv);
auto result = deco::cli::parse<BenchmarkOptions>(args);
if(!result.has_value()) {
std::println(stderr, "Error: {}", result.error().message);
@@ -231,7 +233,7 @@ int main(int argc, const char** argv) {
if(opts.help.value_or(false) || !opts.cdb_path.has_value()) {
std::ostringstream oss;
kota::deco::cli::write_usage_for<BenchmarkOptions>(oss, "scan_benchmark [OPTIONS] <cdb>");
deco::cli::write_usage_for<BenchmarkOptions>(oss, "scan_benchmark [OPTIONS] <cdb>");
std::print("{}", oss.str());
return opts.help.value_or(false) ? 0 : 1;
}

View File

@@ -25,22 +25,6 @@ function(setup_llvm LLVM_VERSION)
list(APPEND LLVM_SETUP_ARGS "--offline")
endif()
if(DEFINED CLICE_TARGET_TRIPLE)
if(CLICE_TARGET_TRIPLE MATCHES "linux")
list(APPEND LLVM_SETUP_ARGS "--target-platform" "Linux")
elseif(CLICE_TARGET_TRIPLE MATCHES "darwin")
list(APPEND LLVM_SETUP_ARGS "--target-platform" "macosx")
elseif(CLICE_TARGET_TRIPLE MATCHES "windows")
list(APPEND LLVM_SETUP_ARGS "--target-platform" "Windows")
endif()
if(CLICE_TARGET_TRIPLE MATCHES "^aarch64")
list(APPEND LLVM_SETUP_ARGS "--target-arch" "arm64")
elseif(CLICE_TARGET_TRIPLE MATCHES "^x86_64")
list(APPEND LLVM_SETUP_ARGS "--target-arch" "x64")
endif()
endif()
execute_process(
COMMAND "${Python3_EXECUTABLE}" "${LLVM_SETUP_SCRIPT}" ${LLVM_SETUP_ARGS}
RESULT_VARIABLE LLVM_SETUP_RESULT
@@ -117,15 +101,8 @@ function(setup_llvm LLVM_VERSION)
clangToolingSyntax
)
else()
file(GLOB LLVM_LIBRARIES CONFIGURE_DEPENDS "${LLVM_INSTALL_PATH}/lib/${CMAKE_STATIC_LIBRARY_PREFIX}LLVM[a-zA-Z]*${CMAKE_STATIC_LIBRARY_SUFFIX}")
file(GLOB CLANG_LIBRARIES CONFIGURE_DEPENDS "${LLVM_INSTALL_PATH}/lib/${CMAKE_STATIC_LIBRARY_PREFIX}clang[a-zA-Z]*${CMAKE_STATIC_LIBRARY_SUFFIX}")
# TODO: find a better way to find out whether zlib and zstd are needed
# Currently link if present in the LLVM lib directory
file(GLOB OTHER_REQUIRED_LIBS CONFIGURE_DEPENDS
"${LLVM_INSTALL_PATH}/lib/${CMAKE_STATIC_LIBRARY_PREFIX}z${CMAKE_STATIC_LIBRARY_SUFFIX}"
"${LLVM_INSTALL_PATH}/lib/${CMAKE_STATIC_LIBRARY_PREFIX}zstd${CMAKE_STATIC_LIBRARY_SUFFIX}"
)
target_link_libraries(llvm-libs INTERFACE ${LLVM_LIBRARIES} ${CLANG_LIBRARIES} ${OTHER_REQUIRED_LIBS})
file(GLOB LLVM_LIBRARIES CONFIGURE_DEPENDS "${LLVM_INSTALL_PATH}/lib/*${CMAKE_STATIC_LIBRARY_SUFFIX}")
target_link_libraries(llvm-libs INTERFACE ${LLVM_LIBRARIES})
target_compile_definitions(llvm-libs INTERFACE CLANG_BUILD_STATIC=1)
endif()
endfunction()

View File

@@ -1,7 +1,7 @@
include_guard()
include(${CMAKE_CURRENT_LIST_DIR}/llvm.cmake)
setup_llvm("21.1.8")
setup_llvm("21.1.4+r1")
# install dependencies
include(FetchContent)
@@ -39,18 +39,18 @@ set(FLATBUFFERS_BUILD_TESTS OFF CACHE BOOL "" FORCE)
set(FLATBUFFERS_BUILD_FLATHASH OFF CACHE BOOL "" FORCE)
FetchContent_Declare(
kotatsu
GIT_REPOSITORY https://github.com/clice-io/kotatsu
eventide
GIT_REPOSITORY https://github.com/clice-io/eventide
GIT_TAG main
GIT_SHALLOW TRUE
)
set(KOTA_ENABLE_ZEST ON)
set(KOTA_ENABLE_TEST OFF)
set(KOTA_CODEC_ENABLE_SIMDJSON ON)
set(KOTA_CODEC_ENABLE_YYJSON ON)
set(KOTA_CODEC_ENABLE_TOML ON)
set(KOTA_ENABLE_EXCEPTIONS OFF)
set(KOTA_ENABLE_RTTI OFF)
set(ETD_ENABLE_ZEST ON)
set(ETD_ENABLE_TEST OFF)
set(ETD_SERDE_ENABLE_SIMDJSON ON)
set(ETD_SERDE_ENABLE_YYJSON ON)
set(ETD_SERDE_ENABLE_TOML ON)
set(ETD_ENABLE_EXCEPTIONS OFF)
set(ETD_ENABLE_RTTI OFF)
FetchContent_MakeAvailable(kotatsu spdlog croaring flatbuffers)
FetchContent_MakeAvailable(eventide spdlog croaring flatbuffers)

View File

@@ -1,29 +1,5 @@
cmake_minimum_required(VERSION 3.30)
# Cross-compilation support via CLICE_TARGET_TRIPLE.
# Examples:
# -DCLICE_TARGET_TRIPLE=x86_64-apple-darwin (macOS x64 from arm64)
# -DCLICE_TARGET_TRIPLE=aarch64-linux-gnu (Linux arm64 from x64)
# -DCLICE_TARGET_TRIPLE=aarch64-pc-windows-msvc (Windows arm64 from x64)
if(DEFINED CLICE_TARGET_TRIPLE)
if(CLICE_TARGET_TRIPLE MATCHES "^x86_64-apple-darwin")
set(CMAKE_OSX_ARCHITECTURES "x86_64" CACHE STRING "")
elseif(CLICE_TARGET_TRIPLE MATCHES "^aarch64-.*linux")
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
set(CMAKE_C_COMPILER_TARGET "aarch64-linux-gnu" CACHE STRING "")
set(CMAKE_CXX_COMPILER_TARGET "aarch64-linux-gnu" CACHE STRING "")
if(DEFINED ENV{CONDA_PREFIX} AND NOT DEFINED CMAKE_SYSROOT)
set(CMAKE_SYSROOT "$ENV{CONDA_PREFIX}/aarch64-conda-linux-gnu/sysroot" CACHE PATH "")
endif()
elseif(CLICE_TARGET_TRIPLE MATCHES "^aarch64-.*-windows")
set(CMAKE_SYSTEM_NAME Windows)
set(CMAKE_SYSTEM_PROCESSOR ARM64)
set(CMAKE_C_COMPILER_TARGET "aarch64-pc-windows-msvc" CACHE STRING "")
set(CMAKE_CXX_COMPILER_TARGET "aarch64-pc-windows-msvc" CACHE STRING "")
endif()
endif()
set(CMAKE_C_COMPILER clang CACHE STRING "")
set(CMAKE_CXX_COMPILER clang++ CACHE STRING "")

View File

@@ -1,142 +1,83 @@
[
{
"version": "21.1.8",
"filename": "aarch64-linux-gnu-releasedbg-lto.tar.xz",
"sha256": "f3444ee840b50933c23656cbee7c4d010e752ac55ca66095b97f7c0e997b13b5",
"lto": true,
"asan": false,
"platform": "linux",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"filename": "aarch64-linux-gnu-releasedbg.tar.xz",
"sha256": "b9012bf059e4d8673fb564b5780e5fc78c6a2e47f5cc6a39f444d1879b42dd2a",
"lto": false,
"asan": false,
"platform": "linux",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"filename": "aarch64-windows-msvc-releasedbg-lto.tar.xz",
"sha256": "8870d16141ba7f9ea12f5147b8d91329abbbaa4376cd4576667dd323d896dd08",
"lto": true,
"asan": false,
"platform": "windows",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"filename": "aarch64-windows-msvc-releasedbg.tar.xz",
"sha256": "ad394e79ec85dd40f942671bb0342ffe54a103eb2baabacb773999d57d80134b",
"lto": false,
"asan": false,
"platform": "windows",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"version": "21.1.4+r1",
"filename": "arm64-macos-clang-debug-asan.tar.xz",
"sha256": "b02d20e4f7294ee33f49a09dfdd765b3b44135e003ef50e3a760aeee39e3f993",
"sha256": "7da4b7d63edefecaf11773e7e701c575140d1a07329bbbb038673b6ee4516ff5",
"lto": false,
"asan": true,
"platform": "macosx",
"arch": "arm64",
"build_type": "Debug"
},
{
"version": "21.1.8",
"version": "21.1.4+r1",
"filename": "arm64-macos-clang-releasedbg-lto.tar.xz",
"sha256": "e40c21eb0d0b91d9d4ab31212a5cb01ea46707f5c29839414567857e4147604d",
"sha256": "300455b169448f9f01ae95e3bc269f489558a4ca3955e3032171cc75feca0e30",
"lto": true,
"asan": false,
"platform": "macosx",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"version": "21.1.4+r1",
"filename": "arm64-macos-clang-releasedbg.tar.xz",
"sha256": "e1b01de34f0edfd41c118e4981a93afb35556ae369597e864f4a393db623b926",
"sha256": "9abfc6cd65b957d734ffb97610a634fb4a66d3fbe0fcfb5a1c9124ef693c1495",
"lto": false,
"asan": false,
"platform": "macosx",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"version": "21.1.4+r1",
"filename": "x64-linux-gnu-debug-asan.tar.xz",
"sha256": "76bb82d822b5377fb5e0fac8abcfba125142e6a0acc02bb36d1fa1532a268646",
"sha256": "c1ad3ec476911596a842ac67dd9c9c9475ce9f0a77b81101d3c801840292e7bc",
"lto": false,
"asan": true,
"platform": "linux",
"arch": "x64",
"build_type": "Debug"
},
{
"version": "21.1.8",
"version": "21.1.4+r1",
"filename": "x64-linux-gnu-releasedbg-lto.tar.xz",
"sha256": "32f5edddec1e689124f045b586fb402ae30febc05203af7391b088bc8494cd53",
"sha256": "8a869c2184d139dbba704e2d712e7a68336458ad2d70622b3eb906c3e3511e54",
"lto": true,
"asan": false,
"platform": "linux",
"arch": "x64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"version": "21.1.4+r1",
"filename": "x64-linux-gnu-releasedbg.tar.xz",
"sha256": "8ba3c84f23a2a81a86c54780754a61adf99048aa2ac0dc9b9708d0f842d553de",
"sha256": "552bab86f715d4f2c027f07eaaf5b3d6b8e430af0b74b470142f3f00da4feec6",
"lto": false,
"asan": false,
"platform": "linux",
"arch": "x64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"filename": "x64-macos-clang-releasedbg-lto.tar.xz",
"sha256": "97e81d6296896d7237f118f728d05291707b9e4e5791e07ce4be8aee0517505d",
"lto": true,
"asan": false,
"platform": "macosx",
"arch": "x64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"filename": "x64-macos-clang-releasedbg.tar.xz",
"sha256": "53c13f8e1082fa2fe2f9c05303de48cb3133bf5f24271f4b3062f1dec578159c",
"version": "21.1.4+r1",
"filename": "x64-windows-msvc-debug-asan.tar.xz",
"sha256": "093667a493d336c22ff3c604c5f1fea2a7d2c927c1179cec44e9a03726906ac1",
"lto": false,
"asan": false,
"platform": "macosx",
"arch": "x64",
"build_type": "RelWithDebInfo"
"asan": true,
"platform": "windows",
"build_type": "Debug"
},
{
"version": "21.1.8",
"version": "21.1.4+r1",
"filename": "x64-windows-msvc-releasedbg-lto.tar.xz",
"sha256": "16bcf0e4cbc3d2b1204edd619a3837004dacea28eeff0a101c8d0212f936427d",
"sha256": "010539e85621dc3c6ecf359d899feb4075aeca5d0bba6625cdbec0e570e79129",
"lto": true,
"asan": false,
"platform": "windows",
"arch": "x64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"version": "21.1.4+r1",
"filename": "x64-windows-msvc-releasedbg.tar.xz",
"sha256": "81d31fad05e200726c8178314b0b2045c947483dddd8cb974f4c376ae5f441fa",
"sha256": "f473c09fbea10053fac00be409d75dc228d4a38bcbc5e4aeb58b56a4b0dde78e",
"lto": false,
"asan": false,
"platform": "windows",
"arch": "x64",
"build_type": "RelWithDebInfo"
}
]

View File

@@ -20,7 +20,7 @@ max_active_file = 8
cache_dir = "${workspace}/.clice/cache"
# Directory for storing index files.
index_dir = "${workspace}/.clice/index"
logging_dir = "${workspace}/.clice/logs"
logging_dir = "${workspace}/.clice/logging"
# Compile commands files or directories to search for compile_commands.json files.
compile_commands_paths = ["${workspace}/build"]

View File

@@ -91,7 +91,7 @@ The worker pool (`src/server/worker_pool.cpp`) manages spawning and communicatin
### Communication
Workers communicate with the master via **stdio pipes** using a **bincode** serialization format (via `kota::ipc::BincodePeer`). This is more compact and faster than JSON for internal IPC, while the master handles JSON for the external LSP protocol.
Workers communicate with the master via **stdio pipes** using a **bincode** serialization format (via `eventide::ipc::BincodePeer`). This is more compact and faster than JSON for internal IPC, while the master handles JSON for the external LSP protocol.
### Stateful Worker Routing
@@ -111,7 +111,7 @@ The stateful worker (`src/server/stateful_worker.cpp`) caches compiled ASTs in m
- **Feature queries**: Look up the cached AST and invoke the corresponding `feature::*` function (hover, semantic tokens, etc.), serializing the result to JSON
- **Document updates**: Received as notifications — the worker updates the stored text and marks the document as `dirty`, causing feature queries to return `null` until recompilation
- **Eviction**: LRU-based; evicts the oldest document when capacity is exceeded, notifying the master
- **Concurrency**: Each document has a per-document `kota::mutex` (strand) to serialize compilation and feature queries. Heavy work (compilation, feature extraction) runs on a thread pool via `kota::queue`.
- **Concurrency**: Each document has a per-document `et::mutex` (strand) to serialize compilation and feature queries. Heavy work (compilation, feature extraction) runs on a thread pool via `et::queue`.
## Stateless Worker
@@ -123,7 +123,7 @@ The stateless worker (`src/server/stateless_worker.cpp`) handles one-shot reques
- **Build PCM**: Compiles a C++20 module interface to a temporary file
- **Index**: Compiles a file for indexing (TUIndex generation — currently a stub)
All requests are dispatched to a thread pool via `kota::queue`.
All requests are dispatched to a thread pool via `et::queue`.
## Compile Graph
@@ -132,7 +132,7 @@ The compile graph (`src/server/compile_graph.cpp`) tracks compilation unit depen
- **Registration**: Each file registers its included dependencies
- **Cascade invalidation**: When a file changes, all transitive dependents are marked dirty and their ongoing compilations are cancelled
- **Dependency compilation**: Before compiling a file, `compile_deps` ensures all dependencies (PCH, PCMs) are built first
- **Cancellation**: Uses `kota::cancellation_source` to abort in-flight compilations when files are invalidated
- **Cancellation**: Uses `et::cancellation_source` to abort in-flight compilations when files are invalidated
## Configuration

View File

@@ -32,6 +32,18 @@ pixi run integration-test Debug
> [!TIP]
> If you want to develop directly with `cmake`, `ninja`, `clang++`, etc., run `pixi shell` to enter a shell with all env vars configured.
### XMake
We also support building with XMake:
```shell
# config & build (default releasedbg)
pixi run xmake
# unit & integration
pixi run xmake-test
```
## Manual Build
If you plan to build manually, first ensure your toolchain matches the versions defined in `pixi.toml`.
@@ -58,13 +70,30 @@ Optional build options:
| CLICE_USE_LIBCXX | OFF | Build clice with libc++ (adds `-std=libc++`); if enabled, ensure the LLVM libs are also built with libc++ |
| CLICE_CI_ENVIRONMENT | OFF | Enable the `CLICE_CI_ENVIRONMENT` macro; some tests only run in CI |
### XMake
Build clice with:
```bash
xmake f -c --mode=releasedbg --toolchain=clang
xmake build --all
```
Optional build options:
| Option | Default | Effect |
| ------------- | ------- | ---------------------------------------- |
| --llvm | "" | Build clice with LLVM from a custom path |
| --enable_test | false | Build clice unit tests |
| --ci | false | Enable `CLICE_CI_ENVIRONMENT` |
## About LLVM
clice calls Clang APIs to parse C++ code, so it must link against LLVM/Clang. Because clice uses Clang's private headers (usually absent from distro packages), the system LLVM package cannot be used directly.
Two ways to satisfy this dependency:
1. We publish prebuilt binaries of the LLVM version we use at [clice-llvm](https://github.com/clice-io/clice-llvm/releases) for CI and release builds. During builds, cmake downloads these LLVM libs by default.
1. We publish prebuilt binaries of the LLVM version we use at [clice-llvm](https://github.com/clice-io/clice-llvm/releases) for CI and release builds. During builds, cmake and xmake download these LLVM libs by default.
> [!IMPORTANT]
>

View File

@@ -18,6 +18,13 @@ We use pytest to run integration tests. Please refer to `pyproject.toml` to inst
$ pytest -s --log-cli-level=INFO tests/integration --executable=./build/bin/clice
```
If you use xmake as your build system, you can run the tests directly with xmake:
```shell
$ xmake run --verbose unit_tests
$ xmake test --verbose integration_tests/default
```
## Debug
If you want to attach a debugger to clice for debugging, it is recommended to first start clice in socket mode independently, and then connect the client to it.

View File

@@ -54,73 +54,14 @@ bazel run @hedron_compile_commands//:refresh_all
### Visual Studio
Visual Studio (2019 16.1+) can generate a compilation database via CMake integration. Open your project as a CMake project, then configure the generation in `CMakeSettings.json`:
```json
{
"configurations": [
{
"name": "x64-Debug",
"generator": "Ninja",
"buildRoot": "${projectDir}\\build",
"cmakeCommandArgs": "-DCMAKE_EXPORT_COMPILE_COMMANDS=ON"
}
]
}
```
Alternatively, for MSBuild-based projects (`.vcxproj`), you can use [compiledb-vs](https://github.com/pjbroad/compiledb-vs) or [catter](https://github.com/clice-io/catter) to generate the compilation database.
TODO:
### Makefile
For Makefile-based projects, use [bear](https://github.com/rizsotto/Bear) to intercept compilation commands:
```bash
bear -- make
```
This will generate a `compile_commands.json` in the current directory. Note that `bear` requires a clean build to capture all commands — run `make clean` before `bear -- make` if needed.
Alternatively, if you use GNU Make, you can use [compiledb](https://github.com/nicktimko/compiledb):
```bash
compiledb make
```
### Meson
Meson generates a compilation database automatically during setup:
```bash
meson setup build
```
The `compile_commands.json` will be in the `build` directory.
TODO:
### Xmake
Use one of the following approaches to generate a compilation database.
#### Command Line
Run the following command to manually generate a compilation database:
```bash
xmake project -k compile_commands --lsp=clangd build
```
> Compilation database generated manually doesn't automatically update itself. Re-generate if changes are made to the project.
#### VSCode Extension
The Xmake official VSCode extension automatically generates the compilation database when `xmake.lua` is updated. However, it generates the database to the `.vscode` directory by default. Add this setting in `settings.json`:
```json
"xmake.compileCommandsDirectory": "build"
```
to explicitly ask the extension to generate the compilation database in `build`.
### Others
For any other build system, you can use [catter](https://github.com/clice-io/catter) to generate a compilation database. It captures compilation commands through a fake compiler approach and is designed to work reliably with any build system that invokes a compiler executable.
For any other build system, you can try using [bear](https://github.com/rizsotto/Bear) or [scan-build](https://github.com/rizsotto/scan-build) to intercept compilation commands and obtain the compilation database (no guarantee of success). We plan to write a **new tool** in the future that captures compilation commands through a fake compiler approach.

View File

@@ -32,6 +32,18 @@ pixi run integration-test Debug
> [!TIP]
> 如果你想直接使用 `cmake`, `ninja`, `clang++` 等命令进行开发,请运行 `pixi shell` 进入已配置好环境变量的终端
### XMake
我们同样支持使用 XMake 构建:
```shell
# config & build (default releasedbg)
pixi run xmake
# unit & integration
pixi run xmake-test
```
## Manual Build
如果你打算手动构建,请务必先确认你的工具链满足 pixi.toml 中定义的版本要求。
@@ -58,13 +70,30 @@ cmake -B build -G Ninja \
| CLICE_USE_LIBCXX | OFF | 是否使用 libc++ 来构建 clice添加 `-std=libc++`),如果开启,请确保 LLVM 库也是使用 libc++ 编译的 |
| CLICE_CI_ENVIRONMENT | OFF | 是否打开 `CLICE_CI_ENVIRONMENT` 这个宏,有些测试在 CI 环境才会执行 |
### XMake
使用如下命令即可构建 clice
```bash
xmake f -c --mode=releasedbg --toolchain=clang
xmake build --all
```
可选的构建选项:
| 选项 | 默认值 | 效果 |
| ------------- | ------ | ------------------------------------ |
| --llvm | "" | 使用自定义路径的 LLVM 库来构建 clice |
| --enable_test | false | 是否构建 clice 的单元测试 |
| --ci | false | 是否打开 `CLICE_CI_ENVIRONMENT` |
## About LLVM
clice 调用 Clang API 来解析 C++ 代码,因此必须链接 LLVM/Clang 库。由于 clice 使用了 Clang 的私有头文件(这些文件通常不包含在发行版中),不能直接使用系统安装的 LLVM 包。
主要有两种方式解决这个依赖问题:
1. 我们在 [clice-llvm](https://github.com/clice-io/clice-llvm/releases) 上会发布使用的 LLVM 版本的预编译二进制,用于 CI 或者 release 构建。在构建时 cmake 默认会从此处下载 LLVM 库然后使用。
1. 我们在 [clice-llvm](https://github.com/clice-io/clice-llvm/releases) 上会发布使用的 LLVM 版本的预编译二进制,用于 CI 或者 release 构建。在构建时 cmake 和 xmake 默认会从此处下载 LLVM 库然后使用。
> [!IMPORTANT]
>

View File

@@ -30,7 +30,7 @@ pixi run publish-vscode
1. `pixi shell -e node`
2.`editors/vscode` 下运行 `pnpm run watch`(增量构建)
3. VSCode 中使用Run Extension/Launch Extension”调试配置或执行 `code --extensionDevelopmentPath=$(pwd)/editors/vscode`
3. VSCode 中使用Run Extension/Launch Extension” 调试配置,或执行 `code --extensionDevelopmentPath=$(pwd)/editors/vscode`
常用脚本(在 `pixi shell -e node` 下):

View File

@@ -18,6 +18,13 @@ $ ./build/bin/unit_tests --test-dir="./tests/data"
$ pytest -s --log-cli-level=INFO tests/integration --executable=./build/bin/clice
```
如果你使用 xmake 作为构建系统,可以直接通过 xmake 运行测试:
```shell
$ xmake run --verbose unit_tests
$ xmake test --verbose integration_tests/default
```
## Debug
如果想在 clice 上附加调试器并进行调试,推荐先单独以 socket 模式启动 clice然后再将客户端连接到 clice 上

View File

@@ -54,73 +54,14 @@ bazel run @hedron_compile_commands//:refresh_all
### Visual Studio
Visual Studio2019 16.1+)可以通过 CMake 集成来生成编译数据库。将项目作为 CMake 项目打开,然后在 `CMakeSettings.json` 中配置:
```json
{
"configurations": [
{
"name": "x64-Debug",
"generator": "Ninja",
"buildRoot": "${projectDir}\\build",
"cmakeCommandArgs": "-DCMAKE_EXPORT_COMPILE_COMMANDS=ON"
}
]
}
```
对于基于 MSBuild 的项目(`.vcxproj`),可以使用 [compiledb-vs](https://github.com/pjbroad/compiledb-vs) 或 [catter](https://github.com/clice-io/catter) 来生成编译数据库。
TODO:
### Makefile
对于基于 Makefile 的项目,使用 [bear](https://github.com/rizsotto/Bear) 来拦截编译命令:
```bash
bear -- make
```
这会在当前目录生成 `compile_commands.json`。注意 `bear` 需要干净的构建来捕获所有命令——如果需要的话,在运行 `bear -- make` 之前先执行 `make clean`
另外,如果使用 GNU Make也可以使用 [compiledb](https://github.com/nicktimko/compiledb)
```bash
compiledb make
```
### Meson
Meson 在配置阶段会自动生成编译数据库:
```bash
meson setup build
```
`compile_commands.json` 会生成在 `build` 目录下。
TODO:
### Xmake
用下列任意方法生成编译数据库。
#### 命令行手动生成
在命令行中执行以下命令:
```bash
xmake project -k compile_commands --lsp=clangd build
```
> 通过这种方法生成的编译数据库无法自动更新,需要在项目编译配置更改时手动重新生成。
#### VSCode 扩展
Xmake 提供的官方 VSCode 扩展会在 `xmake.lua` 更新时自动生成编译数据库。然而默认情况下,它将编译数据库生成到了 `.vscode` 文件夹。在 `settings.json` 中添加以下配置:
```json
"xmake.compileCommandsDirectory": "build"
```
以将编译数据库的生成目录调整到 `build`,供 clice 使用。
### Others
对于任意其它的构建系统,可以使用 [catter](https://github.com/clice-io/catter) 来生成编译数据库。它通过伪装编译器的方式来捕获编译命令,能够可靠地与任何调用编译器可执行文件的构建系统配合工作
对于任意其它的构建系统,可以尝试使用 [bear](https://github.com/rizsotto/Bear) 或者 [scan-build](https://github.com/rizsotto/scan-build) 来拦截编译命令并获取到编译数据库(不保证成功)。我们计划在未来编写一个**新的工具**,通过假编译器的方式来实现编译命令的捕获

View File

@@ -0,0 +1,9 @@
{
// See http://go.microsoft.com/fwlink/?LinkId=827846
// for the documentation about the extensions.json format
"recommendations": [
"dbaeumer.vscode-eslint",
"amodio.tsl-problem-matcher",
"ms-vscode.extension-test-runner"
]
}

23
editors/vscode/.vscode/launch.json vendored Normal file
View File

@@ -0,0 +1,23 @@
{
"version": "0.2.0",
"configurations": [
{
"name": "Run Extension (socket)",
"type": "extensionHost",
"request": "launch",
"args": ["--disable-extensions", "--extensionDevelopmentPath=${workspaceFolder}"],
"env": { "CLICE_MODE": "socket" },
"outFiles": ["${workspaceFolder}/dist/**/*.js"],
"preLaunchTask": "${defaultBuildTask}"
},
{
"name": "Run Extension (pipe)",
"type": "extensionHost",
"request": "launch",
"args": ["--disable-extensions", "--extensionDevelopmentPath=${workspaceFolder}"],
"env": { "CLICE_MODE": "pipe" },
"outFiles": ["${workspaceFolder}/dist/**/*.js"],
"preLaunchTask": "${defaultBuildTask}"
}
]
}

13
editors/vscode/.vscode/settings.json vendored Normal file
View File

@@ -0,0 +1,13 @@
// Place your settings in this file to overwrite default and user settings.
{
"files.exclude": {
"out": false, // set this to true to hide the "out" folder with the compiled JS files
"dist": false // set this to true to hide the "dist" folder with the compiled JS files
},
"search.exclude": {
"out": true, // set this to false to include "out" folder in search results
"dist": true // set this to false to include "dist" folder in search results
},
// Turn off tsc task auto detection since we have the necessary tasks as npm scripts
"typescript.tsc.autoDetect": "off"
}

37
editors/vscode/.vscode/tasks.json vendored Normal file
View File

@@ -0,0 +1,37 @@
// See https://go.microsoft.com/fwlink/?LinkId=733558
// for the documentation about the tasks.json format
{
"version": "2.0.0",
"tasks": [
{
"type": "npm",
"script": "watch",
"problemMatcher": "$ts-webpack-watch",
"isBackground": true,
"presentation": {
"reveal": "never",
"group": "watchers"
},
"group": {
"kind": "build",
"isDefault": true
}
},
{
"type": "npm",
"script": "watch-tests",
"problemMatcher": "$tsc-watch",
"isBackground": true,
"presentation": {
"reveal": "never",
"group": "watchers"
},
"group": "build"
},
{
"label": "tasks: watch-tests",
"dependsOn": ["npm: watch", "npm: watch-tests"],
"problemMatcher": []
}
]
}

View File

@@ -0,0 +1,42 @@
cmake_minimum_required(VERSION 3.28)
project(clice_vscode_cmake_sample LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
add_library(sample_greeting greeting.cc)
target_include_directories(sample_greeting PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}")
add_executable(sample_app main.cc)
target_link_libraries(sample_app PRIVATE sample_greeting)
set(SAMPLE_MODULES_SUPPORTED OFF)
if(CMAKE_CXX_COMPILER_ID STREQUAL "Clang" AND
CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 16)
set(SAMPLE_MODULES_SUPPORTED ON)
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" AND
CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 14)
set(SAMPLE_MODULES_SUPPORTED ON)
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC" AND
CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 19.34)
set(SAMPLE_MODULES_SUPPORTED ON)
endif()
if(SAMPLE_MODULES_SUPPORTED)
add_executable(sample_module_app)
target_sources(sample_module_app PRIVATE main_module.cc)
target_sources(sample_module_app
PRIVATE
FILE_SET CXX_MODULES
FILES greeting_module.cppm
)
else()
message(STATUS
"Skipping sample_module_app because the active compiler lacks "
"CMake C++20 module scanning support. Use Clang >= 16, GCC >= 14, "
"or MSVC 19.34+ to enable it."
)
endif()

View File

@@ -0,0 +1,38 @@
# VS Code CMake Sample
This workspace is a standalone CMake project for attaching the VS Code extension to a real `clice` session.
`clice` already auto-detects `build/compile_commands.json`, so this sample does not need any helper scripts or extra CMake glue.
The workspace contains two entry points:
- `main.cc`: a traditional include-based example.
- `main_module.cc`: a C++20 modules example that imports `greeting_module.cppm`.
## Prepare The Workspace
From this directory:
```sh
cmake -S . -B build -G Ninja -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
```
That is enough to generate `build/compile_commands.json`, which `clice` can discover automatically when this folder is opened as the workspace.
If you also want the sample binary:
```sh
cmake --build build
```
## C++20 Modules
The module example is enabled automatically when CMake is using a compiler it can scan for C++20 modules:
- Clang 16+
- GCC 14+
- MSVC 19.34+
If the active compiler is older than that, CMake still configures the workspace and builds `sample_app`, but it skips `sample_module_app`.
When you do have a supported compiler, `main_module.cc` and `greeting_module.cppm` will also appear in `build/compile_commands.json`, which makes this workspace useful for testing clice's module handling in an editor.

View File

@@ -0,0 +1,7 @@
#include "greeting.h"
#include <string>
std::string build_greeting(std::string_view name) {
return "Hello, " + std::string(name) + " from the CMake sample.";
}

View File

@@ -0,0 +1,6 @@
#pragma once
#include <string>
#include <string_view>
std::string build_greeting(std::string_view name);

View File

@@ -0,0 +1,14 @@
module;
#include <string>
#include <string_view>
export module sample.greeting;
export namespace sample {
std::string build_module_greeting(std::string_view name) {
return "Hello, " + std::string(name) + " from the C++20 module sample.";
}
}

View File

@@ -0,0 +1,8 @@
#include "greeting.h"
#include <iostream>
int main() {
std::cout << build_greeting("clice") << '\n';
return 0;
}

View File

@@ -0,0 +1,8 @@
#include <iostream>
import sample.greeting;
int main() {
std::cout << sample::build_module_greeting("clice") << '\n';
return 0;
}

View File

@@ -0,0 +1,6 @@
#include <cstdio>
int main() {
printf("Hello, World!\n");
return 0;
}

5702
pixi.lock generated

File diff suppressed because it is too large Load Diff

135
pixi.toml
View File

@@ -14,24 +14,17 @@ readme = "README.md"
documentation = "https://docs.clice.io/clice/"
repository = "https://github.com/clice-io/clice"
channels = ["conda-forge"]
platforms = ["win-64", "linux-64", "osx-arm64", "osx-64", "linux-aarch64", "win-arm64"]
platforms = ["win-64", "linux-64", "osx-arm64"]
[environments]
default = ["build", "test"]
package = ["build", "test", "package"]
cross-macos-x64 = ["build", "package", "cross-macos-x64"]
cross-linux-aarch64 = ["build", "package", "cross-linux-aarch64"]
cross-windows-arm64 = ["build", "package", "cross-windows-arm64"]
node = ["node"]
format = ["format"]
test-run = ["test"]
# ============================================================================== #
# DEPENDENCIES #
# ============================================================================== #
[feature.build]
platforms = ["win-64", "linux-64", "osx-arm64", "osx-64", "linux-aarch64"]
[feature.build.dependencies]
python = ">=3.13"
cmake = ">=3.30"
@@ -40,9 +33,7 @@ clang = "==20.1.8"
clangxx = "==20.1.8"
lld = "==20.1.8"
llvm-tools = "==20.1.8"
clang-tools = "==20.1.8"
compiler-rt = "==20.1.8"
flatbuffers = "==25.9.23"
[feature.build.target.win-64.dependencies]
sccache = "*"
@@ -62,43 +53,6 @@ scripts = ["scripts/activate_linux.sh"]
[feature.build.target.win-64.activation]
scripts = ["scripts/activate_asan.bat"]
# macOS x64 (from arm64): clang natively supports cross-arch, no extra deps.
[feature.cross-macos-x64.target.osx-arm64.dependencies]
[feature.cross-macos-x64.target.osx-arm64.activation]
scripts = ["scripts/activate_cross_macos.sh"]
# Linux aarch64 (from x64): needs aarch64 sysroot and cross gcc for libstdc++.
[feature.cross-linux-aarch64.target.linux-64.dependencies]
sysroot_linux-aarch64 = "==2.17"
gcc_linux-aarch64 = "==14.2.0"
gxx_linux-aarch64 = "==14.2.0"
[feature.cross-linux-aarch64.target.linux-64.activation]
scripts = ["scripts/activate_cross_linux.sh"]
# Windows arm64 (from x64): Windows SDK on CI already includes ARM64 libs.
[feature.cross-windows-arm64.target.win-64.dependencies]
[feature.cross-windows-arm64.target.win-64.activation]
scripts = ["scripts/activate_cross_windows.bat"]
[feature.test.dependencies]
python = ">=3.13"
# On macOS, the system Apple clang emits vendor-specific flags that upstream
# LLVM cannot parse. Providing upstream clang + lld in PATH prevents
# fallback to /usr/bin/clang++ and satisfies toolchain.cmake's -fuse-ld=lld.
[feature.test.target.osx-64.dependencies]
clang = "==20.1.8"
clangxx = "==20.1.8"
lld = "==20.1.8"
[feature.test.target.osx-arm64.dependencies]
clang = "==20.1.8"
clangxx = "==20.1.8"
lld = "==20.1.8"
[feature.test.pypi-dependencies]
pytest = "*"
pytest-asyncio = ">=1.1.0"
@@ -108,22 +62,6 @@ lsprotocol = ">=2024.0.0"
[feature.package.dependencies]
xz = ">=5.8.1,<6"
[feature.package.tasks.package-config]
args = [{ arg = "type", default = "RelWithDebInfo" }]
cmd = """
cmake -B build/{{ type }} -G Ninja \
-DCMAKE_BUILD_TYPE={{ type }} \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchain.cmake \
-DCLICE_RELEASE=ON
"""
[feature.package.tasks.package]
args = [{ arg = "type", default = "RelWithDebInfo" }]
depends-on = [
{ task = "package-config", args = ["{{ type }}"] },
{ task = "cmake-build", args = ["{{ type }}"] },
]
# ============================================================================== #
# CMAKE #
# ============================================================================== #
@@ -131,13 +69,14 @@ depends-on = [
args = [
{ arg = "type", default = "RelWithDebInfo" },
{ arg = "ci", default = "OFF" },
{ arg = "extra", default = "" },
]
cmd = """
cmake -B build/{{ type }} -G Ninja \
-DCMAKE_BUILD_TYPE={{ type }} \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchain.cmake \
-DCLICE_ENABLE_TEST=ON \
-DCLICE_CI_ENVIRONMENT={{ ci }}
-DCLICE_CI_ENVIRONMENT={{ ci }} {{extra}} \
"""
[feature.build.tasks.cmake-build]
@@ -148,17 +87,14 @@ cmd = "cmake --build build/{{ type }}"
args = [
{ arg = "type", default = "RelWithDebInfo" },
{ arg = "ci", default = "OFF" },
{ arg = "extra", default = "" },
]
depends-on = [
{ task = "cmake-config", args = ["{{ type }}", "{{ ci }}"] },
{ task = "cmake-config", args = ["{{ type }}", "{{ ci }}", "{{extra}}"] },
{ task = "cmake-build", args = ["{{ type }}"] },
]
[feature.build.tasks.clang-tidy]
args = [{ arg = "type", default = "RelWithDebInfo" }]
depends-on = [{ task = "lint-cpp", args = ["{{ type }}"] }]
[feature.test.tasks.unit-test]
[feature.build.tasks.unit-test]
args = [{ arg = "type", default = "RelWithDebInfo" }]
cmd = './build/{{ type }}/bin/unit_tests --test-dir="./tests/data"'
@@ -181,9 +117,41 @@ args = [{ arg = "type", default = "RelWithDebInfo" }]
depends-on = [
{ task = "unit-test", args = ["{{ type }}"] },
{ task = "integration-test", args = ["{{ type }}"] },
{ task = "smoke-test", args = ["{{ type }}"] },
]
# ============================================================================== #
# XMAKE #
# ============================================================================== #
[feature.build.tasks.xmake-config]
args = [
{ arg = "type", default = "releasedbg" },
{ arg = "ci", default = "n" },
]
cmd = "xmake config --yes --mode={{ type }} --toolchain=clang --ci={{ ci }}"
[feature.build.tasks.xmake-build]
cmd = "xmake build --verbose --diagnosis --all"
[feature.build.tasks.xmake]
args = [
{ arg = "type", default = "releasedbg" },
{ arg = "ci", default = "n" },
]
depends-on = [
{ task = "xmake-config", args = ["{{ type }}", "{{ ci }}"] },
{ task = "xmake-build" },
]
[feature.test.tasks.xmake-test]
cmd = "xmake test --verbose"
[feature.package.tasks.package]
cmd = """
xmake config --yes --toolchain=clang --mode=releasedbg \
--enable_test=n --dev=n --release=y && \
xmake pack --verbose
"""
# ============================================================================== #
# HELPER TASKS #
# ============================================================================== #
@@ -203,14 +171,9 @@ gh workflow run upload-llvm.yml \
args = ["file_name"]
cmd = ["scripts/delete-artifacts.bash", "{{ file_name }}"]
[dependencies]
# ============================================================================== #
# DOCS & VSCODE EXTENSION #
# ============================================================================== #
[feature.node]
platforms = ["win-64", "linux-64", "osx-arm64", "osx-64", "linux-aarch64"]
[feature.node.dependencies]
nodejs = ">=20"
pnpm = "*"
@@ -236,9 +199,6 @@ outputs = ["editors/vscode/node_modules/.modules.yaml"]
# ============================================================================== #
# FORMAT #
# ============================================================================== #
[feature.format]
platforms = ["win-64", "linux-64", "osx-arm64", "osx-64", "linux-aarch64"]
[feature.format.dependencies]
ruff = "*"
tombi = "*"
@@ -269,20 +229,3 @@ format = { depends-on = [
"format-toml",
"format-yaml"
] }
# ============================================================================== #
# LINT #
# ============================================================================== #
[feature.format.tasks.lint-python]
cmd = "ruff check ."
[feature.build.tasks.lint-cpp]
args = [{ arg = "type", default = "RelWithDebInfo" }]
cmd = "python scripts/run_clang_tidy.py build/{{ type }}"
[feature.build.tasks.lint]
args = [{ arg = "type", default = "RelWithDebInfo" }]
depends-on = [
"lint-python",
{ task = "lint-cpp", args = ["{{ type }}"] },
]

View File

@@ -1,12 +0,0 @@
#!/bin/sh
# Clear conda cross-gcc flags so host x86_64 paths don't leak into the
# aarch64 build. conda's gcc_linux-aarch64 activation sets
# CFLAGS/CXXFLAGS/CPPFLAGS/LDFLAGS with -isystem/-L pointing at $CONDA_PREFIX
# (x86_64 host paths). LIBRARY_PATH from ld_impl_linux-64 likewise points at
# host libs. Empty-string export reliably overrides conda-installed values
# regardless of whether pixi sources or calls this script.
export CFLAGS=
export CXXFLAGS=
export CPPFLAGS=
export LDFLAGS=
export LIBRARY_PATH=

View File

@@ -1,8 +0,0 @@
#!/bin/sh
# Clear conda host flags so arm64 host paths don't leak into the x86_64-macos
# cross build. See scripts/activate_cross_linux.sh for rationale.
export CFLAGS=
export CXXFLAGS=
export CPPFLAGS=
export LDFLAGS=
export LIBRARY_PATH=

View File

@@ -1,8 +0,0 @@
@echo off
REM Clear conda host flags so host x64 paths don't leak into the aarch64-windows
REM cross build. See scripts/activate_cross_linux.sh for rationale.
set "CFLAGS="
set "CXXFLAGS="
set "CPPFLAGS="
set "LDFLAGS="
set "LIBRARY_PATH="

View File

@@ -0,0 +1,489 @@
#!/usr/bin/env python3
"""Use Codex to analyze clangd semantic-highlighting logic from a GitHub blob URL.
Example:
python3 scripts/analyzers/extract_semantic_highlighting_codex.py \
https://github.com/llvm/llvm-project/blob/d8ba56ce3f98871ae4e5782c4af2df4c98bebde7/clang-tools-extra/clangd/SemanticHighlighting.cpp \
--output semantic-highlighting.md
"""
from __future__ import annotations
import argparse
import json
import os
import shutil
import subprocess
import sys
import tempfile
from dataclasses import dataclass
from pathlib import Path
from typing import Literal
from urllib.error import HTTPError, URLError
from urllib.parse import urlparse
from urllib.request import ProxyHandler, Request, build_opener, getproxies
PROMPT = """You are analyzing one C++ source file from clangd.
Task:
- Extract the cases in which semantic highlighting is applied.
- Produce an exhaustive segmentation that covers every source line exactly once.
Kinds:
- nop: this line does not materially participate in deciding or applying a highlight.
- condition: this line or contiguous range establishes a boolean/branching condition that gates a later highlight resolution.
- resolution: this line or contiguous range selects or applies a concrete semantic-highlighting outcome, such as a HighlightingKind, modifier, token emission, or an equivalent concrete result.
Output rules:
- Return JSON only.
- Segments must be in ascending order and non-overlapping.
- Every line from 1 through the last line must be covered exactly once.
- Use the smallest practical contiguous ranges.
- It is fine to compress long nop runs into ranges; the caller may expand them later.
- For nop segments, use an empty summary string.
- For condition segments, write one short sentence in plain English.
- For resolution segments, write one short sentence describing the concrete outcome.
- For resolution segments, populate depends_on with the exact condition line ranges that directly gate this outcome when present.
- Use the provided line numbers exactly; never invent lines.
- Do not use any external tools or local files; analyze only the numbered source text provided in the prompt.
"""
SCHEMA = {
"type": "object",
"properties": {
"segments": {
"type": "array",
"items": {
"type": "object",
"properties": {
"start_line": {"type": "integer", "minimum": 1},
"end_line": {"type": "integer", "minimum": 1},
"kind": {
"type": "string",
"enum": ["nop", "condition", "resolution"],
},
"summary": {"type": "string"},
"depends_on": {
"type": "array",
"items": {
"type": "object",
"properties": {
"start_line": {"type": "integer", "minimum": 1},
"end_line": {"type": "integer", "minimum": 1},
},
"required": ["start_line", "end_line"],
"additionalProperties": False,
},
},
},
"required": [
"start_line",
"end_line",
"kind",
"summary",
"depends_on",
],
"additionalProperties": False,
},
}
},
"required": ["segments"],
"additionalProperties": False,
}
SegmentKind = Literal["nop", "condition", "resolution"]
PROXY_ENV_KEYS = ("http_proxy", "https_proxy", "all_proxy", "no_proxy")
@dataclass(frozen=True)
class GitHubBlobRef:
owner: str
repo: str
rev: str
path: str
blob_url: str
raw_url: str
@property
def title(self) -> str:
return Path(self.path).stem
@dataclass(frozen=True)
class LineRange:
start_line: int
end_line: int
@dataclass(frozen=True)
class Segment:
start_line: int
end_line: int
kind: SegmentKind
summary: str
depends_on: tuple[LineRange, ...]
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Ask Codex to analyze a GitHub-hosted semantic-highlighting file.",
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
)
parser.add_argument(
"url",
help="GitHub blob URL pinned to a specific revision.",
)
parser.add_argument(
"--model",
default=None,
help="Codex model to use via `codex exec`.",
)
parser.add_argument(
"--reasoning-effort",
default=None,
choices=["low", "medium", "high", "xhigh"],
help="Reasoning effort override passed to Codex CLI.",
)
parser.add_argument(
"--timeout",
type=int,
default=30,
help="HTTP timeout in seconds for fetching the source file.",
)
parser.add_argument(
"--codex-bin",
default="codex",
help="Codex CLI executable to invoke.",
)
parser.add_argument(
"--output",
type=Path,
help="Write the markdown output to this path instead of stdout.",
)
return parser.parse_args()
def parse_github_blob_url(url: str) -> GitHubBlobRef:
parsed = urlparse(url)
if parsed.scheme not in {"http", "https"} or parsed.netloc != "github.com":
raise ValueError("expected a GitHub https://github.com/.../blob/... URL")
parts = [part for part in parsed.path.split("/") if part]
if len(parts) < 5 or parts[2] != "blob":
raise ValueError("expected a GitHub blob URL with /owner/repo/blob/rev/path")
owner, repo = parts[0], parts[1]
rev = parts[3]
path = "/".join(parts[4:])
if not path:
raise ValueError("missing file path in GitHub blob URL")
raw_url = f"https://raw.githubusercontent.com/{owner}/{repo}/{rev}/{path}"
normalized_blob_url = f"https://github.com/{owner}/{repo}/blob/{rev}/{path}"
return GitHubBlobRef(
owner=owner,
repo=repo,
rev=rev,
path=path,
blob_url=normalized_blob_url,
raw_url=raw_url,
)
def fetch_text(url: str, timeout: int) -> str:
request = Request(url, headers={"User-Agent": "semantic-highlighting-codex/1.0"})
opener = build_opener(ProxyHandler(getproxies()))
try:
with opener.open(request, timeout=timeout) as response:
return response.read().decode("utf-8").replace("\r\n", "\n")
except HTTPError as exc:
raise RuntimeError(f"failed to fetch {url}: HTTP {exc.code}") from exc
except URLError as exc:
raise RuntimeError(f"failed to fetch {url}: {exc.reason}") from exc
def number_source(text: str) -> tuple[str, int]:
lines = text.splitlines()
width = len(str(max(len(lines), 1)))
numbered = "\n".join(
f"{idx:>{width}} | {line}" for idx, line in enumerate(lines, 1)
)
return numbered, len(lines)
def build_codex_prompt(
blob: GitHubBlobRef,
source_text: str,
line_count: int,
) -> str:
numbered_source, _ = number_source(source_text)
return (
f"{PROMPT}\n\n"
f"GitHub blob URL: {blob.blob_url}\n"
f"LLVM revision: {blob.rev}\n"
f"File path: {blob.path}\n"
f"File title: {blob.title}\n"
f"Total lines: {line_count}\n\n"
"Analyze only the following numbered source file:\n"
f"{numbered_source}\n"
)
def analyze_with_codex_cli(
codex_bin: str,
blob: GitHubBlobRef,
source_text: str,
line_count: int,
model: str | None,
reasoning_effort: str | None,
) -> list[Segment]:
if shutil.which(codex_bin) is None:
raise RuntimeError(f"Codex CLI executable `{codex_bin}` was not found in PATH.")
prompt = build_codex_prompt(
blob=blob, source_text=source_text, line_count=line_count
)
child_env = build_codex_env()
with tempfile.TemporaryDirectory(prefix="semantic-highlighting-codex-") as temp_dir:
temp_path = Path(temp_dir)
schema_path = temp_path / "schema.json"
output_path = temp_path / "last-message.json"
schema_path.write_text(json.dumps(SCHEMA, indent=2), encoding="utf-8")
command = [
codex_bin,
"exec",
"--skip-git-repo-check",
"--ephemeral",
"--color",
"never",
"--sandbox",
"read-only",
*([] if model is None else ["--model", model]),
*(
[]
if reasoning_effort is None
else ["--config", f"model_reasoning_effort={json.dumps(reasoning_effort)}"]
),
"--output-schema",
str(schema_path),
"--output-last-message",
str(output_path),
"-",
]
try:
completed = subprocess.run(
command,
input=prompt,
text=True,
capture_output=True,
check=True,
env=child_env,
)
except FileNotFoundError as exc:
raise RuntimeError(
f"failed to execute `{codex_bin}`: command not found"
) from exc
except subprocess.CalledProcessError as exc:
detail = exc.stderr.strip() or exc.stdout.strip() or str(exc)
raise RuntimeError(f"Codex CLI failed: {detail}") from exc
if not output_path.exists():
detail = completed.stderr.strip() or completed.stdout.strip()
raise RuntimeError(
"Codex CLI did not produce an output message."
+ (f" Details: {detail}" if detail else "")
)
output_text = output_path.read_text(encoding="utf-8").strip()
if not output_text:
raise RuntimeError("Codex CLI returned an empty final message.")
try:
payload = json.loads(output_text)
except json.JSONDecodeError as exc:
raise RuntimeError(f"Codex returned invalid JSON:\n{output_text}") from exc
return normalize_segments(payload.get("segments", []), line_count)
def build_codex_env() -> dict[str, str]:
env = os.environ.copy()
for key in PROXY_ENV_KEYS:
value = env.get(key)
upper_key = key.upper()
if value and upper_key not in env:
env[upper_key] = value
return env
def normalize_segments(raw_segments: list[dict], line_count: int) -> list[Segment]:
normalized: list[Segment] = []
for item in raw_segments:
kind = item["kind"]
if kind not in {"nop", "condition", "resolution"}:
raise ValueError(f"unknown segment kind: {kind}")
start_line = int(item["start_line"])
end_line = int(item["end_line"])
if start_line < 1 or end_line < start_line or end_line > line_count:
raise ValueError(
f"invalid segment range {start_line}-{end_line} for file with {line_count} lines"
)
depends_on = tuple(
LineRange(start_line=int(dep["start_line"]), end_line=int(dep["end_line"]))
for dep in item.get("depends_on", [])
)
normalized.append(
Segment(
start_line=start_line,
end_line=end_line,
kind=kind,
summary=item.get("summary", "").strip(),
depends_on=depends_on,
)
)
normalized.sort(key=lambda segment: (segment.start_line, segment.end_line))
stitched: list[Segment] = []
next_line = 1
for segment in normalized:
if segment.start_line < next_line:
raise ValueError(
f"overlapping segments around line {segment.start_line}: model output is invalid"
)
while next_line < segment.start_line:
stitched.append(
Segment(
start_line=next_line,
end_line=next_line,
kind="nop",
summary="",
depends_on=(),
)
)
next_line += 1
stitched.extend(expand_nop_segment(segment))
next_line = segment.end_line + 1
while next_line <= line_count:
stitched.append(
Segment(
start_line=next_line,
end_line=next_line,
kind="nop",
summary="",
depends_on=(),
)
)
next_line += 1
return stitched
def expand_nop_segment(segment: Segment) -> list[Segment]:
if segment.kind != "nop" or segment.start_line == segment.end_line:
return [segment]
return [
Segment(
start_line=line_no,
end_line=line_no,
kind="nop",
summary="",
depends_on=(),
)
for line_no in range(segment.start_line, segment.end_line + 1)
]
def blob_anchor_url(blob: GitHubBlobRef, line_range: LineRange) -> str:
if line_range.start_line == line_range.end_line:
return f"{blob.blob_url}#L{line_range.start_line}"
return f"{blob.blob_url}#L{line_range.start_line}-L{line_range.end_line}"
def format_line_range(start_line: int, end_line: int) -> str:
if start_line == end_line:
return f"Line {start_line}"
return f"Line {start_line}-{end_line}"
def format_loc_reference(blob: GitHubBlobRef, line_range: LineRange) -> str:
label = (
f"LoC {line_range.start_line}"
if line_range.start_line == line_range.end_line
else f"LoC {line_range.start_line}-{line_range.end_line}"
)
return f"[{label}]({blob_anchor_url(blob, line_range)})"
def render_segment_summary(blob: GitHubBlobRef, segment: Segment) -> str:
summary = segment.summary.strip()
if segment.kind == "nop" or not summary:
return ""
if segment.kind == "resolution" and segment.depends_on:
refs = [format_loc_reference(blob, dep) for dep in segment.depends_on]
if len(refs) == 1:
prefix = f"By condition at {refs[0]}, "
else:
prefix = f"By conditions at {', '.join(refs)}, "
return prefix + decapitalize_summary(summary)
return summary
def decapitalize_summary(summary: str) -> str:
if not summary:
return summary
if len(summary) >= 2 and summary[0].isupper() and summary[1].islower():
return summary[:1].lower() + summary[1:]
return summary
def render_markdown(blob: GitHubBlobRef, segments: list[Segment]) -> str:
lines = [f"- LLVMRevHash: `{blob.rev}`", f"# {blob.title}", ""]
for segment in segments:
lines.append(
f"## {format_line_range(segment.start_line, segment.end_line)} (kind: {segment.kind})"
)
summary = render_segment_summary(blob, segment)
if summary:
lines.append(summary)
lines.append("")
return "\n".join(lines).rstrip() + "\n"
def main() -> int:
args = parse_args()
try:
blob = parse_github_blob_url(args.url)
source_text = fetch_text(blob.raw_url, timeout=args.timeout)
_, line_count = number_source(source_text)
segments = analyze_with_codex_cli(
codex_bin=args.codex_bin,
blob=blob,
source_text=source_text,
line_count=line_count,
model=args.model,
reasoning_effort=args.reasoning_effort,
)
markdown = render_markdown(blob, segments)
except Exception as exc:
print(f"Error: {exc}", file=sys.stderr)
return 1
if args.output:
args.output.write_text(markdown, encoding="utf-8")
else:
sys.stdout.write(markdown)
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -4,7 +4,6 @@ import subprocess
import shutil
import argparse
import os
import json
from pathlib import Path
@@ -23,66 +22,6 @@ def normalize_mode(value: str) -> str:
)
def build_native_tools(project_root: Path, build_dir: Path) -> Path:
"""Build native host tablegen tools for cross-compilation.
When cross-compiling LLVM, build tools like llvm-tblgen must run on the
host but would otherwise be compiled for the target architecture. This
function performs a minimal native build and returns the bin directory
containing host-runnable executables.
"""
native_dir = build_dir.parent / f"{build_dir.name}-native-tools"
native_dir.mkdir(exist_ok=True)
source_dir = project_root / "llvm"
cmake_args = [
"-G",
"Ninja",
"-DCMAKE_BUILD_TYPE=Release",
"-DLLVM_ENABLE_PROJECTS=clang;clang-tools-extra",
"-DLLVM_TARGETS_TO_BUILD=Native",
"-DLLVM_DISABLE_ASSEMBLY_FILES=ON",
"-DCMAKE_C_FLAGS=-w",
"-DCMAKE_CXX_FLAGS=-w",
]
if sys.platform == "win32":
cmake_args += [
"-DCMAKE_C_COMPILER=clang-cl",
"-DCMAKE_CXX_COMPILER=clang-cl",
]
else:
cmake_args += [
"-DCMAKE_C_COMPILER=clang",
"-DCMAKE_CXX_COMPILER=clang++",
]
print(f"\nConfiguring native host tools in {native_dir}...")
subprocess.check_call(
["cmake", "-S", str(source_dir), "-B", str(native_dir)] + cmake_args
)
required_tools = ["llvm-tblgen", "llvm-min-tblgen", "clang-tblgen"]
optional_tools = ["clang-tidy-confusable-chars-gen"]
for tool in required_tools:
print(f"Building native {tool}...")
subprocess.check_call(["cmake", "--build", str(native_dir), "--target", tool])
for tool in optional_tools:
try:
print(f"Building native {tool} (optional)...")
subprocess.check_call(
["cmake", "--build", str(native_dir), "--target", tool]
)
except subprocess.CalledProcessError:
print(f" {tool} not available, skipping.")
bin_dir = native_dir / "bin"
print(f"Native host tools ready in {bin_dir}")
return bin_dir
def main():
parser = argparse.ArgumentParser(
description="Build LLVM with specific configurations."
@@ -109,10 +48,6 @@ def main():
"--build-dir",
help="Custom build directory (relative to project root or absolute)",
)
parser.add_argument(
"--target-triple",
help="Cross-compilation target triple (e.g. x86_64-apple-darwin, aarch64-linux-gnu, aarch64-pc-windows-msvc)",
)
args = parser.parse_args()
@@ -150,46 +85,118 @@ def main():
print("--- Configuration ---")
print(f"Mode: {args.mode}")
print(f"LTO: {args.lto}")
print(f"Target Triple: {args.target_triple or '(native)'}")
print(f"Root: {project_root}")
print(f"Build Dir: {build_dir}")
print(f"Install Prefix: {install_prefix}")
print(f"Toolchain: {toolchain_file}")
print("---------------------")
components_path = Path(__file__).resolve().parent / "llvm-components.json"
with components_path.open() as f:
llvm_distribution_components = json.load(f)["components"]
llvm_distribution_components = [
"LLVMDemangle",
"LLVMSupport",
"LLVMCore",
"LLVMOption",
"LLVMBinaryFormat",
"LLVMMC",
"LLVMMCParser",
"LLVMObject",
"LLVMProfileData",
"LLVMBitReader",
"LLVMBitstreamReader",
"LLVMRemarks",
"LLVMObjectYAML",
"LLVMAggressiveInstCombine",
"LLVMInstCombine",
"LLVMIRReader",
"LLVMTextAPI",
"LLVMSymbolize",
"LLVMDebugInfoDWARF",
"LLVMDebugInfoDWARFLowLevel",
"LLVMDebugInfoCodeView",
"LLVMDebugInfoGSYM",
"LLVMDebugInfoPDB",
"LLVMDebugInfoBTF",
"LLVMDebugInfoMSF",
"LLVMAsmParser",
"LLVMTargetParser",
"LLVMTransformUtils",
"LLVMAnalysis",
"LLVMScalarOpts",
"LLVMFrontendHLSL",
"LLVMFrontendOpenMP",
"LLVMFrontendOffloading",
"LLVMFrontendAtomic",
"LLVMFrontendDirective",
"LLVMWindowsDriver",
"clangIndex",
"clangAPINotes",
"clangAST",
"clangASTMatchers",
"clangBasic",
"clangDriver",
"clangFormat",
"clangFrontend",
"clangLex",
"clangParse",
"clangSema",
"clangSerialization",
"clangRewrite",
"clangAnalysis",
"clangEdit",
"clangSupport",
"clangStaticAnalyzerCore",
"clangStaticAnalyzerFrontend",
"clangTidy",
"clangTidyUtils",
"clangTidyAndroidModule",
"clangTidyAbseilModule",
"clangTidyAlteraModule",
"clangTidyBoostModule",
"clangTidyBugproneModule",
"clangTidyCERTModule",
"clangTidyConcurrencyModule",
"clangTidyCppCoreGuidelinesModule",
"clangTidyDarwinModule",
"clangTidyFuchsiaModule",
"clangTidyGoogleModule",
"clangTidyHICPPModule",
"clangTidyLinuxKernelModule",
"clangTidyLLVMModule",
"clangTidyLLVMLibcModule",
"clangTidyMiscModule",
"clangTidyModernizeModule",
"clangTidyObjCModule",
"clangTidyOpenMPModule",
"clangTidyPerformanceModule",
"clangTidyPortabilityModule",
"clangTidyReadabilityModule",
"clangTidyZirconModule",
"clangTooling",
"clangToolingCore",
"clangToolingInclusions",
"clangToolingInclusionsStdlib",
"clangToolingSyntax",
"clangToolingRefactoring",
"clangTransformer",
"clangCrossTU",
"clangAnalysisFlowSensitive",
"clangAnalysisFlowSensitiveModels",
"clangStaticAnalyzerCheckers",
"clangIncludeCleaner",
"llvm-headers",
"clang-headers",
"clang-tidy-headers",
"clang-resource-headers",
]
components_joined = ";".join(llvm_distribution_components)
cmake_args = [
"-G",
"Ninja",
f"-DCMAKE_TOOLCHAIN_FILE={toolchain_file.as_posix()}",
f"-DCMAKE_INSTALL_PREFIX={install_prefix}",
]
if sys.platform == "win32":
# Use clang-cl (MSVC driver) on Windows so that LLVM's CMake
# generates correct MSVC-style linker flags for LTO, etc.
c_flags = "-w"
if args.target_triple:
c_flags += f" --target={args.target_triple}"
cmake_args += [
"-DCMAKE_C_COMPILER=clang-cl",
"-DCMAKE_CXX_COMPILER=clang-cl",
f"-DCMAKE_C_FLAGS={c_flags}",
f"-DCMAKE_CXX_FLAGS={c_flags}",
"-DLLVM_USE_LINKER=lld-link",
]
else:
cmake_args += [
f"-DCMAKE_TOOLCHAIN_FILE={toolchain_file.as_posix()}",
"-DCMAKE_C_FLAGS=-w",
"-DCMAKE_CXX_FLAGS=-w",
"-DLLVM_USE_LINKER=lld",
]
cmake_args += [
"-DCMAKE_C_FLAGS=-w",
"-DCMAKE_CXX_FLAGS=-w",
"-DLLVM_ENABLE_ZLIB=OFF",
"-DLLVM_ENABLE_ZSTD=OFF",
"-DLLVM_ENABLE_LIBXML2=OFF",
@@ -224,6 +231,7 @@ def main():
"-DCMAKE_JOB_POOL_LINK=console",
"-DLLVM_ENABLE_PROJECTS=clang;clang-tools-extra",
"-DLLVM_TARGETS_TO_BUILD=all",
"-DLLVM_USE_LINKER=lld",
"-DLLVM_DISABLE_ASSEMBLY_FILES=ON",
# Distribution
f"-DLLVM_DISTRIBUTION_COMPONENTS={components_joined}",
@@ -248,10 +256,8 @@ def main():
is_shared = "OFF"
if args.mode == "Debug":
cmake_args.append("-DCMAKE_BUILD_TYPE=Debug")
# ASAN is incompatible with -MDd on Windows (clang-cl), skip it there.
if sys.platform != "win32":
cmake_args.append("-DLLVM_USE_SANITIZER=Address")
is_shared = "ON"
cmake_args.append("-DLLVM_USE_SANITIZER=Address")
is_shared = "ON"
elif args.mode == "Release":
cmake_args.append("-DCMAKE_BUILD_TYPE=Release")
elif args.mode == "RelWithDebInfo":
@@ -266,24 +272,6 @@ def main():
else:
cmake_args.append("-DLLVM_ENABLE_LTO=OFF")
if args.target_triple:
cmake_args.append(f"-DCLICE_TARGET_TRIPLE={args.target_triple}")
cmake_args.append(f"-DLLVM_HOST_TRIPLE={args.target_triple}")
# When cross-compiling, clear conda's host-platform flags so they
# don't leak into the target build (e.g. -L pointing to x86_64 libs).
# This must happen before the native-tools build too so we don't
# contaminate the native configure with target-arch link flags.
for var in ["LIBRARY_PATH", "LDFLAGS", "CFLAGS", "CXXFLAGS", "CPPFLAGS"]:
os.environ.pop(var, None)
# Cross-compilation needs native host tools (tablegen, etc.) that can
# run on the build machine. macOS handles this transparently via
# Rosetta 2, but Linux and Windows require a separate native build.
if sys.platform != "darwin":
native_bin_dir = build_native_tools(project_root, build_dir)
cmake_args.append(f"-DLLVM_NATIVE_TOOL_DIR={native_bin_dir}")
build_dir.mkdir(exist_ok=True)
print(f"\nConfiguring in {build_dir}...")

View File

@@ -1,99 +0,0 @@
{
"components": [
"LLVMDemangle",
"LLVMSupport",
"LLVMCore",
"LLVMOption",
"LLVMBinaryFormat",
"LLVMMC",
"LLVMMCParser",
"LLVMObject",
"LLVMProfileData",
"LLVMBitReader",
"LLVMBitstreamReader",
"LLVMRemarks",
"LLVMObjectYAML",
"LLVMAggressiveInstCombine",
"LLVMInstCombine",
"LLVMIRReader",
"LLVMTextAPI",
"LLVMSymbolize",
"LLVMDebugInfoDWARF",
"LLVMDebugInfoDWARFLowLevel",
"LLVMDebugInfoCodeView",
"LLVMDebugInfoGSYM",
"LLVMDebugInfoPDB",
"LLVMDebugInfoBTF",
"LLVMDebugInfoMSF",
"LLVMAsmParser",
"LLVMTargetParser",
"LLVMTransformUtils",
"LLVMAnalysis",
"LLVMScalarOpts",
"LLVMFrontendHLSL",
"LLVMFrontendOpenMP",
"LLVMFrontendOffloading",
"LLVMFrontendAtomic",
"LLVMFrontendDirective",
"LLVMWindowsDriver",
"clangIndex",
"clangAPINotes",
"clangAST",
"clangASTMatchers",
"clangBasic",
"clangDriver",
"clangFormat",
"clangFrontend",
"clangLex",
"clangParse",
"clangSema",
"clangSerialization",
"clangRewrite",
"clangAnalysis",
"clangEdit",
"clangSupport",
"clangStaticAnalyzerCore",
"clangStaticAnalyzerFrontend",
"clangTidy",
"clangTidyUtils",
"clangTidyAndroidModule",
"clangTidyAbseilModule",
"clangTidyAlteraModule",
"clangTidyBoostModule",
"clangTidyBugproneModule",
"clangTidyCERTModule",
"clangTidyConcurrencyModule",
"clangTidyCppCoreGuidelinesModule",
"clangTidyDarwinModule",
"clangTidyFuchsiaModule",
"clangTidyGoogleModule",
"clangTidyHICPPModule",
"clangTidyLinuxKernelModule",
"clangTidyLLVMModule",
"clangTidyLLVMLibcModule",
"clangTidyMiscModule",
"clangTidyModernizeModule",
"clangTidyObjCModule",
"clangTidyOpenMPModule",
"clangTidyPerformanceModule",
"clangTidyPortabilityModule",
"clangTidyReadabilityModule",
"clangTidyZirconModule",
"clangTooling",
"clangToolingCore",
"clangToolingInclusions",
"clangToolingInclusionsStdlib",
"clangToolingSyntax",
"clangToolingRefactoring",
"clangTransformer",
"clangCrossTU",
"clangAnalysisFlowSensitive",
"clangAnalysisFlowSensitiveModels",
"clangStaticAnalyzerCheckers",
"clangIncludeCleaner",
"llvm-headers",
"clang-headers",
"clang-tidy-headers",
"clang-resource-headers"
]
}

View File

@@ -1,66 +0,0 @@
#!/usr/bin/env python3
"""Run clang-tidy in parallel on all files in compile_commands.json."""
import json
import subprocess
import sys
import threading
from concurrent.futures import ThreadPoolExecutor, as_completed
from pathlib import Path
def main():
build_dir = sys.argv[1] if len(sys.argv) > 1 else "build/RelWithDebInfo"
cdb_path = Path(build_dir) / "compile_commands.json"
if not cdb_path.exists():
print(f"Error: {cdb_path} not found. Run cmake-config first.", file=sys.stderr)
sys.exit(1)
project_root = Path(__file__).resolve().parent.parent
src_dirs = (project_root / "src", project_root / "tests")
cdb = json.loads(cdb_path.read_text())
files = [
entry["file"]
for entry in cdb
if any(Path(entry["file"]).resolve().is_relative_to(d) for d in src_dirs)
]
total = len(files)
lock = threading.Lock()
done = 0
failed = []
def run(file: str) -> tuple[str, int, str]:
result = subprocess.run(
["clang-tidy", "-p", build_dir, "--quiet", file],
capture_output=True,
text=True,
)
return file, result.returncode, result.stdout + result.stderr
with ThreadPoolExecutor() as pool:
futures = {pool.submit(run, f): f for f in files}
for future in as_completed(futures):
file, code, output = future.result()
with lock:
done += 1
name = Path(file).name
if code != 0:
failed.append(file)
print(f"[{done}/{total}] FAIL {name}")
if output.strip():
print(output, end="")
else:
print(f"[{done}/{total}] OK {name}")
if failed:
print(f"\nclang-tidy failed on {len(failed)}/{total} files.", file=sys.stderr)
sys.exit(1)
print(f"\nclang-tidy passed on {total} files.")
if __name__ == "__main__":
main()

View File

@@ -40,52 +40,23 @@ def detect_platform() -> str:
raise RuntimeError(f"Unsupported platform: {plat}")
def detect_arch() -> str:
import platform
machine = platform.machine().lower()
if machine in ("x86_64", "amd64"):
return "x64"
if machine in ("aarch64", "arm64"):
return "arm64"
raise RuntimeError(f"Unsupported architecture: {machine}")
def pick_artifact(
manifest: list[dict],
version: str,
build_type: str,
is_lto: bool,
platform: str,
arch: str,
manifest: list[dict], version: str, build_type: str, is_lto: bool, platform: str
) -> dict:
base_version = version.split("+", 1)[0]
saw_missing_arch = False
for entry in manifest:
if entry.get("version") != version:
continue
if entry.get("platform") != platform.lower():
continue
entry_arch = entry.get("arch")
if entry_arch is None:
saw_missing_arch = True
continue
if entry_arch != arch:
continue
if entry.get("build_type") != build_type:
continue
if bool(entry.get("lto")) != is_lto:
continue
return entry
if saw_missing_arch:
raise RuntimeError(
f"Manifest contains entries without an 'arch' field for version={base_version}, "
f"platform={platform}. The manifest format changed to require explicit "
f"architectures; regenerate it via scripts/update-llvm-version.py."
)
raise RuntimeError(
f"No matching LLVM artifact in manifest for version={base_version}, platform={platform}, "
f"arch={arch}, build_type={build_type}, lto={is_lto}"
f"build_type={build_type}, lto={is_lto}"
)
@@ -293,14 +264,6 @@ def main() -> None:
parser.add_argument("--install-path")
parser.add_argument("--enable-lto", action="store_true")
parser.add_argument("--offline", action="store_true")
parser.add_argument(
"--target-platform",
help="Override platform for cross-compilation (e.g. macosx, linux, windows)",
)
parser.add_argument(
"--target-arch",
help="Override architecture for cross-compilation (e.g. x64, arm64)",
)
parser.add_argument("--output", required=True)
args = parser.parse_args()
@@ -312,11 +275,8 @@ def main() -> None:
)
token = os.environ.get("GH_TOKEN") or os.environ.get("GITHUB_TOKEN")
build_type = args.build_type
platform_name = args.target_platform if args.target_platform else detect_platform()
arch_name = args.target_arch if args.target_arch else detect_arch()
log(
f"Platform: {platform_name}, arch: {arch_name}, normalized build type: {build_type}"
)
platform_name = detect_platform()
log(f"Platform detected: {platform_name}, normalized build type: {build_type}")
manifest = read_manifest(Path(args.manifest))
binary_dir = Path(args.binary_dir).resolve()
@@ -344,12 +304,7 @@ def main() -> None:
if install_path is None:
needs_install = True
artifact = pick_artifact(
manifest,
args.version,
build_type,
args.enable_lto,
platform_name,
arch_name,
manifest, args.version, build_type, args.enable_lto, platform_name
)
log(f"Selected artifact: {artifact.get('filename')} for download")
filename = artifact["filename"]
@@ -362,12 +317,7 @@ def main() -> None:
install_path = install_root
elif needs_install:
artifact = pick_artifact(
manifest,
args.version,
build_type,
args.enable_lto,
platform_name,
arch_name,
manifest, args.version, build_type, args.enable_lto, platform_name
)
log(f"Selected artifact: {artifact.get('filename')} for download")
filename = artifact["filename"]

View File

@@ -1,162 +0,0 @@
#!/usr/bin/env python3
import argparse
import json
import re
import sys
from pathlib import Path
def copy_manifest(src: Path, dest: Path) -> None:
text = src.read_text(encoding="utf-8")
try:
data = json.loads(text)
except json.JSONDecodeError as err:
print(f"Error: {src} is not valid JSON: {err}", file=sys.stderr)
sys.exit(1)
if not isinstance(data, list) or len(data) == 0:
print(f"Error: {src} must be a non-empty JSON array", file=sys.stderr)
sys.exit(1)
dest.parent.mkdir(parents=True, exist_ok=True)
with dest.open("w", encoding="utf-8") as handle:
json.dump(data, handle, indent=2)
handle.write("\n")
print(f"Copied manifest: {src} -> {dest} ({len(data)} entries)")
def update_package_cmake(path: Path, version: str) -> None:
text = path.read_text(encoding="utf-8")
pattern = r'setup_llvm\("[^"]*"\)'
matches = re.findall(pattern, text)
if len(matches) == 0:
print(f"Error: no setup_llvm(...) call found in {path}", file=sys.stderr)
sys.exit(1)
if len(matches) > 1:
print(
f"Error: expected exactly 1 setup_llvm(...) call in {path}, "
f"found {len(matches)}",
file=sys.stderr,
)
sys.exit(1)
old_call = matches[0]
new_call = f'setup_llvm("{version}")'
if old_call == new_call:
print(f"Version in {path} is already {version}, no change needed")
return
updated = text.replace(old_call, new_call)
path.write_text(updated, encoding="utf-8")
print(f"Updated {path}: {old_call} -> {new_call}")
def check_package_cmake(path: Path) -> None:
"""Verify package.cmake has exactly one setup_llvm(...) call that the
update script can rewrite. Used by CI to catch drift before the next bump."""
text = path.read_text(encoding="utf-8")
matches = re.findall(r'setup_llvm\("[^"]*"\)', text)
if len(matches) == 0:
print(f"Error: no setup_llvm(...) call found in {path}", file=sys.stderr)
sys.exit(1)
if len(matches) > 1:
print(
f"Error: expected exactly 1 setup_llvm(...) call in {path}, "
f"found {len(matches)}: {matches}",
file=sys.stderr,
)
sys.exit(1)
print(f"OK: {path} has a single setup_llvm(...) call: {matches[0]}")
def check_manifest(path: Path) -> None:
"""Verify the manifest is a well-formed non-empty array with required fields."""
try:
data = json.loads(path.read_text(encoding="utf-8"))
except json.JSONDecodeError as err:
print(f"Error: {path} is not valid JSON: {err}", file=sys.stderr)
sys.exit(1)
if not isinstance(data, list) or len(data) == 0:
print(f"Error: {path} must be a non-empty JSON array", file=sys.stderr)
sys.exit(1)
required = ("version", "platform", "arch", "build_type", "filename", "sha256")
for idx, entry in enumerate(data):
missing = [k for k in required if k not in entry]
if missing:
print(
f"Error: {path} entry {idx} is missing fields: {missing}",
file=sys.stderr,
)
sys.exit(1)
print(f"OK: {path} has {len(data)} well-formed entries")
def main() -> None:
parser = argparse.ArgumentParser(
description="Update LLVM version references in the clice project."
)
parser.add_argument(
"--check",
action="store_true",
help="Validate existing state without modifying files (for CI drift checks)",
)
parser.add_argument(
"--version",
help="New LLVM version string (e.g. 21.2.0); required unless --check",
)
parser.add_argument(
"--manifest-src",
help="Path to the source llvm-manifest.json; required unless --check",
)
parser.add_argument(
"--manifest-dest",
required=True,
help="Path to destination manifest (e.g. config/llvm-manifest.json)",
)
parser.add_argument(
"--package-cmake",
required=True,
help="Path to cmake/package.cmake",
)
args = parser.parse_args()
manifest_dest = Path(args.manifest_dest)
package_cmake = Path(args.package_cmake)
if not package_cmake.is_file():
print(f"Error: package.cmake not found: {package_cmake}", file=sys.stderr)
sys.exit(1)
if args.check:
check_package_cmake(package_cmake)
check_manifest(manifest_dest)
print("Done (check mode).")
return
if not args.version or not args.manifest_src:
print(
"Error: --version and --manifest-src are required unless --check is set",
file=sys.stderr,
)
sys.exit(1)
manifest_src = Path(args.manifest_src)
if not manifest_src.is_file():
print(f"Error: manifest source not found: {manifest_src}", file=sys.stderr)
sys.exit(1)
copy_manifest(manifest_src, manifest_dest)
update_package_cmake(package_cmake, args.version)
print("Done.")
if __name__ == "__main__":
main()

View File

@@ -27,15 +27,6 @@ def parse_platform(name: str) -> str:
raise ValueError(f"Unable to determine platform from filename: {name}")
def parse_arch(name: str) -> str:
lowered = name.lower()
if lowered.startswith("aarch64-") or lowered.startswith("arm64-"):
return "arm64"
if lowered.startswith("x64-") or lowered.startswith("x86_64-"):
return "x64"
raise ValueError(f"Unable to determine arch from filename: {name}")
def parse_build_type(name: str) -> str:
lowered = name.lower()
if "debug" in lowered:
@@ -52,7 +43,6 @@ def build_metadata_entry(path: Path, version: str) -> dict:
"lto": "-lto" in filename.lower(),
"asan": "-asan" in filename.lower(),
"platform": parse_platform(filename),
"arch": parse_arch(filename),
"build_type": parse_build_type(filename),
}

View File

@@ -1,163 +0,0 @@
#!/usr/bin/env python3
"""
Validate the LLVM distribution component list against the actual LLVM source tree.
Scans the LLVM source for CMake library targets and compares them against
a components JSON file to detect stale or misspelled entries.
"""
import argparse
import difflib
import json
import re
import sys
from pathlib import Path
# CMake function calls that define library targets.
# The captured group uses [^\s)]+ to grab the target name without
# trailing parentheses or whitespace.
LLVM_LIB_PATTERNS = [
re.compile(r"add_llvm_component_library\(\s*([^\s)]+)"),
re.compile(r"add_llvm_library\(\s*([^\s)]+)"),
]
CLANG_LIB_PATTERNS = [
re.compile(r"add_clang_library\(\s*([^\s)]+)"),
]
# Header-only / custom install targets.
HEADER_PATTERNS = [
re.compile(r"add_llvm_install_targets\(\s*([^\s)]+)"),
re.compile(r"add_custom_target\(\s*([^\s)]+)"),
re.compile(r"add_library\(\s*([^\s)]+)"),
]
# Targets we recognise as header-only distribution components.
KNOWN_HEADER_TARGETS = {
"llvm-headers",
"clang-headers",
"clang-tidy-headers",
"clang-resource-headers",
}
def scan_targets(directory: Path, patterns: list[re.Pattern]) -> set[str]:
"""Recursively scan *directory* for CMakeLists.txt files and extract target names."""
targets: set[str] = set()
if not directory.is_dir():
return targets
for cmake_file in directory.rglob("CMakeLists.txt"):
text = cmake_file.read_text(errors="replace")
for pattern in patterns:
for match in pattern.finditer(text):
targets.add(match.group(1))
return targets
def scan_header_targets(llvm_src: Path) -> set[str]:
"""Scan for well-known header / custom-install targets across the tree."""
found: set[str] = set()
for cmake_file in llvm_src.rglob("CMakeLists.txt"):
text = cmake_file.read_text(errors="replace")
for pattern in HEADER_PATTERNS:
for match in pattern.finditer(text):
name = match.group(1)
if name in KNOWN_HEADER_TARGETS:
found.add(name)
return found
def collect_source_targets(llvm_src: Path) -> set[str]:
"""Return the full set of library / header targets found in the LLVM source tree."""
targets: set[str] = set()
targets |= scan_targets(llvm_src / "llvm" / "lib", LLVM_LIB_PATTERNS)
targets |= scan_targets(llvm_src / "clang" / "lib", CLANG_LIB_PATTERNS)
targets |= scan_targets(llvm_src / "clang-tools-extra", CLANG_LIB_PATTERNS)
targets |= scan_header_targets(llvm_src)
return targets
def load_components(path: Path) -> list[str]:
with path.open("r", encoding="utf-8") as handle:
data = json.load(handle)
if isinstance(data, dict):
data = data.get("components", [])
if not isinstance(data, list) or not data:
print(f"Error: no component list found in {path}", file=sys.stderr)
sys.exit(1)
return data
def main() -> None:
parser = argparse.ArgumentParser(
description="Validate LLVM distribution components against the source tree."
)
parser.add_argument(
"--llvm-src",
required=True,
help="Path to the llvm-project source root",
)
parser.add_argument(
"--components-file",
required=True,
help="Path to llvm-components.json",
)
args = parser.parse_args()
llvm_src = Path(args.llvm_src).expanduser().resolve()
components_file = Path(args.components_file).expanduser().resolve()
if not llvm_src.is_dir():
print(f"Error: LLVM source directory not found: {llvm_src}")
sys.exit(1)
if not (llvm_src / "llvm" / "CMakeLists.txt").exists():
print(f"Error: {llvm_src} does not look like an llvm-project root.")
sys.exit(1)
if not components_file.is_file():
print(f"Error: components file not found: {components_file}")
sys.exit(1)
components = load_components(components_file)
source_targets = collect_source_targets(llvm_src)
print(f"Found {len(source_targets)} targets in LLVM source tree")
print(f"Components file lists {len(components)} entries")
# Check for components that are missing from the source tree.
missing: list[tuple[str, list[str]]] = []
for name in components:
if name not in source_targets:
suggestions = difflib.get_close_matches(
name, source_targets, n=3, cutoff=0.6
)
missing.append((name, suggestions))
if missing:
print(f"\nError: {len(missing)} component(s) not found in the source tree:\n")
for name, suggestions in missing:
print(f" - {name}")
if suggestions:
print(f" Did you mean: {', '.join(suggestions)}?")
sys.exit(1)
# Warn about source targets not present in the component list.
component_set = set(components)
new_targets = sorted(source_targets - component_set - KNOWN_HEADER_TARGETS)
# Filter to targets that follow LLVM/Clang naming conventions to reduce noise.
noteworthy = [t for t in new_targets if t.startswith(("LLVM", "clang", "Clang"))]
if noteworthy:
print(
f"\nWarning: {len(noteworthy)} target(s) in source not listed in components:"
)
for name in noteworthy:
print(f" + {name}")
print("\nAll components validated successfully.")
sys.exit(0)
if __name__ == "__main__":
main()

24
scripts/watch-socket.sh Executable file
View File

@@ -0,0 +1,24 @@
#!/usr/bin/env bash
set -euo pipefail
BUILD_CMD=${BUILD_CMD:-".clice/build.sh"}
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "${ROOT_DIR}"
if ! command -v watchexec >/dev/null 2>&1; then
echo "watchexec is not installed or not in PATH." >&2
exit 1
fi
exec watchexec \
--project-origin "${ROOT_DIR}" \
--watch "${ROOT_DIR}/config" \
--watch "${ROOT_DIR}/src" \
--watch "${ROOT_DIR}/cmake" \
--watch "${ROOT_DIR}/CMakeLists.txt" \
--restart \
--clear \
--shell=bash \
-- "$BUILD_CMD && ./build/bin/clice --mode socket --port 50051"

View File

@@ -1,81 +1,64 @@
#include <csignal>
#include <cstdint>
#include <iostream>
#include <print>
#include <string>
#include "eventide/async/async.h"
#include "eventide/deco/deco.h"
#include "eventide/ipc/peer.h"
#include "eventide/ipc/recording_transport.h"
#include "eventide/ipc/transport.h"
#include "server/master_server.h"
#include "server/stateful_worker.h"
#include "server/stateless_worker.h"
#include "support/logging.h"
#include "kota/async/async.h"
#include "kota/deco/deco.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/peer.h"
#include "kota/ipc/recording_transport.h"
#include "kota/ipc/transport.h"
namespace clice {
using kota::deco::decl::KVStyle;
struct Options {
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Running mode: pipe, socket, stateless-worker, stateful-worker",
required = false)
DecoKV(names = {"--mode"};
help = "Running mode: pipe, socket, stateless-worker, stateful-worker";
required = false;)
<std::string> mode;
DecoKV(style = KVStyle::JoinedOrSeparate, help = "Socket mode address", required = false)
DecoKV(names = {"--host"}; help = "Socket mode address"; required = false;)
<std::string> host = "127.0.0.1";
DecoKV(style = KVStyle::JoinedOrSeparate, help = "Socket mode port", required = false)
DecoKV(names = {"--port"}; help = "Socket mode port"; required = false;)
<int> port = 50051;
DecoKV(style = KVStyle::JoinedOrSeparate,
names = {"--log-level", "--log-level="},
help = "Log level: trace, debug, info, warn, error, off",
required = false)
<std::string> log_level = "info";
DecoKV(names = {"--stateful-worker-count"}; help = "Number of stateful workers";
required = false;)
<std::uint32_t> stateful_worker_count;
DecoKV(style = KVStyle::JoinedOrSeparate,
help = "Record LSP input to file for replay testing",
required = false)
<std::string> record;
DecoKV(names = {"--stateless-worker-count"}; help = "Number of stateless workers";
required = false;)
<std::uint32_t> stateless_worker_count;
// Internal options (passed from master to worker processes)
DecoKV(style = KVStyle::JoinedOrSeparate,
names = {"--worker-memory-limit", "--worker-memory-limit="},
required = false)
DecoKV(names = {"--worker-memory-limit"}; help = "Memory limit per stateful worker (bytes)";
required = false;)
<std::uint64_t> worker_memory_limit;
DecoKV(style = KVStyle::JoinedOrSeparate,
names = {"--worker-name", "--worker-name="},
required = false)
<std::string> worker_name;
DecoKV(names = {"--log-level"}; help = "Log level: trace, debug, info, warn, error, off";
required = false;)
<std::string> log_level = "info";
DecoKV(style = KVStyle::JoinedOrSeparate, names = {"--log-dir", "--log-dir="}, required = false)
<std::string> log_dir;
DecoKV(names = {"--record"}; help = "Record LSP input to file for replay testing";
required = false;)
<std::string> record;
DecoFlag(names = {"-h", "--help"}, help = "Show help message", required = false)
DecoFlag(names = {"-h", "--help"}; help = "Show help message"; required = false;)
help;
DecoFlag(names = {"-v", "--version"}, help = "Show version", required = false)
DecoFlag(names = {"-v", "--version"}; help = "Show version"; required = false;)
version;
};
} // namespace clice
int main(int argc, const char** argv) {
#ifndef _WIN32
// On POSIX systems, ignore SIGPIPE so that writing to a closed pipe
// (e.g. when the LSP client disconnects) returns EPIPE instead of
// killing the process. This is standard practice for pipe-based servers.
signal(SIGPIPE, SIG_IGN);
#endif
auto args = kota::deco::util::argvify(argc, argv);
auto result = kota::deco::cli::parse<clice::Options>(args);
auto args = deco::util::argvify(argc, argv);
auto result = deco::cli::parse<clice::Options>(args);
if(!result.has_value()) {
LOG_ERROR("{}", result.error().message);
@@ -85,7 +68,7 @@ int main(int argc, const char** argv) {
auto& opts = result->options;
if(opts.help.value_or(false)) {
kota::deco::cli::write_usage_for<clice::Options>(std::cout, "clice [OPTIONS]");
deco::cli::write_usage_for<clice::Options>(std::cout, "clice [OPTIONS]");
return 0;
}
@@ -114,42 +97,35 @@ int main(int argc, const char** argv) {
auto& mode = *opts.mode;
auto worker_name = opts.worker_name.value_or("");
auto log_dir = opts.log_dir.value_or("");
if(mode == "stateless-worker") {
return clice::run_stateless_worker_mode(worker_name.empty() ? "stateless-worker"
: worker_name,
log_dir);
return clice::run_stateless_worker_mode();
}
if(mode == "stateful-worker") {
auto mem_limit = opts.worker_memory_limit.value_or(4ULL * 1024 * 1024 * 1024);
return clice::run_stateful_worker_mode(mem_limit,
worker_name.empty() ? "stateful-worker"
: worker_name,
log_dir);
return clice::run_stateful_worker_mode(mem_limit);
}
if(mode == "pipe") {
clice::logging::stderr_logger("master", clice::logging::options);
kota::event_loop loop;
namespace et = eventide;
et::event_loop loop;
auto transport = kota::ipc::StreamTransport::open_stdio(loop);
auto transport = et::ipc::StreamTransport::open_stdio(loop);
if(!transport) {
LOG_ERROR("failed to open stdio transport");
return 1;
}
std::unique_ptr<kota::ipc::Transport> final_transport = std::move(*transport);
std::unique_ptr<et::ipc::Transport> final_transport = std::move(*transport);
if(opts.record.has_value()) {
final_transport =
std::make_unique<kota::ipc::RecordingTransport>(std::move(final_transport),
*opts.record);
std::make_unique<et::ipc::RecordingTransport>(std::move(final_transport),
*opts.record);
}
kota::ipc::JsonPeer peer(loop, std::move(final_transport));
et::ipc::JsonPeer peer(loop, std::move(final_transport));
clice::MasterServer server(loop, peer, std::move(self_path));
server.register_handlers();
@@ -161,12 +137,13 @@ int main(int argc, const char** argv) {
if(mode == "socket") {
clice::logging::stderr_logger("master", clice::logging::options);
kota::event_loop loop;
namespace et = eventide;
et::event_loop loop;
auto host = opts.host.value_or("127.0.0.1");
auto port = opts.port.value_or(50051);
auto acceptor = kota::tcp::listen(host, port, {}, loop);
auto acceptor = et::tcp::listen(host, port, {}, loop);
if(!acceptor) {
LOG_ERROR("failed to listen on {}:{}", host, port);
return 1;
@@ -174,7 +151,7 @@ int main(int argc, const char** argv) {
LOG_INFO("Listening on {}:{} ...", host, port);
auto task = [&]() -> kota::task<> {
auto task = [&]() -> et::task<> {
auto client = co_await acceptor->accept();
if(!client.has_value()) {
LOG_ERROR("failed to accept connection");
@@ -184,13 +161,13 @@ int main(int argc, const char** argv) {
LOG_INFO("Client connected");
std::unique_ptr<kota::ipc::Transport> transport =
std::make_unique<kota::ipc::StreamTransport>(std::move(client.value()));
std::unique_ptr<et::ipc::Transport> transport =
std::make_unique<et::ipc::StreamTransport>(std::move(client.value()));
if(opts.record.has_value()) {
transport = std::make_unique<kota::ipc::RecordingTransport>(std::move(transport),
*opts.record);
transport = std::make_unique<et::ipc::RecordingTransport>(std::move(transport),
*opts.record);
}
kota::ipc::JsonPeer peer(loop, std::move(transport));
et::ipc::JsonPeer peer(loop, std::move(transport));
clice::MasterServer server(loop, peer, std::string(self_path));
server.register_handlers();

View File

@@ -23,44 +23,6 @@ namespace ranges = std::ranges;
} // namespace
std::vector<const char*> CompileCommand::to_argv() const {
std::vector<const char*> argv;
argv.reserve(resolved.flags.size() + 4);
if(resolved.is_cc1 && source_file) {
// cc1 mode requires TWO file-related arguments (both are needed):
// 1. -main-file-name <basename> — used by clang for diagnostics/debug info
// 2. <source_file> at the end — the actual input file path
// These are NOT duplicates: (1) is just the basename, (2) is the full path.
for(std::size_t i = 0; i < resolved.flags.size(); ++i) {
argv.push_back(resolved.flags[i]);
if(resolved.flags[i] == llvm::StringRef("-cc1")) {
argv.push_back("-main-file-name");
// path::filename returns a suffix of source_file (a pointer into
// the same buffer), so .data() is null-terminated because source_file is.
argv.push_back(path::filename(source_file).data());
}
}
} else {
argv.insert(argv.end(), resolved.flags.begin(), resolved.flags.end());
}
if(source_file) {
argv.push_back(source_file);
}
return argv;
}
std::vector<std::string> CompileCommand::to_string_argv() const {
auto argv = to_argv();
std::vector<std::string> result;
result.reserve(argv.size());
for(auto* arg: argv) {
result.emplace_back(arg);
}
return result;
}
CompilationDatabase::CompilationDatabase() = default;
CompilationDatabase::~CompilationDatabase() = default;
@@ -367,8 +329,8 @@ std::size_t CompilationDatabase::load(llvm::StringRef path) {
return entries.size();
}
llvm::SmallVector<CompileCommand> CompilationDatabase::lookup(llvm::StringRef file,
const CommandOptions& options) {
llvm::SmallVector<CompilationContext> CompilationDatabase::lookup(llvm::StringRef file,
const CommandOptions& options) {
auto path_id = paths.intern(file);
auto matched = find_entries(path_id);
@@ -376,18 +338,17 @@ llvm::SmallVector<CompileCommand> CompilationDatabase::lookup(llvm::StringRef fi
render_arg_to([&](llvm::StringRef s) { out.push_back(strings.save(s).data()); }, arg);
};
/// Build one CompileCommand from a single CompilationInfo.
auto build_command = [&](object_ptr<CompilationInfo> info) -> CompileCommand {
/// Build one CompilationContext from a single CompilationInfo.
auto build_context = [&](object_ptr<CompilationInfo> info) -> CompilationContext {
llvm::StringRef directory = info->directory;
std::vector<const char*> flags;
bool is_cc1 = false;
std::vector<const char*> arguments;
auto append_arg = [&](llvm::StringRef s) {
flags.emplace_back(strings.save(s).data());
arguments.emplace_back(strings.save(s).data());
};
auto append_args = [&](llvm::ArrayRef<const char*> args) {
flags.insert(flags.end(), args.begin(), args.end());
arguments.insert(arguments.end(), args.begin(), args.end());
};
if(options.query_toolchain) {
@@ -400,20 +361,23 @@ llvm::SmallVector<CompileCommand> CompilationDatabase::lookup(llvm::StringRef fi
append_args(info->canonical->arguments);
append_args(info->patch);
} else {
flags.assign(cached.begin(), cached.end());
flags.pop_back(); // remove temp source file
arguments.assign(cached.begin(), cached.end());
// TODO: add an assertion that the last arg is the temp source
// file (e.g., contains "query-toolchain") to guard against
// future changes in clang cc1 argument ordering.
arguments.pop_back(); // remove temp source file
// Replace resource dir if needed.
if(!resource_dir().empty()) {
llvm::StringRef old_resource_dir;
for(std::size_t i = 0; i + 1 < flags.size(); ++i) {
if(flags[i] == llvm::StringRef("-resource-dir")) {
old_resource_dir = flags[i + 1];
for(std::size_t i = 0; i + 1 < arguments.size(); ++i) {
if(arguments[i] == llvm::StringRef("-resource-dir")) {
old_resource_dir = arguments[i + 1];
break;
}
}
if(!old_resource_dir.empty() && old_resource_dir != resource_dir()) {
for(auto& arg: flags) {
for(auto& arg: arguments) {
llvm::StringRef s(arg);
if(s.starts_with(old_resource_dir)) {
auto replaced =
@@ -426,42 +390,39 @@ llvm::SmallVector<CompileCommand> CompilationDatabase::lookup(llvm::StringRef fi
append_args(info->patch);
// Strip -main-file-name and its value from flags (to_argv() will
// re-inject it with the correct basename when is_cc1 is set).
std::vector<const char*> cleaned;
cleaned.reserve(flags.size());
for(std::size_t i = 0; i < flags.size(); ++i) {
if(flags[i] == llvm::StringRef("-main-file-name") && i + 1 < flags.size()) {
++i; // skip the value
// Fix -main-file-name to match the actual file.
bool next_main_file = false;
for(auto& arg: arguments) {
if(arg == llvm::StringRef("-main-file-name")) {
next_main_file = true;
continue;
}
cleaned.push_back(flags[i]);
if(next_main_file) {
arg = strings.save(path::filename(file)).data();
next_main_file = false;
}
}
flags = std::move(cleaned);
}
// Detect cc1 mode (search rather than assuming index).
is_cc1 = ranges::contains(flags, llvm::StringRef("-cc1"));
// Inject our resource dir if not already present.
if(!resource_dir().empty()) {
bool has_resource_dir = false;
for(auto& arg: arguments) {
if(arg == llvm::StringRef("-resource-dir")) {
has_resource_dir = true;
break;
}
}
if(!has_resource_dir) {
append_arg("-resource-dir");
append_arg(resource_dir());
}
}
} else {
append_args(info->canonical->arguments);
append_args(info->patch);
}
// Inject our resource dir if not already present.
if(options.inject_resource_dir && !resource_dir().empty()) {
bool has_resource_dir = false;
for(auto& arg: flags) {
if(arg == llvm::StringRef("-resource-dir")) {
has_resource_dir = true;
break;
}
}
if(!has_resource_dir) {
append_arg("-resource-dir");
append_arg(resource_dir());
}
}
// Apply remove filter.
if(!options.remove.empty()) {
using Arg = std::unique_ptr<llvm::opt::Arg>;
@@ -479,12 +440,12 @@ llvm::SmallVector<CompileCommand> CompilationDatabase::lookup(llvm::StringRef fi
};
std::ranges::sort(remove_args, {}, get_id);
auto saved_flags = std::move(flags);
flags.clear();
flags.push_back(saved_flags.front());
auto saved_args = std::move(arguments);
arguments.clear();
arguments.push_back(saved_args.front());
parser->parse(
llvm::ArrayRef(saved_flags).drop_front(),
llvm::ArrayRef(saved_args).drop_front(),
[&](Arg arg) {
auto id = arg->getOption().getID();
auto range = std::ranges::equal_range(remove_args, id, {}, get_id);
@@ -500,7 +461,7 @@ llvm::SmallVector<CompileCommand> CompilationDatabase::lookup(llvm::StringRef fi
return;
}
}
render_arg(flags, *arg);
render_arg(arguments, *arg);
},
[](int, int) {});
}
@@ -509,34 +470,26 @@ llvm::SmallVector<CompileCommand> CompilationDatabase::lookup(llvm::StringRef fi
append_arg(arg);
}
return CompileCommand{
ResolvedFlags{directory, std::move(flags), is_cc1},
paths.resolve(path_id).data()
};
arguments.emplace_back(paths.resolve(path_id).data());
return CompilationContext(directory, std::move(arguments));
};
llvm::SmallVector<CompileCommand> results;
llvm::SmallVector<CompilationContext> results;
if(!matched.empty()) {
for(auto& entry: matched) {
results.push_back(build_command(entry.info));
results.push_back(build_context(entry.info));
}
} else {
// No matching entry — synthesize a default command.
std::vector<const char*> flags;
std::vector<const char*> arguments;
if(file.ends_with(".cpp") || file.ends_with(".hpp") || file.ends_with(".cc")) {
flags = {"clang++", "-std=c++20"};
arguments = {"clang++", "-std=c++20"};
} else {
flags = {"clang"};
arguments = {"clang"};
}
if(options.inject_resource_dir && !resource_dir().empty()) {
flags.push_back(strings.save("-resource-dir").data());
flags.push_back(strings.save(resource_dir()).data());
}
results.push_back(CompileCommand{
ResolvedFlags{{}, std::move(flags), false},
paths.resolve(path_id).data()
});
arguments.emplace_back(paths.resolve(path_id).data());
results.push_back(CompilationContext({}, std::move(arguments)));
}
return results;
@@ -560,8 +513,8 @@ SearchConfig CompilationDatabase::lookup_search_config(llvm::StringRef file,
}
auto results = lookup(file, options);
auto& cmd = results.front();
auto config = extract_search_config(cmd.to_argv(), cmd.resolved.directory);
auto& ctx = results.front();
auto config = extract_search_config(ctx.arguments, ctx.directory);
if(cacheable) {
auto key = ConfigCacheKey{matched.front().info.ptr, options_bits(options)};
@@ -697,11 +650,6 @@ std::uint32_t CompilationDatabase::intern_path(llvm::StringRef path) {
return paths.intern(path);
}
bool CompilationDatabase::has_entry(llvm::StringRef file) {
auto path_id = paths.intern(file);
return !find_entries(path_id).empty();
}
llvm::ArrayRef<CompilationEntry> CompilationDatabase::get_entries() const {
return entries;
}

View File

@@ -29,11 +29,6 @@ struct CommandOptions {
/// Set true in unittests to avoid cluttering test output.
bool suppress_logging = false;
/// Inject our resource dir into the flags if not already present.
/// Enabled by default so clang tools always use matching builtin headers.
/// Disable in unit tests that assert exact argument counts.
bool inject_resource_dir = true;
/// Extra arguments to remove from the original command line.
llvm::ArrayRef<std::string> remove;
@@ -41,35 +36,12 @@ struct CommandOptions {
llvm::ArrayRef<std::string> append;
};
/// File-independent compilation flags (shareable, suitable as cache key input).
/// Does NOT contain source file path or -main-file-name.
struct ResolvedFlags {
struct CompilationContext {
/// The working directory of compilation.
llvm::StringRef directory;
/// All flags excluding source file path and -main-file-name.
std::vector<const char*> flags;
/// Whether flags come from toolchain query (cc1 mode).
/// When true, flags are cc1 frontend args (resolved clang binary + "-cc1" + ...),
/// NOT the original driver command. to_argv() scans for "-cc1" in flags and
/// inserts -main-file-name immediately after it.
bool is_cc1 = false;
};
/// Compilation command = resolved flags + source file identity.
struct CompileCommand {
ResolvedFlags resolved;
/// Interned, pointer-stable. Must be null-terminated (required by to_argv()
/// and path::filename().data() which relies on the suffix being null-terminated).
const char* source_file = nullptr;
/// Produce full argv: flags + [-main-file-name <basename> if cc1] + source_file.
std::vector<const char*> to_argv() const;
/// Convenience: to_argv() converted to vector<string>.
std::vector<std::string> to_string_argv() const;
/// The compilation arguments.
std::vector<const char*> arguments;
};
/// Shared compiler identity — driver + all semantics-affecting flags.
@@ -202,10 +174,10 @@ public:
/// but toolchain cache survives. Returns the number of entries loaded.
std::size_t load(llvm::StringRef path);
/// Lookup the compile commands for a file. A file may have multiple
/// Lookup the compilation contexts for a file. A file may have multiple
/// compilation commands (e.g. different build configurations); all are returned.
llvm::SmallVector<CompileCommand> lookup(llvm::StringRef file,
const CommandOptions& options = {});
llvm::SmallVector<CompilationContext> lookup(llvm::StringRef file,
const CommandOptions& options = {});
/// Combined lookup + extract_search_config with internal caching.
SearchConfig lookup_search_config(llvm::StringRef file, const CommandOptions& options = {});
@@ -219,10 +191,6 @@ public:
/// Intern a file path and return its path_id.
std::uint32_t intern_path(llvm::StringRef path);
/// Check if a file has an explicit entry in the compilation database
/// (as opposed to a synthesized default).
bool has_entry(llvm::StringRef file);
/// All compilation entries (sorted by path_id).
llvm::ArrayRef<CompilationEntry> get_entries() const;

View File

@@ -5,10 +5,10 @@
#include <vector>
#include "command/argument_parser.h"
#include "eventide/reflection/enum.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "kota/meta/enum.h"
#include "llvm/ADT/ScopeExit.h"
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/FileSystem.h"
@@ -363,7 +363,7 @@ std::vector<const char*> query_toolchain(const QueryParams& params) {
case CompilerFamily::Unknown: {
/// TODO: nvcc and intel compilers need further exploration.
LOG_ERROR("Fail to query driver, unknown supported driver kind: {}, driver is {}",
kota::meta::enum_name(family),
eventide::refl::enum_name(family),
driver);
std::vector<const char*> result;

View File

@@ -219,10 +219,9 @@ public:
auto CreateASTConsumer(clang::CompilerInstance& instance, llvm::StringRef file)
-> std::unique_ptr<clang::ASTConsumer> final {
auto consumer = WrapperFrontendAction::CreateASTConsumer(instance, file);
if(!consumer)
return nullptr;
return std::make_unique<ProxyASTConsumer>(std::move(consumer), unit);
return std::make_unique<ProxyASTConsumer>(
WrapperFrontendAction::CreateASTConsumer(instance, file),
unit);
}
/// Make this public.

View File

@@ -81,16 +81,13 @@ auto CompilationUnitRef::file_offset(clang::SourceLocation location) -> std::uin
}
auto CompilationUnitRef::file_path(clang::FileID fid) -> llvm::StringRef {
if(!fid.isValid())
return {};
assert(fid.isValid() && "Invalid fid");
if(auto it = self->path_cache.find(fid); it != self->path_cache.end()) {
return it->second;
}
auto entry = self->SM().getFileEntryRefForID(fid);
if(!entry) {
return {};
}
assert(entry && "Invalid file entry");
llvm::SmallString<128> path;
@@ -245,19 +242,13 @@ std::vector<std::string> CompilationUnitRef::deps() {
for(auto& [fid, directive]: directives()) {
for(auto& include: directive.includes) {
if(!include.skipped) {
auto path = file_path(include.fid);
if(!path.empty()) {
deps.try_emplace(path);
}
deps.try_emplace(file_path(include.fid));
}
}
for(auto& has_include: directive.has_includes) {
if(has_include.fid.isValid()) {
auto path = file_path(has_include.fid);
if(!path.empty()) {
deps.try_emplace(path);
}
deps.try_emplace(file_path(has_include.fid));
}
}
}

View File

@@ -65,36 +65,11 @@ public:
/// Rewritten Preprocessor Callbacks
/// ============================================================================
void HasEmbed(clang::SourceLocation location,
llvm::StringRef filename,
bool is_angled,
clang::OptionalFileEntryRef file) override {
unit->directives[unit.file_id(location)].has_embeds.emplace_back(clice::HasEmbed{
.file_name = filename,
.file = file,
.is_angled = is_angled,
.loc = location,
});
}
void EmbedDirective(clang::SourceLocation location,
clang::StringRef filename,
bool is_angled,
clang::OptionalFileEntryRef file,
const clang::LexEmbedParametersResult&) override {
unit->directives[unit.file_id(location)].embeds.emplace_back(Embed{
.file_name = filename,
.file = file,
.is_angled = is_angled,
.loc = location,
});
}
void InclusionDirective(clang::SourceLocation hash_loc,
const clang::Token& include_tok,
llvm::StringRef,
bool,
clang::CharSourceRange,
clang::CharSourceRange filename_range,
clang::OptionalFileEntryRef,
llvm::StringRef,
llvm::StringRef,
@@ -108,6 +83,7 @@ public:
unit->directives[prev_fid].includes.emplace_back(Include{
.fid = {},
.location = include_tok.getLocation(),
.filename_range = filename_range.getAsRange(),
});
}

View File

@@ -20,8 +20,11 @@ struct Include {
/// The file id of included file.
clang::FileID fid;
/// Location of the `include` keyword.
/// Location of the `include`.
clang::SourceLocation location;
/// The range of filename(includes `""` or `<>`).
clang::SourceRange filename_range;
};
/// Information about `__has_include` directive.
@@ -129,39 +132,6 @@ struct Import {
std::vector<clang::SourceLocation> name_locations;
};
/// Information about `#embed` directive.
struct Embed {
/// The file name in the embed directive, not including quotes or angle brackets.
llvm::StringRef file_name;
/// The actual file that may be embedded by this embed directive.
clang::OptionalFileEntryRef file;
/// Whether the file name is angled.
bool is_angled;
/// Location of the `#` token.
clang::SourceLocation loc;
/// TODO: Currently we do not store parameters of the embed directive.
/// See clang::LexEmbedParametersResult for details.
};
/// Information about `__has_embed` directive.
struct HasEmbed {
/// The file name in the embed directive, not including quotes or angle brackets.
llvm::StringRef file_name;
/// The actual file that may be embedded by this embed directive.
clang::OptionalFileEntryRef file;
/// Whether the file name is angled.
bool is_angled;
/// Location of the `__has_embed` token.
clang::SourceLocation loc;
};
struct Directive {
std::vector<Include> includes;
std::vector<HasInclude> has_includes;
@@ -169,8 +139,6 @@ struct Directive {
std::vector<MacroRef> macros;
std::vector<Pragma> pragmas;
std::vector<Import> imports;
std::vector<Embed> embeds;
std::vector<HasEmbed> has_embeds;
};
} // namespace clice

View File

@@ -53,8 +53,8 @@ auto completion_kind(const clang::NamedDecl* decl) -> protocol::CompletionItemKi
return protocol::CompletionItemKind::Module;
}
if(llvm::isa<clang::CXXConstructorDecl>(decl)) {
return protocol::CompletionItemKind::Constructor;
if(llvm::isa<clang::FunctionDecl, clang::FunctionTemplateDecl>(decl)) {
return protocol::CompletionItemKind::Function;
}
if(llvm::isa<clang::CXXMethodDecl,
@@ -64,8 +64,8 @@ auto completion_kind(const clang::NamedDecl* decl) -> protocol::CompletionItemKi
return protocol::CompletionItemKind::Method;
}
if(llvm::isa<clang::FunctionDecl, clang::FunctionTemplateDecl>(decl)) {
return protocol::CompletionItemKind::Function;
if(llvm::isa<clang::CXXConstructorDecl>(decl)) {
return protocol::CompletionItemKind::Constructor;
}
if(llvm::isa<clang::FieldDecl, clang::IndirectFieldDecl>(decl)) {
@@ -109,115 +109,6 @@ auto completion_kind(const clang::NamedDecl* decl) -> protocol::CompletionItemKi
return protocol::CompletionItemKind::Text;
}
/// Extract the function signature (parameter list) from a CodeCompletionString.
/// Returns something like "(int x, float y)" for display in labelDetails.detail.
auto extract_signature(const clang::CodeCompletionString& ccs) -> std::string {
std::string signature;
bool in_parens = false;
for(const auto& chunk: ccs) {
using CK = clang::CodeCompletionString::ChunkKind;
switch(chunk.Kind) {
case CK::CK_LeftParen:
in_parens = true;
signature += '(';
break;
case CK::CK_RightParen:
signature += ')';
in_parens = false;
break;
case CK::CK_Placeholder:
case CK::CK_CurrentParameter:
if(in_parens && chunk.Text) {
signature += chunk.Text;
}
break;
case CK::CK_Text:
case CK::CK_Informative:
if(in_parens && chunk.Text) {
signature += chunk.Text;
}
break;
case CK::CK_LeftAngle:
signature += '<';
in_parens = true;
break;
case CK::CK_RightAngle:
signature += '>';
in_parens = false;
break;
case CK::CK_Comma:
if(in_parens) {
signature += ", ";
}
break;
default: break;
}
}
return signature;
}
/// Build a snippet string from a CodeCompletionString.
/// Produces e.g. "funcName(${1:int x}, ${2:float y})" for functions,
/// or "ClassName<${1:T}>" for class templates.
auto build_snippet(const clang::CodeCompletionString& ccs) -> std::string {
std::string snippet;
unsigned placeholder_index = 0;
for(const auto& chunk: ccs) {
using CK = clang::CodeCompletionString::ChunkKind;
switch(chunk.Kind) {
case CK::CK_TypedText:
if(chunk.Text) {
snippet += chunk.Text;
}
break;
case CK::CK_Placeholder:
if(chunk.Text) {
snippet += std::format("${{{0}:{1}}}", ++placeholder_index, chunk.Text);
}
break;
case CK::CK_LeftParen: snippet += '('; break;
case CK::CK_RightParen: snippet += ')'; break;
case CK::CK_LeftAngle: snippet += '<'; break;
case CK::CK_RightAngle: snippet += '>'; break;
case CK::CK_Comma: snippet += ", "; break;
case CK::CK_Text:
if(chunk.Text) {
snippet += chunk.Text;
}
break;
case CK::CK_Optional:
// Optional chunks contain default arguments — skip for snippet.
break;
case CK::CK_Informative:
case CK::CK_ResultType:
case CK::CK_CurrentParameter:
// Display-only chunks, not part of insertion.
break;
default: break;
}
}
// If no placeholders were generated, return empty to signal plain text.
if(placeholder_index == 0) {
return {};
}
return snippet;
}
/// Extract the return type from a CodeCompletionString.
auto extract_return_type(const clang::CodeCompletionString& ccs) -> std::string {
for(const auto& chunk: ccs) {
if(chunk.Kind == clang::CodeCompletionString::CK_ResultType && chunk.Text) {
return chunk.Text;
}
}
return {};
}
struct OverloadItem {
protocol::CompletionItem item;
float score = 0.0F;
@@ -268,45 +159,29 @@ public:
overloads.reserve(candidate_count);
std::unordered_map<std::string, std::size_t> overload_index;
bool prefix_starts_with_underscore = prefix.spelling.starts_with("_");
auto build_item =
[&](llvm::StringRef label, protocol::CompletionItemKind kind, llvm::StringRef insert) {
protocol::CompletionItem item{
.label = label.str(),
};
item.kind = kind;
auto build_item = [&](llvm::StringRef label,
protocol::CompletionItemKind kind,
llvm::StringRef insert,
bool is_snippet = false) {
protocol::CompletionItem item{
.label = label.str(),
protocol::TextEdit edit{
.range = replace_range,
.new_text = insert.empty() ? label.str() : insert.str(),
};
item.text_edit = std::move(edit);
return item;
};
item.kind = kind;
protocol::TextEdit edit{
.range = replace_range,
.new_text = insert.empty() ? label.str() : insert.str(),
};
item.text_edit = std::move(edit);
if(is_snippet) {
item.insert_text_format = protocol::InsertTextFormat::Snippet;
}
return item;
};
auto try_add = [&](llvm::StringRef label,
protocol::CompletionItemKind kind,
llvm::StringRef insert_text,
llvm::StringRef overload_key,
llvm::StringRef signature = {},
llvm::StringRef return_type = {},
bool is_snippet = false,
bool is_deprecated = false) {
llvm::StringRef overload_key) {
if(label.empty()) {
return;
}
// Filter out _/__ prefixed internal symbols unless user typed _.
if(!prefix_starts_with_underscore && label.starts_with("_")) {
return;
}
auto score = matcher.match(label);
if(!score.has_value()) {
return;
@@ -316,21 +191,8 @@ public:
auto [it, inserted] =
overload_index.try_emplace(overload_key.str(), overloads.size());
if(inserted) {
auto item = build_item(label, kind, insert_text, is_snippet);
auto item = build_item(label, kind, insert_text);
item.sort_text = std::format("{}", *score);
if(!signature.empty() || !return_type.empty()) {
protocol::CompletionItemLabelDetails details;
if(!signature.empty()) {
details.detail = signature.str();
}
if(!return_type.empty()) {
details.description = return_type.str();
}
item.label_details = std::move(details);
}
if(is_deprecated) {
item.tags = std::vector{protocol::CompletionItemTag::Deprecated};
}
overloads.push_back({
.item = std::move(item),
.score = *score,
@@ -347,21 +209,8 @@ public:
return;
}
auto item = build_item(label, kind, insert_text, is_snippet);
auto item = build_item(label, kind, insert_text);
item.sort_text = std::format("{}", *score);
if(!signature.empty() || !return_type.empty()) {
protocol::CompletionItemLabelDetails details;
if(!signature.empty()) {
details.detail = signature.str();
}
if(!return_type.empty()) {
details.description = return_type.str();
}
item.label_details = std::move(details);
}
if(is_deprecated) {
item.tags = std::vector{protocol::CompletionItemTag::Deprecated};
}
collected.push_back(std::move(item));
};
@@ -393,60 +242,16 @@ public:
break;
}
auto label = ast::name_of(declaration);
auto kind = completion_kind(declaration);
// For constructors and deduction guides, use the class name
// (without template args) instead of the full type name.
// e.g. "vector" instead of "vector<_Tp, _Alloc>".
std::string label;
if(auto* ctor = llvm::dyn_cast<clang::CXXConstructorDecl>(declaration)) {
label = ctor->getParent()->getName().str();
} else if(auto* guide =
llvm::dyn_cast<clang::CXXDeductionGuideDecl>(declaration)) {
label = guide->getDeducedTemplate()->getName().str();
} else {
label = ast::name_of(declaration);
}
llvm::SmallString<256> qualified_name;
bool is_callable = kind == protocol::CompletionItemKind::Function ||
kind == protocol::CompletionItemKind::Method ||
kind == protocol::CompletionItemKind::Constructor;
if(options.bundle_overloads && is_callable) {
if(options.bundle_overloads && kind == protocol::CompletionItemKind::Function) {
llvm::raw_svector_ostream stream(qualified_name);
declaration->printQualifiedName(stream);
}
std::string signature;
std::string return_type;
std::string snippet;
auto* ccs =
candidate.CreateCodeCompletionString(sema,
context,
getAllocator(),
getCodeCompletionTUInfo(),
/*IncludeBriefComments=*/false);
if(ccs) {
signature = extract_signature(*ccs);
return_type = extract_return_type(*ccs);
// Generate snippet for non-bundled callables.
if(is_callable && !options.bundle_overloads &&
options.enable_function_arguments_snippet) {
snippet = build_snippet(*ccs);
}
}
bool has_snippet = !snippet.empty();
auto insert = has_snippet ? llvm::StringRef(snippet) : llvm::StringRef(label);
bool deprecated = candidate.Availability == CXAvailability_Deprecated;
try_add(label,
kind,
insert,
qualified_name.str(),
signature,
return_type,
has_snippet,
deprecated);
try_add(label, kind, label, qualified_name.str());
break;
}
}
@@ -454,48 +259,11 @@ public:
for(auto& entry: overloads) {
if(entry.count > 1) {
protocol::CompletionItemLabelDetails details;
details.detail = std::format("(…) +{} overloads", entry.count);
entry.item.label_details = std::move(details);
entry.item.detail = "(...)";
}
collected.push_back(std::move(entry.item));
}
// In bundle mode, deduplicate by label: when the same name appears as
// both a class and its constructors/deduction guides, keep only the
// highest-priority kind (Class > Function/Method > others).
if(options.bundle_overloads) {
auto kind_priority = [](protocol::CompletionItemKind k) -> int {
switch(k) {
case protocol::CompletionItemKind::Class:
case protocol::CompletionItemKind::Struct: return 3;
case protocol::CompletionItemKind::Function:
case protocol::CompletionItemKind::Method: return 2;
case protocol::CompletionItemKind::Constructor: return 1;
default: return 0;
}
};
std::unordered_map<std::string, std::size_t> label_index;
std::vector<protocol::CompletionItem> deduped;
deduped.reserve(collected.size());
for(auto& item: collected) {
auto [it, inserted] = label_index.try_emplace(item.label, deduped.size());
if(inserted) {
deduped.push_back(std::move(item));
} else {
auto& existing = deduped[it->second];
int old_prio = existing.kind.has_value() ? kind_priority(*existing.kind) : 0;
int new_prio = item.kind.has_value() ? kind_priority(*item.kind) : 0;
if(new_prio > old_prio) {
existing = std::move(item);
}
}
}
collected.swap(deduped);
}
output.clear();
output.swap(collected);
}

View File

@@ -2,15 +2,14 @@
#include <string>
#include <vector>
#include "eventide/ipc/lsp/uri.h"
#include "feature/feature.h"
#include "kota/ipc/lsp/uri.h"
namespace clice::feature {
namespace {
namespace lsp = kota::ipc::lsp;
namespace lsp = eventide::ipc::lsp;
auto to_uri(llvm::StringRef file) -> std::string {
const auto file_view = std::string_view(file.data(), file.size());

View File

@@ -1,12 +1,14 @@
#include <algorithm>
#include <cstdint>
#include <string>
#include <vector>
#include "feature/feature.h"
#include "syntax/lexer.h"
namespace clice::feature {
namespace {} // namespace
auto document_links(CompilationUnitRef unit, PositionEncoding encoding)
-> std::vector<protocol::DocumentLink> {
std::vector<protocol::DocumentLink> links;
@@ -20,42 +22,50 @@ auto document_links(CompilationUnitRef unit, PositionEncoding encoding)
auto content = unit.interested_content();
PositionMapper converter(content, encoding);
auto& directives = directives_it->second;
auto* lang_opts = &unit.lang_options();
auto add_link = [&](clang::SourceLocation loc, llvm::StringRef target) {
auto [fid, offset] = unit.decompose_location(loc);
if(fid != interested || offset >= content.size())
return;
auto range = find_directive_argument(content, offset, lang_opts);
if(!range)
return;
protocol::DocumentLink link{.range = to_range(converter, *range)};
link.target = target.str();
links.push_back(std::move(link));
};
links.reserve(directives.includes.size() + directives.has_includes.size());
for(const auto& include: directives.includes) {
if(include.fid.isValid()) {
add_link(include.location, unit.file_path(include.fid));
auto [fid, range] = unit.decompose_range(include.filename_range);
if(fid != interested || !range.valid()) {
continue;
}
protocol::DocumentLink link{
.range = to_range(converter, range),
};
link.target = std::string(unit.file_path(include.fid));
links.push_back(std::move(link));
}
for(const auto& has_include: directives.has_includes) {
if(has_include.fid.isValid()) {
add_link(has_include.location, unit.file_path(has_include.fid));
if(has_include.fid.isInvalid()) {
continue;
}
}
for(const auto& embed: directives.embeds) {
if(embed.file) {
add_link(embed.loc, embed.file->getName());
auto [fid, offset] = unit.decompose_location(has_include.location);
if(fid != interested || offset >= content.size()) {
continue;
}
}
for(const auto& has_embed: directives.has_embeds) {
if(has_embed.file) {
add_link(has_embed.loc, has_embed.file->getName());
auto tail = content.substr(offset);
char open = tail.front();
if(open != '<' && open != '"') {
continue;
}
char close = open == '<' ? '>' : '"';
auto close_index = tail.find(close, 1);
if(close_index == llvm::StringRef::npos) {
continue;
}
LocalSourceRange range(offset, offset + static_cast<std::uint32_t>(close_index + 1));
protocol::DocumentLink link{
.range = to_range(converter, range),
};
link.target = std::string(unit.file_path(has_include.fid));
links.push_back(std::move(link));
}
return links;

View File

@@ -7,9 +7,8 @@
#include "compile/compilation.h"
#include "compile/compilation_unit.h"
#include "kota/ipc/lsp/position.h"
#include "kota/ipc/lsp/protocol.h"
#include "eventide/ipc/lsp/position.h"
#include "eventide/ipc/lsp/protocol.h"
namespace clang {
@@ -19,11 +18,11 @@ class NamedDecl;
namespace clice::feature {
namespace protocol = kota::ipc::protocol;
namespace protocol = eventide::ipc::protocol;
using kota::ipc::lsp::PositionEncoding;
using kota::ipc::lsp::PositionMapper;
using kota::ipc::lsp::parse_position_encoding;
using eventide::ipc::lsp::PositionEncoding;
using eventide::ipc::lsp::PositionMapper;
using eventide::ipc::lsp::parse_position_encoding;
inline auto to_range(const PositionMapper& converter, LocalSourceRange range) -> protocol::Range {
return protocol::Range{

View File

@@ -12,7 +12,6 @@
#include "clang/AST/Attr.h"
#include "clang/Basic/IdentifierTable.h"
#include "clang/Basic/Module.h"
namespace clice::feature {
@@ -24,8 +23,12 @@ struct RawToken {
std::uint32_t modifiers = 0;
};
constexpr std::uint32_t bit(SymbolModifiers::Kind kind) {
return static_cast<std::uint32_t>(kind);
}
void add_modifier(std::uint32_t& modifiers, SymbolModifiers::Kind kind) {
modifiers |= SymbolModifiers::to_mask(kind);
modifiers |= bit(kind);
}
auto type_index(SymbolKind kind) -> std::uint32_t {
@@ -36,132 +39,6 @@ auto encode_modifiers(std::uint32_t modifiers) -> std::uint32_t {
return modifiers;
}
bool is_dependent(const clang::Decl* D) {
return isa<clang::UnresolvedUsingValueDecl>(D);
}
/// Returns true if `decl` is considered to be from a default/system library.
/// This currently checks the systemness of the file by include type, although
/// different heuristics may be used in the future (e.g. sysroot paths).
bool is_default_library(const clang::Decl* decl) {
clang::SourceLocation location = decl->getLocation();
if(!location.isValid()) {
return false;
}
return decl->getASTContext().getSourceManager().isInSystemHeader(location);
}
// "Static" means many things in C++, only some get the "static" modifier.
//
// Meanings that do:
// - Members associated with the class rather than the instance.
// This is what 'static' most often means across languages.
// - static local variables
// These are similarly "detached from their context" by the static keyword.
// In practice, these are rarely used inside classes, reducing confusion.
//
// Meanings that don't:
// - Namespace-scoped variables, which have static storage class.
// This is implicit, so the keyword "static" isn't so strongly associated.
// If we want a modifier for these, "global scope" is probably the concept.
// - Namespace-scoped variables/functions explicitly marked "static".
// There the keyword changes *linkage* , which is a totally different concept.
// If we want to model this, "file scope" would be a nice modifier.
//
// This is confusing, and maybe we should use another name, but because "static"
// is a standard LSP modifier, having one with that name has advantages.
bool is_static(const clang::Decl* decl) {
if(const auto* method = llvm::dyn_cast<clang::CXXMethodDecl>(decl)) {
return method->isStatic();
}
if(const auto* var_decl = llvm::dyn_cast<clang::VarDecl>(decl)) {
return var_decl->isStaticDataMember() || var_decl->isStaticLocal();
}
if(const auto* objc_property = llvm::dyn_cast<clang::ObjCPropertyDecl>(decl)) {
return objc_property->isClassProperty();
}
if(const auto* objc_method = llvm::dyn_cast<clang::ObjCMethodDecl>(decl)) {
return objc_method->isClassMethod();
}
if(const auto* function = llvm::dyn_cast<clang::FunctionDecl>(decl)) {
return function->isStatic();
}
return false;
}
// Whether `type` is const in a loose sense: would a value of this type be readonly?
bool is_const(clang::QualType type) {
if(type.isNull()) {
return false;
}
type = type.getNonReferenceType();
if(type.isConstQualified()) {
return true;
}
if(const auto* array_type = type->getAsArrayTypeUnsafe()) {
return is_const(array_type->getElementType());
}
if(is_const(type->getPointeeType())) {
return true;
}
return false;
}
// Whether `decl` is const in a loose sense (should it be highlighted as such?)
// FIXME: This is separate from whether a particular usage can mutate `decl`.
// We may want a receiver in `value.size()` to be readonly even if `value` is mutable.
bool is_const(const clang::Decl* decl) {
if(llvm::isa<clang::EnumConstantDecl>(decl) ||
llvm::isa<clang::NonTypeTemplateParmDecl>(decl)) {
return true;
}
if(llvm::isa<clang::FieldDecl>(decl) || llvm::isa<clang::VarDecl>(decl) ||
llvm::isa<clang::MSPropertyDecl>(decl) || llvm::isa<clang::BindingDecl>(decl)) {
if(is_const(llvm::cast<clang::ValueDecl>(decl)->getType())) {
return true;
}
}
if(const auto* objc_property = llvm::dyn_cast<clang::ObjCPropertyDecl>(decl)) {
if(objc_property->isReadOnly()) {
return true;
}
}
if(const auto* ms_property = llvm::dyn_cast<clang::MSPropertyDecl>(decl)) {
if(!ms_property->hasSetter()) {
return true;
}
}
if(const auto* method = llvm::dyn_cast<clang::CXXMethodDecl>(decl)) {
if(method->isConst()) {
return true;
}
}
if(const auto* function = llvm::dyn_cast<clang::FunctionDecl>(decl)) {
return is_const(function->getReturnType());
}
return false;
}
// Indicates whether declaration `decl` is abstract in cases where it is a struct or a
// class.
bool is_abstract(const clang::Decl* decl) {
if(const auto* method = llvm::dyn_cast<clang::CXXMethodDecl>(decl)) {
return method->isPureVirtual();
}
if(const auto* record = llvm::dyn_cast<clang::CXXRecordDecl>(decl)) {
return record->hasDefinition() && record->isAbstract();
}
return false;
}
// Indicates whether declaration `decl` is virtual in cases where it is a method.
bool is_virtual(const clang::Decl* decl) {
if(const auto* method = llvm::dyn_cast<clang::CXXMethodDecl>(decl)) {
return method->isVirtual();
}
return false;
}
class SemanticTokensCollector : public SemanticVisitor<SemanticTokensCollector> {
public:
explicit SemanticTokensCollector(CompilationUnitRef unit) : SemanticVisitor(unit, true) {}
@@ -169,7 +46,6 @@ public:
auto collect() -> std::vector<RawToken> {
highlight_lexical(unit.interested_file());
run();
highlight_modules();
merge_tokens();
return std::move(tokens);
}
@@ -179,8 +55,6 @@ public:
clang::SourceLocation location) {
std::uint32_t modifiers = 0;
if(relation.is_one_of(RelationKind::Definition)) {
// todo: clangd add both Declaration and Definition modifiers for definitions.
// add_modifier(modifiers, SymbolModifiers::Declaration);
add_modifier(modifiers, SymbolModifiers::Definition);
} else if(relation.is_one_of(RelationKind::Declaration)) {
add_modifier(modifiers, SymbolModifiers::Declaration);
@@ -189,42 +63,6 @@ public:
if(ast::is_templated(decl)) {
add_modifier(modifiers, SymbolModifiers::Templated);
}
// Apply attribute-style modifiers to the underlying declaration.
// The attribute tests don't want to look at the template.
if(const auto* template_decl = llvm::dyn_cast<clang::TemplateDecl>(decl)) {
if(const auto* templated_decl = template_decl->getTemplatedDecl())
decl = templated_decl;
}
// TODO: add scope-based modifiers once the local model supports them.
// if (auto Mod = scopeModifier(Decl))
// Tok.addModifier(*Mod);
if(is_const(decl)) {
add_modifier(modifiers, SymbolModifiers::Readonly);
}
if(is_static(decl)) {
add_modifier(modifiers, SymbolModifiers::Static);
}
if(is_abstract(decl)) {
add_modifier(modifiers, SymbolModifiers::Abstract);
}
if(is_virtual(decl)) {
add_modifier(modifiers, SymbolModifiers::Virtual);
}
if(is_default_library(decl)) {
add_modifier(modifiers, SymbolModifiers::DefaultLibrary);
}
if(decl->isDeprecated()) {
add_modifier(modifiers, SymbolModifiers::Deprecated);
}
if(is_dependent(decl)) {
add_modifier(modifiers, SymbolModifiers::DependentName);
}
if(llvm::isa<clang::CXXConstructorDecl>(decl) ||
llvm::isa<clang::CXXDestructorDecl>(decl)) {
add_modifier(modifiers, SymbolModifiers::ConstructorOrDestructor);
}
add_token(location, SymbolKind::from(decl), modifiers);
}
@@ -242,10 +80,6 @@ public:
add_token(location, SymbolKind::Macro, modifiers);
}
// handleModuleOccurrence
// handleRelation
void handleAttrOccurrence(const clang::Attr* attr, clang::SourceRange range) {
auto [begin, end] = range;
if(llvm::isa<clang::FinalAttr, clang::OverrideAttr>(attr)) {
@@ -293,58 +127,6 @@ private:
});
}
void highlight_modules() {
auto interested = unit.interested_file();
auto directives_it = unit.directives().find(interested);
if(directives_it != unit.directives().end()) {
for(const auto& import: directives_it->second.imports) {
add_token(import.location, SymbolKind::Keyword, 0);
for(auto loc: import.name_locations) {
add_token(loc, SymbolKind::Module, 0);
}
}
}
auto* mod = unit.context().getCurrentNamedModule();
if(!mod) {
return;
}
auto def_loc = mod->DefinitionLoc;
if(!def_loc.isValid() || !def_loc.isFileID()) {
return;
}
auto [fid, offset] = unit.decompose_location(def_loc);
if(fid != interested) {
return;
}
auto content = unit.file_content(fid);
auto& lang_opts = unit.lang_options();
Lexer lexer(content.substr(offset), false, &lang_opts);
auto module_token = lexer.advance();
if(module_token.is_identifier()) {
auto range = LocalSourceRange(offset + module_token.range.begin,
offset + module_token.range.end);
tokens.push_back({.range = range, .kind = SymbolKind::Keyword, .modifiers = 0});
}
// Scan for identifiers (module name parts) until semicolon/eof.
while(true) {
auto token = lexer.advance();
if(token.is_eof() || token.kind == clang::tok::semi) {
break;
}
if(token.is_identifier()) {
auto range = LocalSourceRange(offset + token.range.begin, offset + token.range.end);
tokens.push_back({.range = range, .kind = SymbolKind::Module, .modifiers = 0});
}
}
}
void highlight_lexical(clang::FileID fid) {
auto content = unit.file_content(fid);
auto& lang_opts = unit.lang_options();
@@ -376,6 +158,7 @@ private:
case clang::tok::utf16_string_literal:
case clang::tok::utf32_string_literal: kind = SymbolKind::String; break;
case clang::tok::header_name: kind = SymbolKind::Header; break;
case clang::tok::identifier: break;
case clang::tok::raw_identifier: {
auto previous = lexer.last();
if(previous.is_pp_keyword && previous.text(content) == "define") {
@@ -389,8 +172,457 @@ private:
}
break;
}
default: break;
/* Keywords */
case clang::tok::kw_auto:
case clang::tok::kw_break:
case clang::tok::kw_case:
case clang::tok::kw_char:
case clang::tok::kw_const:
case clang::tok::kw_continue:
case clang::tok::kw_default:
case clang::tok::kw_do:
case clang::tok::kw_double:
case clang::tok::kw_else:
case clang::tok::kw_enum:
case clang::tok::kw_extern:
case clang::tok::kw_float:
case clang::tok::kw_for:
case clang::tok::kw_goto:
case clang::tok::kw_if:
case clang::tok::kw_int:
case clang::tok::kw__ExtInt:
case clang::tok::kw__BitInt:
case clang::tok::kw_long:
case clang::tok::kw_register:
case clang::tok::kw_return:
case clang::tok::kw_short:
case clang::tok::kw_signed:
case clang::tok::kw_sizeof:
case clang::tok::kw___datasizeof:
case clang::tok::kw_static:
case clang::tok::kw_struct:
case clang::tok::kw_switch:
case clang::tok::kw_typedef:
case clang::tok::kw_union:
case clang::tok::kw_unsigned:
case clang::tok::kw_void:
case clang::tok::kw_volatile:
case clang::tok::kw_while:
case clang::tok::kw__Alignas:
case clang::tok::kw__Alignof:
case clang::tok::kw__Atomic:
case clang::tok::kw__Bool:
case clang::tok::kw__Complex:
case clang::tok::kw__Generic:
case clang::tok::kw__Imaginary:
case clang::tok::kw__Noreturn:
case clang::tok::kw__Static_assert:
case clang::tok::kw__Thread_local:
case clang::tok::kw___func__:
case clang::tok::kw___objc_yes:
case clang::tok::kw___objc_no:
case clang::tok::kw___ptrauth:
case clang::tok::kw__Countof:
case clang::tok::kw_asm:
case clang::tok::kw_bool:
case clang::tok::kw_catch:
case clang::tok::kw_class:
case clang::tok::kw_const_cast:
case clang::tok::kw_delete:
case clang::tok::kw_dynamic_cast:
case clang::tok::kw_explicit:
case clang::tok::kw_export:
case clang::tok::kw_false:
case clang::tok::kw_friend:
case clang::tok::kw_mutable:
case clang::tok::kw_namespace:
case clang::tok::kw_new:
case clang::tok::kw_operator:
case clang::tok::kw_private:
case clang::tok::kw_protected:
case clang::tok::kw_public:
case clang::tok::kw_reinterpret_cast:
case clang::tok::kw_static_cast:
case clang::tok::kw_template:
case clang::tok::kw_this:
case clang::tok::kw_throw:
case clang::tok::kw_true:
case clang::tok::kw_try:
case clang::tok::kw_typename:
case clang::tok::kw_typeid:
case clang::tok::kw_using:
case clang::tok::kw_virtual:
case clang::tok::kw_wchar_t:
case clang::tok::kw_restrict:
case clang::tok::kw_inline:
case clang::tok::kw_alignas:
case clang::tok::kw_alignof:
case clang::tok::kw_char16_t:
case clang::tok::kw_char32_t:
case clang::tok::kw_constexpr:
case clang::tok::kw_decltype:
case clang::tok::kw_noexcept:
case clang::tok::kw_nullptr:
case clang::tok::kw_static_assert:
case clang::tok::kw_thread_local:
case clang::tok::kw_co_await:
case clang::tok::kw_co_return:
case clang::tok::kw_co_yield:
case clang::tok::kw_module:
case clang::tok::kw_import:
case clang::tok::kw_consteval:
case clang::tok::kw_constinit:
case clang::tok::kw_concept:
case clang::tok::kw_requires:
case clang::tok::kw_char8_t:
case clang::tok::kw__Float16:
case clang::tok::kw_typeof:
case clang::tok::kw_typeof_unqual:
case clang::tok::kw__Accum:
case clang::tok::kw__Fract:
case clang::tok::kw__Sat:
case clang::tok::kw__Decimal32:
case clang::tok::kw__Decimal64:
case clang::tok::kw__Decimal128:
case clang::tok::kw___null:
case clang::tok::kw___alignof:
case clang::tok::kw___attribute:
case clang::tok::kw___builtin_choose_expr:
case clang::tok::kw___builtin_offsetof:
case clang::tok::kw___builtin_FILE:
case clang::tok::kw___builtin_FILE_NAME:
case clang::tok::kw___builtin_FUNCTION:
case clang::tok::kw___builtin_FUNCSIG:
case clang::tok::kw___builtin_LINE:
case clang::tok::kw___builtin_COLUMN:
case clang::tok::kw___builtin_source_location:
case clang::tok::kw___builtin_types_compatible_p:
case clang::tok::kw___builtin_va_arg:
case clang::tok::kw___extension__:
case clang::tok::kw___float128:
case clang::tok::kw___ibm128:
case clang::tok::kw___imag:
case clang::tok::kw___int128:
case clang::tok::kw___label__:
case clang::tok::kw___real:
case clang::tok::kw___thread:
case clang::tok::kw___FUNCTION__:
case clang::tok::kw___PRETTY_FUNCTION__:
case clang::tok::kw___auto_type:
case clang::tok::kw___FUNCDNAME__:
case clang::tok::kw___FUNCSIG__:
case clang::tok::kw_L__FUNCTION__:
case clang::tok::kw_L__FUNCSIG__:
case clang::tok::kw___is_interface_class:
case clang::tok::kw___is_sealed:
case clang::tok::kw___is_destructible:
case clang::tok::kw___is_trivially_destructible:
case clang::tok::kw___is_nothrow_destructible:
case clang::tok::kw___is_nothrow_assignable:
case clang::tok::kw___is_constructible:
case clang::tok::kw___is_nothrow_constructible:
case clang::tok::kw___is_assignable:
case clang::tok::kw___has_nothrow_move_assign:
case clang::tok::kw___has_trivial_move_assign:
case clang::tok::kw___has_trivial_move_constructor:
case clang::tok::kw___builtin_is_implicit_lifetime:
case clang::tok::kw___builtin_is_virtual_base_of:
case clang::tok::kw___has_nothrow_assign:
case clang::tok::kw___has_nothrow_copy:
case clang::tok::kw___has_nothrow_constructor:
case clang::tok::kw___has_trivial_assign:
case clang::tok::kw___has_trivial_copy:
case clang::tok::kw___has_trivial_constructor:
case clang::tok::kw___has_trivial_destructor:
case clang::tok::kw___has_virtual_destructor:
case clang::tok::kw___is_abstract:
case clang::tok::kw___is_aggregate:
case clang::tok::kw___is_base_of:
case clang::tok::kw___is_class:
case clang::tok::kw___is_convertible_to:
case clang::tok::kw___is_empty:
case clang::tok::kw___is_enum:
case clang::tok::kw___is_final:
case clang::tok::kw___is_literal:
case clang::tok::kw___is_pod:
case clang::tok::kw___is_polymorphic:
case clang::tok::kw___is_standard_layout:
case clang::tok::kw___is_trivial:
case clang::tok::kw___is_trivially_assignable:
case clang::tok::kw___is_trivially_constructible:
case clang::tok::kw___is_trivially_copyable:
case clang::tok::kw___is_union:
case clang::tok::kw___has_unique_object_representations:
case clang::tok::kw___is_layout_compatible:
case clang::tok::kw___is_pointer_interconvertible_base_of:
case clang::tok::kw___add_lvalue_reference:
case clang::tok::kw___add_pointer:
case clang::tok::kw___add_rvalue_reference:
case clang::tok::kw___decay:
case clang::tok::kw___make_signed:
case clang::tok::kw___make_unsigned:
case clang::tok::kw___remove_all_extents:
case clang::tok::kw___remove_const:
case clang::tok::kw___remove_cv:
case clang::tok::kw___remove_cvref:
case clang::tok::kw___remove_extent:
case clang::tok::kw___remove_pointer:
case clang::tok::kw___remove_reference_t:
case clang::tok::kw___remove_restrict:
case clang::tok::kw___remove_volatile:
case clang::tok::kw___underlying_type:
case clang::tok::kw___is_trivially_equality_comparable:
case clang::tok::kw___is_bounded_array:
case clang::tok::kw___is_unbounded_array:
case clang::tok::kw___is_scoped_enum:
case clang::tok::kw___can_pass_in_regs:
case clang::tok::kw___reference_binds_to_temporary:
case clang::tok::kw___reference_constructs_from_temporary:
case clang::tok::kw___reference_converts_from_temporary:
case clang::tok::kw_:
case clang::tok::kw___builtin_is_cpp_trivially_relocatable:
case clang::tok::kw___is_trivially_relocatable:
case clang::tok::kw___is_bitwise_cloneable:
case clang::tok::kw___builtin_is_replaceable:
case clang::tok::kw___builtin_structured_binding_size:
case clang::tok::kw___is_lvalue_expr:
case clang::tok::kw___is_rvalue_expr:
case clang::tok::kw___is_arithmetic:
case clang::tok::kw___is_floating_point:
case clang::tok::kw___is_integral:
case clang::tok::kw___is_complete_type:
case clang::tok::kw___is_void:
case clang::tok::kw___is_array:
case clang::tok::kw___is_function:
case clang::tok::kw___is_reference:
case clang::tok::kw___is_lvalue_reference:
case clang::tok::kw___is_rvalue_reference:
case clang::tok::kw___is_fundamental:
case clang::tok::kw___is_object:
case clang::tok::kw___is_scalar:
case clang::tok::kw___is_compound:
case clang::tok::kw___is_pointer:
case clang::tok::kw___is_member_object_pointer:
case clang::tok::kw___is_member_function_pointer:
case clang::tok::kw___is_member_pointer:
case clang::tok::kw___is_const:
case clang::tok::kw___is_volatile:
case clang::tok::kw___is_signed:
case clang::tok::kw___is_unsigned:
case clang::tok::kw___is_same:
case clang::tok::kw___is_convertible:
case clang::tok::kw___is_nothrow_convertible:
case clang::tok::kw___array_rank:
case clang::tok::kw___array_extent:
case clang::tok::kw___private_extern__:
case clang::tok::kw___module_private__:
case clang::tok::kw___builtin_ptrauth_type_discriminator:
case clang::tok::kw___declspec:
case clang::tok::kw___cdecl:
case clang::tok::kw___stdcall:
case clang::tok::kw___fastcall:
case clang::tok::kw___thiscall:
case clang::tok::kw___regcall:
case clang::tok::kw___vectorcall:
case clang::tok::kw___forceinline:
case clang::tok::kw___unaligned:
case clang::tok::kw___super:
case clang::tok::kw___global:
case clang::tok::kw___local:
case clang::tok::kw___constant:
case clang::tok::kw___private:
case clang::tok::kw___generic:
case clang::tok::kw___kernel:
case clang::tok::kw___read_only:
case clang::tok::kw___write_only:
case clang::tok::kw___read_write:
case clang::tok::kw___builtin_astype:
case clang::tok::kw_vec_step:
case clang::tok::kw_image1d_t:
case clang::tok::kw_image1d_array_t:
case clang::tok::kw_image1d_buffer_t:
case clang::tok::kw_image2d_t:
case clang::tok::kw_image2d_array_t:
case clang::tok::kw_image2d_depth_t:
case clang::tok::kw_image2d_array_depth_t:
case clang::tok::kw_image2d_msaa_t:
case clang::tok::kw_image2d_array_msaa_t:
case clang::tok::kw_image2d_msaa_depth_t:
case clang::tok::kw_image2d_array_msaa_depth_t:
case clang::tok::kw_image3d_t:
case clang::tok::kw_pipe:
case clang::tok::kw_addrspace_cast:
case clang::tok::kw___noinline__:
case clang::tok::kw_cbuffer:
case clang::tok::kw_tbuffer:
case clang::tok::kw_groupshared:
case clang::tok::kw_in:
case clang::tok::kw_inout:
case clang::tok::kw_out:
case clang::tok::kw___hlsl_resource_t:
case clang::tok::kw___builtin_hlsl_is_scalarized_layout_compatible:
case clang::tok::kw___builtin_hlsl_is_intangible:
case clang::tok::kw___builtin_hlsl_is_typed_resource_element_compatible:
case clang::tok::kw___builtin_omp_required_simd_align:
case clang::tok::kw___pascal:
case clang::tok::kw___vector:
case clang::tok::kw___pixel:
case clang::tok::kw___bool:
case clang::tok::kw___bf16:
case clang::tok::kw_half:
case clang::tok::kw___bridge:
case clang::tok::kw___bridge_transfer:
case clang::tok::kw___bridge_retained:
case clang::tok::kw___bridge_retain:
case clang::tok::kw___covariant:
case clang::tok::kw___contravariant:
case clang::tok::kw___kindof:
case clang::tok::kw__Nonnull:
case clang::tok::kw__Nullable:
case clang::tok::kw__Nullable_result:
case clang::tok::kw__Null_unspecified:
case clang::tok::kw___funcref:
case clang::tok::kw___ptr64:
case clang::tok::kw___ptr32:
case clang::tok::kw___sptr:
case clang::tok::kw___uptr:
case clang::tok::kw___w64:
case clang::tok::kw___uuidof:
case clang::tok::kw___try:
case clang::tok::kw___finally:
case clang::tok::kw___leave:
case clang::tok::kw___int64:
case clang::tok::kw___if_exists:
case clang::tok::kw___if_not_exists:
case clang::tok::kw___single_inheritance:
case clang::tok::kw___multiple_inheritance:
case clang::tok::kw___virtual_inheritance:
case clang::tok::kw___interface:
case clang::tok::kw___builtin_convertvector:
case clang::tok::kw___builtin_vectorelements:
case clang::tok::kw___builtin_bit_cast:
case clang::tok::kw___builtin_available:
case clang::tok::kw___builtin_sycl_unique_stable_name:
case clang::tok::kw___arm_agnostic:
case clang::tok::kw___arm_in:
case clang::tok::kw___arm_inout:
case clang::tok::kw___arm_locally_streaming:
case clang::tok::kw___arm_new:
case clang::tok::kw___arm_out:
case clang::tok::kw___arm_preserves:
case clang::tok::kw___arm_streaming:
case clang::tok::kw___arm_streaming_compatible:
case clang::tok::kw___unknown_anytype: kind = SymbolKind::Keyword; break;
/* Operators */
case clang::tok::l_square:
case clang::tok::r_square:
case clang::tok::l_paren:
case clang::tok::r_paren:
case clang::tok::l_brace:
case clang::tok::r_brace:
case clang::tok::period:
case clang::tok::ellipsis:
case clang::tok::amp:
case clang::tok::ampamp:
case clang::tok::ampequal:
case clang::tok::star:
case clang::tok::starequal:
case clang::tok::plus:
case clang::tok::plusplus:
case clang::tok::plusequal:
case clang::tok::minus:
case clang::tok::arrow:
case clang::tok::minusminus:
case clang::tok::minusequal:
case clang::tok::tilde:
case clang::tok::exclaim:
case clang::tok::exclaimequal:
case clang::tok::slash:
case clang::tok::slashequal:
case clang::tok::percent:
case clang::tok::percentequal:
case clang::tok::less:
case clang::tok::lessless:
case clang::tok::lessequal:
case clang::tok::lesslessequal:
case clang::tok::greater:
case clang::tok::greatergreater:
case clang::tok::greaterequal:
case clang::tok::greatergreaterequal:
case clang::tok::caret:
case clang::tok::caretequal:
case clang::tok::pipe:
case clang::tok::pipepipe:
case clang::tok::pipeequal:
case clang::tok::question:
case clang::tok::colon:
case clang::tok::semi:
case clang::tok::equal:
case clang::tok::equalequal:
case clang::tok::comma:
case clang::tok::hashat:
case clang::tok::periodstar:
case clang::tok::arrowstar:
case clang::tok::coloncolon:
case clang::tok::at:
case clang::tok::lesslessless:
case clang::tok::greatergreatergreater: break;
case clang::tok::annot_cxxscope:
case clang::tok::annot_typename:
case clang::tok::annot_template_id:
case clang::tok::annot_non_type:
case clang::tok::annot_non_type_undeclared:
case clang::tok::annot_non_type_dependent:
case clang::tok::annot_overload_set:
case clang::tok::annot_primary_expr:
case clang::tok::annot_decltype:
case clang::tok::annot_pack_indexing_type:
case clang::tok::annot_pragma_unused:
case clang::tok::annot_pragma_vis:
case clang::tok::annot_pragma_pack:
case clang::tok::annot_pragma_parser_crash:
case clang::tok::annot_pragma_captured:
case clang::tok::annot_pragma_dump:
case clang::tok::annot_pragma_msstruct:
case clang::tok::annot_pragma_align:
case clang::tok::annot_pragma_weak:
case clang::tok::annot_pragma_weakalias:
case clang::tok::annot_pragma_redefine_extname:
case clang::tok::annot_pragma_fp_contract:
case clang::tok::annot_pragma_fenv_access:
case clang::tok::annot_pragma_fenv_access_ms:
case clang::tok::annot_pragma_fenv_round:
case clang::tok::annot_pragma_cx_limited_range:
case clang::tok::annot_pragma_float_control:
case clang::tok::annot_pragma_ms_pointers_to_members:
case clang::tok::annot_pragma_ms_vtordisp:
case clang::tok::annot_pragma_ms_pragma:
case clang::tok::annot_pragma_opencl_extension:
case clang::tok::annot_attr_openmp:
case clang::tok::annot_pragma_openmp:
case clang::tok::annot_pragma_openmp_end:
case clang::tok::annot_pragma_openacc:
case clang::tok::annot_pragma_openacc_end:
case clang::tok::annot_pragma_loop_hint:
case clang::tok::annot_pragma_fp:
case clang::tok::annot_pragma_attribute:
case clang::tok::annot_pragma_riscv:
case clang::tok::annot_module_include:
case clang::tok::annot_module_begin:
case clang::tok::annot_module_end:
case clang::tok::annot_header_unit:
case clang::tok::annot_repl_input_end:
case clang::tok::annot_embed: break;
/* Others */
case clang::tok::spaceship:
case clang::tok::binary_data:
case clang::tok::hash:
case clang::tok::hashhash:
case clang::tok::unknown:
case clang::tok::eof:
case clang::tok::eod:
case clang::tok::code_completion:
case clang::tok::NUM_TOKENS: break;
}
}
@@ -399,17 +631,10 @@ private:
}
static void resolve_conflict(RawToken& last, const RawToken& current) {
(void)current;
if(last.kind == SymbolKind::Conflict) {
return;
}
// Directive is a low-priority lexical kind; semantic tokens override it.
if(last.kind == SymbolKind::Directive) {
last = current;
return;
}
if(current.kind == SymbolKind::Directive) {
return;
}
last.kind = SymbolKind::Conflict;
}

View File

@@ -14,8 +14,8 @@ namespace {
class Builder : public SemanticVisitor<Builder> {
public:
Builder(TUIndex& result, CompilationUnitRef unit, bool interested_only) :
SemanticVisitor<Builder>(unit, interested_only), result(result) {
Builder(TUIndex& result, CompilationUnitRef unit) :
SemanticVisitor<Builder>(unit, false), result(result) {
result.graph = IncludeGraph::from(unit);
}
@@ -188,11 +188,11 @@ std::array<std::uint8_t, 32> FileIndex::hash() {
return hasher.final();
}
TUIndex TUIndex::build(CompilationUnitRef unit, bool interested_only) {
TUIndex TUIndex::build(CompilationUnitRef unit) {
TUIndex index;
index.built_at = unit.build_at();
Builder builder(index, unit, interested_only);
Builder builder(index, unit);
builder.build();
return index;

View File

@@ -85,7 +85,7 @@ struct TUIndex {
FileIndex main_file_index;
static TUIndex build(CompilationUnitRef unit, bool interested_only = false);
static TUIndex build(CompilationUnitRef unit);
void serialize(llvm::raw_ostream& os) const;

View File

@@ -37,6 +37,10 @@ std::string name_of(const clang::NamedDecl* decl);
std::string display_name_of(const clang::NamedDecl* decl);
clang::NestedNameSpecifierLoc get_qualifier_loc(const clang::NamedDecl* decl);
std::string print_template_specialization_args(const clang::NamedDecl* decl);
/// To response go-to-type-definition request. Some decls actually have a type
/// for example the result of `typeof(var)` is the type of `var`. This function
/// returns the type for the decl if any.

1449
src/semantic/find_target.cpp Normal file

File diff suppressed because it is too large Load Diff

136
src/semantic/find_target.h Normal file
View File

@@ -0,0 +1,136 @@
#pragma once
#include "llvm/ADT/STLFunctionalExtras.h"
#include "llvm/ADT/SmallVector.h"
#include "llvm/Support/raw_ostream.h"
#include "clang/AST/ASTContext.h"
#include "clang/AST/ASTTypeTraits.h"
#include "clang/AST/NestedNameSpecifier.h"
#include "clang/AST/Stmt.h"
#include "clang/Basic/SourceLocation.h"
namespace clang {
class Decl;
class NamedDecl;
} // namespace clang
namespace clice {
class TemplateResolver;
/// Information about a reference written in the source code, independent of
/// the AST node that contains it.
struct ReferenceLoc {
/// Qualifier written in the source code, e.g. `ns::` for `ns::foo`.
clang::NestedNameSpecifierLoc Qualifier;
/// Start location of the last name part, e.g. `foo` in `ns::foo<int>`.
clang::SourceLocation NameLoc;
/// True when the reference is introducing a declaration or definition.
bool IsDecl = false;
/// The declarations referenced by the written name.
llvm::SmallVector<const clang::NamedDecl*, 1> Targets;
};
enum class DeclRelation : unsigned {
/// The written name is an alias that should be preserved in results.
Alias = 1u << 0,
/// The target was reached by desugaring or following the aliased entity.
Underlying = 1u << 1,
/// The target is a concrete template instantiation.
TemplateInstantiation = 1u << 2,
/// The target is the template pattern underlying an instantiation.
TemplatePattern = 1u << 3,
};
struct DeclRelationSet {
unsigned bits = 0;
constexpr DeclRelationSet() = default;
constexpr DeclRelationSet(DeclRelation relation) : bits(static_cast<unsigned>(relation)) {}
constexpr explicit DeclRelationSet(unsigned bits) : bits(bits) {}
constexpr bool contains(DeclRelationSet other) const {
return (bits & other.bits) == other.bits;
}
constexpr bool contains(DeclRelation relation) const {
return (bits & static_cast<unsigned>(relation)) != 0;
}
constexpr explicit operator bool() const {
return bits != 0;
}
constexpr DeclRelationSet& operator|=(DeclRelationSet other) {
bits |= other.bits;
return *this;
}
constexpr DeclRelationSet& operator|=(DeclRelation relation) {
bits |= static_cast<unsigned>(relation);
return *this;
}
};
constexpr DeclRelationSet operator|(DeclRelationSet lhs, DeclRelationSet rhs) {
return DeclRelationSet(lhs.bits | rhs.bits);
}
constexpr DeclRelationSet operator|(DeclRelationSet lhs, DeclRelation rhs) {
return lhs | DeclRelationSet(rhs);
}
constexpr DeclRelationSet operator|(DeclRelation lhs, DeclRelationSet rhs) {
return DeclRelationSet(lhs) | rhs;
}
constexpr DeclRelationSet operator|(DeclRelation lhs, DeclRelation rhs) {
return DeclRelationSet(lhs) | rhs;
}
constexpr DeclRelationSet operator&(DeclRelationSet lhs, DeclRelationSet rhs) {
return DeclRelationSet(lhs.bits & rhs.bits);
}
constexpr DeclRelationSet operator&(DeclRelationSet lhs, DeclRelation rhs) {
return lhs & DeclRelationSet(rhs);
}
struct TargetDecl {
const clang::NamedDecl* Decl = nullptr;
DeclRelationSet Relations;
};
llvm::raw_ostream& operator<<(llvm::raw_ostream& os, ReferenceLoc ref);
/// Finds all declarations a selected AST node may refer to, including alias
/// and template-instantiation relationships that higher-level APIs may filter.
auto all_target_decls(const clang::DynTypedNode& node, TemplateResolver* resolver = nullptr)
-> llvm::SmallVector<TargetDecl, 1>;
/// Recursively traverses \p stmt and reports all references explicitly written in
/// the source code.
void explicit_references(const clang::Stmt* stmt,
llvm::function_ref<void(ReferenceLoc)> out,
TemplateResolver* resolver = nullptr);
/// Recursively traverses \p decl and reports all references explicitly written in
/// the source code.
void explicit_references(const clang::Decl* decl,
llvm::function_ref<void(ReferenceLoc)> out,
TemplateResolver* resolver = nullptr);
/// Recursively traverses the full AST and reports all references explicitly
/// written in the source code.
void explicit_references(const clang::ASTContext& ast,
llvm::function_ref<void(ReferenceLoc)> out,
TemplateResolver* resolver = nullptr);
} // namespace clice

File diff suppressed because it is too large Load Diff

View File

@@ -2,11 +2,11 @@
#include "clang/AST/ExprCXX.h"
#include "clang/AST/Type.h"
#include "clang/Basic/SourceManager.h"
#include "clang/Sema/Sema.h"
namespace clang {
class Sema;
}
namespace clice {
@@ -17,12 +17,9 @@ namespace clice {
/// completion, you cannot get go-to-definition, etc. To avoid this, we just use
/// some heuristics to simplify the dependent names as normal type/expression.
/// For example, `std::vector<T>::value_type` can be simplified as `T`.
///
/// Thread safety: NOT thread-safe. Each compilation unit should have its own resolver.
/// The `resolved` cache persists across multiple resolve() calls on the same unit.
class TemplateResolver {
public:
explicit TemplateResolver(clang::Sema& sema) : sema(sema) {}
TemplateResolver(clang::Sema& sema) : sema(sema) {}
clang::QualType resolve(clang::QualType type);
@@ -30,7 +27,7 @@ public:
void resolve(clang::UnresolvedLookupExpr* expr);
// TODO: Use a clearer approach for resolving UnresolvedLookupExpr.
// TODO: use a relative clear way to resolve `UnresolvedLookupExpr`.
void resolve(clang::UnresolvedUsingType* type);
@@ -43,6 +40,10 @@ public:
/// Look up the name in the given nested name specifier.
lookup_result lookup(const clang::NestedNameSpecifier* NNS, clang::DeclarationName name);
lookup_result lookup(clang::DeclarationName name) {
return sema.getASTContext().getTranslationUnitDecl()->lookup(name);
}
lookup_result lookup(const clang::DependentNameType* type) {
return lookup(type->getQualifier(), type->getIdentifier());
}
@@ -53,7 +54,7 @@ public:
if(identifier) {
return lookup(template_name.getQualifier(), identifier);
} else {
/// TODO: Operators don't have an IdentifierInfo; need DeclarationName-based lookup.
/// FIXME: Operators does't have a name.
return {};
}
}
@@ -63,7 +64,7 @@ public:
}
lookup_result lookup(const clang::UnresolvedLookupExpr* expr) {
/// TODO: Only returns the first TemplateDecl; should handle overloaded lookups.
/// FIXME:
for(auto decl: expr->decls()) {
if(auto TD = llvm::dyn_cast<clang::TemplateDecl>(decl)) {
return lookup_result(TD);
@@ -77,8 +78,8 @@ public:
return {};
}
/// TODO: Implement dependent member expression lookup (e.g. `x.template foo<T>()`).
lookup_result lookup(const clang::CXXDependentScopeMemberExpr* expr) {
/// TODO:
lookup_result lookup(clang::CXXDependentScopeMemberExpr* expr) {
return {};
}
@@ -86,20 +87,30 @@ public:
return lookup(decl->getQualifier(), decl->getDeclName());
}
lookup_result lookup(const clang::UnresolvedUsingTypenameDecl* decl) {
lookup_result resolve(const clang::UnresolvedUsingTypenameDecl* decl) {
return lookup(decl->getQualifier(), decl->getDeclName());
}
#ifndef NDEBUG
static inline bool debug = false;
#endif
private:
clang::Sema& sema;
/// Cache of resolved dependent types, keyed by AST node pointer.
/// Shared across resolve() calls within the same TU for performance.
/// This is safe because a given AST node (DependentNameType*, etc.) has a
/// unique identity within the TU — the same pointer always refers to the same
/// syntactic occurrence. Different syntactic occurrences of the "same" type
/// have different AST node pointers.
llvm::DenseMap<const void*, clang::QualType> resolved;
public:
auto source_manager() -> clang::SourceManager& {
return sema.getSourceManager();
}
auto lang_options() const -> const clang::LangOptions& {
return sema.getLangOpts();
}
auto ast_context() -> clang::ASTContext& {
return sema.getASTContext();
}
};
} // namespace clice

View File

@@ -131,6 +131,33 @@ public:
}
}
}
// if(auto module = unit.context().getCurrentNamedModule()) {
// auto keyword = module->DefinitionLoc;
// auto begin = TB.spelledTokenContaining(keyword);
// // assert(begin->kind() == clang::tok::identifier && begin->text(SM) == "module" &&
// // "Invalid module declaration");
//
// begin += 1;
// auto end = TB.spelledTokens(unit.file_id(keyword)).end();
//
// for(auto iter = begin; iter != end; ++iter) {
// if(iter->kind() == clang::tok::identifier) {
// if(auto next = iter + 1; next != end && (next->kind() == clang::tok::period ||
// next->kind() == clang::tok::colon)) {
// iter += 1;
// continue;
// }
//
// end = iter + 1;
// break;
// }
//
// std::unreachable();
// }
//
// handleModuleOccurrence(keyword, llvm::ArrayRef<clang::syntax::Token>(begin, end));
//}
}
public:

View File

@@ -80,79 +80,27 @@ private:
struct SymbolModifiers {
enum Kind : std::uint32_t {
/// Represents that the symbol is a declaration(e.g. function declaration).
Declaration = 0,
Declaration = 1u << 0,
/// Represents that the symbol is a definition(e.g. function definition).
Definition = 1,
Definition = 1u << 1,
/// Represents that the symbol is const modified(e.g. `const` variable).
Const = 2,
Const = 1u << 2,
/// Represents that the symbol is overloaded(e.g. overloaded functions and operators).
Overloaded = 3,
Overloaded = 1u << 3,
/// Represents that the symbol is a part of type(e.g. `*` in `int*`).
Typed = 4,
Typed = 1u << 4,
/// Represents that the symbol is a template(e.g. class template or function template).
Templated = 5,
/// Represents that the symbol is deprecated.
Deprecated = 6,
/// Represents that the symbol is deduced.
Deduced = 7,
/// Represents that the symbol is readonly.
Readonly = 8,
/// Represents that the symbol is static.
Static = 9,
/// Represents that the symbol is abstract.
Abstract = 10,
/// Represents that the symbol is virtual.
Virtual = 11,
/// Represents that the symbol is a dependent name.
DependentName = 12,
/// Represents that the symbol comes from the default library.
DefaultLibrary = 13,
/// Represents that the symbol is used through a mutable reference.
UsedAsMutableReference = 14,
/// Represents that the symbol is used through a mutable pointer.
UsedAsMutablePointer = 15,
/// Represents that the symbol is a constructor or destructor.
ConstructorOrDestructor = 16,
/// Represents that the symbol is user-defined.
UserDefined = 17,
/// Represents that the symbol is function-scoped.
FunctionScope = 18,
/// Represents that the symbol is class-scoped.
ClassScope = 19,
/// Represents that the symbol is file-scoped.
FileScope = 20,
/// Represents that the symbol is global-scoped.
GlobalScope = 21,
Templated = 1u << 5,
};
constexpr static std::uint32_t to_mask(Kind kind) {
return std::uint32_t(1) << static_cast<std::uint32_t>(kind);
}
constexpr SymbolModifiers() = default;
constexpr SymbolModifiers(Kind kind) : value(to_mask(kind)) {}
constexpr SymbolModifiers(Kind kind) : value(static_cast<std::uint32_t>(kind)) {}
constexpr explicit SymbolModifiers(std::uint32_t bits) : value(bits) {}
@@ -161,7 +109,7 @@ struct SymbolModifiers {
}
constexpr bool contains(Kind kind) const {
return (value & to_mask(kind)) != 0;
return (value & static_cast<std::uint32_t>(kind)) != 0;
}
private:

View File

@@ -6,8 +6,6 @@
namespace clice {
namespace ranges = std::ranges;
CompileGraph::CompileGraph(dispatch_fn dispatch, resolve_fn resolve) :
dispatch(std::move(dispatch)), resolve(std::move(resolve)) {}
@@ -33,19 +31,13 @@ void CompileGraph::ensure_resolved(std::uint32_t path_id) {
}
}
kota::task<bool> CompileGraph::compile_deps(std::uint32_t path_id) {
llvm::DenseSet<std::uint32_t> ancestors;
co_return co_await compile_impl(path_id, ancestors, false);
}
kota::task<bool> CompileGraph::compile(std::uint32_t path_id) {
et::task<bool> CompileGraph::compile(std::uint32_t path_id) {
llvm::DenseSet<std::uint32_t> ancestors;
co_return co_await compile_impl(path_id, ancestors);
}
kota::task<bool> CompileGraph::compile_impl(std::uint32_t path_id,
llvm::DenseSet<std::uint32_t> ancestors,
bool dispatch_self) {
et::task<bool> CompileGraph::compile_impl(std::uint32_t path_id,
llvm::DenseSet<std::uint32_t> ancestors) {
ensure_resolved(path_id);
// Cycle detection: if this unit is already in the compile chain, bail out.
@@ -56,27 +48,6 @@ kota::task<bool> CompileGraph::compile_impl(std::uint32_t path_id,
// Re-lookup after ensure_resolved may have mutated the map.
auto it = units.find(path_id);
// For deps-only mode, compile dependencies concurrently and return.
if(!dispatch_self) {
auto deps = it->second.dependencies;
if(deps.empty()) {
co_return true;
}
std::vector<kota::task<bool>> dep_tasks;
dep_tasks.reserve(deps.size());
for(auto dep_id: deps) {
dep_tasks.push_back(compile_impl(dep_id, ancestors));
}
auto results = co_await kota::when_all(std::move(dep_tasks));
for(auto ok: results) {
if(!ok) {
co_return false;
}
}
co_return true;
}
// Already clean.
if(!it->second.dirty) {
co_return true;
@@ -93,16 +64,9 @@ kota::task<bool> CompileGraph::compile_impl(std::uint32_t path_id,
co_return !units.find(path_id)->second.dirty;
}
// Begin compilation. The finish lambda ensures compiling/completion state
// is always cleaned up, regardless of how the function exits.
// Begin compilation.
it->second.compiling = true;
it->second.completion = std::make_unique<kota::event>();
auto finish = [&, path_id] {
auto& u = units.find(path_id)->second;
u.compiling = false;
u.completion->set();
};
it->second.completion = std::make_unique<et::event>();
// Copy deps and capture generation before co_await (DenseMap iterator safety).
auto deps = it->second.dependencies;
@@ -113,49 +77,60 @@ kota::task<bool> CompileGraph::compile_impl(std::uint32_t path_id,
// Deadlocks from cross-branch cycles (e.g. 1->{2,3}, 2->3, 3->2) are
// prevented by has_wait_cycle() checking before completion.wait().
if(!deps.empty()) {
std::vector<kota::task<bool, void, kota::cancellation>> dep_tasks;
std::vector<et::task<bool, void, et::cancellation>> dep_tasks;
dep_tasks.reserve(deps.size());
for(auto dep_id: deps) {
dep_tasks.push_back(kota::with_token(compile_impl(dep_id, ancestors), token));
dep_tasks.push_back(et::with_token(compile_impl(dep_id, ancestors), token));
}
auto results = co_await kota::when_all(std::move(dep_tasks));
auto results = co_await et::when_all(std::move(dep_tasks));
auto& u = units.find(path_id)->second;
if(results.is_cancelled()) {
finish();
co_await kota::cancel();
u.compiling = false;
u.completion->set();
co_await et::cancel();
}
for(auto ok: *results) {
if(!ok) {
finish();
u.compiling = false;
u.completion->set();
co_return false;
}
}
}
// Dispatch the actual compilation, cancellable via the pre-captured token.
auto result = co_await kota::with_token(dispatch(path_id), token);
// Using the token captured before co_await ensures cancellation propagates
// correctly even if update() replaces the source during dependency compilation.
{
auto result = co_await et::with_token(dispatch(path_id), token);
if(!result.has_value()) {
finish();
co_await kota::cancel();
}
if(!*result) {
finish();
co_return false;
auto& u = units.find(path_id)->second;
if(!result.has_value()) {
u.compiling = false;
u.completion->set();
co_await et::cancel();
}
if(!*result) {
u.compiling = false;
u.completion->set();
co_return false;
}
}
// Success — only clear dirty if update() hasn't bumped the generation.
auto& final_unit = units.find(path_id)->second;
if(final_unit.generation != gen) {
finish();
// update() was called while dispatch was in flight.
final_unit.compiling = false;
final_unit.completion->set();
co_return false;
}
final_unit.dirty = false;
finish();
final_unit.compiling = false;
final_unit.completion->set();
co_return true;
}
@@ -190,7 +165,8 @@ llvm::SmallVector<std::uint32_t> CompileGraph::update(std::uint32_t path_id) {
auto dep_it = units.find(dep_id);
if(dep_it != units.end()) {
auto& dependents = dep_it->second.dependents;
dependents.erase(ranges::remove(dependents, path_id).begin(), dependents.end());
dependents.erase(std::remove(dependents.begin(), dependents.end(), path_id),
dependents.end());
}
}
unit.dependencies.clear();
@@ -199,7 +175,7 @@ llvm::SmallVector<std::uint32_t> CompileGraph::update(std::uint32_t path_id) {
// Cancel in-flight compilation if running.
if(unit.compiling) {
unit.source->cancel();
unit.source = std::make_unique<kota::cancellation_source>();
unit.source = std::make_unique<et::cancellation_source>();
}
unit.dirty = true;
unit.generation++;
@@ -247,7 +223,7 @@ bool CompileGraph::has_wait_cycle(std::uint32_t target,
void CompileGraph::cancel_all() {
for(auto& [_, unit]: units) {
unit.source->cancel();
unit.source = std::make_unique<kota::cancellation_source>();
unit.source = std::make_unique<et::cancellation_source>();
}
}

View File

@@ -4,13 +4,16 @@
#include <functional>
#include <memory>
#include "kota/async/async.h"
#include "eventide/async/async.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/SmallVector.h"
namespace clice {
namespace et = eventide;
struct CompileUnit {
std::uint32_t path_id = 0;
@@ -30,15 +33,14 @@ struct CompileUnit {
/// stale completions without ABA risk from raw-pointer comparison.
std::uint64_t generation = 0;
std::unique_ptr<kota::cancellation_source> source =
std::make_unique<kota::cancellation_source>();
std::unique_ptr<kota::event> completion;
std::unique_ptr<et::cancellation_source> source = std::make_unique<et::cancellation_source>();
std::unique_ptr<et::event> completion;
};
class CompileGraph {
public:
/// Performs the actual compilation (e.g. produce PCM file).
using dispatch_fn = std::function<kota::task<bool>(std::uint32_t path_id)>;
using dispatch_fn = std::function<et::task<bool>(std::uint32_t path_id)>;
/// Returns the dependency path_ids for a given path_id (called lazily on first compile).
using resolve_fn = std::function<llvm::SmallVector<std::uint32_t>(std::uint32_t path_id)>;
@@ -46,11 +48,7 @@ public:
CompileGraph(dispatch_fn dispatch, resolve_fn resolve);
/// Compile a unit and all its transitive dependencies.
kota::task<bool> compile(std::uint32_t path_id);
/// Compile all transitive module dependencies of path_id, but NOT path_id itself.
/// Used for non-module files (plain .cpp) that import modules.
kota::task<bool> compile_deps(std::uint32_t path_id);
et::task<bool> compile(std::uint32_t path_id);
/// Mark path_id and all transitive dependents as dirty,
/// cancelling any in-progress compilations.
@@ -68,9 +66,7 @@ private:
void ensure_resolved(std::uint32_t path_id);
/// Internal compile with ancestor tracking for cycle detection.
kota::task<bool> compile_impl(std::uint32_t path_id,
llvm::DenseSet<std::uint32_t> ancestors,
bool dispatch_self = true);
et::task<bool> compile_impl(std::uint32_t path_id, llvm::DenseSet<std::uint32_t> ancestors);
/// Check if waiting on `target` would deadlock given our `ancestors` chain.
/// Walks the dependency graph through compiling units to see if any dep

View File

@@ -1,952 +0,0 @@
#include "server/compiler.h"
#include <format>
#include <ranges>
#include <string>
#include "command/search_config.h"
#include "index/tu_index.h"
#include "server/protocol.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "syntax/include_resolver.h"
#include "syntax/scan.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/lsp/position.h"
#include "kota/ipc/lsp/uri.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/MemoryBuffer.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/xxhash.h"
namespace clice {
namespace lsp = kota::ipc::lsp;
using serde_raw = kota::codec::RawValue;
/// Detect whether the cursor is inside a preamble directive (include/import).
Compiler::Compiler(kota::event_loop& loop,
kota::ipc::JsonPeer& peer,
Workspace& workspace,
WorkerPool& pool,
llvm::DenseMap<std::uint32_t, Session>& sessions) :
loop(loop), peer(peer), workspace(workspace), pool(pool), sessions(sessions) {}
Compiler::~Compiler() {
workspace.cancel_all();
}
void Compiler::init_compile_graph() {
if(workspace.path_to_module.empty()) {
LOG_INFO("No C++20 modules detected, skipping CompileGraph");
return;
}
// Lazy dependency resolver: scans a module file on demand to discover imports.
auto resolve = [this](std::uint32_t path_id) -> llvm::SmallVector<std::uint32_t> {
auto file_path = workspace.path_pool.resolve(path_id);
std::vector<std::string> rule_append, rule_remove;
workspace.config.match_rules(file_path, rule_append, rule_remove);
auto results = workspace.cdb.lookup(file_path,
{.query_toolchain = true,
.suppress_logging = true,
.remove = rule_remove,
.append = rule_append});
if(results.empty())
return {};
auto& cmd = results[0];
auto scan_result = scan_precise(cmd.to_argv(), cmd.resolved.directory);
llvm::SmallVector<std::uint32_t> deps;
for(auto& mod_name: scan_result.modules) {
auto mod_ids = workspace.dep_graph.lookup_module(mod_name);
if(!mod_ids.empty()) {
deps.push_back(mod_ids[0]);
}
}
// Module implementation units implicitly depend on their interface unit.
if(!scan_result.module_name.empty() && !scan_result.is_interface_unit) {
auto mod_ids = workspace.dep_graph.lookup_module(scan_result.module_name);
if(!mod_ids.empty()) {
deps.push_back(mod_ids[0]);
}
}
return deps;
};
// Dispatch: sends BuildPCM request to a stateless worker.
auto dispatch = [this](std::uint32_t path_id) -> kota::task<bool> {
auto mod_it = workspace.path_to_module.find(path_id);
if(mod_it == workspace.path_to_module.end())
co_return false;
auto file_path = std::string(workspace.path_pool.resolve(path_id));
worker::BuildParams bp;
bp.kind = worker::BuildKind::BuildPCM;
bp.file = file_path;
if(!fill_compile_args(file_path, bp.directory, bp.arguments))
co_return false;
// Compute deterministic content-addressed PCM path.
auto safe_module_name = mod_it->second;
std::ranges::replace(safe_module_name, ':', '-');
std::string hash_input = file_path;
for(auto& arg: bp.arguments) {
hash_input += arg;
}
auto args_hash = llvm::xxh3_64bits(llvm::StringRef(hash_input));
auto pcm_filename = std::format("{}-{:016x}.pcm", safe_module_name, args_hash);
auto pcm_path =
path::join(workspace.config.project.cache_dir, "cache", "pcm", pcm_filename);
// Check if cached PCM is still valid.
if(auto pcm_it = workspace.pcm_cache.find(path_id); pcm_it != workspace.pcm_cache.end()) {
if(!pcm_it->second.path.empty() && llvm::sys::fs::exists(pcm_it->second.path) &&
!deps_changed(workspace.path_pool, pcm_it->second.deps)) {
workspace.pcm_paths[path_id] = pcm_it->second.path;
co_return true;
}
}
bp.module_name = mod_it->second;
bp.output_path = pcm_path;
// Clang needs ALL transitive PCM deps, not just direct imports.
workspace.fill_pcm_deps(bp.pcms);
auto result = co_await pool.send_stateless(bp);
if(!result.has_value() || !result.value().success) {
LOG_WARN("BuildPCM failed for module {}: {}",
mod_it->second,
result.has_value() ? result.value().error : result.error().message);
co_return false;
}
workspace.pcm_paths[path_id] = result.value().output_path;
workspace.pcm_cache[path_id] = {
result.value().output_path,
capture_deps_snapshot(workspace.path_pool, result.value().deps)};
LOG_INFO("Built PCM for module {}: {}", mod_it->second, result.value().output_path);
// Persist cache metadata after successful build.
workspace.save_cache();
// Signal that new index data is available for background merge.
if(on_indexing_needed)
on_indexing_needed();
co_return true;
};
workspace.compile_graph =
std::make_unique<CompileGraph>(std::move(dispatch), std::move(resolve));
LOG_INFO("CompileGraph initialized with {} module(s)", workspace.path_to_module.size());
}
bool Compiler::fill_compile_args(llvm::StringRef path,
std::string& directory,
std::vector<std::string>& arguments,
Session* session) {
auto path_id = workspace.path_pool.intern(path);
// 1. If the session has an active header context via switchContext,
// use the host source's CDB entry with file path replaced and preamble injected.
if(session && session->active_context.has_value()) {
return fill_header_context_args(path, path_id, directory, arguments, session);
}
// 2. Normal CDB lookup for the file itself.
// Apply rules from config (append/remove flags based on file patterns).
std::vector<std::string> rule_append, rule_remove;
workspace.config.match_rules(path, rule_append, rule_remove);
CommandOptions opts{.query_toolchain = true, .remove = rule_remove, .append = rule_append};
auto results = workspace.cdb.lookup(path, opts);
if(!results.empty()) {
auto& cmd = results.front();
directory = cmd.resolved.directory.str();
arguments = cmd.to_string_argv();
return true;
}
// 3. No CDB entry — try automatic header context resolution.
return fill_header_context_args(path, path_id, directory, arguments, session);
}
bool Compiler::fill_header_context_args(llvm::StringRef path,
std::uint32_t path_id,
std::string& directory,
std::vector<std::string>& arguments,
Session* session) {
// Use cached context if available; otherwise resolve.
// If an active context override exists, invalidate cache if it points to
// a different host so we re-resolve with the correct one.
const HeaderFileContext* ctx_ptr = nullptr;
if(session && session->header_context.has_value()) {
if(session->active_context.has_value() &&
session->header_context->host_path_id != *session->active_context) {
session->header_context.reset();
} else {
ctx_ptr = &*session->header_context;
}
}
if(!ctx_ptr) {
auto resolved = resolve_header_context(path_id, session);
if(!resolved) {
LOG_WARN("No CDB entry and no header context for {}", path);
return false;
}
if(session) {
session->header_context = std::move(*resolved);
ctx_ptr = &*session->header_context;
} else {
// Background indexing path — no session to store on.
// Use a temporary (caller will use it immediately).
// Store in a local and return.
static thread_local std::optional<HeaderFileContext> tl_ctx;
tl_ctx = std::move(*resolved);
ctx_ptr = &*tl_ctx;
}
}
auto host_path = workspace.path_pool.resolve(ctx_ptr->host_path_id);
// Apply rules matching the HEADER path (what the user is editing) on top of
// the host's command — rules are expected to apply uniformly to every file.
std::vector<std::string> rule_append, rule_remove;
workspace.config.match_rules(path, rule_append, rule_remove);
auto host_results = workspace.cdb.lookup(
host_path,
{.query_toolchain = true, .remove = rule_remove, .append = rule_append});
if(host_results.empty()) {
LOG_WARN("fill_header_context_args: host {} has no CDB entry", host_path);
return false;
}
auto& host_cmd = host_results.front();
directory = host_cmd.resolved.directory.str();
// Replace source_file and inject -include preamble into flags directly.
CompileCommand header_cmd = host_cmd;
header_cmd.source_file = workspace.path_pool.resolve(path_id).data();
// Inject -include <preamble> into flags: after "-cc1" for cc1, after driver otherwise.
std::size_t inject_pos = header_cmd.resolved.is_cc1 ? 2 : 1;
header_cmd.resolved.flags.insert(header_cmd.resolved.flags.begin() + inject_pos,
ctx_ptr->preamble_path.c_str());
header_cmd.resolved.flags.insert(header_cmd.resolved.flags.begin() + inject_pos, "-include");
arguments = header_cmd.to_string_argv();
LOG_INFO("fill_compile_args: header context for {} (host={}, preamble={})",
path,
host_path,
ctx_ptr->preamble_path);
return true;
}
std::optional<HeaderFileContext> Compiler::resolve_header_context(std::uint32_t header_path_id,
Session* session) {
// Find source files that transitively include this header.
auto hosts = workspace.dep_graph.find_host_sources(header_path_id);
if(hosts.empty()) {
LOG_DEBUG("resolve_header_context: no host sources for path_id={}", header_path_id);
return std::nullopt;
}
// If there's an active context override, prefer that host.
std::uint32_t host_path_id = 0;
std::vector<std::uint32_t> chain;
if(session && session->active_context.has_value()) {
auto preferred = *session->active_context;
auto preferred_path = workspace.path_pool.resolve(preferred);
auto results = workspace.cdb.lookup(preferred_path, {.suppress_logging = true});
if(!results.empty()) {
auto c = workspace.dep_graph.find_include_chain(preferred, header_path_id);
if(!c.empty()) {
host_path_id = preferred;
chain = std::move(c);
}
}
}
// Fall back to the first available host that has a CDB entry.
if(chain.empty()) {
for(auto candidate: hosts) {
auto candidate_path = workspace.path_pool.resolve(candidate);
auto results = workspace.cdb.lookup(candidate_path, {.suppress_logging = true});
if(results.empty())
continue;
auto c = workspace.dep_graph.find_include_chain(candidate, header_path_id);
if(c.empty())
continue;
host_path_id = candidate;
chain = std::move(c);
break;
}
}
if(chain.empty()) {
LOG_DEBUG("resolve_header_context: no usable host with include chain for path_id={}",
header_path_id);
return std::nullopt;
}
// Build preamble text: for each file in the chain except the last (target),
// append all content up to (but not including) the line that includes the
// next file in the chain.
std::string preamble;
for(std::size_t i = 0; i + 1 < chain.size(); ++i) {
auto cur_id = chain[i];
auto next_id = chain[i + 1];
auto cur_path = workspace.path_pool.resolve(cur_id);
auto next_path = workspace.path_pool.resolve(next_id);
auto next_filename = llvm::sys::path::filename(next_path);
// Prefer in-memory document text over disk content.
// Use the session if this file matches the session's path, otherwise
// fall back to disk.
std::string content;
// Note: we don't have the sessions map here, so we always read from disk
// for intermediate chain files. The session parameter only covers the
// header file itself (the target), not intermediate files in the chain.
auto buf = llvm::MemoryBuffer::getFile(cur_path);
if(!buf) {
LOG_WARN("resolve_header_context: cannot read {}", cur_path);
return std::nullopt;
}
content = (*buf)->getBuffer().str();
// Scan line by line for the #include that brings in next_filename.
llvm::StringRef content_ref(content);
std::size_t line_start = 0;
std::size_t include_line_start = std::string::npos;
while(line_start <= content_ref.size()) {
auto newline_pos = content_ref.find('\n', line_start);
auto line_end =
(newline_pos == llvm::StringRef::npos) ? content_ref.size() : newline_pos;
auto line = content_ref.slice(line_start, line_end).trim();
if(line.starts_with("#include") || line.starts_with("# include")) {
// Extract the filename from the #include directive.
// Handles: #include "foo.h", #include <foo.h>, # include "foo.h"
auto quote_start = line.find_first_of("\"<");
auto quote_end = llvm::StringRef::npos;
if(quote_start != llvm::StringRef::npos) {
char close = (line[quote_start] == '"') ? '"' : '>';
quote_end = line.find(close, quote_start + 1);
}
if(quote_start != llvm::StringRef::npos && quote_end != llvm::StringRef::npos) {
auto included = line.slice(quote_start + 1, quote_end);
auto included_filename = llvm::sys::path::filename(included);
if(included_filename == next_filename) {
include_line_start = line_start;
break;
}
}
}
line_start =
(newline_pos == llvm::StringRef::npos) ? content_ref.size() + 1 : newline_pos + 1;
}
// Emit a #line marker then all content before the include line.
preamble += std::format("#line 1 \"{}\"\n", cur_path.str());
if(include_line_start != std::string::npos) {
preamble += content_ref.substr(0, include_line_start).str();
} else {
// No matching include line found — emit the whole file to be safe.
LOG_DEBUG("resolve_header_context: include line for {} not found in {}, emitting full",
next_filename,
cur_path);
preamble += content;
}
}
// Hash the preamble and write to cache directory.
auto preamble_hash = llvm::xxh3_64bits(llvm::StringRef(preamble));
auto preamble_filename = std::format("{:016x}.h", preamble_hash);
auto preamble_dir = path::join(workspace.config.project.cache_dir, "header_context");
auto preamble_path = path::join(preamble_dir, preamble_filename);
if(!llvm::sys::fs::exists(preamble_path)) {
auto ec = llvm::sys::fs::create_directories(preamble_dir);
if(ec) {
LOG_WARN("resolve_header_context: cannot create dir {}: {}",
preamble_dir,
ec.message());
return std::nullopt;
}
if(auto result = fs::write(preamble_path, preamble); !result) {
LOG_WARN("resolve_header_context: cannot write preamble {}: {}",
preamble_path,
result.error().message());
return std::nullopt;
}
LOG_INFO("resolve_header_context: wrote preamble {} for header path_id={}",
preamble_path,
header_path_id);
}
return HeaderFileContext{host_path_id, preamble_path, preamble_hash};
}
std::string uri_to_path(const std::string& uri) {
auto parsed = lsp::URI::parse(uri);
if(parsed.has_value()) {
auto path = parsed->file_path();
if(path.has_value()) {
return std::move(*path);
}
}
return uri;
}
void Compiler::publish_diagnostics(const std::string& uri,
int version,
const kota::codec::RawValue& diagnostics_json) {
std::vector<protocol::Diagnostic> diagnostics;
if(!diagnostics_json.empty()) {
auto status = kota::codec::json::from_json(diagnostics_json.data, diagnostics);
if(!status) {
LOG_WARN("Failed to deserialize diagnostics JSON for {}", uri);
}
}
protocol::PublishDiagnosticsParams params;
params.uri = uri;
params.version = version;
params.diagnostics = std::move(diagnostics);
peer.send_notification(params);
}
void Compiler::clear_diagnostics(const std::string& uri) {
protocol::PublishDiagnosticsParams params;
params.uri = uri;
params.diagnostics = {};
peer.send_notification(params);
}
kota::task<bool> Compiler::ensure_pch(Session& session,
const std::string& directory,
const std::vector<std::string>& arguments) {
auto path_id = session.path_id;
auto path = workspace.path_pool.resolve(path_id);
auto& text = session.text;
auto bound = compute_preamble_bound(text);
if(bound == 0) {
// No preamble directives — PCH would be empty. Clear any stale entry.
workspace.pch_cache.erase(path_id);
session.pch_ref.reset();
co_return true;
}
// FIXME: hash should also include compile flags that affect preprocessing
// (e.g. -D, -I, -isystem, -std) so that files with the same preamble text
// but different flags produce separate PCHs. Currently only the preamble
// text is hashed — the source file path must be excluded from the hash
// to allow sharing across files with identical preambles.
auto preamble_text = llvm::StringRef(text).substr(0, bound);
auto preamble_hash = llvm::xxh3_64bits(preamble_text);
// Deterministic content-addressed PCH path.
auto pch_path = path::join(workspace.config.project.cache_dir,
"cache",
"pch",
std::format("{:016x}.pch", preamble_hash));
// Reuse existing PCH if preamble content and deps haven't changed.
if(auto it = workspace.pch_cache.find(path_id); it != workspace.pch_cache.end()) {
auto& st = it->second;
if(st.hash == preamble_hash && !st.path.empty() &&
!deps_changed(workspace.path_pool, st.deps)) {
st.bound = bound;
session.pch_ref = Session::PCHRef{path_id, preamble_hash, bound};
co_return true;
}
}
// Preamble incomplete (user still typing) — defer rebuild, reuse old PCH if available.
if(!is_preamble_complete(text, bound)) {
LOG_DEBUG("Preamble incomplete for {}, deferring PCH rebuild", path);
co_return workspace.pch_cache.count(path_id) && !workspace.pch_cache[path_id].path.empty();
}
// If another coroutine is already building PCH for this file, wait for it.
if(auto it = workspace.pch_cache.find(path_id);
it != workspace.pch_cache.end() && it->second.building) {
co_await it->second.building->wait();
if(auto it2 = workspace.pch_cache.find(path_id); it2 != workspace.pch_cache.end()) {
session.pch_ref = Session::PCHRef{path_id, it2->second.hash, it2->second.bound};
}
co_return workspace.pch_cache.count(path_id) && !workspace.pch_cache[path_id].path.empty();
}
// Register in-flight build so concurrent requests wait on us.
auto completion = std::make_shared<kota::event>();
workspace.pch_cache[path_id].building = completion;
if(workspace.config.project.cache_dir.empty()) {
LOG_WARN("PCH build skipped: cache_dir is not configured");
workspace.pch_cache[path_id].building.reset();
completion->set();
co_return false;
}
// Ensure the PCH cache directory exists.
auto pch_dir = path::join(workspace.config.project.cache_dir, "cache", "pch");
if(auto ec = llvm::sys::fs::create_directories(pch_dir)) {
LOG_WARN("Cannot create PCH cache dir {}: {}", pch_dir, ec.message());
workspace.pch_cache[path_id].building.reset();
completion->set();
co_return false;
}
// Build a new PCH via stateless worker.
worker::BuildParams bp;
bp.kind = worker::BuildKind::BuildPCH;
bp.file = std::string(path);
bp.directory = directory;
bp.arguments = arguments;
bp.text = text;
bp.preamble_bound = bound;
bp.output_path = pch_path;
LOG_DEBUG("Building PCH for {}, bound={}, output={}", path, bound, pch_path);
auto result = co_await pool.send_stateless(bp);
if(!result.has_value() || !result.value().success) {
LOG_WARN("PCH build failed for {}: {}",
path,
result.has_value() ? result.value().error : result.error().message);
workspace.pch_cache[path_id].building.reset();
completion->set();
co_return false;
}
auto& st = workspace.pch_cache[path_id];
st.path = result.value().output_path;
st.bound = bound;
st.hash = preamble_hash;
st.deps = capture_deps_snapshot(workspace.path_pool, result.value().deps);
st.document_links_json = std::move(result.value().pch_links_json);
st.building.reset();
session.pch_ref = Session::PCHRef{path_id, preamble_hash, bound};
LOG_INFO("PCH built for {}: {}", path, result.value().output_path);
// Persist cache metadata after successful build.
workspace.save_cache();
completion->set();
co_return true;
}
/// Compile module dependencies, build/reuse PCH, and fill PCM paths.
/// Shared preparation step used by both ensure_compiled() (stateful path)
/// and forward_stateless() (completion/signatureHelp path).
kota::task<bool> Compiler::ensure_deps(Session& session,
const std::string& directory,
const std::vector<std::string>& arguments,
std::pair<std::string, uint32_t>& pch,
std::unordered_map<std::string, std::string>& pcms) {
auto path_id = session.path_id;
// Compile C++20 module dependencies (PCMs).
if(workspace.compile_graph && !co_await workspace.compile_graph->compile_deps(path_id)) {
co_return false;
}
// Scan buffer text for module imports that might not be in compile_graph yet.
// When a user adds `import std;` without saving, the compile_graph (disk-based)
// doesn't know about the new dependency. Scan the in-memory text to find them.
{
auto scan_result = scan(session.text);
for(auto& mod_name: scan_result.modules) {
if(mod_name.empty())
continue;
bool found = false;
for(auto& [pid, name]: workspace.path_to_module) {
if(name == mod_name) {
// If PCM not already built, try to build it.
if(workspace.pcm_paths.find(pid) == workspace.pcm_paths.end()) {
if(workspace.compile_graph && workspace.compile_graph->has_unit(pid)) {
co_await workspace.compile_graph->compile_deps(pid);
}
}
found = true;
break;
}
}
if(!found) {
LOG_DEBUG("Buffer imports unknown module '{}', skipping", mod_name);
}
}
}
// Build or reuse PCH.
auto pch_ok = co_await ensure_pch(session, directory, arguments);
if(pch_ok) {
if(auto pch_it = workspace.pch_cache.find(path_id); pch_it != workspace.pch_cache.end()) {
pch = {pch_it->second.path, pch_it->second.bound};
}
}
// Fill all available PCM paths, excluding the file's own PCM
// to avoid "multiple module declarations".
workspace.fill_pcm_deps(pcms, path_id);
co_return true;
}
bool Compiler::is_stale(const Session& session) {
if(session.ast_deps.has_value() && deps_changed(workspace.path_pool, *session.ast_deps))
return true;
// Check PCH staleness via the session's pch_ref.
if(session.pch_ref.has_value()) {
auto pch_it = workspace.pch_cache.find(session.pch_ref->path_id);
if(pch_it != workspace.pch_cache.end() &&
deps_changed(workspace.path_pool, pch_it->second.deps))
return true;
}
return false;
}
void Compiler::record_deps(Session& session, llvm::ArrayRef<std::string> deps) {
session.ast_deps = capture_deps_snapshot(workspace.path_pool, deps);
}
/// Pull-based compilation entry point for user-opened files.
///
/// Called lazily by forward_query() / forward_build() before every
/// feature request (hover, semantic tokens, etc.). Guarantees that when it
/// returns true the stateful worker assigned to `path_id` holds an up-to-date
/// AST and diagnostics have been published to the client.
///
/// Lifecycle overview (pull-based model):
///
/// didOpen / didChange only update Session, mark ast_dirty
/// didSave mark dependents dirty, queue indexing
/// feature request arrives calls ensure_compiled() first
/// 1. Fast-path exit if AST is already clean (!ast_dirty).
/// 2. Compile any C++20 module dependencies (PCMs) via CompileGraph.
/// 3. Build / reuse the precompiled header (PCH) via ensure_pch().
/// 4. Send CompileParams to the stateful worker, which builds the AST.
/// 5. On success: publish diagnostics, clear ast_dirty, schedule indexing.
/// 6. On generation mismatch (user edited during compile): keep dirty,
/// the next feature request will trigger another compile cycle.
///
/// Only the opened file itself is remapped (its in-memory text is sent to the
/// worker); every other file is read from disk by the compiler.
///
/// Concurrency: multiple concurrent feature requests for the same file will
/// each call ensure_compiled(). The first one launches a detached compile
/// task via loop.schedule(); subsequent ones wait on the shared event.
/// The detached task cannot be cancelled by LSP $/cancelRequest, preventing
/// the race where cancellation wakes all waiters and they all start compiles.
kota::task<bool> Compiler::ensure_compiled(Session& session) {
auto path_id = session.path_id;
LOG_DEBUG("ensure_compiled: path_id={} version={} gen={} ast_dirty={}",
path_id,
session.version,
session.generation,
session.ast_dirty);
if(!session.ast_dirty) {
if(!is_stale(session)) {
co_return true;
}
session.ast_dirty = true;
}
// If another compile is already in flight, wait for it.
// This co_await may be cancelled by LSP $/cancelRequest — that's fine,
// it just means this particular feature request is abandoned. The
// detached compile task keeps running independently.
while(session.compiling) {
auto pending = session.compiling;
co_await pending->done.wait();
if(!session.ast_dirty)
co_return true;
}
// No compile in flight and AST is dirty — launch a detached compile task.
// The detached task is scheduled via loop.schedule() so it is NOT subject
// to LSP $/cancelRequest cancellation. This eliminates the race where
// cancellation fires the RAII guard, waking all waiters simultaneously
// and causing them all to start new compiles.
auto pending_compile = std::make_shared<Session::PendingCompile>();
session.compiling = pending_compile;
LOG_INFO("ensure_compiled: launching detached compile path_id={} gen={}",
path_id,
session.generation);
// Capture path_id by value so the detached lambda can re-lookup the session
// from the sessions map after co_await (DenseMap may invalidate pointers).
loop.schedule([](Compiler* self,
std::uint32_t pid,
std::shared_ptr<Session::PendingCompile> pc) -> kota::task<> {
// Re-lookup session from the sessions map (pointer may have been
// invalidated by DenseMap growth during co_await).
auto find_session = [&]() -> Session* {
auto it = self->sessions.find(pid);
return it != self->sessions.end() ? &it->second : nullptr;
};
auto* sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
auto finish_compile = [&]() {
auto* s = find_session();
if(s && s->compiling == pc) {
s->compiling.reset();
}
LOG_INFO("ensure_compiled: finish_compile (detached) path_id={}", pid);
pc->done.set();
};
auto gen = sess->generation;
LOG_INFO("ensure_compiled: starting compile (detached) path_id={} gen={}", pid, gen);
auto file_path = std::string(self->workspace.path_pool.resolve(pid));
auto uri = lsp::URI::from_file_path(file_path);
std::string uri_str = uri.has_value() ? uri->str() : file_path;
worker::CompileParams params;
params.path = file_path;
params.version = sess->version;
params.text = sess->text;
if(!self->fill_compile_args(file_path, params.directory, params.arguments, sess)) {
finish_compile();
co_return;
}
if(!co_await self
->ensure_deps(*sess, params.directory, params.arguments, params.pch, params.pcms)) {
LOG_WARN("Dependency preparation failed for {}, skipping compile", uri_str);
finish_compile();
co_return;
}
// Re-lookup after co_await (DenseMap may have grown).
sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
auto result = co_await self->pool.send_stateful(pid, params);
// Re-lookup after co_await.
sess = find_session();
if(!sess) {
pc->done.set();
co_return;
}
if(sess->generation != gen) {
LOG_INFO("ensure_compiled: generation mismatch ({} vs {}) for {}",
sess->generation,
gen,
uri_str);
finish_compile();
co_return;
}
if(!result.has_value()) {
LOG_WARN("Compile failed for {}: {}", uri_str, result.error().message);
self->clear_diagnostics(uri_str);
finish_compile();
co_return;
}
sess->ast_dirty = false;
pc->succeeded = true;
self->record_deps(*sess, result.value().deps);
// Store open file index from the stateful worker's TUIndex.
if(!result.value().tu_index_data.empty()) {
auto tu_index = index::TUIndex::from(result.value().tu_index_data.data());
OpenFileIndex ofi;
ofi.file_index = std::move(tu_index.main_file_index);
ofi.symbols = std::move(tu_index.symbols);
ofi.content = sess->text;
ofi.mapper.emplace(ofi.content, lsp::PositionEncoding::UTF16);
sess->file_index = std::move(ofi);
}
auto version = sess->version;
finish_compile();
// Publish diagnostics AFTER marking compile as done, so that concurrent
// forward_query() calls can proceed immediately.
self->publish_diagnostics(uri_str, version, result.value().diagnostics);
if(self->on_indexing_needed)
self->on_indexing_needed();
}(this, path_id, pending_compile));
// Wait for the detached compile to finish. If this wait is cancelled
// by LSP $/cancelRequest, the detached task continues unaffected.
co_await pending_compile->done.wait();
co_return !session.ast_dirty;
}
Compiler::RawResult Compiler::forward_query(worker::QueryKind kind,
Session& session,
std::optional<protocol::Position> position,
std::optional<protocol::Range> range) {
auto path_id = session.path_id;
auto path = std::string(workspace.path_pool.resolve(path_id));
// Cache text before co_await — session reference may dangle if didClose
// erases the entry from the sessions map during suspension.
auto text = session.text;
if(!co_await ensure_compiled(session)) {
co_return serde_raw{"null"};
}
auto sit = sessions.find(path_id);
if(sit == sessions.end() || sit->second.ast_dirty) {
co_return serde_raw{"null"};
}
worker::QueryParams wp;
wp.kind = kind;
wp.path = path;
lsp::PositionMapper mapper(text, lsp::PositionEncoding::UTF16);
if(position) {
auto offset = mapper.to_offset(*position);
if(!offset)
co_return serde_raw{"null"};
wp.offset = *offset;
}
if(range) {
auto start = mapper.to_offset(range->start);
auto end = mapper.to_offset(range->end);
if(start && end) {
wp.range = {*start, *end};
}
}
auto result = co_await pool.send_stateful(path_id, wp);
if(!result.has_value()) {
co_return serde_raw{};
}
co_return std::move(result.value());
}
Compiler::RawResult Compiler::forward_build(worker::BuildKind kind,
const protocol::Position& position,
Session& session) {
auto path_id = session.path_id;
auto path = std::string(workspace.path_pool.resolve(path_id));
worker::BuildParams wp;
wp.kind = kind;
wp.file = path;
// Cache session fields before co_await — session reference may dangle
// if didClose erases the entry from the sessions map during suspension.
wp.version = session.version;
wp.text = session.text;
if(!fill_compile_args(path, wp.directory, wp.arguments, &session)) {
co_return serde_raw{};
}
if(!co_await ensure_deps(session, wp.directory, wp.arguments, wp.pch, wp.pcms)) {
co_return serde_raw{};
}
// After co_await, verify session still exists.
if(sessions.find(path_id) == sessions.end()) {
co_return serde_raw{};
}
lsp::PositionMapper mapper(wp.text, lsp::PositionEncoding::UTF16);
auto offset = mapper.to_offset(position);
if(!offset)
co_return serde_raw{"null"};
wp.offset = *offset;
auto result = co_await pool.send_stateless(wp);
if(!result.has_value()) {
co_return serde_raw{};
}
co_return std::move(result.value().result_json);
}
Compiler::RawResult Compiler::handle_completion(const protocol::Position& position,
Session& session) {
auto path_id = session.path_id;
auto path = std::string(workspace.path_pool.resolve(path_id));
lsp::PositionMapper mapper(session.text, lsp::PositionEncoding::UTF16);
auto offset = mapper.to_offset(position);
if(offset) {
auto pctx = detect_completion_context(session.text, *offset);
if(pctx.kind == CompletionContext::IncludeQuoted ||
pctx.kind == CompletionContext::IncludeAngled) {
std::string directory;
std::vector<std::string> arguments;
if(!fill_compile_args(path, directory, arguments))
co_return serde_raw{"[]"};
std::vector<const char*> args_ptrs;
args_ptrs.reserve(arguments.size());
for(auto& arg: arguments)
args_ptrs.push_back(arg.c_str());
auto search_config = extract_search_config(args_ptrs, directory);
DirListingCache dir_cache;
auto resolved = resolve_search_config(search_config, dir_cache);
bool angled = (pctx.kind == CompletionContext::IncludeAngled);
auto candidates = complete_include_path(resolved, pctx.prefix, angled, dir_cache);
std::vector<protocol::CompletionItem> items;
items.reserve(candidates.size());
for(auto& c: candidates) {
protocol::CompletionItem item;
item.label = c.is_directory ? c.name + "/" : c.name;
item.kind = protocol::CompletionItemKind::File;
items.push_back(std::move(item));
}
auto json = kota::codec::json::to_json<kota::ipc::lsp_config>(items);
co_return serde_raw{json ? std::move(*json) : "[]"};
}
if(pctx.kind == CompletionContext::Import) {
auto module_names = complete_module_import(workspace.path_to_module, pctx.prefix);
std::vector<protocol::CompletionItem> items;
items.reserve(module_names.size());
for(auto& name: module_names) {
protocol::CompletionItem item;
item.label = name;
item.kind = protocol::CompletionItemKind::Module;
item.insert_text = name + ";";
items.push_back(std::move(item));
}
auto json = kota::codec::json::to_json<kota::ipc::lsp_config>(items);
co_return serde_raw{json ? std::move(*json) : "[]"};
}
}
co_return co_await forward_build(worker::BuildKind::Completion, position, session);
}
} // namespace clice

View File

@@ -1,134 +0,0 @@
#pragma once
#include <cstdint>
#include <functional>
#include <optional>
#include <string>
#include <unordered_map>
#include <vector>
#include "command/command.h"
#include "server/session.h"
#include "server/worker_pool.h"
#include "server/workspace.h"
#include "syntax/completion.h"
#include "kota/async/async.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/peer.h"
#include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/SmallVector.h"
#include "llvm/ADT/StringRef.h"
namespace clice {
namespace protocol = kota::ipc::protocol;
/// Convert a file:// URI to a local file path.
std::string uri_to_path(const std::string& uri);
/// Compilation service — drives worker processes to build ASTs, PCHs, and PCMs.
///
/// Compiler holds no persistent state of its own. All project-wide data
/// lives in Workspace; per-file data lives in Session. Compiler reads from
/// both and writes compilation results back to Session (file_index, pch_ref,
/// ast_deps, diagnostics).
///
/// Responsibilities:
/// - AST compilation lifecycle (ensure_compiled → ensure_pch → ensure_deps)
/// - Feature request forwarding to stateful/stateless workers
/// - Compile argument resolution (CDB lookup + header context fallback)
/// - Compile graph initialization (module DAG setup)
///
/// NOT responsible for:
/// - Document lifecycle (didOpen/didChange/didClose) — handled by MasterServer
/// - Index queries — handled by Indexer
/// - Background indexing scheduling — handled by Indexer
class Compiler {
public:
Compiler(kota::event_loop& loop,
kota::ipc::JsonPeer& peer,
Workspace& workspace,
WorkerPool& pool,
llvm::DenseMap<std::uint32_t, Session>& sessions);
~Compiler();
void init_compile_graph();
/// Fill compile arguments for a file (CDB lookup + header context fallback).
/// @param session If non-null, used for header context resolution on open files.
bool fill_compile_args(llvm::StringRef path,
std::string& directory,
std::vector<std::string>& arguments,
Session* session = nullptr);
/// Compile an open file's AST if dirty. On success, updates session's
/// file_index, pch_ref, ast_deps, and publishes diagnostics.
kota::task<bool> ensure_compiled(Session& session);
using RawResult = kota::task<kota::codec::RawValue, kota::ipc::Error>;
/// Forward a query to the stateful worker that holds this file's AST.
/// Ensures compilation first. For position-sensitive queries (hover,
/// goto-definition), pass a Position. For range-sensitive queries
/// (inlay hints), pass a Range.
RawResult forward_query(worker::QueryKind kind,
Session& session,
std::optional<protocol::Position> position = {},
std::optional<protocol::Range> range = {});
/// Forward a build request (signature help, etc.) to a stateless worker.
/// Sends the full buffer content and compile arguments.
RawResult forward_build(worker::BuildKind kind,
const protocol::Position& position,
Session& session);
/// Handle completion requests. Detects preamble context (include/import)
/// and serves those locally; delegates code completion to a stateless worker.
RawResult handle_completion(const protocol::Position& position, Session& session);
/// Send an empty diagnostics notification to clear stale markers in the editor.
void clear_diagnostics(const std::string& uri);
/// Callback invoked when indexing should be scheduled.
std::function<void()> on_indexing_needed;
private:
kota::task<bool> ensure_deps(Session& session,
const std::string& directory,
const std::vector<std::string>& arguments,
std::pair<std::string, uint32_t>& pch,
std::unordered_map<std::string, std::string>& pcms);
kota::task<bool> ensure_pch(Session& session,
const std::string& directory,
const std::vector<std::string>& arguments);
bool is_stale(const Session& session);
void record_deps(Session& session, llvm::ArrayRef<std::string> deps);
void publish_diagnostics(const std::string& uri,
int version,
const kota::codec::RawValue& diags);
std::optional<HeaderFileContext> resolve_header_context(std::uint32_t header_path_id,
Session* session);
bool fill_header_context_args(llvm::StringRef path,
std::uint32_t path_id,
std::string& directory,
std::vector<std::string>& arguments,
Session* session);
private:
kota::event_loop& loop;
kota::ipc::JsonPeer& peer;
Workspace& workspace;
WorkerPool& pool;
llvm::DenseMap<std::uint32_t, Session>& sessions;
};
} // namespace clice

View File

@@ -1,191 +1,93 @@
#include "server/config.h"
#include <algorithm>
#include <thread>
#include "eventide/serde/toml.h"
#include "support/filesystem.h"
#include "support/glob_pattern.h"
#include "support/logging.h"
#include "kota/codec/json/json.h"
#include "kota/codec/toml/toml.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/Process.h"
#include "llvm/Support/xxhash.h"
namespace clice {
/// Replace all occurrences of ${workspace} with the workspace root.
/// No-op when workspace_root is empty, to avoid producing paths like "/cache"
/// from "${workspace}/cache".
static void substitute_workspace(std::string& value, llvm::StringRef workspace_root) {
if(workspace_root.empty())
return;
static void substitute_workspace(std::string& value, const std::string& workspace_root) {
constexpr std::string_view placeholder = "${workspace}";
std::size_t pos = 0;
std::string::size_type pos = 0;
while((pos = value.find(placeholder, pos)) != std::string::npos) {
value.replace(pos, placeholder.size(), workspace_root);
pos += workspace_root.size();
}
}
/// Try to resolve the default cache directory using XDG_CACHE_HOME.
/// Returns empty string on failure.
static std::string resolve_xdg_cache_dir(llvm::StringRef workspace_root) {
// Determine base: $XDG_CACHE_HOME or ~/.cache
std::string base;
if(auto xdg = llvm::sys::Process::GetEnv("XDG_CACHE_HOME"); xdg && !xdg->empty()) {
base = std::move(*xdg);
} else if(auto home = llvm::sys::Process::GetEnv("HOME"); home && !home->empty()) {
base = path::join(*home, ".cache");
} else {
return {};
void CliceConfig::apply_defaults(const std::string& workspace_root) {
auto cpu_count = std::thread::hardware_concurrency();
if(cpu_count == 0)
cpu_count = 4;
if(stateful_worker_count == 0) {
stateful_worker_count = std::max(1u, cpu_count / 4);
}
if(stateless_worker_count == 0) {
stateless_worker_count = std::max(1u, cpu_count / 4);
}
if(worker_memory_limit == 0) {
worker_memory_limit = 4ULL * 1024 * 1024 * 1024; // 4GB default
}
if(cache_dir.empty() && !workspace_root.empty()) {
cache_dir = path::join(workspace_root, ".clice");
}
// Use a hash of workspace_root to create a unique subdirectory.
auto hash = llvm::xxh3_64bits(workspace_root);
auto dir = path::join(base, "clice", std::format("{:016x}", hash));
if(auto ec = llvm::sys::fs::create_directories(dir)) {
LOG_WARN("Failed to create XDG cache directory {}: {}", dir, ec.message());
return {};
if(index_dir.empty() && !cache_dir.empty()) {
index_dir = path::join(cache_dir, "index");
}
return dir;
// Apply variable substitution to string fields
substitute_workspace(compile_commands_path, workspace_root);
substitute_workspace(cache_dir, workspace_root);
substitute_workspace(index_dir, workspace_root);
}
void Config::apply_defaults(llvm::StringRef workspace_root) {
auto& p = project;
if(p.max_active_file == 0)
p.max_active_file = 8;
if(!p.enable_indexing)
p.enable_indexing = true;
if(!p.idle_timeout_ms)
p.idle_timeout_ms = 3000;
if(p.stateful_worker_count == 0)
p.stateful_worker_count = 2;
if(p.stateless_worker_count == 0)
p.stateless_worker_count = 3;
if(p.worker_memory_limit == 0)
p.worker_memory_limit = 4ULL * 1024 * 1024 * 1024; // 4GB
if(p.cache_dir.empty() && !workspace_root.empty()) {
p.cache_dir = resolve_xdg_cache_dir(workspace_root);
if(p.cache_dir.empty())
p.cache_dir = path::join(workspace_root, ".clice");
}
if(p.index_dir.empty() && !p.cache_dir.empty())
p.index_dir = path::join(p.cache_dir, "index");
if(p.logging_dir.empty() && !p.cache_dir.empty())
p.logging_dir = path::join(p.cache_dir, "logs");
// Variable substitution on string fields.
substitute_workspace(p.cache_dir, workspace_root);
substitute_workspace(p.index_dir, workspace_root);
substitute_workspace(p.logging_dir, workspace_root);
for(auto& entry: p.compile_commands_paths)
substitute_workspace(entry, workspace_root);
// Pre-compile glob patterns from rules.
compiled_rules.clear();
for(auto& rule: rules) {
CompiledRule compiled;
for(auto& pattern_str: rule.patterns) {
auto pat = GlobPattern::create(pattern_str);
if(!pat) {
LOG_WARN("Invalid glob pattern in rule: {}", pattern_str);
continue;
}
compiled.patterns.push_back(std::move(*pat));
}
// Drop the whole rule if no pattern compiled successfully — otherwise the
// append/remove flags would be silently attached to a rule that can never match.
if(compiled.patterns.empty()) {
if(!rule.patterns.empty())
LOG_WARN("Rule dropped: all glob patterns failed to compile");
continue;
}
compiled.append.assign(rule.append.begin(), rule.append.end());
compiled.remove.assign(rule.remove.begin(), rule.remove.end());
compiled_rules.push_back(std::move(compiled));
}
}
void Config::match_rules(llvm::StringRef file_path,
std::vector<std::string>& append,
std::vector<std::string>& remove) const {
// Rules are processed in declaration order so that a later rule can
// override an earlier one. Specifically, when a later rule removes
// an argument, we also strip any string-equal entry already added
// to `append` by an earlier matching rule — otherwise the append
// would silently survive (lookup applies removes to the base flags
// only, not to entries contributed via `append`).
for(auto& rule: compiled_rules) {
bool matched =
std::ranges::any_of(rule.patterns, [&](auto& pat) { return pat.match(file_path); });
if(!matched)
continue;
for(auto& r: rule.remove) {
std::erase(append, r);
remove.push_back(r);
}
append.insert(append.end(), rule.append.begin(), rule.append.end());
}
}
std::optional<Config> Config::load(llvm::StringRef path, llvm::StringRef workspace_root) {
std::optional<CliceConfig> CliceConfig::load(const std::string& path,
const std::string& workspace_root) {
auto content = fs::read(path);
if(!content)
if(!content) {
return std::nullopt;
}
auto result = kota::codec::toml::parse<Config>(*content);
auto result = eventide::serde::toml::parse<CliceConfig>(*content);
if(!result) {
LOG_ERROR("Invalid clice.toml {}: {}", path, result.error().to_string());
LOG_WARN("Failed to parse config file {}", path);
return std::nullopt;
}
auto config = std::move(*result);
config.apply_defaults(workspace_root);
LOG_INFO("Loaded config from {}", path);
return config;
}
std::optional<Config> Config::load_from_json(llvm::StringRef json, llvm::StringRef workspace_root) {
auto result = kota::codec::json::from_json<Config>(json);
if(!result) {
LOG_WARN("Failed to parse initializationOptions JSON: {}", result.error().message());
return std::nullopt;
}
auto config = std::move(*result);
config.apply_defaults(workspace_root);
LOG_INFO("Loaded config from initializationOptions");
return config;
}
Config Config::load_from_workspace(llvm::StringRef workspace_root) {
CliceConfig CliceConfig::load_from_workspace(const std::string& workspace_root) {
if(!workspace_root.empty()) {
// Try standard config file locations
for(auto* name: {"clice.toml", ".clice/config.toml"}) {
auto config_path = path::join(workspace_root, name);
if(!llvm::sys::fs::exists(config_path))
continue;
if(auto config = load(config_path, workspace_root))
return std::move(*config);
// Present but malformed: fall through to defaults, but surface
// the situation clearly so users know their config wasn't applied.
LOG_WARN("Falling back to default configuration because {} is invalid", config_path);
if(llvm::sys::fs::exists(config_path)) {
auto config = load(config_path, workspace_root);
if(config)
return std::move(*config);
}
}
}
Config config;
// No config file found; use defaults
CliceConfig config;
config.apply_defaults(workspace_root);
LOG_INFO(
"No clice.toml found, using default configuration " "(stateful={}, stateless={}, memory_limit={}MB)",
config.project.stateful_worker_count.value,
config.project.stateless_worker_count.value,
config.project.worker_memory_limit.value / (1024 * 1024));
config.stateful_worker_count,
config.stateless_worker_count,
config.worker_memory_limit / (1024 * 1024));
return config;
}

View File

@@ -3,77 +3,44 @@
#include <cstdint>
#include <optional>
#include <string>
#include <vector>
#include "support/glob_pattern.h"
#include "kota/meta/annotation.h"
#include "llvm/ADT/StringRef.h"
namespace clice {
using kota::meta::defaulted;
/// Configuration for the clice LSP server, loadable from clice.toml.
struct CliceConfig {
// Worker configuration (0 = auto-detect from system resources)
std::uint32_t stateful_worker_count = 0;
std::uint32_t stateless_worker_count = 0;
std::uint64_t worker_memory_limit = 0; // bytes; 0 = auto
/// A file-pattern rule that appends/removes compilation flags.
/// Corresponds to `[[rules]]` in clice.toml.
struct ConfigRule {
defaulted<std::vector<std::string>> patterns;
defaulted<std::vector<std::string>> append;
defaulted<std::vector<std::string>> remove;
};
// Compilation database path (empty = auto-detect)
std::string compile_commands_path;
/// Corresponds to the `[project]` section in clice.toml.
struct ProjectConfig {
defaulted<bool> clang_tidy = {};
defaulted<int> max_active_file = {};
// Cache directory (empty = default: <workspace>/.clice/)
std::string cache_dir;
defaulted<std::string> cache_dir;
defaulted<std::string> index_dir;
defaulted<std::string> logging_dir;
// Index storage directory (default: <cache_dir>/index/)
std::string index_dir;
defaulted<std::vector<std::string>> compile_commands_paths;
// Debounce interval for re-compilation after edits (milliseconds)
int debounce_ms = 200;
std::optional<bool> enable_indexing;
std::optional<int> idle_timeout_ms;
defaulted<std::uint32_t> stateful_worker_count = {};
defaulted<std::uint32_t> stateless_worker_count = {};
defaulted<std::uint64_t> worker_memory_limit = {};
};
struct CompiledRule {
std::vector<GlobPattern> patterns;
std::vector<std::string> append;
std::vector<std::string> remove;
};
/// Configuration for the clice LSP server, loadable from clice.toml
/// or passed via LSP initializationOptions.
struct Config {
defaulted<ProjectConfig> project;
defaulted<std::vector<ConfigRule>> rules;
kota::meta::annotation<std::vector<CompiledRule>, kota::meta::attrs::skip> compiled_rules;
// Background indexing
bool enable_indexing = true;
int idle_timeout_ms = 3000;
/// Compute default values for any field left at its zero/empty sentinel.
void apply_defaults(llvm::StringRef workspace_root);
/// Collect append/remove flags from all rules whose patterns match `path`.
void match_rules(llvm::StringRef path,
std::vector<std::string>& append,
std::vector<std::string>& remove) const;
void apply_defaults(const std::string& workspace_root);
/// Try to load configuration from a TOML file.
static std::optional<Config> load(llvm::StringRef path, llvm::StringRef workspace_root);
/// Try to load configuration from a JSON string (e.g. initializationOptions).
static std::optional<Config> load_from_json(llvm::StringRef json,
llvm::StringRef workspace_root);
/// Performs ${workspace} variable substitution in string fields.
/// Returns std::nullopt if the file does not exist or cannot be parsed.
static std::optional<CliceConfig> load(const std::string& path,
const std::string& workspace_root);
/// Load config from the workspace, trying standard locations.
/// Returns a default config (with apply_defaults) if no file is found.
static Config load_from_workspace(llvm::StringRef workspace_root);
static CliceConfig load_from_workspace(const std::string& workspace_root);
};
} // namespace clice

View File

@@ -1,696 +0,0 @@
#include "server/indexer.h"
#include <string>
#include <variant>
#include <vector>
#include "index/tu_index.h"
#include "server/compiler.h"
#include "server/protocol.h"
#include "server/session.h"
#include "server/worker_pool.h"
#include "support/filesystem.h"
#include "support/logging.h"
#include "kota/ipc/lsp/position.h"
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/lsp/uri.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/MemoryBuffer.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/raw_ostream.h"
namespace clice {
namespace lsp = kota::ipc::lsp;
void Indexer::merge(const void* tu_index_data, std::size_t size) {
auto tu_index = index::TUIndex::from(tu_index_data);
if(tu_index.graph.paths.empty()) {
LOG_WARN("Ignoring TUIndex with empty path graph");
return;
}
auto file_ids_map = workspace.project_index.merge(tu_index);
auto main_tu_path_id = static_cast<std::uint32_t>(tu_index.graph.paths.size() - 1);
auto merge_file_index = [&](std::uint32_t tu_path_id, index::FileIndex& file_idx) {
auto global_path_id = file_ids_map[tu_path_id];
auto& shard = workspace.merged_indices[global_path_id];
if(tu_path_id == main_tu_path_id) {
std::vector<index::IncludeLocation> include_locs;
for(auto& loc: tu_index.graph.locations) {
index::IncludeLocation remapped = loc;
remapped.path_id = file_ids_map[loc.path_id];
include_locs.push_back(remapped);
}
auto file_path = workspace.project_index.path_pool.path(global_path_id);
llvm::StringRef file_content;
std::string file_content_storage;
auto buf = llvm::MemoryBuffer::getFile(file_path);
if(buf) {
file_content_storage = (*buf)->getBuffer().str();
file_content = file_content_storage;
}
shard.index.merge(global_path_id,
tu_index.built_at,
std::move(include_locs),
file_idx,
file_content);
} else {
std::optional<std::uint32_t> include_id;
for(std::uint32_t i = 0; i < tu_index.graph.locations.size(); ++i) {
if(tu_index.graph.locations[i].path_id == tu_path_id) {
include_id = i;
break;
}
}
if(!include_id) {
LOG_WARN("Skip merge for path {}: include location not found", global_path_id);
return;
}
auto header_path = workspace.project_index.path_pool.path(global_path_id);
llvm::StringRef header_content;
std::string header_content_storage;
auto header_buf = llvm::MemoryBuffer::getFile(header_path);
if(header_buf) {
header_content_storage = (*header_buf)->getBuffer().str();
header_content = header_content_storage;
}
shard.index.merge(global_path_id, *include_id, file_idx, header_content);
}
shard.invalidate_mapper();
};
for(auto& [tu_path_id, file_idx]: tu_index.path_file_indices) {
merge_file_index(tu_path_id, file_idx);
}
merge_file_index(main_tu_path_id, tu_index.main_file_index);
LOG_INFO("Merged TUIndex: {} paths, {} symbols, {} merged_shards",
tu_index.graph.paths.size(),
tu_index.symbols.size(),
workspace.merged_indices.size());
}
void Indexer::save(llvm::StringRef index_dir) {
if(index_dir.empty())
return;
auto ec = llvm::sys::fs::create_directories(index_dir);
if(ec) {
LOG_WARN("Failed to create index directory {}: {}", std::string(index_dir), ec.message());
return;
}
auto project_path = path::join(index_dir, "project.idx");
{
std::error_code write_ec;
llvm::raw_fd_ostream os(project_path, write_ec);
if(!write_ec) {
workspace.project_index.serialize(os);
LOG_INFO("Saved ProjectIndex to {}", project_path);
} else {
LOG_WARN("Failed to save ProjectIndex: {}", write_ec.message());
}
}
auto shards_dir = path::join(index_dir, "shards");
ec = llvm::sys::fs::create_directories(shards_dir);
if(ec) {
LOG_WARN("Failed to create shards directory: {}", ec.message());
return;
}
std::size_t saved = 0;
for(auto& [path_id, shard]: workspace.merged_indices) {
if(!shard.index.need_rewrite())
continue;
auto shard_path = path::join(shards_dir, std::to_string(path_id) + ".idx");
std::error_code write_ec;
llvm::raw_fd_ostream os(shard_path, write_ec);
if(!write_ec) {
shard.index.serialize(os);
++saved;
}
}
LOG_INFO("Saved {} MergedIndex shards (of {} total)", saved, workspace.merged_indices.size());
}
void Indexer::load(llvm::StringRef index_dir) {
if(index_dir.empty())
return;
auto project_path = path::join(index_dir, "project.idx");
auto buf = llvm::MemoryBuffer::getFile(project_path);
if(buf) {
workspace.project_index = index::ProjectIndex::from((*buf)->getBufferStart());
LOG_INFO("Loaded ProjectIndex: {} symbols", workspace.project_index.symbols.size());
}
auto shards_dir = path::join(index_dir, "shards");
std::error_code ec;
for(auto it = llvm::sys::fs::directory_iterator(shards_dir, ec);
!ec && it != llvm::sys::fs::directory_iterator();
it.increment(ec)) {
auto filename = llvm::sys::path::filename(it->path());
if(!filename.ends_with(".idx"))
continue;
auto stem = filename.drop_back(4);
std::uint32_t path_id = 0;
if(stem.getAsInteger(10, path_id))
continue;
workspace.merged_indices[path_id] = MergedIndexShard{index::MergedIndex::load(it->path())};
}
if(!workspace.merged_indices.empty()) {
LOG_INFO("Loaded {} MergedIndex shards", workspace.merged_indices.size());
}
}
bool Indexer::need_update(llvm::StringRef file_path) {
auto cache_it = workspace.project_index.path_pool.find(file_path);
if(cache_it == workspace.project_index.path_pool.cache.end())
return true;
auto merged_it = workspace.merged_indices.find(cache_it->second);
if(merged_it == workspace.merged_indices.end())
return true;
llvm::SmallVector<llvm::StringRef> path_mapping;
for(auto& p: workspace.project_index.path_pool.paths) {
path_mapping.push_back(p);
}
return merged_it->second.index.need_update(path_mapping);
}
bool Indexer::find_symbol_info(index::SymbolHash hash, std::string& name, SymbolKind& kind) const {
for(auto& [_, session]: sessions) {
if(!session.file_index)
continue;
auto it = session.file_index->symbols.find(hash);
if(it != session.file_index->symbols.end()) {
name = it->second.name;
kind = it->second.kind;
return true;
}
}
auto it = workspace.project_index.symbols.find(hash);
if(it != workspace.project_index.symbols.end()) {
name = it->second.name;
kind = it->second.kind;
return true;
}
return false;
}
Indexer::CursorHit Indexer::resolve_cursor(llvm::StringRef path,
const protocol::Position& position,
Session* session) {
// Try the session's open file index first.
if(session && session->file_index) {
auto& index = *session->file_index;
if(!index.mapper)
return {};
auto offset = index.mapper->to_offset(position);
if(!offset)
return {};
if(auto found = index.find_occurrence(*offset))
return {found->first, found->second};
return {};
}
// Fallback to MergedIndex, using session text (or reading from disk) for position -> offset.
const std::string* doc_text = session ? &session->text : nullptr;
if(!doc_text)
return {};
lsp::PositionMapper doc_mapper(*doc_text, lsp::PositionEncoding::UTF16);
auto offset = doc_mapper.to_offset(position);
if(!offset)
return {};
auto proj_it = workspace.project_index.path_pool.find(path);
if(proj_it == workspace.project_index.path_pool.cache.end())
return {};
auto shard_it = workspace.merged_indices.find(proj_it->second);
if(shard_it == workspace.merged_indices.end())
return {};
if(auto found = shard_it->second.find_occurrence(*offset))
return {found->first, found->second};
return {};
}
std::vector<protocol::Location> Indexer::query_relations(llvm::StringRef path,
const protocol::Position& position,
RelationKind kind,
Session* session) {
auto hit = resolve_cursor(path, position, session);
if(hit.hash == 0)
return {};
std::vector<protocol::Location> locations;
auto sym_it = workspace.project_index.symbols.find(hit.hash);
if(sym_it != workspace.project_index.symbols.end()) {
for(auto file_id: sym_it->second.reference_files) {
if(is_proj_path_open(file_id))
continue;
auto shard_it = workspace.merged_indices.find(file_id);
if(shard_it == workspace.merged_indices.end())
continue;
auto uri = lsp::URI::from_file_path(workspace.project_index.path_pool.path(file_id));
if(!uri)
continue;
shard_it->second.find_relations(hit.hash,
kind,
[&](const auto&, protocol::Range range) {
locations.push_back({uri->str(), range});
return true;
});
}
}
for(auto& [id, sess]: sessions) {
if(!sess.file_index)
continue;
auto uri = lsp::URI::from_file_path(std::string(workspace.path_pool.resolve(id)));
if(!uri)
continue;
sess.file_index->find_relations(hit.hash, kind, [&](const auto&, protocol::Range range) {
locations.push_back({uri->str(), range});
return true;
});
}
return locations;
}
std::optional<SymbolInfo> Indexer::lookup_symbol(const std::string& uri,
llvm::StringRef path,
const protocol::Position& position,
Session* session) {
auto hit = resolve_cursor(path, position, session);
if(hit.hash == 0)
return std::nullopt;
std::string name;
SymbolKind sym_kind;
if(!find_symbol_info(hit.hash, name, sym_kind))
return std::nullopt;
return SymbolInfo{hit.hash, std::move(name), sym_kind, uri, hit.range};
}
std::optional<protocol::Location> Indexer::find_definition_location(index::SymbolHash hash) {
// Open file indices first (fresher data for actively-edited files).
for(auto& [id, sess]: sessions) {
if(!sess.file_index)
continue;
auto uri = lsp::URI::from_file_path(std::string(workspace.path_pool.resolve(id)));
if(!uri)
continue;
std::optional<protocol::Location> result;
sess.file_index->find_relations(hash,
RelationKind::Definition,
[&](const auto&, protocol::Range range) {
result = protocol::Location{uri->str(), range};
return false;
});
if(result)
return result;
}
// Fall back to ProjectIndex reference files.
auto sym_it = workspace.project_index.symbols.find(hash);
if(sym_it == workspace.project_index.symbols.end())
return std::nullopt;
for(auto file_id: sym_it->second.reference_files) {
if(is_proj_path_open(file_id))
continue;
auto shard_it = workspace.merged_indices.find(file_id);
if(shard_it == workspace.merged_indices.end())
continue;
auto uri = lsp::URI::from_file_path(workspace.project_index.path_pool.path(file_id));
if(!uri)
continue;
std::optional<protocol::Location> result;
shard_it->second.find_relations(hash,
RelationKind::Definition,
[&](const auto&, protocol::Range range) {
result = protocol::Location{uri->str(), range};
return false;
});
if(result)
return result;
}
return std::nullopt;
}
std::optional<SymbolInfo>
Indexer::resolve_hierarchy_item(const std::string& uri,
llvm::StringRef path,
const protocol::Range& range,
const std::optional<protocol::LSPAny>& data,
Session* session) {
if(data) {
if(auto* int_val = std::get_if<std::int64_t>(&*data)) {
auto hash = static_cast<index::SymbolHash>(*int_val);
std::string name;
SymbolKind kind;
if(find_symbol_info(hash, name, kind)) {
return SymbolInfo{hash, std::move(name), kind, uri, range};
}
}
}
return lookup_symbol(uri, path, range.start, session);
}
void Indexer::collect_grouped_relations(
index::SymbolHash hash,
RelationKind kind,
llvm::DenseMap<index::SymbolHash, std::vector<protocol::Range>>& target_ranges) {
auto sym_it = workspace.project_index.symbols.find(hash);
if(sym_it != workspace.project_index.symbols.end()) {
for(auto file_id: sym_it->second.reference_files) {
if(is_proj_path_open(file_id))
continue;
auto shard_it = workspace.merged_indices.find(file_id);
if(shard_it == workspace.merged_indices.end())
continue;
shard_it->second.find_relations(hash, kind, [&](const auto& r, protocol::Range range) {
target_ranges[r.target_symbol].push_back(range);
return true;
});
}
}
for(auto& [_, sess]: sessions) {
if(!sess.file_index)
continue;
sess.file_index->find_relations(hash, kind, [&](const auto& r, protocol::Range range) {
target_ranges[r.target_symbol].push_back(range);
return true;
});
}
}
void Indexer::collect_unique_targets(index::SymbolHash hash,
RelationKind kind,
llvm::SmallVectorImpl<index::SymbolHash>& targets) {
llvm::DenseSet<index::SymbolHash> seen;
auto sym_it = workspace.project_index.symbols.find(hash);
if(sym_it != workspace.project_index.symbols.end()) {
for(auto file_id: sym_it->second.reference_files) {
if(is_proj_path_open(file_id))
continue;
auto shard_it = workspace.merged_indices.find(file_id);
if(shard_it == workspace.merged_indices.end())
continue;
/// No position conversion needed -- just collect target symbol hashes.
shard_it->second.index.lookup(hash, kind, [&](const index::Relation& r) {
if(seen.insert(r.target_symbol).second) {
targets.push_back(r.target_symbol);
}
return true;
});
}
}
for(auto& [_, sess]: sessions) {
if(!sess.file_index)
continue;
auto rel_it = sess.file_index->file_index.relations.find(hash);
if(rel_it == sess.file_index->file_index.relations.end())
continue;
for(auto& r: rel_it->second) {
if(r.kind & kind) {
if(seen.insert(r.target_symbol).second) {
targets.push_back(r.target_symbol);
}
}
}
}
}
/// Resolve a symbol hash into a SymbolInfo with definition location.
/// Returns nullopt if the symbol or its definition cannot be found.
std::optional<SymbolInfo> Indexer::resolve_symbol(index::SymbolHash hash) {
std::string name;
SymbolKind kind;
if(!find_symbol_info(hash, name, kind))
return std::nullopt;
auto def_loc = find_definition_location(hash);
if(!def_loc)
return std::nullopt;
return SymbolInfo{hash, std::move(name), kind, def_loc->uri, def_loc->range};
}
std::vector<protocol::CallHierarchyIncomingCall>
Indexer::find_incoming_calls(index::SymbolHash hash) {
llvm::DenseMap<index::SymbolHash, std::vector<protocol::Range>> caller_ranges;
collect_grouped_relations(hash, RelationKind::Caller, caller_ranges);
std::vector<protocol::CallHierarchyIncomingCall> results;
for(auto& [caller_hash, ranges]: caller_ranges) {
auto info = resolve_symbol(caller_hash);
if(!info)
continue;
results.push_back({build_call_hierarchy_item(*info), std::move(ranges)});
}
return results;
}
std::vector<protocol::CallHierarchyOutgoingCall>
Indexer::find_outgoing_calls(index::SymbolHash hash) {
llvm::DenseMap<index::SymbolHash, std::vector<protocol::Range>> callee_ranges;
collect_grouped_relations(hash, RelationKind::Callee, callee_ranges);
std::vector<protocol::CallHierarchyOutgoingCall> results;
for(auto& [callee_hash, ranges]: callee_ranges) {
auto info = resolve_symbol(callee_hash);
if(!info)
continue;
results.push_back({build_call_hierarchy_item(*info), std::move(ranges)});
}
return results;
}
std::vector<protocol::TypeHierarchyItem> Indexer::find_supertypes(index::SymbolHash hash) {
llvm::SmallVector<index::SymbolHash> base_hashes;
collect_unique_targets(hash, RelationKind::Base, base_hashes);
std::vector<protocol::TypeHierarchyItem> results;
for(auto target_hash: base_hashes) {
auto info = resolve_symbol(target_hash);
if(!info)
continue;
results.push_back(build_type_hierarchy_item(*info));
}
return results;
}
std::vector<protocol::TypeHierarchyItem> Indexer::find_subtypes(index::SymbolHash hash) {
llvm::SmallVector<index::SymbolHash> derived_hashes;
collect_unique_targets(hash, RelationKind::Derived, derived_hashes);
std::vector<protocol::TypeHierarchyItem> results;
for(auto target_hash: derived_hashes) {
auto info = resolve_symbol(target_hash);
if(!info)
continue;
results.push_back(build_type_hierarchy_item(*info));
}
return results;
}
std::vector<protocol::SymbolInformation> Indexer::search_symbols(llvm::StringRef query,
std::size_t max_results) {
std::string query_lower = query.lower();
auto is_indexable_kind = [](SymbolKind sk) {
return sk == SymbolKind::Namespace || sk == SymbolKind::Class || sk == SymbolKind::Struct ||
sk == SymbolKind::Union || sk == SymbolKind::Enum || sk == SymbolKind::Type ||
sk == SymbolKind::Field || sk == SymbolKind::EnumMember ||
sk == SymbolKind::Function || sk == SymbolKind::Method ||
sk == SymbolKind::Variable || sk == SymbolKind::Parameter ||
sk == SymbolKind::Macro || sk == SymbolKind::Concept || sk == SymbolKind::Module ||
sk == SymbolKind::Operator || sk == SymbolKind::MacroParameter ||
sk == SymbolKind::Label || sk == SymbolKind::Attribute;
};
auto matches_query = [&](llvm::StringRef name) {
if(query_lower.empty())
return true;
return llvm::StringRef(name).lower().find(query_lower) != std::string::npos;
};
std::vector<protocol::SymbolInformation> results;
llvm::DenseSet<index::SymbolHash> seen;
for(auto& [hash, symbol]: workspace.project_index.symbols) {
if(results.size() >= max_results)
break;
if(!is_indexable_kind(symbol.kind) || symbol.name.empty())
continue;
if(!matches_query(symbol.name))
continue;
auto def_loc = find_definition_location(hash);
if(!def_loc)
continue;
protocol::SymbolInformation info;
info.name = symbol.name;
info.kind = to_lsp_symbol_kind(symbol.kind);
info.location = std::move(*def_loc);
results.push_back(std::move(info));
seen.insert(hash);
}
for(auto& [_, sess]: sessions) {
if(results.size() >= max_results)
break;
if(!sess.file_index)
continue;
for(auto& [hash, symbol]: sess.file_index->symbols) {
if(results.size() >= max_results)
break;
if(seen.contains(hash))
continue;
if(!is_indexable_kind(symbol.kind) || symbol.name.empty())
continue;
if(!matches_query(symbol.name))
continue;
auto def_loc = find_definition_location(hash);
if(!def_loc)
continue;
protocol::SymbolInformation info;
info.name = symbol.name;
info.kind = to_lsp_symbol_kind(symbol.kind);
info.location = std::move(*def_loc);
results.push_back(std::move(info));
seen.insert(hash);
}
}
return results;
}
protocol::SymbolKind Indexer::to_lsp_symbol_kind(SymbolKind kind) {
switch(kind) {
case SymbolKind::Namespace: return protocol::SymbolKind::Namespace;
case SymbolKind::Class: return protocol::SymbolKind::Class;
case SymbolKind::Struct: return protocol::SymbolKind::Struct;
case SymbolKind::Union: return protocol::SymbolKind::Class;
case SymbolKind::Enum: return protocol::SymbolKind::Enum;
case SymbolKind::Type: return protocol::SymbolKind::TypeParameter;
case SymbolKind::Field: return protocol::SymbolKind::Field;
case SymbolKind::EnumMember: return protocol::SymbolKind::EnumMember;
case SymbolKind::Function: return protocol::SymbolKind::Function;
case SymbolKind::Method: return protocol::SymbolKind::Method;
case SymbolKind::Variable: return protocol::SymbolKind::Variable;
case SymbolKind::Parameter: return protocol::SymbolKind::Variable;
case SymbolKind::Macro: return protocol::SymbolKind::Function;
case SymbolKind::Concept: return protocol::SymbolKind::Interface;
case SymbolKind::Module: return protocol::SymbolKind::Module;
case SymbolKind::Operator: return protocol::SymbolKind::Operator;
default: return protocol::SymbolKind::Variable;
}
}
protocol::CallHierarchyItem Indexer::build_call_hierarchy_item(const SymbolInfo& info) {
protocol::CallHierarchyItem item;
item.name = info.name;
item.kind = to_lsp_symbol_kind(info.kind);
item.uri = info.uri;
item.range = info.range;
item.selection_range = info.range;
item.data = protocol::LSPAny(static_cast<std::int64_t>(info.hash));
return item;
}
protocol::TypeHierarchyItem Indexer::build_type_hierarchy_item(const SymbolInfo& info) {
protocol::TypeHierarchyItem item;
item.name = info.name;
item.kind = to_lsp_symbol_kind(info.kind);
item.uri = info.uri;
item.range = info.range;
item.selection_range = info.range;
item.data = protocol::LSPAny(static_cast<std::int64_t>(info.hash));
return item;
}
void Indexer::enqueue(std::uint32_t server_path_id) {
index_queue.push_back(server_path_id);
}
void Indexer::schedule() {
if(!*workspace.config.project.enable_indexing || indexing_active || indexing_scheduled)
return;
indexing_scheduled = true;
if(!index_idle_timer) {
index_idle_timer = std::make_shared<kota::timer>(kota::timer::create(loop));
}
index_idle_timer->start(std::chrono::milliseconds(*workspace.config.project.idle_timeout_ms));
loop.schedule(run_background_indexing());
}
kota::task<> Indexer::run_background_indexing() {
if(index_idle_timer) {
co_await index_idle_timer->wait();
}
indexing_scheduled = false;
if(index_queue_pos >= index_queue.size()) {
LOG_DEBUG("Background indexing: queue exhausted");
co_return;
}
indexing_active = true;
std::size_t processed = 0;
while(index_queue_pos < index_queue.size()) {
auto server_path_id = index_queue[index_queue_pos];
index_queue_pos++;
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
if(sessions.contains(server_path_id))
continue;
if(!need_update(file_path))
continue;
worker::BuildParams params;
params.kind = worker::BuildKind::Index;
params.file = file_path;
if(!compiler.fill_compile_args(file_path, params.directory, params.arguments, nullptr))
continue;
workspace.fill_pcm_deps(params.pcms);
LOG_INFO("Background indexing: {}", file_path);
auto result = co_await pool.send_stateless(params);
if(result.has_value() && result.value().success && !result.value().tu_index_data.empty()) {
LOG_INFO("Background indexing got TUIndex for {}: {} bytes",
file_path,
result.value().tu_index_data.size());
merge(result.value().tu_index_data.data(), result.value().tu_index_data.size());
++processed;
} else if(result.has_value() && !result.value().success) {
LOG_WARN("Background index failed for {}: {}", file_path, result.value().error);
} else if(result.has_value() && result.value().tu_index_data.empty()) {
LOG_WARN("Background index returned empty TUIndex for {}", file_path);
} else {
LOG_WARN("Background index IPC error for {}: {}", file_path, result.error().message);
}
}
indexing_active = false;
LOG_INFO("Background indexing complete: {} files processed", processed);
save(workspace.config.project.index_dir);
}
} // namespace clice

View File

@@ -1,188 +0,0 @@
#pragma once
#include <cstdint>
#include <functional>
#include <memory>
#include <optional>
#include <string>
#include <vector>
#include "semantic/relation_kind.h"
#include "semantic/symbol_kind.h"
#include "server/workspace.h"
#include "kota/async/async.h"
#include "kota/ipc/lsp/position.h"
#include "kota/ipc/lsp/protocol.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/SmallVector.h"
#include "llvm/ADT/StringRef.h"
namespace clice {
namespace protocol = kota::ipc::protocol;
namespace lsp = kota::ipc::lsp;
struct Session;
class Compiler;
class WorkerPool;
/// Information about a symbol at a given position.
struct SymbolInfo {
index::SymbolHash hash = 0;
std::string name;
SymbolKind kind;
std::string uri;
protocol::Range range;
};
/// Index query layer and background indexing scheduler.
///
/// Indexer holds no index data of its own. All persistent data lives in
/// Workspace (disk-derived ProjectIndex + MergedIndex shards) and per-file
/// data lives in Session (OpenFileIndex from unsaved buffers).
///
/// Responsibilities:
/// - Cross-file navigation queries (definition, references, hierarchy)
/// - Symbol search (workspace/symbol)
/// - Background indexing scheduling (enqueue → idle timer → worker dispatch)
/// - Merging TUIndex results into Workspace's ProjectIndex
///
/// NOT responsible for:
/// - Compilation — handled by Compiler
/// - Document lifecycle — handled by MasterServer
class Indexer {
public:
Indexer(kota::event_loop& loop,
Workspace& workspace,
llvm::DenseMap<std::uint32_t, Session>& sessions,
WorkerPool& pool,
Compiler& compiler,
std::function<bool(std::uint32_t)> is_file_open = {}) :
loop(loop), workspace(workspace), sessions(sessions), pool(pool), compiler(compiler),
is_file_open(std::move(is_file_open)) {}
/// Add a file to the background indexing queue.
void enqueue(std::uint32_t server_path_id);
/// Schedule background indexing (respects idle timeout and dedup).
void schedule();
/// Merge a TUIndex result into Workspace's ProjectIndex and MergedIndex shards.
void merge(const void* tu_index_data, std::size_t size);
/// Save Workspace's ProjectIndex and MergedIndex shards to disk.
void save(llvm::StringRef index_dir);
/// Load Workspace's ProjectIndex and MergedIndex shards from disk.
void load(llvm::StringRef index_dir);
/// Check whether a file needs re-indexing (stale or missing shard).
bool need_update(llvm::StringRef file_path);
/// Query relations (Definition, Reference, etc.) for a symbol at cursor.
/// @param session Active Session for this file, or nullptr to use MergedIndex only.
std::vector<protocol::Location> query_relations(llvm::StringRef path,
const protocol::Position& position,
RelationKind kind,
Session* session);
/// Look up symbol info (hash, name, kind, range) at a cursor position.
/// @param session Active Session for this file, or nullptr.
std::optional<SymbolInfo> lookup_symbol(const std::string& uri,
llvm::StringRef path,
const protocol::Position& position,
Session* session);
/// Find the definition location of a symbol by hash.
std::optional<protocol::Location> find_definition_location(index::SymbolHash hash);
/// Find a symbol's name and kind by hash.
bool find_symbol_info(index::SymbolHash hash, std::string& name, SymbolKind& kind) const;
/// Resolve a hierarchy item (from stored data or by position lookup).
/// @param session Active Session for this file, or nullptr.
std::optional<SymbolInfo> resolve_hierarchy_item(const std::string& uri,
llvm::StringRef path,
const protocol::Range& range,
const std::optional<protocol::LSPAny>& data,
Session* session);
/// Find incoming calls to a function.
std::vector<protocol::CallHierarchyIncomingCall> find_incoming_calls(index::SymbolHash hash);
/// Find outgoing calls from a function.
std::vector<protocol::CallHierarchyOutgoingCall> find_outgoing_calls(index::SymbolHash hash);
/// Find supertypes (base classes) of a type.
std::vector<protocol::TypeHierarchyItem> find_supertypes(index::SymbolHash hash);
/// Find subtypes (derived classes) of a type.
std::vector<protocol::TypeHierarchyItem> find_subtypes(index::SymbolHash hash);
/// Search symbols by name substring.
std::vector<protocol::SymbolInformation> search_symbols(llvm::StringRef query,
std::size_t max_results = 100);
/// Convert internal SymbolKind to LSP SymbolKind.
static protocol::SymbolKind to_lsp_symbol_kind(SymbolKind kind);
/// Build hierarchy items from SymbolInfo.
static protocol::CallHierarchyItem build_call_hierarchy_item(const SymbolInfo& info);
static protocol::TypeHierarchyItem build_type_hierarchy_item(const SymbolInfo& info);
private:
/// Result of resolving a symbol at a cursor position.
struct CursorHit {
index::SymbolHash hash = 0;
protocol::Range range{};
};
/// Resolve the symbol at (position), checking Session's file_index first
/// then falling back to Workspace's MergedIndex.
CursorHit resolve_cursor(llvm::StringRef path,
const protocol::Position& position,
Session* session);
/// Collect relations grouped by target symbol, across all index sources.
void collect_grouped_relations(
index::SymbolHash hash,
RelationKind kind,
llvm::DenseMap<index::SymbolHash, std::vector<protocol::Range>>& target_ranges);
/// Collect unique target symbol hashes for a relation kind.
void collect_unique_targets(index::SymbolHash hash,
RelationKind kind,
llvm::SmallVectorImpl<index::SymbolHash>& targets);
/// Resolve a symbol hash into a SymbolInfo with definition location.
std::optional<SymbolInfo> resolve_symbol(index::SymbolHash hash);
/// Check whether a project-level path_id has an active Session.
bool is_proj_path_open(std::uint32_t proj_path_id) const {
return is_file_open && is_file_open(proj_path_id);
}
private:
kota::event_loop& loop;
Workspace& workspace;
llvm::DenseMap<std::uint32_t, Session>& sessions;
WorkerPool& pool;
Compiler& compiler;
/// Callback that checks if a *project-level* path_id has an active
/// Session. Set by the owner (e.g. MasterServer) to bridge the
/// server-path-id-keyed sessions map to project-level path_ids.
std::function<bool(std::uint32_t)> is_file_open;
/// Background indexing queue and scheduling state.
std::vector<std::uint32_t> index_queue;
std::size_t index_queue_pos = 0;
bool indexing_active = false;
bool indexing_scheduled = false;
std::shared_ptr<kota::timer> index_idle_timer;
kota::task<> run_background_indexing();
};
} // namespace clice

File diff suppressed because it is too large Load Diff

View File

@@ -3,21 +3,40 @@
#include <cstdint>
#include <memory>
#include <string>
#include <vector>
#include "server/compiler.h"
#include "server/indexer.h"
#include "server/session.h"
#include "command/command.h"
#include "eventide/async/async.h"
#include "eventide/ipc/lsp/protocol.h"
#include "eventide/ipc/peer.h"
#include "eventide/serde/serde/raw_value.h"
#include "index/merged_index.h"
#include "index/project_index.h"
#include "semantic/relation_kind.h"
#include "server/compile_graph.h"
#include "server/config.h"
#include "server/worker_pool.h"
#include "server/workspace.h"
#include "support/path_pool.h"
#include "syntax/dependency_graph.h"
#include "kota/async/async.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/peer.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/SmallVector.h"
#include "llvm/ADT/StringMap.h"
#include "llvm/ADT/StringRef.h"
namespace clice {
namespace et = eventide;
namespace protocol = et::ipc::protocol;
struct DocumentState {
int version = 0;
std::string text;
std::uint64_t generation = 0;
bool build_running = false;
bool build_requested = false;
bool drain_scheduled = false;
};
enum class ServerLifecycle : std::uint8_t {
Uninitialized,
Initialized,
@@ -26,56 +45,177 @@ enum class ServerLifecycle : std::uint8_t {
Exited,
};
/// Top-level LSP server — the single orchestration point for the language
/// server process.
///
/// Responsibilities:
/// - Owns the two-layer state model: Workspace (disk truth) and Sessions
/// (per-open-file volatile state).
/// - Manages Session lifecycle directly: didOpen creates, didChange mutates,
/// didSave syncs to Workspace, didClose destroys.
/// - Dispatches compilation and feature queries to Compiler.
/// - Dispatches index lookups and background indexing to Indexer.
///
/// Design principle:
/// Open files are never depended upon by other files. Dependencies always
/// point to disk files. The only path from Session to Workspace is didSave.
class MasterServer {
public:
MasterServer(kota::event_loop& loop, kota::ipc::JsonPeer& peer, std::string self_path);
MasterServer(et::event_loop& loop, et::ipc::JsonPeer& peer, std::string self_path);
~MasterServer();
void register_handlers();
private:
kota::event_loop& loop;
kota::ipc::JsonPeer& peer;
/// Persistent project-wide state (config, CDB, path pool, dependency
/// graphs, compilation caches, symbol index).
Workspace workspace;
/// Per-file editing sessions, keyed by server-level path_id.
llvm::DenseMap<std::uint32_t, Session> sessions;
/// Worker process pool for offloading compilation and queries.
et::event_loop& loop;
et::ipc::JsonPeer& peer;
WorkerPool pool;
/// Compilation lifecycle manager (reads/writes workspace and sessions).
Compiler compiler;
/// Index query and background scheduling (reads from workspace and sessions).
Indexer indexer;
PathPool path_pool;
ServerLifecycle lifecycle = ServerLifecycle::Uninitialized;
std::string self_path;
std::string workspace_root;
std::string session_log_dir;
std::string init_options_json; ///< Raw JSON from initializationOptions, consumed once.
CliceConfig config;
void load_workspace();
CompilationDatabase cdb;
DependencyGraph dependency_graph;
using RawResult = kota::task<kota::codec::RawValue, kota::ipc::Error>;
// Module compilation graph (lazy dependency resolution).
std::unique_ptr<CompileGraph> compile_graph;
// path_id -> built PCM output path (set after successful module build).
llvm::DenseMap<std::uint32_t, std::string> pcm_paths;
// path_id -> module name (for files that provide a module interface).
llvm::DenseMap<std::uint32_t, std::string> path_to_module;
// path_id -> built PCH file path.
llvm::DenseMap<std::uint32_t, std::string> pch_paths;
// path_id -> preamble bound (byte offset) used when building the PCH.
llvm::DenseMap<std::uint32_t, std::uint32_t> pch_bounds;
// path_id -> hash of preamble content at PCH build time (for staleness detection).
llvm::DenseMap<std::uint32_t, std::uint64_t> pch_hashes;
// path_id -> in-flight PCH build event (later arrivals co_await the same build).
llvm::DenseMap<std::uint32_t, std::shared_ptr<et::event>> pch_building;
// === Index state ===
// Global symbol table and path mapping for the project.
index::ProjectIndex project_index;
// Per-file merged index shards (keyed by project-level path_id).
llvm::DenseMap<std::uint32_t, index::MergedIndex> merged_indices;
// Files queued for background indexing (server-level path_ids from CDB).
std::vector<std::uint32_t> index_queue;
// Index of next file to process in index_queue.
std::size_t index_queue_pos = 0;
// Whether background indexing is currently in progress.
bool indexing_active = false;
// Whether a background indexing coroutine has been scheduled (waiting on timer).
bool indexing_scheduled = false;
// Timer for idle-triggered background indexing.
std::shared_ptr<et::timer> index_idle_timer;
// Document state: path_id -> DocumentState
llvm::DenseMap<std::uint32_t, DocumentState> documents;
// Per-document debounce timers (shared_ptr so drain coroutines survive didClose)
llvm::DenseMap<std::uint32_t, std::shared_ptr<et::timer>> debounce_timers;
// Helper: convert URI to file path
std::string uri_to_path(const std::string& uri);
// Publish diagnostics to client
void publish_diagnostics(const std::string& uri,
int version,
const eventide::serde::RawValue& diagnostics_json);
void clear_diagnostics(const std::string& uri);
// Schedule a build after debounce
void schedule_build(std::uint32_t path_id, const std::string& uri);
// Build drain coroutine: waits for debounce, then runs compile loop
et::task<> run_build_drain(std::uint32_t path_id, std::string uri);
// Ensure a file has been compiled before servicing feature requests
et::task<bool> ensure_compiled(std::uint32_t path_id, const std::string& uri);
// Load CDB and build initial include graph
et::task<> load_workspace();
// Helper: fill compile arguments from CDB into worker params
bool fill_compile_args(llvm::StringRef path,
std::string& directory,
std::vector<std::string>& arguments);
// Build or reuse PCH for a source file. Returns true if PCH is available.
et::task<bool> ensure_pch(std::uint32_t path_id,
llvm::StringRef path,
const std::string& text,
const std::string& directory,
const std::vector<std::string>& arguments);
// Schedule background indexing when idle.
void schedule_indexing();
// Background indexing coroutine: picks files from queue and dispatches to workers.
et::task<> run_background_indexing();
// Merge a TUIndex result into ProjectIndex and MergedIndex shards.
void merge_index_result(const void* tu_index_data, std::size_t size);
// Persist index state to disk.
void save_index();
// Load index state from disk.
void load_index();
// Forwarding helpers for feature requests (RawValue passthrough)
using RawResult = et::task<et::serde::RawValue, et::ipc::Error>;
/// Forward a simple stateful request (path-only worker params).
template <typename WorkerParams>
RawResult forward_stateful(const std::string& uri);
/// Forward a stateful request with position-to-offset conversion.
template <typename WorkerParams>
RawResult forward_stateful(const std::string& uri, const protocol::Position& position);
/// Forward a stateless request with document content and compile args.
template <typename WorkerParams>
RawResult forward_stateless(const std::string& uri, const protocol::Position& position);
/// Query index for symbol relations (GoToDefinition, FindReferences, etc.).
/// Returns LSP Location array as RawValue.
RawResult query_index_relations(const std::string& uri,
const protocol::Position& position,
RelationKind kind);
/// Information about a symbol at a given position.
struct SymbolInfo {
index::SymbolHash hash = 0;
std::string name;
SymbolKind kind;
std::string uri;
protocol::Range range;
};
/// Look up a symbol at a position, returning its hash, name, kind, and range.
et::task<std::optional<SymbolInfo>>
lookup_symbol_at_position(const std::string& uri, const protocol::Position& position);
/// Find the definition location (uri + range) of a symbol by its hash.
std::optional<protocol::Location> find_symbol_definition_location(index::SymbolHash hash);
/// Convert clice::SymbolKind to LSP protocol::SymbolKind.
static protocol::SymbolKind to_lsp_symbol_kind(SymbolKind kind);
/// Build a CallHierarchyItem from a SymbolInfo.
protocol::CallHierarchyItem build_call_hierarchy_item(const SymbolInfo& info);
/// Build a TypeHierarchyItem from a SymbolInfo.
protocol::TypeHierarchyItem build_type_hierarchy_item(const SymbolInfo& info);
/// Resolve SymbolInfo from a hierarchy item's stored data (symbol hash).
/// Falls back to position-based lookup if data is missing.
et::task<std::optional<SymbolInfo>>
resolve_hierarchy_item(const std::string& uri,
const protocol::Range& range,
const std::optional<protocol::LSPAny>& data);
};
} // namespace clice

View File

@@ -7,38 +7,16 @@
#include <utility>
#include <vector>
#include "syntax/token.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/protocol.h"
#include "eventide/ipc/lsp/protocol.h"
#include "eventide/ipc/protocol.h"
#include "eventide/serde/serde/raw_value.h"
namespace clice::worker {
namespace protocol = kota::ipc::protocol;
namespace protocol = eventide::ipc::protocol;
/// Kind of AST query dispatched to a stateful worker.
enum class QueryKind : uint8_t {
Hover,
GoToDefinition,
SemanticTokens,
InlayHints,
FoldingRange,
DocumentSymbol,
DocumentLink,
CodeAction,
};
// === StatefulWorker Requests ===
/// Unified parameters for all stateful AST queries.
/// The worker dispatches to the appropriate feature handler based on `kind`.
struct QueryParams {
QueryKind kind;
std::string path;
uint32_t offset = 0; ///< Byte offset for position-sensitive queries (Hover, GoToDefinition).
LocalSourceRange range; ///< Byte range for range-sensitive queries (InlayHints).
};
/// Parameters for stateful compilation (builds AST, publishes diagnostics).
struct CompileParams {
std::string path;
int version;
@@ -52,64 +30,115 @@ struct CompileParams {
struct CompileResult {
int version;
/// Diagnostics serialized as JSON (RawValue) to avoid bincode/serde annotation conflicts.
kota::codec::RawValue diagnostics;
eventide::serde::RawValue diagnostics;
std::size_t memory_usage;
std::vector<std::string> deps;
/// Serialized TUIndex for the main file (interested_only=true).
std::string tu_index_data;
};
/// Kind of build task dispatched to a stateless worker.
enum class BuildKind : uint8_t {
BuildPCH,
BuildPCM,
Index,
Completion,
SignatureHelp,
struct HoverParams {
std::string path;
uint32_t offset;
};
/// Unified parameters for all stateless build/compilation tasks.
/// Fields are used selectively based on `kind`:
/// - All: file, directory, arguments
/// - BuildPCH: + content, preamble_bound, output_path
/// - BuildPCM: + module_name, pcms, output_path
/// - Index: + pcms
/// - Completion: + text, version, offset, pch, pcms
/// - SignatureHelp: + text, version, offset, pch, pcms
struct BuildParams {
BuildKind kind;
struct SemanticTokensParams {
std::string path;
};
struct InlayHintsParams {
std::string path;
};
struct FoldingRangeParams {
std::string path;
};
struct DocumentSymbolParams {
std::string path;
};
struct DocumentLinkParams {
std::string path;
};
struct CodeActionParams {
std::string path;
};
struct GoToDefinitionParams {
std::string path;
uint32_t offset;
};
// === StatelessWorker Requests ===
struct CompletionParams {
std::string path;
int version;
std::string text;
std::string directory;
std::vector<std::string> arguments;
std::pair<std::string, uint32_t> pch;
std::unordered_map<std::string, std::string> pcms;
uint32_t offset;
};
struct SignatureHelpParams {
std::string path;
int version;
std::string text;
std::string directory;
std::vector<std::string> arguments;
std::pair<std::string, uint32_t> pch;
std::unordered_map<std::string, std::string> pcms;
uint32_t offset;
};
struct BuildPCHParams {
std::string file;
std::string directory;
std::vector<std::string> arguments;
/// Source text for Completion/SignatureHelp, preamble content for BuildPCH.
std::string text;
int version = 0;
uint32_t offset = 0;
std::pair<std::string, uint32_t> pch;
std::unordered_map<std::string, std::string> pcms;
std::string output_path; ///< BuildPCH, BuildPCM
std::string module_name; ///< BuildPCM
uint32_t preamble_bound = UINT32_MAX; ///< BuildPCH
std::string content;
std::uint32_t preamble_bound = UINT32_MAX;
};
/// Unified result for stateless build tasks.
/// For Completion/SignatureHelp, the result JSON is in `result_json`.
/// For BuildPCH/BuildPCM/Index, structured fields are used.
struct BuildResult {
bool success = true;
struct BuildPCHResult {
bool success;
std::string error;
std::string output_path; ///< PCH or PCM path
std::vector<std::string> deps;
std::string tu_index_data;
std::string pch_links_json; ///< Pre-serialized DocumentLink[] from PCH
kota::codec::RawValue result_json; ///< Completion/SignatureHelp result
std::string pch_path;
};
struct BuildPCMParams {
std::string file;
std::string directory;
std::vector<std::string> arguments;
std::string module_name;
std::unordered_map<std::string, std::string> pcms;
};
struct BuildPCMResult {
bool success;
std::string error;
std::string pcm_path;
};
struct IndexParams {
std::string file;
std::string directory;
std::vector<std::string> arguments;
std::unordered_map<std::string, std::string> pcms;
};
struct IndexResult {
bool success;
std::string error;
std::string tu_index_data;
};
// === Notifications ===
struct DocumentUpdateParams {
std::string path;
int version;
std::string text;
};
struct EvictParams {
@@ -122,44 +151,9 @@ struct EvictedParams {
} // namespace clice::worker
namespace clice::ext {
namespace eventide::ipc::protocol {
struct ContextItem {
std::string label;
std::string description;
std::string uri;
};
struct QueryContextParams {
std::string uri;
std::optional<int> offset;
};
struct QueryContextResult {
std::vector<ContextItem> contexts;
int total;
};
struct CurrentContextParams {
std::string uri;
};
struct CurrentContextResult {
std::optional<ContextItem> context;
};
struct SwitchContextParams {
std::string uri;
std::string context_uri;
};
struct SwitchContextResult {
bool success;
};
} // namespace clice::ext
namespace kota::ipc::protocol {
// === Stateful Requests ===
template <>
struct RequestTraits<clice::worker::CompileParams> {
@@ -168,17 +162,87 @@ struct RequestTraits<clice::worker::CompileParams> {
};
template <>
struct RequestTraits<clice::worker::QueryParams> {
using Result = kota::codec::RawValue;
constexpr inline static std::string_view method = "clice/worker/query";
struct RequestTraits<clice::worker::HoverParams> {
using Result = eventide::serde::RawValue;
constexpr inline static std::string_view method = "clice/worker/hover";
};
template <>
struct RequestTraits<clice::worker::BuildParams> {
using Result = clice::worker::BuildResult;
constexpr inline static std::string_view method = "clice/worker/build";
struct RequestTraits<clice::worker::SemanticTokensParams> {
using Result = eventide::serde::RawValue;
constexpr inline static std::string_view method = "clice/worker/semanticTokens";
};
template <>
struct RequestTraits<clice::worker::InlayHintsParams> {
using Result = eventide::serde::RawValue;
constexpr inline static std::string_view method = "clice/worker/inlayHints";
};
template <>
struct RequestTraits<clice::worker::FoldingRangeParams> {
using Result = eventide::serde::RawValue;
constexpr inline static std::string_view method = "clice/worker/foldingRange";
};
template <>
struct RequestTraits<clice::worker::DocumentSymbolParams> {
using Result = eventide::serde::RawValue;
constexpr inline static std::string_view method = "clice/worker/documentSymbol";
};
template <>
struct RequestTraits<clice::worker::DocumentLinkParams> {
using Result = eventide::serde::RawValue;
constexpr inline static std::string_view method = "clice/worker/documentLink";
};
template <>
struct RequestTraits<clice::worker::CodeActionParams> {
using Result = eventide::serde::RawValue;
constexpr inline static std::string_view method = "clice/worker/codeAction";
};
template <>
struct RequestTraits<clice::worker::GoToDefinitionParams> {
using Result = eventide::serde::RawValue;
constexpr inline static std::string_view method = "clice/worker/goToDefinition";
};
// === Stateless Requests ===
template <>
struct RequestTraits<clice::worker::CompletionParams> {
using Result = eventide::serde::RawValue;
constexpr inline static std::string_view method = "clice/worker/completion";
};
template <>
struct RequestTraits<clice::worker::SignatureHelpParams> {
using Result = eventide::serde::RawValue;
constexpr inline static std::string_view method = "clice/worker/signatureHelp";
};
template <>
struct RequestTraits<clice::worker::BuildPCHParams> {
using Result = clice::worker::BuildPCHResult;
constexpr inline static std::string_view method = "clice/worker/buildPCH";
};
template <>
struct RequestTraits<clice::worker::BuildPCMParams> {
using Result = clice::worker::BuildPCMResult;
constexpr inline static std::string_view method = "clice/worker/buildPCM";
};
template <>
struct RequestTraits<clice::worker::IndexParams> {
using Result = clice::worker::IndexResult;
constexpr inline static std::string_view method = "clice/worker/index";
};
// === Notifications ===
template <>
struct NotificationTraits<clice::worker::DocumentUpdateParams> {
constexpr inline static std::string_view method = "clice/worker/documentUpdate";
@@ -194,4 +258,4 @@ struct NotificationTraits<clice::worker::EvictedParams> {
constexpr inline static std::string_view method = "clice/worker/evicted";
};
} // namespace kota::ipc::protocol
} // namespace eventide::ipc::protocol

View File

@@ -1,82 +0,0 @@
#pragma once
#include <cstdint>
#include <memory>
#include <optional>
#include <string>
#include "server/workspace.h"
#include "kota/async/async.h"
#include "llvm/ADT/SmallVector.h"
namespace clice {
/// An editing session for a single file opened in the editor.
///
/// Design principle: open files are never depended upon by other files.
/// Dependencies always point to disk files. The only path from Session
/// to Workspace is didSave, which tells Workspace to rescan the disk file.
///
/// Created on didOpen, destroyed on didClose. All fields are local to this
/// file's translation unit and NEVER leak to Workspace or other Sessions.
/// Sessions may READ from Workspace (e.g. to obtain PCH/PCM paths, module
/// mappings, include graph) but all compilation results stay here.
struct Session {
/// Path ID of this file in PathPool. Set on creation, never changes.
std::uint32_t path_id = 0;
/// LSP document version, incremented by the client on each edit.
int version = 0;
/// Current buffer content (may differ from disk until saved).
std::string text;
/// Monotonic generation counter, incremented on every didChange.
/// Used to detect stale compilation results (ABA prevention).
std::uint64_t generation = 0;
/// Whether the AST needs to be rebuilt before serving queries.
bool ast_dirty = true;
/// Non-null while a compilation is in flight for this file.
/// Other queries wait on the event; the compilation task itself
/// runs independently and cannot be cancelled by LSP $/cancelRequest.
struct PendingCompile {
kota::event done;
bool succeeded = false;
};
std::shared_ptr<PendingCompile> compiling;
/// Reference to the PCH entry in Workspace.pch_cache, if any.
/// The PCH itself is owned by Workspace (shared, content-addressed);
/// Session only stores enough to locate and validate it.
struct PCHRef {
std::uint32_t path_id = 0; ///< Key into Workspace.pch_cache.
std::uint64_t hash = 0; ///< Preamble hash at build time.
std::uint32_t bound = 0; ///< Preamble byte boundary.
};
std::optional<PCHRef> pch_ref;
/// Dependency snapshot from the last successful AST compilation.
/// Used for two-layer staleness detection (mtime + content hash).
std::optional<DepsSnapshot> ast_deps;
/// Compilation context for header files that lack their own CDB entry.
/// Stores the host source file and synthesized preamble for this header.
std::optional<HeaderFileContext> header_context;
/// User-selected compilation context override (via clice/switchContext).
/// When set, overrides automatic header context resolution.
std::optional<std::uint32_t> active_context;
/// Symbol index built from the latest compilation of this file's buffer.
/// Used for queries (hover, goto, references) on this file.
/// NOT merged into Workspace.project_index — that only gets disk-derived
/// data from background indexing.
std::optional<OpenFileIndex> file_index;
};
} // namespace clice

View File

@@ -1,6 +1,7 @@
#include "server/stateful_worker.h"
#include <atomic>
#include <chrono>
#include <cstdint>
#include <list>
#include <memory>
@@ -8,23 +9,23 @@
#include <vector>
#include "compile/compilation.h"
#include "eventide/async/async.h"
#include "eventide/ipc/json_codec.h"
#include "eventide/ipc/peer.h"
#include "eventide/ipc/transport.h"
#include "eventide/serde/json/serializer.h"
#include "eventide/serde/serde/raw_value.h"
#include "feature/feature.h"
#include "index/tu_index.h"
#include "server/protocol.h"
#include "server/worker_common.h"
#include "support/logging.h"
#include "kota/async/async.h"
#include "kota/ipc/codec/bincode.h"
#include "kota/ipc/peer.h"
#include "kota/ipc/transport.h"
#include "llvm/ADT/StringMap.h"
#include "llvm/Support/raw_ostream.h"
namespace clice {
using kota::ipc::RequestResult;
using RequestContext = kota::ipc::BincodePeer::RequestContext;
namespace et = eventide;
using et::ipc::RequestResult;
using RequestContext = et::ipc::BincodePeer::RequestContext;
struct DocumentEntry {
int version = 0;
@@ -35,7 +36,7 @@ struct DocumentEntry {
// Signaled when the first compilation completes (has_ast becomes true).
// Feature handlers co_await this before accessing the AST.
kota::event ast_ready{false};
et::event ast_ready{false};
// Compilation context (from CompileParams)
std::string directory;
@@ -44,14 +45,40 @@ struct DocumentEntry {
llvm::StringMap<std::string> pcms;
// Per-document serialization mutex
kota::mutex strand;
et::mutex strand;
};
struct ScopedTimer {
std::chrono::steady_clock::time_point start = std::chrono::steady_clock::now();
long long ms() const {
return std::chrono::duration_cast<std::chrono::milliseconds>(
std::chrono::steady_clock::now() - start)
.count();
}
};
static void fill_args(CompilationParams& cp,
const std::string& directory,
const std::vector<std::string>& arguments) {
cp.directory = directory;
for(auto& arg: arguments) {
cp.arguments.push_back(arg.c_str());
}
}
/// Serialize any value to LSP JSON RawValue.
template <typename T>
static et::serde::RawValue to_raw(const T& value) {
auto json = et::serde::json::to_json<et::ipc::lsp_config>(value);
return et::serde::RawValue{json ? std::move(*json) : "null"};
}
class StatefulWorker {
kota::ipc::BincodePeer& peer;
et::ipc::BincodePeer& peer;
std::uint64_t memory_limit;
llvm::StringMap<std::shared_ptr<DocumentEntry>> documents;
llvm::StringMap<std::unique_ptr<DocumentEntry>> documents;
// LRU tracking — owns keys so they don't dangle after request handler returns
std::list<std::string> lru;
@@ -79,43 +106,41 @@ class StatefulWorker {
}
}
std::shared_ptr<DocumentEntry> get_or_create(llvm::StringRef path) {
DocumentEntry& get_or_create(llvm::StringRef path) {
auto [it, inserted] = documents.try_emplace(path, nullptr);
if(inserted) {
it->second = std::make_shared<DocumentEntry>();
it->second = std::make_unique<DocumentEntry>();
LOG_DEBUG("Created new document entry: {}", path.str());
}
return it->second;
return *it->second;
}
/// Look up document, wait for AST, lock strand, run fn(doc) on thread pool, unlock.
/// Returns "null" if document not found or AST not usable.
template <typename F>
kota::task<kota::codec::RawValue> with_ast(llvm::StringRef path, F&& fn) {
et::task<et::serde::RawValue> with_ast(llvm::StringRef path, F&& fn) {
auto it = documents.find(path);
if(it == documents.end()) {
co_return kota::codec::RawValue{"null"};
}
if(it == documents.end())
co_return et::serde::RawValue{"null"};
// Hold shared_ptr so Evict can't destroy the entry mid-request.
auto doc = it->second;
auto& doc = *it->second;
touch_lru(path);
co_await doc->ast_ready.wait();
co_await doc->strand.lock();
co_await doc.ast_ready.wait();
co_await doc.strand.lock();
auto result = co_await kota::queue([&]() -> kota::codec::RawValue {
if(!doc->has_ast || (!doc->unit.completed() && !doc->unit.fatal_error()))
return kota::codec::RawValue{"null"};
return fn(*doc);
auto result = co_await et::queue([&]() -> et::serde::RawValue {
if(!doc.has_ast || (!doc.unit.completed() && !doc.unit.fatal_error()))
return et::serde::RawValue{"null"};
return fn(doc);
});
doc->strand.unlock();
doc.strand.unlock();
co_return result.value();
}
public:
StatefulWorker(kota::ipc::BincodePeer& peer, std::uint64_t memory_limit) :
StatefulWorker(et::ipc::BincodePeer& peer, std::uint64_t memory_limit) :
peer(peer), memory_limit(memory_limit) {}
void register_handlers();
@@ -128,83 +153,68 @@ void StatefulWorker::register_handlers() {
const worker::CompileParams& params) -> RequestResult<worker::CompileParams> {
LOG_INFO("Compile request: path={}, version={}", params.path, params.version);
// Hold shared_ptr so Evict can't destroy the entry mid-compile.
auto doc = get_or_create(params.path);
touch_lru(params.path);
co_await doc->strand.lock();
// Copy params to doc AFTER acquiring the strand lock, so that
// concurrent Compile requests waiting on the strand don't
// overwrite our fields before we use them.
doc->version = params.version;
doc->text = params.text;
doc->directory = params.directory;
doc->arguments = params.arguments;
doc->pch = params.pch;
doc->pcms.clear();
auto& doc = get_or_create(params.path);
doc.version = params.version;
doc.text = params.text;
doc.directory = params.directory;
doc.arguments = params.arguments;
doc.pch = params.pch;
doc.pcms.clear();
for(auto& [name, pcm_path]: params.pcms) {
doc->pcms.try_emplace(name, pcm_path);
doc.pcms.try_emplace(name, pcm_path);
}
auto compile_result = co_await kota::queue([&]() -> worker::CompileResult {
touch_lru(params.path);
co_await doc.strand.lock();
auto compile_result = co_await et::queue([&]() -> worker::CompileResult {
LOG_DEBUG("Compiling: path={}, {} args", params.path, doc.arguments.size());
ScopedTimer timer;
CompilationParams cp;
cp.kind = CompilationKind::Content;
fill_args(cp, doc->directory, doc->arguments);
if(!doc->pch.first.empty()) {
cp.pch = doc->pch;
fill_args(cp, doc.directory, doc.arguments);
if(!doc.pch.first.empty()) {
cp.pch = doc.pch;
}
cp.add_remapped_file(params.path, doc->text);
for(auto& entry: doc->pcms) {
cp.add_remapped_file(params.path, doc.text);
for(auto& entry: doc.pcms) {
cp.pcms.try_emplace(entry.getKey(), entry.getValue());
}
doc->unit = compile(cp);
doc->has_ast = true;
doc->dirty.store(false, std::memory_order_release);
doc.unit = compile(cp);
doc.has_ast = true;
doc.dirty.store(false, std::memory_order_release);
worker::CompileResult result;
result.version = doc->version;
if(doc->unit.completed() || doc->unit.fatal_error()) {
auto diags = feature::diagnostics(doc->unit);
auto json = kota::codec::json::to_json<kota::ipc::lsp_config>(diags);
result.diagnostics = kota::codec::RawValue{json ? std::move(*json) : "[]"};
result.version = doc.version;
if(doc.unit.completed() || doc.unit.fatal_error()) {
auto diags = feature::diagnostics(doc.unit);
auto json = et::serde::json::to_json<et::ipc::lsp_config>(diags);
result.diagnostics = et::serde::RawValue{json ? std::move(*json) : "[]"};
LOG_INFO("Compile done: path={}, {}ms, {} diags, fatal={}",
params.path,
timer.ms(),
diags.size(),
doc->unit.fatal_error());
doc.unit.fatal_error());
} else {
result.diagnostics = kota::codec::RawValue{"[]"};
result.diagnostics = et::serde::RawValue{"[]"};
LOG_WARN("Compile incomplete: path={}, {}ms", params.path, timer.ms());
}
result.memory_usage = 0; // TODO: query actual memory
if(doc->unit.completed()) {
result.deps = doc->unit.deps();
// Build index for main file only (interested_only=true).
auto tu_index = index::TUIndex::build(doc->unit, true);
llvm::raw_string_ostream os(result.tu_index_data);
tu_index.serialize(os);
}
return result;
});
doc->strand.unlock();
doc->ast_ready.set();
doc.strand.unlock();
doc.ast_ready.set();
shrink_if_over_limit();
co_return compile_result.value();
});
// === DocumentUpdate ===
// Only mark the document dirty — do NOT update doc.text or doc.version
// here. The kota::queue compilation work may be reading doc.text on the
// thread pool concurrently, so writing it from the event loop would be
// a data race. The next Compile request will bring the correct text
// and update it inside the strand lock.
peer.on_notification([this](const worker::DocumentUpdateParams& params) {
LOG_TRACE("DocumentUpdate: path={}, version={}", params.path, params.version);
@@ -214,7 +224,10 @@ void StatefulWorker::register_handlers() {
return;
}
it->second->dirty.store(true, std::memory_order_release);
auto& doc = *it->second;
doc.version = params.version;
doc.text = params.text;
doc.dirty.store(true, std::memory_order_release);
});
// === Evict ===
@@ -229,70 +242,90 @@ void StatefulWorker::register_handlers() {
documents.erase(params.path);
});
// === Query (hover, definition, semantic tokens, etc.) ===
// === Hover ===
peer.on_request(
[this](RequestContext& ctx,
const worker::QueryParams& params) -> RequestResult<worker::QueryParams> {
using K = worker::QueryKind;
switch(params.kind) {
case K::Hover:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
auto result = feature::hover(doc.unit, params.offset);
return result ? to_raw(*result) : kota::codec::RawValue{"null"};
});
case K::GoToDefinition:
// TODO: Implement go-to-definition
co_return kota::codec::RawValue{"[]"};
case K::SemanticTokens:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::semantic_tokens(doc.unit));
});
case K::InlayHints:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
auto range = params.range;
if(range.begin == static_cast<uint32_t>(-1))
range = LocalSourceRange{0, static_cast<uint32_t>(doc.text.size())};
return to_raw(feature::inlay_hints(doc.unit, range));
});
case K::FoldingRange:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::folding_ranges(doc.unit));
});
case K::DocumentSymbol:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::document_symbols(doc.unit));
});
case K::DocumentLink:
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::document_links(doc.unit));
});
case K::CodeAction:
// TODO: Implement code actions
co_return kota::codec::RawValue{"[]"};
}
co_return kota::codec::RawValue{"null"};
const worker::HoverParams& params) -> RequestResult<worker::HoverParams> {
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
auto result = feature::hover(doc.unit, params.offset);
return result ? to_raw(*result) : et::serde::RawValue{"null"};
});
});
// === SemanticTokens ===
peer.on_request([this](RequestContext& ctx, const worker::SemanticTokensParams& params)
-> RequestResult<worker::SemanticTokensParams> {
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::semantic_tokens(doc.unit));
});
});
// === InlayHints ===
peer.on_request(
[this](RequestContext& ctx,
const worker::InlayHintsParams& params) -> RequestResult<worker::InlayHintsParams> {
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
LocalSourceRange range{0, static_cast<uint32_t>(doc.text.size())};
return to_raw(feature::inlay_hints(doc.unit, range));
});
});
// === FoldingRange ===
peer.on_request([this](RequestContext& ctx, const worker::FoldingRangeParams& params)
-> RequestResult<worker::FoldingRangeParams> {
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::folding_ranges(doc.unit));
});
});
// === DocumentSymbol ===
peer.on_request([this](RequestContext& ctx, const worker::DocumentSymbolParams& params)
-> RequestResult<worker::DocumentSymbolParams> {
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::document_symbols(doc.unit));
});
});
// === DocumentLink ===
peer.on_request([this](RequestContext& ctx, const worker::DocumentLinkParams& params)
-> RequestResult<worker::DocumentLinkParams> {
co_return co_await with_ast(params.path, [&](DocumentEntry& doc) {
return to_raw(feature::document_links(doc.unit));
});
});
// === CodeAction ===
peer.on_request(
[this](RequestContext& ctx,
const worker::CodeActionParams& params) -> RequestResult<worker::CodeActionParams> {
LOG_TRACE("CodeAction request: path={}", params.path);
// TODO: Implement code actions
co_return et::serde::RawValue{"[]"};
});
// === GoToDefinition ===
peer.on_request([this](RequestContext& ctx, const worker::GoToDefinitionParams& params)
-> RequestResult<worker::GoToDefinitionParams> {
LOG_TRACE("GoToDefinition request: path={}, offset={}", params.path, params.offset);
// TODO: Implement go-to-definition
co_return et::serde::RawValue{"[]"};
});
}
int run_stateful_worker_mode(std::uint64_t memory_limit,
const std::string& worker_name,
const std::string& log_dir) {
logging::stderr_logger(worker_name, logging::options);
if(!log_dir.empty()) {
logging::file_logger(worker_name, log_dir, logging::options);
}
int run_stateful_worker_mode(std::uint64_t memory_limit) {
logging::stderr_logger("stateful-worker", logging::options);
LOG_INFO("Starting stateful worker, memory_limit={}MB", memory_limit / (1024 * 1024));
kota::event_loop loop;
et::event_loop loop;
auto transport_result = kota::ipc::StreamTransport::open_stdio(loop);
auto transport_result = et::ipc::StreamTransport::open_stdio(loop);
if(!transport_result) {
LOG_ERROR("Failed to open stdio transport");
return 1;
}
kota::ipc::BincodePeer peer(loop, std::move(*transport_result));
et::ipc::BincodePeer peer(loop, std::move(*transport_result));
StatefulWorker worker(peer, memory_limit);
worker.register_handlers();

View File

@@ -1,15 +1,12 @@
#pragma once
#include <cstdint>
#include <string>
namespace clice {
/// Run the stateful worker process mode.
/// The worker holds compiled ASTs and handles feature requests
/// (hover, semantic tokens, etc.) alongside compile requests.
int run_stateful_worker_mode(std::uint64_t memory_limit,
const std::string& worker_name,
const std::string& log_dir);
int run_stateful_worker_mode(std::uint64_t memory_limit);
} // namespace clice

View File

@@ -1,293 +1,231 @@
#include "server/stateless_worker.h"
#include <chrono>
#include "compile/compilation.h"
#include "eventide/async/async.h"
#include "eventide/ipc/json_codec.h"
#include "eventide/ipc/peer.h"
#include "eventide/ipc/transport.h"
#include "eventide/serde/json/serializer.h"
#include "eventide/serde/serde/raw_value.h"
#include "feature/feature.h"
#include "index/tu_index.h"
#include "server/protocol.h"
#include "server/worker_common.h"
#include "support/logging.h"
#include "kota/async/async.h"
#include "kota/ipc/codec/bincode.h"
#include "kota/ipc/peer.h"
#include "kota/ipc/transport.h"
#include "llvm/Support/raw_ostream.h"
namespace clice {
using kota::ipc::RequestResult;
using RequestContext = kota::ipc::BincodePeer::RequestContext;
namespace et = eventide;
using et::ipc::RequestResult;
using RequestContext = et::ipc::BincodePeer::RequestContext;
/// Extract error messages from compilation diagnostics.
static std::string collect_errors(CompilationUnit& unit) {
std::string errors;
for(auto& diag: unit.diagnostics()) {
if(diag.id.level >= DiagnosticLevel::Error) {
if(!errors.empty())
errors += "; ";
errors += diag.message;
}
struct ScopedTimer {
std::chrono::steady_clock::time_point start = std::chrono::steady_clock::now();
long long ms() const {
return std::chrono::duration_cast<std::chrono::milliseconds>(
std::chrono::steady_clock::now() - start)
.count();
}
return errors;
}
};
/// Build a TUIndex, serialize it, and return as a string.
static std::string serialize_tu_index(CompilationUnit& unit, bool interested_only = false) {
auto tu_index = index::TUIndex::build(unit, interested_only);
if(!interested_only) {
tu_index.main_file_index = index::FileIndex();
}
std::string serialized;
llvm::raw_string_ostream os(serialized);
tu_index.serialize(os);
return serialized;
}
/// Write compilation output to disk, handling tmp+rename pattern.
/// Returns the final path on success, or empty string on failure.
static std::string finalize_output(const std::string& output_path,
const std::string& tmp_path,
const std::string& file,
const char* label) {
if(!output_path.empty()) {
auto ec = fs::rename(tmp_path, output_path);
if(ec) {
return output_path;
} else {
LOG_WARN("{}: rename {} -> {} failed: {}",
label,
tmp_path,
output_path,
ec.error().message());
return tmp_path;
}
}
return tmp_path;
}
static worker::BuildResult handle_build_pch(const worker::BuildParams& params) {
ScopedTimer timer;
CompilationParams cp;
cp.kind = CompilationKind::Preamble;
fill_args(cp, params.directory, params.arguments);
cp.add_remapped_file(params.file, params.text, params.preamble_bound);
std::string tmp_path;
bool has_output = !params.output_path.empty();
if(has_output) {
tmp_path = params.output_path + ".tmp";
} else {
auto tmp = fs::createTemporaryFile("clice-pch", "pch");
if(!tmp) {
LOG_ERROR("BuildPCH: failed to create temp file");
return {false, "Failed to create temporary PCH file"};
}
tmp_path = *tmp;
}
cp.output_file = tmp_path;
PCHInfo pch_info;
auto unit = compile(cp, pch_info);
bool success = unit.completed();
std::string errors;
if(!success)
errors = collect_errors(unit);
std::string tu_index_data;
std::string pch_links_json;
if(success) {
tu_index_data = serialize_tu_index(unit);
auto links = feature::document_links(unit);
auto raw = to_raw(links);
pch_links_json = std::move(raw.data);
}
// Destroy CompilationUnit to flush PCH to disk.
unit = CompilationUnit(nullptr);
if(success) {
auto final_path = finalize_output(params.output_path, tmp_path, params.file, "BuildPCH");
LOG_INFO("BuildPCH done: file={}, output={}, {}ms", params.file, final_path, timer.ms());
worker::BuildResult result;
result.success = true;
result.output_path = std::move(final_path);
result.deps = pch_info.deps;
result.tu_index_data = std::move(tu_index_data);
result.pch_links_json = std::move(pch_links_json);
return result;
} else {
LOG_WARN("BuildPCH failed: file={}, {}ms, errors=[{}]", params.file, timer.ms(), errors);
fs::remove(tmp_path);
return {false, errors.empty() ? "PCH compilation failed" : errors};
static void fill_args(CompilationParams& cp,
const std::string& directory,
const std::vector<std::string>& arguments) {
cp.directory = directory;
for(auto& arg: arguments) {
cp.arguments.push_back(arg.c_str());
}
}
static worker::BuildResult handle_build_pcm(const worker::BuildParams& params) {
ScopedTimer timer;
CompilationParams cp;
cp.kind = CompilationKind::ModuleInterface;
fill_args(cp, params.directory, params.arguments);
for(auto& [name, path]: params.pcms) {
cp.pcms.try_emplace(name, path);
}
std::string tmp_path;
bool has_output = !params.output_path.empty();
if(has_output) {
tmp_path = params.output_path + ".tmp";
} else {
auto tmp = fs::createTemporaryFile("clice-pcm", "pcm");
if(!tmp) {
LOG_ERROR("BuildPCM: failed to create temp file");
return {false, "Failed to create temporary PCM file"};
}
tmp_path = *tmp;
}
cp.output_file = tmp_path;
PCMInfo pcm_info;
auto unit = compile(cp, pcm_info);
bool success = unit.completed();
std::string errors;
if(!success)
errors = collect_errors(unit);
std::string tu_index_data;
if(success)
tu_index_data = serialize_tu_index(unit, true);
unit = CompilationUnit(nullptr);
if(success) {
auto final_path = finalize_output(params.output_path, tmp_path, params.file, "BuildPCM");
LOG_INFO("BuildPCM done: module={}, {}ms", params.module_name, timer.ms());
worker::BuildResult result;
result.success = true;
result.output_path = std::move(final_path);
result.deps = pcm_info.deps;
result.tu_index_data = std::move(tu_index_data);
return result;
} else {
LOG_WARN("BuildPCM failed: module={}, {}ms, errors=[{}]",
params.module_name,
timer.ms(),
errors);
fs::remove(tmp_path);
return {false, errors.empty() ? "PCM compilation failed" : errors};
}
template <typename T>
static et::serde::RawValue to_raw(const T& value) {
auto json = et::serde::json::to_json<et::ipc::lsp_config>(value);
return et::serde::RawValue{json ? std::move(*json) : "null"};
}
static worker::BuildResult handle_index(const worker::BuildParams& params) {
ScopedTimer timer;
CompilationParams cp;
cp.kind = CompilationKind::Indexing;
fill_args(cp, params.directory, params.arguments);
for(auto& [name, path]: params.pcms) {
cp.pcms.try_emplace(name, path);
}
auto unit = compile(cp);
if(!unit.completed()) {
LOG_WARN("Index failed: file={}, {}ms", params.file, timer.ms());
return {false, "Index compilation failed"};
}
auto tu_index = index::TUIndex::build(unit);
std::string serialized;
llvm::raw_string_ostream os(serialized);
tu_index.serialize(os);
LOG_INFO("Index done: file={}, {} symbols, {}ms",
params.file,
tu_index.symbols.size(),
timer.ms());
worker::BuildResult result;
result.success = true;
result.tu_index_data = std::move(serialized);
return result;
}
static worker::BuildResult handle_completion(const worker::BuildParams& params) {
ScopedTimer timer;
CompilationParams cp;
cp.kind = CompilationKind::Completion;
fill_args(cp, params.directory, params.arguments);
if(!params.pch.first.empty()) {
cp.pch = params.pch;
}
for(auto& [name, path]: params.pcms) {
cp.pcms.try_emplace(name, path);
}
cp.add_remapped_file(params.file, params.text);
cp.completion = {params.file, params.offset};
auto items = feature::code_complete(cp);
LOG_DEBUG("Completion done: {} items, {}ms", items.size(), timer.ms());
worker::BuildResult result;
result.result_json = to_raw(items);
return result;
}
static worker::BuildResult handle_signature_help(const worker::BuildParams& params) {
ScopedTimer timer;
CompilationParams cp;
cp.kind = CompilationKind::Completion;
fill_args(cp, params.directory, params.arguments);
if(!params.pch.first.empty()) {
cp.pch = params.pch;
}
for(auto& [name, path]: params.pcms) {
cp.pcms.try_emplace(name, path);
}
cp.add_remapped_file(params.file, params.text);
cp.completion = {params.file, params.offset};
auto help = feature::signature_help(cp);
LOG_DEBUG("SignatureHelp done: {}ms", timer.ms());
worker::BuildResult result;
result.result_json = to_raw(help);
return result;
}
int run_stateless_worker_mode(const std::string& worker_name, const std::string& log_dir) {
logging::stderr_logger(worker_name, logging::options);
if(!log_dir.empty()) {
logging::file_logger(worker_name, log_dir, logging::options);
}
int run_stateless_worker_mode() {
logging::stderr_logger("stateless-worker", logging::options);
LOG_INFO("Starting stateless worker");
kota::event_loop loop;
et::event_loop loop;
auto transport_result = kota::ipc::StreamTransport::open_stdio(loop);
auto transport_result = et::ipc::StreamTransport::open_stdio(loop);
if(!transport_result) {
LOG_ERROR("Failed to open stdio transport");
return 1;
}
kota::ipc::BincodePeer peer(loop, std::move(*transport_result));
et::ipc::BincodePeer peer(loop, std::move(*transport_result));
peer.on_request([&](RequestContext& ctx,
const worker::BuildParams& params) -> RequestResult<worker::BuildParams> {
using K = worker::BuildKind;
auto result = co_await kota::queue([&]() -> worker::BuildResult {
switch(params.kind) {
case K::BuildPCH: return handle_build_pch(params);
case K::BuildPCM: return handle_build_pcm(params);
case K::Index: return handle_index(params);
case K::Completion: return handle_completion(params);
case K::SignatureHelp: return handle_signature_help(params);
// === BuildPCH ===
peer.on_request(
[&](RequestContext& ctx,
const worker::BuildPCHParams& params) -> RequestResult<worker::BuildPCHParams> {
LOG_INFO("BuildPCH request: file={}", params.file);
auto result = co_await et::queue([&]() -> worker::BuildPCHResult {
ScopedTimer timer;
CompilationParams cp;
cp.kind = CompilationKind::Preamble;
fill_args(cp, params.directory, params.arguments);
cp.add_remapped_file(params.file, params.content, params.preamble_bound);
auto tmp = fs::createTemporaryFile("clice-pch", "pch");
if(!tmp) {
LOG_ERROR("BuildPCH: failed to create temp file");
return {false, "Failed to create temporary PCH file", ""};
}
cp.output_file = *tmp;
PCHInfo pch_info;
auto unit = compile(cp, pch_info);
if(unit.completed()) {
LOG_INFO("BuildPCH done: file={}, output={}, {}ms",
params.file,
cp.output_file,
timer.ms());
return {true, "", std::string(cp.output_file)};
} else {
LOG_WARN("BuildPCH failed: file={}, {}ms", params.file, timer.ms());
fs::remove(cp.output_file);
return {false, "PCH compilation failed", ""};
}
});
co_return result.value();
});
// === BuildPCM ===
peer.on_request(
[&](RequestContext& ctx,
const worker::BuildPCMParams& params) -> RequestResult<worker::BuildPCMParams> {
LOG_INFO("BuildPCM request: file={}, module={}", params.file, params.module_name);
auto result = co_await et::queue([&]() -> worker::BuildPCMResult {
ScopedTimer timer;
CompilationParams cp;
cp.kind = CompilationKind::ModuleInterface;
fill_args(cp, params.directory, params.arguments);
for(auto& [name, path]: params.pcms) {
cp.pcms.try_emplace(name, path);
}
auto tmp = fs::createTemporaryFile("clice-pcm", "pcm");
if(!tmp) {
LOG_ERROR("BuildPCM: failed to create temp file");
return {false, "Failed to create temporary PCM file"};
}
cp.output_file = *tmp;
PCMInfo pcm_info;
auto unit = compile(cp, pcm_info);
if(unit.completed()) {
LOG_INFO("BuildPCM done: module={}, {}ms", params.module_name, timer.ms());
return {true, "", std::string(cp.output_file)};
} else {
LOG_WARN("BuildPCM failed: module={}, {}ms", params.module_name, timer.ms());
return {false, "PCM compilation failed", ""};
}
});
co_return result.value();
});
// === Completion ===
peer.on_request(
[&](RequestContext& ctx,
const worker::CompletionParams& params) -> RequestResult<worker::CompletionParams> {
LOG_DEBUG("Completion request: path={}, offset={}", params.path, params.offset);
auto result = co_await et::queue([&]() -> et::serde::RawValue {
ScopedTimer timer;
CompilationParams cp;
cp.kind = CompilationKind::Completion;
fill_args(cp, params.directory, params.arguments);
if(!params.pch.first.empty()) {
cp.pch = params.pch;
}
for(auto& [name, path]: params.pcms) {
cp.pcms.try_emplace(name, path);
}
cp.add_remapped_file(params.path, params.text);
cp.completion = {params.path, params.offset};
auto items = feature::code_complete(cp);
LOG_DEBUG("Completion done: {} items, {}ms", items.size(), timer.ms());
return to_raw(items);
});
co_return result.value();
});
// === SignatureHelp ===
peer.on_request([&](RequestContext& ctx, const worker::SignatureHelpParams& params)
-> RequestResult<worker::SignatureHelpParams> {
LOG_DEBUG("SignatureHelp request: path={}, offset={}", params.path, params.offset);
auto result = co_await et::queue([&]() -> et::serde::RawValue {
ScopedTimer timer;
CompilationParams cp;
cp.kind = CompilationKind::Completion;
fill_args(cp, params.directory, params.arguments);
if(!params.pch.first.empty()) {
cp.pch = params.pch;
}
return {false, "Unknown build kind"};
for(auto& [name, path]: params.pcms) {
cp.pcms.try_emplace(name, path);
}
cp.add_remapped_file(params.path, params.text);
cp.completion = {params.path, params.offset};
auto help = feature::signature_help(cp);
LOG_DEBUG("SignatureHelp done: {}ms", timer.ms());
return to_raw(help);
});
co_return result.value();
});
// === Index ===
peer.on_request([&](RequestContext& ctx,
const worker::IndexParams& params) -> RequestResult<worker::IndexParams> {
LOG_INFO("Index request: file={}", params.file);
auto result = co_await et::queue([&]() -> worker::IndexResult {
ScopedTimer timer;
CompilationParams cp;
cp.kind = CompilationKind::Indexing;
fill_args(cp, params.directory, params.arguments);
for(auto& [name, path]: params.pcms) {
cp.pcms.try_emplace(name, path);
}
auto unit = compile(cp);
if(!unit.completed()) {
LOG_WARN("Index failed: file={}, {}ms", params.file, timer.ms());
return {false, "Index compilation failed", ""};
}
auto tu_index = index::TUIndex::build(unit);
std::string serialized;
llvm::raw_string_ostream os(serialized);
tu_index.serialize(os);
LOG_INFO("Index done: file={}, {} symbols, {}ms",
params.file,
tu_index.symbols.size(),
timer.ms());
return {true, "", std::move(serialized)};
});
co_return result.value();
});

View File

@@ -1,13 +1,11 @@
#pragma once
#include <string>
namespace clice {
/// Run the stateless worker process mode.
/// The worker receives one-shot compilation tasks (BuildPCH, BuildPCM,
/// Completion, SignatureHelp, Index) via stdin/stdout bincode IPC,
/// executes them on a thread pool, and returns results.
int run_stateless_worker_mode(const std::string& worker_name, const std::string& log_dir);
int run_stateless_worker_mode();
} // namespace clice

View File

@@ -1,44 +0,0 @@
#pragma once
/// Shared utilities for stateful and stateless worker processes.
#include <chrono>
#include <string>
#include <vector>
#include "compile/compilation.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
namespace clice {
/// RAII timer for measuring elapsed milliseconds.
struct ScopedTimer {
std::chrono::steady_clock::time_point start = std::chrono::steady_clock::now();
long long ms() const {
return std::chrono::duration_cast<std::chrono::milliseconds>(
std::chrono::steady_clock::now() - start)
.count();
}
};
/// Fill CompilationParams directory and arguments from worker request fields.
inline void fill_args(CompilationParams& cp,
const std::string& directory,
const std::vector<std::string>& arguments) {
cp.directory = directory;
for(auto& arg: arguments) {
cp.arguments.push_back(arg.c_str());
}
}
/// Serialize a value to JSON RawValue using LSP config.
template <typename T>
inline kota::codec::RawValue to_raw(const T& value) {
auto json = kota::codec::json::to_json<kota::ipc::lsp_config>(value);
return kota::codec::RawValue{json ? std::move(*json) : "null"};
}
} // namespace clice

View File

@@ -3,22 +3,21 @@
#include <csignal>
#include <string>
#include "eventide/ipc/transport.h"
#include "support/logging.h"
#include "kota/ipc/transport.h"
namespace clice {
namespace {
/// Coroutine that drains a worker's stderr pipe.
/// Workers write their own log files, so this only captures unexpected output
/// (crash stacktraces, assertion failures, etc.) that bypasses spdlog.
kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
/// Coroutine that reads lines from a worker's stderr pipe and logs them
/// with a prefix like [SL-0] or [SF-1].
et::task<> drain_stderr(et::pipe stderr_pipe, std::string prefix) {
std::string buffer;
while(true) {
auto result = co_await stderr_pipe.read();
if(!result.has_value()) {
// EOF or error — worker has exited
break;
}
auto& chunk = result.value();
@@ -27,6 +26,7 @@ kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
buffer += chunk;
// Log complete lines
std::size_t pos = 0;
while(true) {
auto nl = buffer.find('\n', pos);
@@ -34,15 +34,16 @@ kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
break;
auto line = buffer.substr(pos, nl - pos);
if(!line.empty()) {
LOG_DEBUG("{} {}", prefix, line);
LOG_INFO("{} {}", prefix, line);
}
pos = nl + 1;
}
buffer.erase(0, pos);
}
// Flush any remaining partial line
if(!buffer.empty()) {
LOG_DEBUG("{} {}", prefix, buffer);
LOG_INFO("{} {}", prefix, buffer);
}
}
@@ -51,11 +52,7 @@ kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
bool WorkerPool::spawn_worker(const std::string& self_path,
bool stateful,
std::uint64_t memory_limit) {
auto& workers = stateful ? stateful_workers : stateless_workers;
auto worker_index = workers.size();
std::string worker_name = std::string(stateful ? "SF-" : "SL-") + std::to_string(worker_index);
kota::process::options opts;
et::process::options opts;
opts.file = self_path;
if(stateful) {
opts.args = {self_path,
@@ -66,22 +63,13 @@ bool WorkerPool::spawn_worker(const std::string& self_path,
} else {
opts.args = {self_path, "--mode", "stateless-worker"};
}
opts.args.push_back("--worker-name");
opts.args.push_back(worker_name);
if(!log_dir_.empty()) {
opts.args.push_back("--log-dir");
opts.args.push_back(log_dir_);
}
opts.streams = {
kota::process::stdio::pipe(true, false), // stdin: child reads
kota::process::stdio::pipe(false, true), // stdout: child writes
kota::process::stdio::pipe(false, true), // stderr: child writes
et::process::stdio::pipe(true, false), // stdin: child reads
et::process::stdio::pipe(false, true), // stdout: child writes
et::process::stdio::pipe(false, true), // stderr: child writes
};
auto result = kota::process::spawn(opts, loop);
auto result = et::process::spawn(opts, loop);
if(!result) {
LOG_ERROR("Failed to spawn {} worker: {}",
stateful ? "stateful" : "stateless",
@@ -93,12 +81,18 @@ bool WorkerPool::spawn_worker(const std::string& self_path,
// StreamTransport: input = child's stdout (parent reads), output = child's stdin (parent
// writes)
auto transport = std::make_unique<kota::ipc::StreamTransport>(std::move(spawn.stdout_pipe),
std::move(spawn.stdin_pipe));
auto peer = std::make_unique<kota::ipc::BincodePeer>(loop, std::move(transport));
auto transport = std::make_unique<et::ipc::StreamTransport>(std::move(spawn.stdout_pipe),
std::move(spawn.stdin_pipe));
auto peer = std::make_unique<et::ipc::BincodePeer>(loop, std::move(transport));
auto& workers = stateful ? stateful_workers : stateless_workers;
auto worker_index = workers.size();
// Build log prefix: [SF-0] for stateful, [SL-0] for stateless
std::string prefix =
std::string("[") + (stateful ? "SF-" : "SL-") + std::to_string(worker_index) + "]";
// Schedule stderr log collection
std::string prefix = "[" + worker_name + "]";
loop.schedule(drain_stderr(std::move(spawn.stderr_pipe), prefix));
workers.push_back(WorkerProcess{
@@ -114,8 +108,6 @@ bool WorkerPool::spawn_worker(const std::string& self_path,
}
bool WorkerPool::start(const WorkerPoolOptions& options) {
log_dir_ = options.log_dir;
for(std::uint32_t i = 0; i < options.stateless_count; ++i) {
if(!spawn_worker(options.self_path, false, 0)) {
return false;
@@ -143,7 +135,7 @@ bool WorkerPool::start(const WorkerPoolOptions& options) {
return true;
}
kota::task<> WorkerPool::stop() {
et::task<> WorkerPool::stop() {
LOG_INFO("WorkerPool stopping...");
// Close output pipes to signal workers to exit gracefully

View File

@@ -6,46 +6,47 @@
#include <list>
#include <memory>
#include "eventide/async/async.h"
#include "eventide/ipc/peer.h"
#include "server/protocol.h"
#include "kota/async/async.h"
#include "kota/ipc/codec/bincode.h"
#include "kota/ipc/peer.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/SmallVector.h"
namespace clice {
using kota::ipc::RequestResult;
/// Default timeout for IPC requests to worker processes.
constexpr inline auto kWorkerRequestTimeout = std::chrono::milliseconds(30000);
namespace et = eventide;
using et::ipc::RequestResult;
struct WorkerPoolOptions {
std::string self_path;
std::uint32_t stateless_count = 2;
std::uint32_t stateful_count = 2;
std::uint64_t worker_memory_limit = 4ULL * 1024 * 1024 * 1024; // 4GB default
std::string log_dir;
};
class WorkerPool {
public:
WorkerPool(kota::event_loop& loop) : loop(loop) {}
WorkerPool(et::event_loop& loop) : loop(loop) {}
/// Spawn all worker processes. Returns false on failure.
bool start(const WorkerPoolOptions& options);
/// Gracefully stop all workers.
kota::task<> stop();
et::task<> stop();
/// Send a request to a stateful worker with path_id affinity routing.
template <typename Params>
RequestResult<Params> send_stateful(std::uint32_t path_id,
const Params& params,
kota::ipc::request_options opts = {});
et::ipc::request_options opts = {});
/// Send a request to a stateless worker with round-robin dispatch.
template <typename Params>
RequestResult<Params> send_stateless(const Params& params,
kota::ipc::request_options opts = {});
RequestResult<Params> send_stateless(const Params& params, et::ipc::request_options opts = {});
/// Send a notification to the stateful worker owning path_id (if any).
template <typename Params>
@@ -61,12 +62,12 @@ public:
private:
struct WorkerProcess {
kota::process proc;
std::unique_ptr<kota::ipc::BincodePeer> peer;
et::process proc;
std::unique_ptr<et::ipc::BincodePeer> peer;
std::size_t owned_documents = 0;
};
kota::event_loop& loop;
et::event_loop& loop;
llvm::SmallVector<WorkerProcess> stateless_workers;
llvm::SmallVector<WorkerProcess> stateful_workers;
std::size_t next_stateless = 0;
@@ -80,30 +81,33 @@ private:
void clear_owner(std::size_t worker_index);
std::size_t pick_least_loaded();
std::string log_dir_;
bool spawn_worker(const std::string& self_path, bool stateful, std::uint64_t memory_limit);
};
// --- Template implementations ---------------------------------------------------
template <typename Params>
RequestResult<Params> WorkerPool::send_stateful(std::uint32_t path_id,
const Params& params,
kota::ipc::request_options opts) {
et::ipc::request_options opts) {
if(stateful_workers.empty()) {
co_return kota::outcome_error(kota::ipc::Error{"No stateful workers available"});
co_return et::outcome_error(et::ipc::Error{"No stateful workers available"});
}
if(!opts.timeout.has_value()) {
opts.timeout = kWorkerRequestTimeout;
}
// No timeout: compile tasks run as detached tasks (loop.schedule) that
// are immune to LSP $/cancelRequest. Adding a timeout here would use
// kotatsu's with_token/when_any which has a spurious-cancellation bug
// that kills requests within milliseconds instead of the configured period.
auto idx = assign_worker(path_id);
co_return co_await stateful_workers[idx].peer->send_request(params, opts);
}
template <typename Params>
RequestResult<Params> WorkerPool::send_stateless(const Params& params,
kota::ipc::request_options opts) {
et::ipc::request_options opts) {
if(stateless_workers.empty()) {
co_return kota::outcome_error(kota::ipc::Error{"No stateless workers available"});
co_return et::outcome_error(et::ipc::Error{"No stateless workers available"});
}
if(!opts.timeout.has_value()) {
opts.timeout = kWorkerRequestTimeout;
}
auto idx = next_stateless;
next_stateless = (next_stateless + 1) % stateless_workers.size();

Some files were not shown because too many files have changed in this diff Show More