7 Commits

Author SHA1 Message Date
Myriad-Dreamin
9d75659fb1 feat: implement explicit_reference_targets 2026-04-04 03:18:45 +08:00
Myriad-Dreamin
09e95bbc7e feat: explore gap between clice and clangd 2026-04-04 01:51:56 +08:00
Myriad-Dreamin
69454812bf feat: explicit specify lexical tokens to handle 2026-04-04 00:38:22 +08:00
Myriad-Dreamin
511b71f19a dev: add module example 2026-04-04 00:04:55 +08:00
Myriad-Dreamin
ed8b8b7745 dev: add single-file sample 2026-04-03 23:49:03 +08:00
Myriad-Dreamin
a303e13f58 dev: add cmake-workspace sample 2026-04-03 23:48:08 +08:00
Myriad-Dreamin
72c0a74609 dev: watch socket 2026-04-03 23:47:58 +08:00
215 changed files with 9266 additions and 29035 deletions

View File

@@ -100,7 +100,7 @@ SortIncludes: true
SortUsingDeclarations: Never SortUsingDeclarations: Never
IncludeBlocks: Regroup IncludeBlocks: Regroup
IncludeCategories: IncludeCategories:
- Regex: '^["<](spdlog|toml\+\+|coraing|cpptrace|flatbuffers|kota)/' - Regex: '^["<](spdlog|toml\+\+|coraing|cpptrace|flatbuffers)/'
Priority: 30 Priority: 30
SortPriority: 31 SortPriority: 31

View File

@@ -1,50 +0,0 @@
---
Checks: >
-*,
bugprone-*,
modernize-*,
performance-*,
readability-*,
-modernize-use-trailing-return-type,
-readability-magic-numbers,
-readability-else-after-return,
-readability-braces-around-statements,
-readability-avoid-const-params-in-decls,
-readability-named-parameter,
-readability-implicit-bool-conversion,
-readability-use-anyofallof,
-bugprone-easily-swappable-parameters,
-bugprone-exception-escape,
-bugprone-narrowing-conversions,
-modernize-use-nodiscard,
WarningsAsErrors: ""
HeaderFilterRegex: "(src|tests)/.*"
CheckOptions:
# Naming conventions matching project style
- key: readability-identifier-naming.ClassCase
value: CamelCase
- key: readability-identifier-naming.StructCase
value: CamelCase
- key: readability-identifier-naming.EnumCase
value: CamelCase
- key: readability-identifier-naming.EnumConstantCase
value: CamelCase
- key: readability-identifier-naming.TemplateParameterCase
value: CamelCase
- key: readability-identifier-naming.TypeAliasCase
value: CamelCase
- key: readability-identifier-naming.FunctionCase
value: lower_case
- key: readability-identifier-naming.MethodCase
value: lower_case
- key: readability-identifier-naming.VariableCase
value: lower_case
- key: readability-identifier-naming.ParameterCase
value: lower_case
- key: readability-identifier-naming.MemberCase
value: lower_case
- key: readability-identifier-naming.NamespaceCase
value: lower_case

View File

@@ -1,268 +0,0 @@
# clice — Project Guide
## Project Overview
clice is a next-generation C++ language server (LSP) built on LLVM/Clang, targeting modern C++ (C++20/23). It uses a multi-process architecture with a master server coordinating stateless and stateful workers.
## Core Correction Patterns — Lessons from Past Interactions
The following patterns were extracted from extensive real-world collaboration. These are recurring mistakes that MUST be avoided. Read them carefully — they represent hard-won lessons, not hypothetical concerns.
### Pattern 1: Misjudging Real-World Priorities
AI tends to optimize whatever metric looks most impressive, rather than what actually matters in the user's real scenario.
**Example**: During performance optimization, the AI proudly reported "hot cache is 4-7x faster!" — but the function in question runs at LSP server startup, which is ALWAYS a cold start. Optimizing hot cache was completely meaningless.
**Rule**: Before optimizing or analyzing anything, first understand the REAL usage scenario. Ask yourself: "When does this code actually run? What does the user actually experience?" Do not chase metrics that look good on paper but are irrelevant in practice.
### Pattern 2: Pushing Without Local Verification
The most common and most damaging pattern. AI proposes a fix, pushes it immediately, CI fails, then another fix, push, fail again — wasting CI cycles and the user's time.
**Rule**: NEVER push code that you haven't verified locally. Before every push:
- Build locally with the same configuration CI uses.
- Run the relevant tests locally and confirm they pass.
- If you cannot reproduce the CI environment locally, say so — do not just "try and see."
- "It compiles" is NOT sufficient. Tests must pass.
### Pattern 3: Superficial Refactoring
When asked to refactor, AI tends to do mechanical code movement (copy functions from A to B) without understanding the deeper design intent (ownership, responsibility boundaries, API cleanliness).
**Example**: When splitting `MasterServer` into `Workspace` and `Session`, the AI moved functions but kept ugly APIs like `f(path_id, sessions_map)` instead of the clean `f(Session&)` that the refactoring was meant to achieve.
**Rule**: When refactoring, understand the WHY. Ask: "What design problem is this refactoring solving?" If you're just moving code around without improving the abstractions, you're not refactoring — you're rearranging deck chairs.
### Pattern 4: Fixing Only the Immediate Instance, Not the Pattern
When given a cleanup instruction, AI applies it to the single file or function currently being discussed, ignoring all other occurrences in the project.
**Example**: User says "remove decorative `===` comment separators." AI removes them from `workspace.cpp` only. User has to say: "The other files too!"
**Rule**: When given a cleanup or style instruction, apply it project-wide. Use `Grep` to find ALL occurrences and fix them all in one pass. Think: "Where else does this pattern appear?"
### Pattern 5: Never Skip, Disable, or Work Around Failing Tests
When stuck on a difficult bug (especially flaky CI, race conditions, platform-specific issues), AI may propose marking tests as `continue-on-error`, skipping them, or adding `expected-failure` annotations to make CI green.
**Rule**: This is ABSOLUTELY FORBIDDEN. If a test fails, fix the root cause. There are ZERO exceptions. Skipping a test to make CI green is not "fixing" — it is hiding a bug. If you ever find yourself thinking "maybe we should just skip this test," stop and reconsider your approach entirely.
### Pattern 6: Excessive Confirmation Seeking vs. Premature Execution
AI oscillates between two extremes: asking "should I do X?" for every trivial decision, or silently executing major changes without confirmation.
**Rule**: Calibrate based on reversibility and impact:
- **Small, reversible changes** (formatting, renaming a local variable, adding a test): just do it.
- **Architecture decisions, API changes, large refactors**: propose the plan first, wait for confirmation.
- **Pushing to remote, creating PRs, modifying CI**: always confirm.
- When the user says "go ahead" or "do it," execute fully without asking again mid-way.
## Code Reuse & Understanding Before Implementation
**This is the single most important rule in this project.** Before writing ANY new code, you MUST thoroughly read and understand the existing codebase first. This project has a rich set of utilities, abstractions, and patterns already in place — duplicating them wastes effort and creates maintenance burden.
Concrete requirements:
1. **Read before you write.** Before implementing a feature or fix, explore the relevant modules in `src/`. Search for existing helpers, utilities, and patterns that solve the same or similar problems. Use `Grep`, `Glob`, and `Agent` tools to investigate thoroughly — do not assume something doesn't exist just because you haven't seen it yet.
2. **Reuse existing infrastructure.** This project already has:
- A `Lexer` class (`src/syntax/lexer.h`) — do not hand-write token scanning logic.
- A `PositionMapper` for source location conversion — do not reimplement offset-to-line/column math.
- `CompilationUnitRef` methods (`decompose_location`, `decompose_range`, `file_path`, `directives`, etc.) — use them instead of raw Clang APIs.
- `SemanticVisitor` for AST traversal — extend it, do not write custom recursive AST walkers.
- `Tester` framework for unit tests with VFS, annotation support, and multi-phase compilation — use it, do not create ad-hoc test setups.
- Utility functions in `src/support/` — check there before writing new helpers.
3. **Follow established patterns.** When adding a new feature (e.g., a new LSP request handler), look at how 2-3 existing features of the same kind are implemented. Match their structure: same file organization, same function signatures, same error handling patterns. If every other feature in `src/feature/` follows a certain pattern, yours should too.
4. **Do not reinvent what the project already has.** If you find yourself writing a helper function that feels generic (string manipulation, path handling, JSON serialization, source range conversion), STOP and search the codebase first. There is a high probability it already exists. Creating duplicates leads to inconsistencies and bugs when one copy gets updated but the other doesn't.
5. **When in doubt, ask.** If you're unsure whether an existing utility covers your use case or whether to extend an existing abstraction vs. create a new one, ask the user rather than guessing.
## Source Layout
- `src/server/` — LSP server core: master server, compiler, indexer, stateful/stateless workers
- `src/feature/` — LSP feature implementations: hover, completion, document links, semantic tokens, etc.
- `src/compile/` — Compilation orchestration: compilation unit, directives, diagnostics
- `src/index/` — Symbol indexing: TUIndex, ProjectIndex, MergedIndex, include graph
- `src/semantic/` — Semantic analysis: symbol kinds, relations, AST visitor, template resolver
- `src/syntax/` — Lexer, scanner, token types, dependency graph
- `src/command/` — CLI parsing, compilation database, toolchain detection
- `src/support/` — Utilities: logging, filesystem, JSON, string helpers
## Build System
- Uses **pixi** for environment management and **CMake + Ninja** for building.
- Two build types: `Debug` and `RelWithDebInfo` (default).
- Build output goes to `build/[type]/`.
- See `/build`, `/test`, `/format` commands for common operations.
## Commit Message Format
Use **conventional commits** — enforced by CI:
```
<type>(<scope>): <short description>
```
- **Types**: `feat`, `fix`, `refactor`, `chore`, `docs`, `ci`, `test`
- **Scopes**: match `src/` subdirectories or feature names, e.g. `completion`, `server`, `index`, `tests`, `document links`
- Keep the subject line under 70 characters.
## Tests
Three types of tests, all must pass before committing:
- **Unit tests** (`tests/unit/`): C++ tests using the project's own test framework. Test names should be at most 4 words.
- **Integration tests** (`tests/integration/`): Python pytest tests that start a real clice server and communicate via LSP.
- **Smoke tests** (`tests/smoke/`): Replay recorded LSP sessions via `tests/replay.py`.
### Integration Test Style
- Keep tests concise. Do NOT write large comment blocks explaining the test layout or expected behavior.
- Use descriptive test function names and short inline comments only where logic is non-obvious.
## Pre-PR Review
Before opening a PR, launch **3 parallel subagents** to review the diff independently:
1. **Correctness reviewer**: Check for logic errors, edge cases, undefined behavior, and off-by-one mistakes.
2. **Style reviewer**: Verify the code follows this project's naming conventions, coding style, and CLAUDE.md rules.
3. **Test reviewer**: Confirm test coverage is adequate — new functionality has tests, edge cases are covered, and no existing tests were broken or weakened.
Each agent should read the full diff (`git diff main...HEAD`) and report issues. Fix all reported issues before opening the PR.
## Pre-commit Checklist
Before committing code, you MUST:
1. **Run `pixi run format`** to format all source files.
2. **Pass all three types of tests:**
- Unit tests: `pixi run unit-test [type]`
- Integration tests: `pixi run integration-test [type]`
- Smoke tests: `pixi run smoke-test [type]`
3. **All test failures must be fixed before committing.** This is a HARD REQUIREMENT with NO exceptions:
- If a test fails, it MUST be fixed before you commit. Do NOT commit with known failures.
- Do NOT skip, disable, or mark tests as expected-failure to work around breakage.
- Do NOT argue "this test was already broken before my changes" — if it fails on your branch, it is YOUR responsibility to fix it before committing. The main branch CI is green; any failure on your branch is caused by your changes, period.
- Do NOT defer fixing to a follow-up PR. Fix it NOW, in this branch, before committing.
---
## C++ Coding Style
### Template & Type Traits
- Do NOT blindly add `std::remove_cvref_t` on every template parameter. Understand C++ template argument deduction rules:
- `template<typename T> void f(T x)``T` is always deduced as a non-reference, non-cv-qualified type. No need for `remove_cvref_t`.
- `template<typename T> void f(T& x)``T` is deduced as the referred-to type (possibly cv-qualified, but never a reference). No need for `remove_cvref_t` to strip references.
- `template<typename T> void f(const T& x)``T` is deduced as a non-const, non-reference type. No need for `remove_cvref_t`.
- `template<typename T> void f(T&& x)`**forwarding reference**: `T` CAN be deduced as an lvalue reference (e.g., `int&`). This is the ONLY case where `std::remove_cvref_t<T>` is needed to get the bare type.
- Class template parameters and return types are also never deduced as references; don't add `remove_cvref_t` on them either.
### Type Traits & Concepts (C++20/23)
- This project targets C++20/23. Use variable templates directly for type traits — do NOT use the old pattern of wrapping a class template static member in a variable template. Prefer:
```cpp
// Good: directly specialize a variable template
template<typename T>
inline constexpr bool is_my_type_v = false;
template<>
inline constexpr bool is_my_type_v<MyType> = true;
```
```cpp
// Bad: unnecessary class template wrapper
template<typename T>
struct is_my_type : std::false_type {};
template<>
struct is_my_type<MyType> : std::true_type {};
template<typename T>
inline constexpr bool is_my_type_v = is_my_type<T>::value;
```
- When defining a concept that checks a type trait, do NOT add `std::remove_cvref_t` unless you specifically intend the concept to see through references/cv-qualifiers. If the concept is meant for a bare type, just use `T` directly — the caller is responsible for passing the right type.
```cpp
// Good
template<typename T>
concept MyTrait = is_my_type_v<T>;
// Bad: unnecessary remove_cvref_t
template<typename T>
concept MyTrait = is_my_type_v<std::remove_cvref_t<T>>;
```
### Naming Conventions
- **Variables, member fields, function names**: `snake_case`. Class member fields do NOT use any special suffix/prefix (no trailing `_`, no `m_` prefix).
- **Class names, template parameter names, enum names**: `PascalCase`. Exception: some class names also use `snake_case` — follow the existing style in the project.
- **Enum values**: `PascalCase`.
### String Literals
- Prefer C++11 raw string literals `R"(...)"` over escaped strings. Avoid `\"`, `\\`, `\n` in string literals when a raw literal is cleaner.
### Error Handling
- **Prefer `if` with init-statements to tightly scope error variables**, but avoid them when they compromise code readability or flatten control flow.
- **Omit redundant conditions:** If the error type provides an `operator bool` or evaluates implicitly (e.g., standard error codes, custom error wrappers), omit the redundant condition check.
- **Avoid forced `else` branches:** If scoping the variable inside the `if` requires you to introduce an `else` block for the success path (especially when returning early on error), declare the variable in the local scope instead to keep the control flow flat.
```cpp
// Good: Omit redundant condition when the type has operator bool
if (auto err = foo()) {
/* handle error */
}
// Bad: Redundant condition check
if (auto err = foo(); err) {
/* handle error */
}
// Good: Use init-statement when a custom condition is required,
// AND the variable isn't needed outside the if-statement
if (auto result = foo(); !result.has_value()) {
/* handle error */
}
// --- Scope and Control Flow Considerations ---
// Bad: Using init-statement forces an 'else' block because 'result'
// goes out of scope, leading to nested/redundant code.
if (auto result = get_data(); !result.has_value()) {
return result.error();
} else {
process(result.value()); // Success path is forced into a nested block
}
// Good: Declare as a regular local variable to allow early exit
// and keep the success path un-nested (flat control flow).
auto result = get_data();
if (!result.has_value()) {
return result.error();
}
process(result.value());
```
### Style
- Prefer `[[maybe_unused]]` over `(void)` for intentionally unused variables or parameters.
### Modern C++ Usage
- Use C++20/23 APIs whenever possible. Do NOT use `<iostream>` facilities (`std::cout`, `std::cin`, `std::cerr`, etc.). Also do NOT use C-style I/O (`printf`, `fprintf`, etc.).
- Prefer `std::ranges` / `std::views` APIs over raw loops and traditional `<algorithm>` calls.
- If the project depends on LLVM, prefer LLVM's efficient data structures (e.g., `llvm::SmallVector`, `llvm::DenseMap`, `llvm::StringMap`, `llvm::StringRef`) over their `std` counterparts when appropriate.
### Parameter Passing Preferences
- For string parameters, prefer `llvm::StringRef` > `std::string_view` > `const std::string&`.
- For array/span parameters, prefer `llvm::ArrayRef` > `std::span` > `const std::vector&`.

View File

@@ -1,15 +0,0 @@
Build the project. Accepts an optional argument for build type: `Debug` or `RelWithDebInfo` (default).
Available build commands:
- CMake configure only: `pixi run cmake-config [type]`
- CMake build only (skip configure): `pixi run cmake-build [type]`
- Full build (configure + build): `pixi run build [type]`
- Build a specific target: `pixi run cmake-build [type]` then `cmake --build build/[type] --target [target]`
Common targets: `clice`, `unit_tests`
Example usage:
- `/build` — full build RelWithDebInfo
- `/build Debug` — full build Debug

View File

@@ -1,5 +0,0 @@
Format all project source files.
Run: `pixi run format`
Formats C++, Python, Lua, JS/TS, Markdown, JSON, TOML, and YAML files.

View File

@@ -1,19 +0,0 @@
Run tests. Accepts an optional argument for build type: `Debug` or `RelWithDebInfo` (default).
Available test commands:
- Unit tests: `pixi run unit-test [type]`
- Integration tests: `pixi run integration-test [type]`
- Smoke tests: `pixi run smoke-test [type]`
- All tests (unit + integration): `pixi run test [type]`
Filtering specific tests:
- Unit tests: `pixi run unit-test [type] --test-filter=SuiteName.CaseName`
- Integration tests: `pixi run pytest tests/integration -k "test_name" --executable=./build/[type]/bin/clice`
- Smoke tests: `pixi run python tests/replay.py tests/smoke/specific.jsonl --clice=./build/[type]/bin/clice`
Example usage:
- `/test` — run all tests (RelWithDebInfo)
- `/test Debug` — run all tests (Debug)

View File

@@ -1,7 +1,2 @@
chat: chat:
auto_reply: false auto_reply: false
reviews:
auto_review:
enabled: true
summary:
enabled: false

View File

@@ -13,7 +13,7 @@ runs:
- name: Setup Pixi - name: Setup Pixi
uses: prefix-dev/setup-pixi@v0.9.3 uses: prefix-dev/setup-pixi@v0.9.3
with: with:
pixi-version: v0.67.0 pixi-version: v0.62.0
environments: ${{ inputs.environments }} environments: ${{ inputs.environments }}
activate-environment: true activate-environment: true
cache: true cache: true

View File

@@ -1,7 +1,8 @@
name: benchmark name: benchmark
on: on:
workflow_dispatch: pull_request:
branches: [main]
jobs: jobs:
benchmark: benchmark:
@@ -21,7 +22,7 @@ jobs:
- name: Build scan_benchmark - name: Build scan_benchmark
run: | run: |
pixi run cmake-config RelWithDebInfo ON -- -DCLICE_ENABLE_BENCHMARK=ON pixi run cmake-config RelWithDebInfo ON "-DCLICE_ENABLE_BENCHMARK=ON"
cmake --build build/RelWithDebInfo --target scan_benchmark cmake --build build/RelWithDebInfo --target scan_benchmark
- name: Clone LLVM - name: Clone LLVM

View File

@@ -1,22 +1,6 @@
name: build llvm name: build llvm
on: on:
workflow_dispatch:
inputs:
llvm_version:
description: "LLVM version to build (e.g., 21.1.8)"
required: true
type: string
skip_upload:
description: "Skip upload and PR creation (build-only mode)"
required: false
type: boolean
default: false
skip_pr:
description: "Skip PR creation (upload only, no PR)"
required: false
type: boolean
default: false
pull_request: pull_request:
# if you want to run this workflow, change the branch name to main, # if you want to run this workflow, change the branch name to main,
# if you want to turn off it, change it to non existent branch. # if you want to turn off it, change it to non existent branch.
@@ -28,7 +12,9 @@ jobs:
fail-fast: false fail-fast: false
matrix: matrix:
include: include:
# Native builds - os: windows-2025
llvm_mode: Debug
lto: OFF
- os: windows-2025 - os: windows-2025
llvm_mode: RelWithDebInfo llvm_mode: RelWithDebInfo
lto: OFF lto: OFF
@@ -53,42 +39,6 @@ jobs:
- os: macos-15 - os: macos-15
llvm_mode: RelWithDebInfo llvm_mode: RelWithDebInfo
lto: ON lto: ON
# Cross-compilation builds
# macOS x64 (from arm64 macos-15)
- os: macos-15
llvm_mode: RelWithDebInfo
lto: OFF
target_triple: x86_64-apple-darwin
- os: macos-15
llvm_mode: RelWithDebInfo
lto: ON
target_triple: x86_64-apple-darwin
# Linux aarch64 (from x64 ubuntu-24.04)
- os: ubuntu-24.04
llvm_mode: RelWithDebInfo
lto: OFF
target_triple: aarch64-linux-gnu
pixi_env: cross-linux-aarch64
- os: ubuntu-24.04
llvm_mode: RelWithDebInfo
lto: ON
target_triple: aarch64-linux-gnu
pixi_env: cross-linux-aarch64
# Windows arm64 (from x64 windows-2025)
- os: windows-2025
llvm_mode: RelWithDebInfo
lto: OFF
target_triple: aarch64-pc-windows-msvc
pixi_env: cross-windows-arm64
- os: windows-2025
llvm_mode: RelWithDebInfo
lto: ON
target_triple: aarch64-pc-windows-msvc
pixi_env: cross-windows-arm64
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
steps: steps:
- name: Checkout repository - name: Checkout repository
@@ -117,91 +67,49 @@ jobs:
free -h free -h
df -h df -h
- uses: ./.github/actions/setup-pixi - name: Setup Pixi
uses: prefix-dev/setup-pixi@v0.9.3
with: with:
environments: ${{ matrix.pixi_env || 'package' }} pixi-version: v0.59.0
environments: package
activate-environment: true
cache: true
locked: true
- name: Clone llvm-project - name: Clone llvm-project (21.1.4)
shell: bash shell: bash
run: | run: |
VERSION="${{ inputs.llvm_version || '21.1.8' }}" git clone --branch llvmorg-21.1.4 --depth 1 https://github.com/llvm/llvm-project.git .llvm
echo "Cloning LLVM ${VERSION}..."
git clone --branch "llvmorg-${VERSION}" --depth 1 https://github.com/llvm/llvm-project.git .llvm
- name: Validate distribution components
shell: bash
run: |
python3 scripts/validate-llvm-components.py \
--llvm-src=.llvm \
--components-file=scripts/llvm-components.json
- name: Build LLVM (install-distribution) - name: Build LLVM (install-distribution)
shell: bash shell: bash
run: | run: |
ENV="${{ matrix.pixi_env || 'package' }}" pixi run build-llvm --llvm-src=.llvm --mode="${{ matrix.llvm_mode }}" --lto="${{ matrix.lto }}" --build-dir=build
EXTRA_ARGS=""
if [[ -n "${{ matrix.target_triple }}" ]]; then
EXTRA_ARGS="--target-triple=${{ matrix.target_triple }}"
fi
pixi run -e "$ENV" build-llvm \
--llvm-src=.llvm \
--mode="${{ matrix.llvm_mode }}" \
--lto="${{ matrix.lto }}" \
--build-dir=build \
${EXTRA_ARGS}
- name: Build clice using installed LLVM - name: Build clice using installed LLVM
if: ${{ !matrix.target_triple }}
shell: bash shell: bash
run: | run: |
pixi run cmake-config ${{ matrix.llvm_mode }} ON -- \ cmake -B build -G Ninja \
"-DCLICE_ENABLE_LTO=${{ matrix.lto }}" \ -DCMAKE_BUILD_TYPE=${{ matrix.llvm_mode }} \
"-DLLVM_INSTALL_PATH=.llvm/build-install" -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain.cmake \
pixi run cmake-build ${{ matrix.llvm_mode }} -DCLICE_ENABLE_TEST=ON \
-DCLICE_CI_ENVIRONMENT=ON \
- name: Build clice using installed LLVM (cross-compile) -DCLICE_ENABLE_LTO=${{ matrix.lto }} \
if: ${{ matrix.target_triple }} -DLLVM_INSTALL_PATH=".llvm/build-install"
shell: bash cmake --build build
run: |
ENV="${{ matrix.pixi_env || 'package' }}"
pixi run -e "$ENV" cmake-config ${{ matrix.llvm_mode }} ON -- \
"-DCLICE_ENABLE_LTO=${{ matrix.lto }}" \
"-DCLICE_TARGET_TRIPLE=${{ matrix.target_triple }}" \
"-DLLVM_INSTALL_PATH=.llvm/build-install"
pixi run -e "$ENV" cmake-build ${{ matrix.llvm_mode }}
- name: Verify cross-compiled binary architecture
if: ${{ matrix.target_triple && runner.os != 'Windows' }}
shell: bash
run: |
BINARY="build/${{ matrix.llvm_mode }}/bin/clice"
echo "Binary info:"
file "$BINARY"
case "${{ matrix.target_triple }}" in
aarch64-linux-gnu) file "$BINARY" | grep -q "aarch64" ;;
x86_64-apple-darwin) file "$BINARY" | grep -q "x86_64" ;;
esac
- name: Upload cross-compiled clice for functional test
if: ${{ matrix.target_triple && matrix.lto == 'OFF' }}
uses: actions/upload-artifact@v4
with:
name: cross-clice-${{ matrix.target_triple }}-${{ matrix.llvm_mode }}
path: |
build/${{ matrix.llvm_mode }}/bin/
build/${{ matrix.llvm_mode }}/lib/
if-no-files-found: error
retention-days: 1
- name: Run tests - name: Run tests
if: ${{ !matrix.target_triple }}
shell: bash shell: bash
run: pixi run test ${{ matrix.llvm_mode }} run: |
EXE_EXT=""
if [[ "${{ runner.os }}" == "Windows" ]]; then
EXE_EXT=".exe"
fi
./build/bin/unit_tests${EXE_EXT} --test-dir="./tests/data"
uv run --project tests pytest -s --log-cli-level=INFO tests/integration --executable=./build/bin/clice${EXE_EXT}
# Prune is only supported for native builds (requires linking clice to test).
# Cross-compiled targets reuse the native prune manifest of the same OS.
- name: Prune LLVM static libraries (Debug/RelWithDebInfo no LTO) - name: Prune LLVM static libraries (Debug/RelWithDebInfo no LTO)
if: (!matrix.target_triple) && (matrix.llvm_mode == 'Debug' || (matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF')) if: matrix.llvm_mode == 'Debug' || (matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF')
shell: bash shell: bash
run: | run: |
MANIFEST="pruned-libs-${{ matrix.os }}.json" MANIFEST="pruned-libs-${{ matrix.os }}.json"
@@ -209,13 +117,13 @@ jobs:
python3 scripts/prune-llvm-bin.py \ python3 scripts/prune-llvm-bin.py \
--action discover \ --action discover \
--install-dir ".llvm/build-install/lib" \ --install-dir ".llvm/build-install/lib" \
--build-dir "build/${{ matrix.llvm_mode }}" \ --build-dir "build" \
--max-attempts 60 \ --max-attempts 60 \
--sleep-seconds 60 \ --sleep-seconds 60 \
--manifest "${MANIFEST}" --manifest "${MANIFEST}"
- name: Upload pruned-libs manifest - name: Upload pruned-libs manifest
if: (!matrix.target_triple) && matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF' if: matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'OFF'
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v4
with: with:
name: llvm-pruned-libs-${{ matrix.os }} name: llvm-pruned-libs-${{ matrix.os }}
@@ -223,8 +131,8 @@ jobs:
if-no-files-found: error if-no-files-found: error
compression-level: 0 compression-level: 0
- name: Apply pruned-libs manifest (RelWithDebInfo + LTO, native only) - name: Apply pruned-libs manifest (RelWithDebInfo + LTO)
if: (!matrix.target_triple) && matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'ON' if: matrix.llvm_mode == 'RelWithDebInfo' && matrix.lto == 'ON'
shell: bash shell: bash
env: env:
GH_TOKEN: ${{ github.token }} GH_TOKEN: ${{ github.token }}
@@ -234,27 +142,7 @@ jobs:
--action apply \ --action apply \
--manifest "${MANIFEST}" \ --manifest "${MANIFEST}" \
--install-dir ".llvm/build-install/lib" \ --install-dir ".llvm/build-install/lib" \
--build-dir "build/${{ matrix.llvm_mode }}" \ --build-dir "build" \
--gh-run-id "${{ github.run_id }}" \
--gh-artifact "llvm-pruned-libs-${{ matrix.os }}" \
--gh-download-dir "artifacts" \
--max-attempts 60 \
--sleep-seconds 60
# For cross-compiled LTO builds, apply the native prune manifest.
# The unused library set is arch-independent (same API surface).
- name: Apply pruned-libs manifest (cross-compile + LTO)
if: matrix.target_triple && matrix.lto == 'ON'
shell: bash
env:
GH_TOKEN: ${{ github.token }}
run: |
MANIFEST="pruned-libs-${{ matrix.os }}.json"
python3 scripts/prune-llvm-bin.py \
--action apply \
--manifest "${MANIFEST}" \
--install-dir ".llvm/build-install/lib" \
--build-dir "build/${{ matrix.llvm_mode }}" \
--gh-run-id "${{ github.run_id }}" \ --gh-run-id "${{ github.run_id }}" \
--gh-artifact "llvm-pruned-libs-${{ matrix.os }}" \ --gh-artifact "llvm-pruned-libs-${{ matrix.os }}" \
--gh-download-dir "artifacts" \ --gh-download-dir "artifacts" \
@@ -269,35 +157,23 @@ jobs:
MODE_TAG="debug" MODE_TAG="debug"
fi fi
# Determine arch/platform/toolchain from target triple or runner OS ARCH="x64"
if [[ -n "${{ matrix.target_triple }}" ]]; then PLATFORM="linux"
case "${{ matrix.target_triple }}" in TOOLCHAIN="gnu"
x86_64-apple-darwin) if [[ "${{ matrix.os }}" == windows-* ]]; then
ARCH="x64"; PLATFORM="macos"; TOOLCHAIN="clang" ;; PLATFORM="windows"
aarch64-linux-gnu) TOOLCHAIN="msvc"
ARCH="aarch64"; PLATFORM="linux"; TOOLCHAIN="gnu" ;; elif [[ "${{ matrix.os }}" == macos-* ]]; then
aarch64-pc-windows-msvc) ARCH="arm64"
ARCH="aarch64"; PLATFORM="windows"; TOOLCHAIN="msvc" ;; PLATFORM="macos"
esac TOOLCHAIN="clang"
else
ARCH="x64"
PLATFORM="linux"
TOOLCHAIN="gnu"
if [[ "${{ matrix.os }}" == windows-* ]]; then
PLATFORM="windows"
TOOLCHAIN="msvc"
elif [[ "${{ matrix.os }}" == macos-* ]]; then
ARCH="arm64"
PLATFORM="macos"
TOOLCHAIN="clang"
fi
fi fi
SUFFIX="" SUFFIX=""
if [[ "${{ matrix.lto }}" == "ON" ]]; then if [[ "${{ matrix.lto }}" == "ON" ]]; then
SUFFIX="-lto" SUFFIX="-lto"
fi fi
if [[ "${{ matrix.llvm_mode }}" == "Debug" && "${{ matrix.os }}" != windows-* ]]; then if [[ "${{ matrix.llvm_mode }}" == "Debug" ]]; then
SUFFIX="${SUFFIX}-asan" SUFFIX="${SUFFIX}-asan"
fi fi
@@ -313,134 +189,3 @@ jobs:
name: ${{ env.LLVM_INSTALL_ARCHIVE }} name: ${{ env.LLVM_INSTALL_ARCHIVE }}
path: ${{ env.LLVM_INSTALL_ARCHIVE }} path: ${{ env.LLVM_INSTALL_ARCHIVE }}
if-no-files-found: error if-no-files-found: error
test-cross:
needs: build
strategy:
fail-fast: false
matrix:
include:
- os: macos-15-intel
llvm_mode: RelWithDebInfo
target_triple: x86_64-apple-darwin
- os: ubuntu-24.04-arm
llvm_mode: RelWithDebInfo
target_triple: aarch64-linux-gnu
- os: windows-11-arm
llvm_mode: RelWithDebInfo
target_triple: aarch64-pc-windows-msvc
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: test-run
- name: Download cross-compiled clice
uses: actions/download-artifact@v4
with:
name: cross-clice-${{ matrix.target_triple }}-${{ matrix.llvm_mode }}
path: build/${{ matrix.llvm_mode }}/
- name: Make binaries executable
if: runner.os != 'Windows'
run: chmod +x build/${{ matrix.llvm_mode }}/bin/*
- name: Run tests
run: pixi run -e test-run test ${{ matrix.llvm_mode }}
upload:
needs: build
if: ${{ !cancelled() && inputs.llvm_version && !inputs.skip_upload }}
runs-on: ubuntu-24.04
permissions:
contents: read
steps:
- uses: actions/checkout@v4
- name: Download all build artifacts
env:
GH_TOKEN: ${{ github.token }}
run: scripts/download-llvm.sh "${{ github.run_id }}"
- name: Upload to clice-llvm
env:
GH_TOKEN: ${{ secrets.UPLOAD_LLVM }}
TARGET_REPO: clice-io/clice-llvm
run: python3 scripts/upload-llvm.py "${{ inputs.llvm_version }}" "${TARGET_REPO}" "${{ github.run_id }}"
- name: Save manifest for update-clice job
uses: actions/upload-artifact@v4
with:
name: llvm-manifest-final
path: artifacts/llvm-manifest.json
if-no-files-found: error
compression-level: 0
update-clice:
needs: upload
if: ${{ !inputs.skip_pr }}
runs-on: ubuntu-24.04
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v4
- name: Download manifest
uses: actions/download-artifact@v4
with:
name: llvm-manifest-final
path: .
- name: Update manifest and version
run: |
python3 scripts/update-llvm-version.py \
--version "${{ inputs.llvm_version }}" \
--manifest-src llvm-manifest.json \
--manifest-dest config/llvm-manifest.json \
--package-cmake cmake/package.cmake
- name: Create or update PR
env:
GH_TOKEN: ${{ github.token }}
run: |
VERSION="${{ inputs.llvm_version }}"
BRANCH="chore/update-llvm-${VERSION}"
RUN_URL="https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}"
RELEASE_URL="https://github.com/clice-io/clice-llvm/releases/tag/${VERSION}"
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git checkout -b "${BRANCH}"
git add config/llvm-manifest.json cmake/package.cmake
git commit -m "chore: update LLVM to ${VERSION}"
git push --force-with-lease origin "${BRANCH}"
# Check if PR already exists for this branch
EXISTING_PR=$(gh pr list --head "${BRANCH}" --json number --jq '.[0].number // empty')
BODY="$(cat <<EOF
## Summary
- Update LLVM prebuilt binaries to version ${VERSION}
- Updated \`config/llvm-manifest.json\` with new SHA256 hashes
- Updated \`cmake/package.cmake\` version string
**Artifacts:** [clice-llvm release](${RELEASE_URL})
**Build:** [workflow run](${RUN_URL})
> Auto-generated by build-llvm workflow
EOF
)"
if [[ -n "${EXISTING_PR}" ]]; then
echo "Updating existing PR #${EXISTING_PR}"
gh pr edit "${EXISTING_PR}" --body "${BODY}"
else
gh pr create \
--title "chore: update LLVM to ${VERSION}" \
--body "${BODY}" \
--base main
fi

View File

@@ -14,12 +14,6 @@ jobs:
with: with:
environments: format environments: format
- name: Validate update-llvm-version.py can still patch package.cmake
run: |
python3 scripts/update-llvm-version.py --check \
--manifest-dest config/llvm-manifest.json \
--package-cmake cmake/package.cmake
- name: Run formatter - name: Run formatter
run: pixi run format run: pixi run format
continue-on-error: true continue-on-error: true

View File

@@ -19,7 +19,7 @@ jobs:
clice: ${{ steps.filter.outputs.clice }} clice: ${{ steps.filter.outputs.clice }}
vscode: ${{ steps.filter.outputs.vscode }} vscode: ${{ steps.filter.outputs.vscode }}
cmake: ${{ steps.filter.outputs.cmake }} cmake: ${{ steps.filter.outputs.cmake }}
xmake: ${{ steps.filter.outputs.xmake }}
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- uses: dorny/paths-filter@v3 - uses: dorny/paths-filter@v3
@@ -46,31 +46,13 @@ jobs:
- 'tests/**' - 'tests/**'
- 'config/**' - 'config/**'
- '.github/workflows/test-cmake.yml' - '.github/workflows/test-cmake.yml'
xmake:
conventional-commit: - 'xmake.lua'
if: ${{ !startsWith(github.ref, 'refs/tags/') }} - 'src/**'
runs-on: ubuntu-latest - 'include/**'
steps: - 'tests/**'
- name: Check conventional commit format - 'config/**'
env: - '.github/workflows/test-xmake.yml'
IS_PR: ${{ github.event_name == 'pull_request' }}
PR_TITLE: ${{ github.event.pull_request.title }}
COMMIT_MSG: ${{ github.event.head_commit.message }}
run: |
pattern='^(feat|fix|refactor|chore|build|ci|docs|test|perf|style|revert)(\(.+\))?: .+'
if [[ "$IS_PR" == "true" ]]; then
subject="$PR_TITLE"
label="PR title"
else
subject=$(echo "$COMMIT_MSG" | head -n1)
label="Commit message"
fi
if [[ ! "$subject" =~ $pattern ]]; then
echo "::error::$label must follow conventional commit format: type(scope)?: description"
echo " Valid types: feat, fix, refactor, chore, build, ci, docs, test, perf, style, revert"
echo " Got: '$subject'"
exit 1
fi
format: format:
needs: changes needs: changes
@@ -100,6 +82,11 @@ jobs:
if: ${{ needs.changes.outputs.cmake == 'true' }} if: ${{ needs.changes.outputs.cmake == 'true' }}
uses: ./.github/workflows/test-cmake.yml uses: ./.github/workflows/test-cmake.yml
# xmake:
# needs: changes
# if: ${{ needs.changes.outputs.xmake == 'true' }}
# uses: ./.github/workflows/test-xmake.yml
release-clice: release-clice:
permissions: permissions:
contents: write contents: write
@@ -117,17 +104,16 @@ jobs:
checks-passed: checks-passed:
if: ${{ always() && !startsWith(github.ref, 'refs/tags/') }} if: ${{ always() && !startsWith(github.ref, 'refs/tags/') }}
needs: needs:
- conventional-commit
- format - format
- deploy - deploy
# - clice # - clice
- vscode - vscode
- cmake - cmake
# - xmake
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Check results - name: Check results
uses: re-actors/alls-green@release/v1 uses: re-actors/alls-green@release/v1
with: with:
allowed-skips: conventional-commit,format,deploy,clice,vscode,cmake allowed-skips: format,deploy,clice,vscode,cmake,xmake
jobs: ${{ toJSON(needs) }} jobs: ${{ toJSON(needs) }}

View File

@@ -9,7 +9,6 @@ jobs:
fail-fast: false fail-fast: false
matrix: matrix:
include: include:
# Native builds
- os: windows-2025 - os: windows-2025
artifact_name: clice.zip artifact_name: clice.zip
asset_name: clice-x64-windows-msvc.zip asset_name: clice-x64-windows-msvc.zip
@@ -28,63 +27,43 @@ jobs:
symbol_artifact_name: clice-symbol.tar.gz symbol_artifact_name: clice-symbol.tar.gz
symbol_asset_name: clice-arm64-macos-darwin-symbol.tar.gz symbol_asset_name: clice-arm64-macos-darwin-symbol.tar.gz
# Cross-compilation builds
- os: macos-15
target_triple: x86_64-apple-darwin
pixi_env: cross-macos-x64
artifact_name: clice.tar.gz
asset_name: clice-x86_64-macos-darwin.tar.gz
symbol_artifact_name: clice-symbol.tar.gz
symbol_asset_name: clice-x86_64-macos-darwin-symbol.tar.gz
- os: ubuntu-24.04
target_triple: aarch64-linux-gnu
pixi_env: cross-linux-aarch64
artifact_name: clice.tar.gz
asset_name: clice-aarch64-linux-gnu.tar.gz
symbol_artifact_name: clice-symbol.tar.gz
symbol_asset_name: clice-aarch64-linux-gnu-symbol.tar.gz
- os: windows-2025
target_triple: aarch64-pc-windows-msvc
pixi_env: cross-windows-arm64
artifact_name: clice.zip
asset_name: clice-aarch64-windows-msvc.zip
symbol_artifact_name: clice-symbol.zip
symbol_asset_name: clice-aarch64-windows-msvc-symbol.zip
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Setup xmake
uses: xmake-io/github-action-setup-xmake@v1
with:
xmake-version: 3.0.5
actions-cache-folder: ".xmake-cache"
actions-cache-key: ${{ matrix.os }}
package-cache: true
package-cache-key: ${{ matrix.os }}-pkg-release-v1
build-cache: true
build-cache-key: ${{ matrix.os }}-build-release-v1
- uses: ./.github/actions/setup-pixi - uses: ./.github/actions/setup-pixi
with: with:
environments: ${{ matrix.pixi_env || 'package' }} environments: package
- name: Package (native) - name: Remove ci llvm toolchain on Windows
if: ${{ !matrix.target_triple }} if: runner.os == 'Windows'
run: pixi run package
- name: Package (cross-compile)
if: ${{ matrix.target_triple }}
run: | run: |
ENV="${{ matrix.pixi_env }}" # @see https://github.com/xmake-io/xmake/issues/7158
pixi run -e "$ENV" package-config -- \ xmake lua os.rmdir "C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/Llvm"
"-DCLICE_TARGET_TRIPLE=${{ matrix.target_triple }}" xmake lua os.rmdir "C:/Program Files/LLVM"
pixi run -e "$ENV" cmake-build
- name: Package
run: pixi run package
- name: Upload Main Package to Release - name: Upload Main Package to Release
if: startsWith(github.ref, 'refs/tags/v') if: startsWith(github.ref, 'refs/tags/v')
uses: svenstaro/upload-release-action@v2 uses: svenstaro/upload-release-action@v2
with: with:
repo_token: ${{ secrets.GITHUB_TOKEN }} repo_token: ${{ secrets.GITHUB_TOKEN }}
file: build/RelWithDebInfo/${{ matrix.artifact_name }} file: build/xpack/clice/${{ matrix.artifact_name }}
asset_name: ${{ matrix.asset_name }} asset_name: ${{ matrix.asset_name }}
tag: ${{ github.ref }} tag: ${{ github.ref }}
overwrite: true overwrite: true
@@ -94,7 +73,7 @@ jobs:
uses: svenstaro/upload-release-action@v2 uses: svenstaro/upload-release-action@v2
with: with:
repo_token: ${{ secrets.GITHUB_TOKEN }} repo_token: ${{ secrets.GITHUB_TOKEN }}
file: build/RelWithDebInfo/${{ matrix.symbol_artifact_name }} file: build/xpack/clice/${{ matrix.symbol_artifact_name }}
asset_name: ${{ matrix.symbol_asset_name }} asset_name: ${{ matrix.symbol_asset_name }}
tag: ${{ github.ref }} tag: ${{ github.ref }}
overwrite: true overwrite: true

View File

@@ -17,134 +17,53 @@ jobs:
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
include: os: [windows-2025, ubuntu-24.04, macos-15]
# Native builds build_type: [Debug, RelWithDebInfo]
- os: windows-2025
build_type: RelWithDebInfo
- os: ubuntu-24.04
build_type: Debug
- os: ubuntu-24.04
build_type: RelWithDebInfo
- os: macos-15
build_type: Debug
- os: macos-15
build_type: RelWithDebInfo
# Cross-compile (build only; tests run on native runners)
- os: macos-15
build_type: RelWithDebInfo
target_triple: x86_64-apple-darwin
build_only: true
- os: ubuntu-24.04
build_type: RelWithDebInfo
target_triple: aarch64-linux-gnu
build_only: true
pixi_env: cross-linux-aarch64
- os: windows-2025
build_type: RelWithDebInfo
target_triple: aarch64-pc-windows-msvc
build_only: true
pixi_env: cross-windows-arm64
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@v4 uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi - uses: ./.github/actions/setup-pixi
with:
environments: ${{ matrix.pixi_env || 'default' }}
- name: Restore compiler cache - name: Restore compiler cache
uses: actions/cache@v4 uses: actions/cache@v4
with: with:
path: ${{ runner.os == 'Windows' && '.cache/sccache' || '.cache/ccache' }} path: ${{ runner.os == 'Windows' && '.cache/sccache' || '.cache/ccache' }}
key: ${{ runner.os }}-${{ matrix.build_type }}-${{ matrix.target_triple || 'native' }}-ccache-${{ github.sha }} key: ${{ runner.os }}-${{ matrix.build_type }}-ccache-${{ github.sha }}
restore-keys: | restore-keys: |
${{ runner.os }}-${{ matrix.build_type }}-${{ matrix.target_triple || 'native' }}-ccache- ${{ runner.os }}-${{ matrix.build_type }}-ccache-
- name: Zero cache stats - name: Zero cache stats
run: | run: |
ENV="${{ matrix.pixi_env || 'default' }}"
if [ "$RUNNER_OS" = "Windows" ]; then if [ "$RUNNER_OS" = "Windows" ]; then
pixi run -e "$ENV" -- sccache --stop-server || true pixi run -- sccache --stop-server || true
pixi run -e "$ENV" -- sccache --zero-stats || true pixi run -- sccache --zero-stats || true
else else
pixi run -e "$ENV" -- ccache --zero-stats || true pixi run -- ccache --zero-stats || true
fi fi
shell: bash shell: bash
- name: Build (native) - name: Build
if: ${{ !matrix.target_triple }}
run: pixi run build ${{ matrix.build_type }} ON run: pixi run build ${{ matrix.build_type }} ON
- name: Build (cross-compile) - name: Unit Test
if: ${{ matrix.target_triple }} run: pixi run unit-test ${{ matrix.build_type }}
shell: bash
run: |
ENV="${{ matrix.pixi_env || 'default' }}"
pixi run -e "$ENV" cmake-config ${{ matrix.build_type }} OFF -- \
"-DCLICE_TARGET_TRIPLE=${{ matrix.target_triple }}"
pixi run -e "$ENV" cmake-build ${{ matrix.build_type }}
- name: Upload cross-compiled binaries - name: Integration Test
if: ${{ matrix.build_only }} run: pixi run integration-test ${{ matrix.build_type }}
uses: actions/upload-artifact@v4
with:
name: cross-build-${{ matrix.target_triple }}
path: |
build/${{ matrix.build_type }}/bin/
build/${{ matrix.build_type }}/lib/
if-no-files-found: error
retention-days: 1
- name: Run tests - name: Smoke Test
if: ${{ !matrix.build_only }} if: success() || failure()
run: pixi run test ${{ matrix.build_type }} run: pixi run smoke-test ${{ matrix.build_type }}
- name: Print cache stats and stop server - name: Print cache stats and stop server
if: always() if: always()
run: | run: |
ENV="${{ matrix.pixi_env || 'default' }}"
if [ "$RUNNER_OS" = "Windows" ]; then if [ "$RUNNER_OS" = "Windows" ]; then
pixi run -e "$ENV" -- sccache --show-stats pixi run -- sccache --show-stats
pixi run -e "$ENV" -- sccache --stop-server || true pixi run -- sccache --stop-server || true
else else
pixi run -e "$ENV" -- ccache --show-stats pixi run -- ccache --show-stats
fi fi
shell: bash shell: bash
test-cross:
needs: build
strategy:
fail-fast: false
matrix:
include:
- os: macos-15-intel
build_type: RelWithDebInfo
target_triple: x86_64-apple-darwin
- os: ubuntu-24.04-arm
build_type: RelWithDebInfo
target_triple: aarch64-linux-gnu
- os: windows-11-arm
build_type: RelWithDebInfo
target_triple: aarch64-pc-windows-msvc
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- uses: ./.github/actions/setup-pixi
with:
environments: test-run
- name: Download cross-compiled binaries
uses: actions/download-artifact@v4
with:
name: cross-build-${{ matrix.target_triple }}
path: build/${{ matrix.build_type }}/
- name: Make binaries executable
if: runner.os != 'Windows'
run: chmod +x build/${{ matrix.build_type }}/bin/*
- name: Run tests
run: pixi run -e test-run test ${{ matrix.build_type }}

42
.github/workflows/test-xmake.yml vendored Normal file
View File

@@ -0,0 +1,42 @@
name: xmake
on:
workflow_call:
jobs:
build:
strategy:
fail-fast: false
matrix:
os: [windows-2025, ubuntu-24.04, macos-15]
build_type: [debug, releasedbg]
exclude:
- os: windows-2025
build_type: debug
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup xmake
uses: xmake-io/github-action-setup-xmake@v1
with:
xmake-version: 3.0.5
actions-cache-folder: ".xmake-cache"
actions-cache-key: ${{ matrix.os }}
package-cache: true
package-cache-key: ${{ matrix.os }}-pixi
build-cache: true
build-cache-key: ${{ matrix.os }}-${{ matrix.build_type }}
- uses: ./.github/actions/setup-pixi
- name: Build
run: pixi run xmake ${{ matrix.build_type }}
- name: Test
run: pixi run xmake-test
- name: Remove llvm package (Linux)
if: runner.os == 'Linux'
run: xmake require --uninstall clice-llvm

12
.gitignore vendored
View File

@@ -35,7 +35,7 @@
*build*/ *build*/
temp/ temp/
.cache/ .cache/
.xmake/
.llvm*/ .llvm*/
.clice/ .clice/
compile_commands.json compile_commands.json
@@ -56,11 +56,10 @@ __pycache__/
tests/unit/Local/ tests/unit/Local/
# IDEs & Editors # IDEs & Editors
/.vscode/* /.vscode/
!/.vscode/launch.json
!/.vscode/tasks.json
.vs/ .vs/
.idea/ .idea/
.claude
.clangd .clangd
# pixi environments # pixi environments
@@ -69,6 +68,5 @@ tests/unit/Local/
!.pixi/config.toml !.pixi/config.toml
.codex/ .codex/
.claude/* .claude/
!.claude/CLAUDE.md openspec/
!.claude/commands/

83
.vscode/launch.json vendored
View File

@@ -1,83 +0,0 @@
{
"version": "0.2.0",
"configurations": [
{
"type": "lldb",
"request": "launch",
"name": "Debug clice",
"program": "${workspaceFolder}/build/Debug/bin/clice",
"args": ["--mode=socket", "--port=50051"],
"cwd": "${workspaceFolder}"
},
{
"type": "lldb",
"request": "launch",
"name": "Debug clice (socket, RelWithDebInfo)",
"program": "${workspaceFolder}/build/RelWithDebInfo/bin/clice",
"args": ["--mode=socket", "--port=50051"],
"cwd": "${workspaceFolder}"
},
{
"name": "VSCode Extension (pipe)",
"type": "extensionHost",
"request": "launch",
"args": [
"--disable-extension=llvm-vs-code-extensions.vscode-clangd",
"--disable-extension=ms-vscode.cpptools",
"--disable-extension=ms-vscode.cpptools-extension-pack",
"--extensionDevelopmentPath=${workspaceFolder}/editors/vscode"
],
"env": {
"CLICE_MODE": "pipe"
},
"outFiles": ["${workspaceFolder}/editors/vscode/dist/**/*.js"],
"preLaunchTask": "npm: watch vscode ext"
},
{
"name": "VSCode Extension (socket)",
"type": "extensionHost",
"request": "launch",
"args": [
"--disable-extension=llvm-vs-code-extensions.vscode-clangd",
"--disable-extension=ms-vscode.cpptools",
"--disable-extension=ms-vscode.cpptools-extension-pack",
"--extensionDevelopmentPath=${workspaceFolder}/editors/vscode"
],
"env": {
"CLICE_MODE": "socket"
},
"outFiles": ["${workspaceFolder}/editors/vscode/dist/**/*.js"],
"preLaunchTask": "npm: watch vscode ext"
},
{
"type": "lldb",
"request": "launch",
"name": "Unit Test",
"program": "${workspaceFolder}/build/Debug/bin/unit_tests",
"args": ["--test-dir=./tests/data", "--test-filter=${input:filter}"],
"cwd": "${workspaceFolder}"
},
{
"type": "lldb",
"request": "launch",
"name": "Release Unit Test",
"program": "${workspaceFolder}/build/RelWithDebInfo/bin/unit_tests",
"args": ["--test-dir=./tests/data", "--test-filter=${input:filter}"],
"cwd": "${workspaceFolder}"
}
],
"compounds": [
{
"name": "clice + VSCode Extension (socket)",
"configurations": ["Debug clice", "VSCode Extension (socket)"],
"stopAll": true
}
],
"inputs": [
{
"id": "filter",
"type": "promptString",
"description": "Unit Test Filter"
}
]
}

42
.vscode/tasks.json vendored
View File

@@ -1,42 +0,0 @@
{
"version": "2.0.0",
"tasks": [
{
"label": "npm: install vscode deps",
"type": "shell",
"command": "pnpm",
"args": ["install"],
"options": {
"cwd": "${workspaceFolder}/editors/vscode"
},
"problemMatcher": []
},
{
"label": "npm: watch vscode ext",
"type": "shell",
"command": "pnpm",
"args": ["run", "watch"],
"options": {
"cwd": "${workspaceFolder}/editors/vscode"
},
"dependsOn": "npm: install vscode deps",
"problemMatcher": "$ts-webpack-watch",
"isBackground": true,
"presentation": {
"reveal": "never",
"group": "watchers"
}
},
{
"label": "npm: package vscode ext",
"type": "shell",
"command": "pnpm",
"args": ["run", "package"],
"options": {
"cwd": "${workspaceFolder}/editors/vscode"
},
"dependsOn": "npm: install vscode deps",
"problemMatcher": []
}
]
}

View File

@@ -127,16 +127,9 @@ endif()
set(FBS_SCHEMA_FILE "${PROJECT_SOURCE_DIR}/src/index/schema.fbs") set(FBS_SCHEMA_FILE "${PROJECT_SOURCE_DIR}/src/index/schema.fbs")
set(GENERATED_HEADER "${PROJECT_BINARY_DIR}/generated/schema_generated.h") set(GENERATED_HEADER "${PROJECT_BINARY_DIR}/generated/schema_generated.h")
if(CMAKE_CROSSCOMPILING)
find_program(FLATC_EXECUTABLE flatc REQUIRED)
set(FLATC_CMD "${FLATC_EXECUTABLE}")
else()
set(FLATC_CMD "$<TARGET_FILE:flatc>")
endif()
add_custom_command( add_custom_command(
OUTPUT "${GENERATED_HEADER}" OUTPUT "${GENERATED_HEADER}"
COMMAND ${FLATC_CMD} --cpp -o "${PROJECT_BINARY_DIR}/generated" "${FBS_SCHEMA_FILE}" COMMAND $<TARGET_FILE:flatc> --cpp -o "${PROJECT_BINARY_DIR}/generated" "${FBS_SCHEMA_FILE}"
DEPENDS "${FBS_SCHEMA_FILE}" DEPENDS "${FBS_SCHEMA_FILE}"
COMMENT "Generating C++ header from ${FBS_SCHEMA_FILE}" COMMENT "Generating C++ header from ${FBS_SCHEMA_FILE}"
) )
@@ -158,13 +151,13 @@ target_link_libraries(clice-core PUBLIC
spdlog::spdlog spdlog::spdlog
roaring::roaring roaring::roaring
flatbuffers flatbuffers
kota::ipc::lsp eventide::ipc::lsp
kota::codec::toml eventide::serde::toml
simdjson::simdjson simdjson::simdjson
) )
add_executable(clice "${PROJECT_SOURCE_DIR}/src/clice.cc") add_executable(clice "${PROJECT_SOURCE_DIR}/src/clice.cc")
target_link_libraries(clice PRIVATE clice::core kota::deco) target_link_libraries(clice PRIVATE clice::core eventide::deco)
install(TARGETS clice RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}) install(TARGETS clice RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
add_custom_target(copy_clang_resource ALL add_custom_target(copy_clang_resource ALL
@@ -196,7 +189,7 @@ if(CLICE_ENABLE_TEST)
"${PROJECT_SOURCE_DIR}/src" "${PROJECT_SOURCE_DIR}/src"
"${PROJECT_SOURCE_DIR}/tests/unit" "${PROJECT_SOURCE_DIR}/tests/unit"
) )
target_link_libraries(unit_tests PRIVATE clice::core kota::zest kota::deco) target_link_libraries(unit_tests PRIVATE clice::core eventide::zest eventide::deco)
endif() endif()
if(CLICE_ENABLE_BENCHMARK) if(CLICE_ENABLE_BENCHMARK)
@@ -206,7 +199,7 @@ if(CLICE_ENABLE_BENCHMARK)
target_include_directories(scan_benchmark PRIVATE target_include_directories(scan_benchmark PRIVATE
"${PROJECT_SOURCE_DIR}/src" "${PROJECT_SOURCE_DIR}/src"
) )
target_link_libraries(scan_benchmark PRIVATE clice::core kota::deco) target_link_libraries(scan_benchmark PRIVATE clice::core eventide::deco)
endif() endif()
if(CLICE_RELEASE) if(CLICE_RELEASE)

View File

@@ -21,15 +21,17 @@
#include <thread> #include <thread>
#include "command/command.h" #include "command/command.h"
#include "eventide/deco/deco.h"
#include "eventide/serde/json/serializer.h"
#include "support/filesystem.h" #include "support/filesystem.h"
#include "support/logging.h" #include "support/logging.h"
#include "support/path_pool.h" #include "support/path_pool.h"
#include "syntax/dependency_graph.h" #include "syntax/dependency_graph.h"
#include "kota/codec/json/serializer.h"
#include "kota/deco/deco.h"
#include "llvm/Support/FileSystem.h" #include "llvm/Support/FileSystem.h"
namespace et = eventide;
using namespace clice; using namespace clice;
struct BenchmarkOptions { struct BenchmarkOptions {
@@ -95,7 +97,7 @@ void export_graph_json(const PathPool& path_pool,
export_data.files.push_back(std::move(node)); export_data.files.push_back(std::move(node));
} }
auto json = kota::codec::json::to_json(export_data); auto json = et::serde::json::to_json(export_data);
if(!json) { if(!json) {
std::println(stderr, "Failed to serialize dependency graph"); std::println(stderr, "Failed to serialize dependency graph");
return; return;
@@ -219,8 +221,8 @@ void print_report(const ScanReport& report) {
} }
int main(int argc, const char** argv) { int main(int argc, const char** argv) {
auto args = kota::deco::util::argvify(argc, argv); auto args = deco::util::argvify(argc, argv);
auto result = kota::deco::cli::parse<BenchmarkOptions>(args); auto result = deco::cli::parse<BenchmarkOptions>(args);
if(!result.has_value()) { if(!result.has_value()) {
std::println(stderr, "Error: {}", result.error().message); std::println(stderr, "Error: {}", result.error().message);
@@ -231,7 +233,7 @@ int main(int argc, const char** argv) {
if(opts.help.value_or(false) || !opts.cdb_path.has_value()) { if(opts.help.value_or(false) || !opts.cdb_path.has_value()) {
std::ostringstream oss; std::ostringstream oss;
kota::deco::cli::write_usage_for<BenchmarkOptions>(oss, "scan_benchmark [OPTIONS] <cdb>"); deco::cli::write_usage_for<BenchmarkOptions>(oss, "scan_benchmark [OPTIONS] <cdb>");
std::print("{}", oss.str()); std::print("{}", oss.str());
return opts.help.value_or(false) ? 0 : 1; return opts.help.value_or(false) ? 0 : 1;
} }

View File

@@ -25,22 +25,6 @@ function(setup_llvm LLVM_VERSION)
list(APPEND LLVM_SETUP_ARGS "--offline") list(APPEND LLVM_SETUP_ARGS "--offline")
endif() endif()
if(DEFINED CLICE_TARGET_TRIPLE)
if(CLICE_TARGET_TRIPLE MATCHES "linux")
list(APPEND LLVM_SETUP_ARGS "--target-platform" "Linux")
elseif(CLICE_TARGET_TRIPLE MATCHES "darwin")
list(APPEND LLVM_SETUP_ARGS "--target-platform" "macosx")
elseif(CLICE_TARGET_TRIPLE MATCHES "windows")
list(APPEND LLVM_SETUP_ARGS "--target-platform" "Windows")
endif()
if(CLICE_TARGET_TRIPLE MATCHES "^aarch64")
list(APPEND LLVM_SETUP_ARGS "--target-arch" "arm64")
elseif(CLICE_TARGET_TRIPLE MATCHES "^x86_64")
list(APPEND LLVM_SETUP_ARGS "--target-arch" "x64")
endif()
endif()
execute_process( execute_process(
COMMAND "${Python3_EXECUTABLE}" "${LLVM_SETUP_SCRIPT}" ${LLVM_SETUP_ARGS} COMMAND "${Python3_EXECUTABLE}" "${LLVM_SETUP_SCRIPT}" ${LLVM_SETUP_ARGS}
RESULT_VARIABLE LLVM_SETUP_RESULT RESULT_VARIABLE LLVM_SETUP_RESULT
@@ -117,15 +101,8 @@ function(setup_llvm LLVM_VERSION)
clangToolingSyntax clangToolingSyntax
) )
else() else()
file(GLOB LLVM_LIBRARIES CONFIGURE_DEPENDS "${LLVM_INSTALL_PATH}/lib/${CMAKE_STATIC_LIBRARY_PREFIX}LLVM[a-zA-Z]*${CMAKE_STATIC_LIBRARY_SUFFIX}") file(GLOB LLVM_LIBRARIES CONFIGURE_DEPENDS "${LLVM_INSTALL_PATH}/lib/*${CMAKE_STATIC_LIBRARY_SUFFIX}")
file(GLOB CLANG_LIBRARIES CONFIGURE_DEPENDS "${LLVM_INSTALL_PATH}/lib/${CMAKE_STATIC_LIBRARY_PREFIX}clang[a-zA-Z]*${CMAKE_STATIC_LIBRARY_SUFFIX}") target_link_libraries(llvm-libs INTERFACE ${LLVM_LIBRARIES})
# TODO: find a better way to find out whether zlib and zstd are needed
# Currently link if present in the LLVM lib directory
file(GLOB OTHER_REQUIRED_LIBS CONFIGURE_DEPENDS
"${LLVM_INSTALL_PATH}/lib/${CMAKE_STATIC_LIBRARY_PREFIX}z${CMAKE_STATIC_LIBRARY_SUFFIX}"
"${LLVM_INSTALL_PATH}/lib/${CMAKE_STATIC_LIBRARY_PREFIX}zstd${CMAKE_STATIC_LIBRARY_SUFFIX}"
)
target_link_libraries(llvm-libs INTERFACE ${LLVM_LIBRARIES} ${CLANG_LIBRARIES} ${OTHER_REQUIRED_LIBS})
target_compile_definitions(llvm-libs INTERFACE CLANG_BUILD_STATIC=1) target_compile_definitions(llvm-libs INTERFACE CLANG_BUILD_STATIC=1)
endif() endif()
endfunction() endfunction()

View File

@@ -1,7 +1,7 @@
include_guard() include_guard()
include(${CMAKE_CURRENT_LIST_DIR}/llvm.cmake) include(${CMAKE_CURRENT_LIST_DIR}/llvm.cmake)
setup_llvm("21.1.8") setup_llvm("21.1.4+r1")
# install dependencies # install dependencies
include(FetchContent) include(FetchContent)
@@ -39,18 +39,18 @@ set(FLATBUFFERS_BUILD_TESTS OFF CACHE BOOL "" FORCE)
set(FLATBUFFERS_BUILD_FLATHASH OFF CACHE BOOL "" FORCE) set(FLATBUFFERS_BUILD_FLATHASH OFF CACHE BOOL "" FORCE)
FetchContent_Declare( FetchContent_Declare(
kotatsu eventide
GIT_REPOSITORY https://github.com/clice-io/kotatsu GIT_REPOSITORY https://github.com/clice-io/eventide
GIT_TAG main GIT_TAG main
GIT_SHALLOW TRUE GIT_SHALLOW TRUE
) )
set(KOTA_ENABLE_ZEST ON) set(ETD_ENABLE_ZEST ON)
set(KOTA_ENABLE_TEST OFF) set(ETD_ENABLE_TEST OFF)
set(KOTA_CODEC_ENABLE_SIMDJSON ON) set(ETD_SERDE_ENABLE_SIMDJSON ON)
set(KOTA_CODEC_ENABLE_YYJSON ON) set(ETD_SERDE_ENABLE_YYJSON ON)
set(KOTA_CODEC_ENABLE_TOML ON) set(ETD_SERDE_ENABLE_TOML ON)
set(KOTA_ENABLE_EXCEPTIONS OFF) set(ETD_ENABLE_EXCEPTIONS OFF)
set(KOTA_ENABLE_RTTI OFF) set(ETD_ENABLE_RTTI OFF)
FetchContent_MakeAvailable(kotatsu spdlog croaring flatbuffers) FetchContent_MakeAvailable(eventide spdlog croaring flatbuffers)

View File

@@ -1,29 +1,5 @@
cmake_minimum_required(VERSION 3.30) cmake_minimum_required(VERSION 3.30)
# Cross-compilation support via CLICE_TARGET_TRIPLE.
# Examples:
# -DCLICE_TARGET_TRIPLE=x86_64-apple-darwin (macOS x64 from arm64)
# -DCLICE_TARGET_TRIPLE=aarch64-linux-gnu (Linux arm64 from x64)
# -DCLICE_TARGET_TRIPLE=aarch64-pc-windows-msvc (Windows arm64 from x64)
if(DEFINED CLICE_TARGET_TRIPLE)
if(CLICE_TARGET_TRIPLE MATCHES "^x86_64-apple-darwin")
set(CMAKE_OSX_ARCHITECTURES "x86_64" CACHE STRING "")
elseif(CLICE_TARGET_TRIPLE MATCHES "^aarch64-.*linux")
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
set(CMAKE_C_COMPILER_TARGET "aarch64-linux-gnu" CACHE STRING "")
set(CMAKE_CXX_COMPILER_TARGET "aarch64-linux-gnu" CACHE STRING "")
if(DEFINED ENV{CONDA_PREFIX} AND NOT DEFINED CMAKE_SYSROOT)
set(CMAKE_SYSROOT "$ENV{CONDA_PREFIX}/aarch64-conda-linux-gnu/sysroot" CACHE PATH "")
endif()
elseif(CLICE_TARGET_TRIPLE MATCHES "^aarch64-.*-windows")
set(CMAKE_SYSTEM_NAME Windows)
set(CMAKE_SYSTEM_PROCESSOR ARM64)
set(CMAKE_C_COMPILER_TARGET "aarch64-pc-windows-msvc" CACHE STRING "")
set(CMAKE_CXX_COMPILER_TARGET "aarch64-pc-windows-msvc" CACHE STRING "")
endif()
endif()
set(CMAKE_C_COMPILER clang CACHE STRING "") set(CMAKE_C_COMPILER clang CACHE STRING "")
set(CMAKE_CXX_COMPILER clang++ CACHE STRING "") set(CMAKE_CXX_COMPILER clang++ CACHE STRING "")

View File

@@ -1,142 +1,83 @@
[ [
{ {
"version": "21.1.8", "version": "21.1.4+r1",
"filename": "aarch64-linux-gnu-releasedbg-lto.tar.xz",
"sha256": "f3444ee840b50933c23656cbee7c4d010e752ac55ca66095b97f7c0e997b13b5",
"lto": true,
"asan": false,
"platform": "linux",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"filename": "aarch64-linux-gnu-releasedbg.tar.xz",
"sha256": "b9012bf059e4d8673fb564b5780e5fc78c6a2e47f5cc6a39f444d1879b42dd2a",
"lto": false,
"asan": false,
"platform": "linux",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"filename": "aarch64-windows-msvc-releasedbg-lto.tar.xz",
"sha256": "8870d16141ba7f9ea12f5147b8d91329abbbaa4376cd4576667dd323d896dd08",
"lto": true,
"asan": false,
"platform": "windows",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"filename": "aarch64-windows-msvc-releasedbg.tar.xz",
"sha256": "ad394e79ec85dd40f942671bb0342ffe54a103eb2baabacb773999d57d80134b",
"lto": false,
"asan": false,
"platform": "windows",
"arch": "arm64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"filename": "arm64-macos-clang-debug-asan.tar.xz", "filename": "arm64-macos-clang-debug-asan.tar.xz",
"sha256": "b02d20e4f7294ee33f49a09dfdd765b3b44135e003ef50e3a760aeee39e3f993", "sha256": "7da4b7d63edefecaf11773e7e701c575140d1a07329bbbb038673b6ee4516ff5",
"lto": false, "lto": false,
"asan": true, "asan": true,
"platform": "macosx", "platform": "macosx",
"arch": "arm64",
"build_type": "Debug" "build_type": "Debug"
}, },
{ {
"version": "21.1.8", "version": "21.1.4+r1",
"filename": "arm64-macos-clang-releasedbg-lto.tar.xz", "filename": "arm64-macos-clang-releasedbg-lto.tar.xz",
"sha256": "e40c21eb0d0b91d9d4ab31212a5cb01ea46707f5c29839414567857e4147604d", "sha256": "300455b169448f9f01ae95e3bc269f489558a4ca3955e3032171cc75feca0e30",
"lto": true, "lto": true,
"asan": false, "asan": false,
"platform": "macosx", "platform": "macosx",
"arch": "arm64",
"build_type": "RelWithDebInfo" "build_type": "RelWithDebInfo"
}, },
{ {
"version": "21.1.8", "version": "21.1.4+r1",
"filename": "arm64-macos-clang-releasedbg.tar.xz", "filename": "arm64-macos-clang-releasedbg.tar.xz",
"sha256": "e1b01de34f0edfd41c118e4981a93afb35556ae369597e864f4a393db623b926", "sha256": "9abfc6cd65b957d734ffb97610a634fb4a66d3fbe0fcfb5a1c9124ef693c1495",
"lto": false, "lto": false,
"asan": false, "asan": false,
"platform": "macosx", "platform": "macosx",
"arch": "arm64",
"build_type": "RelWithDebInfo" "build_type": "RelWithDebInfo"
}, },
{ {
"version": "21.1.8", "version": "21.1.4+r1",
"filename": "x64-linux-gnu-debug-asan.tar.xz", "filename": "x64-linux-gnu-debug-asan.tar.xz",
"sha256": "76bb82d822b5377fb5e0fac8abcfba125142e6a0acc02bb36d1fa1532a268646", "sha256": "c1ad3ec476911596a842ac67dd9c9c9475ce9f0a77b81101d3c801840292e7bc",
"lto": false, "lto": false,
"asan": true, "asan": true,
"platform": "linux", "platform": "linux",
"arch": "x64",
"build_type": "Debug" "build_type": "Debug"
}, },
{ {
"version": "21.1.8", "version": "21.1.4+r1",
"filename": "x64-linux-gnu-releasedbg-lto.tar.xz", "filename": "x64-linux-gnu-releasedbg-lto.tar.xz",
"sha256": "32f5edddec1e689124f045b586fb402ae30febc05203af7391b088bc8494cd53", "sha256": "8a869c2184d139dbba704e2d712e7a68336458ad2d70622b3eb906c3e3511e54",
"lto": true, "lto": true,
"asan": false, "asan": false,
"platform": "linux", "platform": "linux",
"arch": "x64",
"build_type": "RelWithDebInfo" "build_type": "RelWithDebInfo"
}, },
{ {
"version": "21.1.8", "version": "21.1.4+r1",
"filename": "x64-linux-gnu-releasedbg.tar.xz", "filename": "x64-linux-gnu-releasedbg.tar.xz",
"sha256": "8ba3c84f23a2a81a86c54780754a61adf99048aa2ac0dc9b9708d0f842d553de", "sha256": "552bab86f715d4f2c027f07eaaf5b3d6b8e430af0b74b470142f3f00da4feec6",
"lto": false, "lto": false,
"asan": false, "asan": false,
"platform": "linux", "platform": "linux",
"arch": "x64",
"build_type": "RelWithDebInfo" "build_type": "RelWithDebInfo"
}, },
{ {
"version": "21.1.8", "version": "21.1.4+r1",
"filename": "x64-macos-clang-releasedbg-lto.tar.xz", "filename": "x64-windows-msvc-debug-asan.tar.xz",
"sha256": "97e81d6296896d7237f118f728d05291707b9e4e5791e07ce4be8aee0517505d", "sha256": "093667a493d336c22ff3c604c5f1fea2a7d2c927c1179cec44e9a03726906ac1",
"lto": true,
"asan": false,
"platform": "macosx",
"arch": "x64",
"build_type": "RelWithDebInfo"
},
{
"version": "21.1.8",
"filename": "x64-macos-clang-releasedbg.tar.xz",
"sha256": "53c13f8e1082fa2fe2f9c05303de48cb3133bf5f24271f4b3062f1dec578159c",
"lto": false, "lto": false,
"asan": false, "asan": true,
"platform": "macosx", "platform": "windows",
"arch": "x64", "build_type": "Debug"
"build_type": "RelWithDebInfo"
}, },
{ {
"version": "21.1.8", "version": "21.1.4+r1",
"filename": "x64-windows-msvc-releasedbg-lto.tar.xz", "filename": "x64-windows-msvc-releasedbg-lto.tar.xz",
"sha256": "16bcf0e4cbc3d2b1204edd619a3837004dacea28eeff0a101c8d0212f936427d", "sha256": "010539e85621dc3c6ecf359d899feb4075aeca5d0bba6625cdbec0e570e79129",
"lto": true, "lto": true,
"asan": false, "asan": false,
"platform": "windows", "platform": "windows",
"arch": "x64",
"build_type": "RelWithDebInfo" "build_type": "RelWithDebInfo"
}, },
{ {
"version": "21.1.8", "version": "21.1.4+r1",
"filename": "x64-windows-msvc-releasedbg.tar.xz", "filename": "x64-windows-msvc-releasedbg.tar.xz",
"sha256": "81d31fad05e200726c8178314b0b2045c947483dddd8cb974f4c376ae5f441fa", "sha256": "f473c09fbea10053fac00be409d75dc228d4a38bcbc5e4aeb58b56a4b0dde78e",
"lto": false, "lto": false,
"asan": false, "asan": false,
"platform": "windows", "platform": "windows",
"arch": "x64",
"build_type": "RelWithDebInfo" "build_type": "RelWithDebInfo"
} }
] ]

View File

@@ -20,7 +20,7 @@ max_active_file = 8
cache_dir = "${workspace}/.clice/cache" cache_dir = "${workspace}/.clice/cache"
# Directory for storing index files. # Directory for storing index files.
index_dir = "${workspace}/.clice/index" index_dir = "${workspace}/.clice/index"
logging_dir = "${workspace}/.clice/logs" logging_dir = "${workspace}/.clice/logging"
# Compile commands files or directories to search for compile_commands.json files. # Compile commands files or directories to search for compile_commands.json files.
compile_commands_paths = ["${workspace}/build"] compile_commands_paths = ["${workspace}/build"]

View File

@@ -91,7 +91,7 @@ The worker pool (`src/server/worker_pool.cpp`) manages spawning and communicatin
### Communication ### Communication
Workers communicate with the master via **stdio pipes** using a **bincode** serialization format (via `kota::ipc::BincodePeer`). This is more compact and faster than JSON for internal IPC, while the master handles JSON for the external LSP protocol. Workers communicate with the master via **stdio pipes** using a **bincode** serialization format (via `eventide::ipc::BincodePeer`). This is more compact and faster than JSON for internal IPC, while the master handles JSON for the external LSP protocol.
### Stateful Worker Routing ### Stateful Worker Routing
@@ -111,7 +111,7 @@ The stateful worker (`src/server/stateful_worker.cpp`) caches compiled ASTs in m
- **Feature queries**: Look up the cached AST and invoke the corresponding `feature::*` function (hover, semantic tokens, etc.), serializing the result to JSON - **Feature queries**: Look up the cached AST and invoke the corresponding `feature::*` function (hover, semantic tokens, etc.), serializing the result to JSON
- **Document updates**: Received as notifications — the worker updates the stored text and marks the document as `dirty`, causing feature queries to return `null` until recompilation - **Document updates**: Received as notifications — the worker updates the stored text and marks the document as `dirty`, causing feature queries to return `null` until recompilation
- **Eviction**: LRU-based; evicts the oldest document when capacity is exceeded, notifying the master - **Eviction**: LRU-based; evicts the oldest document when capacity is exceeded, notifying the master
- **Concurrency**: Each document has a per-document `kota::mutex` (strand) to serialize compilation and feature queries. Heavy work (compilation, feature extraction) runs on a thread pool via `kota::queue`. - **Concurrency**: Each document has a per-document `et::mutex` (strand) to serialize compilation and feature queries. Heavy work (compilation, feature extraction) runs on a thread pool via `et::queue`.
## Stateless Worker ## Stateless Worker
@@ -123,7 +123,7 @@ The stateless worker (`src/server/stateless_worker.cpp`) handles one-shot reques
- **Build PCM**: Compiles a C++20 module interface to a temporary file - **Build PCM**: Compiles a C++20 module interface to a temporary file
- **Index**: Compiles a file for indexing (TUIndex generation — currently a stub) - **Index**: Compiles a file for indexing (TUIndex generation — currently a stub)
All requests are dispatched to a thread pool via `kota::queue`. All requests are dispatched to a thread pool via `et::queue`.
## Compile Graph ## Compile Graph
@@ -132,7 +132,7 @@ The compile graph (`src/server/compile_graph.cpp`) tracks compilation unit depen
- **Registration**: Each file registers its included dependencies - **Registration**: Each file registers its included dependencies
- **Cascade invalidation**: When a file changes, all transitive dependents are marked dirty and their ongoing compilations are cancelled - **Cascade invalidation**: When a file changes, all transitive dependents are marked dirty and their ongoing compilations are cancelled
- **Dependency compilation**: Before compiling a file, `compile_deps` ensures all dependencies (PCH, PCMs) are built first - **Dependency compilation**: Before compiling a file, `compile_deps` ensures all dependencies (PCH, PCMs) are built first
- **Cancellation**: Uses `kota::cancellation_source` to abort in-flight compilations when files are invalidated - **Cancellation**: Uses `et::cancellation_source` to abort in-flight compilations when files are invalidated
## Configuration ## Configuration

View File

@@ -32,6 +32,18 @@ pixi run integration-test Debug
> [!TIP] > [!TIP]
> If you want to develop directly with `cmake`, `ninja`, `clang++`, etc., run `pixi shell` to enter a shell with all env vars configured. > If you want to develop directly with `cmake`, `ninja`, `clang++`, etc., run `pixi shell` to enter a shell with all env vars configured.
### XMake
We also support building with XMake:
```shell
# config & build (default releasedbg)
pixi run xmake
# unit & integration
pixi run xmake-test
```
## Manual Build ## Manual Build
If you plan to build manually, first ensure your toolchain matches the versions defined in `pixi.toml`. If you plan to build manually, first ensure your toolchain matches the versions defined in `pixi.toml`.
@@ -58,13 +70,30 @@ Optional build options:
| CLICE_USE_LIBCXX | OFF | Build clice with libc++ (adds `-std=libc++`); if enabled, ensure the LLVM libs are also built with libc++ | | CLICE_USE_LIBCXX | OFF | Build clice with libc++ (adds `-std=libc++`); if enabled, ensure the LLVM libs are also built with libc++ |
| CLICE_CI_ENVIRONMENT | OFF | Enable the `CLICE_CI_ENVIRONMENT` macro; some tests only run in CI | | CLICE_CI_ENVIRONMENT | OFF | Enable the `CLICE_CI_ENVIRONMENT` macro; some tests only run in CI |
### XMake
Build clice with:
```bash
xmake f -c --mode=releasedbg --toolchain=clang
xmake build --all
```
Optional build options:
| Option | Default | Effect |
| ------------- | ------- | ---------------------------------------- |
| --llvm | "" | Build clice with LLVM from a custom path |
| --enable_test | false | Build clice unit tests |
| --ci | false | Enable `CLICE_CI_ENVIRONMENT` |
## About LLVM ## About LLVM
clice calls Clang APIs to parse C++ code, so it must link against LLVM/Clang. Because clice uses Clang's private headers (usually absent from distro packages), the system LLVM package cannot be used directly. clice calls Clang APIs to parse C++ code, so it must link against LLVM/Clang. Because clice uses Clang's private headers (usually absent from distro packages), the system LLVM package cannot be used directly.
Two ways to satisfy this dependency: Two ways to satisfy this dependency:
1. We publish prebuilt binaries of the LLVM version we use at [clice-llvm](https://github.com/clice-io/clice-llvm/releases) for CI and release builds. During builds, cmake downloads these LLVM libs by default. 1. We publish prebuilt binaries of the LLVM version we use at [clice-llvm](https://github.com/clice-io/clice-llvm/releases) for CI and release builds. During builds, cmake and xmake download these LLVM libs by default.
> [!IMPORTANT] > [!IMPORTANT]
> >

View File

@@ -18,6 +18,13 @@ We use pytest to run integration tests. Please refer to `pyproject.toml` to inst
$ pytest -s --log-cli-level=INFO tests/integration --executable=./build/bin/clice $ pytest -s --log-cli-level=INFO tests/integration --executable=./build/bin/clice
``` ```
If you use xmake as your build system, you can run the tests directly with xmake:
```shell
$ xmake run --verbose unit_tests
$ xmake test --verbose integration_tests/default
```
## Debug ## Debug
If you want to attach a debugger to clice for debugging, it is recommended to first start clice in socket mode independently, and then connect the client to it. If you want to attach a debugger to clice for debugging, it is recommended to first start clice in socket mode independently, and then connect the client to it.

View File

@@ -54,73 +54,14 @@ bazel run @hedron_compile_commands//:refresh_all
### Visual Studio ### Visual Studio
Visual Studio (2019 16.1+) can generate a compilation database via CMake integration. Open your project as a CMake project, then configure the generation in `CMakeSettings.json`: TODO:
```json
{
"configurations": [
{
"name": "x64-Debug",
"generator": "Ninja",
"buildRoot": "${projectDir}\\build",
"cmakeCommandArgs": "-DCMAKE_EXPORT_COMPILE_COMMANDS=ON"
}
]
}
```
Alternatively, for MSBuild-based projects (`.vcxproj`), you can use [compiledb-vs](https://github.com/pjbroad/compiledb-vs) or [catter](https://github.com/clice-io/catter) to generate the compilation database.
### Makefile ### Makefile
For Makefile-based projects, use [bear](https://github.com/rizsotto/Bear) to intercept compilation commands: TODO:
```bash
bear -- make
```
This will generate a `compile_commands.json` in the current directory. Note that `bear` requires a clean build to capture all commands — run `make clean` before `bear -- make` if needed.
Alternatively, if you use GNU Make, you can use [compiledb](https://github.com/nicktimko/compiledb):
```bash
compiledb make
```
### Meson
Meson generates a compilation database automatically during setup:
```bash
meson setup build
```
The `compile_commands.json` will be in the `build` directory.
### Xmake ### Xmake
Use one of the following approaches to generate a compilation database.
#### Command Line
Run the following command to manually generate a compilation database:
```bash
xmake project -k compile_commands --lsp=clangd build
```
> Compilation database generated manually doesn't automatically update itself. Re-generate if changes are made to the project.
#### VSCode Extension
The Xmake official VSCode extension automatically generates the compilation database when `xmake.lua` is updated. However, it generates the database to the `.vscode` directory by default. Add this setting in `settings.json`:
```json
"xmake.compileCommandsDirectory": "build"
```
to explicitly ask the extension to generate the compilation database in `build`.
### Others ### Others
For any other build system, you can use [catter](https://github.com/clice-io/catter) to generate a compilation database. It captures compilation commands through a fake compiler approach and is designed to work reliably with any build system that invokes a compiler executable. For any other build system, you can try using [bear](https://github.com/rizsotto/Bear) or [scan-build](https://github.com/rizsotto/scan-build) to intercept compilation commands and obtain the compilation database (no guarantee of success). We plan to write a **new tool** in the future that captures compilation commands through a fake compiler approach.

View File

@@ -32,6 +32,18 @@ pixi run integration-test Debug
> [!TIP] > [!TIP]
> 如果你想直接使用 `cmake`, `ninja`, `clang++` 等命令进行开发,请运行 `pixi shell` 进入已配置好环境变量的终端 > 如果你想直接使用 `cmake`, `ninja`, `clang++` 等命令进行开发,请运行 `pixi shell` 进入已配置好环境变量的终端
### XMake
我们同样支持使用 XMake 构建:
```shell
# config & build (default releasedbg)
pixi run xmake
# unit & integration
pixi run xmake-test
```
## Manual Build ## Manual Build
如果你打算手动构建,请务必先确认你的工具链满足 pixi.toml 中定义的版本要求。 如果你打算手动构建,请务必先确认你的工具链满足 pixi.toml 中定义的版本要求。
@@ -58,13 +70,30 @@ cmake -B build -G Ninja \
| CLICE_USE_LIBCXX | OFF | 是否使用 libc++ 来构建 clice添加 `-std=libc++`),如果开启,请确保 LLVM 库也是使用 libc++ 编译的 | | CLICE_USE_LIBCXX | OFF | 是否使用 libc++ 来构建 clice添加 `-std=libc++`),如果开启,请确保 LLVM 库也是使用 libc++ 编译的 |
| CLICE_CI_ENVIRONMENT | OFF | 是否打开 `CLICE_CI_ENVIRONMENT` 这个宏,有些测试在 CI 环境才会执行 | | CLICE_CI_ENVIRONMENT | OFF | 是否打开 `CLICE_CI_ENVIRONMENT` 这个宏,有些测试在 CI 环境才会执行 |
### XMake
使用如下命令即可构建 clice
```bash
xmake f -c --mode=releasedbg --toolchain=clang
xmake build --all
```
可选的构建选项:
| 选项 | 默认值 | 效果 |
| ------------- | ------ | ------------------------------------ |
| --llvm | "" | 使用自定义路径的 LLVM 库来构建 clice |
| --enable_test | false | 是否构建 clice 的单元测试 |
| --ci | false | 是否打开 `CLICE_CI_ENVIRONMENT` |
## About LLVM ## About LLVM
clice 调用 Clang API 来解析 C++ 代码,因此必须链接 LLVM/Clang 库。由于 clice 使用了 Clang 的私有头文件(这些文件通常不包含在发行版中),不能直接使用系统安装的 LLVM 包。 clice 调用 Clang API 来解析 C++ 代码,因此必须链接 LLVM/Clang 库。由于 clice 使用了 Clang 的私有头文件(这些文件通常不包含在发行版中),不能直接使用系统安装的 LLVM 包。
主要有两种方式解决这个依赖问题: 主要有两种方式解决这个依赖问题:
1. 我们在 [clice-llvm](https://github.com/clice-io/clice-llvm/releases) 上会发布使用的 LLVM 版本的预编译二进制,用于 CI 或者 release 构建。在构建时 cmake 默认会从此处下载 LLVM 库然后使用。 1. 我们在 [clice-llvm](https://github.com/clice-io/clice-llvm/releases) 上会发布使用的 LLVM 版本的预编译二进制,用于 CI 或者 release 构建。在构建时 cmake 和 xmake 默认会从此处下载 LLVM 库然后使用。
> [!IMPORTANT] > [!IMPORTANT]
> >

View File

@@ -30,7 +30,7 @@ pixi run publish-vscode
1. `pixi shell -e node` 1. `pixi shell -e node`
2.`editors/vscode` 下运行 `pnpm run watch`(增量构建) 2.`editors/vscode` 下运行 `pnpm run watch`(增量构建)
3. VSCode 中使用Run Extension/Launch Extension”调试配置或执行 `code --extensionDevelopmentPath=$(pwd)/editors/vscode` 3. VSCode 中使用Run Extension/Launch Extension” 调试配置,或执行 `code --extensionDevelopmentPath=$(pwd)/editors/vscode`
常用脚本(在 `pixi shell -e node` 下): 常用脚本(在 `pixi shell -e node` 下):

View File

@@ -18,6 +18,13 @@ $ ./build/bin/unit_tests --test-dir="./tests/data"
$ pytest -s --log-cli-level=INFO tests/integration --executable=./build/bin/clice $ pytest -s --log-cli-level=INFO tests/integration --executable=./build/bin/clice
``` ```
如果你使用 xmake 作为构建系统,可以直接通过 xmake 运行测试:
```shell
$ xmake run --verbose unit_tests
$ xmake test --verbose integration_tests/default
```
## Debug ## Debug
如果想在 clice 上附加调试器并进行调试,推荐先单独以 socket 模式启动 clice然后再将客户端连接到 clice 上 如果想在 clice 上附加调试器并进行调试,推荐先单独以 socket 模式启动 clice然后再将客户端连接到 clice 上

View File

@@ -54,73 +54,14 @@ bazel run @hedron_compile_commands//:refresh_all
### Visual Studio ### Visual Studio
Visual Studio2019 16.1+)可以通过 CMake 集成来生成编译数据库。将项目作为 CMake 项目打开,然后在 `CMakeSettings.json` 中配置: TODO:
```json
{
"configurations": [
{
"name": "x64-Debug",
"generator": "Ninja",
"buildRoot": "${projectDir}\\build",
"cmakeCommandArgs": "-DCMAKE_EXPORT_COMPILE_COMMANDS=ON"
}
]
}
```
对于基于 MSBuild 的项目(`.vcxproj`),可以使用 [compiledb-vs](https://github.com/pjbroad/compiledb-vs) 或 [catter](https://github.com/clice-io/catter) 来生成编译数据库。
### Makefile ### Makefile
对于基于 Makefile 的项目,使用 [bear](https://github.com/rizsotto/Bear) 来拦截编译命令: TODO:
```bash
bear -- make
```
这会在当前目录生成 `compile_commands.json`。注意 `bear` 需要干净的构建来捕获所有命令——如果需要的话,在运行 `bear -- make` 之前先执行 `make clean`
另外,如果使用 GNU Make也可以使用 [compiledb](https://github.com/nicktimko/compiledb)
```bash
compiledb make
```
### Meson
Meson 在配置阶段会自动生成编译数据库:
```bash
meson setup build
```
`compile_commands.json` 会生成在 `build` 目录下。
### Xmake ### Xmake
用下列任意方法生成编译数据库。
#### 命令行手动生成
在命令行中执行以下命令:
```bash
xmake project -k compile_commands --lsp=clangd build
```
> 通过这种方法生成的编译数据库无法自动更新,需要在项目编译配置更改时手动重新生成。
#### VSCode 扩展
Xmake 提供的官方 VSCode 扩展会在 `xmake.lua` 更新时自动生成编译数据库。然而默认情况下,它将编译数据库生成到了 `.vscode` 文件夹。在 `settings.json` 中添加以下配置:
```json
"xmake.compileCommandsDirectory": "build"
```
以将编译数据库的生成目录调整到 `build`,供 clice 使用。
### Others ### Others
对于任意其它的构建系统,可以使用 [catter](https://github.com/clice-io/catter) 来生成编译数据库。它通过伪装编译器的方式来捕获编译命令,能够可靠地与任何调用编译器可执行文件的构建系统配合工作 对于任意其它的构建系统,可以尝试使用 [bear](https://github.com/rizsotto/Bear) 或者 [scan-build](https://github.com/rizsotto/scan-build) 来拦截编译命令并获取到编译数据库(不保证成功)。我们计划在未来编写一个**新的工具**,通过假编译器的方式来实现编译命令的捕获

View File

@@ -0,0 +1,9 @@
{
// See http://go.microsoft.com/fwlink/?LinkId=827846
// for the documentation about the extensions.json format
"recommendations": [
"dbaeumer.vscode-eslint",
"amodio.tsl-problem-matcher",
"ms-vscode.extension-test-runner"
]
}

23
editors/vscode/.vscode/launch.json vendored Normal file
View File

@@ -0,0 +1,23 @@
{
"version": "0.2.0",
"configurations": [
{
"name": "Run Extension (socket)",
"type": "extensionHost",
"request": "launch",
"args": ["--disable-extensions", "--extensionDevelopmentPath=${workspaceFolder}"],
"env": { "CLICE_MODE": "socket" },
"outFiles": ["${workspaceFolder}/dist/**/*.js"],
"preLaunchTask": "${defaultBuildTask}"
},
{
"name": "Run Extension (pipe)",
"type": "extensionHost",
"request": "launch",
"args": ["--disable-extensions", "--extensionDevelopmentPath=${workspaceFolder}"],
"env": { "CLICE_MODE": "pipe" },
"outFiles": ["${workspaceFolder}/dist/**/*.js"],
"preLaunchTask": "${defaultBuildTask}"
}
]
}

13
editors/vscode/.vscode/settings.json vendored Normal file
View File

@@ -0,0 +1,13 @@
// Place your settings in this file to overwrite default and user settings.
{
"files.exclude": {
"out": false, // set this to true to hide the "out" folder with the compiled JS files
"dist": false // set this to true to hide the "dist" folder with the compiled JS files
},
"search.exclude": {
"out": true, // set this to false to include "out" folder in search results
"dist": true // set this to false to include "dist" folder in search results
},
// Turn off tsc task auto detection since we have the necessary tasks as npm scripts
"typescript.tsc.autoDetect": "off"
}

37
editors/vscode/.vscode/tasks.json vendored Normal file
View File

@@ -0,0 +1,37 @@
// See https://go.microsoft.com/fwlink/?LinkId=733558
// for the documentation about the tasks.json format
{
"version": "2.0.0",
"tasks": [
{
"type": "npm",
"script": "watch",
"problemMatcher": "$ts-webpack-watch",
"isBackground": true,
"presentation": {
"reveal": "never",
"group": "watchers"
},
"group": {
"kind": "build",
"isDefault": true
}
},
{
"type": "npm",
"script": "watch-tests",
"problemMatcher": "$tsc-watch",
"isBackground": true,
"presentation": {
"reveal": "never",
"group": "watchers"
},
"group": "build"
},
{
"label": "tasks: watch-tests",
"dependsOn": ["npm: watch", "npm: watch-tests"],
"problemMatcher": []
}
]
}

View File

@@ -0,0 +1,42 @@
cmake_minimum_required(VERSION 3.28)
project(clice_vscode_cmake_sample LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
add_library(sample_greeting greeting.cc)
target_include_directories(sample_greeting PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}")
add_executable(sample_app main.cc)
target_link_libraries(sample_app PRIVATE sample_greeting)
set(SAMPLE_MODULES_SUPPORTED OFF)
if(CMAKE_CXX_COMPILER_ID STREQUAL "Clang" AND
CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 16)
set(SAMPLE_MODULES_SUPPORTED ON)
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" AND
CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 14)
set(SAMPLE_MODULES_SUPPORTED ON)
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC" AND
CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 19.34)
set(SAMPLE_MODULES_SUPPORTED ON)
endif()
if(SAMPLE_MODULES_SUPPORTED)
add_executable(sample_module_app)
target_sources(sample_module_app PRIVATE main_module.cc)
target_sources(sample_module_app
PRIVATE
FILE_SET CXX_MODULES
FILES greeting_module.cppm
)
else()
message(STATUS
"Skipping sample_module_app because the active compiler lacks "
"CMake C++20 module scanning support. Use Clang >= 16, GCC >= 14, "
"or MSVC 19.34+ to enable it."
)
endif()

View File

@@ -0,0 +1,38 @@
# VS Code CMake Sample
This workspace is a standalone CMake project for attaching the VS Code extension to a real `clice` session.
`clice` already auto-detects `build/compile_commands.json`, so this sample does not need any helper scripts or extra CMake glue.
The workspace contains two entry points:
- `main.cc`: a traditional include-based example.
- `main_module.cc`: a C++20 modules example that imports `greeting_module.cppm`.
## Prepare The Workspace
From this directory:
```sh
cmake -S . -B build -G Ninja -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
```
That is enough to generate `build/compile_commands.json`, which `clice` can discover automatically when this folder is opened as the workspace.
If you also want the sample binary:
```sh
cmake --build build
```
## C++20 Modules
The module example is enabled automatically when CMake is using a compiler it can scan for C++20 modules:
- Clang 16+
- GCC 14+
- MSVC 19.34+
If the active compiler is older than that, CMake still configures the workspace and builds `sample_app`, but it skips `sample_module_app`.
When you do have a supported compiler, `main_module.cc` and `greeting_module.cppm` will also appear in `build/compile_commands.json`, which makes this workspace useful for testing clice's module handling in an editor.

View File

@@ -0,0 +1,7 @@
#include "greeting.h"
#include <string>
std::string build_greeting(std::string_view name) {
return "Hello, " + std::string(name) + " from the CMake sample.";
}

View File

@@ -0,0 +1,6 @@
#pragma once
#include <string>
#include <string_view>
std::string build_greeting(std::string_view name);

View File

@@ -0,0 +1,14 @@
module;
#include <string>
#include <string_view>
export module sample.greeting;
export namespace sample {
std::string build_module_greeting(std::string_view name) {
return "Hello, " + std::string(name) + " from the C++20 module sample.";
}
}

View File

@@ -0,0 +1,8 @@
#include "greeting.h"
#include <iostream>
int main() {
std::cout << build_greeting("clice") << '\n';
return 0;
}

View File

@@ -0,0 +1,8 @@
#include <iostream>
import sample.greeting;
int main() {
std::cout << sample::build_module_greeting("clice") << '\n';
return 0;
}

View File

@@ -0,0 +1,6 @@
#include <cstdio>
int main() {
printf("Hello, World!\n");
return 0;
}

View File

@@ -1,2 +0,0 @@
schema: spec-driven
created: 2026-04-22

View File

@@ -1,113 +0,0 @@
## Downloaded Upstream Reference
Downloaded from GitHub tag `llvmorg-21.1.8` into `openspec/changes/explore-improve-folding-range-support/reference/clangd/llvmorg-21.1.8/` using `curl`.
Files downloaded:
- `clang-tools-extra/clangd/SemanticSelection.cpp`
- `clang-tools-extra/clangd/SemanticSelection.h`
- `clang-tools-extra/clangd/ClangdServer.cpp`
- `clang-tools-extra/clangd/ClangdServer.h`
- `clang-tools-extra/clangd/ClangdLSPServer.cpp`
- `clang-tools-extra/clangd/Protocol.h`
- `clang-tools-extra/clangd/Protocol.cpp`
- `clang-tools-extra/clangd/test/folding-range.test`
- `clang-tools-extra/clangd/unittests/SemanticSelectionTests.cpp`
Raw GitHub URLs used:
- `https://raw.githubusercontent.com/llvm/llvm-project/llvmorg-21.1.8/clang-tools-extra/clangd/SemanticSelection.cpp`
- `https://raw.githubusercontent.com/llvm/llvm-project/llvmorg-21.1.8/clang-tools-extra/clangd/SemanticSelection.h`
- `https://raw.githubusercontent.com/llvm/llvm-project/llvmorg-21.1.8/clang-tools-extra/clangd/ClangdServer.cpp`
- `https://raw.githubusercontent.com/llvm/llvm-project/llvmorg-21.1.8/clang-tools-extra/clangd/ClangdServer.h`
- `https://raw.githubusercontent.com/llvm/llvm-project/llvmorg-21.1.8/clang-tools-extra/clangd/ClangdLSPServer.cpp`
- `https://raw.githubusercontent.com/llvm/llvm-project/llvmorg-21.1.8/clang-tools-extra/clangd/Protocol.h`
- `https://raw.githubusercontent.com/llvm/llvm-project/llvmorg-21.1.8/clang-tools-extra/clangd/Protocol.cpp`
- `https://raw.githubusercontent.com/llvm/llvm-project/llvmorg-21.1.8/clang-tools-extra/clangd/test/folding-range.test`
- `https://raw.githubusercontent.com/llvm/llvm-project/llvmorg-21.1.8/clang-tools-extra/clangd/unittests/SemanticSelectionTests.cpp`
## Clice Reference Files
Current branch files compared against:
- `src/feature/folding_ranges.cpp`
- `src/server/master_server.cpp`
- `tests/unit/feature/folding_range_tests.cpp`
## Confirmed Comparison Findings
### 1. clangd already has dedicated comment folding and line-only rendering
clangd's pseudo-parser folding path in `SemanticSelection.cpp` explicitly handles:
- bracket folds with line-only adjustment at `SemanticSelection.cpp:223-235`
- multiline block and contiguous comment-group folds at `SemanticSelection.cpp:238-269`
The request path wires `LineFoldingOnly` from client capabilities in:
- `ClangdLSPServer.cpp:545`
- `ClangdServer.cpp:967-980`
Regression coverage exists in:
- `test/folding-range.test:6-20`
- `unittests/SemanticSelectionTests.cpp:269-455`
Current clice does not have any comment collector in `src/feature/folding_ranges.cpp`, and the server request path in `src/server/master_server.cpp:517-525` forwards folding requests without any folding-specific options.
### 2. clice already folds more AST structure than clangd
clangd's AST-oriented `getFoldingRanges(ParsedAST &AST)` is intentionally narrow and only walks syntax-tree compound statements in `SemanticSelection.cpp:170-175`.
Current clice already folds:
- namespaces at `src/feature/folding_ranges.cpp:66-80`
- records and access-specifier regions at `src/feature/folding_ranges.cpp:82-121`
- function parameter lists and bodies at `src/feature/folding_ranges.cpp:123-144`, `246-269`
- lambda captures at `src/feature/folding_ranges.cpp:134-143`
- call argument lists at `src/feature/folding_ranges.cpp:146-179`
- initializer lists at `src/feature/folding_ranges.cpp:181-185`
- compound statements at `src/feature/folding_ranges.cpp:271-284`
That is materially broader than clangd's current AST folding baseline.
### 3. clice still exposes richer but less compatible output
Current clice maps many internal categories directly to custom kind strings in `src/feature/folding_ranges.cpp:35-54`, and carries `collapsed_text` through `src/feature/folding_ranges.cpp:56-60` and `src/feature/folding_ranges.cpp:363-376`.
clangd's downloaded protocol reference exposes only folding kinds in `Protocol.h:1970-1981` and serializes them in `Protocol.cpp:1680-1692`. The downloaded clangd protocol does not expose `collapsedText`, so `collapsedText` is a clice-specific protocol improvement rather than a clangd parity requirement.
### 4. clice still has an incomplete `#endif` branch closure bug
Current clice closes a prior conditional branch only when `#else` is seen at `src/feature/folding_ranges.cpp:302-311`. On `#endif`, it only pops the stack at `src/feature/folding_ranges.cpp:314-317` and emits no range for the final branch body.
clangd does not solve this either. Upstream `SemanticSelection.cpp:178-190` still leaves PP conditional regions and disabled regions as FIXME items. This means `#if` branch folding remains a clice extension opportunity, not a direct clangd parity target.
### 5. clice lacks client-capability plumbing for folding
Current clice only advertises `caps.folding_range_provider = true` in `src/server/master_server.cpp:244`, and the request handler in `src/server/master_server.cpp:517-525` forwards no `lineFoldingOnly`, `rangeLimit`, or `collapsedText` support signals into the feature layer.
clangd at least threads `LineFoldingOnly` from the client into folding generation via `ClangdLSPServer.cpp:545` and `ClangdServer.cpp:974-976`.
### 6. clice test coverage is still weaker in the most important gap areas
Current clice has structural tests, but the directive and pragma-region cases remain placeholder-only in `tests/unit/feature/folding_range_tests.cpp:398-430`. The tests also do not assert folding kinds.
clangd's downloaded tests cover:
- AST folding
- comment folding
- line-folding-only behavior
- macro-related exclusion cases
Those are visible in `unittests/SemanticSelectionTests.cpp:269-455`.
## Planning Implications
The downloaded source narrows the real parity target:
- confirmed clangd parity gaps for clice: comment folding, `lineFoldingOnly`, standard public kind behavior, stronger tests
- confirmed clice advantages over clangd: namespaces, access-specifier regions, lambda captures, function parameter folds, function-call folds, initializer folds, pragma regions, collapsed text
- confirmed clice-specific extension space beyond clangd: inactive-branch folding, complete `#if/#elif/#else/#endif` folding, macro-definition folding, include/import grouping
The earlier `third_party` vendor plan was the wrong storage model for this branch. The correct model is a change-local downloaded reference under `openspec/changes/explore-improve-folding-range-support/reference/`.

View File

@@ -1,279 +0,0 @@
## Context
`clice` currently implements folding ranges in `src/feature/folding_ranges.cpp`. The implementation is primarily an AST visitor with extra handling for conditional compilation and `#pragma region` data from `CompilationUnitRef::directives()`. It already covers many structural folds that clangd does not currently expose, such as namespaces, records, function parameter lists, lambda captures, call argument lists, access-specifier sections, and initializer lists.
The request path is currently split across:
- `src/feature/folding_ranges.cpp` for collection and rendering
- `src/server/master_server.cpp` for request plumbing and capability advertisement
- generated `kota` LSP protocol types for request/response shapes
- `tests/unit/feature/folding_range_tests.cpp` for unit coverage
That split reveals three immediate shortcomings:
- the current collector has no comment path at all
- folding-specific client capabilities such as `lineFoldingOnly`, `rangeLimit`, and `collapsedText` are not threaded through the request path
- directive-related tests are mostly placeholders and do not assert important behavior
The comparison target for this exploration change should be fixed and versioned. At tag `llvmorg-21.1.8`, clangd's folding behavior is centered on `clang-tools-extra/clangd/SemanticSelection.cpp`, with request plumbing in `ClangdServer.cpp` and `ClangdLSPServer.cpp`, protocol types in `Protocol.h` and `Protocol.cpp`, and regression coverage in `test/folding-range.test` plus `unittests/SemanticSelectionTests.cpp`. Those files have been downloaded into `openspec/changes/explore-improve-folding-range-support/reference/clangd/llvmorg-21.1.8/`, and the side-by-side analysis lives in `comparison.md`.
Compared with that clangd baseline, the current gap is clear:
- clangd already has behavior that `clice` still lacks:
- multiline comment folding
- contiguous `//` comment-group folding
- `lineFoldingOnly` rendering behavior wired from client capabilities into folding generation
- consistent use of standard public folding kinds
- a more complete and assertion-backed folding-range test matrix
- `clice` already has behavior that clangd does not:
- richer AST-structure folding
- `#pragma region` and some conditional-compilation folding
- `collapsedText`
- `clice` still has obvious opportunities that are not fully implemented yet:
- fully closing the last `#if/#elif/#else` branch at `#endif`
- folding inactive branches
- folding multiline macro definitions
- grouping contiguous `#include` / `import` blocks
- capability-aware `kind` and `collapsedText` rendering
In addition, the downloaded clangd source confirms that clangd still does not implement PP conditional regions, include grouping, or access-specifier folding in `SemanticSelection.cpp`; those are explicitly left as FIXME items upstream. The real parity target is therefore narrower than "match everything clangd does": comments, line-only rendering, standard kinds, and test discipline are the confirmed baseline gaps. Everything around directive groups, inactive branches, and richer structural categories remains a clice-specific extension opportunity.
## Goals / Non-Goals
**Goals:**
- Download a focused clangd reference set from `llvmorg-21.1.8` into this change directory and use it as the explicit comparison baseline for this branch.
- Preserve `clice`'s current advantage in AST-structure folding instead of regressing to clangd's much narrower block-only baseline.
- Fill the high-value baseline gaps that clangd already covers, especially multiline comments and `lineFoldingOnly`.
- Turn preprocessor metadata into a differentiating `clice` capability covering conditional branches, macro definitions, and include/import grouping.
- Make folding-range output respect client capabilities with predictable fallback behavior.
- Lock behavior down with unit and integration tests across AST, comments, preprocessor handling, and protocol negotiation.
**Non-Goals:**
- Import clangd implementation code directly into `clice` production paths or make the build depend on the downloaded reference files.
- Achieve byte-for-byte or range-for-range parity with clangd in this change.
- Add fine-grained folding for every C++ syntax detail such as template parameter lists, requires-clauses, or attribute arguments before their value is proven.
- Introduce editor-specific behavior that only exists to satisfy one frontend.
- Add cross-file or index-backed folding behavior.
## Decisions
### 1. Download a focused clangd reference set into the change directory before implementation work
The branch should first download a small, reviewable set of clangd's folding-related sources from tag `llvmorg-21.1.8` into `openspec/changes/explore-improve-folding-range-support/reference/clangd/llvmorg-21.1.8/`. The downloaded set should include the implementation, request plumbing, protocol types, and relevant tests that explain folding behavior, rather than the whole LLVM tree.
Why:
- it creates a stable review artifact for this exploration branch
- later implementation work can point at local upstream code instead of external URLs
- it keeps the eventual runtime change honest about what is parity work and what is a clice-specific extension
- it avoids adding a repo-level vendor location for a one-branch study artifact
Alternative considered:
- Put the files under `third_party/`. Rejected because this is an exploration artifact, not a production dependency.
### 2. Split the folding-range pipeline into collection, normalization, and rendering
The current implementation mixes "how a range is discovered" with "how it is emitted as LSP". The new design separates this into three layers:
- collection: produce internal `RawFoldingRange` entries from AST, comment scanning, and preprocessor metadata
- normalization: sort, deduplicate, validate, and reconcile nested or overlapping ranges
- rendering: decide line/column boundaries, `kind`, and `collapsedText` based on client capabilities
Why:
- `lineFoldingOnly`, `collapsedText`, and standards-compatible kind downgrading are rendering concerns and should not pollute collection logic
- comments, macros, and include/import groups do not naturally belong inside the AST visitor
- future range limiting or prioritization should also live in normalization/rendering instead of collector code
Follow-up discussion narrows this design point: the existing `RawFoldingRange` model is finished for the current pipeline work and should not be redesigned here. The missing part is an explicit options object, passed as `Opts`/`FoldingRangeOptions`, that lets callers configure renderer behavior such as `line_folding_only`.
Alternative considered:
- Keep generating final LSP ranges directly inside the visitor. Rejected because capability negotiation and multi-source collection will keep making the function larger and harder to test.
### 3. Keep rich internal categories, but only promise standard-compatible public kinds
Internally, the implementation may still distinguish namespace, class, function body, macro definition, conditional branch, and similar categories so tests, prioritization, and `collapsedText` selection remain precise. However, public LSP output should default to standard kinds only:
- comment folds -> `comment`
- contiguous include/import groups -> `imports`
- all other structural and preprocessor folds -> `region`
If some client later proves it needs clice-specific kinds, that can be evaluated separately. This change does not make non-standard kind strings part of the compatibility contract.
Why:
- many current custom strings will not be understood by clients and do not produce stable UI semantics
- the real differentiator is what `clice` can fold, not the literal `kind` label
- once public kinds are standardized, `collapsedText` and range boundaries become the primary user-visible expression
Alternative considered:
- Continue exposing all custom kinds directly. Rejected because that leaves client compatibility up to luck rather than protocol design.
### 4. Use the downloaded clangd files as a behavior reference, not as a direct implementation template
clangd's folding logic is text- and token-oriented rather than AST-oriented. `clice` should study the upstream behavior to match the useful parts, but it should not force its own collector architecture to look like clangd's when `CompilationUnitRef::directives()` and the existing AST visitor provide better raw data.
Why:
- parity should be measured at the behavior boundary, not by mirroring file structure
- `clice` already has data sources that clangd does not, especially for directive metadata
- this keeps the change focused on correctness and value, not on source-level imitation
Alternative considered:
- Rewrite `clice` folding collection to resemble clangd's text parser closely. Rejected because that would discard existing strengths without a clear benefit.
### 5. Implement comment folding through lexical/source scanning, not AST
Multiline comments are handled independently in clangd's pseudo-parser path, and `clice` should do the same. The design adds a comment collector that scans the main-file source or token stream directly:
- fold multiline `/* ... */` block comments
- fold contiguous `//` comment groups
- do not fold single-line comments
- preserve source spans that let the renderer adjust closing boundaries for `lineFoldingOnly` mode
Why:
- comments are not AST structure, so trying to derive them from AST produces fragile behavior
- lexical scanning naturally handles adjacent-comment grouping and block-comment boundaries
Alternative considered:
- Only support block comments. Rejected because clangd already demonstrates that contiguous `//` comment groups are a useful folding case.
### 6. Rework preprocessor folding around complete branch blocks instead of the current half-open stack
Today `collect_condition_directives()` only closes the previous branch when it sees `#else`, but when it sees `#endif` it only pops the stack and does not emit a folding range for the final `#if/#elif/#else` branch. As a result, `#if` folding is incomplete.
The new design treats conditional compilation as an explicit branch-group model:
- maintain the ordered branch chain for each `#if` group
- allow every branch to close at the next `#elif`, `#else`, or `#endif`
- distinguish active and inactive branches
- allow inactive branches to produce region folds, optionally with distinct `collapsedText`
Why:
- this is the minimum sound model needed to fix the current logical gap
- `Condition::ConditionValue` already records true/false/skipped state and can drive inactive-branch folding directly
Alternative considered:
- Patch only the `#endif` closing case. Rejected because nested conditions, inactive branches, and range ordering would remain structurally weak.
### 7. Add dedicated directive-based collectors for macros and include/import groups
`clice` already collects:
- `directive.macros`
- `directive.includes`
- `directive.imports`
The new design therefore adds directive-based folding collectors for:
- multiline `#define` macro definitions, using continuation backslashes or stable definition ranges
- contiguous `#include` blocks, merged into a single `imports` folding range
- contiguous `import Foo;` / `import Foo:Bar;` module-import blocks, also emitted as `imports`
Why:
- the necessary data already exists in preprocessing metadata and does not require new AST modeling
- this is one of the easiest places for `clice` to provide value beyond clangd
Alternative considered:
- Leave include/import grouping for a later change. Rejected because the metadata already exists, the implementation cost is relatively low, and the editor-facing value is immediate.
### 8. Separate clangd parity capabilities from clice-only protocol improvements
This change should treat comment folding, `lineFoldingOnly`, and standard public kinds as clangd parity work. `collapsedText` gating and deterministic `rangeLimit` trimming remain clice-side protocol improvements. The downloaded clangd `Protocol.h` / `Protocol.cpp` reference does not expose `collapsedText`, so the design and tests should not imply that clangd already provides that capability.
Why:
- it keeps the comparison honest
- it allows reviewer discussion to separate "must match upstream baseline" from "valuable extra behavior"
- it keeps spec language compatible with LSP without overstating clangd
Alternative considered:
- Treat all capability work as a clangd parity gap. Rejected because clangd's known folding path does not establish that broader claim.
### 9. Folding-range output must be explicitly bound to client capabilities
The master server currently only advertises `foldingRangeProvider = true`, but it does not read or propagate folding-specific client capabilities. The new design requires the session to track at least:
- `lineFoldingOnly`
- whether `collapsedText` is supported
- optional `rangeLimit`
Capability state should be translated into a feature-layer options object before rendering. The initial option needed by the current discussion is:
```cpp
struct FoldingRangeOptions {
bool line_folding_only = false;
};
```
The feature API should accept that options object separately from the source collector inputs, for example as `folding_ranges(unit, opts, encoding)`. Later protocol work can extend the same object for collapsed-text gating or range limiting without changing collectors.
Rendering rules:
- when `opts.line_folding_only = true`, only emit ranges that remain meaningful as line-based folds, adjusting end lines where necessary
- when the client does not support `collapsedText`, omit it
- when a `rangeLimit` is declared, trim results deterministically rather than arbitrarily
Alternative considered:
- Continue always returning exact columns and `collapsedText`. Rejected because that relies on client tolerance instead of following the protocol contract.
- Thread capability state into collectors directly. Rejected because it would reopen the raw model and collection contract even though line-only behavior is a renderer policy.
### 10. Organize tests by source category and protocol behavior
Tests will be split into two dimensions:
- source-category unit tests: AST structure, comments, conditional compilation, multiline macros, `#pragma region`, and include/import groups
- protocol-behavior tests: `lineFoldingOnly`, `collapsedText` support, public kind mapping, and range limiting
In particular, the current `tests/unit/feature/folding_range_tests.cpp` contains `Directive` and `PragmaRegion` cases that do not actually assert results. This change upgrades them into strong assertion-based tests.
Alternative considered:
- Rely mostly on manual editor validation. Rejected because folding details regress easily, especially for preprocessor handling and line-only rendering.
## Risks / Trade-offs
- [The downloaded clangd reference set could sprawl or become noisy in review] -> Mitigation: keep only the small folding-related file set needed for comparison under the change directory and record the exact URLs in `comparison.md`.
- [Client capabilities must flow from initialize state into request-time rendering] -> Mitigation: introduce a dedicated folding-options structure so session details do not leak broadly into the feature layer.
- [Inactive-branch and macro-definition ranges can be unstable around expansion locations] -> Mitigation: prefer spelling/main-file ranges and explicitly filter or special-case macro-expansion ranges when necessary.
- [Adding comments, macros, and include/import groups can increase the number of ranges quickly] -> Mitigation: implement stable sorting and `rangeLimit` trimming in the normalization layer.
- [Mapping public kinds back to standard values changes current metadata output] -> Mitigation: the folds themselves remain; the user-visible change is mostly in optional metadata, and tests plus change notes will make that explicit.
- [Multiple collectors may produce overlapping or duplicate ranges] -> Mitigation: normalize by source category and boundary rules so collectors do not amplify noise.
## Migration Plan
1. Download the focused clangd `llvmorg-21.1.8` folding reference files into `openspec/changes/explore-improve-folding-range-support/reference/clangd/llvmorg-21.1.8/`.
2. Record the confirmed clangd-vs-clice comparison in this change, including exact URLs, which behaviors are parity gaps, and which are clice-specific extensions.
3. Keep the existing `RawFoldingRange` data flow, add `FoldingRangeOptions` for `line_folding_only`, and add standard kind mapping.
4. Add the comment collector and assertion-backed tests for multiline comment folding.
5. Rewrite conditional-directive and `#pragma region` collection so `#if` branches close correctly through `#endif`.
6. Add multiline macro folding and grouped include/import collectors.
7. Wire folding client capabilities through initialize/request handling and add integration coverage.
8. Add `rangeLimit` trimming and regression cleanup after the new collectors are in place.
Rollback strategy:
- If the downloaded reference set becomes more distracting than useful, keep only the documented comparison notes and delete the change-local downloads before merging.
- If protocol negotiation proves unstable, keep the new collectors but temporarily disable outward behavior changes tied to `collapsedText` or `rangeLimit`.
- If a particular new fold category proves noisy, roll it back collector-by-collector instead of reverting the entire folding-range refactor.
## Open Questions
- Are `test/folding-range.test` and `unittests/SemanticSelectionTests.cpp` enough for ongoing comparison, or will later implementation work need more upstream folding-related tests?
- Should multiline macro folding cover only the macro body, or the full `#define NAME(...)` line plus body as one fold region?
- Should `rangeLimit` prioritize outer structure, top-of-file regions, or longer ranges when trimming results?
- For structural AST folds originating from macro expansion, should `clice` preserve current behavior or restrict itself to cases with stable spelling ranges only?

View File

@@ -1,36 +0,0 @@
## Why
`clice` already goes beyond clangd in several structural folding cases: it can fold namespaces, records, function parameter lists and bodies, lambda captures, call argument lists, access-specifier sections, and some preprocessor regions. However, the current implementation in `src/feature/folding_ranges.cpp` still misses several baseline behaviors that clangd already exposes well, especially multiline comment folding, line-only folding rendering, standard public folding kinds, and a stronger regression test matrix. Its preprocessor branch folding is also not yet fully closed.
This exploration branch needs a fixed upstream reference instead of relying on memory. At tag `llvmorg-21.1.8`, clangd's folding implementation is centered around `clang-tools-extra/clangd/SemanticSelection.cpp`, with request plumbing in `ClangdServer.cpp` and `ClangdLSPServer.cpp`, protocol types in `Protocol.h` and `Protocol.cpp`, and folding coverage in `test/folding-range.test` plus `unittests/SemanticSelectionTests.cpp`. Downloading those files into this change directory with `curl` gives the branch a stable, reviewable baseline for side-by-side comparison without introducing a repository-level vendor tree.
More importantly, `clice` already has preprocessor metadata that clangd does not fully exploit, such as `directive.macros`, `directive.includes`, `directive.imports`, and evaluated conditional-branch state. That means `clice` should not stop at matching clangd: after filling the real parity gaps, folding ranges can become a more useful C/C++ feature by covering macro definitions, `#if` branches, and include/import groups that clangd does not currently handle well.
Follow-up discussion clarified the split with `split-folding-range-pipeline`: the existing `RawFoldingRange` model is finished for the current architecture work. The missing capability path is explicit folding options, passed as `Opts`/`FoldingRangeOptions`, so `line_folding_only` can be requested by server capability plumbing and consumed by the renderer.
## What Changes
- Download the clangd folding-range reference files for tag `llvmorg-21.1.8` into `openspec/changes/explore-improve-folding-range-support/reference/clangd/llvmorg-21.1.8/` using `curl` from GitHub raw URLs.
- Record a concrete clangd-vs-clice comparison in `comparison.md`, including the upstream files consulted, exact download URLs, confirmed parity gaps, clice-only capabilities, and known implementation bugs.
- Fill the remaining folding-range baseline gaps between `clice` and clangd, especially multiline comment folding, line-only folding rendering, and standard public kind mapping.
- Complete preprocessor-related folding so full `#if/#elif/#else/#endif` branch regions, nested `#pragma region` blocks, and inactive branches have well-defined behavior.
- Add folding features that take advantage of `clice`'s existing preprocessor metadata, including multiline macro definitions and grouped `#include` / `import` blocks.
- Normalize `FoldingRange.kind` output so standard kinds remain compatible while clice-specific fold categories degrade predictably.
- Make folding range responses honor client capabilities such as `lineFoldingOnly`, optional `collapsedText` support, and range limiting, using folding options rather than collector-specific state.
- Expand unit and integration coverage for AST folds, comments, preprocessor regions, macros, include/import groups, and protocol negotiation behavior.
## Capabilities
### New Capabilities
- `folding-ranges`: Provide LSP-compatible, C/C++-focused folding regions that cover AST structure, comments, preprocessor branches, macro definitions, and include/import groups.
### Modified Capabilities
- None.
## Impact
- A change-local upstream reference set has been downloaded under `openspec/changes/explore-improve-folding-range-support/reference/clangd/llvmorg-21.1.8/`, limited to the folding-range implementation, protocol, and tests needed for analysis.
- The side-by-side analysis is recorded in `openspec/changes/explore-improve-folding-range-support/comparison.md`.
- Primary runtime impact is in `src/feature/folding_ranges.cpp`, the compile-unit/preprocessor metadata access paths, request handling in `src/server/master_server.cpp`, and the folding options object used to carry capability-derived rendering choices.
- Tests need expansion in `tests/unit/feature/folding_range_tests.cpp`, server/integration coverage, and any required fixtures for preprocessor and module scenarios.
- User-visible behavior will be folding results that are closer to clangd where clangd already has coverage, while also adding high-value C/C++ folds that clangd does not currently provide well, especially macro-definition and conditional-compilation folding.

View File

@@ -1,531 +0,0 @@
//===--- ClangdServer.h - Main clangd server code ----------------*- C++-*-===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_CLANGDSERVER_H
#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_CLANGDSERVER_H
#include "CodeComplete.h"
#include "ConfigProvider.h"
#include "Diagnostics.h"
#include "DraftStore.h"
#include "FeatureModule.h"
#include "GlobalCompilationDatabase.h"
#include "Hover.h"
#include "ModulesBuilder.h"
#include "Protocol.h"
#include "SemanticHighlighting.h"
#include "TUScheduler.h"
#include "XRefs.h"
#include "index/Background.h"
#include "index/FileIndex.h"
#include "index/Index.h"
#include "refactor/Rename.h"
#include "refactor/Tweak.h"
#include "support/Function.h"
#include "support/MemoryTree.h"
#include "support/Path.h"
#include "support/ThreadsafeFS.h"
#include "clang/Tooling/Core/Replacement.h"
#include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/FunctionExtras.h"
#include "llvm/ADT/StringRef.h"
#include <functional>
#include <memory>
#include <optional>
#include <string>
#include <tuple>
#include <vector>
namespace clang {
namespace clangd {
/// Manages a collection of source files and derived data (ASTs, indexes),
/// and provides language-aware features such as code completion.
///
/// The primary client is ClangdLSPServer which exposes these features via
/// the Language Server protocol. ClangdServer may also be embedded directly,
/// though its API is not stable over time.
///
/// ClangdServer should be used from a single thread. Many potentially-slow
/// operations have asynchronous APIs and deliver their results on another
/// thread.
/// Such operations support cancellation: if the caller sets up a cancelable
/// context, many operations will notice cancellation and fail early.
/// (ClangdLSPServer uses this to implement $/cancelRequest).
class ClangdServer {
public:
/// Interface with hooks for users of ClangdServer to be notified of events.
class Callbacks {
public:
virtual ~Callbacks() = default;
/// Called by ClangdServer when \p Diagnostics for \p File are ready.
/// These pushed diagnostics might correspond to an older version of the
/// file, they do not interfere with "pull-based" ClangdServer::diagnostics.
/// May be called concurrently for separate files, not for a single file.
virtual void onDiagnosticsReady(PathRef File, llvm::StringRef Version,
llvm::ArrayRef<Diag> Diagnostics) {}
/// Called whenever the file status is updated.
/// May be called concurrently for separate files, not for a single file.
virtual void onFileUpdated(PathRef File, const TUStatus &Status) {}
/// Called when background indexing tasks are enqueued/started/completed.
/// Not called concurrently.
virtual void
onBackgroundIndexProgress(const BackgroundQueue::Stats &Stats) {}
/// Called when the meaning of a source code may have changed without an
/// edit. Usually clients assume that responses to requests are valid until
/// they next edit the file. If they're invalidated at other times, we
/// should tell the client. In particular, when an asynchronous preamble
/// build finishes, we can provide more accurate semantic tokens, so we
/// should tell the client to refresh.
virtual void onSemanticsMaybeChanged(PathRef File) {}
/// Called by ClangdServer when some \p InactiveRegions for \p File are
/// ready.
virtual void onInactiveRegionsReady(PathRef File,
std::vector<Range> InactiveRegions) {}
};
/// Creates a context provider that loads and installs config.
/// Errors in loading config are reported as diagnostics via Callbacks.
/// (This is typically used as ClangdServer::Options::ContextProvider).
static std::function<Context(PathRef)>
createConfiguredContextProvider(const config::Provider *Provider,
ClangdServer::Callbacks *);
struct Options {
/// To process requests asynchronously, ClangdServer spawns worker threads.
/// If this is zero, no threads are spawned. All work is done on the calling
/// thread, and callbacks are invoked before "async" functions return.
unsigned AsyncThreadsCount = getDefaultAsyncThreadsCount();
/// AST caching policy. The default is to keep up to 3 ASTs in memory.
ASTRetentionPolicy RetentionPolicy;
/// Cached preambles are potentially large. If false, store them on disk.
bool StorePreamblesInMemory = true;
/// Call hierarchy's outgoing calls feature requires additional index
/// serving structures which increase memory usage. If false, these are
/// not created and the feature is not enabled.
bool EnableOutgoingCalls = true;
/// This throttler controls which preambles may be built at a given time.
clangd::PreambleThrottler *PreambleThrottler = nullptr;
/// Manages to build module files.
ModulesBuilder *ModulesManager = nullptr;
/// If true, ClangdServer builds a dynamic in-memory index for symbols in
/// opened files and uses the index to augment code completion results.
bool BuildDynamicSymbolIndex = false;
/// If true, ClangdServer automatically indexes files in the current project
/// on background threads. The index is stored in the project root.
bool BackgroundIndex = false;
llvm::ThreadPriority BackgroundIndexPriority = llvm::ThreadPriority::Low;
/// If set, use this index to augment code completion results.
SymbolIndex *StaticIndex = nullptr;
/// If set, queried to derive a processing context for some work.
/// Usually used to inject Config (see createConfiguredContextProvider).
///
/// When the provider is called, the active context will be that inherited
/// from the request (e.g. addDocument()), or from the ClangdServer
/// constructor if there is no such request (e.g. background indexing).
///
/// The path is an absolute path of the file being processed.
/// If there is no particular file (e.g. project loading) then it is empty.
std::function<Context(PathRef)> ContextProvider;
/// The Options provider to use when running clang-tidy. If null, clang-tidy
/// checks will be disabled.
TidyProviderRef ClangTidyProvider;
/// Clangd's workspace root. Relevant for "workspace" operations not bound
/// to a particular file.
/// FIXME: If not set, should use the current working directory.
std::optional<std::string> WorkspaceRoot;
/// The resource directory is used to find internal headers, overriding
/// defaults and -resource-dir compiler flag).
/// If std::nullopt, ClangdServer calls
/// CompilerInvocation::GetResourcePath() to obtain the standard resource
/// directory.
std::optional<std::string> ResourceDir;
/// Time to wait after a new file version before computing diagnostics.
DebouncePolicy UpdateDebounce = DebouncePolicy{
/*Min=*/std::chrono::milliseconds(50),
/*Max=*/std::chrono::milliseconds(500),
/*RebuildRatio=*/1,
};
/// Cancel certain requests if the file changes before they begin running.
/// This is useful for "transient" actions like enumerateTweaks that were
/// likely implicitly generated, and avoids redundant work if clients forget
/// to cancel. Clients that always cancel stale requests should clear this.
bool ImplicitCancellation = true;
/// Clangd will execute compiler drivers matching one of these globs to
/// fetch system include path.
std::vector<std::string> QueryDriverGlobs;
// Whether the client supports folding only complete lines.
bool LineFoldingOnly = false;
FeatureModuleSet *FeatureModules = nullptr;
/// If true, use the dirty buffer contents when building Preambles.
bool UseDirtyHeaders = false;
// If true, parse emplace-like functions in the preamble.
bool PreambleParseForwardingFunctions = true;
/// Whether include fixer insertions for Objective-C code should use #import
/// instead of #include.
bool ImportInsertions = false;
/// Whether to collect and publish information about inactive preprocessor
/// regions in the document.
bool PublishInactiveRegions = false;
explicit operator TUScheduler::Options() const;
};
// Sensible default options for use in tests.
// Features like indexing must be enabled if desired.
static Options optsForTest();
/// Creates a new ClangdServer instance.
///
/// ClangdServer uses \p CDB to obtain compilation arguments for parsing. Note
/// that ClangdServer only obtains compilation arguments once for each newly
/// added file (i.e., when processing a first call to addDocument) and reuses
/// those arguments for subsequent reparses. However, ClangdServer will check
/// if compilation arguments changed on calls to forceReparse().
ClangdServer(const GlobalCompilationDatabase &CDB, const ThreadsafeFS &TFS,
const Options &Opts, Callbacks *Callbacks = nullptr);
~ClangdServer();
/// Gets the installed feature module of a given type, if any.
/// This exposes access the public interface of feature modules that have one.
template <typename Mod> Mod *featureModule() {
return FeatureModules ? FeatureModules->get<Mod>() : nullptr;
}
template <typename Mod> const Mod *featureModule() const {
return FeatureModules ? FeatureModules->get<Mod>() : nullptr;
}
/// Add a \p File to the list of tracked C++ files or update the contents if
/// \p File is already tracked. Also schedules parsing of the AST for it on a
/// separate thread. When the parsing is complete, DiagConsumer passed in
/// constructor will receive onDiagnosticsReady callback.
/// Version identifies this snapshot and is propagated to ASTs, preambles,
/// diagnostics etc built from it. If empty, a version number is generated.
void addDocument(PathRef File, StringRef Contents,
llvm::StringRef Version = "null",
WantDiagnostics WD = WantDiagnostics::Auto,
bool ForceRebuild = false);
/// Remove \p File from list of tracked files, schedule a request to free
/// resources associated with it. Pending diagnostics for closed files may not
/// be delivered, even if requested with WantDiags::Auto or WantDiags::Yes.
/// An empty set of diagnostics will be delivered, with Version = "".
void removeDocument(PathRef File);
/// Requests a reparse of currently opened files using their latest source.
/// This will typically only rebuild if something other than the source has
/// changed (e.g. the CDB yields different flags, or files included in the
/// preamble have been modified).
void reparseOpenFilesIfNeeded(
llvm::function_ref<bool(llvm::StringRef File)> Filter);
/// Run code completion for \p File at \p Pos.
///
/// This method should only be called for currently tracked files.
void codeComplete(PathRef File, Position Pos,
const clangd::CodeCompleteOptions &Opts,
Callback<CodeCompleteResult> CB);
/// Provide signature help for \p File at \p Pos. This method should only be
/// called for tracked files.
void signatureHelp(PathRef File, Position Pos, MarkupKind DocumentationFormat,
Callback<SignatureHelp> CB);
/// Find declaration/definition locations of symbol at a specified position.
void locateSymbolAt(PathRef File, Position Pos,
Callback<std::vector<LocatedSymbol>> CB);
/// Switch to a corresponding source file when given a header file, and vice
/// versa.
void switchSourceHeader(PathRef Path,
Callback<std::optional<clangd::Path>> CB);
/// Get document highlights for a given position.
void findDocumentHighlights(PathRef File, Position Pos,
Callback<std::vector<DocumentHighlight>> CB);
/// Get code hover for a given position.
void findHover(PathRef File, Position Pos,
Callback<std::optional<HoverInfo>> CB);
/// Get information about type hierarchy for a given position.
void typeHierarchy(PathRef File, Position Pos, int Resolve,
TypeHierarchyDirection Direction,
Callback<std::vector<TypeHierarchyItem>> CB);
/// Get direct parents of a type hierarchy item.
void superTypes(const TypeHierarchyItem &Item,
Callback<std::optional<std::vector<TypeHierarchyItem>>> CB);
/// Get direct children of a type hierarchy item.
void subTypes(const TypeHierarchyItem &Item,
Callback<std::vector<TypeHierarchyItem>> CB);
/// Resolve type hierarchy item in the given direction.
void resolveTypeHierarchy(TypeHierarchyItem Item, int Resolve,
TypeHierarchyDirection Direction,
Callback<std::optional<TypeHierarchyItem>> CB);
/// Get information about call hierarchy for a given position.
void prepareCallHierarchy(PathRef File, Position Pos,
Callback<std::vector<CallHierarchyItem>> CB);
/// Resolve incoming calls for a given call hierarchy item.
void incomingCalls(const CallHierarchyItem &Item,
Callback<std::vector<CallHierarchyIncomingCall>>);
/// Resolve outgoing calls for a given call hierarchy item.
void outgoingCalls(const CallHierarchyItem &Item,
Callback<std::vector<CallHierarchyOutgoingCall>>);
/// Resolve inlay hints for a given document.
void inlayHints(PathRef File, std::optional<Range> RestrictRange,
Callback<std::vector<InlayHint>>);
/// Retrieve the top symbols from the workspace matching a query.
void workspaceSymbols(StringRef Query, int Limit,
Callback<std::vector<SymbolInformation>> CB);
/// Retrieve the symbols within the specified file.
void documentSymbols(StringRef File,
Callback<std::vector<DocumentSymbol>> CB);
/// Retrieve ranges that can be used to fold code within the specified file.
void foldingRanges(StringRef File, Callback<std::vector<FoldingRange>> CB);
/// Retrieve implementations for virtual method.
void findImplementations(PathRef File, Position Pos,
Callback<std::vector<LocatedSymbol>> CB);
/// Retrieve symbols for types referenced at \p Pos.
void findType(PathRef File, Position Pos,
Callback<std::vector<LocatedSymbol>> CB);
/// Retrieve locations for symbol references.
void findReferences(PathRef File, Position Pos, uint32_t Limit,
bool AddContainer, Callback<ReferencesResult> CB);
/// Run formatting for the \p File with content \p Code.
/// If \p Rng is non-empty, formats only those regions.
void formatFile(PathRef File, const std::vector<Range> &Rngs,
Callback<tooling::Replacements> CB);
/// Run formatting after \p TriggerText was typed at \p Pos in \p File with
/// content \p Code.
void formatOnType(PathRef File, Position Pos, StringRef TriggerText,
Callback<std::vector<TextEdit>> CB);
/// Test the validity of a rename operation.
///
/// If NewName is provided, it performs a name validation.
void prepareRename(PathRef File, Position Pos,
std::optional<std::string> NewName,
const RenameOptions &RenameOpts,
Callback<RenameResult> CB);
/// Rename all occurrences of the symbol at the \p Pos in \p File to
/// \p NewName.
/// If WantFormat is false, the final TextEdit will be not formatted,
/// embedders could use this method to get all occurrences of the symbol (e.g.
/// highlighting them in prepare stage).
void rename(PathRef File, Position Pos, llvm::StringRef NewName,
const RenameOptions &Opts, Callback<RenameResult> CB);
struct TweakRef {
std::string ID; /// ID to pass for applyTweak.
std::string Title; /// A single-line message to show in the UI.
llvm::StringLiteral Kind;
};
// Ref to the clangd::Diag.
struct DiagRef {
clangd::Range Range;
std::string Message;
bool operator==(const DiagRef &Other) const {
return std::tie(Range, Message) == std::tie(Other.Range, Other.Message);
}
bool operator<(const DiagRef &Other) const {
return std::tie(Range, Message) < std::tie(Other.Range, Other.Message);
}
};
struct CodeActionInputs {
std::string File;
Range Selection;
/// Requested kind of actions to return.
std::vector<std::string> RequestedActionKinds;
/// Diagnostics attached to the code action request.
std::vector<DiagRef> Diagnostics;
/// Tweaks where Filter returns false will not be checked or included.
std::function<bool(const Tweak &)> TweakFilter;
};
struct CodeActionResult {
std::string Version;
struct QuickFix {
DiagRef Diag;
Fix F;
};
std::vector<QuickFix> QuickFixes;
std::vector<TweakRef> TweakRefs;
struct Rename {
DiagRef Diag;
std::string FixMessage;
std::string NewName;
};
std::vector<Rename> Renames;
};
/// Surface code actions (quick-fixes for diagnostics, or available code
/// tweaks) for a given range in a file.
void codeAction(const CodeActionInputs &Inputs,
Callback<CodeActionResult> CB);
/// Apply the code tweak with a specified \p ID.
void applyTweak(PathRef File, Range Sel, StringRef ID,
Callback<Tweak::Effect> CB);
/// Called when an event occurs for a watched file in the workspace.
void onFileEvent(const DidChangeWatchedFilesParams &Params);
/// Get symbol info for given position.
/// Clangd extension - not part of official LSP.
void symbolInfo(PathRef File, Position Pos,
Callback<std::vector<SymbolDetails>> CB);
/// Get semantic ranges around a specified position in a file.
void semanticRanges(PathRef File, const std::vector<Position> &Pos,
Callback<std::vector<SelectionRange>> CB);
/// Get all document links in a file.
void documentLinks(PathRef File, Callback<std::vector<DocumentLink>> CB);
void semanticHighlights(PathRef File,
Callback<std::vector<HighlightingToken>>);
/// Describe the AST subtree for a piece of code.
void getAST(PathRef File, std::optional<Range> R,
Callback<std::optional<ASTNode>> CB);
/// Runs an arbitrary action that has access to the AST of the specified file.
/// The action will execute on one of ClangdServer's internal threads.
/// The AST is only valid for the duration of the callback.
/// As with other actions, the file must have been opened.
void customAction(PathRef File, llvm::StringRef Name,
Callback<InputsAndAST> Action);
/// Fetches diagnostics for current version of the \p File. This might fail if
/// server is busy (building a preamble) and would require a long time to
/// prepare diagnostics. If it fails, clients should wait for
/// onSemanticsMaybeChanged and then retry.
/// These 'pulled' diagnostics do not interfere with the diagnostics 'pushed'
/// to Callbacks::onDiagnosticsReady, and clients may use either or both.
void diagnostics(PathRef File, Callback<std::vector<Diag>> CB);
/// Returns estimated memory usage and other statistics for each of the
/// currently open files.
/// Overall memory usage of clangd may be significantly more than reported
/// here, as this metric does not account (at least) for:
/// - memory occupied by static and dynamic index,
/// - memory required for in-flight requests,
/// FIXME: those metrics might be useful too, we should add them.
llvm::StringMap<TUScheduler::FileStats> fileStats() const;
/// Gets the contents of a currently tracked file. Returns nullptr if the file
/// isn't being tracked.
std::shared_ptr<const std::string> getDraft(PathRef File) const;
// Blocks the main thread until the server is idle. Only for use in tests.
// Returns false if the timeout expires.
// FIXME: various subcomponents each get the full timeout, so it's more of
// an order of magnitude than a hard deadline.
[[nodiscard]] bool
blockUntilIdleForTest(std::optional<double> TimeoutSeconds = 10);
/// Builds a nested representation of memory used by components.
void profile(MemoryTree &MT) const;
private:
FeatureModuleSet *FeatureModules;
const GlobalCompilationDatabase &CDB;
const ThreadsafeFS &getHeaderFS() const {
return UseDirtyHeaders ? *DirtyFS : TFS;
}
const ThreadsafeFS &TFS;
Path ResourceDir;
// The index used to look up symbols. This could be:
// - null (all index functionality is optional)
// - the dynamic index owned by ClangdServer (DynamicIdx)
// - the static index passed to the constructor
// - a merged view of a static and dynamic index (MergedIndex)
const SymbolIndex *Index = nullptr;
// If present, an index of symbols in open files. Read via *Index.
std::unique_ptr<FileIndex> DynamicIdx;
// If present, the new "auto-index" maintained in background threads.
std::unique_ptr<BackgroundIndex> BackgroundIdx;
// Storage for merged views of the various indexes.
std::vector<std::unique_ptr<SymbolIndex>> MergedIdx;
// Manage module files.
ModulesBuilder *ModulesManager = nullptr;
// When set, provides clang-tidy options for a specific file.
TidyProviderRef ClangTidyProvider;
bool UseDirtyHeaders = false;
// Whether the client supports folding only complete lines.
bool LineFoldingOnly = false;
bool PreambleParseForwardingFunctions = true;
bool ImportInsertions = false;
bool PublishInactiveRegions = false;
// GUARDED_BY(CachedCompletionFuzzyFindRequestMutex)
llvm::StringMap<std::optional<FuzzyFindRequest>>
CachedCompletionFuzzyFindRequestByFile;
mutable std::mutex CachedCompletionFuzzyFindRequestMutex;
std::optional<std::string> WorkspaceRoot;
std::optional<AsyncTaskRunner> IndexTasks; // for stdlib indexing.
std::optional<TUScheduler> WorkScheduler;
// Invalidation policy used for actions that we assume are "transient".
TUScheduler::ASTActionInvalidation Transient;
// Store of the current versions of the open documents.
// Only written from the main thread (despite being threadsafe).
DraftStore DraftMgr;
std::unique_ptr<ThreadsafeFS> DirtyFS;
};
} // namespace clangd
} // namespace clang
#endif

View File

@@ -1,274 +0,0 @@
//===--- SemanticSelection.cpp -----------------------------------*- C++-*-===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
#include "SemanticSelection.h"
#include "ParsedAST.h"
#include "Protocol.h"
#include "Selection.h"
#include "SourceCode.h"
#include "clang/AST/DeclBase.h"
#include "clang/Basic/SourceLocation.h"
#include "clang/Basic/SourceManager.h"
#include "clang/Tooling/Syntax/BuildTree.h"
#include "clang/Tooling/Syntax/Nodes.h"
#include "clang/Tooling/Syntax/TokenBufferTokenManager.h"
#include "clang/Tooling/Syntax/Tree.h"
#include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/StringRef.h"
#include "llvm/Support/Casting.h"
#include "llvm/Support/Error.h"
#include "support/Bracket.h"
#include "support/DirectiveTree.h"
#include "support/Token.h"
#include <optional>
#include <queue>
#include <vector>
namespace clang {
namespace clangd {
namespace {
// Adds Range \p R to the Result if it is distinct from the last added Range.
// Assumes that only consecutive ranges can coincide.
void addIfDistinct(const Range &R, std::vector<Range> &Result) {
if (Result.empty() || Result.back() != R) {
Result.push_back(R);
}
}
std::optional<FoldingRange> toFoldingRange(SourceRange SR,
const SourceManager &SM) {
const auto Begin = SM.getDecomposedLoc(SR.getBegin()),
End = SM.getDecomposedLoc(SR.getEnd());
// Do not produce folding ranges if either range ends is not within the main
// file. Macros have their own FileID so this also checks if locations are not
// within the macros.
if ((Begin.first != SM.getMainFileID()) || (End.first != SM.getMainFileID()))
return std::nullopt;
FoldingRange Range;
Range.startCharacter = SM.getColumnNumber(Begin.first, Begin.second) - 1;
Range.startLine = SM.getLineNumber(Begin.first, Begin.second) - 1;
Range.endCharacter = SM.getColumnNumber(End.first, End.second) - 1;
Range.endLine = SM.getLineNumber(End.first, End.second) - 1;
return Range;
}
std::optional<FoldingRange>
extractFoldingRange(const syntax::Node *Node,
const syntax::TokenBufferTokenManager &TM) {
if (const auto *Stmt = dyn_cast<syntax::CompoundStatement>(Node)) {
const auto *LBrace = cast_or_null<syntax::Leaf>(
Stmt->findChild(syntax::NodeRole::OpenParen));
// FIXME(kirillbobyrev): This should find the last child. Compound
// statements have only one pair of braces so this is valid but for other
// node kinds it might not be correct.
const auto *RBrace = cast_or_null<syntax::Leaf>(
Stmt->findChild(syntax::NodeRole::CloseParen));
if (!LBrace || !RBrace)
return std::nullopt;
// Fold the entire range within braces, including whitespace.
const SourceLocation LBraceLocInfo =
TM.getToken(LBrace->getTokenKey())->endLocation(),
RBraceLocInfo =
TM.getToken(RBrace->getTokenKey())->location();
auto Range = toFoldingRange(SourceRange(LBraceLocInfo, RBraceLocInfo),
TM.sourceManager());
// Do not generate folding range for compound statements without any
// nodes and newlines.
if (Range && Range->startLine != Range->endLine)
return Range;
}
return std::nullopt;
}
// Traverse the tree and collect folding ranges along the way.
std::vector<FoldingRange>
collectFoldingRanges(const syntax::Node *Root,
const syntax::TokenBufferTokenManager &TM) {
std::queue<const syntax::Node *> Nodes;
Nodes.push(Root);
std::vector<FoldingRange> Result;
while (!Nodes.empty()) {
const syntax::Node *Node = Nodes.front();
Nodes.pop();
const auto Range = extractFoldingRange(Node, TM);
if (Range)
Result.push_back(*Range);
if (const auto *T = dyn_cast<syntax::Tree>(Node))
for (const auto *NextNode = T->getFirstChild(); NextNode;
NextNode = NextNode->getNextSibling())
Nodes.push(NextNode);
}
return Result;
}
} // namespace
llvm::Expected<SelectionRange> getSemanticRanges(ParsedAST &AST, Position Pos) {
std::vector<Range> Ranges;
const auto &SM = AST.getSourceManager();
const auto &LangOpts = AST.getLangOpts();
auto FID = SM.getMainFileID();
auto Offset = positionToOffset(SM.getBufferData(FID), Pos);
if (!Offset) {
return Offset.takeError();
}
// Get node under the cursor.
SelectionTree ST = SelectionTree::createRight(
AST.getASTContext(), AST.getTokens(), *Offset, *Offset);
for (const auto *Node = ST.commonAncestor(); Node != nullptr;
Node = Node->Parent) {
if (const Decl *D = Node->ASTNode.get<Decl>()) {
if (llvm::isa<TranslationUnitDecl>(D)) {
break;
}
}
auto SR = toHalfOpenFileRange(SM, LangOpts, Node->ASTNode.getSourceRange());
if (!SR || SM.getFileID(SR->getBegin()) != SM.getMainFileID()) {
continue;
}
Range R;
R.start = sourceLocToPosition(SM, SR->getBegin());
R.end = sourceLocToPosition(SM, SR->getEnd());
addIfDistinct(R, Ranges);
}
if (Ranges.empty()) {
// LSP provides no way to signal "the point is not within a semantic range".
// Return an empty range at the point.
SelectionRange Empty;
Empty.range.start = Empty.range.end = Pos;
return std::move(Empty);
}
// Convert to the LSP linked-list representation.
SelectionRange Head;
Head.range = std::move(Ranges.front());
SelectionRange *Tail = &Head;
for (auto &Range :
llvm::MutableArrayRef(Ranges.data(), Ranges.size()).drop_front()) {
Tail->parent = std::make_unique<SelectionRange>();
Tail = Tail->parent.get();
Tail->range = std::move(Range);
}
return std::move(Head);
}
// FIXME(kirillbobyrev): Collect comments, PP conditional regions, includes and
// other code regions (e.g. public/private/protected sections of classes,
// control flow statement bodies).
// Related issue: https://github.com/clangd/clangd/issues/310
llvm::Expected<std::vector<FoldingRange>> getFoldingRanges(ParsedAST &AST) {
syntax::Arena A;
syntax::TokenBufferTokenManager TM(AST.getTokens(), AST.getLangOpts(),
AST.getSourceManager());
const auto *SyntaxTree = syntax::buildSyntaxTree(A, TM, AST.getASTContext());
return collectFoldingRanges(SyntaxTree, TM);
}
// FIXME( usaxena95): Collect PP conditional regions, includes and other code
// regions (e.g. public/private/protected sections of classes, control flow
// statement bodies).
// Related issue: https://github.com/clangd/clangd/issues/310
llvm::Expected<std::vector<FoldingRange>>
getFoldingRanges(const std::string &Code, bool LineFoldingOnly) {
auto OrigStream = lex(Code, genericLangOpts());
auto DirectiveStructure = DirectiveTree::parse(OrigStream);
chooseConditionalBranches(DirectiveStructure, OrigStream);
// FIXME: Provide ranges in the disabled-PP regions as well.
auto Preprocessed = DirectiveStructure.stripDirectives(OrigStream);
auto ParseableStream = cook(Preprocessed, genericLangOpts());
pairBrackets(ParseableStream);
std::vector<FoldingRange> Result;
auto AddFoldingRange = [&](Position Start, Position End,
llvm::StringLiteral Kind) {
if (Start.line >= End.line)
return;
FoldingRange FR;
FR.startLine = Start.line;
FR.startCharacter = Start.character;
FR.endLine = End.line;
FR.endCharacter = End.character;
FR.kind = Kind.str();
Result.push_back(FR);
};
auto OriginalToken = [&](const Token &T) {
return OrigStream.tokens()[T.OriginalIndex];
};
auto StartOffset = [&](const Token &T) {
return OriginalToken(T).text().data() - Code.data();
};
auto StartPosition = [&](const Token &T) {
return offsetToPosition(Code, StartOffset(T));
};
auto EndOffset = [&](const Token &T) {
return StartOffset(T) + OriginalToken(T).Length;
};
auto EndPosition = [&](const Token &T) {
return offsetToPosition(Code, EndOffset(T));
};
auto Tokens = ParseableStream.tokens();
// Brackets.
for (const auto &Tok : Tokens) {
if (auto *Paired = Tok.pair()) {
// Process only token at the start of the range. Avoid ranges on a single
// line.
if (Tok.Line < Paired->Line) {
Position Start = offsetToPosition(Code, 1 + StartOffset(Tok));
Position End = StartPosition(*Paired);
if (LineFoldingOnly)
End.line--;
AddFoldingRange(Start, End, FoldingRange::REGION_KIND);
}
}
}
auto IsBlockComment = [&](const Token &T) {
assert(T.Kind == tok::comment);
return OriginalToken(T).Length >= 2 &&
Code.substr(StartOffset(T), 2) == "/*";
};
// Multi-line comments.
for (auto *T = Tokens.begin(); T != Tokens.end();) {
if (T->Kind != tok::comment) {
T++;
continue;
}
Token *FirstComment = T;
// Show starting sentinals (// and /*) of the comment.
Position Start = offsetToPosition(Code, 2 + StartOffset(*FirstComment));
Token *LastComment = T;
Position End = EndPosition(*T);
while (T != Tokens.end() && T->Kind == tok::comment &&
StartPosition(*T).line <= End.line + 1) {
End = EndPosition(*T);
LastComment = T;
T++;
}
if (IsBlockComment(*FirstComment)) {
if (LineFoldingOnly)
// Show last line of a block comment.
End.line--;
if (IsBlockComment(*LastComment))
// Show ending sentinal "*/" of the block comment.
End.character -= 2;
}
AddFoldingRange(Start, End, FoldingRange::COMMENT_KIND);
}
return Result;
}
} // namespace clangd
} // namespace clang

View File

@@ -1,41 +0,0 @@
//===--- SemanticSelection.h -------------------------------------*- C++-*-===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
//
// Features for giving interesting semantic ranges around the cursor.
//
//===----------------------------------------------------------------------===//
#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_SEMANTICSELECTION_H
#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_SEMANTICSELECTION_H
#include "ParsedAST.h"
#include "Protocol.h"
#include "llvm/Support/Error.h"
#include <string>
#include <vector>
namespace clang {
namespace clangd {
/// Returns the list of all interesting ranges around the Position \p Pos.
/// The interesting ranges corresponds to the AST nodes in the SelectionTree
/// containing \p Pos.
/// If pos is not in any interesting range, return [Pos, Pos).
llvm::Expected<SelectionRange> getSemanticRanges(ParsedAST &AST, Position Pos);
/// Returns a list of ranges whose contents might be collapsible in an editor.
/// This should include large scopes, preprocessor blocks etc.
llvm::Expected<std::vector<FoldingRange>> getFoldingRanges(ParsedAST &AST);
/// Returns a list of ranges whose contents might be collapsible in an editor.
/// This version uses the pseudoparser which does not require the AST.
llvm::Expected<std::vector<FoldingRange>>
getFoldingRanges(const std::string &Code, bool LineFoldingOnly);
} // namespace clangd
} // namespace clang
#endif // LLVM_CLANG_TOOLS_EXTRA_CLANGD_SEMANTICSELECTION_H

View File

@@ -1,24 +0,0 @@
# RUN: clangd -lit-test < %s | FileCheck -strict-whitespace %s
void f() {
}
---
{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":123,"rootPath":"clangd","capabilities":{"textDocument": {"foldingRange": {"lineFoldingOnly": true}}},"trace":"off"}}
---
{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"languageId":"cpp","text":"void f() {\n\n}\n","uri":"test:///foo.cpp","version":1}}}
---
{"id":1,"jsonrpc":"2.0","method":"textDocument/foldingRange","params":{"textDocument":{"uri":"test:///foo.cpp"}}}
# CHECK: "id": 1,
# CHECK-NEXT: "jsonrpc": "2.0",
# CHECK-NEXT: "result": [
# CHECK-NEXT: {
# CHECK-NEXT: "endLine": 1,
# CHECK-NEXT: "kind": "region",
# CHECK-NEXT: "startCharacter": 10,
# CHECK-NEXT: "startLine": 0
# CHECK-NEXT: }
# CHECK-NEXT: ]
---
{"jsonrpc":"2.0","id":5,"method":"shutdown"}
---
{"jsonrpc":"2.0","method":"exit"}

View File

@@ -1,459 +0,0 @@
//===-- SemanticSelectionTests.cpp ----------------*- C++ -*--------------===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
#include "Annotations.h"
#include "ClangdServer.h"
#include "Protocol.h"
#include "SemanticSelection.h"
#include "SyncAPI.h"
#include "TestFS.h"
#include "TestTU.h"
#include "llvm/ADT/ArrayRef.h"
#include "llvm/Support/Error.h"
#include "gmock/gmock.h"
#include "gtest/gtest.h"
#include <vector>
namespace clang {
namespace clangd {
namespace {
using ::testing::ElementsAre;
using ::testing::ElementsAreArray;
using ::testing::UnorderedElementsAreArray;
// front() is SR.range, back() is outermost range.
std::vector<Range> gatherRanges(const SelectionRange &SR) {
std::vector<Range> Ranges;
for (const SelectionRange *S = &SR; S; S = S->parent.get())
Ranges.push_back(S->range);
return Ranges;
}
std::vector<Range>
gatherFoldingRanges(llvm::ArrayRef<FoldingRange> FoldingRanges) {
std::vector<Range> Ranges;
Range NextRange;
for (const auto &R : FoldingRanges) {
NextRange.start.line = R.startLine;
NextRange.start.character = R.startCharacter;
NextRange.end.line = R.endLine;
NextRange.end.character = R.endCharacter;
Ranges.push_back(NextRange);
}
return Ranges;
}
TEST(SemanticSelection, All) {
const char *Tests[] = {
R"cpp( // Single statement in a function body.
[[void func() [[{
[[[[int v = [[1^00]]]];]]
}]]]]
)cpp",
R"cpp( // Expression
[[void func() [[{
int a = 1;
// int v = (10 + 2) * (a + a);
[[[[int v = [[[[([[[[10^]] + 2]])]] * (a + a)]]]];]]
}]]]]
)cpp",
R"cpp( // Function call.
int add(int x, int y) { return x + y; }
[[void callee() [[{
// int res = add(11, 22);
[[[[int res = [[add([[1^1]], 22)]]]];]]
}]]]]
)cpp",
R"cpp( // Tricky macros.
#define MUL ) * (
[[void func() [[{
// int var = (4 + 15 MUL 6 + 10);
[[[[int var = [[[[([[4 + [[1^5]]]] MUL]] 6 + 10)]]]];]]
}]]]]
)cpp",
R"cpp( // Cursor inside a macro.
#define HASH(x) ((x) % 10)
[[void func() [[{
[[[[int a = [[HASH([[[[2^3]] + 34]])]]]];]]
}]]]]
)cpp",
R"cpp( // Cursor on a macro.
#define HASH(x) ((x) % 10)
[[void func() [[{
[[[[int a = [[HA^SH(23)]]]];]]
}]]]]
)cpp",
R"cpp( // Multiple declaration.
[[void func() [[{
[[[[int var1, var^2]], var3;]]
}]]]]
)cpp",
R"cpp( // Before comment.
[[void func() [[{
int var1 = 1;
[[[[int var2 = [[[[var1]]^ /*some comment*/ + 41]]]];]]
}]]]]
)cpp",
// Empty file.
"[[^]]",
// FIXME: We should get the whole DeclStmt as a range.
R"cpp( // Single statement in TU.
[[int v = [[1^00]]]];
)cpp",
R"cpp( // Cursor at end of VarDecl.
[[int v = [[100]]^]];
)cpp",
// FIXME: No node found associated to the position.
R"cpp( // Cursor in between spaces.
void func() {
int v = 100 + [[^]] 100;
}
)cpp",
// Structs.
R"cpp(
struct AAA { struct BBB { static int ccc(); };};
[[void func() [[{
// int x = AAA::BBB::ccc();
[[[[int x = [[[[AAA::BBB::c^cc]]()]]]];]]
}]]]]
)cpp",
R"cpp(
struct AAA { struct BBB { static int ccc(); };};
[[void func() [[{
// int x = AAA::BBB::ccc();
[[[[int x = [[[[[[[[[[AA^A]]::]]BBB::]]ccc]]()]]]];]]
}]]]]
)cpp",
R"cpp( // Inside struct.
struct A { static int a(); };
[[struct B {
[[static int b() [[{
[[return [[[[1^1]] + 2]]]];
}]]]]
}]];
)cpp",
// Namespaces.
R"cpp(
[[namespace nsa {
[[namespace nsb {
static int ccc();
[[void func() [[{
// int x = nsa::nsb::ccc();
[[[[int x = [[[[nsa::nsb::cc^c]]()]]]];]]
}]]]]
}]]
}]]
)cpp",
};
for (const char *Test : Tests) {
auto T = Annotations(Test);
auto AST = TestTU::withCode(T.code()).build();
EXPECT_THAT(gatherRanges(llvm::cantFail(getSemanticRanges(AST, T.point()))),
ElementsAreArray(T.ranges()))
<< Test;
}
}
TEST(SemanticSelection, RunViaClangdServer) {
MockFS FS;
MockCompilationDatabase CDB;
ClangdServer Server(CDB, FS, ClangdServer::optsForTest());
auto FooH = testPath("foo.h");
FS.Files[FooH] = R"cpp(
int foo(int x);
#define HASH(x) ((x) % 10)
)cpp";
auto FooCpp = testPath("Foo.cpp");
const char *SourceContents = R"cpp(
#include "foo.h"
[[void bar(int& inp) [[{
// inp = HASH(foo(inp));
[[inp = [[HASH([[foo([[in^p]])]])]]]];
}]]]]
$empty[[^]]
)cpp";
Annotations SourceAnnotations(SourceContents);
FS.Files[FooCpp] = std::string(SourceAnnotations.code());
Server.addDocument(FooCpp, SourceAnnotations.code());
auto Ranges = runSemanticRanges(Server, FooCpp, SourceAnnotations.points());
ASSERT_TRUE(bool(Ranges))
<< "getSemanticRange returned an error: " << Ranges.takeError();
ASSERT_EQ(Ranges->size(), SourceAnnotations.points().size());
EXPECT_THAT(gatherRanges(Ranges->front()),
ElementsAreArray(SourceAnnotations.ranges()));
EXPECT_THAT(gatherRanges(Ranges->back()),
ElementsAre(SourceAnnotations.range("empty")));
}
TEST(FoldingRanges, ASTAll) {
const char *Tests[] = {
R"cpp(
#define FOO int foo() {\
int Variable = 42; \
return 0; \
}
// Do not generate folding range for braces within macro expansion.
FOO
// Do not generate folding range within macro arguments.
#define FUNCTOR(functor) functor
void func() {[[
FUNCTOR([](){});
]]}
// Do not generate folding range with a brace coming from macro.
#define LBRACE {
void bar() LBRACE
int X = 42;
}
)cpp",
R"cpp(
void func() {[[
int Variable = 100;
if (Variable > 5) {[[
Variable += 42;
]]} else if (Variable++)
++Variable;
else {[[
Variable--;
]]}
// Do not generate FoldingRange for empty CompoundStmts.
for (;;) {}
// If there are newlines between {}, we should generate one.
for (;;) {[[
]]}
]]}
)cpp",
R"cpp(
class Foo {
public:
Foo() {[[
int X = 1;
]]}
private:
int getBar() {[[
return 42;
]]}
// Braces are located at the same line: no folding range here.
void getFooBar() { }
};
)cpp",
};
for (const char *Test : Tests) {
auto T = Annotations(Test);
auto AST = TestTU::withCode(T.code()).build();
EXPECT_THAT(gatherFoldingRanges(llvm::cantFail(getFoldingRanges(AST))),
UnorderedElementsAreArray(T.ranges()))
<< Test;
}
}
TEST(FoldingRanges, PseudoParserWithoutLineFoldings) {
const char *Tests[] = {
R"cpp(
#define FOO int foo() {\
int Variable = 42; \
}
// Do not generate folding range for braces within macro expansion.
FOO
// Do not generate folding range within macro arguments.
#define FUNCTOR(functor) functor
void func() {[[
FUNCTOR([](){});
]]}
// Do not generate folding range with a brace coming from macro.
#define LBRACE {
void bar() LBRACE
int X = 42;
}
)cpp",
R"cpp(
void func() {[[
int Variable = 100;
if (Variable > 5) {[[
Variable += 42;
]]} else if (Variable++)
++Variable;
else {[[
Variable--;
]]}
// Do not generate FoldingRange for empty CompoundStmts.
for (;;) {}
// If there are newlines between {}, we should generate one.
for (;;) {[[
]]}
]]}
)cpp",
R"cpp(
class Foo {[[
public:
Foo() {[[
int X = 1;
]]}
private:
int getBar() {[[
return 42;
]]}
// Braces are located at the same line: no folding range here.
void getFooBar() { }
]]};
)cpp",
R"cpp(
// Range boundaries on escaped newlines.
class Foo \
\
{[[ \
public:
Foo() {[[\
int X = 1;
]]} \
]]};
)cpp",
R"cpp(
/*[[ Multi
* line
* comment
]]*/
)cpp",
R"cpp(
//[[ Comment
// 1]]
//[[ Comment
// 2]]
// No folding for single line comment.
/*[[ comment 3
]]*/
/*[[ comment 4
]]*/
/*[[ foo */
/* bar ]]*/
/*[[ foo */
// baz
/* bar ]]*/
/*[[ foo */
/* bar*/
// baz]]
//[[ foo
/* bar */]]
)cpp",
};
for (const char *Test : Tests) {
auto T = Annotations(Test);
EXPECT_THAT(gatherFoldingRanges(llvm::cantFail(getFoldingRanges(
T.code().str(), /*LineFoldingsOnly=*/false))),
UnorderedElementsAreArray(T.ranges()))
<< Test;
}
}
TEST(FoldingRanges, PseudoParserLineFoldingsOnly) {
const char *Tests[] = {
R"cpp(
void func(int a) {[[
a++;]]
}
)cpp",
R"cpp(
// Always exclude last line for brackets.
void func(int a) {[[
if(a == 1) {[[
a++;]]
} else if (a == 2){[[
a--;]]
} else { // No folding for 2 line bracketed ranges.
}]]
}
)cpp",
R"cpp(
/*[[ comment
* comment]]
*/
/* No folding for this comment.
*/
// No folding for this comment.
//[[ 2 single line comment.
// 2 single line comment.]]
//[[ >=2 line comments.
// >=2 line comments.
// >=2 line comments.]]
//[[ foo\
bar\
baz]]
/*[[ foo */
/* bar */]]
/* baz */
/*[[ foo */
/* bar]]
* This does not fold me */
//[[ foo
/* bar */]]
)cpp",
// FIXME: Support folding template arguments.
// R"cpp(
// template <[[typename foo, class bar]]> struct baz {};
// )cpp",
};
auto StripColumns = [](const std::vector<Range> &Ranges) {
std::vector<Range> Res;
for (Range R : Ranges) {
R.start.character = R.end.character = 0;
Res.push_back(R);
}
return Res;
};
for (const char *Test : Tests) {
auto T = Annotations(Test);
EXPECT_THAT(
StripColumns(gatherFoldingRanges(llvm::cantFail(
getFoldingRanges(T.code().str(), /*LineFoldingsOnly=*/true)))),
UnorderedElementsAreArray(StripColumns(T.ranges())))
<< Test;
}
}
} // namespace
} // namespace clangd
} // namespace clang

View File

@@ -1,70 +0,0 @@
## ADDED Requirements
### Requirement: Folding range responses honor client capabilities
The server SHALL render folding ranges according to the client's declared folding capabilities instead of always returning the richest possible payload.
#### Scenario: Line-only folding is respected
- **WHEN** the client declares `textDocument.foldingRange.lineFoldingOnly = true`
- **THEN** the server MUST return folding ranges that remain valid when interpreted as whole-line folds, including adjusting end boundaries for bracketed or comment ranges whose closing delimiter is on the last line
#### Scenario: Client line-only support is propagated through folding options
- **WHEN** the client declares `textDocument.foldingRange.lineFoldingOnly = true`
- **THEN** the server MUST invoke folding rendering with options equivalent to `line_folding_only = true`
- **AND** collectors MUST NOT need to inspect client capability state to produce different raw ranges
#### Scenario: Collapsed text is gated by client support
- **WHEN** the client does not declare support for `textDocument.foldingRange.foldingRange.collapsedText`
- **THEN** the server MUST omit `collapsedText` from the folding range response
#### Scenario: Preferred range limits are applied deterministically
- **WHEN** the client declares `textDocument.foldingRange.rangeLimit = N` and the server can produce more than `N` folding ranges for a document
- **THEN** the server MUST return no more than `N` ranges and MUST choose them using a deterministic ordering rule
#### Scenario: Standard kinds are emitted compatibly
- **WHEN** a folding range represents a comment block, an include/import block, or any other foldable region
- **THEN** the server MUST emit `kind = comment`, `kind = imports`, or `kind = region` respectively, and MUST NOT require clients to understand clice-specific kind strings in order to fold correctly
### Requirement: Structural and comment folding baseline
The server SHALL provide folding ranges for multi-line C/C++ structural regions and multi-line comments in the main file.
#### Scenario: Multi-line comment blocks can be folded
- **WHEN** a document contains a multi-line `/* ... */` comment or a contiguous block of `//` comments spanning more than one line
- **THEN** the server MUST return a folding range for that comment block with `kind = comment`
#### Scenario: Single-line comments are not folded
- **WHEN** a document contains a single-line comment that does not extend across multiple lines and is not part of a larger contiguous comment block
- **THEN** the server MUST NOT return a folding range for that comment
#### Scenario: Existing structural regions remain foldable
- **WHEN** a document contains a multi-line namespace, record, function body, parameter list, lambda body, initializer list, or other supported structural region already collected by clice
- **THEN** the server MUST continue to return a folding range for that region if its boundaries can be mapped back to the main file
### Requirement: Preprocessor regions fold as complete branch blocks
The server SHALL provide complete and nested folding ranges for preprocessor branch structures instead of leaving the final branch in a conditional block unclosed.
#### Scenario: Final conditional branch closes at endif
- **WHEN** a document contains a `#if/#elif/#else/#endif` chain
- **THEN** the server MUST generate a folding range for each multi-line branch body, including the last branch body that ends at `#endif`
#### Scenario: Inactive conditional branches can be folded
- **WHEN** a conditional branch is known to be inactive or skipped in the current preprocessing configuration
- **THEN** the server MUST be able to return a folding range covering that inactive branch region using `kind = region`
#### Scenario: Nested pragma regions are folded
- **WHEN** a document contains nested `#pragma region` / `#pragma endregion` pairs in the main file
- **THEN** the server MUST return properly nested folding ranges for each matched region pair
### Requirement: C/C++ directive groups and multiline macros are foldable
The server SHALL use clice's preprocessor metadata to expose foldable ranges that clangd does not currently provide.
#### Scenario: Multi-line macro definitions can be folded
- **WHEN** a document contains a multi-line macro definition whose body spans more than one physical line
- **THEN** the server MUST return a folding range for that macro definition using `kind = region`
#### Scenario: Consecutive include directives are grouped
- **WHEN** a document contains a contiguous block of `#include` directives with no intervening non-trivia code lines
- **THEN** the server MUST return a folding range covering that include block using `kind = imports`
#### Scenario: Consecutive module imports are grouped
- **WHEN** a document contains a contiguous block of C++ module `import` declarations with no intervening non-trivia code lines
- **THEN** the server MUST return a folding range covering that import block using `kind = imports`

View File

@@ -1,39 +0,0 @@
## 1. Reference Snapshot
- [x] 1.1 Download the clangd folding-range reference files for `llvmorg-21.1.8` into `openspec/changes/explore-improve-folding-range-support/reference/clangd/llvmorg-21.1.8/` using `curl` against GitHub raw URLs.
- [x] 1.2 Include `SemanticSelection.{cpp,h}`, `ClangdServer.{cpp,h}`, `ClangdLSPServer.cpp`, `Protocol.{h,cpp}`, `test/folding-range.test`, and `unittests/SemanticSelectionTests.cpp`.
- [x] 1.3 Record the exact raw GitHub URLs and downloaded file layout in a change-local comparison note.
## 2. Comparison and Pipeline
- [x] 2.1 Record a side-by-side comparison in the change artifacts between clangd's folding path and clice's current path, calling out confirmed parity gaps, clice-only capabilities, and known bugs.
- [ ] 2.2 Keep the existing `RawFoldingRange` model as the settled collection contract while completing normalization and options-driven rendering.
- [ ] 2.3 Replace direct exposure of clice-specific public folding kinds with a stable mapping to standard LSP `comment` / `imports` / `region` kinds.
## 3. Comment and Structural Baseline
- [ ] 3.1 Add a comment collector that folds multi-line block comments and contiguous multi-line `//` comment groups in the main file.
- [ ] 3.2 Preserve existing AST structural folding behavior while routing it through the new normalization/rendering pipeline.
- [ ] 3.3 Add focused unit tests for comment folding, single-line comment exclusion, and structural folding regressions.
## 4. Preprocessor Folding
- [ ] 4.1 Rework conditional-directive collection so each `#if/#elif/#else/#endif` branch body closes correctly, including the final branch ending at `#endif`.
- [ ] 4.2 Add folding support for inactive conditional branches using the existing preprocessor condition metadata.
- [ ] 4.3 Strengthen `#pragma region` handling and convert the current placeholder directive tests into assertion-backed coverage.
## 5. Protocol and Rendering
- [ ] 5.1 Capture client folding capabilities during initialize and translate them into `FoldingRangeOptions`/`Opts` when serving `textDocument/foldingRange`.
- [ ] 5.2 Honor `lineFoldingOnly` through `opts.line_folding_only`, gate `collapsedText`, and apply deterministic `rangeLimit` trimming during folding-range rendering.
- [ ] 5.3 Add integration coverage for line-only rendering, standard kind output, optional collapsed text, and range limiting.
## 6. Clice-Specific Folding Extensions
- [ ] 6.1 Add folding ranges for multi-line macro definitions using `directive.macros` and stable main-file source ranges.
- [ ] 6.2 Add grouping folds for contiguous `#include` blocks and return them as `imports` ranges.
- [ ] 6.3 Add grouping folds for contiguous C++ module `import` declarations and cover mixed include/import layouts with tests.
## 7. Verification
- [ ] 7.1 Run the relevant folding-range unit and integration tests, then fix any ordering, deduplication, or boundary regressions found during verification.

View File

@@ -1,2 +0,0 @@
schema: spec-driven
created: 2026-04-22

View File

@@ -1,192 +0,0 @@
## Context
This change extracts decision `2` from `openspec/changes/explore-improve-folding-range-support/design.md` into a standalone proposal. The current folding implementation in `src/feature/folding_ranges.cpp` already has an internal `RawFoldingRange` handoff, but it still leaves two important concerns too implicit:
- deciding which ranges survive deduplication and validation
- shaping the final LSP response, including client-specific output rules such as `line_folding_only`
Follow-up discussion clarified that the existing `RawFoldingRange` shape is finished for this extracted change. The remaining architectural gap is an explicit options path, passed as `Opts`/`FoldingRangeOptions`, so rendering can be configured without reworking collectors. This proposal therefore keeps scope narrow: it does not add new fold categories or redesign raw ranges, but it creates the normalization and options-driven rendering boundaries that later changes can build on without destabilizing existing structural folding.
The downloaded clangd reference confirms both the value and the limit of the upstream design. clangd has useful, tested folding behavior for brace bodies, comment blocks, contiguous `//` groups, and `lineFoldingOnly`, but its implementation largely emits protocol-shaped `FoldingRange` objects directly from collection code. In `SemanticSelection.cpp`, both the AST path and the pseudo-parser path build `FoldingRange` results directly, and the pseudo-parser applies rendering details such as delimiter trimming and `lineFoldingOnly` adjustments while collecting ranges. That is a good behavior reference, but it is not the architecture this extracted change should copy.
`clice` already has stronger ingredients for a real pipeline:
- the existing `RawFoldingRange` gives collectors a feature-local representation that is not a final LSP response
- `LocalSourceRange` gives us a main-file, half-open offset representation that is independent of LSP position encoding
- directive metadata already captures information clangd does not expose well, including conditional-branch state, pragma regions, includes, imports, and macro references
- the current tests are boundary-oriented, which makes them a good fit for validating raw spans before protocol rendering
The design therefore separates "what fold exists in the source" from "how that fold should be emitted to this client". clangd's tested boundary rules are still relevant, but they should become renderer policy selected by options and normalization rules rather than collector output format.
## Goals / Non-Goals
**Goals:**
- Keep the existing `RawFoldingRange` collection contract stable while completing normalization and rendering boundaries.
- Preserve the existing AST structural folding categories already supported by `clice`.
- Make ordering, deduplication, and boundary validation deterministic and testable.
- Add an explicit folding options object so `line_folding_only` can be configured by callers and consumed only by rendering.
- Give later changes a stable extension point for comments, directives, and client-driven rendering options.
**Non-Goals:**
- Add comment folding in this change.
- Fix preprocessor branch-closing behavior in this change.
- Add new fold categories such as macro definitions or include/import grouping.
- Redesign or replace the existing `RawFoldingRange` model.
- Depend on initialize-time client capability plumbing being implemented first.
## Decisions
### 1. Use clangd as a behavior reference, not an architecture template
This change should borrow clangd's confirmed folding behavior where it is useful, especially around multiline comments, contiguous `//` comment groups, main-file-only filtering, and `lineFoldingOnly` boundary shaping. It should not copy clangd's habit of emitting protocol-shaped `FoldingRange` objects directly from collection logic.
Why:
- clangd's tests are valuable because they pin down tricky folding behavior around comments, macro boundaries, and line-only rendering
- clangd's data flow is intentionally narrow and mixes collection with response shaping
- `clice` already has richer file-local and directive metadata that supports a cleaner internal representation
Alternative considered:
- Treat clangd's direct `FoldingRange` construction as the architecture to reproduce. Rejected because it would preserve the same coupling this extracted change is meant to remove.
### 2. Treat the existing raw internal folding-range model as finished
Collectors should continue to emit the existing internal `RawFoldingRange` structure instead of final LSP protocol objects. The raw model is sufficient for this extracted change and should not be redesigned as part of adding `line_folding_only` support.
The raw model should remain shaped around file-local source structure, not client capability state. In the current implementation it carries:
- a main-file `LocalSourceRange` span using half-open byte offsets
- an optional public folding kind to preserve existing behavior
- an optional collapsed-text hint
```cpp
struct RawFoldingRange {
LocalSourceRange range;
std::optional<protocol::FoldingRangeKind> kind;
std::string collapsed_text;
};
```
The important design choice is that `range` represents the foldable source envelope in the main file while client-specific rendering state stays out of the raw model. For example:
- brace-based structural folds keep their source span and let the renderer decide line-only boundary shaping
- future block comments can keep the full `/* ... */` span and let the renderer decide whether to hide the closing delimiter or final line
- future contiguous `//` groups can keep the grouped span and let the renderer decide line-only output
Why:
- collectors should describe what was found, not how it will be serialized
- `LocalSourceRange` is already the natural coordinate system for `clice`
- future comment and directive collectors can share the same pipeline contract
- tests can validate collection independently from rendering
- the missing `line_folding_only` behavior belongs in options and rendering, not in the raw range shape
Alternatives considered:
- Continue emitting LSP ranges directly from collectors. Rejected because it keeps protocol concerns entangled with source discovery.
- Expand `RawFoldingRange` now with render-hint fields for line-only behavior. Rejected because follow-up discussion established the raw model as finished for this slice, and line-only support can be configured through rendering options instead.
### 3. Normalize ranges before rendering
All collected ranges should pass through a normalization step before any response is emitted. Normalization is responsible for deterministic ordering, duplicate removal, and rejection of degenerate or unmappable ranges.
Normalization should operate on raw spans and raw metadata, not on already-rendered LSP line/character fields. Its responsibilities include:
- deterministic ordering independent of collector traversal order
- duplicate collapse for collectors that discover the same fold
- invalid-range filtering after raw spans are mapped and validated
- stable tie-breaking for overlapping ranges from different origins
Collectors may still reject obviously invalid inputs, such as non-main-file locations that cannot be mapped to `LocalSourceRange`, but normalization remains the phase that decides which collected folds survive to rendering.
Why:
- duplicate or invalid ranges are easier to reason about in one place than across many collectors
- stable ordering reduces regression noise and makes range limiting predictable later
- metadata-aware normalization preserves fold meaning until the renderer maps it to public output
- normalization lets new collectors plug in without each collector re-implementing cleanup logic
Alternative considered:
- Let each collector manage its own sorting and duplicate suppression. Rejected because cross-collector interactions would still remain undefined.
### 4. Keep the current AST visitor as the first collector boundary
The initial extraction should preserve the current AST visitor as one collector feeding the raw model. This reduces refactor risk while still creating the new phase boundaries.
Why:
- the existing structural fold coverage is valuable and should not be rewritten unnecessarily
- an adapter-style refactor is easier to verify against current tests than a full collector redesign
Alternative considered:
- Rewrite collection around a brand-new multi-source manager immediately. Rejected because it adds scope before the phase split is proven.
### 5. Move output shaping into an options-driven renderer
The renderer should translate normalized ranges into LSP folding ranges. Boundary shaping, output kinds, and optional metadata emission should live there, even if some options still use default values until later protocol plumbing exists.
Renderer input should be the normalized raw model plus a separate `FoldingRangeOptions` structure. The public feature API should follow the existing feature-options style, for example:
```cpp
struct FoldingRangeOptions {
bool line_folding_only = false;
};
auto folding_ranges(CompilationUnitRef unit,
const FoldingRangeOptions& opts = {},
PositionEncoding encoding = PositionEncoding::UTF16)
-> std::vector<protocol::FoldingRange>;
```
`line_folding_only` defaults to `false`, preserving the current behavior for existing callers. When server capability plumbing is added later, the server should translate `textDocument.foldingRange.lineFoldingOnly` into this option instead of exposing session state to collectors.
The renderer then becomes responsible for:
- converting `LocalSourceRange` into protocol positions for the requested encoding
- applying line-only adjustments when `opts.line_folding_only = true`
- mapping raw kind metadata to emitted LSP kinds
- deciding whether collapsed text is emitted or suppressed
- later applying deterministic `rangeLimit` trimming without changing collectors
This is the key point where `clice` should intentionally diverge from clangd. clangd threads `lineFoldingOnly` into collection and directly produces protocol objects. `clice` should keep those capability and transport decisions isolated in rendering so collectors remain stable as client support evolves.
Why:
- rendering rules are a separate concern from source discovery
- later work on line-only output, metadata gating, or public kind mapping should not force collector rewrites
- clangd-style line-only shaping is still supported, but as renderer policy rather than collector output
- isolating rendering makes behavioral diffs easier to review
- a small options object makes the missing `line_folding_only` support explicit without expanding `RawFoldingRange`
Alternative considered:
- Keep final boundary shaping next to the AST collector and only add a small helper for sorting. Rejected because it only moves a symptom, not the architectural problem.
## Risks / Trade-offs
- [Refactoring the current path can accidentally change fold ordering] -> Mitigation: add deterministic-order assertions and compare outputs for existing structural fixtures.
- [The raw model could become too abstract too early] -> Mitigation: do not redesign `RawFoldingRange` in this change; keep the existing fields unless implementation proves a concrete need.
- [Line-only behavior can be accidentally encoded in collectors] -> Mitigation: expose `line_folding_only` only through `FoldingRangeOptions` and assert renderer-level behavior in tests.
- [A renderer abstraction may appear premature before full capability plumbing exists] -> Mitigation: keep default render options aligned with current behavior and treat future options as extension points, not immediate scope.
## Migration Plan
1. Keep the existing `RawFoldingRange` collection path stable behind the current entrypoint.
2. Introduce `FoldingRangeOptions` with `line_folding_only = false` by default.
3. Insert normalization between collection and response emission.
4. Move LSP object construction into a dedicated renderer that consumes normalized ranges plus options.
5. Add line-only renderer tests and verify that existing structural folding fixtures still produce the expected default ranges.
Rollback strategy:
- If the refactor destabilizes output, keep the new helper types but temporarily route the old direct-emission path until normalization and rendering regressions are resolved.
## Open Questions
- Whether public kind remapping should land in this extracted change or remain a follow-up proposal once the renderer boundary exists.
- Whether `FoldingRangeOptions` should initially contain only `line_folding_only`, or also reserve fields for later collapsed-text and `rangeLimit` behavior.

View File

@@ -1,29 +0,0 @@
## Why
`explore-improve-folding-range-support` combines several different concerns: upstream comparison work, baseline folding fixes, preprocessor extensions, and folding renderer behavior. The second design point in that change, splitting the folding-range pipeline into collection, normalization, and rendering, is the architectural slice that other work depends on and should be referenceable as its own proposal.
Follow-up discussion clarified that the existing internal `RawFoldingRange` shape is finished for this slice. The missing architectural part is not another raw-range redesign; it is an explicit folding options path so callers can request client-specific rendering behavior, starting with `line_folding_only`.
## What Changes
- Extract the pipeline-splitting work from `explore-improve-folding-range-support` into a standalone change focused on folding-range architecture.
- Treat the existing `RawFoldingRange` model as the settled internal collection contract for this change.
- Define a normalization phase that performs deterministic sorting, duplicate removal, and boundary validation before response generation.
- Define a folding options object, passed as `Opts`/`FoldingRangeOptions`, that configures rendering without changing collectors.
- Define a rendering phase that owns line/column shaping, including `line_folding_only`, and optional metadata emission instead of mixing those concerns into collectors.
- Preserve the current AST structural folding coverage while establishing extension points for future comment, directive, and capability-aware rendering work.
## Capabilities
### New Capabilities
- `folding-range-pipeline`: Provide a deterministic folding-range pipeline that separates collection, normalization, and rendering while preserving existing structural folds.
### Modified Capabilities
- None.
## Impact
- `src/feature/folding_ranges.cpp` will keep raw-range collection but gain explicit normalization/rendering boundaries and options-driven rendering.
- `src/feature/feature.h` will need a folding options type or equivalent public API extension so `line_folding_only` can be configured without changing collection.
- `tests/unit/feature/folding_range_tests.cpp` will need regression coverage for structural folds, deterministic ordering, and `line_folding_only` boundary shaping.
- `openspec/changes/explore-improve-folding-range-support/design.md` remains the source change from which this standalone proposal was extracted.

View File

@@ -1,50 +0,0 @@
## ADDED Requirements
### Requirement: Folding ranges are normalized before response emission
The server SHALL convert collected folding candidates into a deterministic normalized set before emitting the folding range response.
#### Scenario: Duplicate candidates collapse to one emitted fold
- **WHEN** multiple collectors produce the same folding candidate for the same source span and raw metadata
- **THEN** the server MUST emit at most one folding range for that candidate
#### Scenario: Invalid candidates are dropped during normalization
- **WHEN** a collected folding candidate does not span multiple lines or cannot be mapped back to the main file
- **THEN** the server MUST omit that candidate from the emitted folding ranges
#### Scenario: Output ordering is deterministic
- **WHEN** the same document is analyzed repeatedly without source changes
- **THEN** the server MUST emit folding ranges in a deterministic order that does not depend on collector traversal order
### Requirement: Existing structural folding survives the pipeline split
The server SHALL preserve the currently supported AST structural folding categories after collection, normalization, and rendering are separated.
#### Scenario: Supported structural regions remain foldable
- **WHEN** a document contains a supported multi-line namespace, record, function body, parameter list, lambda body, initializer list, call argument list, or compound statement
- **THEN** the server MUST still return a folding range for that region when its boundaries can be mapped to the main file
#### Scenario: Structural coverage is preserved through normalization
- **WHEN** the document contains only currently supported AST-driven folding categories
- **THEN** normalization and rendering MUST NOT remove a valid structural fold except when it is an exact duplicate or an invalid range
### Requirement: Rendering decisions are applied after normalization
The server SHALL derive final LSP folding-range output from normalized internal ranges instead of requiring collectors to emit protocol-shaped results directly.
#### Scenario: Rendering options do not require collector changes
- **WHEN** rendering rules change how line or metadata output is shaped for a normalized fold
- **THEN** the server MUST apply that change in the rendering phase without requiring collector-specific logic changes
#### Scenario: Metadata hints remain optional until rendering
- **WHEN** a collected or normalized fold carries optional kind or collapsed-text hints
- **THEN** the renderer MUST decide whether to surface, transform, or suppress that metadata in the emitted LSP range
### Requirement: Folding rendering is configured through explicit options
The server SHALL expose folding-specific rendering options so client capability behavior can be selected without changing collectors or raw ranges.
#### Scenario: Default options preserve existing output
- **WHEN** folding ranges are requested without explicit folding options
- **THEN** rendering MUST behave as if `line_folding_only = false`
#### Scenario: Line-only rendering is selected by options
- **WHEN** folding ranges are rendered with `line_folding_only = true`
- **THEN** the renderer MUST emit ranges that remain valid when interpreted as whole-line folds
- **AND** collectors MUST NOT need to inspect client capability state or emit different raw ranges for line-only clients

View File

@@ -1,19 +0,0 @@
## 1. Existing Raw Model and Collector Boundary
- [x] 1.1 Treat the existing `RawFoldingRange` as the finished internal collection model for this change.
- [ ] 1.2 Keep the existing AST structural folding path routed through raw ranges instead of reintroducing direct collector-to-LSP emission.
- [ ] 1.3 Add regression fixtures or assertions that cover the currently supported structural fold categories before further rendering changes.
## 2. Normalization, Opts, and Rendering
- [ ] 2.1 Implement normalization for deterministic sorting, duplicate removal, and invalid-range filtering.
- [ ] 2.2 Introduce `FoldingRangeOptions`/`Opts` with `line_folding_only = false` as the default.
- [ ] 2.3 Introduce a dedicated renderer that converts normalized ranges plus `Opts` into final LSP folding-range objects.
- [ ] 2.4 Honor `line_folding_only` in rendering by shaping emitted boundaries for clients that only support whole-line folds.
- [ ] 2.5 Keep default rendered output compatible with current structural behavior while exposing extension points for future collectors and render rules.
## 3. Verification
- [ ] 3.1 Compare pre-refactor and post-refactor outputs for the existing structural folding test cases.
- [ ] 3.2 Add focused tests for `line_folding_only` output using the new folding options path.
- [ ] 3.3 Run relevant folding-range unit tests and fix any ordering, deduplication, or boundary regressions introduced by the new pipeline.

5702
pixi.lock generated

File diff suppressed because it is too large Load Diff

135
pixi.toml
View File

@@ -14,24 +14,17 @@ readme = "README.md"
documentation = "https://docs.clice.io/clice/" documentation = "https://docs.clice.io/clice/"
repository = "https://github.com/clice-io/clice" repository = "https://github.com/clice-io/clice"
channels = ["conda-forge"] channels = ["conda-forge"]
platforms = ["win-64", "linux-64", "osx-arm64", "osx-64", "linux-aarch64", "win-arm64"] platforms = ["win-64", "linux-64", "osx-arm64"]
[environments] [environments]
default = ["build", "test"] default = ["build", "test"]
package = ["build", "test", "package"] package = ["build", "test", "package"]
cross-macos-x64 = ["build", "package", "cross-macos-x64"]
cross-linux-aarch64 = ["build", "package", "cross-linux-aarch64"]
cross-windows-arm64 = ["build", "package", "cross-windows-arm64"]
node = ["node"] node = ["node"]
format = ["format"] format = ["format"]
test-run = ["test"]
# ============================================================================== # # ============================================================================== #
# DEPENDENCIES # # DEPENDENCIES #
# ============================================================================== # # ============================================================================== #
[feature.build]
platforms = ["win-64", "linux-64", "osx-arm64", "osx-64", "linux-aarch64"]
[feature.build.dependencies] [feature.build.dependencies]
python = ">=3.13" python = ">=3.13"
cmake = ">=3.30" cmake = ">=3.30"
@@ -40,9 +33,7 @@ clang = "==20.1.8"
clangxx = "==20.1.8" clangxx = "==20.1.8"
lld = "==20.1.8" lld = "==20.1.8"
llvm-tools = "==20.1.8" llvm-tools = "==20.1.8"
clang-tools = "==20.1.8"
compiler-rt = "==20.1.8" compiler-rt = "==20.1.8"
flatbuffers = "==25.9.23"
[feature.build.target.win-64.dependencies] [feature.build.target.win-64.dependencies]
sccache = "*" sccache = "*"
@@ -62,43 +53,6 @@ scripts = ["scripts/activate_linux.sh"]
[feature.build.target.win-64.activation] [feature.build.target.win-64.activation]
scripts = ["scripts/activate_asan.bat"] scripts = ["scripts/activate_asan.bat"]
# macOS x64 (from arm64): clang natively supports cross-arch, no extra deps.
[feature.cross-macos-x64.target.osx-arm64.dependencies]
[feature.cross-macos-x64.target.osx-arm64.activation]
scripts = ["scripts/activate_cross_macos.sh"]
# Linux aarch64 (from x64): needs aarch64 sysroot and cross gcc for libstdc++.
[feature.cross-linux-aarch64.target.linux-64.dependencies]
sysroot_linux-aarch64 = "==2.17"
gcc_linux-aarch64 = "==14.2.0"
gxx_linux-aarch64 = "==14.2.0"
[feature.cross-linux-aarch64.target.linux-64.activation]
scripts = ["scripts/activate_cross_linux.sh"]
# Windows arm64 (from x64): Windows SDK on CI already includes ARM64 libs.
[feature.cross-windows-arm64.target.win-64.dependencies]
[feature.cross-windows-arm64.target.win-64.activation]
scripts = ["scripts/activate_cross_windows.bat"]
[feature.test.dependencies]
python = ">=3.13"
# On macOS, the system Apple clang emits vendor-specific flags that upstream
# LLVM cannot parse. Providing upstream clang + lld in PATH prevents
# fallback to /usr/bin/clang++ and satisfies toolchain.cmake's -fuse-ld=lld.
[feature.test.target.osx-64.dependencies]
clang = "==20.1.8"
clangxx = "==20.1.8"
lld = "==20.1.8"
[feature.test.target.osx-arm64.dependencies]
clang = "==20.1.8"
clangxx = "==20.1.8"
lld = "==20.1.8"
[feature.test.pypi-dependencies] [feature.test.pypi-dependencies]
pytest = "*" pytest = "*"
pytest-asyncio = ">=1.1.0" pytest-asyncio = ">=1.1.0"
@@ -108,22 +62,6 @@ lsprotocol = ">=2024.0.0"
[feature.package.dependencies] [feature.package.dependencies]
xz = ">=5.8.1,<6" xz = ">=5.8.1,<6"
[feature.package.tasks.package-config]
args = [{ arg = "type", default = "RelWithDebInfo" }]
cmd = """
cmake -B build/{{ type }} -G Ninja \
-DCMAKE_BUILD_TYPE={{ type }} \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchain.cmake \
-DCLICE_RELEASE=ON
"""
[feature.package.tasks.package]
args = [{ arg = "type", default = "RelWithDebInfo" }]
depends-on = [
{ task = "package-config", args = ["{{ type }}"] },
{ task = "cmake-build", args = ["{{ type }}"] },
]
# ============================================================================== # # ============================================================================== #
# CMAKE # # CMAKE #
# ============================================================================== # # ============================================================================== #
@@ -131,13 +69,14 @@ depends-on = [
args = [ args = [
{ arg = "type", default = "RelWithDebInfo" }, { arg = "type", default = "RelWithDebInfo" },
{ arg = "ci", default = "OFF" }, { arg = "ci", default = "OFF" },
{ arg = "extra", default = "" },
] ]
cmd = """ cmd = """
cmake -B build/{{ type }} -G Ninja \ cmake -B build/{{ type }} -G Ninja \
-DCMAKE_BUILD_TYPE={{ type }} \ -DCMAKE_BUILD_TYPE={{ type }} \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchain.cmake \ -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain.cmake \
-DCLICE_ENABLE_TEST=ON \ -DCLICE_ENABLE_TEST=ON \
-DCLICE_CI_ENVIRONMENT={{ ci }} -DCLICE_CI_ENVIRONMENT={{ ci }} {{extra}} \
""" """
[feature.build.tasks.cmake-build] [feature.build.tasks.cmake-build]
@@ -148,17 +87,14 @@ cmd = "cmake --build build/{{ type }}"
args = [ args = [
{ arg = "type", default = "RelWithDebInfo" }, { arg = "type", default = "RelWithDebInfo" },
{ arg = "ci", default = "OFF" }, { arg = "ci", default = "OFF" },
{ arg = "extra", default = "" },
] ]
depends-on = [ depends-on = [
{ task = "cmake-config", args = ["{{ type }}", "{{ ci }}"] }, { task = "cmake-config", args = ["{{ type }}", "{{ ci }}", "{{extra}}"] },
{ task = "cmake-build", args = ["{{ type }}"] }, { task = "cmake-build", args = ["{{ type }}"] },
] ]
[feature.build.tasks.clang-tidy] [feature.build.tasks.unit-test]
args = [{ arg = "type", default = "RelWithDebInfo" }]
depends-on = [{ task = "lint-cpp", args = ["{{ type }}"] }]
[feature.test.tasks.unit-test]
args = [{ arg = "type", default = "RelWithDebInfo" }] args = [{ arg = "type", default = "RelWithDebInfo" }]
cmd = './build/{{ type }}/bin/unit_tests --test-dir="./tests/data"' cmd = './build/{{ type }}/bin/unit_tests --test-dir="./tests/data"'
@@ -181,9 +117,41 @@ args = [{ arg = "type", default = "RelWithDebInfo" }]
depends-on = [ depends-on = [
{ task = "unit-test", args = ["{{ type }}"] }, { task = "unit-test", args = ["{{ type }}"] },
{ task = "integration-test", args = ["{{ type }}"] }, { task = "integration-test", args = ["{{ type }}"] },
{ task = "smoke-test", args = ["{{ type }}"] },
] ]
# ============================================================================== #
# XMAKE #
# ============================================================================== #
[feature.build.tasks.xmake-config]
args = [
{ arg = "type", default = "releasedbg" },
{ arg = "ci", default = "n" },
]
cmd = "xmake config --yes --mode={{ type }} --toolchain=clang --ci={{ ci }}"
[feature.build.tasks.xmake-build]
cmd = "xmake build --verbose --diagnosis --all"
[feature.build.tasks.xmake]
args = [
{ arg = "type", default = "releasedbg" },
{ arg = "ci", default = "n" },
]
depends-on = [
{ task = "xmake-config", args = ["{{ type }}", "{{ ci }}"] },
{ task = "xmake-build" },
]
[feature.test.tasks.xmake-test]
cmd = "xmake test --verbose"
[feature.package.tasks.package]
cmd = """
xmake config --yes --toolchain=clang --mode=releasedbg \
--enable_test=n --dev=n --release=y && \
xmake pack --verbose
"""
# ============================================================================== # # ============================================================================== #
# HELPER TASKS # # HELPER TASKS #
# ============================================================================== # # ============================================================================== #
@@ -203,14 +171,9 @@ gh workflow run upload-llvm.yml \
args = ["file_name"] args = ["file_name"]
cmd = ["scripts/delete-artifacts.bash", "{{ file_name }}"] cmd = ["scripts/delete-artifacts.bash", "{{ file_name }}"]
[dependencies]
# ============================================================================== # # ============================================================================== #
# DOCS & VSCODE EXTENSION # # DOCS & VSCODE EXTENSION #
# ============================================================================== # # ============================================================================== #
[feature.node]
platforms = ["win-64", "linux-64", "osx-arm64", "osx-64", "linux-aarch64"]
[feature.node.dependencies] [feature.node.dependencies]
nodejs = ">=20" nodejs = ">=20"
pnpm = "*" pnpm = "*"
@@ -236,9 +199,6 @@ outputs = ["editors/vscode/node_modules/.modules.yaml"]
# ============================================================================== # # ============================================================================== #
# FORMAT # # FORMAT #
# ============================================================================== # # ============================================================================== #
[feature.format]
platforms = ["win-64", "linux-64", "osx-arm64", "osx-64", "linux-aarch64"]
[feature.format.dependencies] [feature.format.dependencies]
ruff = "*" ruff = "*"
tombi = "*" tombi = "*"
@@ -269,20 +229,3 @@ format = { depends-on = [
"format-toml", "format-toml",
"format-yaml" "format-yaml"
] } ] }
# ============================================================================== #
# LINT #
# ============================================================================== #
[feature.format.tasks.lint-python]
cmd = "ruff check ."
[feature.build.tasks.lint-cpp]
args = [{ arg = "type", default = "RelWithDebInfo" }]
cmd = "python scripts/run_clang_tidy.py build/{{ type }}"
[feature.build.tasks.lint]
args = [{ arg = "type", default = "RelWithDebInfo" }]
depends-on = [
"lint-python",
{ task = "lint-cpp", args = ["{{ type }}"] },
]

View File

@@ -1,12 +0,0 @@
#!/bin/sh
# Clear conda cross-gcc flags so host x86_64 paths don't leak into the
# aarch64 build. conda's gcc_linux-aarch64 activation sets
# CFLAGS/CXXFLAGS/CPPFLAGS/LDFLAGS with -isystem/-L pointing at $CONDA_PREFIX
# (x86_64 host paths). LIBRARY_PATH from ld_impl_linux-64 likewise points at
# host libs. Empty-string export reliably overrides conda-installed values
# regardless of whether pixi sources or calls this script.
export CFLAGS=
export CXXFLAGS=
export CPPFLAGS=
export LDFLAGS=
export LIBRARY_PATH=

View File

@@ -1,8 +0,0 @@
#!/bin/sh
# Clear conda host flags so arm64 host paths don't leak into the x86_64-macos
# cross build. See scripts/activate_cross_linux.sh for rationale.
export CFLAGS=
export CXXFLAGS=
export CPPFLAGS=
export LDFLAGS=
export LIBRARY_PATH=

View File

@@ -1,8 +0,0 @@
@echo off
REM Clear conda host flags so host x64 paths don't leak into the aarch64-windows
REM cross build. See scripts/activate_cross_linux.sh for rationale.
set "CFLAGS="
set "CXXFLAGS="
set "CPPFLAGS="
set "LDFLAGS="
set "LIBRARY_PATH="

View File

@@ -0,0 +1,489 @@
#!/usr/bin/env python3
"""Use Codex to analyze clangd semantic-highlighting logic from a GitHub blob URL.
Example:
python3 scripts/analyzers/extract_semantic_highlighting_codex.py \
https://github.com/llvm/llvm-project/blob/d8ba56ce3f98871ae4e5782c4af2df4c98bebde7/clang-tools-extra/clangd/SemanticHighlighting.cpp \
--output semantic-highlighting.md
"""
from __future__ import annotations
import argparse
import json
import os
import shutil
import subprocess
import sys
import tempfile
from dataclasses import dataclass
from pathlib import Path
from typing import Literal
from urllib.error import HTTPError, URLError
from urllib.parse import urlparse
from urllib.request import ProxyHandler, Request, build_opener, getproxies
PROMPT = """You are analyzing one C++ source file from clangd.
Task:
- Extract the cases in which semantic highlighting is applied.
- Produce an exhaustive segmentation that covers every source line exactly once.
Kinds:
- nop: this line does not materially participate in deciding or applying a highlight.
- condition: this line or contiguous range establishes a boolean/branching condition that gates a later highlight resolution.
- resolution: this line or contiguous range selects or applies a concrete semantic-highlighting outcome, such as a HighlightingKind, modifier, token emission, or an equivalent concrete result.
Output rules:
- Return JSON only.
- Segments must be in ascending order and non-overlapping.
- Every line from 1 through the last line must be covered exactly once.
- Use the smallest practical contiguous ranges.
- It is fine to compress long nop runs into ranges; the caller may expand them later.
- For nop segments, use an empty summary string.
- For condition segments, write one short sentence in plain English.
- For resolution segments, write one short sentence describing the concrete outcome.
- For resolution segments, populate depends_on with the exact condition line ranges that directly gate this outcome when present.
- Use the provided line numbers exactly; never invent lines.
- Do not use any external tools or local files; analyze only the numbered source text provided in the prompt.
"""
SCHEMA = {
"type": "object",
"properties": {
"segments": {
"type": "array",
"items": {
"type": "object",
"properties": {
"start_line": {"type": "integer", "minimum": 1},
"end_line": {"type": "integer", "minimum": 1},
"kind": {
"type": "string",
"enum": ["nop", "condition", "resolution"],
},
"summary": {"type": "string"},
"depends_on": {
"type": "array",
"items": {
"type": "object",
"properties": {
"start_line": {"type": "integer", "minimum": 1},
"end_line": {"type": "integer", "minimum": 1},
},
"required": ["start_line", "end_line"],
"additionalProperties": False,
},
},
},
"required": [
"start_line",
"end_line",
"kind",
"summary",
"depends_on",
],
"additionalProperties": False,
},
}
},
"required": ["segments"],
"additionalProperties": False,
}
SegmentKind = Literal["nop", "condition", "resolution"]
PROXY_ENV_KEYS = ("http_proxy", "https_proxy", "all_proxy", "no_proxy")
@dataclass(frozen=True)
class GitHubBlobRef:
owner: str
repo: str
rev: str
path: str
blob_url: str
raw_url: str
@property
def title(self) -> str:
return Path(self.path).stem
@dataclass(frozen=True)
class LineRange:
start_line: int
end_line: int
@dataclass(frozen=True)
class Segment:
start_line: int
end_line: int
kind: SegmentKind
summary: str
depends_on: tuple[LineRange, ...]
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Ask Codex to analyze a GitHub-hosted semantic-highlighting file.",
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
)
parser.add_argument(
"url",
help="GitHub blob URL pinned to a specific revision.",
)
parser.add_argument(
"--model",
default=None,
help="Codex model to use via `codex exec`.",
)
parser.add_argument(
"--reasoning-effort",
default=None,
choices=["low", "medium", "high", "xhigh"],
help="Reasoning effort override passed to Codex CLI.",
)
parser.add_argument(
"--timeout",
type=int,
default=30,
help="HTTP timeout in seconds for fetching the source file.",
)
parser.add_argument(
"--codex-bin",
default="codex",
help="Codex CLI executable to invoke.",
)
parser.add_argument(
"--output",
type=Path,
help="Write the markdown output to this path instead of stdout.",
)
return parser.parse_args()
def parse_github_blob_url(url: str) -> GitHubBlobRef:
parsed = urlparse(url)
if parsed.scheme not in {"http", "https"} or parsed.netloc != "github.com":
raise ValueError("expected a GitHub https://github.com/.../blob/... URL")
parts = [part for part in parsed.path.split("/") if part]
if len(parts) < 5 or parts[2] != "blob":
raise ValueError("expected a GitHub blob URL with /owner/repo/blob/rev/path")
owner, repo = parts[0], parts[1]
rev = parts[3]
path = "/".join(parts[4:])
if not path:
raise ValueError("missing file path in GitHub blob URL")
raw_url = f"https://raw.githubusercontent.com/{owner}/{repo}/{rev}/{path}"
normalized_blob_url = f"https://github.com/{owner}/{repo}/blob/{rev}/{path}"
return GitHubBlobRef(
owner=owner,
repo=repo,
rev=rev,
path=path,
blob_url=normalized_blob_url,
raw_url=raw_url,
)
def fetch_text(url: str, timeout: int) -> str:
request = Request(url, headers={"User-Agent": "semantic-highlighting-codex/1.0"})
opener = build_opener(ProxyHandler(getproxies()))
try:
with opener.open(request, timeout=timeout) as response:
return response.read().decode("utf-8").replace("\r\n", "\n")
except HTTPError as exc:
raise RuntimeError(f"failed to fetch {url}: HTTP {exc.code}") from exc
except URLError as exc:
raise RuntimeError(f"failed to fetch {url}: {exc.reason}") from exc
def number_source(text: str) -> tuple[str, int]:
lines = text.splitlines()
width = len(str(max(len(lines), 1)))
numbered = "\n".join(
f"{idx:>{width}} | {line}" for idx, line in enumerate(lines, 1)
)
return numbered, len(lines)
def build_codex_prompt(
blob: GitHubBlobRef,
source_text: str,
line_count: int,
) -> str:
numbered_source, _ = number_source(source_text)
return (
f"{PROMPT}\n\n"
f"GitHub blob URL: {blob.blob_url}\n"
f"LLVM revision: {blob.rev}\n"
f"File path: {blob.path}\n"
f"File title: {blob.title}\n"
f"Total lines: {line_count}\n\n"
"Analyze only the following numbered source file:\n"
f"{numbered_source}\n"
)
def analyze_with_codex_cli(
codex_bin: str,
blob: GitHubBlobRef,
source_text: str,
line_count: int,
model: str | None,
reasoning_effort: str | None,
) -> list[Segment]:
if shutil.which(codex_bin) is None:
raise RuntimeError(f"Codex CLI executable `{codex_bin}` was not found in PATH.")
prompt = build_codex_prompt(
blob=blob, source_text=source_text, line_count=line_count
)
child_env = build_codex_env()
with tempfile.TemporaryDirectory(prefix="semantic-highlighting-codex-") as temp_dir:
temp_path = Path(temp_dir)
schema_path = temp_path / "schema.json"
output_path = temp_path / "last-message.json"
schema_path.write_text(json.dumps(SCHEMA, indent=2), encoding="utf-8")
command = [
codex_bin,
"exec",
"--skip-git-repo-check",
"--ephemeral",
"--color",
"never",
"--sandbox",
"read-only",
*([] if model is None else ["--model", model]),
*(
[]
if reasoning_effort is None
else ["--config", f"model_reasoning_effort={json.dumps(reasoning_effort)}"]
),
"--output-schema",
str(schema_path),
"--output-last-message",
str(output_path),
"-",
]
try:
completed = subprocess.run(
command,
input=prompt,
text=True,
capture_output=True,
check=True,
env=child_env,
)
except FileNotFoundError as exc:
raise RuntimeError(
f"failed to execute `{codex_bin}`: command not found"
) from exc
except subprocess.CalledProcessError as exc:
detail = exc.stderr.strip() or exc.stdout.strip() or str(exc)
raise RuntimeError(f"Codex CLI failed: {detail}") from exc
if not output_path.exists():
detail = completed.stderr.strip() or completed.stdout.strip()
raise RuntimeError(
"Codex CLI did not produce an output message."
+ (f" Details: {detail}" if detail else "")
)
output_text = output_path.read_text(encoding="utf-8").strip()
if not output_text:
raise RuntimeError("Codex CLI returned an empty final message.")
try:
payload = json.loads(output_text)
except json.JSONDecodeError as exc:
raise RuntimeError(f"Codex returned invalid JSON:\n{output_text}") from exc
return normalize_segments(payload.get("segments", []), line_count)
def build_codex_env() -> dict[str, str]:
env = os.environ.copy()
for key in PROXY_ENV_KEYS:
value = env.get(key)
upper_key = key.upper()
if value and upper_key not in env:
env[upper_key] = value
return env
def normalize_segments(raw_segments: list[dict], line_count: int) -> list[Segment]:
normalized: list[Segment] = []
for item in raw_segments:
kind = item["kind"]
if kind not in {"nop", "condition", "resolution"}:
raise ValueError(f"unknown segment kind: {kind}")
start_line = int(item["start_line"])
end_line = int(item["end_line"])
if start_line < 1 or end_line < start_line or end_line > line_count:
raise ValueError(
f"invalid segment range {start_line}-{end_line} for file with {line_count} lines"
)
depends_on = tuple(
LineRange(start_line=int(dep["start_line"]), end_line=int(dep["end_line"]))
for dep in item.get("depends_on", [])
)
normalized.append(
Segment(
start_line=start_line,
end_line=end_line,
kind=kind,
summary=item.get("summary", "").strip(),
depends_on=depends_on,
)
)
normalized.sort(key=lambda segment: (segment.start_line, segment.end_line))
stitched: list[Segment] = []
next_line = 1
for segment in normalized:
if segment.start_line < next_line:
raise ValueError(
f"overlapping segments around line {segment.start_line}: model output is invalid"
)
while next_line < segment.start_line:
stitched.append(
Segment(
start_line=next_line,
end_line=next_line,
kind="nop",
summary="",
depends_on=(),
)
)
next_line += 1
stitched.extend(expand_nop_segment(segment))
next_line = segment.end_line + 1
while next_line <= line_count:
stitched.append(
Segment(
start_line=next_line,
end_line=next_line,
kind="nop",
summary="",
depends_on=(),
)
)
next_line += 1
return stitched
def expand_nop_segment(segment: Segment) -> list[Segment]:
if segment.kind != "nop" or segment.start_line == segment.end_line:
return [segment]
return [
Segment(
start_line=line_no,
end_line=line_no,
kind="nop",
summary="",
depends_on=(),
)
for line_no in range(segment.start_line, segment.end_line + 1)
]
def blob_anchor_url(blob: GitHubBlobRef, line_range: LineRange) -> str:
if line_range.start_line == line_range.end_line:
return f"{blob.blob_url}#L{line_range.start_line}"
return f"{blob.blob_url}#L{line_range.start_line}-L{line_range.end_line}"
def format_line_range(start_line: int, end_line: int) -> str:
if start_line == end_line:
return f"Line {start_line}"
return f"Line {start_line}-{end_line}"
def format_loc_reference(blob: GitHubBlobRef, line_range: LineRange) -> str:
label = (
f"LoC {line_range.start_line}"
if line_range.start_line == line_range.end_line
else f"LoC {line_range.start_line}-{line_range.end_line}"
)
return f"[{label}]({blob_anchor_url(blob, line_range)})"
def render_segment_summary(blob: GitHubBlobRef, segment: Segment) -> str:
summary = segment.summary.strip()
if segment.kind == "nop" or not summary:
return ""
if segment.kind == "resolution" and segment.depends_on:
refs = [format_loc_reference(blob, dep) for dep in segment.depends_on]
if len(refs) == 1:
prefix = f"By condition at {refs[0]}, "
else:
prefix = f"By conditions at {', '.join(refs)}, "
return prefix + decapitalize_summary(summary)
return summary
def decapitalize_summary(summary: str) -> str:
if not summary:
return summary
if len(summary) >= 2 and summary[0].isupper() and summary[1].islower():
return summary[:1].lower() + summary[1:]
return summary
def render_markdown(blob: GitHubBlobRef, segments: list[Segment]) -> str:
lines = [f"- LLVMRevHash: `{blob.rev}`", f"# {blob.title}", ""]
for segment in segments:
lines.append(
f"## {format_line_range(segment.start_line, segment.end_line)} (kind: {segment.kind})"
)
summary = render_segment_summary(blob, segment)
if summary:
lines.append(summary)
lines.append("")
return "\n".join(lines).rstrip() + "\n"
def main() -> int:
args = parse_args()
try:
blob = parse_github_blob_url(args.url)
source_text = fetch_text(blob.raw_url, timeout=args.timeout)
_, line_count = number_source(source_text)
segments = analyze_with_codex_cli(
codex_bin=args.codex_bin,
blob=blob,
source_text=source_text,
line_count=line_count,
model=args.model,
reasoning_effort=args.reasoning_effort,
)
markdown = render_markdown(blob, segments)
except Exception as exc:
print(f"Error: {exc}", file=sys.stderr)
return 1
if args.output:
args.output.write_text(markdown, encoding="utf-8")
else:
sys.stdout.write(markdown)
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -4,7 +4,6 @@ import subprocess
import shutil import shutil
import argparse import argparse
import os import os
import json
from pathlib import Path from pathlib import Path
@@ -23,66 +22,6 @@ def normalize_mode(value: str) -> str:
) )
def build_native_tools(project_root: Path, build_dir: Path) -> Path:
"""Build native host tablegen tools for cross-compilation.
When cross-compiling LLVM, build tools like llvm-tblgen must run on the
host but would otherwise be compiled for the target architecture. This
function performs a minimal native build and returns the bin directory
containing host-runnable executables.
"""
native_dir = build_dir.parent / f"{build_dir.name}-native-tools"
native_dir.mkdir(exist_ok=True)
source_dir = project_root / "llvm"
cmake_args = [
"-G",
"Ninja",
"-DCMAKE_BUILD_TYPE=Release",
"-DLLVM_ENABLE_PROJECTS=clang;clang-tools-extra",
"-DLLVM_TARGETS_TO_BUILD=Native",
"-DLLVM_DISABLE_ASSEMBLY_FILES=ON",
"-DCMAKE_C_FLAGS=-w",
"-DCMAKE_CXX_FLAGS=-w",
]
if sys.platform == "win32":
cmake_args += [
"-DCMAKE_C_COMPILER=clang-cl",
"-DCMAKE_CXX_COMPILER=clang-cl",
]
else:
cmake_args += [
"-DCMAKE_C_COMPILER=clang",
"-DCMAKE_CXX_COMPILER=clang++",
]
print(f"\nConfiguring native host tools in {native_dir}...")
subprocess.check_call(
["cmake", "-S", str(source_dir), "-B", str(native_dir)] + cmake_args
)
required_tools = ["llvm-tblgen", "llvm-min-tblgen", "clang-tblgen"]
optional_tools = ["clang-tidy-confusable-chars-gen"]
for tool in required_tools:
print(f"Building native {tool}...")
subprocess.check_call(["cmake", "--build", str(native_dir), "--target", tool])
for tool in optional_tools:
try:
print(f"Building native {tool} (optional)...")
subprocess.check_call(
["cmake", "--build", str(native_dir), "--target", tool]
)
except subprocess.CalledProcessError:
print(f" {tool} not available, skipping.")
bin_dir = native_dir / "bin"
print(f"Native host tools ready in {bin_dir}")
return bin_dir
def main(): def main():
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
description="Build LLVM with specific configurations." description="Build LLVM with specific configurations."
@@ -109,10 +48,6 @@ def main():
"--build-dir", "--build-dir",
help="Custom build directory (relative to project root or absolute)", help="Custom build directory (relative to project root or absolute)",
) )
parser.add_argument(
"--target-triple",
help="Cross-compilation target triple (e.g. x86_64-apple-darwin, aarch64-linux-gnu, aarch64-pc-windows-msvc)",
)
args = parser.parse_args() args = parser.parse_args()
@@ -150,46 +85,118 @@ def main():
print("--- Configuration ---") print("--- Configuration ---")
print(f"Mode: {args.mode}") print(f"Mode: {args.mode}")
print(f"LTO: {args.lto}") print(f"LTO: {args.lto}")
print(f"Target Triple: {args.target_triple or '(native)'}")
print(f"Root: {project_root}") print(f"Root: {project_root}")
print(f"Build Dir: {build_dir}") print(f"Build Dir: {build_dir}")
print(f"Install Prefix: {install_prefix}") print(f"Install Prefix: {install_prefix}")
print(f"Toolchain: {toolchain_file}") print(f"Toolchain: {toolchain_file}")
print("---------------------") print("---------------------")
components_path = Path(__file__).resolve().parent / "llvm-components.json" llvm_distribution_components = [
with components_path.open() as f: "LLVMDemangle",
llvm_distribution_components = json.load(f)["components"] "LLVMSupport",
"LLVMCore",
"LLVMOption",
"LLVMBinaryFormat",
"LLVMMC",
"LLVMMCParser",
"LLVMObject",
"LLVMProfileData",
"LLVMBitReader",
"LLVMBitstreamReader",
"LLVMRemarks",
"LLVMObjectYAML",
"LLVMAggressiveInstCombine",
"LLVMInstCombine",
"LLVMIRReader",
"LLVMTextAPI",
"LLVMSymbolize",
"LLVMDebugInfoDWARF",
"LLVMDebugInfoDWARFLowLevel",
"LLVMDebugInfoCodeView",
"LLVMDebugInfoGSYM",
"LLVMDebugInfoPDB",
"LLVMDebugInfoBTF",
"LLVMDebugInfoMSF",
"LLVMAsmParser",
"LLVMTargetParser",
"LLVMTransformUtils",
"LLVMAnalysis",
"LLVMScalarOpts",
"LLVMFrontendHLSL",
"LLVMFrontendOpenMP",
"LLVMFrontendOffloading",
"LLVMFrontendAtomic",
"LLVMFrontendDirective",
"LLVMWindowsDriver",
"clangIndex",
"clangAPINotes",
"clangAST",
"clangASTMatchers",
"clangBasic",
"clangDriver",
"clangFormat",
"clangFrontend",
"clangLex",
"clangParse",
"clangSema",
"clangSerialization",
"clangRewrite",
"clangAnalysis",
"clangEdit",
"clangSupport",
"clangStaticAnalyzerCore",
"clangStaticAnalyzerFrontend",
"clangTidy",
"clangTidyUtils",
"clangTidyAndroidModule",
"clangTidyAbseilModule",
"clangTidyAlteraModule",
"clangTidyBoostModule",
"clangTidyBugproneModule",
"clangTidyCERTModule",
"clangTidyConcurrencyModule",
"clangTidyCppCoreGuidelinesModule",
"clangTidyDarwinModule",
"clangTidyFuchsiaModule",
"clangTidyGoogleModule",
"clangTidyHICPPModule",
"clangTidyLinuxKernelModule",
"clangTidyLLVMModule",
"clangTidyLLVMLibcModule",
"clangTidyMiscModule",
"clangTidyModernizeModule",
"clangTidyObjCModule",
"clangTidyOpenMPModule",
"clangTidyPerformanceModule",
"clangTidyPortabilityModule",
"clangTidyReadabilityModule",
"clangTidyZirconModule",
"clangTooling",
"clangToolingCore",
"clangToolingInclusions",
"clangToolingInclusionsStdlib",
"clangToolingSyntax",
"clangToolingRefactoring",
"clangTransformer",
"clangCrossTU",
"clangAnalysisFlowSensitive",
"clangAnalysisFlowSensitiveModels",
"clangStaticAnalyzerCheckers",
"clangIncludeCleaner",
"llvm-headers",
"clang-headers",
"clang-tidy-headers",
"clang-resource-headers",
]
components_joined = ";".join(llvm_distribution_components) components_joined = ";".join(llvm_distribution_components)
cmake_args = [ cmake_args = [
"-G", "-G",
"Ninja", "Ninja",
f"-DCMAKE_TOOLCHAIN_FILE={toolchain_file.as_posix()}",
f"-DCMAKE_INSTALL_PREFIX={install_prefix}", f"-DCMAKE_INSTALL_PREFIX={install_prefix}",
] "-DCMAKE_C_FLAGS=-w",
"-DCMAKE_CXX_FLAGS=-w",
if sys.platform == "win32":
# Use clang-cl (MSVC driver) on Windows so that LLVM's CMake
# generates correct MSVC-style linker flags for LTO, etc.
c_flags = "-w"
if args.target_triple:
c_flags += f" --target={args.target_triple}"
cmake_args += [
"-DCMAKE_C_COMPILER=clang-cl",
"-DCMAKE_CXX_COMPILER=clang-cl",
f"-DCMAKE_C_FLAGS={c_flags}",
f"-DCMAKE_CXX_FLAGS={c_flags}",
"-DLLVM_USE_LINKER=lld-link",
]
else:
cmake_args += [
f"-DCMAKE_TOOLCHAIN_FILE={toolchain_file.as_posix()}",
"-DCMAKE_C_FLAGS=-w",
"-DCMAKE_CXX_FLAGS=-w",
"-DLLVM_USE_LINKER=lld",
]
cmake_args += [
"-DLLVM_ENABLE_ZLIB=OFF", "-DLLVM_ENABLE_ZLIB=OFF",
"-DLLVM_ENABLE_ZSTD=OFF", "-DLLVM_ENABLE_ZSTD=OFF",
"-DLLVM_ENABLE_LIBXML2=OFF", "-DLLVM_ENABLE_LIBXML2=OFF",
@@ -224,6 +231,7 @@ def main():
"-DCMAKE_JOB_POOL_LINK=console", "-DCMAKE_JOB_POOL_LINK=console",
"-DLLVM_ENABLE_PROJECTS=clang;clang-tools-extra", "-DLLVM_ENABLE_PROJECTS=clang;clang-tools-extra",
"-DLLVM_TARGETS_TO_BUILD=all", "-DLLVM_TARGETS_TO_BUILD=all",
"-DLLVM_USE_LINKER=lld",
"-DLLVM_DISABLE_ASSEMBLY_FILES=ON", "-DLLVM_DISABLE_ASSEMBLY_FILES=ON",
# Distribution # Distribution
f"-DLLVM_DISTRIBUTION_COMPONENTS={components_joined}", f"-DLLVM_DISTRIBUTION_COMPONENTS={components_joined}",
@@ -248,10 +256,8 @@ def main():
is_shared = "OFF" is_shared = "OFF"
if args.mode == "Debug": if args.mode == "Debug":
cmake_args.append("-DCMAKE_BUILD_TYPE=Debug") cmake_args.append("-DCMAKE_BUILD_TYPE=Debug")
# ASAN is incompatible with -MDd on Windows (clang-cl), skip it there. cmake_args.append("-DLLVM_USE_SANITIZER=Address")
if sys.platform != "win32": is_shared = "ON"
cmake_args.append("-DLLVM_USE_SANITIZER=Address")
is_shared = "ON"
elif args.mode == "Release": elif args.mode == "Release":
cmake_args.append("-DCMAKE_BUILD_TYPE=Release") cmake_args.append("-DCMAKE_BUILD_TYPE=Release")
elif args.mode == "RelWithDebInfo": elif args.mode == "RelWithDebInfo":
@@ -266,24 +272,6 @@ def main():
else: else:
cmake_args.append("-DLLVM_ENABLE_LTO=OFF") cmake_args.append("-DLLVM_ENABLE_LTO=OFF")
if args.target_triple:
cmake_args.append(f"-DCLICE_TARGET_TRIPLE={args.target_triple}")
cmake_args.append(f"-DLLVM_HOST_TRIPLE={args.target_triple}")
# When cross-compiling, clear conda's host-platform flags so they
# don't leak into the target build (e.g. -L pointing to x86_64 libs).
# This must happen before the native-tools build too so we don't
# contaminate the native configure with target-arch link flags.
for var in ["LIBRARY_PATH", "LDFLAGS", "CFLAGS", "CXXFLAGS", "CPPFLAGS"]:
os.environ.pop(var, None)
# Cross-compilation needs native host tools (tablegen, etc.) that can
# run on the build machine. macOS handles this transparently via
# Rosetta 2, but Linux and Windows require a separate native build.
if sys.platform != "darwin":
native_bin_dir = build_native_tools(project_root, build_dir)
cmake_args.append(f"-DLLVM_NATIVE_TOOL_DIR={native_bin_dir}")
build_dir.mkdir(exist_ok=True) build_dir.mkdir(exist_ok=True)
print(f"\nConfiguring in {build_dir}...") print(f"\nConfiguring in {build_dir}...")

View File

@@ -1,99 +0,0 @@
{
"components": [
"LLVMDemangle",
"LLVMSupport",
"LLVMCore",
"LLVMOption",
"LLVMBinaryFormat",
"LLVMMC",
"LLVMMCParser",
"LLVMObject",
"LLVMProfileData",
"LLVMBitReader",
"LLVMBitstreamReader",
"LLVMRemarks",
"LLVMObjectYAML",
"LLVMAggressiveInstCombine",
"LLVMInstCombine",
"LLVMIRReader",
"LLVMTextAPI",
"LLVMSymbolize",
"LLVMDebugInfoDWARF",
"LLVMDebugInfoDWARFLowLevel",
"LLVMDebugInfoCodeView",
"LLVMDebugInfoGSYM",
"LLVMDebugInfoPDB",
"LLVMDebugInfoBTF",
"LLVMDebugInfoMSF",
"LLVMAsmParser",
"LLVMTargetParser",
"LLVMTransformUtils",
"LLVMAnalysis",
"LLVMScalarOpts",
"LLVMFrontendHLSL",
"LLVMFrontendOpenMP",
"LLVMFrontendOffloading",
"LLVMFrontendAtomic",
"LLVMFrontendDirective",
"LLVMWindowsDriver",
"clangIndex",
"clangAPINotes",
"clangAST",
"clangASTMatchers",
"clangBasic",
"clangDriver",
"clangFormat",
"clangFrontend",
"clangLex",
"clangParse",
"clangSema",
"clangSerialization",
"clangRewrite",
"clangAnalysis",
"clangEdit",
"clangSupport",
"clangStaticAnalyzerCore",
"clangStaticAnalyzerFrontend",
"clangTidy",
"clangTidyUtils",
"clangTidyAndroidModule",
"clangTidyAbseilModule",
"clangTidyAlteraModule",
"clangTidyBoostModule",
"clangTidyBugproneModule",
"clangTidyCERTModule",
"clangTidyConcurrencyModule",
"clangTidyCppCoreGuidelinesModule",
"clangTidyDarwinModule",
"clangTidyFuchsiaModule",
"clangTidyGoogleModule",
"clangTidyHICPPModule",
"clangTidyLinuxKernelModule",
"clangTidyLLVMModule",
"clangTidyLLVMLibcModule",
"clangTidyMiscModule",
"clangTidyModernizeModule",
"clangTidyObjCModule",
"clangTidyOpenMPModule",
"clangTidyPerformanceModule",
"clangTidyPortabilityModule",
"clangTidyReadabilityModule",
"clangTidyZirconModule",
"clangTooling",
"clangToolingCore",
"clangToolingInclusions",
"clangToolingInclusionsStdlib",
"clangToolingSyntax",
"clangToolingRefactoring",
"clangTransformer",
"clangCrossTU",
"clangAnalysisFlowSensitive",
"clangAnalysisFlowSensitiveModels",
"clangStaticAnalyzerCheckers",
"clangIncludeCleaner",
"llvm-headers",
"clang-headers",
"clang-tidy-headers",
"clang-resource-headers"
]
}

View File

@@ -1,66 +0,0 @@
#!/usr/bin/env python3
"""Run clang-tidy in parallel on all files in compile_commands.json."""
import json
import subprocess
import sys
import threading
from concurrent.futures import ThreadPoolExecutor, as_completed
from pathlib import Path
def main():
build_dir = sys.argv[1] if len(sys.argv) > 1 else "build/RelWithDebInfo"
cdb_path = Path(build_dir) / "compile_commands.json"
if not cdb_path.exists():
print(f"Error: {cdb_path} not found. Run cmake-config first.", file=sys.stderr)
sys.exit(1)
project_root = Path(__file__).resolve().parent.parent
src_dirs = (project_root / "src", project_root / "tests")
cdb = json.loads(cdb_path.read_text())
files = [
entry["file"]
for entry in cdb
if any(Path(entry["file"]).resolve().is_relative_to(d) for d in src_dirs)
]
total = len(files)
lock = threading.Lock()
done = 0
failed = []
def run(file: str) -> tuple[str, int, str]:
result = subprocess.run(
["clang-tidy", "-p", build_dir, "--quiet", file],
capture_output=True,
text=True,
)
return file, result.returncode, result.stdout + result.stderr
with ThreadPoolExecutor() as pool:
futures = {pool.submit(run, f): f for f in files}
for future in as_completed(futures):
file, code, output = future.result()
with lock:
done += 1
name = Path(file).name
if code != 0:
failed.append(file)
print(f"[{done}/{total}] FAIL {name}")
if output.strip():
print(output, end="")
else:
print(f"[{done}/{total}] OK {name}")
if failed:
print(f"\nclang-tidy failed on {len(failed)}/{total} files.", file=sys.stderr)
sys.exit(1)
print(f"\nclang-tidy passed on {total} files.")
if __name__ == "__main__":
main()

View File

@@ -40,52 +40,23 @@ def detect_platform() -> str:
raise RuntimeError(f"Unsupported platform: {plat}") raise RuntimeError(f"Unsupported platform: {plat}")
def detect_arch() -> str:
import platform
machine = platform.machine().lower()
if machine in ("x86_64", "amd64"):
return "x64"
if machine in ("aarch64", "arm64"):
return "arm64"
raise RuntimeError(f"Unsupported architecture: {machine}")
def pick_artifact( def pick_artifact(
manifest: list[dict], manifest: list[dict], version: str, build_type: str, is_lto: bool, platform: str
version: str,
build_type: str,
is_lto: bool,
platform: str,
arch: str,
) -> dict: ) -> dict:
base_version = version.split("+", 1)[0] base_version = version.split("+", 1)[0]
saw_missing_arch = False
for entry in manifest: for entry in manifest:
if entry.get("version") != version: if entry.get("version") != version:
continue continue
if entry.get("platform") != platform.lower(): if entry.get("platform") != platform.lower():
continue continue
entry_arch = entry.get("arch")
if entry_arch is None:
saw_missing_arch = True
continue
if entry_arch != arch:
continue
if entry.get("build_type") != build_type: if entry.get("build_type") != build_type:
continue continue
if bool(entry.get("lto")) != is_lto: if bool(entry.get("lto")) != is_lto:
continue continue
return entry return entry
if saw_missing_arch:
raise RuntimeError(
f"Manifest contains entries without an 'arch' field for version={base_version}, "
f"platform={platform}. The manifest format changed to require explicit "
f"architectures; regenerate it via scripts/update-llvm-version.py."
)
raise RuntimeError( raise RuntimeError(
f"No matching LLVM artifact in manifest for version={base_version}, platform={platform}, " f"No matching LLVM artifact in manifest for version={base_version}, platform={platform}, "
f"arch={arch}, build_type={build_type}, lto={is_lto}" f"build_type={build_type}, lto={is_lto}"
) )
@@ -293,14 +264,6 @@ def main() -> None:
parser.add_argument("--install-path") parser.add_argument("--install-path")
parser.add_argument("--enable-lto", action="store_true") parser.add_argument("--enable-lto", action="store_true")
parser.add_argument("--offline", action="store_true") parser.add_argument("--offline", action="store_true")
parser.add_argument(
"--target-platform",
help="Override platform for cross-compilation (e.g. macosx, linux, windows)",
)
parser.add_argument(
"--target-arch",
help="Override architecture for cross-compilation (e.g. x64, arm64)",
)
parser.add_argument("--output", required=True) parser.add_argument("--output", required=True)
args = parser.parse_args() args = parser.parse_args()
@@ -312,11 +275,8 @@ def main() -> None:
) )
token = os.environ.get("GH_TOKEN") or os.environ.get("GITHUB_TOKEN") token = os.environ.get("GH_TOKEN") or os.environ.get("GITHUB_TOKEN")
build_type = args.build_type build_type = args.build_type
platform_name = args.target_platform if args.target_platform else detect_platform() platform_name = detect_platform()
arch_name = args.target_arch if args.target_arch else detect_arch() log(f"Platform detected: {platform_name}, normalized build type: {build_type}")
log(
f"Platform: {platform_name}, arch: {arch_name}, normalized build type: {build_type}"
)
manifest = read_manifest(Path(args.manifest)) manifest = read_manifest(Path(args.manifest))
binary_dir = Path(args.binary_dir).resolve() binary_dir = Path(args.binary_dir).resolve()
@@ -344,12 +304,7 @@ def main() -> None:
if install_path is None: if install_path is None:
needs_install = True needs_install = True
artifact = pick_artifact( artifact = pick_artifact(
manifest, manifest, args.version, build_type, args.enable_lto, platform_name
args.version,
build_type,
args.enable_lto,
platform_name,
arch_name,
) )
log(f"Selected artifact: {artifact.get('filename')} for download") log(f"Selected artifact: {artifact.get('filename')} for download")
filename = artifact["filename"] filename = artifact["filename"]
@@ -362,12 +317,7 @@ def main() -> None:
install_path = install_root install_path = install_root
elif needs_install: elif needs_install:
artifact = pick_artifact( artifact = pick_artifact(
manifest, manifest, args.version, build_type, args.enable_lto, platform_name
args.version,
build_type,
args.enable_lto,
platform_name,
arch_name,
) )
log(f"Selected artifact: {artifact.get('filename')} for download") log(f"Selected artifact: {artifact.get('filename')} for download")
filename = artifact["filename"] filename = artifact["filename"]

View File

@@ -1,162 +0,0 @@
#!/usr/bin/env python3
import argparse
import json
import re
import sys
from pathlib import Path
def copy_manifest(src: Path, dest: Path) -> None:
text = src.read_text(encoding="utf-8")
try:
data = json.loads(text)
except json.JSONDecodeError as err:
print(f"Error: {src} is not valid JSON: {err}", file=sys.stderr)
sys.exit(1)
if not isinstance(data, list) or len(data) == 0:
print(f"Error: {src} must be a non-empty JSON array", file=sys.stderr)
sys.exit(1)
dest.parent.mkdir(parents=True, exist_ok=True)
with dest.open("w", encoding="utf-8") as handle:
json.dump(data, handle, indent=2)
handle.write("\n")
print(f"Copied manifest: {src} -> {dest} ({len(data)} entries)")
def update_package_cmake(path: Path, version: str) -> None:
text = path.read_text(encoding="utf-8")
pattern = r'setup_llvm\("[^"]*"\)'
matches = re.findall(pattern, text)
if len(matches) == 0:
print(f"Error: no setup_llvm(...) call found in {path}", file=sys.stderr)
sys.exit(1)
if len(matches) > 1:
print(
f"Error: expected exactly 1 setup_llvm(...) call in {path}, "
f"found {len(matches)}",
file=sys.stderr,
)
sys.exit(1)
old_call = matches[0]
new_call = f'setup_llvm("{version}")'
if old_call == new_call:
print(f"Version in {path} is already {version}, no change needed")
return
updated = text.replace(old_call, new_call)
path.write_text(updated, encoding="utf-8")
print(f"Updated {path}: {old_call} -> {new_call}")
def check_package_cmake(path: Path) -> None:
"""Verify package.cmake has exactly one setup_llvm(...) call that the
update script can rewrite. Used by CI to catch drift before the next bump."""
text = path.read_text(encoding="utf-8")
matches = re.findall(r'setup_llvm\("[^"]*"\)', text)
if len(matches) == 0:
print(f"Error: no setup_llvm(...) call found in {path}", file=sys.stderr)
sys.exit(1)
if len(matches) > 1:
print(
f"Error: expected exactly 1 setup_llvm(...) call in {path}, "
f"found {len(matches)}: {matches}",
file=sys.stderr,
)
sys.exit(1)
print(f"OK: {path} has a single setup_llvm(...) call: {matches[0]}")
def check_manifest(path: Path) -> None:
"""Verify the manifest is a well-formed non-empty array with required fields."""
try:
data = json.loads(path.read_text(encoding="utf-8"))
except json.JSONDecodeError as err:
print(f"Error: {path} is not valid JSON: {err}", file=sys.stderr)
sys.exit(1)
if not isinstance(data, list) or len(data) == 0:
print(f"Error: {path} must be a non-empty JSON array", file=sys.stderr)
sys.exit(1)
required = ("version", "platform", "arch", "build_type", "filename", "sha256")
for idx, entry in enumerate(data):
missing = [k for k in required if k not in entry]
if missing:
print(
f"Error: {path} entry {idx} is missing fields: {missing}",
file=sys.stderr,
)
sys.exit(1)
print(f"OK: {path} has {len(data)} well-formed entries")
def main() -> None:
parser = argparse.ArgumentParser(
description="Update LLVM version references in the clice project."
)
parser.add_argument(
"--check",
action="store_true",
help="Validate existing state without modifying files (for CI drift checks)",
)
parser.add_argument(
"--version",
help="New LLVM version string (e.g. 21.2.0); required unless --check",
)
parser.add_argument(
"--manifest-src",
help="Path to the source llvm-manifest.json; required unless --check",
)
parser.add_argument(
"--manifest-dest",
required=True,
help="Path to destination manifest (e.g. config/llvm-manifest.json)",
)
parser.add_argument(
"--package-cmake",
required=True,
help="Path to cmake/package.cmake",
)
args = parser.parse_args()
manifest_dest = Path(args.manifest_dest)
package_cmake = Path(args.package_cmake)
if not package_cmake.is_file():
print(f"Error: package.cmake not found: {package_cmake}", file=sys.stderr)
sys.exit(1)
if args.check:
check_package_cmake(package_cmake)
check_manifest(manifest_dest)
print("Done (check mode).")
return
if not args.version or not args.manifest_src:
print(
"Error: --version and --manifest-src are required unless --check is set",
file=sys.stderr,
)
sys.exit(1)
manifest_src = Path(args.manifest_src)
if not manifest_src.is_file():
print(f"Error: manifest source not found: {manifest_src}", file=sys.stderr)
sys.exit(1)
copy_manifest(manifest_src, manifest_dest)
update_package_cmake(package_cmake, args.version)
print("Done.")
if __name__ == "__main__":
main()

View File

@@ -27,15 +27,6 @@ def parse_platform(name: str) -> str:
raise ValueError(f"Unable to determine platform from filename: {name}") raise ValueError(f"Unable to determine platform from filename: {name}")
def parse_arch(name: str) -> str:
lowered = name.lower()
if lowered.startswith("aarch64-") or lowered.startswith("arm64-"):
return "arm64"
if lowered.startswith("x64-") or lowered.startswith("x86_64-"):
return "x64"
raise ValueError(f"Unable to determine arch from filename: {name}")
def parse_build_type(name: str) -> str: def parse_build_type(name: str) -> str:
lowered = name.lower() lowered = name.lower()
if "debug" in lowered: if "debug" in lowered:
@@ -52,7 +43,6 @@ def build_metadata_entry(path: Path, version: str) -> dict:
"lto": "-lto" in filename.lower(), "lto": "-lto" in filename.lower(),
"asan": "-asan" in filename.lower(), "asan": "-asan" in filename.lower(),
"platform": parse_platform(filename), "platform": parse_platform(filename),
"arch": parse_arch(filename),
"build_type": parse_build_type(filename), "build_type": parse_build_type(filename),
} }

View File

@@ -1,163 +0,0 @@
#!/usr/bin/env python3
"""
Validate the LLVM distribution component list against the actual LLVM source tree.
Scans the LLVM source for CMake library targets and compares them against
a components JSON file to detect stale or misspelled entries.
"""
import argparse
import difflib
import json
import re
import sys
from pathlib import Path
# CMake function calls that define library targets.
# The captured group uses [^\s)]+ to grab the target name without
# trailing parentheses or whitespace.
LLVM_LIB_PATTERNS = [
re.compile(r"add_llvm_component_library\(\s*([^\s)]+)"),
re.compile(r"add_llvm_library\(\s*([^\s)]+)"),
]
CLANG_LIB_PATTERNS = [
re.compile(r"add_clang_library\(\s*([^\s)]+)"),
]
# Header-only / custom install targets.
HEADER_PATTERNS = [
re.compile(r"add_llvm_install_targets\(\s*([^\s)]+)"),
re.compile(r"add_custom_target\(\s*([^\s)]+)"),
re.compile(r"add_library\(\s*([^\s)]+)"),
]
# Targets we recognise as header-only distribution components.
KNOWN_HEADER_TARGETS = {
"llvm-headers",
"clang-headers",
"clang-tidy-headers",
"clang-resource-headers",
}
def scan_targets(directory: Path, patterns: list[re.Pattern]) -> set[str]:
"""Recursively scan *directory* for CMakeLists.txt files and extract target names."""
targets: set[str] = set()
if not directory.is_dir():
return targets
for cmake_file in directory.rglob("CMakeLists.txt"):
text = cmake_file.read_text(errors="replace")
for pattern in patterns:
for match in pattern.finditer(text):
targets.add(match.group(1))
return targets
def scan_header_targets(llvm_src: Path) -> set[str]:
"""Scan for well-known header / custom-install targets across the tree."""
found: set[str] = set()
for cmake_file in llvm_src.rglob("CMakeLists.txt"):
text = cmake_file.read_text(errors="replace")
for pattern in HEADER_PATTERNS:
for match in pattern.finditer(text):
name = match.group(1)
if name in KNOWN_HEADER_TARGETS:
found.add(name)
return found
def collect_source_targets(llvm_src: Path) -> set[str]:
"""Return the full set of library / header targets found in the LLVM source tree."""
targets: set[str] = set()
targets |= scan_targets(llvm_src / "llvm" / "lib", LLVM_LIB_PATTERNS)
targets |= scan_targets(llvm_src / "clang" / "lib", CLANG_LIB_PATTERNS)
targets |= scan_targets(llvm_src / "clang-tools-extra", CLANG_LIB_PATTERNS)
targets |= scan_header_targets(llvm_src)
return targets
def load_components(path: Path) -> list[str]:
with path.open("r", encoding="utf-8") as handle:
data = json.load(handle)
if isinstance(data, dict):
data = data.get("components", [])
if not isinstance(data, list) or not data:
print(f"Error: no component list found in {path}", file=sys.stderr)
sys.exit(1)
return data
def main() -> None:
parser = argparse.ArgumentParser(
description="Validate LLVM distribution components against the source tree."
)
parser.add_argument(
"--llvm-src",
required=True,
help="Path to the llvm-project source root",
)
parser.add_argument(
"--components-file",
required=True,
help="Path to llvm-components.json",
)
args = parser.parse_args()
llvm_src = Path(args.llvm_src).expanduser().resolve()
components_file = Path(args.components_file).expanduser().resolve()
if not llvm_src.is_dir():
print(f"Error: LLVM source directory not found: {llvm_src}")
sys.exit(1)
if not (llvm_src / "llvm" / "CMakeLists.txt").exists():
print(f"Error: {llvm_src} does not look like an llvm-project root.")
sys.exit(1)
if not components_file.is_file():
print(f"Error: components file not found: {components_file}")
sys.exit(1)
components = load_components(components_file)
source_targets = collect_source_targets(llvm_src)
print(f"Found {len(source_targets)} targets in LLVM source tree")
print(f"Components file lists {len(components)} entries")
# Check for components that are missing from the source tree.
missing: list[tuple[str, list[str]]] = []
for name in components:
if name not in source_targets:
suggestions = difflib.get_close_matches(
name, source_targets, n=3, cutoff=0.6
)
missing.append((name, suggestions))
if missing:
print(f"\nError: {len(missing)} component(s) not found in the source tree:\n")
for name, suggestions in missing:
print(f" - {name}")
if suggestions:
print(f" Did you mean: {', '.join(suggestions)}?")
sys.exit(1)
# Warn about source targets not present in the component list.
component_set = set(components)
new_targets = sorted(source_targets - component_set - KNOWN_HEADER_TARGETS)
# Filter to targets that follow LLVM/Clang naming conventions to reduce noise.
noteworthy = [t for t in new_targets if t.startswith(("LLVM", "clang", "Clang"))]
if noteworthy:
print(
f"\nWarning: {len(noteworthy)} target(s) in source not listed in components:"
)
for name in noteworthy:
print(f" + {name}")
print("\nAll components validated successfully.")
sys.exit(0)
if __name__ == "__main__":
main()

24
scripts/watch-socket.sh Executable file
View File

@@ -0,0 +1,24 @@
#!/usr/bin/env bash
set -euo pipefail
BUILD_CMD=${BUILD_CMD:-".clice/build.sh"}
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "${ROOT_DIR}"
if ! command -v watchexec >/dev/null 2>&1; then
echo "watchexec is not installed or not in PATH." >&2
exit 1
fi
exec watchexec \
--project-origin "${ROOT_DIR}" \
--watch "${ROOT_DIR}/config" \
--watch "${ROOT_DIR}/src" \
--watch "${ROOT_DIR}/cmake" \
--watch "${ROOT_DIR}/CMakeLists.txt" \
--restart \
--clear \
--shell=bash \
-- "$BUILD_CMD && ./build/bin/clice --mode socket --port 50051"

View File

@@ -1,81 +1,64 @@
#include <csignal>
#include <cstdint> #include <cstdint>
#include <iostream> #include <iostream>
#include <print> #include <print>
#include <string> #include <string>
#include "eventide/async/async.h"
#include "eventide/deco/deco.h"
#include "eventide/ipc/peer.h"
#include "eventide/ipc/recording_transport.h"
#include "eventide/ipc/transport.h"
#include "server/master_server.h" #include "server/master_server.h"
#include "server/stateful_worker.h" #include "server/stateful_worker.h"
#include "server/stateless_worker.h" #include "server/stateless_worker.h"
#include "support/logging.h" #include "support/logging.h"
#include "kota/async/async.h"
#include "kota/deco/deco.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/peer.h"
#include "kota/ipc/recording_transport.h"
#include "kota/ipc/transport.h"
namespace clice { namespace clice {
using kota::deco::decl::KVStyle;
struct Options { struct Options {
DecoKV(style = KVStyle::JoinedOrSeparate, DecoKV(names = {"--mode"};
help = "Running mode: pipe, socket, stateless-worker, stateful-worker", help = "Running mode: pipe, socket, stateless-worker, stateful-worker";
required = false) required = false;)
<std::string> mode; <std::string> mode;
DecoKV(style = KVStyle::JoinedOrSeparate, help = "Socket mode address", required = false) DecoKV(names = {"--host"}; help = "Socket mode address"; required = false;)
<std::string> host = "127.0.0.1"; <std::string> host = "127.0.0.1";
DecoKV(style = KVStyle::JoinedOrSeparate, help = "Socket mode port", required = false) DecoKV(names = {"--port"}; help = "Socket mode port"; required = false;)
<int> port = 50051; <int> port = 50051;
DecoKV(style = KVStyle::JoinedOrSeparate, DecoKV(names = {"--stateful-worker-count"}; help = "Number of stateful workers";
names = {"--log-level", "--log-level="}, required = false;)
help = "Log level: trace, debug, info, warn, error, off", <std::uint32_t> stateful_worker_count;
required = false)
<std::string> log_level = "info";
DecoKV(style = KVStyle::JoinedOrSeparate, DecoKV(names = {"--stateless-worker-count"}; help = "Number of stateless workers";
help = "Record LSP input to file for replay testing", required = false;)
required = false) <std::uint32_t> stateless_worker_count;
<std::string> record;
// Internal options (passed from master to worker processes) DecoKV(names = {"--worker-memory-limit"}; help = "Memory limit per stateful worker (bytes)";
DecoKV(style = KVStyle::JoinedOrSeparate, required = false;)
names = {"--worker-memory-limit", "--worker-memory-limit="},
required = false)
<std::uint64_t> worker_memory_limit; <std::uint64_t> worker_memory_limit;
DecoKV(style = KVStyle::JoinedOrSeparate, DecoKV(names = {"--log-level"}; help = "Log level: trace, debug, info, warn, error, off";
names = {"--worker-name", "--worker-name="}, required = false;)
required = false) <std::string> log_level = "info";
<std::string> worker_name;
DecoKV(style = KVStyle::JoinedOrSeparate, names = {"--log-dir", "--log-dir="}, required = false) DecoKV(names = {"--record"}; help = "Record LSP input to file for replay testing";
<std::string> log_dir; required = false;)
<std::string> record;
DecoFlag(names = {"-h", "--help"}, help = "Show help message", required = false) DecoFlag(names = {"-h", "--help"}; help = "Show help message"; required = false;)
help; help;
DecoFlag(names = {"-v", "--version"}, help = "Show version", required = false) DecoFlag(names = {"-v", "--version"}; help = "Show version"; required = false;)
version; version;
}; };
} // namespace clice } // namespace clice
int main(int argc, const char** argv) { int main(int argc, const char** argv) {
#ifndef _WIN32 auto args = deco::util::argvify(argc, argv);
// On POSIX systems, ignore SIGPIPE so that writing to a closed pipe auto result = deco::cli::parse<clice::Options>(args);
// (e.g. when the LSP client disconnects) returns EPIPE instead of
// killing the process. This is standard practice for pipe-based servers.
signal(SIGPIPE, SIG_IGN);
#endif
auto args = kota::deco::util::argvify(argc, argv);
auto result = kota::deco::cli::parse<clice::Options>(args);
if(!result.has_value()) { if(!result.has_value()) {
LOG_ERROR("{}", result.error().message); LOG_ERROR("{}", result.error().message);
@@ -85,7 +68,7 @@ int main(int argc, const char** argv) {
auto& opts = result->options; auto& opts = result->options;
if(opts.help.value_or(false)) { if(opts.help.value_or(false)) {
kota::deco::cli::write_usage_for<clice::Options>(std::cout, "clice [OPTIONS]"); deco::cli::write_usage_for<clice::Options>(std::cout, "clice [OPTIONS]");
return 0; return 0;
} }
@@ -114,42 +97,35 @@ int main(int argc, const char** argv) {
auto& mode = *opts.mode; auto& mode = *opts.mode;
auto worker_name = opts.worker_name.value_or("");
auto log_dir = opts.log_dir.value_or("");
if(mode == "stateless-worker") { if(mode == "stateless-worker") {
return clice::run_stateless_worker_mode(worker_name.empty() ? "stateless-worker" return clice::run_stateless_worker_mode();
: worker_name,
log_dir);
} }
if(mode == "stateful-worker") { if(mode == "stateful-worker") {
auto mem_limit = opts.worker_memory_limit.value_or(4ULL * 1024 * 1024 * 1024); auto mem_limit = opts.worker_memory_limit.value_or(4ULL * 1024 * 1024 * 1024);
return clice::run_stateful_worker_mode(mem_limit, return clice::run_stateful_worker_mode(mem_limit);
worker_name.empty() ? "stateful-worker"
: worker_name,
log_dir);
} }
if(mode == "pipe") { if(mode == "pipe") {
clice::logging::stderr_logger("master", clice::logging::options); clice::logging::stderr_logger("master", clice::logging::options);
kota::event_loop loop; namespace et = eventide;
et::event_loop loop;
auto transport = kota::ipc::StreamTransport::open_stdio(loop); auto transport = et::ipc::StreamTransport::open_stdio(loop);
if(!transport) { if(!transport) {
LOG_ERROR("failed to open stdio transport"); LOG_ERROR("failed to open stdio transport");
return 1; return 1;
} }
std::unique_ptr<kota::ipc::Transport> final_transport = std::move(*transport); std::unique_ptr<et::ipc::Transport> final_transport = std::move(*transport);
if(opts.record.has_value()) { if(opts.record.has_value()) {
final_transport = final_transport =
std::make_unique<kota::ipc::RecordingTransport>(std::move(final_transport), std::make_unique<et::ipc::RecordingTransport>(std::move(final_transport),
*opts.record); *opts.record);
} }
kota::ipc::JsonPeer peer(loop, std::move(final_transport)); et::ipc::JsonPeer peer(loop, std::move(final_transport));
clice::MasterServer server(loop, peer, std::move(self_path)); clice::MasterServer server(loop, peer, std::move(self_path));
server.register_handlers(); server.register_handlers();
@@ -161,12 +137,13 @@ int main(int argc, const char** argv) {
if(mode == "socket") { if(mode == "socket") {
clice::logging::stderr_logger("master", clice::logging::options); clice::logging::stderr_logger("master", clice::logging::options);
kota::event_loop loop; namespace et = eventide;
et::event_loop loop;
auto host = opts.host.value_or("127.0.0.1"); auto host = opts.host.value_or("127.0.0.1");
auto port = opts.port.value_or(50051); auto port = opts.port.value_or(50051);
auto acceptor = kota::tcp::listen(host, port, {}, loop); auto acceptor = et::tcp::listen(host, port, {}, loop);
if(!acceptor) { if(!acceptor) {
LOG_ERROR("failed to listen on {}:{}", host, port); LOG_ERROR("failed to listen on {}:{}", host, port);
return 1; return 1;
@@ -174,7 +151,7 @@ int main(int argc, const char** argv) {
LOG_INFO("Listening on {}:{} ...", host, port); LOG_INFO("Listening on {}:{} ...", host, port);
auto task = [&]() -> kota::task<> { auto task = [&]() -> et::task<> {
auto client = co_await acceptor->accept(); auto client = co_await acceptor->accept();
if(!client.has_value()) { if(!client.has_value()) {
LOG_ERROR("failed to accept connection"); LOG_ERROR("failed to accept connection");
@@ -184,13 +161,13 @@ int main(int argc, const char** argv) {
LOG_INFO("Client connected"); LOG_INFO("Client connected");
std::unique_ptr<kota::ipc::Transport> transport = std::unique_ptr<et::ipc::Transport> transport =
std::make_unique<kota::ipc::StreamTransport>(std::move(client.value())); std::make_unique<et::ipc::StreamTransport>(std::move(client.value()));
if(opts.record.has_value()) { if(opts.record.has_value()) {
transport = std::make_unique<kota::ipc::RecordingTransport>(std::move(transport), transport = std::make_unique<et::ipc::RecordingTransport>(std::move(transport),
*opts.record); *opts.record);
} }
kota::ipc::JsonPeer peer(loop, std::move(transport)); et::ipc::JsonPeer peer(loop, std::move(transport));
clice::MasterServer server(loop, peer, std::string(self_path)); clice::MasterServer server(loop, peer, std::string(self_path));
server.register_handlers(); server.register_handlers();

View File

@@ -23,44 +23,6 @@ namespace ranges = std::ranges;
} // namespace } // namespace
std::vector<const char*> CompileCommand::to_argv() const {
std::vector<const char*> argv;
argv.reserve(resolved.flags.size() + 4);
if(resolved.is_cc1 && source_file) {
// cc1 mode requires TWO file-related arguments (both are needed):
// 1. -main-file-name <basename> — used by clang for diagnostics/debug info
// 2. <source_file> at the end — the actual input file path
// These are NOT duplicates: (1) is just the basename, (2) is the full path.
for(std::size_t i = 0; i < resolved.flags.size(); ++i) {
argv.push_back(resolved.flags[i]);
if(resolved.flags[i] == llvm::StringRef("-cc1")) {
argv.push_back("-main-file-name");
// path::filename returns a suffix of source_file (a pointer into
// the same buffer), so .data() is null-terminated because source_file is.
argv.push_back(path::filename(source_file).data());
}
}
} else {
argv.insert(argv.end(), resolved.flags.begin(), resolved.flags.end());
}
if(source_file) {
argv.push_back(source_file);
}
return argv;
}
std::vector<std::string> CompileCommand::to_string_argv() const {
auto argv = to_argv();
std::vector<std::string> result;
result.reserve(argv.size());
for(auto* arg: argv) {
result.emplace_back(arg);
}
return result;
}
CompilationDatabase::CompilationDatabase() = default; CompilationDatabase::CompilationDatabase() = default;
CompilationDatabase::~CompilationDatabase() = default; CompilationDatabase::~CompilationDatabase() = default;
@@ -367,8 +329,8 @@ std::size_t CompilationDatabase::load(llvm::StringRef path) {
return entries.size(); return entries.size();
} }
llvm::SmallVector<CompileCommand> CompilationDatabase::lookup(llvm::StringRef file, llvm::SmallVector<CompilationContext> CompilationDatabase::lookup(llvm::StringRef file,
const CommandOptions& options) { const CommandOptions& options) {
auto path_id = paths.intern(file); auto path_id = paths.intern(file);
auto matched = find_entries(path_id); auto matched = find_entries(path_id);
@@ -376,18 +338,17 @@ llvm::SmallVector<CompileCommand> CompilationDatabase::lookup(llvm::StringRef fi
render_arg_to([&](llvm::StringRef s) { out.push_back(strings.save(s).data()); }, arg); render_arg_to([&](llvm::StringRef s) { out.push_back(strings.save(s).data()); }, arg);
}; };
/// Build one CompileCommand from a single CompilationInfo. /// Build one CompilationContext from a single CompilationInfo.
auto build_command = [&](object_ptr<CompilationInfo> info) -> CompileCommand { auto build_context = [&](object_ptr<CompilationInfo> info) -> CompilationContext {
llvm::StringRef directory = info->directory; llvm::StringRef directory = info->directory;
std::vector<const char*> flags; std::vector<const char*> arguments;
bool is_cc1 = false;
auto append_arg = [&](llvm::StringRef s) { auto append_arg = [&](llvm::StringRef s) {
flags.emplace_back(strings.save(s).data()); arguments.emplace_back(strings.save(s).data());
}; };
auto append_args = [&](llvm::ArrayRef<const char*> args) { auto append_args = [&](llvm::ArrayRef<const char*> args) {
flags.insert(flags.end(), args.begin(), args.end()); arguments.insert(arguments.end(), args.begin(), args.end());
}; };
if(options.query_toolchain) { if(options.query_toolchain) {
@@ -400,20 +361,23 @@ llvm::SmallVector<CompileCommand> CompilationDatabase::lookup(llvm::StringRef fi
append_args(info->canonical->arguments); append_args(info->canonical->arguments);
append_args(info->patch); append_args(info->patch);
} else { } else {
flags.assign(cached.begin(), cached.end()); arguments.assign(cached.begin(), cached.end());
flags.pop_back(); // remove temp source file // TODO: add an assertion that the last arg is the temp source
// file (e.g., contains "query-toolchain") to guard against
// future changes in clang cc1 argument ordering.
arguments.pop_back(); // remove temp source file
// Replace resource dir if needed. // Replace resource dir if needed.
if(!resource_dir().empty()) { if(!resource_dir().empty()) {
llvm::StringRef old_resource_dir; llvm::StringRef old_resource_dir;
for(std::size_t i = 0; i + 1 < flags.size(); ++i) { for(std::size_t i = 0; i + 1 < arguments.size(); ++i) {
if(flags[i] == llvm::StringRef("-resource-dir")) { if(arguments[i] == llvm::StringRef("-resource-dir")) {
old_resource_dir = flags[i + 1]; old_resource_dir = arguments[i + 1];
break; break;
} }
} }
if(!old_resource_dir.empty() && old_resource_dir != resource_dir()) { if(!old_resource_dir.empty() && old_resource_dir != resource_dir()) {
for(auto& arg: flags) { for(auto& arg: arguments) {
llvm::StringRef s(arg); llvm::StringRef s(arg);
if(s.starts_with(old_resource_dir)) { if(s.starts_with(old_resource_dir)) {
auto replaced = auto replaced =
@@ -426,42 +390,39 @@ llvm::SmallVector<CompileCommand> CompilationDatabase::lookup(llvm::StringRef fi
append_args(info->patch); append_args(info->patch);
// Strip -main-file-name and its value from flags (to_argv() will // Fix -main-file-name to match the actual file.
// re-inject it with the correct basename when is_cc1 is set). bool next_main_file = false;
std::vector<const char*> cleaned; for(auto& arg: arguments) {
cleaned.reserve(flags.size()); if(arg == llvm::StringRef("-main-file-name")) {
for(std::size_t i = 0; i < flags.size(); ++i) { next_main_file = true;
if(flags[i] == llvm::StringRef("-main-file-name") && i + 1 < flags.size()) {
++i; // skip the value
continue; continue;
} }
cleaned.push_back(flags[i]); if(next_main_file) {
arg = strings.save(path::filename(file)).data();
next_main_file = false;
}
} }
flags = std::move(cleaned); }
// Detect cc1 mode (search rather than assuming index). // Inject our resource dir if not already present.
is_cc1 = ranges::contains(flags, llvm::StringRef("-cc1")); if(!resource_dir().empty()) {
bool has_resource_dir = false;
for(auto& arg: arguments) {
if(arg == llvm::StringRef("-resource-dir")) {
has_resource_dir = true;
break;
}
}
if(!has_resource_dir) {
append_arg("-resource-dir");
append_arg(resource_dir());
}
} }
} else { } else {
append_args(info->canonical->arguments); append_args(info->canonical->arguments);
append_args(info->patch); append_args(info->patch);
} }
// Inject our resource dir if not already present.
if(options.inject_resource_dir && !resource_dir().empty()) {
bool has_resource_dir = false;
for(auto& arg: flags) {
if(arg == llvm::StringRef("-resource-dir")) {
has_resource_dir = true;
break;
}
}
if(!has_resource_dir) {
append_arg("-resource-dir");
append_arg(resource_dir());
}
}
// Apply remove filter. // Apply remove filter.
if(!options.remove.empty()) { if(!options.remove.empty()) {
using Arg = std::unique_ptr<llvm::opt::Arg>; using Arg = std::unique_ptr<llvm::opt::Arg>;
@@ -479,12 +440,12 @@ llvm::SmallVector<CompileCommand> CompilationDatabase::lookup(llvm::StringRef fi
}; };
std::ranges::sort(remove_args, {}, get_id); std::ranges::sort(remove_args, {}, get_id);
auto saved_flags = std::move(flags); auto saved_args = std::move(arguments);
flags.clear(); arguments.clear();
flags.push_back(saved_flags.front()); arguments.push_back(saved_args.front());
parser->parse( parser->parse(
llvm::ArrayRef(saved_flags).drop_front(), llvm::ArrayRef(saved_args).drop_front(),
[&](Arg arg) { [&](Arg arg) {
auto id = arg->getOption().getID(); auto id = arg->getOption().getID();
auto range = std::ranges::equal_range(remove_args, id, {}, get_id); auto range = std::ranges::equal_range(remove_args, id, {}, get_id);
@@ -500,7 +461,7 @@ llvm::SmallVector<CompileCommand> CompilationDatabase::lookup(llvm::StringRef fi
return; return;
} }
} }
render_arg(flags, *arg); render_arg(arguments, *arg);
}, },
[](int, int) {}); [](int, int) {});
} }
@@ -509,34 +470,26 @@ llvm::SmallVector<CompileCommand> CompilationDatabase::lookup(llvm::StringRef fi
append_arg(arg); append_arg(arg);
} }
return CompileCommand{ arguments.emplace_back(paths.resolve(path_id).data());
ResolvedFlags{directory, std::move(flags), is_cc1}, return CompilationContext(directory, std::move(arguments));
paths.resolve(path_id).data()
};
}; };
llvm::SmallVector<CompileCommand> results; llvm::SmallVector<CompilationContext> results;
if(!matched.empty()) { if(!matched.empty()) {
for(auto& entry: matched) { for(auto& entry: matched) {
results.push_back(build_command(entry.info)); results.push_back(build_context(entry.info));
} }
} else { } else {
// No matching entry — synthesize a default command. // No matching entry — synthesize a default command.
std::vector<const char*> flags; std::vector<const char*> arguments;
if(file.ends_with(".cpp") || file.ends_with(".hpp") || file.ends_with(".cc")) { if(file.ends_with(".cpp") || file.ends_with(".hpp") || file.ends_with(".cc")) {
flags = {"clang++", "-std=c++20"}; arguments = {"clang++", "-std=c++20"};
} else { } else {
flags = {"clang"}; arguments = {"clang"};
} }
if(options.inject_resource_dir && !resource_dir().empty()) { arguments.emplace_back(paths.resolve(path_id).data());
flags.push_back(strings.save("-resource-dir").data()); results.push_back(CompilationContext({}, std::move(arguments)));
flags.push_back(strings.save(resource_dir()).data());
}
results.push_back(CompileCommand{
ResolvedFlags{{}, std::move(flags), false},
paths.resolve(path_id).data()
});
} }
return results; return results;
@@ -560,8 +513,8 @@ SearchConfig CompilationDatabase::lookup_search_config(llvm::StringRef file,
} }
auto results = lookup(file, options); auto results = lookup(file, options);
auto& cmd = results.front(); auto& ctx = results.front();
auto config = extract_search_config(cmd.to_argv(), cmd.resolved.directory); auto config = extract_search_config(ctx.arguments, ctx.directory);
if(cacheable) { if(cacheable) {
auto key = ConfigCacheKey{matched.front().info.ptr, options_bits(options)}; auto key = ConfigCacheKey{matched.front().info.ptr, options_bits(options)};
@@ -697,11 +650,6 @@ std::uint32_t CompilationDatabase::intern_path(llvm::StringRef path) {
return paths.intern(path); return paths.intern(path);
} }
bool CompilationDatabase::has_entry(llvm::StringRef file) {
auto path_id = paths.intern(file);
return !find_entries(path_id).empty();
}
llvm::ArrayRef<CompilationEntry> CompilationDatabase::get_entries() const { llvm::ArrayRef<CompilationEntry> CompilationDatabase::get_entries() const {
return entries; return entries;
} }

View File

@@ -29,11 +29,6 @@ struct CommandOptions {
/// Set true in unittests to avoid cluttering test output. /// Set true in unittests to avoid cluttering test output.
bool suppress_logging = false; bool suppress_logging = false;
/// Inject our resource dir into the flags if not already present.
/// Enabled by default so clang tools always use matching builtin headers.
/// Disable in unit tests that assert exact argument counts.
bool inject_resource_dir = true;
/// Extra arguments to remove from the original command line. /// Extra arguments to remove from the original command line.
llvm::ArrayRef<std::string> remove; llvm::ArrayRef<std::string> remove;
@@ -41,35 +36,12 @@ struct CommandOptions {
llvm::ArrayRef<std::string> append; llvm::ArrayRef<std::string> append;
}; };
/// File-independent compilation flags (shareable, suitable as cache key input). struct CompilationContext {
/// Does NOT contain source file path or -main-file-name.
struct ResolvedFlags {
/// The working directory of compilation. /// The working directory of compilation.
llvm::StringRef directory; llvm::StringRef directory;
/// All flags excluding source file path and -main-file-name. /// The compilation arguments.
std::vector<const char*> flags; std::vector<const char*> arguments;
/// Whether flags come from toolchain query (cc1 mode).
/// When true, flags are cc1 frontend args (resolved clang binary + "-cc1" + ...),
/// NOT the original driver command. to_argv() scans for "-cc1" in flags and
/// inserts -main-file-name immediately after it.
bool is_cc1 = false;
};
/// Compilation command = resolved flags + source file identity.
struct CompileCommand {
ResolvedFlags resolved;
/// Interned, pointer-stable. Must be null-terminated (required by to_argv()
/// and path::filename().data() which relies on the suffix being null-terminated).
const char* source_file = nullptr;
/// Produce full argv: flags + [-main-file-name <basename> if cc1] + source_file.
std::vector<const char*> to_argv() const;
/// Convenience: to_argv() converted to vector<string>.
std::vector<std::string> to_string_argv() const;
}; };
/// Shared compiler identity — driver + all semantics-affecting flags. /// Shared compiler identity — driver + all semantics-affecting flags.
@@ -202,10 +174,10 @@ public:
/// but toolchain cache survives. Returns the number of entries loaded. /// but toolchain cache survives. Returns the number of entries loaded.
std::size_t load(llvm::StringRef path); std::size_t load(llvm::StringRef path);
/// Lookup the compile commands for a file. A file may have multiple /// Lookup the compilation contexts for a file. A file may have multiple
/// compilation commands (e.g. different build configurations); all are returned. /// compilation commands (e.g. different build configurations); all are returned.
llvm::SmallVector<CompileCommand> lookup(llvm::StringRef file, llvm::SmallVector<CompilationContext> lookup(llvm::StringRef file,
const CommandOptions& options = {}); const CommandOptions& options = {});
/// Combined lookup + extract_search_config with internal caching. /// Combined lookup + extract_search_config with internal caching.
SearchConfig lookup_search_config(llvm::StringRef file, const CommandOptions& options = {}); SearchConfig lookup_search_config(llvm::StringRef file, const CommandOptions& options = {});
@@ -219,10 +191,6 @@ public:
/// Intern a file path and return its path_id. /// Intern a file path and return its path_id.
std::uint32_t intern_path(llvm::StringRef path); std::uint32_t intern_path(llvm::StringRef path);
/// Check if a file has an explicit entry in the compilation database
/// (as opposed to a synthesized default).
bool has_entry(llvm::StringRef file);
/// All compilation entries (sorted by path_id). /// All compilation entries (sorted by path_id).
llvm::ArrayRef<CompilationEntry> get_entries() const; llvm::ArrayRef<CompilationEntry> get_entries() const;

View File

@@ -5,10 +5,10 @@
#include <vector> #include <vector>
#include "command/argument_parser.h" #include "command/argument_parser.h"
#include "eventide/reflection/enum.h"
#include "support/filesystem.h" #include "support/filesystem.h"
#include "support/logging.h" #include "support/logging.h"
#include "kota/meta/enum.h"
#include "llvm/ADT/ScopeExit.h" #include "llvm/ADT/ScopeExit.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
#include "llvm/Support/FileSystem.h" #include "llvm/Support/FileSystem.h"
@@ -363,7 +363,7 @@ std::vector<const char*> query_toolchain(const QueryParams& params) {
case CompilerFamily::Unknown: { case CompilerFamily::Unknown: {
/// TODO: nvcc and intel compilers need further exploration. /// TODO: nvcc and intel compilers need further exploration.
LOG_ERROR("Fail to query driver, unknown supported driver kind: {}, driver is {}", LOG_ERROR("Fail to query driver, unknown supported driver kind: {}, driver is {}",
kota::meta::enum_name(family), eventide::refl::enum_name(family),
driver); driver);
std::vector<const char*> result; std::vector<const char*> result;

View File

@@ -87,9 +87,7 @@ auto CompilationUnitRef::file_path(clang::FileID fid) -> llvm::StringRef {
} }
auto entry = self->SM().getFileEntryRefForID(fid); auto entry = self->SM().getFileEntryRefForID(fid);
if(!entry) { assert(entry && "Invalid file entry");
return {};
}
llvm::SmallString<128> path; llvm::SmallString<128> path;
@@ -244,19 +242,13 @@ std::vector<std::string> CompilationUnitRef::deps() {
for(auto& [fid, directive]: directives()) { for(auto& [fid, directive]: directives()) {
for(auto& include: directive.includes) { for(auto& include: directive.includes) {
if(!include.skipped) { if(!include.skipped) {
auto path = file_path(include.fid); deps.try_emplace(file_path(include.fid));
if(!path.empty()) {
deps.try_emplace(path);
}
} }
} }
for(auto& has_include: directive.has_includes) { for(auto& has_include: directive.has_includes) {
if(has_include.fid.isValid()) { if(has_include.fid.isValid()) {
auto path = file_path(has_include.fid); deps.try_emplace(file_path(has_include.fid));
if(!path.empty()) {
deps.try_emplace(path);
}
} }
} }
} }

View File

@@ -65,36 +65,11 @@ public:
/// Rewritten Preprocessor Callbacks /// Rewritten Preprocessor Callbacks
/// ============================================================================ /// ============================================================================
void HasEmbed(clang::SourceLocation location,
llvm::StringRef filename,
bool is_angled,
clang::OptionalFileEntryRef file) override {
unit->directives[unit.file_id(location)].has_embeds.emplace_back(clice::HasEmbed{
.file_name = filename,
.file = file,
.is_angled = is_angled,
.loc = location,
});
}
void EmbedDirective(clang::SourceLocation location,
clang::StringRef filename,
bool is_angled,
clang::OptionalFileEntryRef file,
const clang::LexEmbedParametersResult&) override {
unit->directives[unit.file_id(location)].embeds.emplace_back(Embed{
.file_name = filename,
.file = file,
.is_angled = is_angled,
.loc = location,
});
}
void InclusionDirective(clang::SourceLocation hash_loc, void InclusionDirective(clang::SourceLocation hash_loc,
const clang::Token& include_tok, const clang::Token& include_tok,
llvm::StringRef, llvm::StringRef,
bool, bool,
clang::CharSourceRange, clang::CharSourceRange filename_range,
clang::OptionalFileEntryRef, clang::OptionalFileEntryRef,
llvm::StringRef, llvm::StringRef,
llvm::StringRef, llvm::StringRef,
@@ -108,6 +83,7 @@ public:
unit->directives[prev_fid].includes.emplace_back(Include{ unit->directives[prev_fid].includes.emplace_back(Include{
.fid = {}, .fid = {},
.location = include_tok.getLocation(), .location = include_tok.getLocation(),
.filename_range = filename_range.getAsRange(),
}); });
} }

View File

@@ -20,8 +20,11 @@ struct Include {
/// The file id of included file. /// The file id of included file.
clang::FileID fid; clang::FileID fid;
/// Location of the `include` keyword. /// Location of the `include`.
clang::SourceLocation location; clang::SourceLocation location;
/// The range of filename(includes `""` or `<>`).
clang::SourceRange filename_range;
}; };
/// Information about `__has_include` directive. /// Information about `__has_include` directive.
@@ -129,39 +132,6 @@ struct Import {
std::vector<clang::SourceLocation> name_locations; std::vector<clang::SourceLocation> name_locations;
}; };
/// Information about `#embed` directive.
struct Embed {
/// The file name in the embed directive, not including quotes or angle brackets.
llvm::StringRef file_name;
/// The actual file that may be embedded by this embed directive.
clang::OptionalFileEntryRef file;
/// Whether the file name is angled.
bool is_angled;
/// Location of the `#` token.
clang::SourceLocation loc;
/// TODO: Currently we do not store parameters of the embed directive.
/// See clang::LexEmbedParametersResult for details.
};
/// Information about `__has_embed` directive.
struct HasEmbed {
/// The file name in the embed directive, not including quotes or angle brackets.
llvm::StringRef file_name;
/// The actual file that may be embedded by this embed directive.
clang::OptionalFileEntryRef file;
/// Whether the file name is angled.
bool is_angled;
/// Location of the `__has_embed` token.
clang::SourceLocation loc;
};
struct Directive { struct Directive {
std::vector<Include> includes; std::vector<Include> includes;
std::vector<HasInclude> has_includes; std::vector<HasInclude> has_includes;
@@ -169,8 +139,6 @@ struct Directive {
std::vector<MacroRef> macros; std::vector<MacroRef> macros;
std::vector<Pragma> pragmas; std::vector<Pragma> pragmas;
std::vector<Import> imports; std::vector<Import> imports;
std::vector<Embed> embeds;
std::vector<HasEmbed> has_embeds;
}; };
} // namespace clice } // namespace clice

View File

@@ -53,8 +53,8 @@ auto completion_kind(const clang::NamedDecl* decl) -> protocol::CompletionItemKi
return protocol::CompletionItemKind::Module; return protocol::CompletionItemKind::Module;
} }
if(llvm::isa<clang::CXXConstructorDecl>(decl)) { if(llvm::isa<clang::FunctionDecl, clang::FunctionTemplateDecl>(decl)) {
return protocol::CompletionItemKind::Constructor; return protocol::CompletionItemKind::Function;
} }
if(llvm::isa<clang::CXXMethodDecl, if(llvm::isa<clang::CXXMethodDecl,
@@ -64,8 +64,8 @@ auto completion_kind(const clang::NamedDecl* decl) -> protocol::CompletionItemKi
return protocol::CompletionItemKind::Method; return protocol::CompletionItemKind::Method;
} }
if(llvm::isa<clang::FunctionDecl, clang::FunctionTemplateDecl>(decl)) { if(llvm::isa<clang::CXXConstructorDecl>(decl)) {
return protocol::CompletionItemKind::Function; return protocol::CompletionItemKind::Constructor;
} }
if(llvm::isa<clang::FieldDecl, clang::IndirectFieldDecl>(decl)) { if(llvm::isa<clang::FieldDecl, clang::IndirectFieldDecl>(decl)) {
@@ -109,115 +109,6 @@ auto completion_kind(const clang::NamedDecl* decl) -> protocol::CompletionItemKi
return protocol::CompletionItemKind::Text; return protocol::CompletionItemKind::Text;
} }
/// Extract the function signature (parameter list) from a CodeCompletionString.
/// Returns something like "(int x, float y)" for display in labelDetails.detail.
auto extract_signature(const clang::CodeCompletionString& ccs) -> std::string {
std::string signature;
bool in_parens = false;
for(const auto& chunk: ccs) {
using CK = clang::CodeCompletionString::ChunkKind;
switch(chunk.Kind) {
case CK::CK_LeftParen:
in_parens = true;
signature += '(';
break;
case CK::CK_RightParen:
signature += ')';
in_parens = false;
break;
case CK::CK_Placeholder:
case CK::CK_CurrentParameter:
if(in_parens && chunk.Text) {
signature += chunk.Text;
}
break;
case CK::CK_Text:
case CK::CK_Informative:
if(in_parens && chunk.Text) {
signature += chunk.Text;
}
break;
case CK::CK_LeftAngle:
signature += '<';
in_parens = true;
break;
case CK::CK_RightAngle:
signature += '>';
in_parens = false;
break;
case CK::CK_Comma:
if(in_parens) {
signature += ", ";
}
break;
default: break;
}
}
return signature;
}
/// Build a snippet string from a CodeCompletionString.
/// Produces e.g. "funcName(${1:int x}, ${2:float y})" for functions,
/// or "ClassName<${1:T}>" for class templates.
auto build_snippet(const clang::CodeCompletionString& ccs) -> std::string {
std::string snippet;
unsigned placeholder_index = 0;
for(const auto& chunk: ccs) {
using CK = clang::CodeCompletionString::ChunkKind;
switch(chunk.Kind) {
case CK::CK_TypedText:
if(chunk.Text) {
snippet += chunk.Text;
}
break;
case CK::CK_Placeholder:
if(chunk.Text) {
snippet += std::format("${{{0}:{1}}}", ++placeholder_index, chunk.Text);
}
break;
case CK::CK_LeftParen: snippet += '('; break;
case CK::CK_RightParen: snippet += ')'; break;
case CK::CK_LeftAngle: snippet += '<'; break;
case CK::CK_RightAngle: snippet += '>'; break;
case CK::CK_Comma: snippet += ", "; break;
case CK::CK_Text:
if(chunk.Text) {
snippet += chunk.Text;
}
break;
case CK::CK_Optional:
// Optional chunks contain default arguments — skip for snippet.
break;
case CK::CK_Informative:
case CK::CK_ResultType:
case CK::CK_CurrentParameter:
// Display-only chunks, not part of insertion.
break;
default: break;
}
}
// If no placeholders were generated, return empty to signal plain text.
if(placeholder_index == 0) {
return {};
}
return snippet;
}
/// Extract the return type from a CodeCompletionString.
auto extract_return_type(const clang::CodeCompletionString& ccs) -> std::string {
for(const auto& chunk: ccs) {
if(chunk.Kind == clang::CodeCompletionString::CK_ResultType && chunk.Text) {
return chunk.Text;
}
}
return {};
}
struct OverloadItem { struct OverloadItem {
protocol::CompletionItem item; protocol::CompletionItem item;
float score = 0.0F; float score = 0.0F;
@@ -268,45 +159,29 @@ public:
overloads.reserve(candidate_count); overloads.reserve(candidate_count);
std::unordered_map<std::string, std::size_t> overload_index; std::unordered_map<std::string, std::size_t> overload_index;
bool prefix_starts_with_underscore = prefix.spelling.starts_with("_"); auto build_item =
[&](llvm::StringRef label, protocol::CompletionItemKind kind, llvm::StringRef insert) {
protocol::CompletionItem item{
.label = label.str(),
};
item.kind = kind;
auto build_item = [&](llvm::StringRef label, protocol::TextEdit edit{
protocol::CompletionItemKind kind, .range = replace_range,
llvm::StringRef insert, .new_text = insert.empty() ? label.str() : insert.str(),
bool is_snippet = false) { };
protocol::CompletionItem item{ item.text_edit = std::move(edit);
.label = label.str(), return item;
}; };
item.kind = kind;
protocol::TextEdit edit{
.range = replace_range,
.new_text = insert.empty() ? label.str() : insert.str(),
};
item.text_edit = std::move(edit);
if(is_snippet) {
item.insert_text_format = protocol::InsertTextFormat::Snippet;
}
return item;
};
auto try_add = [&](llvm::StringRef label, auto try_add = [&](llvm::StringRef label,
protocol::CompletionItemKind kind, protocol::CompletionItemKind kind,
llvm::StringRef insert_text, llvm::StringRef insert_text,
llvm::StringRef overload_key, llvm::StringRef overload_key) {
llvm::StringRef signature = {},
llvm::StringRef return_type = {},
bool is_snippet = false,
bool is_deprecated = false) {
if(label.empty()) { if(label.empty()) {
return; return;
} }
// Filter out _/__ prefixed internal symbols unless user typed _.
if(!prefix_starts_with_underscore && label.starts_with("_")) {
return;
}
auto score = matcher.match(label); auto score = matcher.match(label);
if(!score.has_value()) { if(!score.has_value()) {
return; return;
@@ -316,21 +191,8 @@ public:
auto [it, inserted] = auto [it, inserted] =
overload_index.try_emplace(overload_key.str(), overloads.size()); overload_index.try_emplace(overload_key.str(), overloads.size());
if(inserted) { if(inserted) {
auto item = build_item(label, kind, insert_text, is_snippet); auto item = build_item(label, kind, insert_text);
item.sort_text = std::format("{}", *score); item.sort_text = std::format("{}", *score);
if(!signature.empty() || !return_type.empty()) {
protocol::CompletionItemLabelDetails details;
if(!signature.empty()) {
details.detail = signature.str();
}
if(!return_type.empty()) {
details.description = return_type.str();
}
item.label_details = std::move(details);
}
if(is_deprecated) {
item.tags = std::vector{protocol::CompletionItemTag::Deprecated};
}
overloads.push_back({ overloads.push_back({
.item = std::move(item), .item = std::move(item),
.score = *score, .score = *score,
@@ -347,21 +209,8 @@ public:
return; return;
} }
auto item = build_item(label, kind, insert_text, is_snippet); auto item = build_item(label, kind, insert_text);
item.sort_text = std::format("{}", *score); item.sort_text = std::format("{}", *score);
if(!signature.empty() || !return_type.empty()) {
protocol::CompletionItemLabelDetails details;
if(!signature.empty()) {
details.detail = signature.str();
}
if(!return_type.empty()) {
details.description = return_type.str();
}
item.label_details = std::move(details);
}
if(is_deprecated) {
item.tags = std::vector{protocol::CompletionItemTag::Deprecated};
}
collected.push_back(std::move(item)); collected.push_back(std::move(item));
}; };
@@ -393,60 +242,16 @@ public:
break; break;
} }
auto label = ast::name_of(declaration);
auto kind = completion_kind(declaration); auto kind = completion_kind(declaration);
// For constructors and deduction guides, use the class name
// (without template args) instead of the full type name.
// e.g. "vector" instead of "vector<_Tp, _Alloc>".
std::string label;
if(auto* ctor = llvm::dyn_cast<clang::CXXConstructorDecl>(declaration)) {
label = ctor->getParent()->getName().str();
} else if(auto* guide =
llvm::dyn_cast<clang::CXXDeductionGuideDecl>(declaration)) {
label = guide->getDeducedTemplate()->getName().str();
} else {
label = ast::name_of(declaration);
}
llvm::SmallString<256> qualified_name; llvm::SmallString<256> qualified_name;
bool is_callable = kind == protocol::CompletionItemKind::Function || if(options.bundle_overloads && kind == protocol::CompletionItemKind::Function) {
kind == protocol::CompletionItemKind::Method ||
kind == protocol::CompletionItemKind::Constructor;
if(options.bundle_overloads && is_callable) {
llvm::raw_svector_ostream stream(qualified_name); llvm::raw_svector_ostream stream(qualified_name);
declaration->printQualifiedName(stream); declaration->printQualifiedName(stream);
} }
std::string signature; try_add(label, kind, label, qualified_name.str());
std::string return_type;
std::string snippet;
auto* ccs =
candidate.CreateCodeCompletionString(sema,
context,
getAllocator(),
getCodeCompletionTUInfo(),
/*IncludeBriefComments=*/false);
if(ccs) {
signature = extract_signature(*ccs);
return_type = extract_return_type(*ccs);
// Generate snippet for non-bundled callables.
if(is_callable && !options.bundle_overloads &&
options.enable_function_arguments_snippet) {
snippet = build_snippet(*ccs);
}
}
bool has_snippet = !snippet.empty();
auto insert = has_snippet ? llvm::StringRef(snippet) : llvm::StringRef(label);
bool deprecated = candidate.Availability == CXAvailability_Deprecated;
try_add(label,
kind,
insert,
qualified_name.str(),
signature,
return_type,
has_snippet,
deprecated);
break; break;
} }
} }
@@ -454,48 +259,11 @@ public:
for(auto& entry: overloads) { for(auto& entry: overloads) {
if(entry.count > 1) { if(entry.count > 1) {
protocol::CompletionItemLabelDetails details; entry.item.detail = "(...)";
details.detail = std::format("(…) +{} overloads", entry.count);
entry.item.label_details = std::move(details);
} }
collected.push_back(std::move(entry.item)); collected.push_back(std::move(entry.item));
} }
// In bundle mode, deduplicate by label: when the same name appears as
// both a class and its constructors/deduction guides, keep only the
// highest-priority kind (Class > Function/Method > others).
if(options.bundle_overloads) {
auto kind_priority = [](protocol::CompletionItemKind k) -> int {
switch(k) {
case protocol::CompletionItemKind::Class:
case protocol::CompletionItemKind::Struct: return 3;
case protocol::CompletionItemKind::Function:
case protocol::CompletionItemKind::Method: return 2;
case protocol::CompletionItemKind::Constructor: return 1;
default: return 0;
}
};
std::unordered_map<std::string, std::size_t> label_index;
std::vector<protocol::CompletionItem> deduped;
deduped.reserve(collected.size());
for(auto& item: collected) {
auto [it, inserted] = label_index.try_emplace(item.label, deduped.size());
if(inserted) {
deduped.push_back(std::move(item));
} else {
auto& existing = deduped[it->second];
int old_prio = existing.kind.has_value() ? kind_priority(*existing.kind) : 0;
int new_prio = item.kind.has_value() ? kind_priority(*item.kind) : 0;
if(new_prio > old_prio) {
existing = std::move(item);
}
}
}
collected.swap(deduped);
}
output.clear(); output.clear();
output.swap(collected); output.swap(collected);
} }

View File

@@ -2,15 +2,14 @@
#include <string> #include <string>
#include <vector> #include <vector>
#include "eventide/ipc/lsp/uri.h"
#include "feature/feature.h" #include "feature/feature.h"
#include "kota/ipc/lsp/uri.h"
namespace clice::feature { namespace clice::feature {
namespace { namespace {
namespace lsp = kota::ipc::lsp; namespace lsp = eventide::ipc::lsp;
auto to_uri(llvm::StringRef file) -> std::string { auto to_uri(llvm::StringRef file) -> std::string {
const auto file_view = std::string_view(file.data(), file.size()); const auto file_view = std::string_view(file.data(), file.size());

View File

@@ -1,12 +1,14 @@
#include <algorithm>
#include <cstdint> #include <cstdint>
#include <string> #include <string>
#include <vector> #include <vector>
#include "feature/feature.h" #include "feature/feature.h"
#include "syntax/lexer.h"
namespace clice::feature { namespace clice::feature {
namespace {} // namespace
auto document_links(CompilationUnitRef unit, PositionEncoding encoding) auto document_links(CompilationUnitRef unit, PositionEncoding encoding)
-> std::vector<protocol::DocumentLink> { -> std::vector<protocol::DocumentLink> {
std::vector<protocol::DocumentLink> links; std::vector<protocol::DocumentLink> links;
@@ -20,42 +22,50 @@ auto document_links(CompilationUnitRef unit, PositionEncoding encoding)
auto content = unit.interested_content(); auto content = unit.interested_content();
PositionMapper converter(content, encoding); PositionMapper converter(content, encoding);
auto& directives = directives_it->second; auto& directives = directives_it->second;
auto* lang_opts = &unit.lang_options();
auto add_link = [&](clang::SourceLocation loc, llvm::StringRef target) { links.reserve(directives.includes.size() + directives.has_includes.size());
auto [fid, offset] = unit.decompose_location(loc);
if(fid != interested || offset >= content.size())
return;
auto range = find_directive_argument(content, offset, lang_opts);
if(!range)
return;
protocol::DocumentLink link{.range = to_range(converter, *range)};
link.target = target.str();
links.push_back(std::move(link));
};
for(const auto& include: directives.includes) { for(const auto& include: directives.includes) {
if(include.fid.isValid()) { auto [fid, range] = unit.decompose_range(include.filename_range);
add_link(include.location, unit.file_path(include.fid)); if(fid != interested || !range.valid()) {
continue;
} }
protocol::DocumentLink link{
.range = to_range(converter, range),
};
link.target = std::string(unit.file_path(include.fid));
links.push_back(std::move(link));
} }
for(const auto& has_include: directives.has_includes) { for(const auto& has_include: directives.has_includes) {
if(has_include.fid.isValid()) { if(has_include.fid.isInvalid()) {
add_link(has_include.location, unit.file_path(has_include.fid)); continue;
} }
}
for(const auto& embed: directives.embeds) { auto [fid, offset] = unit.decompose_location(has_include.location);
if(embed.file) { if(fid != interested || offset >= content.size()) {
add_link(embed.loc, embed.file->getName()); continue;
} }
}
for(const auto& has_embed: directives.has_embeds) { auto tail = content.substr(offset);
if(has_embed.file) { char open = tail.front();
add_link(has_embed.loc, has_embed.file->getName()); if(open != '<' && open != '"') {
continue;
} }
char close = open == '<' ? '>' : '"';
auto close_index = tail.find(close, 1);
if(close_index == llvm::StringRef::npos) {
continue;
}
LocalSourceRange range(offset, offset + static_cast<std::uint32_t>(close_index + 1));
protocol::DocumentLink link{
.range = to_range(converter, range),
};
link.target = std::string(unit.file_path(has_include.fid));
links.push_back(std::move(link));
} }
return links; return links;

View File

@@ -7,9 +7,8 @@
#include "compile/compilation.h" #include "compile/compilation.h"
#include "compile/compilation_unit.h" #include "compile/compilation_unit.h"
#include "eventide/ipc/lsp/position.h"
#include "kota/ipc/lsp/position.h" #include "eventide/ipc/lsp/protocol.h"
#include "kota/ipc/lsp/protocol.h"
namespace clang { namespace clang {
@@ -19,11 +18,11 @@ class NamedDecl;
namespace clice::feature { namespace clice::feature {
namespace protocol = kota::ipc::protocol; namespace protocol = eventide::ipc::protocol;
using kota::ipc::lsp::PositionEncoding; using eventide::ipc::lsp::PositionEncoding;
using kota::ipc::lsp::PositionMapper; using eventide::ipc::lsp::PositionMapper;
using kota::ipc::lsp::parse_position_encoding; using eventide::ipc::lsp::parse_position_encoding;
inline auto to_range(const PositionMapper& converter, LocalSourceRange range) -> protocol::Range { inline auto to_range(const PositionMapper& converter, LocalSourceRange range) -> protocol::Range {
return protocol::Range{ return protocol::Range{

View File

@@ -12,7 +12,6 @@
#include "clang/AST/Attr.h" #include "clang/AST/Attr.h"
#include "clang/Basic/IdentifierTable.h" #include "clang/Basic/IdentifierTable.h"
#include "clang/Basic/Module.h"
namespace clice::feature { namespace clice::feature {
@@ -24,8 +23,12 @@ struct RawToken {
std::uint32_t modifiers = 0; std::uint32_t modifiers = 0;
}; };
constexpr std::uint32_t bit(SymbolModifiers::Kind kind) {
return static_cast<std::uint32_t>(kind);
}
void add_modifier(std::uint32_t& modifiers, SymbolModifiers::Kind kind) { void add_modifier(std::uint32_t& modifiers, SymbolModifiers::Kind kind) {
modifiers |= SymbolModifiers::to_mask(kind); modifiers |= bit(kind);
} }
auto type_index(SymbolKind kind) -> std::uint32_t { auto type_index(SymbolKind kind) -> std::uint32_t {
@@ -36,132 +39,6 @@ auto encode_modifiers(std::uint32_t modifiers) -> std::uint32_t {
return modifiers; return modifiers;
} }
bool is_dependent(const clang::Decl* D) {
return isa<clang::UnresolvedUsingValueDecl>(D);
}
/// Returns true if `decl` is considered to be from a default/system library.
/// This currently checks the systemness of the file by include type, although
/// different heuristics may be used in the future (e.g. sysroot paths).
bool is_default_library(const clang::Decl* decl) {
clang::SourceLocation location = decl->getLocation();
if(!location.isValid()) {
return false;
}
return decl->getASTContext().getSourceManager().isInSystemHeader(location);
}
// "Static" means many things in C++, only some get the "static" modifier.
//
// Meanings that do:
// - Members associated with the class rather than the instance.
// This is what 'static' most often means across languages.
// - static local variables
// These are similarly "detached from their context" by the static keyword.
// In practice, these are rarely used inside classes, reducing confusion.
//
// Meanings that don't:
// - Namespace-scoped variables, which have static storage class.
// This is implicit, so the keyword "static" isn't so strongly associated.
// If we want a modifier for these, "global scope" is probably the concept.
// - Namespace-scoped variables/functions explicitly marked "static".
// There the keyword changes *linkage* , which is a totally different concept.
// If we want to model this, "file scope" would be a nice modifier.
//
// This is confusing, and maybe we should use another name, but because "static"
// is a standard LSP modifier, having one with that name has advantages.
bool is_static(const clang::Decl* decl) {
if(const auto* method = llvm::dyn_cast<clang::CXXMethodDecl>(decl)) {
return method->isStatic();
}
if(const auto* var_decl = llvm::dyn_cast<clang::VarDecl>(decl)) {
return var_decl->isStaticDataMember() || var_decl->isStaticLocal();
}
if(const auto* objc_property = llvm::dyn_cast<clang::ObjCPropertyDecl>(decl)) {
return objc_property->isClassProperty();
}
if(const auto* objc_method = llvm::dyn_cast<clang::ObjCMethodDecl>(decl)) {
return objc_method->isClassMethod();
}
if(const auto* function = llvm::dyn_cast<clang::FunctionDecl>(decl)) {
return function->isStatic();
}
return false;
}
// Whether `type` is const in a loose sense: would a value of this type be readonly?
bool is_const(clang::QualType type) {
if(type.isNull()) {
return false;
}
type = type.getNonReferenceType();
if(type.isConstQualified()) {
return true;
}
if(const auto* array_type = type->getAsArrayTypeUnsafe()) {
return is_const(array_type->getElementType());
}
if(is_const(type->getPointeeType())) {
return true;
}
return false;
}
// Whether `decl` is const in a loose sense (should it be highlighted as such?)
// FIXME: This is separate from whether a particular usage can mutate `decl`.
// We may want a receiver in `value.size()` to be readonly even if `value` is mutable.
bool is_const(const clang::Decl* decl) {
if(llvm::isa<clang::EnumConstantDecl>(decl) ||
llvm::isa<clang::NonTypeTemplateParmDecl>(decl)) {
return true;
}
if(llvm::isa<clang::FieldDecl>(decl) || llvm::isa<clang::VarDecl>(decl) ||
llvm::isa<clang::MSPropertyDecl>(decl) || llvm::isa<clang::BindingDecl>(decl)) {
if(is_const(llvm::cast<clang::ValueDecl>(decl)->getType())) {
return true;
}
}
if(const auto* objc_property = llvm::dyn_cast<clang::ObjCPropertyDecl>(decl)) {
if(objc_property->isReadOnly()) {
return true;
}
}
if(const auto* ms_property = llvm::dyn_cast<clang::MSPropertyDecl>(decl)) {
if(!ms_property->hasSetter()) {
return true;
}
}
if(const auto* method = llvm::dyn_cast<clang::CXXMethodDecl>(decl)) {
if(method->isConst()) {
return true;
}
}
if(const auto* function = llvm::dyn_cast<clang::FunctionDecl>(decl)) {
return is_const(function->getReturnType());
}
return false;
}
// Indicates whether declaration `decl` is abstract in cases where it is a struct or a
// class.
bool is_abstract(const clang::Decl* decl) {
if(const auto* method = llvm::dyn_cast<clang::CXXMethodDecl>(decl)) {
return method->isPureVirtual();
}
if(const auto* record = llvm::dyn_cast<clang::CXXRecordDecl>(decl)) {
return record->hasDefinition() && record->isAbstract();
}
return false;
}
// Indicates whether declaration `decl` is virtual in cases where it is a method.
bool is_virtual(const clang::Decl* decl) {
if(const auto* method = llvm::dyn_cast<clang::CXXMethodDecl>(decl)) {
return method->isVirtual();
}
return false;
}
class SemanticTokensCollector : public SemanticVisitor<SemanticTokensCollector> { class SemanticTokensCollector : public SemanticVisitor<SemanticTokensCollector> {
public: public:
explicit SemanticTokensCollector(CompilationUnitRef unit) : SemanticVisitor(unit, true) {} explicit SemanticTokensCollector(CompilationUnitRef unit) : SemanticVisitor(unit, true) {}
@@ -169,7 +46,6 @@ public:
auto collect() -> std::vector<RawToken> { auto collect() -> std::vector<RawToken> {
highlight_lexical(unit.interested_file()); highlight_lexical(unit.interested_file());
run(); run();
highlight_modules();
merge_tokens(); merge_tokens();
return std::move(tokens); return std::move(tokens);
} }
@@ -179,8 +55,6 @@ public:
clang::SourceLocation location) { clang::SourceLocation location) {
std::uint32_t modifiers = 0; std::uint32_t modifiers = 0;
if(relation.is_one_of(RelationKind::Definition)) { if(relation.is_one_of(RelationKind::Definition)) {
// todo: clangd add both Declaration and Definition modifiers for definitions.
// add_modifier(modifiers, SymbolModifiers::Declaration);
add_modifier(modifiers, SymbolModifiers::Definition); add_modifier(modifiers, SymbolModifiers::Definition);
} else if(relation.is_one_of(RelationKind::Declaration)) { } else if(relation.is_one_of(RelationKind::Declaration)) {
add_modifier(modifiers, SymbolModifiers::Declaration); add_modifier(modifiers, SymbolModifiers::Declaration);
@@ -189,42 +63,6 @@ public:
if(ast::is_templated(decl)) { if(ast::is_templated(decl)) {
add_modifier(modifiers, SymbolModifiers::Templated); add_modifier(modifiers, SymbolModifiers::Templated);
} }
// Apply attribute-style modifiers to the underlying declaration.
// The attribute tests don't want to look at the template.
if(const auto* template_decl = llvm::dyn_cast<clang::TemplateDecl>(decl)) {
if(const auto* templated_decl = template_decl->getTemplatedDecl())
decl = templated_decl;
}
// TODO: add scope-based modifiers once the local model supports them.
// if (auto Mod = scopeModifier(Decl))
// Tok.addModifier(*Mod);
if(is_const(decl)) {
add_modifier(modifiers, SymbolModifiers::Readonly);
}
if(is_static(decl)) {
add_modifier(modifiers, SymbolModifiers::Static);
}
if(is_abstract(decl)) {
add_modifier(modifiers, SymbolModifiers::Abstract);
}
if(is_virtual(decl)) {
add_modifier(modifiers, SymbolModifiers::Virtual);
}
if(is_default_library(decl)) {
add_modifier(modifiers, SymbolModifiers::DefaultLibrary);
}
if(decl->isDeprecated()) {
add_modifier(modifiers, SymbolModifiers::Deprecated);
}
if(is_dependent(decl)) {
add_modifier(modifiers, SymbolModifiers::DependentName);
}
if(llvm::isa<clang::CXXConstructorDecl>(decl) ||
llvm::isa<clang::CXXDestructorDecl>(decl)) {
add_modifier(modifiers, SymbolModifiers::ConstructorOrDestructor);
}
add_token(location, SymbolKind::from(decl), modifiers); add_token(location, SymbolKind::from(decl), modifiers);
} }
@@ -242,10 +80,6 @@ public:
add_token(location, SymbolKind::Macro, modifiers); add_token(location, SymbolKind::Macro, modifiers);
} }
// handleModuleOccurrence
// handleRelation
void handleAttrOccurrence(const clang::Attr* attr, clang::SourceRange range) { void handleAttrOccurrence(const clang::Attr* attr, clang::SourceRange range) {
auto [begin, end] = range; auto [begin, end] = range;
if(llvm::isa<clang::FinalAttr, clang::OverrideAttr>(attr)) { if(llvm::isa<clang::FinalAttr, clang::OverrideAttr>(attr)) {
@@ -293,58 +127,6 @@ private:
}); });
} }
void highlight_modules() {
auto interested = unit.interested_file();
auto directives_it = unit.directives().find(interested);
if(directives_it != unit.directives().end()) {
for(const auto& import: directives_it->second.imports) {
add_token(import.location, SymbolKind::Keyword, 0);
for(auto loc: import.name_locations) {
add_token(loc, SymbolKind::Module, 0);
}
}
}
auto* mod = unit.context().getCurrentNamedModule();
if(!mod) {
return;
}
auto def_loc = mod->DefinitionLoc;
if(!def_loc.isValid() || !def_loc.isFileID()) {
return;
}
auto [fid, offset] = unit.decompose_location(def_loc);
if(fid != interested) {
return;
}
auto content = unit.file_content(fid);
auto& lang_opts = unit.lang_options();
Lexer lexer(content.substr(offset), false, &lang_opts);
auto module_token = lexer.advance();
if(module_token.is_identifier()) {
auto range = LocalSourceRange(offset + module_token.range.begin,
offset + module_token.range.end);
tokens.push_back({.range = range, .kind = SymbolKind::Keyword, .modifiers = 0});
}
// Scan for identifiers (module name parts) until semicolon/eof.
while(true) {
auto token = lexer.advance();
if(token.is_eof() || token.kind == clang::tok::semi) {
break;
}
if(token.is_identifier()) {
auto range = LocalSourceRange(offset + token.range.begin, offset + token.range.end);
tokens.push_back({.range = range, .kind = SymbolKind::Module, .modifiers = 0});
}
}
}
void highlight_lexical(clang::FileID fid) { void highlight_lexical(clang::FileID fid) {
auto content = unit.file_content(fid); auto content = unit.file_content(fid);
auto& lang_opts = unit.lang_options(); auto& lang_opts = unit.lang_options();
@@ -376,6 +158,7 @@ private:
case clang::tok::utf16_string_literal: case clang::tok::utf16_string_literal:
case clang::tok::utf32_string_literal: kind = SymbolKind::String; break; case clang::tok::utf32_string_literal: kind = SymbolKind::String; break;
case clang::tok::header_name: kind = SymbolKind::Header; break; case clang::tok::header_name: kind = SymbolKind::Header; break;
case clang::tok::identifier: break;
case clang::tok::raw_identifier: { case clang::tok::raw_identifier: {
auto previous = lexer.last(); auto previous = lexer.last();
if(previous.is_pp_keyword && previous.text(content) == "define") { if(previous.is_pp_keyword && previous.text(content) == "define") {
@@ -389,8 +172,457 @@ private:
} }
break; break;
} }
/* Keywords */
default: break; case clang::tok::kw_auto:
case clang::tok::kw_break:
case clang::tok::kw_case:
case clang::tok::kw_char:
case clang::tok::kw_const:
case clang::tok::kw_continue:
case clang::tok::kw_default:
case clang::tok::kw_do:
case clang::tok::kw_double:
case clang::tok::kw_else:
case clang::tok::kw_enum:
case clang::tok::kw_extern:
case clang::tok::kw_float:
case clang::tok::kw_for:
case clang::tok::kw_goto:
case clang::tok::kw_if:
case clang::tok::kw_int:
case clang::tok::kw__ExtInt:
case clang::tok::kw__BitInt:
case clang::tok::kw_long:
case clang::tok::kw_register:
case clang::tok::kw_return:
case clang::tok::kw_short:
case clang::tok::kw_signed:
case clang::tok::kw_sizeof:
case clang::tok::kw___datasizeof:
case clang::tok::kw_static:
case clang::tok::kw_struct:
case clang::tok::kw_switch:
case clang::tok::kw_typedef:
case clang::tok::kw_union:
case clang::tok::kw_unsigned:
case clang::tok::kw_void:
case clang::tok::kw_volatile:
case clang::tok::kw_while:
case clang::tok::kw__Alignas:
case clang::tok::kw__Alignof:
case clang::tok::kw__Atomic:
case clang::tok::kw__Bool:
case clang::tok::kw__Complex:
case clang::tok::kw__Generic:
case clang::tok::kw__Imaginary:
case clang::tok::kw__Noreturn:
case clang::tok::kw__Static_assert:
case clang::tok::kw__Thread_local:
case clang::tok::kw___func__:
case clang::tok::kw___objc_yes:
case clang::tok::kw___objc_no:
case clang::tok::kw___ptrauth:
case clang::tok::kw__Countof:
case clang::tok::kw_asm:
case clang::tok::kw_bool:
case clang::tok::kw_catch:
case clang::tok::kw_class:
case clang::tok::kw_const_cast:
case clang::tok::kw_delete:
case clang::tok::kw_dynamic_cast:
case clang::tok::kw_explicit:
case clang::tok::kw_export:
case clang::tok::kw_false:
case clang::tok::kw_friend:
case clang::tok::kw_mutable:
case clang::tok::kw_namespace:
case clang::tok::kw_new:
case clang::tok::kw_operator:
case clang::tok::kw_private:
case clang::tok::kw_protected:
case clang::tok::kw_public:
case clang::tok::kw_reinterpret_cast:
case clang::tok::kw_static_cast:
case clang::tok::kw_template:
case clang::tok::kw_this:
case clang::tok::kw_throw:
case clang::tok::kw_true:
case clang::tok::kw_try:
case clang::tok::kw_typename:
case clang::tok::kw_typeid:
case clang::tok::kw_using:
case clang::tok::kw_virtual:
case clang::tok::kw_wchar_t:
case clang::tok::kw_restrict:
case clang::tok::kw_inline:
case clang::tok::kw_alignas:
case clang::tok::kw_alignof:
case clang::tok::kw_char16_t:
case clang::tok::kw_char32_t:
case clang::tok::kw_constexpr:
case clang::tok::kw_decltype:
case clang::tok::kw_noexcept:
case clang::tok::kw_nullptr:
case clang::tok::kw_static_assert:
case clang::tok::kw_thread_local:
case clang::tok::kw_co_await:
case clang::tok::kw_co_return:
case clang::tok::kw_co_yield:
case clang::tok::kw_module:
case clang::tok::kw_import:
case clang::tok::kw_consteval:
case clang::tok::kw_constinit:
case clang::tok::kw_concept:
case clang::tok::kw_requires:
case clang::tok::kw_char8_t:
case clang::tok::kw__Float16:
case clang::tok::kw_typeof:
case clang::tok::kw_typeof_unqual:
case clang::tok::kw__Accum:
case clang::tok::kw__Fract:
case clang::tok::kw__Sat:
case clang::tok::kw__Decimal32:
case clang::tok::kw__Decimal64:
case clang::tok::kw__Decimal128:
case clang::tok::kw___null:
case clang::tok::kw___alignof:
case clang::tok::kw___attribute:
case clang::tok::kw___builtin_choose_expr:
case clang::tok::kw___builtin_offsetof:
case clang::tok::kw___builtin_FILE:
case clang::tok::kw___builtin_FILE_NAME:
case clang::tok::kw___builtin_FUNCTION:
case clang::tok::kw___builtin_FUNCSIG:
case clang::tok::kw___builtin_LINE:
case clang::tok::kw___builtin_COLUMN:
case clang::tok::kw___builtin_source_location:
case clang::tok::kw___builtin_types_compatible_p:
case clang::tok::kw___builtin_va_arg:
case clang::tok::kw___extension__:
case clang::tok::kw___float128:
case clang::tok::kw___ibm128:
case clang::tok::kw___imag:
case clang::tok::kw___int128:
case clang::tok::kw___label__:
case clang::tok::kw___real:
case clang::tok::kw___thread:
case clang::tok::kw___FUNCTION__:
case clang::tok::kw___PRETTY_FUNCTION__:
case clang::tok::kw___auto_type:
case clang::tok::kw___FUNCDNAME__:
case clang::tok::kw___FUNCSIG__:
case clang::tok::kw_L__FUNCTION__:
case clang::tok::kw_L__FUNCSIG__:
case clang::tok::kw___is_interface_class:
case clang::tok::kw___is_sealed:
case clang::tok::kw___is_destructible:
case clang::tok::kw___is_trivially_destructible:
case clang::tok::kw___is_nothrow_destructible:
case clang::tok::kw___is_nothrow_assignable:
case clang::tok::kw___is_constructible:
case clang::tok::kw___is_nothrow_constructible:
case clang::tok::kw___is_assignable:
case clang::tok::kw___has_nothrow_move_assign:
case clang::tok::kw___has_trivial_move_assign:
case clang::tok::kw___has_trivial_move_constructor:
case clang::tok::kw___builtin_is_implicit_lifetime:
case clang::tok::kw___builtin_is_virtual_base_of:
case clang::tok::kw___has_nothrow_assign:
case clang::tok::kw___has_nothrow_copy:
case clang::tok::kw___has_nothrow_constructor:
case clang::tok::kw___has_trivial_assign:
case clang::tok::kw___has_trivial_copy:
case clang::tok::kw___has_trivial_constructor:
case clang::tok::kw___has_trivial_destructor:
case clang::tok::kw___has_virtual_destructor:
case clang::tok::kw___is_abstract:
case clang::tok::kw___is_aggregate:
case clang::tok::kw___is_base_of:
case clang::tok::kw___is_class:
case clang::tok::kw___is_convertible_to:
case clang::tok::kw___is_empty:
case clang::tok::kw___is_enum:
case clang::tok::kw___is_final:
case clang::tok::kw___is_literal:
case clang::tok::kw___is_pod:
case clang::tok::kw___is_polymorphic:
case clang::tok::kw___is_standard_layout:
case clang::tok::kw___is_trivial:
case clang::tok::kw___is_trivially_assignable:
case clang::tok::kw___is_trivially_constructible:
case clang::tok::kw___is_trivially_copyable:
case clang::tok::kw___is_union:
case clang::tok::kw___has_unique_object_representations:
case clang::tok::kw___is_layout_compatible:
case clang::tok::kw___is_pointer_interconvertible_base_of:
case clang::tok::kw___add_lvalue_reference:
case clang::tok::kw___add_pointer:
case clang::tok::kw___add_rvalue_reference:
case clang::tok::kw___decay:
case clang::tok::kw___make_signed:
case clang::tok::kw___make_unsigned:
case clang::tok::kw___remove_all_extents:
case clang::tok::kw___remove_const:
case clang::tok::kw___remove_cv:
case clang::tok::kw___remove_cvref:
case clang::tok::kw___remove_extent:
case clang::tok::kw___remove_pointer:
case clang::tok::kw___remove_reference_t:
case clang::tok::kw___remove_restrict:
case clang::tok::kw___remove_volatile:
case clang::tok::kw___underlying_type:
case clang::tok::kw___is_trivially_equality_comparable:
case clang::tok::kw___is_bounded_array:
case clang::tok::kw___is_unbounded_array:
case clang::tok::kw___is_scoped_enum:
case clang::tok::kw___can_pass_in_regs:
case clang::tok::kw___reference_binds_to_temporary:
case clang::tok::kw___reference_constructs_from_temporary:
case clang::tok::kw___reference_converts_from_temporary:
case clang::tok::kw_:
case clang::tok::kw___builtin_is_cpp_trivially_relocatable:
case clang::tok::kw___is_trivially_relocatable:
case clang::tok::kw___is_bitwise_cloneable:
case clang::tok::kw___builtin_is_replaceable:
case clang::tok::kw___builtin_structured_binding_size:
case clang::tok::kw___is_lvalue_expr:
case clang::tok::kw___is_rvalue_expr:
case clang::tok::kw___is_arithmetic:
case clang::tok::kw___is_floating_point:
case clang::tok::kw___is_integral:
case clang::tok::kw___is_complete_type:
case clang::tok::kw___is_void:
case clang::tok::kw___is_array:
case clang::tok::kw___is_function:
case clang::tok::kw___is_reference:
case clang::tok::kw___is_lvalue_reference:
case clang::tok::kw___is_rvalue_reference:
case clang::tok::kw___is_fundamental:
case clang::tok::kw___is_object:
case clang::tok::kw___is_scalar:
case clang::tok::kw___is_compound:
case clang::tok::kw___is_pointer:
case clang::tok::kw___is_member_object_pointer:
case clang::tok::kw___is_member_function_pointer:
case clang::tok::kw___is_member_pointer:
case clang::tok::kw___is_const:
case clang::tok::kw___is_volatile:
case clang::tok::kw___is_signed:
case clang::tok::kw___is_unsigned:
case clang::tok::kw___is_same:
case clang::tok::kw___is_convertible:
case clang::tok::kw___is_nothrow_convertible:
case clang::tok::kw___array_rank:
case clang::tok::kw___array_extent:
case clang::tok::kw___private_extern__:
case clang::tok::kw___module_private__:
case clang::tok::kw___builtin_ptrauth_type_discriminator:
case clang::tok::kw___declspec:
case clang::tok::kw___cdecl:
case clang::tok::kw___stdcall:
case clang::tok::kw___fastcall:
case clang::tok::kw___thiscall:
case clang::tok::kw___regcall:
case clang::tok::kw___vectorcall:
case clang::tok::kw___forceinline:
case clang::tok::kw___unaligned:
case clang::tok::kw___super:
case clang::tok::kw___global:
case clang::tok::kw___local:
case clang::tok::kw___constant:
case clang::tok::kw___private:
case clang::tok::kw___generic:
case clang::tok::kw___kernel:
case clang::tok::kw___read_only:
case clang::tok::kw___write_only:
case clang::tok::kw___read_write:
case clang::tok::kw___builtin_astype:
case clang::tok::kw_vec_step:
case clang::tok::kw_image1d_t:
case clang::tok::kw_image1d_array_t:
case clang::tok::kw_image1d_buffer_t:
case clang::tok::kw_image2d_t:
case clang::tok::kw_image2d_array_t:
case clang::tok::kw_image2d_depth_t:
case clang::tok::kw_image2d_array_depth_t:
case clang::tok::kw_image2d_msaa_t:
case clang::tok::kw_image2d_array_msaa_t:
case clang::tok::kw_image2d_msaa_depth_t:
case clang::tok::kw_image2d_array_msaa_depth_t:
case clang::tok::kw_image3d_t:
case clang::tok::kw_pipe:
case clang::tok::kw_addrspace_cast:
case clang::tok::kw___noinline__:
case clang::tok::kw_cbuffer:
case clang::tok::kw_tbuffer:
case clang::tok::kw_groupshared:
case clang::tok::kw_in:
case clang::tok::kw_inout:
case clang::tok::kw_out:
case clang::tok::kw___hlsl_resource_t:
case clang::tok::kw___builtin_hlsl_is_scalarized_layout_compatible:
case clang::tok::kw___builtin_hlsl_is_intangible:
case clang::tok::kw___builtin_hlsl_is_typed_resource_element_compatible:
case clang::tok::kw___builtin_omp_required_simd_align:
case clang::tok::kw___pascal:
case clang::tok::kw___vector:
case clang::tok::kw___pixel:
case clang::tok::kw___bool:
case clang::tok::kw___bf16:
case clang::tok::kw_half:
case clang::tok::kw___bridge:
case clang::tok::kw___bridge_transfer:
case clang::tok::kw___bridge_retained:
case clang::tok::kw___bridge_retain:
case clang::tok::kw___covariant:
case clang::tok::kw___contravariant:
case clang::tok::kw___kindof:
case clang::tok::kw__Nonnull:
case clang::tok::kw__Nullable:
case clang::tok::kw__Nullable_result:
case clang::tok::kw__Null_unspecified:
case clang::tok::kw___funcref:
case clang::tok::kw___ptr64:
case clang::tok::kw___ptr32:
case clang::tok::kw___sptr:
case clang::tok::kw___uptr:
case clang::tok::kw___w64:
case clang::tok::kw___uuidof:
case clang::tok::kw___try:
case clang::tok::kw___finally:
case clang::tok::kw___leave:
case clang::tok::kw___int64:
case clang::tok::kw___if_exists:
case clang::tok::kw___if_not_exists:
case clang::tok::kw___single_inheritance:
case clang::tok::kw___multiple_inheritance:
case clang::tok::kw___virtual_inheritance:
case clang::tok::kw___interface:
case clang::tok::kw___builtin_convertvector:
case clang::tok::kw___builtin_vectorelements:
case clang::tok::kw___builtin_bit_cast:
case clang::tok::kw___builtin_available:
case clang::tok::kw___builtin_sycl_unique_stable_name:
case clang::tok::kw___arm_agnostic:
case clang::tok::kw___arm_in:
case clang::tok::kw___arm_inout:
case clang::tok::kw___arm_locally_streaming:
case clang::tok::kw___arm_new:
case clang::tok::kw___arm_out:
case clang::tok::kw___arm_preserves:
case clang::tok::kw___arm_streaming:
case clang::tok::kw___arm_streaming_compatible:
case clang::tok::kw___unknown_anytype: kind = SymbolKind::Keyword; break;
/* Operators */
case clang::tok::l_square:
case clang::tok::r_square:
case clang::tok::l_paren:
case clang::tok::r_paren:
case clang::tok::l_brace:
case clang::tok::r_brace:
case clang::tok::period:
case clang::tok::ellipsis:
case clang::tok::amp:
case clang::tok::ampamp:
case clang::tok::ampequal:
case clang::tok::star:
case clang::tok::starequal:
case clang::tok::plus:
case clang::tok::plusplus:
case clang::tok::plusequal:
case clang::tok::minus:
case clang::tok::arrow:
case clang::tok::minusminus:
case clang::tok::minusequal:
case clang::tok::tilde:
case clang::tok::exclaim:
case clang::tok::exclaimequal:
case clang::tok::slash:
case clang::tok::slashequal:
case clang::tok::percent:
case clang::tok::percentequal:
case clang::tok::less:
case clang::tok::lessless:
case clang::tok::lessequal:
case clang::tok::lesslessequal:
case clang::tok::greater:
case clang::tok::greatergreater:
case clang::tok::greaterequal:
case clang::tok::greatergreaterequal:
case clang::tok::caret:
case clang::tok::caretequal:
case clang::tok::pipe:
case clang::tok::pipepipe:
case clang::tok::pipeequal:
case clang::tok::question:
case clang::tok::colon:
case clang::tok::semi:
case clang::tok::equal:
case clang::tok::equalequal:
case clang::tok::comma:
case clang::tok::hashat:
case clang::tok::periodstar:
case clang::tok::arrowstar:
case clang::tok::coloncolon:
case clang::tok::at:
case clang::tok::lesslessless:
case clang::tok::greatergreatergreater: break;
case clang::tok::annot_cxxscope:
case clang::tok::annot_typename:
case clang::tok::annot_template_id:
case clang::tok::annot_non_type:
case clang::tok::annot_non_type_undeclared:
case clang::tok::annot_non_type_dependent:
case clang::tok::annot_overload_set:
case clang::tok::annot_primary_expr:
case clang::tok::annot_decltype:
case clang::tok::annot_pack_indexing_type:
case clang::tok::annot_pragma_unused:
case clang::tok::annot_pragma_vis:
case clang::tok::annot_pragma_pack:
case clang::tok::annot_pragma_parser_crash:
case clang::tok::annot_pragma_captured:
case clang::tok::annot_pragma_dump:
case clang::tok::annot_pragma_msstruct:
case clang::tok::annot_pragma_align:
case clang::tok::annot_pragma_weak:
case clang::tok::annot_pragma_weakalias:
case clang::tok::annot_pragma_redefine_extname:
case clang::tok::annot_pragma_fp_contract:
case clang::tok::annot_pragma_fenv_access:
case clang::tok::annot_pragma_fenv_access_ms:
case clang::tok::annot_pragma_fenv_round:
case clang::tok::annot_pragma_cx_limited_range:
case clang::tok::annot_pragma_float_control:
case clang::tok::annot_pragma_ms_pointers_to_members:
case clang::tok::annot_pragma_ms_vtordisp:
case clang::tok::annot_pragma_ms_pragma:
case clang::tok::annot_pragma_opencl_extension:
case clang::tok::annot_attr_openmp:
case clang::tok::annot_pragma_openmp:
case clang::tok::annot_pragma_openmp_end:
case clang::tok::annot_pragma_openacc:
case clang::tok::annot_pragma_openacc_end:
case clang::tok::annot_pragma_loop_hint:
case clang::tok::annot_pragma_fp:
case clang::tok::annot_pragma_attribute:
case clang::tok::annot_pragma_riscv:
case clang::tok::annot_module_include:
case clang::tok::annot_module_begin:
case clang::tok::annot_module_end:
case clang::tok::annot_header_unit:
case clang::tok::annot_repl_input_end:
case clang::tok::annot_embed: break;
/* Others */
case clang::tok::spaceship:
case clang::tok::binary_data:
case clang::tok::hash:
case clang::tok::hashhash:
case clang::tok::unknown:
case clang::tok::eof:
case clang::tok::eod:
case clang::tok::code_completion:
case clang::tok::NUM_TOKENS: break;
} }
} }
@@ -399,17 +631,10 @@ private:
} }
static void resolve_conflict(RawToken& last, const RawToken& current) { static void resolve_conflict(RawToken& last, const RawToken& current) {
(void)current;
if(last.kind == SymbolKind::Conflict) { if(last.kind == SymbolKind::Conflict) {
return; return;
} }
// Directive is a low-priority lexical kind; semantic tokens override it.
if(last.kind == SymbolKind::Directive) {
last = current;
return;
}
if(current.kind == SymbolKind::Directive) {
return;
}
last.kind = SymbolKind::Conflict; last.kind = SymbolKind::Conflict;
} }

View File

@@ -14,8 +14,8 @@ namespace {
class Builder : public SemanticVisitor<Builder> { class Builder : public SemanticVisitor<Builder> {
public: public:
Builder(TUIndex& result, CompilationUnitRef unit, bool interested_only) : Builder(TUIndex& result, CompilationUnitRef unit) :
SemanticVisitor<Builder>(unit, interested_only), result(result) { SemanticVisitor<Builder>(unit, false), result(result) {
result.graph = IncludeGraph::from(unit); result.graph = IncludeGraph::from(unit);
} }
@@ -188,11 +188,11 @@ std::array<std::uint8_t, 32> FileIndex::hash() {
return hasher.final(); return hasher.final();
} }
TUIndex TUIndex::build(CompilationUnitRef unit, bool interested_only) { TUIndex TUIndex::build(CompilationUnitRef unit) {
TUIndex index; TUIndex index;
index.built_at = unit.build_at(); index.built_at = unit.build_at();
Builder builder(index, unit, interested_only); Builder builder(index, unit);
builder.build(); builder.build();
return index; return index;

View File

@@ -85,7 +85,7 @@ struct TUIndex {
FileIndex main_file_index; FileIndex main_file_index;
static TUIndex build(CompilationUnitRef unit, bool interested_only = false); static TUIndex build(CompilationUnitRef unit);
void serialize(llvm::raw_ostream& os) const; void serialize(llvm::raw_ostream& os) const;

View File

@@ -37,6 +37,10 @@ std::string name_of(const clang::NamedDecl* decl);
std::string display_name_of(const clang::NamedDecl* decl); std::string display_name_of(const clang::NamedDecl* decl);
clang::NestedNameSpecifierLoc get_qualifier_loc(const clang::NamedDecl* decl);
std::string print_template_specialization_args(const clang::NamedDecl* decl);
/// To response go-to-type-definition request. Some decls actually have a type /// To response go-to-type-definition request. Some decls actually have a type
/// for example the result of `typeof(var)` is the type of `var`. This function /// for example the result of `typeof(var)` is the type of `var`. This function
/// returns the type for the decl if any. /// returns the type for the decl if any.

1449
src/semantic/find_target.cpp Normal file

File diff suppressed because it is too large Load Diff

136
src/semantic/find_target.h Normal file
View File

@@ -0,0 +1,136 @@
#pragma once
#include "llvm/ADT/STLFunctionalExtras.h"
#include "llvm/ADT/SmallVector.h"
#include "llvm/Support/raw_ostream.h"
#include "clang/AST/ASTContext.h"
#include "clang/AST/ASTTypeTraits.h"
#include "clang/AST/NestedNameSpecifier.h"
#include "clang/AST/Stmt.h"
#include "clang/Basic/SourceLocation.h"
namespace clang {
class Decl;
class NamedDecl;
} // namespace clang
namespace clice {
class TemplateResolver;
/// Information about a reference written in the source code, independent of
/// the AST node that contains it.
struct ReferenceLoc {
/// Qualifier written in the source code, e.g. `ns::` for `ns::foo`.
clang::NestedNameSpecifierLoc Qualifier;
/// Start location of the last name part, e.g. `foo` in `ns::foo<int>`.
clang::SourceLocation NameLoc;
/// True when the reference is introducing a declaration or definition.
bool IsDecl = false;
/// The declarations referenced by the written name.
llvm::SmallVector<const clang::NamedDecl*, 1> Targets;
};
enum class DeclRelation : unsigned {
/// The written name is an alias that should be preserved in results.
Alias = 1u << 0,
/// The target was reached by desugaring or following the aliased entity.
Underlying = 1u << 1,
/// The target is a concrete template instantiation.
TemplateInstantiation = 1u << 2,
/// The target is the template pattern underlying an instantiation.
TemplatePattern = 1u << 3,
};
struct DeclRelationSet {
unsigned bits = 0;
constexpr DeclRelationSet() = default;
constexpr DeclRelationSet(DeclRelation relation) : bits(static_cast<unsigned>(relation)) {}
constexpr explicit DeclRelationSet(unsigned bits) : bits(bits) {}
constexpr bool contains(DeclRelationSet other) const {
return (bits & other.bits) == other.bits;
}
constexpr bool contains(DeclRelation relation) const {
return (bits & static_cast<unsigned>(relation)) != 0;
}
constexpr explicit operator bool() const {
return bits != 0;
}
constexpr DeclRelationSet& operator|=(DeclRelationSet other) {
bits |= other.bits;
return *this;
}
constexpr DeclRelationSet& operator|=(DeclRelation relation) {
bits |= static_cast<unsigned>(relation);
return *this;
}
};
constexpr DeclRelationSet operator|(DeclRelationSet lhs, DeclRelationSet rhs) {
return DeclRelationSet(lhs.bits | rhs.bits);
}
constexpr DeclRelationSet operator|(DeclRelationSet lhs, DeclRelation rhs) {
return lhs | DeclRelationSet(rhs);
}
constexpr DeclRelationSet operator|(DeclRelation lhs, DeclRelationSet rhs) {
return DeclRelationSet(lhs) | rhs;
}
constexpr DeclRelationSet operator|(DeclRelation lhs, DeclRelation rhs) {
return DeclRelationSet(lhs) | rhs;
}
constexpr DeclRelationSet operator&(DeclRelationSet lhs, DeclRelationSet rhs) {
return DeclRelationSet(lhs.bits & rhs.bits);
}
constexpr DeclRelationSet operator&(DeclRelationSet lhs, DeclRelation rhs) {
return lhs & DeclRelationSet(rhs);
}
struct TargetDecl {
const clang::NamedDecl* Decl = nullptr;
DeclRelationSet Relations;
};
llvm::raw_ostream& operator<<(llvm::raw_ostream& os, ReferenceLoc ref);
/// Finds all declarations a selected AST node may refer to, including alias
/// and template-instantiation relationships that higher-level APIs may filter.
auto all_target_decls(const clang::DynTypedNode& node, TemplateResolver* resolver = nullptr)
-> llvm::SmallVector<TargetDecl, 1>;
/// Recursively traverses \p stmt and reports all references explicitly written in
/// the source code.
void explicit_references(const clang::Stmt* stmt,
llvm::function_ref<void(ReferenceLoc)> out,
TemplateResolver* resolver = nullptr);
/// Recursively traverses \p decl and reports all references explicitly written in
/// the source code.
void explicit_references(const clang::Decl* decl,
llvm::function_ref<void(ReferenceLoc)> out,
TemplateResolver* resolver = nullptr);
/// Recursively traverses the full AST and reports all references explicitly
/// written in the source code.
void explicit_references(const clang::ASTContext& ast,
llvm::function_ref<void(ReferenceLoc)> out,
TemplateResolver* resolver = nullptr);
} // namespace clice

File diff suppressed because it is too large Load Diff

View File

@@ -2,11 +2,11 @@
#include "clang/AST/ExprCXX.h" #include "clang/AST/ExprCXX.h"
#include "clang/AST/Type.h" #include "clang/AST/Type.h"
#include "clang/Basic/SourceManager.h"
#include "clang/Sema/Sema.h"
namespace clang { namespace clang {
class Sema;
} }
namespace clice { namespace clice {
@@ -17,12 +17,9 @@ namespace clice {
/// completion, you cannot get go-to-definition, etc. To avoid this, we just use /// completion, you cannot get go-to-definition, etc. To avoid this, we just use
/// some heuristics to simplify the dependent names as normal type/expression. /// some heuristics to simplify the dependent names as normal type/expression.
/// For example, `std::vector<T>::value_type` can be simplified as `T`. /// For example, `std::vector<T>::value_type` can be simplified as `T`.
///
/// Thread safety: NOT thread-safe. Each compilation unit should have its own resolver.
/// The `resolved` cache persists across multiple resolve() calls on the same unit.
class TemplateResolver { class TemplateResolver {
public: public:
explicit TemplateResolver(clang::Sema& sema) : sema(sema) {} TemplateResolver(clang::Sema& sema) : sema(sema) {}
clang::QualType resolve(clang::QualType type); clang::QualType resolve(clang::QualType type);
@@ -30,7 +27,7 @@ public:
void resolve(clang::UnresolvedLookupExpr* expr); void resolve(clang::UnresolvedLookupExpr* expr);
// TODO: Use a clearer approach for resolving UnresolvedLookupExpr. // TODO: use a relative clear way to resolve `UnresolvedLookupExpr`.
void resolve(clang::UnresolvedUsingType* type); void resolve(clang::UnresolvedUsingType* type);
@@ -43,6 +40,10 @@ public:
/// Look up the name in the given nested name specifier. /// Look up the name in the given nested name specifier.
lookup_result lookup(const clang::NestedNameSpecifier* NNS, clang::DeclarationName name); lookup_result lookup(const clang::NestedNameSpecifier* NNS, clang::DeclarationName name);
lookup_result lookup(clang::DeclarationName name) {
return sema.getASTContext().getTranslationUnitDecl()->lookup(name);
}
lookup_result lookup(const clang::DependentNameType* type) { lookup_result lookup(const clang::DependentNameType* type) {
return lookup(type->getQualifier(), type->getIdentifier()); return lookup(type->getQualifier(), type->getIdentifier());
} }
@@ -53,7 +54,7 @@ public:
if(identifier) { if(identifier) {
return lookup(template_name.getQualifier(), identifier); return lookup(template_name.getQualifier(), identifier);
} else { } else {
/// TODO: Operators don't have an IdentifierInfo; need DeclarationName-based lookup. /// FIXME: Operators does't have a name.
return {}; return {};
} }
} }
@@ -63,7 +64,7 @@ public:
} }
lookup_result lookup(const clang::UnresolvedLookupExpr* expr) { lookup_result lookup(const clang::UnresolvedLookupExpr* expr) {
/// TODO: Only returns the first TemplateDecl; should handle overloaded lookups. /// FIXME:
for(auto decl: expr->decls()) { for(auto decl: expr->decls()) {
if(auto TD = llvm::dyn_cast<clang::TemplateDecl>(decl)) { if(auto TD = llvm::dyn_cast<clang::TemplateDecl>(decl)) {
return lookup_result(TD); return lookup_result(TD);
@@ -77,8 +78,8 @@ public:
return {}; return {};
} }
/// TODO: Implement dependent member expression lookup (e.g. `x.template foo<T>()`). /// TODO:
lookup_result lookup(const clang::CXXDependentScopeMemberExpr* expr) { lookup_result lookup(clang::CXXDependentScopeMemberExpr* expr) {
return {}; return {};
} }
@@ -86,20 +87,30 @@ public:
return lookup(decl->getQualifier(), decl->getDeclName()); return lookup(decl->getQualifier(), decl->getDeclName());
} }
lookup_result lookup(const clang::UnresolvedUsingTypenameDecl* decl) { lookup_result resolve(const clang::UnresolvedUsingTypenameDecl* decl) {
return lookup(decl->getQualifier(), decl->getDeclName()); return lookup(decl->getQualifier(), decl->getDeclName());
} }
#ifndef NDEBUG
static inline bool debug = false;
#endif
private: private:
clang::Sema& sema; clang::Sema& sema;
/// Cache of resolved dependent types, keyed by AST node pointer.
/// Shared across resolve() calls within the same TU for performance.
/// This is safe because a given AST node (DependentNameType*, etc.) has a
/// unique identity within the TU — the same pointer always refers to the same
/// syntactic occurrence. Different syntactic occurrences of the "same" type
/// have different AST node pointers.
llvm::DenseMap<const void*, clang::QualType> resolved; llvm::DenseMap<const void*, clang::QualType> resolved;
public:
auto source_manager() -> clang::SourceManager& {
return sema.getSourceManager();
}
auto lang_options() const -> const clang::LangOptions& {
return sema.getLangOpts();
}
auto ast_context() -> clang::ASTContext& {
return sema.getASTContext();
}
}; };
} // namespace clice } // namespace clice

View File

@@ -131,6 +131,33 @@ public:
} }
} }
} }
// if(auto module = unit.context().getCurrentNamedModule()) {
// auto keyword = module->DefinitionLoc;
// auto begin = TB.spelledTokenContaining(keyword);
// // assert(begin->kind() == clang::tok::identifier && begin->text(SM) == "module" &&
// // "Invalid module declaration");
//
// begin += 1;
// auto end = TB.spelledTokens(unit.file_id(keyword)).end();
//
// for(auto iter = begin; iter != end; ++iter) {
// if(iter->kind() == clang::tok::identifier) {
// if(auto next = iter + 1; next != end && (next->kind() == clang::tok::period ||
// next->kind() == clang::tok::colon)) {
// iter += 1;
// continue;
// }
//
// end = iter + 1;
// break;
// }
//
// std::unreachable();
// }
//
// handleModuleOccurrence(keyword, llvm::ArrayRef<clang::syntax::Token>(begin, end));
//}
} }
public: public:

View File

@@ -80,79 +80,27 @@ private:
struct SymbolModifiers { struct SymbolModifiers {
enum Kind : std::uint32_t { enum Kind : std::uint32_t {
/// Represents that the symbol is a declaration(e.g. function declaration). /// Represents that the symbol is a declaration(e.g. function declaration).
Declaration = 0, Declaration = 1u << 0,
/// Represents that the symbol is a definition(e.g. function definition). /// Represents that the symbol is a definition(e.g. function definition).
Definition = 1, Definition = 1u << 1,
/// Represents that the symbol is const modified(e.g. `const` variable). /// Represents that the symbol is const modified(e.g. `const` variable).
Const = 2, Const = 1u << 2,
/// Represents that the symbol is overloaded(e.g. overloaded functions and operators). /// Represents that the symbol is overloaded(e.g. overloaded functions and operators).
Overloaded = 3, Overloaded = 1u << 3,
/// Represents that the symbol is a part of type(e.g. `*` in `int*`). /// Represents that the symbol is a part of type(e.g. `*` in `int*`).
Typed = 4, Typed = 1u << 4,
/// Represents that the symbol is a template(e.g. class template or function template). /// Represents that the symbol is a template(e.g. class template or function template).
Templated = 5, Templated = 1u << 5,
/// Represents that the symbol is deprecated.
Deprecated = 6,
/// Represents that the symbol is deduced.
Deduced = 7,
/// Represents that the symbol is readonly.
Readonly = 8,
/// Represents that the symbol is static.
Static = 9,
/// Represents that the symbol is abstract.
Abstract = 10,
/// Represents that the symbol is virtual.
Virtual = 11,
/// Represents that the symbol is a dependent name.
DependentName = 12,
/// Represents that the symbol comes from the default library.
DefaultLibrary = 13,
/// Represents that the symbol is used through a mutable reference.
UsedAsMutableReference = 14,
/// Represents that the symbol is used through a mutable pointer.
UsedAsMutablePointer = 15,
/// Represents that the symbol is a constructor or destructor.
ConstructorOrDestructor = 16,
/// Represents that the symbol is user-defined.
UserDefined = 17,
/// Represents that the symbol is function-scoped.
FunctionScope = 18,
/// Represents that the symbol is class-scoped.
ClassScope = 19,
/// Represents that the symbol is file-scoped.
FileScope = 20,
/// Represents that the symbol is global-scoped.
GlobalScope = 21,
}; };
constexpr static std::uint32_t to_mask(Kind kind) {
return std::uint32_t(1) << static_cast<std::uint32_t>(kind);
}
constexpr SymbolModifiers() = default; constexpr SymbolModifiers() = default;
constexpr SymbolModifiers(Kind kind) : value(to_mask(kind)) {} constexpr SymbolModifiers(Kind kind) : value(static_cast<std::uint32_t>(kind)) {}
constexpr explicit SymbolModifiers(std::uint32_t bits) : value(bits) {} constexpr explicit SymbolModifiers(std::uint32_t bits) : value(bits) {}
@@ -161,7 +109,7 @@ struct SymbolModifiers {
} }
constexpr bool contains(Kind kind) const { constexpr bool contains(Kind kind) const {
return (value & to_mask(kind)) != 0; return (value & static_cast<std::uint32_t>(kind)) != 0;
} }
private: private:

Some files were not shown because too many files have changed in this diff Show More