Compare commits
3 Commits
folding-ra
...
raw-foldin
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
183b90d572 | ||
|
|
939ab6d0d4 | ||
|
|
e1202d2fa5 |
4
.github/workflows/main.yml
vendored
4
.github/workflows/main.yml
vendored
@@ -7,6 +7,10 @@ on:
|
||||
pull_request:
|
||||
branches: [main]
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
|
||||
|
||||
jobs:
|
||||
changes:
|
||||
if: ${{ !startsWith(github.ref, 'refs/tags/') }}
|
||||
|
||||
28
.github/workflows/test-cmake.yml
vendored
28
.github/workflows/test-cmake.yml
vendored
@@ -96,9 +96,20 @@ jobs:
|
||||
if-no-files-found: error
|
||||
retention-days: 1
|
||||
|
||||
- name: Run tests
|
||||
- name: Unit tests
|
||||
if: ${{ !matrix.build_only }}
|
||||
run: pixi run test ${{ matrix.build_type }}
|
||||
timeout-minutes: 5
|
||||
run: pixi run unit-test ${{ matrix.build_type }}
|
||||
|
||||
- name: Integration tests
|
||||
if: ${{ !matrix.build_only }}
|
||||
timeout-minutes: 20
|
||||
run: pixi run integration-test ${{ matrix.build_type }}
|
||||
|
||||
- name: Smoke tests
|
||||
if: ${{ !matrix.build_only }}
|
||||
timeout-minutes: 15
|
||||
run: pixi run smoke-test ${{ matrix.build_type }}
|
||||
|
||||
- name: Print cache stats and stop server
|
||||
if: always()
|
||||
@@ -146,5 +157,14 @@ jobs:
|
||||
if: runner.os != 'Windows'
|
||||
run: chmod +x build/${{ matrix.build_type }}/bin/*
|
||||
|
||||
- name: Run tests
|
||||
run: pixi run -e test-run test ${{ matrix.build_type }}
|
||||
- name: Unit tests
|
||||
timeout-minutes: 5
|
||||
run: pixi run -e test-run unit-test ${{ matrix.build_type }}
|
||||
|
||||
- name: Integration tests
|
||||
timeout-minutes: 20
|
||||
run: pixi run -e test-run integration-test ${{ matrix.build_type }}
|
||||
|
||||
- name: Smoke tests
|
||||
timeout-minutes: 10
|
||||
run: pixi run -e test-run smoke-test ${{ matrix.build_type }}
|
||||
|
||||
1
.gitignore
vendored
1
.gitignore
vendored
@@ -72,4 +72,3 @@ tests/unit/Local/
|
||||
.claude/*
|
||||
!.claude/CLAUDE.md
|
||||
!.claude/commands/
|
||||
openspec/
|
||||
|
||||
@@ -26,7 +26,7 @@
|
||||
#include "support/path_pool.h"
|
||||
#include "syntax/dependency_graph.h"
|
||||
|
||||
#include "kota/codec/json/serializer.h"
|
||||
#include "kota/codec/json/json.h"
|
||||
#include "kota/deco/deco.h"
|
||||
#include "llvm/Support/FileSystem.h"
|
||||
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-04-22
|
||||
185
openspec/changes/split-folding-range-pipeline/design.md
Normal file
185
openspec/changes/split-folding-range-pipeline/design.md
Normal file
@@ -0,0 +1,185 @@
|
||||
## Context
|
||||
|
||||
This change extracts decision `2` from `openspec/changes/explore-improve-folding-range-support/design.md` into a standalone proposal. The current folding implementation in `src/feature/folding_ranges.cpp` mixes three responsibilities in one path:
|
||||
|
||||
- discovering foldable structure from AST data
|
||||
- deciding which ranges survive deduplication and validation
|
||||
- shaping the final LSP response, including output metadata
|
||||
|
||||
That coupling makes the code harder to extend safely. Comment folding, directive-based collectors, capability-aware rendering, and range limiting all become riskier when collection and rendering rules share the same code path. The extracted proposal keeps scope narrower: it does not add new fold categories by itself, but it creates the architecture that later changes can build on without destabilizing existing structural folding.
|
||||
|
||||
The downloaded clangd reference confirms both the value and the limit of the upstream design. clangd has useful, tested folding behavior for brace bodies, comment blocks, contiguous `//` groups, and `lineFoldingOnly`, but its implementation largely emits protocol-shaped `FoldingRange` objects directly from collection code. In `SemanticSelection.cpp`, both the AST path and the pseudo-parser path build `FoldingRange` results directly, and the pseudo-parser applies rendering details such as delimiter trimming and `lineFoldingOnly` adjustments while collecting ranges. That is a good behavior reference, but it is not the architecture this extracted change should copy.
|
||||
|
||||
`clice` already has stronger ingredients for a real pipeline:
|
||||
|
||||
- `LocalSourceRange` gives us a main-file, half-open offset representation that is independent of LSP position encoding
|
||||
- directive metadata already captures information clangd does not expose well, including conditional-branch state, pragma regions, includes, imports, and macro references
|
||||
- the current tests are boundary-oriented, which makes them a good fit for validating raw spans before protocol rendering
|
||||
|
||||
The design therefore separates "what fold exists in the source" from "how that fold should be emitted to this client". clangd's tested boundary rules are still relevant, but they should become renderer policy and normalization rules rather than collector output format.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- Separate folding processing into collection, normalization, and rendering phases.
|
||||
- Preserve the existing AST structural folding categories already supported by `clice`.
|
||||
- Make ordering, deduplication, and boundary validation deterministic and testable.
|
||||
- Give later changes a stable extension point for comments, directives, and client-driven rendering options.
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- Add comment folding in this change.
|
||||
- Fix preprocessor branch-closing behavior in this change.
|
||||
- Add new fold categories such as macro definitions or include/import grouping.
|
||||
- Depend on initialize-time client capability plumbing being implemented first.
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. Use clangd as a behavior reference, not an architecture template
|
||||
|
||||
This change should borrow clangd's confirmed folding behavior where it is useful, especially around multiline comments, contiguous `//` comment groups, main-file-only filtering, and `lineFoldingOnly` boundary shaping. It should not copy clangd's habit of emitting protocol-shaped `FoldingRange` objects directly from collection logic.
|
||||
|
||||
Why:
|
||||
|
||||
- clangd's tests are valuable because they pin down tricky folding behavior around comments, macro boundaries, and line-only rendering
|
||||
- clangd's data flow is intentionally narrow and mixes collection with response shaping
|
||||
- `clice` already has richer file-local and directive metadata that supports a cleaner internal representation
|
||||
|
||||
Alternative considered:
|
||||
|
||||
- Treat clangd's direct `FoldingRange` construction as the architecture to reproduce. Rejected because it would preserve the same coupling this extracted change is meant to remove.
|
||||
|
||||
### 2. Introduce a raw internal folding-range model
|
||||
|
||||
Collectors should emit an internal `RawFoldingRange`-style structure instead of final LSP protocol objects. The raw model should preserve source locations, an internal category, and optional metadata hints that later phases may use.
|
||||
|
||||
The raw model should be shaped around file-local source structure, not LSP transport fields. At minimum it should carry:
|
||||
|
||||
- a main-file `LocalSourceRange` span using half-open byte offsets
|
||||
- an internal fold category such as namespace, record, access section, function body, comment block, comment group, conditional branch, pragma region, include group, or import group
|
||||
- the collector origin, such as AST, comment scanning, or directive metadata, so normalization has a stable tie-break and debugging surface
|
||||
- render hints for syntax-specific shaping, such as delimiter trimming, whether line-only folding should hide the final line, and an optional collapsed-text hint
|
||||
|
||||
In other words, the raw model should look closer to:
|
||||
|
||||
```cpp
|
||||
struct RawFoldRenderHint {
|
||||
std::uint8_t trim_start_bytes = 0;
|
||||
std::uint8_t trim_end_bytes = 0;
|
||||
bool hide_last_line_when_line_only = false;
|
||||
std::string collapsed_text_hint;
|
||||
};
|
||||
|
||||
struct RawFoldingRange {
|
||||
LocalSourceRange span;
|
||||
RawFoldCategory category;
|
||||
RawFoldOrigin origin;
|
||||
RawFoldRenderHint render;
|
||||
};
|
||||
```
|
||||
|
||||
The important design choice is that `span` represents the foldable source envelope in the main file, while renderer-specific trimming stays in `render` hints. For example:
|
||||
|
||||
- brace-based structural folds keep the full braced span and let the renderer trim interior boundaries
|
||||
- block comments keep the full `/* ... */` span and let the renderer decide whether to hide the closing delimiter or final line
|
||||
- contiguous `//` groups keep the full grouped span and let the renderer decide how much of the opening sentinel remains visible
|
||||
|
||||
Why:
|
||||
|
||||
- collectors should describe what was found, not how it will be serialized
|
||||
- `LocalSourceRange` is already the natural coordinate system for `clice`
|
||||
- public LSP kinds such as `comment`, `imports`, and `region` are too lossy to use as the internal category model
|
||||
- future comment and directive collectors can share the same pipeline contract
|
||||
- tests can validate collection independently from rendering
|
||||
|
||||
Alternatives considered:
|
||||
|
||||
- Continue emitting LSP ranges directly from collectors. Rejected because it keeps protocol concerns entangled with source discovery.
|
||||
- Make the raw model store already-trimmed visible interior spans instead of the full source envelope. Rejected because line-only rendering, collapsed text, and comment delimiter rules would still leak back into every collector.
|
||||
|
||||
### 3. Normalize ranges before rendering
|
||||
|
||||
All collected ranges should pass through a normalization step before any response is emitted. Normalization is responsible for deterministic ordering, duplicate removal, and rejection of degenerate or unmappable ranges.
|
||||
|
||||
Normalization should operate on raw spans and internal categories, not on final LSP fields. Its responsibilities include:
|
||||
|
||||
- deterministic ordering independent of collector traversal order
|
||||
- duplicate collapse for collectors that discover the same fold
|
||||
- invalid-range filtering after raw spans and render hints are reconciled
|
||||
- stable tie-breaking for overlapping ranges from different origins
|
||||
|
||||
Collectors may still reject obviously invalid inputs, such as non-main-file locations that cannot be mapped to `LocalSourceRange`, but normalization remains the phase that decides which collected folds survive to rendering.
|
||||
|
||||
Why:
|
||||
|
||||
- duplicate or invalid ranges are easier to reason about in one place than across many collectors
|
||||
- stable ordering reduces regression noise and makes range limiting predictable later
|
||||
- category-aware normalization preserves internal meaning until the renderer maps it to public kinds
|
||||
- normalization lets new collectors plug in without each collector re-implementing cleanup logic
|
||||
|
||||
Alternative considered:
|
||||
|
||||
- Let each collector manage its own sorting and duplicate suppression. Rejected because cross-collector interactions would still remain undefined.
|
||||
|
||||
### 4. Keep the current AST visitor as the first collector boundary
|
||||
|
||||
The initial extraction should preserve the current AST visitor as one collector feeding the raw model. This reduces refactor risk while still creating the new phase boundaries.
|
||||
|
||||
Why:
|
||||
|
||||
- the existing structural fold coverage is valuable and should not be rewritten unnecessarily
|
||||
- an adapter-style refactor is easier to verify against current tests than a full collector redesign
|
||||
|
||||
Alternative considered:
|
||||
|
||||
- Rewrite collection around a brand-new multi-source manager immediately. Rejected because it adds scope before the phase split is proven.
|
||||
|
||||
### 5. Move output shaping into a dedicated renderer
|
||||
|
||||
The renderer should translate normalized ranges into LSP folding ranges. Boundary shaping, output kinds, and optional metadata emission should live there, even if some options still use default values until later protocol plumbing exists.
|
||||
|
||||
Renderer input should be the normalized raw model plus a separate `FoldingRenderOptions` structure. The renderer then becomes responsible for:
|
||||
|
||||
- converting `LocalSourceRange` into protocol positions for the requested encoding
|
||||
- applying delimiter trimming and line-only adjustments
|
||||
- mapping internal categories to public LSP kinds
|
||||
- deciding whether collapsed text is emitted or suppressed
|
||||
- later applying deterministic `rangeLimit` trimming without changing collectors
|
||||
|
||||
This is the key point where `clice` should intentionally diverge from clangd. clangd threads `lineFoldingOnly` into collection and directly produces protocol objects. `clice` should keep those capability and transport decisions isolated in rendering so collectors remain stable as client support evolves.
|
||||
|
||||
Why:
|
||||
|
||||
- rendering rules are a separate concern from source discovery
|
||||
- later work on line-only output, metadata gating, or public kind mapping should not force collector rewrites
|
||||
- clangd-style line-only shaping is still supported, but as renderer policy rather than collector output
|
||||
- isolating rendering makes behavioral diffs easier to review
|
||||
|
||||
Alternative considered:
|
||||
|
||||
- Keep final boundary shaping next to the AST collector and only add a small helper for sorting. Rejected because it only moves a symptom, not the architectural problem.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [Refactoring the current path can accidentally change fold ordering] -> Mitigation: add deterministic-order assertions and compare outputs for existing structural fixtures.
|
||||
- [The raw model could become too abstract too early] -> Mitigation: keep the initial fields minimal and only include data already needed by current structural folds.
|
||||
- [Full-envelope raw spans plus render hints may feel less direct than storing already-trimmed ranges] -> Mitigation: use a small, explicit render-hint structure and validate brace/comment shaping with focused renderer tests.
|
||||
- [A renderer abstraction may appear premature before full capability plumbing exists] -> Mitigation: keep default render options aligned with current behavior and treat future options as extension points, not immediate scope.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
1. Introduce raw folding-range and render-option types behind the existing entrypoint.
|
||||
2. Convert the current AST-based collectors to emit raw ranges.
|
||||
3. Insert normalization between collection and response emission.
|
||||
4. Move LSP object construction into a dedicated renderer.
|
||||
5. Verify that existing structural folding fixtures still produce the expected ranges.
|
||||
|
||||
Rollback strategy:
|
||||
|
||||
- If the refactor destabilizes output, keep the new helper types but temporarily route the old direct-emission path until normalization and rendering regressions are resolved.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Whether public kind remapping should land in this extracted change or remain a follow-up proposal once the renderer boundary exists.
|
||||
- Whether collector origin should remain part of the long-term raw model after normalization policy stabilizes, or only exist temporarily as a debugging and tie-break aid.
|
||||
26
openspec/changes/split-folding-range-pipeline/proposal.md
Normal file
26
openspec/changes/split-folding-range-pipeline/proposal.md
Normal file
@@ -0,0 +1,26 @@
|
||||
## Why
|
||||
|
||||
`explore-improve-folding-range-support` combines several different concerns: upstream comparison work, baseline folding fixes, preprocessor extensions, and an internal refactor. The second design point in that change, splitting the folding-range pipeline into collection, normalization, and rendering, is the architectural slice that other work depends on and should be referenceable as its own proposal.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Extract the pipeline-splitting work from `explore-improve-folding-range-support` into a standalone change focused on folding-range architecture.
|
||||
- Introduce an internal raw folding-range model so collectors no longer emit final LSP objects directly.
|
||||
- Define a normalization phase that performs deterministic sorting, duplicate removal, and boundary validation before response generation.
|
||||
- Define a rendering phase that owns line/column shaping and optional metadata emission instead of mixing those concerns into collectors.
|
||||
- Preserve the current AST structural folding coverage while establishing extension points for future comment, directive, and capability-aware rendering work.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `folding-range-pipeline`: Provide a deterministic folding-range pipeline that separates collection, normalization, and rendering while preserving existing structural folds.
|
||||
|
||||
### Modified Capabilities
|
||||
- None.
|
||||
|
||||
## Impact
|
||||
|
||||
- `src/feature/folding_ranges.cpp` will be refactored around raw-range collection, normalization, and rendering boundaries.
|
||||
- Folding-related helper types may be introduced near the folding feature implementation.
|
||||
- `tests/unit/feature/folding_range_tests.cpp` will need regression coverage for structural folds and deterministic ordering.
|
||||
- `openspec/changes/explore-improve-folding-range-support/design.md` remains the source change from which this standalone proposal was extracted.
|
||||
@@ -0,0 +1,38 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Folding ranges are normalized before response emission
|
||||
The server SHALL convert collected folding candidates into a deterministic normalized set before emitting the folding range response.
|
||||
|
||||
#### Scenario: Duplicate candidates collapse to one emitted fold
|
||||
- **WHEN** multiple collectors produce the same folding candidate for the same source span and internal category
|
||||
- **THEN** the server MUST emit at most one folding range for that candidate
|
||||
|
||||
#### Scenario: Invalid candidates are dropped during normalization
|
||||
- **WHEN** a collected folding candidate does not span multiple lines or cannot be mapped back to the main file
|
||||
- **THEN** the server MUST omit that candidate from the emitted folding ranges
|
||||
|
||||
#### Scenario: Output ordering is deterministic
|
||||
- **WHEN** the same document is analyzed repeatedly without source changes
|
||||
- **THEN** the server MUST emit folding ranges in a deterministic order that does not depend on collector traversal order
|
||||
|
||||
### Requirement: Existing structural folding survives the pipeline split
|
||||
The server SHALL preserve the currently supported AST structural folding categories after collection, normalization, and rendering are separated.
|
||||
|
||||
#### Scenario: Supported structural regions remain foldable
|
||||
- **WHEN** a document contains a supported multi-line namespace, record, function body, parameter list, lambda body, initializer list, call argument list, or compound statement
|
||||
- **THEN** the server MUST still return a folding range for that region when its boundaries can be mapped to the main file
|
||||
|
||||
#### Scenario: Structural coverage is preserved through normalization
|
||||
- **WHEN** the document contains only currently supported AST-driven folding categories
|
||||
- **THEN** normalization and rendering MUST NOT remove a valid structural fold except when it is an exact duplicate or an invalid range
|
||||
|
||||
### Requirement: Rendering decisions are applied after normalization
|
||||
The server SHALL derive final LSP folding-range output from normalized internal ranges instead of requiring collectors to emit protocol-shaped results directly.
|
||||
|
||||
#### Scenario: Rendering options do not require collector changes
|
||||
- **WHEN** rendering rules change how line or metadata output is shaped for a normalized fold
|
||||
- **THEN** the server MUST apply that change in the rendering phase without requiring collector-specific logic changes
|
||||
|
||||
#### Scenario: Metadata hints remain optional until rendering
|
||||
- **WHEN** a collected or normalized fold carries optional kind or collapsed-text hints
|
||||
- **THEN** the renderer MUST decide whether to surface, transform, or suppress that metadata in the emitted LSP range
|
||||
16
openspec/changes/split-folding-range-pipeline/tasks.md
Normal file
16
openspec/changes/split-folding-range-pipeline/tasks.md
Normal file
@@ -0,0 +1,16 @@
|
||||
## 1. Raw Model and Collector Boundary
|
||||
|
||||
- [ ] 1.1 Introduce internal raw folding-range and render-option types while keeping the current folding entrypoint stable.
|
||||
- [ ] 1.2 Convert the existing AST structural folding path in `src/feature/folding_ranges.cpp` to emit raw ranges instead of final LSP ranges.
|
||||
- [ ] 1.3 Add regression fixtures or assertions that cover the currently supported structural fold categories before further refactoring.
|
||||
|
||||
## 2. Normalization and Rendering
|
||||
|
||||
- [ ] 2.1 Implement normalization for deterministic sorting, duplicate removal, and invalid-range filtering.
|
||||
- [ ] 2.2 Introduce a dedicated renderer that converts normalized ranges into final LSP folding-range objects.
|
||||
- [ ] 2.3 Keep default rendered output compatible with current structural behavior while exposing extension points for future collectors and render rules.
|
||||
|
||||
## 3. Verification
|
||||
|
||||
- [ ] 3.1 Compare pre-refactor and post-refactor outputs for the existing structural folding test cases.
|
||||
- [ ] 3.2 Run relevant folding-range unit tests and fix any ordering, deduplication, or boundary regressions introduced by the new pipeline.
|
||||
23
pixi.lock
generated
23
pixi.lock
generated
@@ -1078,6 +1078,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
linux-aarch64:
|
||||
- conda: https://conda.anaconda.org/conda-forge/linux-aarch64/_openmp_mutex-4.5-20_gnu.conda
|
||||
@@ -1152,6 +1153,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
osx-64:
|
||||
- conda: https://conda.anaconda.org/conda-forge/osx-64/bzip2-1.0.8-h500dc9f_9.conda
|
||||
@@ -1224,6 +1226,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
osx-arm64:
|
||||
- conda: https://conda.anaconda.org/conda-forge/osx-arm64/bzip2-1.0.8-hd037594_8.conda
|
||||
@@ -1289,6 +1292,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
win-64:
|
||||
- conda: https://conda.anaconda.org/conda-forge/win-64/bzip2-1.0.8-h0ad9c76_8.conda
|
||||
@@ -1343,6 +1347,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
format:
|
||||
channels:
|
||||
@@ -1704,6 +1709,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
linux-aarch64:
|
||||
- conda: https://conda.anaconda.org/conda-forge/linux-aarch64/_openmp_mutex-4.5-20_gnu.conda
|
||||
@@ -1782,6 +1788,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
osx-64:
|
||||
- conda: https://conda.anaconda.org/conda-forge/osx-64/bzip2-1.0.8-h500dc9f_9.conda
|
||||
@@ -1858,6 +1865,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
osx-arm64:
|
||||
- conda: https://conda.anaconda.org/conda-forge/osx-arm64/bzip2-1.0.8-hd037594_8.conda
|
||||
@@ -1926,6 +1934,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
win-64:
|
||||
- conda: https://conda.anaconda.org/conda-forge/win-64/bzip2-1.0.8-h0ad9c76_8.conda
|
||||
@@ -1982,6 +1991,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
test-run:
|
||||
channels:
|
||||
@@ -2025,6 +2035,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
linux-aarch64:
|
||||
- conda: https://conda.anaconda.org/conda-forge/linux-aarch64/_openmp_mutex-4.5-20_gnu.conda
|
||||
@@ -2058,6 +2069,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
osx-64:
|
||||
- conda: https://conda.anaconda.org/conda-forge/osx-64/bzip2-1.0.8-h500dc9f_9.conda
|
||||
@@ -2113,6 +2125,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
osx-arm64:
|
||||
- conda: https://conda.anaconda.org/conda-forge/osx-arm64/bzip2-1.0.8-hd037594_9.conda
|
||||
@@ -2168,6 +2181,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
win-64:
|
||||
- conda: https://conda.anaconda.org/conda-forge/win-64/bzip2-1.0.8-h0ad9c76_9.conda
|
||||
@@ -2199,6 +2213,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
win-arm64:
|
||||
- conda: https://conda.anaconda.org/conda-forge/win-arm64/bzip2-1.0.8-h50b96f5_9.conda
|
||||
@@ -2229,6 +2244,7 @@ environments:
|
||||
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
|
||||
packages:
|
||||
- conda: https://conda.anaconda.org/conda-forge/linux-64/_libgcc_mutex-0.1-conda_forge.tar.bz2
|
||||
@@ -7795,6 +7811,13 @@ packages:
|
||||
- coverage>=6.2 ; extra == 'testing'
|
||||
- hypothesis>=5.7.1 ; extra == 'testing'
|
||||
requires_python: '>=3.10'
|
||||
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
|
||||
name: pytest-timeout
|
||||
version: 2.4.0
|
||||
sha256: c42667e5cdadb151aeb5b26d114aff6bdf5a907f176a007a30b940d3d865b5c2
|
||||
requires_dist:
|
||||
- pytest>=7.0.0
|
||||
requires_python: '>=3.7'
|
||||
- conda: https://conda.anaconda.org/conda-forge/linux-64/python-3.14.2-h32b2ec7_100_cp314.conda
|
||||
build_number: 100
|
||||
sha256: a120fb2da4e4d51dd32918c149b04a08815fd2bd52099dad1334647984bb07f1
|
||||
|
||||
@@ -102,6 +102,7 @@ lld = "==20.1.8"
|
||||
[feature.test.pypi-dependencies]
|
||||
pytest = "*"
|
||||
pytest-asyncio = ">=1.1.0"
|
||||
pytest-timeout = "*"
|
||||
pygls = ">=2.0.0"
|
||||
lsprotocol = ">=2024.0.0"
|
||||
|
||||
@@ -165,8 +166,8 @@ cmd = './build/{{ type }}/bin/unit_tests --test-dir="./tests/data"'
|
||||
[feature.test.tasks.integration-test]
|
||||
args = [{ arg = "type", default = "RelWithDebInfo" }]
|
||||
cmd = """
|
||||
pytest -s --log-cli-level=INFO tests/integration \
|
||||
--executable=./build/{{ type }}/bin/clice
|
||||
pytest -s --log-cli-level=INFO --timeout=300 --timeout-method=thread \
|
||||
tests/integration --executable=./build/{{ type }}/bin/clice
|
||||
"""
|
||||
|
||||
[feature.test.tasks.smoke-test]
|
||||
|
||||
@@ -219,9 +219,10 @@ public:
|
||||
|
||||
auto CreateASTConsumer(clang::CompilerInstance& instance, llvm::StringRef file)
|
||||
-> std::unique_ptr<clang::ASTConsumer> final {
|
||||
return std::make_unique<ProxyASTConsumer>(
|
||||
WrapperFrontendAction::CreateASTConsumer(instance, file),
|
||||
unit);
|
||||
auto consumer = WrapperFrontendAction::CreateASTConsumer(instance, file);
|
||||
if(!consumer)
|
||||
return nullptr;
|
||||
return std::make_unique<ProxyASTConsumer>(std::move(consumer), unit);
|
||||
}
|
||||
|
||||
/// Make this public.
|
||||
|
||||
@@ -81,7 +81,8 @@ auto CompilationUnitRef::file_offset(clang::SourceLocation location) -> std::uin
|
||||
}
|
||||
|
||||
auto CompilationUnitRef::file_path(clang::FileID fid) -> llvm::StringRef {
|
||||
assert(fid.isValid() && "Invalid fid");
|
||||
if(!fid.isValid())
|
||||
return {};
|
||||
if(auto it = self->path_cache.find(fid); it != self->path_cache.end()) {
|
||||
return it->second;
|
||||
}
|
||||
|
||||
@@ -308,6 +308,10 @@ const clang::NamedDecl* decl_of_impl(const void* T) {
|
||||
}
|
||||
|
||||
auto decl_of(clang::QualType type) -> const clang::NamedDecl* {
|
||||
if(type.isNull()) {
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
// Strip type-sugar that wraps the underlying type without adding a decl
|
||||
// (e.g. ElaboratedType for "struct Foo" vs plain "Foo").
|
||||
if(auto ET = type->getAs<clang::ElaboratedType>()) {
|
||||
|
||||
@@ -490,6 +490,22 @@ kota::task<bool> Compiler::ensure_pch(Session& session,
|
||||
auto completion = std::make_shared<kota::event>();
|
||||
workspace.pch_cache[path_id].building = completion;
|
||||
|
||||
if(workspace.config.project.cache_dir.empty()) {
|
||||
LOG_WARN("PCH build skipped: cache_dir is not configured");
|
||||
workspace.pch_cache[path_id].building.reset();
|
||||
completion->set();
|
||||
co_return false;
|
||||
}
|
||||
|
||||
// Ensure the PCH cache directory exists.
|
||||
auto pch_dir = path::join(workspace.config.project.cache_dir, "cache", "pch");
|
||||
if(auto ec = llvm::sys::fs::create_directories(pch_dir)) {
|
||||
LOG_WARN("Cannot create PCH cache dir {}: {}", pch_dir, ec.message());
|
||||
workspace.pch_cache[path_id].building.reset();
|
||||
completion->set();
|
||||
co_return false;
|
||||
}
|
||||
|
||||
// Build a new PCH via stateless worker.
|
||||
worker::BuildParams bp;
|
||||
bp.kind = worker::BuildKind::BuildPCH;
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
#include "syntax/completion.h"
|
||||
|
||||
#include "kota/async/async.h"
|
||||
#include "kota/codec/raw_value.h"
|
||||
#include "kota/codec/json/json.h"
|
||||
#include "kota/ipc/codec/json.h"
|
||||
#include "kota/ipc/lsp/protocol.h"
|
||||
#include "kota/ipc/peer.h"
|
||||
|
||||
@@ -6,8 +6,9 @@
|
||||
#include "support/glob_pattern.h"
|
||||
#include "support/logging.h"
|
||||
|
||||
#include "kota/async/io/system.h"
|
||||
#include "kota/codec/json/json.h"
|
||||
#include "kota/codec/toml.h"
|
||||
#include "kota/codec/toml/toml.h"
|
||||
#include "llvm/Support/FileSystem.h"
|
||||
#include "llvm/Support/Path.h"
|
||||
#include "llvm/Support/Process.h"
|
||||
@@ -65,8 +66,10 @@ void Config::apply_defaults(llvm::StringRef workspace_root) {
|
||||
|
||||
if(p.stateful_worker_count == 0)
|
||||
p.stateful_worker_count = 2;
|
||||
if(p.stateless_worker_count == 0)
|
||||
p.stateless_worker_count = 3;
|
||||
if(p.stateless_worker_count == 0) {
|
||||
auto cores = kota::sys::parallelism();
|
||||
p.stateless_worker_count = std::max(cores / 2, 2u);
|
||||
}
|
||||
if(p.worker_memory_limit == 0)
|
||||
p.worker_memory_limit = 4ULL * 1024 * 1024 * 1024; // 4GB
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
#include "server/indexer.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <string>
|
||||
#include <variant>
|
||||
#include <vector>
|
||||
@@ -624,6 +625,23 @@ void Indexer::enqueue(std::uint32_t server_path_id) {
|
||||
index_queue.push_back(server_path_id);
|
||||
}
|
||||
|
||||
void Indexer::pause_indexing() {
|
||||
++pause_depth;
|
||||
if(pause_depth == 1) {
|
||||
resume_event.reset();
|
||||
LOG_DEBUG("Background indexing paused");
|
||||
}
|
||||
}
|
||||
|
||||
void Indexer::resume_indexing() {
|
||||
if(pause_depth > 0)
|
||||
--pause_depth;
|
||||
if(pause_depth == 0) {
|
||||
resume_event.set();
|
||||
LOG_DEBUG("Background indexing resumed");
|
||||
}
|
||||
}
|
||||
|
||||
void Indexer::schedule() {
|
||||
if(!*workspace.config.project.enable_indexing || indexing_active || indexing_scheduled)
|
||||
return;
|
||||
@@ -636,6 +654,76 @@ void Indexer::schedule() {
|
||||
loop.schedule(run_background_indexing());
|
||||
}
|
||||
|
||||
kota::task<> Indexer::index_one(std::uint32_t server_path_id) {
|
||||
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
|
||||
|
||||
if(sessions.contains(server_path_id))
|
||||
co_return;
|
||||
|
||||
if(!need_update(file_path))
|
||||
co_return;
|
||||
|
||||
// For module interface units, compile their PCM (and transitive deps)
|
||||
// first so the stateless worker has the artifacts it needs.
|
||||
if(workspace.compile_graph && workspace.path_to_module.contains(server_path_id)) {
|
||||
co_await workspace.compile_graph->compile(server_path_id);
|
||||
}
|
||||
|
||||
worker::BuildParams params;
|
||||
params.kind = worker::BuildKind::Index;
|
||||
params.file = file_path;
|
||||
if(!compiler.fill_compile_args(file_path, params.directory, params.arguments, nullptr))
|
||||
co_return;
|
||||
|
||||
workspace.fill_pcm_deps(params.pcms);
|
||||
|
||||
LOG_INFO("Background indexing: {}", file_path);
|
||||
|
||||
auto result = co_await pool.send_stateless(params);
|
||||
if(result.has_value() && result.value().success && !result.value().tu_index_data.empty()) {
|
||||
LOG_INFO("Background indexing got TUIndex for {}: {} bytes",
|
||||
file_path,
|
||||
result.value().tu_index_data.size());
|
||||
merge(result.value().tu_index_data.data(), result.value().tu_index_data.size());
|
||||
} else if(result.has_value() && !result.value().success) {
|
||||
LOG_WARN("Background index failed for {}: {}", file_path, result.value().error);
|
||||
} else if(result.has_value() && result.value().tu_index_data.empty()) {
|
||||
LOG_WARN("Background index returned empty TUIndex for {}", file_path);
|
||||
} else {
|
||||
LOG_WARN("Background index IPC error for {}: {}", file_path, result.error().message);
|
||||
}
|
||||
}
|
||||
|
||||
kota::task<> Indexer::monitor_resources(std::uint32_t generation) {
|
||||
while(generation == monitor_generation) {
|
||||
co_await kota::sleep(std::chrono::milliseconds(3000), loop);
|
||||
|
||||
if(generation != monitor_generation)
|
||||
break;
|
||||
|
||||
auto mem = kota::sys::memory();
|
||||
if(mem.total == 0)
|
||||
continue;
|
||||
|
||||
// Respect cgroup/container limits when present.
|
||||
auto effective_total =
|
||||
(mem.constrained > 0 && mem.constrained < mem.total) ? mem.constrained : mem.total;
|
||||
auto ratio = static_cast<double>(mem.available) / static_cast<double>(effective_total);
|
||||
|
||||
if(ratio < 0.15 && max_concurrent > 1) {
|
||||
--max_concurrent;
|
||||
LOG_INFO("Index concurrency -> {} (memory pressure: {:.0f}% available)",
|
||||
max_concurrent,
|
||||
ratio * 100);
|
||||
} else if(ratio > 0.30 && max_concurrent < baseline_concurrent) {
|
||||
++max_concurrent;
|
||||
LOG_DEBUG("Index concurrency -> {} (memory OK: {:.0f}% available)",
|
||||
max_concurrent,
|
||||
ratio * 100);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
kota::task<> Indexer::run_background_indexing() {
|
||||
if(index_idle_timer) {
|
||||
co_await index_idle_timer->wait();
|
||||
@@ -648,48 +736,88 @@ kota::task<> Indexer::run_background_indexing() {
|
||||
}
|
||||
|
||||
indexing_active = true;
|
||||
std::size_t processed = 0;
|
||||
++monitor_generation;
|
||||
loop.schedule(monitor_resources(monitor_generation));
|
||||
|
||||
while(index_queue_pos < index_queue.size()) {
|
||||
auto server_path_id = index_queue[index_queue_pos];
|
||||
index_queue_pos++;
|
||||
// Put module interface units first so their PCMs are built before
|
||||
// non-module files that might import them.
|
||||
std::stable_partition(
|
||||
index_queue.begin() + index_queue_pos,
|
||||
index_queue.end(),
|
||||
[this](std::uint32_t id) { return workspace.path_to_module.contains(id); });
|
||||
|
||||
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
|
||||
auto batch = index_queue.size() - index_queue_pos;
|
||||
std::size_t dispatched = 0;
|
||||
std::size_t completed = 0;
|
||||
finished = 0;
|
||||
|
||||
if(sessions.contains(server_path_id))
|
||||
continue;
|
||||
|
||||
if(!need_update(file_path))
|
||||
continue;
|
||||
|
||||
worker::BuildParams params;
|
||||
params.kind = worker::BuildKind::Index;
|
||||
params.file = file_path;
|
||||
if(!compiler.fill_compile_args(file_path, params.directory, params.arguments, nullptr))
|
||||
continue;
|
||||
|
||||
workspace.fill_pcm_deps(params.pcms);
|
||||
|
||||
LOG_INFO("Background indexing: {}", file_path);
|
||||
|
||||
auto result = co_await pool.send_stateless(params);
|
||||
if(result.has_value() && result.value().success && !result.value().tu_index_data.empty()) {
|
||||
LOG_INFO("Background indexing got TUIndex for {}: {} bytes",
|
||||
file_path,
|
||||
result.value().tu_index_data.size());
|
||||
merge(result.value().tu_index_data.data(), result.value().tu_index_data.size());
|
||||
++processed;
|
||||
} else if(result.has_value() && !result.value().success) {
|
||||
LOG_WARN("Background index failed for {}: {}", file_path, result.value().error);
|
||||
} else if(result.has_value() && result.value().tu_index_data.empty()) {
|
||||
LOG_WARN("Background index returned empty TUIndex for {}", file_path);
|
||||
// Progress reporting via LSP $/progress.
|
||||
std::optional<lsp::ProgressReporter<kota::ipc::JsonPeer>> progress;
|
||||
if(peer) {
|
||||
progress.emplace(*peer, protocol::ProgressToken(std::string("clice/backgroundIndex")));
|
||||
auto create_result = co_await progress->create();
|
||||
if(!create_result.has_error()) {
|
||||
progress->begin("Indexing", std::format("0/{} files", batch), 0);
|
||||
} else {
|
||||
LOG_WARN("Background index IPC error for {}: {}", file_path, result.error().message);
|
||||
progress.reset();
|
||||
}
|
||||
}
|
||||
|
||||
while(index_queue_pos < index_queue.size() || inflight > 0) {
|
||||
// Dispatch new tasks up to max_concurrent.
|
||||
while(index_queue_pos < index_queue.size() && inflight < max_concurrent) {
|
||||
// Wait if paused by a user request.
|
||||
if(pause_depth > 0) {
|
||||
co_await resume_event.wait();
|
||||
}
|
||||
|
||||
auto server_path_id = index_queue[index_queue_pos++];
|
||||
|
||||
// Quick pre-filter: skip open files and fresh files without
|
||||
// consuming a concurrency slot.
|
||||
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
|
||||
if(sessions.contains(server_path_id) || !need_update(file_path)) {
|
||||
++completed;
|
||||
continue;
|
||||
}
|
||||
|
||||
++inflight;
|
||||
++dispatched;
|
||||
|
||||
// Launch the index task. On completion it decrements
|
||||
// inflight, bumps finished, and signals the event.
|
||||
loop.schedule([](Indexer* self, std::uint32_t id, kota::event& done) -> kota::task<> {
|
||||
co_await self->index_one(id);
|
||||
--self->inflight;
|
||||
++self->finished;
|
||||
done.set();
|
||||
}(this, server_path_id, completion_event));
|
||||
}
|
||||
|
||||
if(inflight == 0)
|
||||
break;
|
||||
|
||||
// Wait for at least one task to finish.
|
||||
co_await completion_event.wait();
|
||||
completion_event.reset();
|
||||
|
||||
// Drain all completions that occurred since last wake.
|
||||
completed += std::exchange(finished, 0);
|
||||
|
||||
// Report progress.
|
||||
if(progress) {
|
||||
auto pct = batch > 0 ? static_cast<std::uint32_t>(completed * 100 / batch) : 100;
|
||||
progress->report(std::format("{}/{} files", completed, batch), pct);
|
||||
}
|
||||
}
|
||||
|
||||
if(progress) {
|
||||
progress->end(std::format("Indexed {} files", dispatched));
|
||||
}
|
||||
|
||||
indexing_active = false;
|
||||
LOG_INFO("Background indexing complete: {} files processed", processed);
|
||||
++monitor_generation; // Stop the monitor coroutine.
|
||||
LOG_INFO("Background indexing complete: {} files dispatched", dispatched);
|
||||
save(workspace.config.project.index_dir);
|
||||
}
|
||||
|
||||
|
||||
@@ -12,7 +12,9 @@
|
||||
#include "server/workspace.h"
|
||||
|
||||
#include "kota/async/async.h"
|
||||
#include "kota/ipc/codec/json.h"
|
||||
#include "kota/ipc/lsp/position.h"
|
||||
#include "kota/ipc/lsp/progress.h"
|
||||
#include "kota/ipc/lsp/protocol.h"
|
||||
#include "llvm/ADT/DenseMap.h"
|
||||
#include "llvm/ADT/SmallVector.h"
|
||||
@@ -62,6 +64,47 @@ public:
|
||||
loop(loop), workspace(workspace), sessions(sessions), pool(pool), compiler(compiler),
|
||||
is_file_open(std::move(is_file_open)) {}
|
||||
|
||||
/// Set the LSP peer for progress reporting. Must be called before
|
||||
/// schedule() if progress notifications are desired.
|
||||
void set_peer(kota::ipc::JsonPeer* p) {
|
||||
peer = p;
|
||||
}
|
||||
|
||||
/// Temporarily pause background indexing to give priority to user
|
||||
/// requests. Indexing tasks already dispatched to workers continue,
|
||||
/// but no new tasks will be sent until resume_indexing() is called.
|
||||
void pause_indexing();
|
||||
|
||||
/// Resume background indexing after a pause.
|
||||
void resume_indexing();
|
||||
|
||||
/// RAII guard that pauses indexing for its lifetime.
|
||||
struct [[nodiscard]] ScopedPause {
|
||||
Indexer& indexer;
|
||||
|
||||
explicit ScopedPause(Indexer& idx) : indexer(idx) {
|
||||
indexer.pause_indexing();
|
||||
}
|
||||
|
||||
~ScopedPause() {
|
||||
indexer.resume_indexing();
|
||||
}
|
||||
|
||||
ScopedPause(const ScopedPause&) = delete;
|
||||
ScopedPause& operator=(const ScopedPause&) = delete;
|
||||
};
|
||||
|
||||
ScopedPause scoped_pause() {
|
||||
return ScopedPause{*this};
|
||||
}
|
||||
|
||||
/// Set the maximum number of concurrent index tasks.
|
||||
/// Also sets the baseline that dynamic adjustment will restore to.
|
||||
void set_max_concurrency(std::size_t n) {
|
||||
max_concurrent = std::max<std::size_t>(n, 1);
|
||||
baseline_concurrent = max_concurrent;
|
||||
}
|
||||
|
||||
/// Add a file to the background indexing queue.
|
||||
void enqueue(std::uint32_t server_path_id);
|
||||
|
||||
@@ -175,6 +218,9 @@ private:
|
||||
/// server-path-id-keyed sessions map to project-level path_ids.
|
||||
std::function<bool(std::uint32_t)> is_file_open;
|
||||
|
||||
/// LSP peer for progress reporting (optional, not owned).
|
||||
kota::ipc::JsonPeer* peer = nullptr;
|
||||
|
||||
/// Background indexing queue and scheduling state.
|
||||
std::vector<std::uint32_t> index_queue;
|
||||
std::size_t index_queue_pos = 0;
|
||||
@@ -182,7 +228,30 @@ private:
|
||||
bool indexing_scheduled = false;
|
||||
std::shared_ptr<kota::timer> index_idle_timer;
|
||||
|
||||
/// Concurrency control for background indexing.
|
||||
std::size_t max_concurrent = 2;
|
||||
std::size_t baseline_concurrent = 2;
|
||||
std::size_t inflight = 0;
|
||||
std::size_t finished = 0; ///< Incremented by each completed dispatch task.
|
||||
|
||||
/// Pause/resume: when paused, new index tasks wait on this event.
|
||||
/// Uses a counter so nested pause/resume pairs work correctly.
|
||||
std::size_t pause_depth = 0;
|
||||
kota::event resume_event{true};
|
||||
|
||||
/// Completion event — signalled by each finished dispatch task so the
|
||||
/// main loop can wake up. Must be a member (not local to the coroutine)
|
||||
/// because inflight tasks capture it by reference and may outlive the
|
||||
/// coroutine frame during server shutdown.
|
||||
kota::event completion_event;
|
||||
|
||||
/// Generation counter — incremented each run so a stale monitor_resources
|
||||
/// coroutine can detect that its owning run has ended.
|
||||
std::uint32_t monitor_generation = 0;
|
||||
|
||||
kota::task<> run_background_indexing();
|
||||
kota::task<> index_one(std::uint32_t server_path_id);
|
||||
kota::task<> monitor_resources(std::uint32_t generation);
|
||||
};
|
||||
|
||||
} // namespace clice
|
||||
|
||||
@@ -56,9 +56,9 @@ MasterServer::MasterServer(kota::event_loop& loop,
|
||||
|
||||
MasterServer::~MasterServer() = default;
|
||||
|
||||
kota::task<> MasterServer::load_workspace() {
|
||||
void MasterServer::load_workspace() {
|
||||
if(workspace_root.empty())
|
||||
co_return;
|
||||
return;
|
||||
|
||||
auto& cfg = workspace.config.project;
|
||||
|
||||
@@ -125,7 +125,7 @@ kota::task<> MasterServer::load_workspace() {
|
||||
|
||||
if(cdb_path.empty()) {
|
||||
LOG_WARN("No compile_commands.json found in workspace {}", workspace_root);
|
||||
co_return;
|
||||
return;
|
||||
}
|
||||
|
||||
auto count = workspace.cdb.load(cdb_path);
|
||||
@@ -331,7 +331,10 @@ void MasterServer::register_handlers() {
|
||||
indexer.schedule();
|
||||
};
|
||||
|
||||
loop.schedule(load_workspace());
|
||||
indexer.set_peer(&peer);
|
||||
indexer.set_max_concurrency(cfg.stateless_worker_count.value);
|
||||
|
||||
load_workspace();
|
||||
});
|
||||
|
||||
peer.on_request(
|
||||
@@ -670,28 +673,33 @@ void MasterServer::register_handlers() {
|
||||
|
||||
/// Feature requests — stateless forwarding.
|
||||
|
||||
peer.on_request([this](RequestContext& ctx,
|
||||
const protocol::CompletionParams& params) -> RawResult {
|
||||
auto path = uri_to_path(params.text_document_position_params.text_document.uri);
|
||||
auto path_id = workspace.path_pool.intern(path);
|
||||
auto sit = sessions.find(path_id);
|
||||
if(sit == sessions.end())
|
||||
co_return serde_raw{"null"};
|
||||
co_return co_await compiler.handle_completion(params.text_document_position_params.position,
|
||||
sit->second);
|
||||
});
|
||||
|
||||
peer.on_request(
|
||||
[this](RequestContext& ctx, const protocol::SignatureHelpParams& params) -> RawResult {
|
||||
[this](RequestContext& ctx, const protocol::CompletionParams& params) -> RawResult {
|
||||
auto path = uri_to_path(params.text_document_position_params.text_document.uri);
|
||||
auto path_id = workspace.path_pool.intern(path);
|
||||
auto sit = sessions.find(path_id);
|
||||
if(sit == sessions.end())
|
||||
co_return serde_raw{"null"};
|
||||
co_return co_await compiler.forward_build(worker::BuildKind::SignatureHelp,
|
||||
auto pause = indexer.scoped_pause();
|
||||
auto result =
|
||||
co_await compiler.handle_completion(params.text_document_position_params.position,
|
||||
sit->second);
|
||||
co_return std::move(result);
|
||||
});
|
||||
|
||||
peer.on_request([this](RequestContext& ctx,
|
||||
const protocol::SignatureHelpParams& params) -> RawResult {
|
||||
auto path = uri_to_path(params.text_document_position_params.text_document.uri);
|
||||
auto path_id = workspace.path_pool.intern(path);
|
||||
auto sit = sessions.find(path_id);
|
||||
if(sit == sessions.end())
|
||||
co_return serde_raw{"null"};
|
||||
auto pause = indexer.scoped_pause();
|
||||
auto result = co_await compiler.forward_build(worker::BuildKind::SignatureHelp,
|
||||
params.text_document_position_params.position,
|
||||
sit->second);
|
||||
});
|
||||
co_return std::move(result);
|
||||
});
|
||||
|
||||
/// Hierarchy queries — index-based.
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
#include "server/workspace.h"
|
||||
|
||||
#include "kota/async/async.h"
|
||||
#include "kota/codec/raw_value.h"
|
||||
#include "kota/codec/json/json.h"
|
||||
#include "kota/ipc/peer.h"
|
||||
#include "llvm/ADT/DenseMap.h"
|
||||
|
||||
@@ -73,7 +73,7 @@ private:
|
||||
std::string session_log_dir;
|
||||
std::string init_options_json; ///< Raw JSON from initializationOptions, consumed once.
|
||||
|
||||
kota::task<> load_workspace();
|
||||
void load_workspace();
|
||||
|
||||
using RawResult = kota::task<kota::codec::RawValue, kota::ipc::Error>;
|
||||
};
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
|
||||
#include "syntax/token.h"
|
||||
|
||||
#include "kota/codec/raw_value.h"
|
||||
#include "kota/codec/json/json.h"
|
||||
#include "kota/ipc/lsp/protocol.h"
|
||||
#include "kota/ipc/protocol.h"
|
||||
|
||||
|
||||
@@ -15,6 +15,22 @@
|
||||
|
||||
namespace clice {
|
||||
|
||||
/// RAII guard that lowers the current process's scheduling priority and
|
||||
/// restores it on destruction.
|
||||
struct ScopedNice {
|
||||
int saved;
|
||||
|
||||
explicit ScopedNice(int increment = 10) {
|
||||
auto p = kota::sys::priority();
|
||||
saved = p ? *p : 0;
|
||||
kota::sys::set_priority(saved + increment);
|
||||
}
|
||||
|
||||
~ScopedNice() {
|
||||
kota::sys::set_priority(saved);
|
||||
}
|
||||
};
|
||||
|
||||
using kota::ipc::RequestResult;
|
||||
using RequestContext = kota::ipc::BincodePeer::RequestContext;
|
||||
|
||||
@@ -283,7 +299,10 @@ int run_stateless_worker_mode(const std::string& worker_name, const std::string&
|
||||
switch(params.kind) {
|
||||
case K::BuildPCH: return handle_build_pch(params);
|
||||
case K::BuildPCM: return handle_build_pcm(params);
|
||||
case K::Index: return handle_index(params);
|
||||
case K::Index: {
|
||||
ScopedNice guard;
|
||||
return handle_index(params);
|
||||
}
|
||||
case K::Completion: return handle_completion(params);
|
||||
case K::SignatureHelp: return handle_signature_help(params);
|
||||
}
|
||||
|
||||
@@ -8,8 +8,7 @@
|
||||
|
||||
#include "compile/compilation.h"
|
||||
|
||||
#include "kota/codec/json/serializer.h"
|
||||
#include "kota/codec/raw_value.h"
|
||||
#include "kota/codec/json/json.h"
|
||||
#include "kota/ipc/codec/json.h"
|
||||
|
||||
namespace clice {
|
||||
|
||||
@@ -13,14 +13,13 @@ namespace {
|
||||
|
||||
/// Coroutine that drains a worker's stderr pipe.
|
||||
/// Workers write their own log files, so this only captures unexpected output
|
||||
/// (crash stacktraces, assertion failures, etc.) that bypasses spdlog.
|
||||
/// (crash stacktraces, assertion failures, sanitizer reports, etc.).
|
||||
kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
|
||||
std::string buffer;
|
||||
while(true) {
|
||||
auto result = co_await stderr_pipe.read();
|
||||
if(!result.has_value()) {
|
||||
if(!result.has_value())
|
||||
break;
|
||||
}
|
||||
auto& chunk = result.value();
|
||||
if(chunk.empty())
|
||||
break;
|
||||
@@ -34,7 +33,7 @@ kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
|
||||
break;
|
||||
auto line = buffer.substr(pos, nl - pos);
|
||||
if(!line.empty()) {
|
||||
LOG_DEBUG("{} {}", prefix, line);
|
||||
LOG_WARN("{} {}", prefix, line);
|
||||
}
|
||||
pos = nl + 1;
|
||||
}
|
||||
@@ -42,7 +41,7 @@ kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
|
||||
}
|
||||
|
||||
if(!buffer.empty()) {
|
||||
LOG_DEBUG("{} {}", prefix, buffer);
|
||||
LOG_WARN("{} {}", prefix, buffer);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -108,24 +107,29 @@ bool WorkerPool::spawn_worker(const std::string& self_path,
|
||||
});
|
||||
|
||||
auto& w = workers.back();
|
||||
w.alive = true;
|
||||
++alive_count_;
|
||||
loop.schedule(w.peer->run());
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool WorkerPool::start(const WorkerPoolOptions& options) {
|
||||
options_ = options;
|
||||
log_dir_ = options.log_dir;
|
||||
|
||||
for(std::uint32_t i = 0; i < options.stateless_count; ++i) {
|
||||
if(!spawn_worker(options.self_path, false, 0)) {
|
||||
return false;
|
||||
}
|
||||
loop.schedule(monitor_worker(stateless_workers.size() - 1, false));
|
||||
}
|
||||
|
||||
for(std::uint32_t i = 0; i < options.stateful_count; ++i) {
|
||||
if(!spawn_worker(options.self_path, true, options.worker_memory_limit)) {
|
||||
return false;
|
||||
}
|
||||
loop.schedule(monitor_worker(stateful_workers.size() - 1, true));
|
||||
}
|
||||
|
||||
// Register evicted notification handler for each stateful worker
|
||||
@@ -145,29 +149,24 @@ bool WorkerPool::start(const WorkerPoolOptions& options) {
|
||||
|
||||
kota::task<> WorkerPool::stop() {
|
||||
LOG_INFO("WorkerPool stopping...");
|
||||
shutting_down_ = true;
|
||||
|
||||
// Close output pipes to signal workers to exit gracefully
|
||||
for(auto& w: stateless_workers) {
|
||||
// Close output pipes to signal workers to exit gracefully.
|
||||
for(auto& w: stateless_workers)
|
||||
w.peer->close_output();
|
||||
}
|
||||
for(auto& w: stateful_workers) {
|
||||
for(auto& w: stateful_workers)
|
||||
w.peer->close_output();
|
||||
}
|
||||
|
||||
// Send SIGTERM to all workers
|
||||
for(auto& w: stateless_workers) {
|
||||
// Send SIGTERM. monitor_worker coroutines handle the wait.
|
||||
for(auto& w: stateless_workers)
|
||||
w.proc.kill(SIGTERM);
|
||||
}
|
||||
for(auto& w: stateful_workers) {
|
||||
for(auto& w: stateful_workers)
|
||||
w.proc.kill(SIGTERM);
|
||||
}
|
||||
|
||||
// Wait for all worker processes to exit
|
||||
for(auto& w: stateless_workers) {
|
||||
co_await w.proc.wait();
|
||||
}
|
||||
for(auto& w: stateful_workers) {
|
||||
co_await w.proc.wait();
|
||||
// Wait until all monitor_worker coroutines have finished.
|
||||
if(alive_count_ > 0) {
|
||||
all_exited_.reset();
|
||||
co_await all_exited_.wait();
|
||||
}
|
||||
|
||||
LOG_INFO("WorkerPool stopped");
|
||||
@@ -198,7 +197,10 @@ std::size_t WorkerPool::assign_worker(std::uint32_t path_id) {
|
||||
std::size_t WorkerPool::pick_least_loaded() {
|
||||
std::size_t best = 0;
|
||||
for(std::size_t i = 1; i < stateful_workers.size(); ++i) {
|
||||
if(stateful_workers[i].owned_documents < stateful_workers[best].owned_documents) {
|
||||
if(!stateful_workers[i].alive)
|
||||
continue;
|
||||
if(!stateful_workers[best].alive ||
|
||||
stateful_workers[i].owned_documents < stateful_workers[best].owned_documents) {
|
||||
best = i;
|
||||
}
|
||||
}
|
||||
@@ -233,4 +235,127 @@ void WorkerPool::clear_owner(std::size_t worker_index) {
|
||||
}
|
||||
}
|
||||
|
||||
kota::task<> WorkerPool::monitor_worker(std::size_t index, bool stateful) {
|
||||
auto& workers = stateful ? stateful_workers : stateless_workers;
|
||||
auto& w = workers[index];
|
||||
auto name = std::string(stateful ? "SF-" : "SL-") + std::to_string(index);
|
||||
|
||||
auto result = co_await w.proc.wait();
|
||||
w.alive = false;
|
||||
--alive_count_;
|
||||
|
||||
if(shutting_down_) {
|
||||
if(alive_count_ == 0)
|
||||
all_exited_.set();
|
||||
co_return;
|
||||
}
|
||||
|
||||
if(result.has_value()) {
|
||||
auto& exit = result.value();
|
||||
if(exit.term_signal != 0) {
|
||||
LOG_ERROR("Worker {} killed by signal {} (restarts: {})",
|
||||
name,
|
||||
exit.term_signal,
|
||||
w.restart_count);
|
||||
} else {
|
||||
LOG_ERROR("Worker {} exited with code {} (restarts: {})",
|
||||
name,
|
||||
exit.status,
|
||||
w.restart_count);
|
||||
}
|
||||
} else {
|
||||
LOG_ERROR("Worker {} lost: {} (restarts: {})",
|
||||
name,
|
||||
result.error().message(),
|
||||
w.restart_count);
|
||||
}
|
||||
|
||||
if(stateful)
|
||||
clear_owner(index);
|
||||
|
||||
constexpr unsigned max_restarts = 5;
|
||||
if(w.restart_count >= max_restarts) {
|
||||
LOG_ERROR("Worker {} exceeded max restarts ({}), giving up", name, max_restarts);
|
||||
co_return;
|
||||
}
|
||||
|
||||
if(!respawn_worker(index, stateful)) {
|
||||
LOG_ERROR("Worker {} respawn failed", name);
|
||||
}
|
||||
}
|
||||
|
||||
bool WorkerPool::respawn_worker(std::size_t index, bool stateful) {
|
||||
auto& workers = stateful ? stateful_workers : stateless_workers;
|
||||
auto old_restart_count = workers[index].restart_count + 1;
|
||||
auto worker_name = std::string(stateful ? "SF-" : "SL-") + std::to_string(index);
|
||||
|
||||
// Close the old peer and retire it so its coroutines (run/write_loop)
|
||||
// can finish naturally before the object is destroyed.
|
||||
if(workers[index].peer) {
|
||||
workers[index].peer->close();
|
||||
retired_peers.push_back(std::move(workers[index].peer));
|
||||
}
|
||||
|
||||
kota::process::options opts;
|
||||
opts.file = options_.self_path;
|
||||
if(stateful) {
|
||||
opts.args = {options_.self_path,
|
||||
"--mode",
|
||||
"stateful-worker",
|
||||
"--worker-memory-limit",
|
||||
std::to_string(options_.worker_memory_limit)};
|
||||
} else {
|
||||
opts.args = {options_.self_path, "--mode", "stateless-worker"};
|
||||
}
|
||||
opts.args.push_back("--worker-name");
|
||||
opts.args.push_back(worker_name);
|
||||
if(!log_dir_.empty()) {
|
||||
opts.args.push_back("--log-dir");
|
||||
opts.args.push_back(log_dir_);
|
||||
}
|
||||
opts.streams = {
|
||||
kota::process::stdio::pipe(true, false),
|
||||
kota::process::stdio::pipe(false, true),
|
||||
kota::process::stdio::pipe(false, true),
|
||||
};
|
||||
|
||||
auto result = kota::process::spawn(opts, loop);
|
||||
if(!result) {
|
||||
LOG_ERROR("Failed to respawn worker {}: {}", worker_name, result.error().message());
|
||||
return false;
|
||||
}
|
||||
|
||||
auto& spawn = *result;
|
||||
auto transport = std::make_unique<kota::ipc::StreamTransport>(std::move(spawn.stdout_pipe),
|
||||
std::move(spawn.stdin_pipe));
|
||||
auto peer = std::make_unique<kota::ipc::BincodePeer>(loop, std::move(transport));
|
||||
|
||||
std::string prefix = "[" + worker_name + "]";
|
||||
loop.schedule(drain_stderr(std::move(spawn.stderr_pipe), prefix));
|
||||
|
||||
workers[index] = WorkerProcess{
|
||||
.proc = std::move(spawn.proc),
|
||||
.peer = std::move(peer),
|
||||
.owned_documents = 0,
|
||||
.alive = true,
|
||||
.restart_count = old_restart_count,
|
||||
};
|
||||
|
||||
auto& w = workers[index];
|
||||
++alive_count_;
|
||||
loop.schedule(w.peer->run());
|
||||
|
||||
if(stateful) {
|
||||
w.peer->on_notification([this](const worker::EvictedParams& params) {
|
||||
if(on_evicted)
|
||||
on_evicted(params.path);
|
||||
});
|
||||
}
|
||||
|
||||
loop.schedule(monitor_worker(index, stateful));
|
||||
|
||||
LOG_INFO("Worker {} restarted (attempt {})", worker_name, old_restart_count);
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace clice
|
||||
|
||||
@@ -64,6 +64,8 @@ private:
|
||||
kota::process proc;
|
||||
std::unique_ptr<kota::ipc::BincodePeer> peer;
|
||||
std::size_t owned_documents = 0;
|
||||
bool alive = true;
|
||||
unsigned restart_count = 0;
|
||||
};
|
||||
|
||||
kota::event_loop& loop;
|
||||
@@ -80,8 +82,19 @@ private:
|
||||
void clear_owner(std::size_t worker_index);
|
||||
std::size_t pick_least_loaded();
|
||||
|
||||
bool shutting_down_ = false;
|
||||
std::size_t alive_count_ = 0;
|
||||
kota::event all_exited_{true}; // Signalled when alive_count_ reaches 0.
|
||||
WorkerPoolOptions options_;
|
||||
std::string log_dir_;
|
||||
|
||||
/// Peers moved here during respawn so their coroutines can finish
|
||||
/// before the object is destroyed.
|
||||
llvm::SmallVector<std::unique_ptr<kota::ipc::BincodePeer>> retired_peers;
|
||||
|
||||
bool spawn_worker(const std::string& self_path, bool stateful, std::uint64_t memory_limit);
|
||||
bool respawn_worker(std::size_t index, bool stateful);
|
||||
kota::task<> monitor_worker(std::size_t index, bool stateful);
|
||||
};
|
||||
|
||||
template <typename Params>
|
||||
@@ -91,11 +104,10 @@ RequestResult<Params> WorkerPool::send_stateful(std::uint32_t path_id,
|
||||
if(stateful_workers.empty()) {
|
||||
co_return kota::outcome_error(kota::ipc::Error{"No stateful workers available"});
|
||||
}
|
||||
// No timeout: compile tasks run as detached tasks (loop.schedule) that
|
||||
// are immune to LSP $/cancelRequest. Adding a timeout here would use
|
||||
// kotatsu's with_token/when_any which has a spurious-cancellation bug
|
||||
// that kills requests within milliseconds instead of the configured period.
|
||||
auto idx = assign_worker(path_id);
|
||||
if(!stateful_workers[idx].alive) {
|
||||
co_return kota::outcome_error(kota::ipc::Error{"Assigned stateful worker is down"});
|
||||
}
|
||||
co_return co_await stateful_workers[idx].peer->send_request(params, opts);
|
||||
}
|
||||
|
||||
@@ -105,9 +117,16 @@ RequestResult<Params> WorkerPool::send_stateless(const Params& params,
|
||||
if(stateless_workers.empty()) {
|
||||
co_return kota::outcome_error(kota::ipc::Error{"No stateless workers available"});
|
||||
}
|
||||
auto idx = next_stateless;
|
||||
next_stateless = (next_stateless + 1) % stateless_workers.size();
|
||||
co_return co_await stateless_workers[idx].peer->send_request(params, opts);
|
||||
// Round-robin, skipping dead workers.
|
||||
auto start = next_stateless;
|
||||
for(std::size_t i = 0; i < stateless_workers.size(); ++i) {
|
||||
auto idx = (start + i) % stateless_workers.size();
|
||||
if(stateless_workers[idx].alive) {
|
||||
next_stateless = (idx + 1) % stateless_workers.size();
|
||||
co_return co_await stateless_workers[idx].peer->send_request(params, opts);
|
||||
}
|
||||
}
|
||||
co_return kota::outcome_error(kota::ipc::Error{"All stateless workers are down"});
|
||||
}
|
||||
|
||||
template <typename Params>
|
||||
@@ -115,6 +134,8 @@ void WorkerPool::notify_stateful(std::uint32_t path_id, const Params& params) {
|
||||
auto it = owner.find(path_id);
|
||||
if(it == owner.end())
|
||||
return;
|
||||
if(!stateful_workers[it->second].alive)
|
||||
return;
|
||||
stateful_workers[it->second].peer->send_notification(params);
|
||||
}
|
||||
|
||||
|
||||
@@ -110,7 +110,11 @@ async def client(
|
||||
|
||||
if workspace is not None:
|
||||
init_options_marker = request.node.get_closest_marker("init_options")
|
||||
init_options = init_options_marker.args[0] if init_options_marker else None
|
||||
init_options = dict(init_options_marker.args[0]) if init_options_marker else {}
|
||||
# Force cache_dir into the workspace so .clice/ cleanup prevents stale PCH.
|
||||
project = dict(init_options.get("project", {}))
|
||||
project.setdefault("cache_dir", str(workspace / ".clice"))
|
||||
init_options["project"] = project
|
||||
await c.initialize(workspace, initialization_options=init_options)
|
||||
|
||||
yield c
|
||||
@@ -165,12 +169,17 @@ async def _shutdown_client(c: CliceClient) -> None:
|
||||
|
||||
try:
|
||||
server = getattr(c, "_server", None)
|
||||
if server and server.stderr:
|
||||
stderr_data = await asyncio.wait_for(server.stderr.read(), timeout=2.0)
|
||||
if stderr_data:
|
||||
for line in stderr_data.decode("utf-8", errors="replace").splitlines():
|
||||
if "[warn]" in line or "[error]" in line:
|
||||
print(f"[server] {line}", flush=True)
|
||||
if server:
|
||||
if server.returncode is not None:
|
||||
print(f"[server] exit code: {server.returncode}", flush=True)
|
||||
if server.stderr:
|
||||
stderr_data = await asyncio.wait_for(server.stderr.read(), timeout=2.0)
|
||||
if stderr_data:
|
||||
for line in stderr_data.decode(
|
||||
"utf-8", errors="replace"
|
||||
).splitlines():
|
||||
if "[warn]" in line or "[error]" in line or "Sanitizer" in line:
|
||||
print(f"[server] {line}", flush=True)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
@@ -13,6 +13,9 @@ import re
|
||||
import signal
|
||||
import sys
|
||||
import time
|
||||
|
||||
# Force line-buffered stdout so CI sees output immediately.
|
||||
sys.stdout.reconfigure(line_buffering=True)
|
||||
from pathlib import Path
|
||||
from urllib.parse import quote, unquote
|
||||
|
||||
@@ -109,7 +112,9 @@ async def write_lsp_message(writer: asyncio.StreamWriter, payload: str):
|
||||
await writer.drain()
|
||||
|
||||
|
||||
async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool | None:
|
||||
async def replay_one(
|
||||
trace_path: Path, clice_bin: Path, timeout: int, wall_timeout: int = 300
|
||||
) -> bool | None:
|
||||
"""Replay a single trace. Returns True=PASS, False=FAIL, None=SKIP."""
|
||||
records = load_trace(trace_path)
|
||||
if not records:
|
||||
@@ -179,8 +184,21 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
|
||||
last_method = None
|
||||
sent_count = 0
|
||||
|
||||
wall_deadline = wall_start + wall_timeout
|
||||
|
||||
def remaining_wall():
|
||||
return max(0, wall_deadline - time.monotonic())
|
||||
|
||||
try:
|
||||
for i, rec in enumerate(records):
|
||||
if remaining_wall() <= 0:
|
||||
elapsed = time.monotonic() - wall_start
|
||||
print(
|
||||
f" result: TIMEOUT (wall-clock {wall_timeout}s exceeded, {elapsed:.1f}s)"
|
||||
)
|
||||
success = False
|
||||
break
|
||||
|
||||
if i > 0:
|
||||
delay = rec["ts"] - records[i - 1]["ts"]
|
||||
if delay > 0:
|
||||
@@ -196,7 +214,7 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
|
||||
try:
|
||||
await asyncio.wait_for(
|
||||
asyncio.gather(*pending.values(), return_exceptions=True),
|
||||
timeout=timeout,
|
||||
timeout=min(timeout, remaining_wall()),
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
elapsed = time.monotonic() - wall_start
|
||||
@@ -210,7 +228,19 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
|
||||
if msg_id is not None and method is not None:
|
||||
pending[msg_id] = asyncio.get_event_loop().create_future()
|
||||
|
||||
await write_lsp_message(proc.stdin, rec["msg"])
|
||||
try:
|
||||
await asyncio.wait_for(
|
||||
write_lsp_message(proc.stdin, rec["msg"]),
|
||||
timeout=min(30, remaining_wall()),
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
elapsed = time.monotonic() - wall_start
|
||||
print(
|
||||
f" result: HANG (write blocked at {last_method},"
|
||||
f" sent={sent_count}/{len(records)}, {elapsed:.1f}s)"
|
||||
)
|
||||
success = False
|
||||
break
|
||||
sent_count = i + 1
|
||||
|
||||
except (ConnectionError, BrokenPipeError):
|
||||
@@ -231,7 +261,7 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
|
||||
try:
|
||||
await asyncio.wait_for(
|
||||
asyncio.gather(*pending.values(), return_exceptions=True),
|
||||
timeout=timeout,
|
||||
timeout=min(timeout, remaining_wall()),
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
elapsed = time.monotonic() - wall_start
|
||||
@@ -294,7 +324,7 @@ async def async_main(args):
|
||||
print(f"SKIP: {trace} (not found)")
|
||||
skipped += 1
|
||||
continue
|
||||
result = await replay_one(trace, args.clice, args.timeout)
|
||||
result = await replay_one(trace, args.clice, args.timeout, args.wall_timeout)
|
||||
if result is None:
|
||||
skipped += 1
|
||||
elif result:
|
||||
@@ -317,7 +347,16 @@ def main():
|
||||
p.add_argument("traces", nargs="+", type=Path, help="JSONL trace files")
|
||||
p.add_argument("--clice", required=True, type=Path, help="Path to clice binary")
|
||||
p.add_argument(
|
||||
"--timeout", type=int, default=120, help="Timeout in seconds (default: 120)"
|
||||
"--timeout",
|
||||
type=int,
|
||||
default=120,
|
||||
help="Per-request timeout in seconds (default: 120)",
|
||||
)
|
||||
p.add_argument(
|
||||
"--wall-timeout",
|
||||
type=int,
|
||||
default=300,
|
||||
help="Max wall-clock time per trace in seconds (default: 300)",
|
||||
)
|
||||
args = p.parse_args()
|
||||
sys.exit(asyncio.run(async_main(args)))
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
#include "support/filesystem.h"
|
||||
|
||||
#include "kota/codec/json/json.h"
|
||||
#include "kota/codec/toml.h"
|
||||
#include "kota/codec/toml/toml.h"
|
||||
|
||||
namespace clice::testing {
|
||||
|
||||
@@ -148,7 +148,7 @@ TEST_CASE(ApplyDefaults) {
|
||||
EXPECT_EQ(*config.project.idle_timeout_ms, 3000);
|
||||
EXPECT_EQ(config.project.max_active_file.value, 8);
|
||||
EXPECT_EQ(config.project.stateful_worker_count.value, 2u);
|
||||
EXPECT_EQ(config.project.stateless_worker_count.value, 3u);
|
||||
EXPECT_GE(config.project.stateless_worker_count.value, 2u);
|
||||
EXPECT_FALSE(config.project.cache_dir.empty());
|
||||
EXPECT_FALSE(config.project.index_dir.empty());
|
||||
EXPECT_FALSE(config.project.logging_dir.empty());
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
#include "server/protocol.h"
|
||||
#include "server/worker_test_helpers.h"
|
||||
|
||||
#include "kota/codec/raw_value.h"
|
||||
#include "kota/codec/json/json.h"
|
||||
|
||||
namespace clice::testing {
|
||||
|
||||
|
||||
@@ -6,7 +6,6 @@
|
||||
#include "server/worker_test_helpers.h"
|
||||
|
||||
#include "kota/codec/bincode/bincode.h"
|
||||
#include "kota/codec/raw_value.h"
|
||||
|
||||
namespace clice::testing {
|
||||
|
||||
|
||||
Reference in New Issue
Block a user