3 Commits

Author SHA1 Message Date
Myriad-Dreamin
183b90d572 Add OpenSpec proposal for folding range pipeline split 2026-04-24 00:13:04 +08:00
ykiko
939ab6d0d4 feat(server): concurrent background indexing with priority control (#432)
## Summary

- Rewrite serial background indexing to concurrent dispatch (up to
`stateless_worker_count / 2` parallel tasks)
- Add depth-counted pause/resume mechanism: completion and
signature-help handlers pause new index dispatches to prioritize user
requests
- Report indexing progress via LSP `$/progress` notifications
(percentage + file count)
- Lower thread scheduling priority (`nice +10`) for index tasks in
stateless workers via RAII `ScopedNice` guard

## Test plan

- [x] `pixi run format` — no changes
- [x] `pixi run unit-test Debug` — 551 passed, 9 skipped (pre-existing)
- [x] `pixi run smoke-test Debug` — 2/2 passed
- [x] `pixi run integration-test Debug` — 121 passed, 3 failed (all
pre-existing on main: header_context x2, staleness x1)
- [ ] Manual test: open a large project (e.g. LLVM), verify progress bar
appears and completion remains responsive during indexing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Pause/resume controls for background indexing
* Concurrent, adaptive background indexing with configurable concurrency
* LSP progress reporting (create/begin/report/end) and updated
completion metrics

* **Behavior Change**
* Code completion and signature help temporarily pause indexing for
responsiveness
* Background indexing runs with reduced scheduling priority on
non-Windows and logs "files dispatched" at finish

* **Tests**
* Test client fixture defaults init options and sets workspace cache dir
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-23 13:28:59 +08:00
ykiko
e1202d2fa5 fix: prevent worker crashes from null ASTConsumer, invalid FileID, and missing PCH cache dir (#435)
## Summary

Three pre-existing bugs cause worker processes to crash with SEGV or
SIGABRT. On the main branch these crashes are silent (workers die,
requests fail fast with "transport closed", tests still pass because
null responses are accepted). However when combined with #432's worker
respawn mechanism, the crash-respawn-crash cycle on low-core CI machines
causes request timeouts and smoke test hangs.

### Fixes

- **compilation.cpp**: `ProxyAction::CreateASTConsumer` now checks for
null before passing to `MultiplexConsumer`. When the wrapped action's
`CreateASTConsumer` fails (e.g. missing system headers during PCH
generation), this previously caused a null pointer dereference, SEGV,
ASAN kills the stateless worker.
- **compilation_unit.cpp**: `file_path()` returns empty `StringRef` on
invalid `FileID` instead of asserting. The assert fired when
`IncludeGraph::from()` called `file_path(interested_file())` on an AST
compiled with synthesized default commands (no compile_commands.json,
clang++ -std=c++20 fallback, no system headers, invalid main file ID),
SIGABRT, stateful worker crash.
- **compiler.cpp**: `ensure_pch` now creates the PCH cache directory
before sending the build request. Previously, when `load_workspace()`
exited early (no compile_commands.json), the cache subdirectories were
never created, causing every PCH write to fail with "No such file or
directory".
- **master_server.cpp/h**: `load_workspace()` changed from
`kota::task<>` to plain `void` -- it contains only synchronous
filesystem operations and no co_await, so the coroutine wrapper was
unnecessary. Called directly instead of via `loop.schedule()`.

## Test plan

- [x] Verified zero SEGV/SIGABRT/assertion crashes in worker stderr
after fix
- [x] rapid_edit.jsonl smoke test passes 3/3 runs consistently (34s
each)
- [x] Behavior matches main branch (both return 134 responses, 0
pending)
- [x] Debug build with ASAN (detect_leaks=0) -- clean run, no sanitizer
reports

<!-- codesmith:footer -->
---
<a
href="https://app.blacksmith.sh/clice-io/codesmith/clice/pr/435"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-in-codesmith-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-in-codesmith-light.svg"><img
alt="View in Codesmith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/view-in-codesmith-dark.svg"></picture></a>
<sup>Codesmith can help with this PR — just tag <code>@codesmith</code>
or enable autofix.</sup>

- [ ] Autofix CI and bot reviews
<!-- /codesmith:footer -->

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved error handling for AST consumer creation with null checks and
a clear failure path.
* Safer file-path access that returns empty for invalid identifiers
instead of asserting.
* PCH cache handling now validates cache configuration, attempts
directory creation, logs warnings, and aborts PCH builds on failure.

* **Refactor**
* Workspace loading changed from asynchronous to synchronous execution.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-23 10:36:03 +08:00
31 changed files with 875 additions and 120 deletions

View File

@@ -7,6 +7,10 @@ on:
pull_request:
branches: [main]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
jobs:
changes:
if: ${{ !startsWith(github.ref, 'refs/tags/') }}

View File

@@ -96,9 +96,20 @@ jobs:
if-no-files-found: error
retention-days: 1
- name: Run tests
- name: Unit tests
if: ${{ !matrix.build_only }}
run: pixi run test ${{ matrix.build_type }}
timeout-minutes: 5
run: pixi run unit-test ${{ matrix.build_type }}
- name: Integration tests
if: ${{ !matrix.build_only }}
timeout-minutes: 20
run: pixi run integration-test ${{ matrix.build_type }}
- name: Smoke tests
if: ${{ !matrix.build_only }}
timeout-minutes: 15
run: pixi run smoke-test ${{ matrix.build_type }}
- name: Print cache stats and stop server
if: always()
@@ -146,5 +157,14 @@ jobs:
if: runner.os != 'Windows'
run: chmod +x build/${{ matrix.build_type }}/bin/*
- name: Run tests
run: pixi run -e test-run test ${{ matrix.build_type }}
- name: Unit tests
timeout-minutes: 5
run: pixi run -e test-run unit-test ${{ matrix.build_type }}
- name: Integration tests
timeout-minutes: 20
run: pixi run -e test-run integration-test ${{ matrix.build_type }}
- name: Smoke tests
timeout-minutes: 10
run: pixi run -e test-run smoke-test ${{ matrix.build_type }}

1
.gitignore vendored
View File

@@ -72,4 +72,3 @@ tests/unit/Local/
.claude/*
!.claude/CLAUDE.md
!.claude/commands/
openspec/

View File

@@ -26,7 +26,7 @@
#include "support/path_pool.h"
#include "syntax/dependency_graph.h"
#include "kota/codec/json/serializer.h"
#include "kota/codec/json/json.h"
#include "kota/deco/deco.h"
#include "llvm/Support/FileSystem.h"

View File

@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-04-22

View File

@@ -0,0 +1,185 @@
## Context
This change extracts decision `2` from `openspec/changes/explore-improve-folding-range-support/design.md` into a standalone proposal. The current folding implementation in `src/feature/folding_ranges.cpp` mixes three responsibilities in one path:
- discovering foldable structure from AST data
- deciding which ranges survive deduplication and validation
- shaping the final LSP response, including output metadata
That coupling makes the code harder to extend safely. Comment folding, directive-based collectors, capability-aware rendering, and range limiting all become riskier when collection and rendering rules share the same code path. The extracted proposal keeps scope narrower: it does not add new fold categories by itself, but it creates the architecture that later changes can build on without destabilizing existing structural folding.
The downloaded clangd reference confirms both the value and the limit of the upstream design. clangd has useful, tested folding behavior for brace bodies, comment blocks, contiguous `//` groups, and `lineFoldingOnly`, but its implementation largely emits protocol-shaped `FoldingRange` objects directly from collection code. In `SemanticSelection.cpp`, both the AST path and the pseudo-parser path build `FoldingRange` results directly, and the pseudo-parser applies rendering details such as delimiter trimming and `lineFoldingOnly` adjustments while collecting ranges. That is a good behavior reference, but it is not the architecture this extracted change should copy.
`clice` already has stronger ingredients for a real pipeline:
- `LocalSourceRange` gives us a main-file, half-open offset representation that is independent of LSP position encoding
- directive metadata already captures information clangd does not expose well, including conditional-branch state, pragma regions, includes, imports, and macro references
- the current tests are boundary-oriented, which makes them a good fit for validating raw spans before protocol rendering
The design therefore separates "what fold exists in the source" from "how that fold should be emitted to this client". clangd's tested boundary rules are still relevant, but they should become renderer policy and normalization rules rather than collector output format.
## Goals / Non-Goals
**Goals:**
- Separate folding processing into collection, normalization, and rendering phases.
- Preserve the existing AST structural folding categories already supported by `clice`.
- Make ordering, deduplication, and boundary validation deterministic and testable.
- Give later changes a stable extension point for comments, directives, and client-driven rendering options.
**Non-Goals:**
- Add comment folding in this change.
- Fix preprocessor branch-closing behavior in this change.
- Add new fold categories such as macro definitions or include/import grouping.
- Depend on initialize-time client capability plumbing being implemented first.
## Decisions
### 1. Use clangd as a behavior reference, not an architecture template
This change should borrow clangd's confirmed folding behavior where it is useful, especially around multiline comments, contiguous `//` comment groups, main-file-only filtering, and `lineFoldingOnly` boundary shaping. It should not copy clangd's habit of emitting protocol-shaped `FoldingRange` objects directly from collection logic.
Why:
- clangd's tests are valuable because they pin down tricky folding behavior around comments, macro boundaries, and line-only rendering
- clangd's data flow is intentionally narrow and mixes collection with response shaping
- `clice` already has richer file-local and directive metadata that supports a cleaner internal representation
Alternative considered:
- Treat clangd's direct `FoldingRange` construction as the architecture to reproduce. Rejected because it would preserve the same coupling this extracted change is meant to remove.
### 2. Introduce a raw internal folding-range model
Collectors should emit an internal `RawFoldingRange`-style structure instead of final LSP protocol objects. The raw model should preserve source locations, an internal category, and optional metadata hints that later phases may use.
The raw model should be shaped around file-local source structure, not LSP transport fields. At minimum it should carry:
- a main-file `LocalSourceRange` span using half-open byte offsets
- an internal fold category such as namespace, record, access section, function body, comment block, comment group, conditional branch, pragma region, include group, or import group
- the collector origin, such as AST, comment scanning, or directive metadata, so normalization has a stable tie-break and debugging surface
- render hints for syntax-specific shaping, such as delimiter trimming, whether line-only folding should hide the final line, and an optional collapsed-text hint
In other words, the raw model should look closer to:
```cpp
struct RawFoldRenderHint {
std::uint8_t trim_start_bytes = 0;
std::uint8_t trim_end_bytes = 0;
bool hide_last_line_when_line_only = false;
std::string collapsed_text_hint;
};
struct RawFoldingRange {
LocalSourceRange span;
RawFoldCategory category;
RawFoldOrigin origin;
RawFoldRenderHint render;
};
```
The important design choice is that `span` represents the foldable source envelope in the main file, while renderer-specific trimming stays in `render` hints. For example:
- brace-based structural folds keep the full braced span and let the renderer trim interior boundaries
- block comments keep the full `/* ... */` span and let the renderer decide whether to hide the closing delimiter or final line
- contiguous `//` groups keep the full grouped span and let the renderer decide how much of the opening sentinel remains visible
Why:
- collectors should describe what was found, not how it will be serialized
- `LocalSourceRange` is already the natural coordinate system for `clice`
- public LSP kinds such as `comment`, `imports`, and `region` are too lossy to use as the internal category model
- future comment and directive collectors can share the same pipeline contract
- tests can validate collection independently from rendering
Alternatives considered:
- Continue emitting LSP ranges directly from collectors. Rejected because it keeps protocol concerns entangled with source discovery.
- Make the raw model store already-trimmed visible interior spans instead of the full source envelope. Rejected because line-only rendering, collapsed text, and comment delimiter rules would still leak back into every collector.
### 3. Normalize ranges before rendering
All collected ranges should pass through a normalization step before any response is emitted. Normalization is responsible for deterministic ordering, duplicate removal, and rejection of degenerate or unmappable ranges.
Normalization should operate on raw spans and internal categories, not on final LSP fields. Its responsibilities include:
- deterministic ordering independent of collector traversal order
- duplicate collapse for collectors that discover the same fold
- invalid-range filtering after raw spans and render hints are reconciled
- stable tie-breaking for overlapping ranges from different origins
Collectors may still reject obviously invalid inputs, such as non-main-file locations that cannot be mapped to `LocalSourceRange`, but normalization remains the phase that decides which collected folds survive to rendering.
Why:
- duplicate or invalid ranges are easier to reason about in one place than across many collectors
- stable ordering reduces regression noise and makes range limiting predictable later
- category-aware normalization preserves internal meaning until the renderer maps it to public kinds
- normalization lets new collectors plug in without each collector re-implementing cleanup logic
Alternative considered:
- Let each collector manage its own sorting and duplicate suppression. Rejected because cross-collector interactions would still remain undefined.
### 4. Keep the current AST visitor as the first collector boundary
The initial extraction should preserve the current AST visitor as one collector feeding the raw model. This reduces refactor risk while still creating the new phase boundaries.
Why:
- the existing structural fold coverage is valuable and should not be rewritten unnecessarily
- an adapter-style refactor is easier to verify against current tests than a full collector redesign
Alternative considered:
- Rewrite collection around a brand-new multi-source manager immediately. Rejected because it adds scope before the phase split is proven.
### 5. Move output shaping into a dedicated renderer
The renderer should translate normalized ranges into LSP folding ranges. Boundary shaping, output kinds, and optional metadata emission should live there, even if some options still use default values until later protocol plumbing exists.
Renderer input should be the normalized raw model plus a separate `FoldingRenderOptions` structure. The renderer then becomes responsible for:
- converting `LocalSourceRange` into protocol positions for the requested encoding
- applying delimiter trimming and line-only adjustments
- mapping internal categories to public LSP kinds
- deciding whether collapsed text is emitted or suppressed
- later applying deterministic `rangeLimit` trimming without changing collectors
This is the key point where `clice` should intentionally diverge from clangd. clangd threads `lineFoldingOnly` into collection and directly produces protocol objects. `clice` should keep those capability and transport decisions isolated in rendering so collectors remain stable as client support evolves.
Why:
- rendering rules are a separate concern from source discovery
- later work on line-only output, metadata gating, or public kind mapping should not force collector rewrites
- clangd-style line-only shaping is still supported, but as renderer policy rather than collector output
- isolating rendering makes behavioral diffs easier to review
Alternative considered:
- Keep final boundary shaping next to the AST collector and only add a small helper for sorting. Rejected because it only moves a symptom, not the architectural problem.
## Risks / Trade-offs
- [Refactoring the current path can accidentally change fold ordering] -> Mitigation: add deterministic-order assertions and compare outputs for existing structural fixtures.
- [The raw model could become too abstract too early] -> Mitigation: keep the initial fields minimal and only include data already needed by current structural folds.
- [Full-envelope raw spans plus render hints may feel less direct than storing already-trimmed ranges] -> Mitigation: use a small, explicit render-hint structure and validate brace/comment shaping with focused renderer tests.
- [A renderer abstraction may appear premature before full capability plumbing exists] -> Mitigation: keep default render options aligned with current behavior and treat future options as extension points, not immediate scope.
## Migration Plan
1. Introduce raw folding-range and render-option types behind the existing entrypoint.
2. Convert the current AST-based collectors to emit raw ranges.
3. Insert normalization between collection and response emission.
4. Move LSP object construction into a dedicated renderer.
5. Verify that existing structural folding fixtures still produce the expected ranges.
Rollback strategy:
- If the refactor destabilizes output, keep the new helper types but temporarily route the old direct-emission path until normalization and rendering regressions are resolved.
## Open Questions
- Whether public kind remapping should land in this extracted change or remain a follow-up proposal once the renderer boundary exists.
- Whether collector origin should remain part of the long-term raw model after normalization policy stabilizes, or only exist temporarily as a debugging and tie-break aid.

View File

@@ -0,0 +1,26 @@
## Why
`explore-improve-folding-range-support` combines several different concerns: upstream comparison work, baseline folding fixes, preprocessor extensions, and an internal refactor. The second design point in that change, splitting the folding-range pipeline into collection, normalization, and rendering, is the architectural slice that other work depends on and should be referenceable as its own proposal.
## What Changes
- Extract the pipeline-splitting work from `explore-improve-folding-range-support` into a standalone change focused on folding-range architecture.
- Introduce an internal raw folding-range model so collectors no longer emit final LSP objects directly.
- Define a normalization phase that performs deterministic sorting, duplicate removal, and boundary validation before response generation.
- Define a rendering phase that owns line/column shaping and optional metadata emission instead of mixing those concerns into collectors.
- Preserve the current AST structural folding coverage while establishing extension points for future comment, directive, and capability-aware rendering work.
## Capabilities
### New Capabilities
- `folding-range-pipeline`: Provide a deterministic folding-range pipeline that separates collection, normalization, and rendering while preserving existing structural folds.
### Modified Capabilities
- None.
## Impact
- `src/feature/folding_ranges.cpp` will be refactored around raw-range collection, normalization, and rendering boundaries.
- Folding-related helper types may be introduced near the folding feature implementation.
- `tests/unit/feature/folding_range_tests.cpp` will need regression coverage for structural folds and deterministic ordering.
- `openspec/changes/explore-improve-folding-range-support/design.md` remains the source change from which this standalone proposal was extracted.

View File

@@ -0,0 +1,38 @@
## ADDED Requirements
### Requirement: Folding ranges are normalized before response emission
The server SHALL convert collected folding candidates into a deterministic normalized set before emitting the folding range response.
#### Scenario: Duplicate candidates collapse to one emitted fold
- **WHEN** multiple collectors produce the same folding candidate for the same source span and internal category
- **THEN** the server MUST emit at most one folding range for that candidate
#### Scenario: Invalid candidates are dropped during normalization
- **WHEN** a collected folding candidate does not span multiple lines or cannot be mapped back to the main file
- **THEN** the server MUST omit that candidate from the emitted folding ranges
#### Scenario: Output ordering is deterministic
- **WHEN** the same document is analyzed repeatedly without source changes
- **THEN** the server MUST emit folding ranges in a deterministic order that does not depend on collector traversal order
### Requirement: Existing structural folding survives the pipeline split
The server SHALL preserve the currently supported AST structural folding categories after collection, normalization, and rendering are separated.
#### Scenario: Supported structural regions remain foldable
- **WHEN** a document contains a supported multi-line namespace, record, function body, parameter list, lambda body, initializer list, call argument list, or compound statement
- **THEN** the server MUST still return a folding range for that region when its boundaries can be mapped to the main file
#### Scenario: Structural coverage is preserved through normalization
- **WHEN** the document contains only currently supported AST-driven folding categories
- **THEN** normalization and rendering MUST NOT remove a valid structural fold except when it is an exact duplicate or an invalid range
### Requirement: Rendering decisions are applied after normalization
The server SHALL derive final LSP folding-range output from normalized internal ranges instead of requiring collectors to emit protocol-shaped results directly.
#### Scenario: Rendering options do not require collector changes
- **WHEN** rendering rules change how line or metadata output is shaped for a normalized fold
- **THEN** the server MUST apply that change in the rendering phase without requiring collector-specific logic changes
#### Scenario: Metadata hints remain optional until rendering
- **WHEN** a collected or normalized fold carries optional kind or collapsed-text hints
- **THEN** the renderer MUST decide whether to surface, transform, or suppress that metadata in the emitted LSP range

View File

@@ -0,0 +1,16 @@
## 1. Raw Model and Collector Boundary
- [ ] 1.1 Introduce internal raw folding-range and render-option types while keeping the current folding entrypoint stable.
- [ ] 1.2 Convert the existing AST structural folding path in `src/feature/folding_ranges.cpp` to emit raw ranges instead of final LSP ranges.
- [ ] 1.3 Add regression fixtures or assertions that cover the currently supported structural fold categories before further refactoring.
## 2. Normalization and Rendering
- [ ] 2.1 Implement normalization for deterministic sorting, duplicate removal, and invalid-range filtering.
- [ ] 2.2 Introduce a dedicated renderer that converts normalized ranges into final LSP folding-range objects.
- [ ] 2.3 Keep default rendered output compatible with current structural behavior while exposing extension points for future collectors and render rules.
## 3. Verification
- [ ] 3.1 Compare pre-refactor and post-refactor outputs for the existing structural folding test cases.
- [ ] 3.2 Run relevant folding-range unit tests and fix any ordering, deduplication, or boundary regressions introduced by the new pipeline.

23
pixi.lock generated
View File

@@ -1078,6 +1078,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
linux-aarch64:
- conda: https://conda.anaconda.org/conda-forge/linux-aarch64/_openmp_mutex-4.5-20_gnu.conda
@@ -1152,6 +1153,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-64:
- conda: https://conda.anaconda.org/conda-forge/osx-64/bzip2-1.0.8-h500dc9f_9.conda
@@ -1224,6 +1226,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-arm64:
- conda: https://conda.anaconda.org/conda-forge/osx-arm64/bzip2-1.0.8-hd037594_8.conda
@@ -1289,6 +1292,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
win-64:
- conda: https://conda.anaconda.org/conda-forge/win-64/bzip2-1.0.8-h0ad9c76_8.conda
@@ -1343,6 +1347,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
format:
channels:
@@ -1704,6 +1709,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
linux-aarch64:
- conda: https://conda.anaconda.org/conda-forge/linux-aarch64/_openmp_mutex-4.5-20_gnu.conda
@@ -1782,6 +1788,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-64:
- conda: https://conda.anaconda.org/conda-forge/osx-64/bzip2-1.0.8-h500dc9f_9.conda
@@ -1858,6 +1865,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-arm64:
- conda: https://conda.anaconda.org/conda-forge/osx-arm64/bzip2-1.0.8-hd037594_8.conda
@@ -1926,6 +1934,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
win-64:
- conda: https://conda.anaconda.org/conda-forge/win-64/bzip2-1.0.8-h0ad9c76_8.conda
@@ -1982,6 +1991,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
test-run:
channels:
@@ -2025,6 +2035,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
linux-aarch64:
- conda: https://conda.anaconda.org/conda-forge/linux-aarch64/_openmp_mutex-4.5-20_gnu.conda
@@ -2058,6 +2069,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-64:
- conda: https://conda.anaconda.org/conda-forge/osx-64/bzip2-1.0.8-h500dc9f_9.conda
@@ -2113,6 +2125,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
osx-arm64:
- conda: https://conda.anaconda.org/conda-forge/osx-arm64/bzip2-1.0.8-hd037594_9.conda
@@ -2168,6 +2181,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
win-64:
- conda: https://conda.anaconda.org/conda-forge/win-64/bzip2-1.0.8-h0ad9c76_9.conda
@@ -2199,6 +2213,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
win-arm64:
- conda: https://conda.anaconda.org/conda-forge/win-arm64/bzip2-1.0.8-h50b96f5_9.conda
@@ -2229,6 +2244,7 @@ environments:
- pypi: https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
- pypi: https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl
packages:
- conda: https://conda.anaconda.org/conda-forge/linux-64/_libgcc_mutex-0.1-conda_forge.tar.bz2
@@ -7795,6 +7811,13 @@ packages:
- coverage>=6.2 ; extra == 'testing'
- hypothesis>=5.7.1 ; extra == 'testing'
requires_python: '>=3.10'
- pypi: https://files.pythonhosted.org/packages/fa/b6/3127540ecdf1464a00e5a01ee60a1b09175f6913f0644ac748494d9c4b21/pytest_timeout-2.4.0-py3-none-any.whl
name: pytest-timeout
version: 2.4.0
sha256: c42667e5cdadb151aeb5b26d114aff6bdf5a907f176a007a30b940d3d865b5c2
requires_dist:
- pytest>=7.0.0
requires_python: '>=3.7'
- conda: https://conda.anaconda.org/conda-forge/linux-64/python-3.14.2-h32b2ec7_100_cp314.conda
build_number: 100
sha256: a120fb2da4e4d51dd32918c149b04a08815fd2bd52099dad1334647984bb07f1

View File

@@ -102,6 +102,7 @@ lld = "==20.1.8"
[feature.test.pypi-dependencies]
pytest = "*"
pytest-asyncio = ">=1.1.0"
pytest-timeout = "*"
pygls = ">=2.0.0"
lsprotocol = ">=2024.0.0"
@@ -165,8 +166,8 @@ cmd = './build/{{ type }}/bin/unit_tests --test-dir="./tests/data"'
[feature.test.tasks.integration-test]
args = [{ arg = "type", default = "RelWithDebInfo" }]
cmd = """
pytest -s --log-cli-level=INFO tests/integration \
--executable=./build/{{ type }}/bin/clice
pytest -s --log-cli-level=INFO --timeout=300 --timeout-method=thread \
tests/integration --executable=./build/{{ type }}/bin/clice
"""
[feature.test.tasks.smoke-test]

View File

@@ -219,9 +219,10 @@ public:
auto CreateASTConsumer(clang::CompilerInstance& instance, llvm::StringRef file)
-> std::unique_ptr<clang::ASTConsumer> final {
return std::make_unique<ProxyASTConsumer>(
WrapperFrontendAction::CreateASTConsumer(instance, file),
unit);
auto consumer = WrapperFrontendAction::CreateASTConsumer(instance, file);
if(!consumer)
return nullptr;
return std::make_unique<ProxyASTConsumer>(std::move(consumer), unit);
}
/// Make this public.

View File

@@ -81,7 +81,8 @@ auto CompilationUnitRef::file_offset(clang::SourceLocation location) -> std::uin
}
auto CompilationUnitRef::file_path(clang::FileID fid) -> llvm::StringRef {
assert(fid.isValid() && "Invalid fid");
if(!fid.isValid())
return {};
if(auto it = self->path_cache.find(fid); it != self->path_cache.end()) {
return it->second;
}

View File

@@ -308,6 +308,10 @@ const clang::NamedDecl* decl_of_impl(const void* T) {
}
auto decl_of(clang::QualType type) -> const clang::NamedDecl* {
if(type.isNull()) {
return nullptr;
}
// Strip type-sugar that wraps the underlying type without adding a decl
// (e.g. ElaboratedType for "struct Foo" vs plain "Foo").
if(auto ET = type->getAs<clang::ElaboratedType>()) {

View File

@@ -490,6 +490,22 @@ kota::task<bool> Compiler::ensure_pch(Session& session,
auto completion = std::make_shared<kota::event>();
workspace.pch_cache[path_id].building = completion;
if(workspace.config.project.cache_dir.empty()) {
LOG_WARN("PCH build skipped: cache_dir is not configured");
workspace.pch_cache[path_id].building.reset();
completion->set();
co_return false;
}
// Ensure the PCH cache directory exists.
auto pch_dir = path::join(workspace.config.project.cache_dir, "cache", "pch");
if(auto ec = llvm::sys::fs::create_directories(pch_dir)) {
LOG_WARN("Cannot create PCH cache dir {}: {}", pch_dir, ec.message());
workspace.pch_cache[path_id].building.reset();
completion->set();
co_return false;
}
// Build a new PCH via stateless worker.
worker::BuildParams bp;
bp.kind = worker::BuildKind::BuildPCH;

View File

@@ -14,7 +14,7 @@
#include "syntax/completion.h"
#include "kota/async/async.h"
#include "kota/codec/raw_value.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/peer.h"

View File

@@ -6,8 +6,9 @@
#include "support/glob_pattern.h"
#include "support/logging.h"
#include "kota/async/io/system.h"
#include "kota/codec/json/json.h"
#include "kota/codec/toml.h"
#include "kota/codec/toml/toml.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Path.h"
#include "llvm/Support/Process.h"
@@ -65,8 +66,10 @@ void Config::apply_defaults(llvm::StringRef workspace_root) {
if(p.stateful_worker_count == 0)
p.stateful_worker_count = 2;
if(p.stateless_worker_count == 0)
p.stateless_worker_count = 3;
if(p.stateless_worker_count == 0) {
auto cores = kota::sys::parallelism();
p.stateless_worker_count = std::max(cores / 2, 2u);
}
if(p.worker_memory_limit == 0)
p.worker_memory_limit = 4ULL * 1024 * 1024 * 1024; // 4GB

View File

@@ -1,5 +1,6 @@
#include "server/indexer.h"
#include <algorithm>
#include <string>
#include <variant>
#include <vector>
@@ -624,6 +625,23 @@ void Indexer::enqueue(std::uint32_t server_path_id) {
index_queue.push_back(server_path_id);
}
void Indexer::pause_indexing() {
++pause_depth;
if(pause_depth == 1) {
resume_event.reset();
LOG_DEBUG("Background indexing paused");
}
}
void Indexer::resume_indexing() {
if(pause_depth > 0)
--pause_depth;
if(pause_depth == 0) {
resume_event.set();
LOG_DEBUG("Background indexing resumed");
}
}
void Indexer::schedule() {
if(!*workspace.config.project.enable_indexing || indexing_active || indexing_scheduled)
return;
@@ -636,6 +654,76 @@ void Indexer::schedule() {
loop.schedule(run_background_indexing());
}
kota::task<> Indexer::index_one(std::uint32_t server_path_id) {
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
if(sessions.contains(server_path_id))
co_return;
if(!need_update(file_path))
co_return;
// For module interface units, compile their PCM (and transitive deps)
// first so the stateless worker has the artifacts it needs.
if(workspace.compile_graph && workspace.path_to_module.contains(server_path_id)) {
co_await workspace.compile_graph->compile(server_path_id);
}
worker::BuildParams params;
params.kind = worker::BuildKind::Index;
params.file = file_path;
if(!compiler.fill_compile_args(file_path, params.directory, params.arguments, nullptr))
co_return;
workspace.fill_pcm_deps(params.pcms);
LOG_INFO("Background indexing: {}", file_path);
auto result = co_await pool.send_stateless(params);
if(result.has_value() && result.value().success && !result.value().tu_index_data.empty()) {
LOG_INFO("Background indexing got TUIndex for {}: {} bytes",
file_path,
result.value().tu_index_data.size());
merge(result.value().tu_index_data.data(), result.value().tu_index_data.size());
} else if(result.has_value() && !result.value().success) {
LOG_WARN("Background index failed for {}: {}", file_path, result.value().error);
} else if(result.has_value() && result.value().tu_index_data.empty()) {
LOG_WARN("Background index returned empty TUIndex for {}", file_path);
} else {
LOG_WARN("Background index IPC error for {}: {}", file_path, result.error().message);
}
}
kota::task<> Indexer::monitor_resources(std::uint32_t generation) {
while(generation == monitor_generation) {
co_await kota::sleep(std::chrono::milliseconds(3000), loop);
if(generation != monitor_generation)
break;
auto mem = kota::sys::memory();
if(mem.total == 0)
continue;
// Respect cgroup/container limits when present.
auto effective_total =
(mem.constrained > 0 && mem.constrained < mem.total) ? mem.constrained : mem.total;
auto ratio = static_cast<double>(mem.available) / static_cast<double>(effective_total);
if(ratio < 0.15 && max_concurrent > 1) {
--max_concurrent;
LOG_INFO("Index concurrency -> {} (memory pressure: {:.0f}% available)",
max_concurrent,
ratio * 100);
} else if(ratio > 0.30 && max_concurrent < baseline_concurrent) {
++max_concurrent;
LOG_DEBUG("Index concurrency -> {} (memory OK: {:.0f}% available)",
max_concurrent,
ratio * 100);
}
}
}
kota::task<> Indexer::run_background_indexing() {
if(index_idle_timer) {
co_await index_idle_timer->wait();
@@ -648,48 +736,88 @@ kota::task<> Indexer::run_background_indexing() {
}
indexing_active = true;
std::size_t processed = 0;
++monitor_generation;
loop.schedule(monitor_resources(monitor_generation));
while(index_queue_pos < index_queue.size()) {
auto server_path_id = index_queue[index_queue_pos];
index_queue_pos++;
// Put module interface units first so their PCMs are built before
// non-module files that might import them.
std::stable_partition(
index_queue.begin() + index_queue_pos,
index_queue.end(),
[this](std::uint32_t id) { return workspace.path_to_module.contains(id); });
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
auto batch = index_queue.size() - index_queue_pos;
std::size_t dispatched = 0;
std::size_t completed = 0;
finished = 0;
if(sessions.contains(server_path_id))
continue;
if(!need_update(file_path))
continue;
worker::BuildParams params;
params.kind = worker::BuildKind::Index;
params.file = file_path;
if(!compiler.fill_compile_args(file_path, params.directory, params.arguments, nullptr))
continue;
workspace.fill_pcm_deps(params.pcms);
LOG_INFO("Background indexing: {}", file_path);
auto result = co_await pool.send_stateless(params);
if(result.has_value() && result.value().success && !result.value().tu_index_data.empty()) {
LOG_INFO("Background indexing got TUIndex for {}: {} bytes",
file_path,
result.value().tu_index_data.size());
merge(result.value().tu_index_data.data(), result.value().tu_index_data.size());
++processed;
} else if(result.has_value() && !result.value().success) {
LOG_WARN("Background index failed for {}: {}", file_path, result.value().error);
} else if(result.has_value() && result.value().tu_index_data.empty()) {
LOG_WARN("Background index returned empty TUIndex for {}", file_path);
// Progress reporting via LSP $/progress.
std::optional<lsp::ProgressReporter<kota::ipc::JsonPeer>> progress;
if(peer) {
progress.emplace(*peer, protocol::ProgressToken(std::string("clice/backgroundIndex")));
auto create_result = co_await progress->create();
if(!create_result.has_error()) {
progress->begin("Indexing", std::format("0/{} files", batch), 0);
} else {
LOG_WARN("Background index IPC error for {}: {}", file_path, result.error().message);
progress.reset();
}
}
while(index_queue_pos < index_queue.size() || inflight > 0) {
// Dispatch new tasks up to max_concurrent.
while(index_queue_pos < index_queue.size() && inflight < max_concurrent) {
// Wait if paused by a user request.
if(pause_depth > 0) {
co_await resume_event.wait();
}
auto server_path_id = index_queue[index_queue_pos++];
// Quick pre-filter: skip open files and fresh files without
// consuming a concurrency slot.
auto file_path = std::string(workspace.path_pool.resolve(server_path_id));
if(sessions.contains(server_path_id) || !need_update(file_path)) {
++completed;
continue;
}
++inflight;
++dispatched;
// Launch the index task. On completion it decrements
// inflight, bumps finished, and signals the event.
loop.schedule([](Indexer* self, std::uint32_t id, kota::event& done) -> kota::task<> {
co_await self->index_one(id);
--self->inflight;
++self->finished;
done.set();
}(this, server_path_id, completion_event));
}
if(inflight == 0)
break;
// Wait for at least one task to finish.
co_await completion_event.wait();
completion_event.reset();
// Drain all completions that occurred since last wake.
completed += std::exchange(finished, 0);
// Report progress.
if(progress) {
auto pct = batch > 0 ? static_cast<std::uint32_t>(completed * 100 / batch) : 100;
progress->report(std::format("{}/{} files", completed, batch), pct);
}
}
if(progress) {
progress->end(std::format("Indexed {} files", dispatched));
}
indexing_active = false;
LOG_INFO("Background indexing complete: {} files processed", processed);
++monitor_generation; // Stop the monitor coroutine.
LOG_INFO("Background indexing complete: {} files dispatched", dispatched);
save(workspace.config.project.index_dir);
}

View File

@@ -12,7 +12,9 @@
#include "server/workspace.h"
#include "kota/async/async.h"
#include "kota/ipc/codec/json.h"
#include "kota/ipc/lsp/position.h"
#include "kota/ipc/lsp/progress.h"
#include "kota/ipc/lsp/protocol.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/SmallVector.h"
@@ -62,6 +64,47 @@ public:
loop(loop), workspace(workspace), sessions(sessions), pool(pool), compiler(compiler),
is_file_open(std::move(is_file_open)) {}
/// Set the LSP peer for progress reporting. Must be called before
/// schedule() if progress notifications are desired.
void set_peer(kota::ipc::JsonPeer* p) {
peer = p;
}
/// Temporarily pause background indexing to give priority to user
/// requests. Indexing tasks already dispatched to workers continue,
/// but no new tasks will be sent until resume_indexing() is called.
void pause_indexing();
/// Resume background indexing after a pause.
void resume_indexing();
/// RAII guard that pauses indexing for its lifetime.
struct [[nodiscard]] ScopedPause {
Indexer& indexer;
explicit ScopedPause(Indexer& idx) : indexer(idx) {
indexer.pause_indexing();
}
~ScopedPause() {
indexer.resume_indexing();
}
ScopedPause(const ScopedPause&) = delete;
ScopedPause& operator=(const ScopedPause&) = delete;
};
ScopedPause scoped_pause() {
return ScopedPause{*this};
}
/// Set the maximum number of concurrent index tasks.
/// Also sets the baseline that dynamic adjustment will restore to.
void set_max_concurrency(std::size_t n) {
max_concurrent = std::max<std::size_t>(n, 1);
baseline_concurrent = max_concurrent;
}
/// Add a file to the background indexing queue.
void enqueue(std::uint32_t server_path_id);
@@ -175,6 +218,9 @@ private:
/// server-path-id-keyed sessions map to project-level path_ids.
std::function<bool(std::uint32_t)> is_file_open;
/// LSP peer for progress reporting (optional, not owned).
kota::ipc::JsonPeer* peer = nullptr;
/// Background indexing queue and scheduling state.
std::vector<std::uint32_t> index_queue;
std::size_t index_queue_pos = 0;
@@ -182,7 +228,30 @@ private:
bool indexing_scheduled = false;
std::shared_ptr<kota::timer> index_idle_timer;
/// Concurrency control for background indexing.
std::size_t max_concurrent = 2;
std::size_t baseline_concurrent = 2;
std::size_t inflight = 0;
std::size_t finished = 0; ///< Incremented by each completed dispatch task.
/// Pause/resume: when paused, new index tasks wait on this event.
/// Uses a counter so nested pause/resume pairs work correctly.
std::size_t pause_depth = 0;
kota::event resume_event{true};
/// Completion event — signalled by each finished dispatch task so the
/// main loop can wake up. Must be a member (not local to the coroutine)
/// because inflight tasks capture it by reference and may outlive the
/// coroutine frame during server shutdown.
kota::event completion_event;
/// Generation counter — incremented each run so a stale monitor_resources
/// coroutine can detect that its owning run has ended.
std::uint32_t monitor_generation = 0;
kota::task<> run_background_indexing();
kota::task<> index_one(std::uint32_t server_path_id);
kota::task<> monitor_resources(std::uint32_t generation);
};
} // namespace clice

View File

@@ -56,9 +56,9 @@ MasterServer::MasterServer(kota::event_loop& loop,
MasterServer::~MasterServer() = default;
kota::task<> MasterServer::load_workspace() {
void MasterServer::load_workspace() {
if(workspace_root.empty())
co_return;
return;
auto& cfg = workspace.config.project;
@@ -125,7 +125,7 @@ kota::task<> MasterServer::load_workspace() {
if(cdb_path.empty()) {
LOG_WARN("No compile_commands.json found in workspace {}", workspace_root);
co_return;
return;
}
auto count = workspace.cdb.load(cdb_path);
@@ -331,7 +331,10 @@ void MasterServer::register_handlers() {
indexer.schedule();
};
loop.schedule(load_workspace());
indexer.set_peer(&peer);
indexer.set_max_concurrency(cfg.stateless_worker_count.value);
load_workspace();
});
peer.on_request(
@@ -670,28 +673,33 @@ void MasterServer::register_handlers() {
/// Feature requests — stateless forwarding.
peer.on_request([this](RequestContext& ctx,
const protocol::CompletionParams& params) -> RawResult {
auto path = uri_to_path(params.text_document_position_params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
co_return serde_raw{"null"};
co_return co_await compiler.handle_completion(params.text_document_position_params.position,
sit->second);
});
peer.on_request(
[this](RequestContext& ctx, const protocol::SignatureHelpParams& params) -> RawResult {
[this](RequestContext& ctx, const protocol::CompletionParams& params) -> RawResult {
auto path = uri_to_path(params.text_document_position_params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
co_return serde_raw{"null"};
co_return co_await compiler.forward_build(worker::BuildKind::SignatureHelp,
auto pause = indexer.scoped_pause();
auto result =
co_await compiler.handle_completion(params.text_document_position_params.position,
sit->second);
co_return std::move(result);
});
peer.on_request([this](RequestContext& ctx,
const protocol::SignatureHelpParams& params) -> RawResult {
auto path = uri_to_path(params.text_document_position_params.text_document.uri);
auto path_id = workspace.path_pool.intern(path);
auto sit = sessions.find(path_id);
if(sit == sessions.end())
co_return serde_raw{"null"};
auto pause = indexer.scoped_pause();
auto result = co_await compiler.forward_build(worker::BuildKind::SignatureHelp,
params.text_document_position_params.position,
sit->second);
});
co_return std::move(result);
});
/// Hierarchy queries — index-based.

View File

@@ -12,7 +12,7 @@
#include "server/workspace.h"
#include "kota/async/async.h"
#include "kota/codec/raw_value.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/peer.h"
#include "llvm/ADT/DenseMap.h"
@@ -73,7 +73,7 @@ private:
std::string session_log_dir;
std::string init_options_json; ///< Raw JSON from initializationOptions, consumed once.
kota::task<> load_workspace();
void load_workspace();
using RawResult = kota::task<kota::codec::RawValue, kota::ipc::Error>;
};

View File

@@ -9,7 +9,7 @@
#include "syntax/token.h"
#include "kota/codec/raw_value.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/lsp/protocol.h"
#include "kota/ipc/protocol.h"

View File

@@ -15,6 +15,22 @@
namespace clice {
/// RAII guard that lowers the current process's scheduling priority and
/// restores it on destruction.
struct ScopedNice {
int saved;
explicit ScopedNice(int increment = 10) {
auto p = kota::sys::priority();
saved = p ? *p : 0;
kota::sys::set_priority(saved + increment);
}
~ScopedNice() {
kota::sys::set_priority(saved);
}
};
using kota::ipc::RequestResult;
using RequestContext = kota::ipc::BincodePeer::RequestContext;
@@ -283,7 +299,10 @@ int run_stateless_worker_mode(const std::string& worker_name, const std::string&
switch(params.kind) {
case K::BuildPCH: return handle_build_pch(params);
case K::BuildPCM: return handle_build_pcm(params);
case K::Index: return handle_index(params);
case K::Index: {
ScopedNice guard;
return handle_index(params);
}
case K::Completion: return handle_completion(params);
case K::SignatureHelp: return handle_signature_help(params);
}

View File

@@ -8,8 +8,7 @@
#include "compile/compilation.h"
#include "kota/codec/json/serializer.h"
#include "kota/codec/raw_value.h"
#include "kota/codec/json/json.h"
#include "kota/ipc/codec/json.h"
namespace clice {

View File

@@ -13,14 +13,13 @@ namespace {
/// Coroutine that drains a worker's stderr pipe.
/// Workers write their own log files, so this only captures unexpected output
/// (crash stacktraces, assertion failures, etc.) that bypasses spdlog.
/// (crash stacktraces, assertion failures, sanitizer reports, etc.).
kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
std::string buffer;
while(true) {
auto result = co_await stderr_pipe.read();
if(!result.has_value()) {
if(!result.has_value())
break;
}
auto& chunk = result.value();
if(chunk.empty())
break;
@@ -34,7 +33,7 @@ kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
break;
auto line = buffer.substr(pos, nl - pos);
if(!line.empty()) {
LOG_DEBUG("{} {}", prefix, line);
LOG_WARN("{} {}", prefix, line);
}
pos = nl + 1;
}
@@ -42,7 +41,7 @@ kota::task<> drain_stderr(kota::pipe stderr_pipe, std::string prefix) {
}
if(!buffer.empty()) {
LOG_DEBUG("{} {}", prefix, buffer);
LOG_WARN("{} {}", prefix, buffer);
}
}
@@ -108,24 +107,29 @@ bool WorkerPool::spawn_worker(const std::string& self_path,
});
auto& w = workers.back();
w.alive = true;
++alive_count_;
loop.schedule(w.peer->run());
return true;
}
bool WorkerPool::start(const WorkerPoolOptions& options) {
options_ = options;
log_dir_ = options.log_dir;
for(std::uint32_t i = 0; i < options.stateless_count; ++i) {
if(!spawn_worker(options.self_path, false, 0)) {
return false;
}
loop.schedule(monitor_worker(stateless_workers.size() - 1, false));
}
for(std::uint32_t i = 0; i < options.stateful_count; ++i) {
if(!spawn_worker(options.self_path, true, options.worker_memory_limit)) {
return false;
}
loop.schedule(monitor_worker(stateful_workers.size() - 1, true));
}
// Register evicted notification handler for each stateful worker
@@ -145,29 +149,24 @@ bool WorkerPool::start(const WorkerPoolOptions& options) {
kota::task<> WorkerPool::stop() {
LOG_INFO("WorkerPool stopping...");
shutting_down_ = true;
// Close output pipes to signal workers to exit gracefully
for(auto& w: stateless_workers) {
// Close output pipes to signal workers to exit gracefully.
for(auto& w: stateless_workers)
w.peer->close_output();
}
for(auto& w: stateful_workers) {
for(auto& w: stateful_workers)
w.peer->close_output();
}
// Send SIGTERM to all workers
for(auto& w: stateless_workers) {
// Send SIGTERM. monitor_worker coroutines handle the wait.
for(auto& w: stateless_workers)
w.proc.kill(SIGTERM);
}
for(auto& w: stateful_workers) {
for(auto& w: stateful_workers)
w.proc.kill(SIGTERM);
}
// Wait for all worker processes to exit
for(auto& w: stateless_workers) {
co_await w.proc.wait();
}
for(auto& w: stateful_workers) {
co_await w.proc.wait();
// Wait until all monitor_worker coroutines have finished.
if(alive_count_ > 0) {
all_exited_.reset();
co_await all_exited_.wait();
}
LOG_INFO("WorkerPool stopped");
@@ -198,7 +197,10 @@ std::size_t WorkerPool::assign_worker(std::uint32_t path_id) {
std::size_t WorkerPool::pick_least_loaded() {
std::size_t best = 0;
for(std::size_t i = 1; i < stateful_workers.size(); ++i) {
if(stateful_workers[i].owned_documents < stateful_workers[best].owned_documents) {
if(!stateful_workers[i].alive)
continue;
if(!stateful_workers[best].alive ||
stateful_workers[i].owned_documents < stateful_workers[best].owned_documents) {
best = i;
}
}
@@ -233,4 +235,127 @@ void WorkerPool::clear_owner(std::size_t worker_index) {
}
}
kota::task<> WorkerPool::monitor_worker(std::size_t index, bool stateful) {
auto& workers = stateful ? stateful_workers : stateless_workers;
auto& w = workers[index];
auto name = std::string(stateful ? "SF-" : "SL-") + std::to_string(index);
auto result = co_await w.proc.wait();
w.alive = false;
--alive_count_;
if(shutting_down_) {
if(alive_count_ == 0)
all_exited_.set();
co_return;
}
if(result.has_value()) {
auto& exit = result.value();
if(exit.term_signal != 0) {
LOG_ERROR("Worker {} killed by signal {} (restarts: {})",
name,
exit.term_signal,
w.restart_count);
} else {
LOG_ERROR("Worker {} exited with code {} (restarts: {})",
name,
exit.status,
w.restart_count);
}
} else {
LOG_ERROR("Worker {} lost: {} (restarts: {})",
name,
result.error().message(),
w.restart_count);
}
if(stateful)
clear_owner(index);
constexpr unsigned max_restarts = 5;
if(w.restart_count >= max_restarts) {
LOG_ERROR("Worker {} exceeded max restarts ({}), giving up", name, max_restarts);
co_return;
}
if(!respawn_worker(index, stateful)) {
LOG_ERROR("Worker {} respawn failed", name);
}
}
bool WorkerPool::respawn_worker(std::size_t index, bool stateful) {
auto& workers = stateful ? stateful_workers : stateless_workers;
auto old_restart_count = workers[index].restart_count + 1;
auto worker_name = std::string(stateful ? "SF-" : "SL-") + std::to_string(index);
// Close the old peer and retire it so its coroutines (run/write_loop)
// can finish naturally before the object is destroyed.
if(workers[index].peer) {
workers[index].peer->close();
retired_peers.push_back(std::move(workers[index].peer));
}
kota::process::options opts;
opts.file = options_.self_path;
if(stateful) {
opts.args = {options_.self_path,
"--mode",
"stateful-worker",
"--worker-memory-limit",
std::to_string(options_.worker_memory_limit)};
} else {
opts.args = {options_.self_path, "--mode", "stateless-worker"};
}
opts.args.push_back("--worker-name");
opts.args.push_back(worker_name);
if(!log_dir_.empty()) {
opts.args.push_back("--log-dir");
opts.args.push_back(log_dir_);
}
opts.streams = {
kota::process::stdio::pipe(true, false),
kota::process::stdio::pipe(false, true),
kota::process::stdio::pipe(false, true),
};
auto result = kota::process::spawn(opts, loop);
if(!result) {
LOG_ERROR("Failed to respawn worker {}: {}", worker_name, result.error().message());
return false;
}
auto& spawn = *result;
auto transport = std::make_unique<kota::ipc::StreamTransport>(std::move(spawn.stdout_pipe),
std::move(spawn.stdin_pipe));
auto peer = std::make_unique<kota::ipc::BincodePeer>(loop, std::move(transport));
std::string prefix = "[" + worker_name + "]";
loop.schedule(drain_stderr(std::move(spawn.stderr_pipe), prefix));
workers[index] = WorkerProcess{
.proc = std::move(spawn.proc),
.peer = std::move(peer),
.owned_documents = 0,
.alive = true,
.restart_count = old_restart_count,
};
auto& w = workers[index];
++alive_count_;
loop.schedule(w.peer->run());
if(stateful) {
w.peer->on_notification([this](const worker::EvictedParams& params) {
if(on_evicted)
on_evicted(params.path);
});
}
loop.schedule(monitor_worker(index, stateful));
LOG_INFO("Worker {} restarted (attempt {})", worker_name, old_restart_count);
return true;
}
} // namespace clice

View File

@@ -64,6 +64,8 @@ private:
kota::process proc;
std::unique_ptr<kota::ipc::BincodePeer> peer;
std::size_t owned_documents = 0;
bool alive = true;
unsigned restart_count = 0;
};
kota::event_loop& loop;
@@ -80,8 +82,19 @@ private:
void clear_owner(std::size_t worker_index);
std::size_t pick_least_loaded();
bool shutting_down_ = false;
std::size_t alive_count_ = 0;
kota::event all_exited_{true}; // Signalled when alive_count_ reaches 0.
WorkerPoolOptions options_;
std::string log_dir_;
/// Peers moved here during respawn so their coroutines can finish
/// before the object is destroyed.
llvm::SmallVector<std::unique_ptr<kota::ipc::BincodePeer>> retired_peers;
bool spawn_worker(const std::string& self_path, bool stateful, std::uint64_t memory_limit);
bool respawn_worker(std::size_t index, bool stateful);
kota::task<> monitor_worker(std::size_t index, bool stateful);
};
template <typename Params>
@@ -91,11 +104,10 @@ RequestResult<Params> WorkerPool::send_stateful(std::uint32_t path_id,
if(stateful_workers.empty()) {
co_return kota::outcome_error(kota::ipc::Error{"No stateful workers available"});
}
// No timeout: compile tasks run as detached tasks (loop.schedule) that
// are immune to LSP $/cancelRequest. Adding a timeout here would use
// kotatsu's with_token/when_any which has a spurious-cancellation bug
// that kills requests within milliseconds instead of the configured period.
auto idx = assign_worker(path_id);
if(!stateful_workers[idx].alive) {
co_return kota::outcome_error(kota::ipc::Error{"Assigned stateful worker is down"});
}
co_return co_await stateful_workers[idx].peer->send_request(params, opts);
}
@@ -105,9 +117,16 @@ RequestResult<Params> WorkerPool::send_stateless(const Params& params,
if(stateless_workers.empty()) {
co_return kota::outcome_error(kota::ipc::Error{"No stateless workers available"});
}
auto idx = next_stateless;
next_stateless = (next_stateless + 1) % stateless_workers.size();
co_return co_await stateless_workers[idx].peer->send_request(params, opts);
// Round-robin, skipping dead workers.
auto start = next_stateless;
for(std::size_t i = 0; i < stateless_workers.size(); ++i) {
auto idx = (start + i) % stateless_workers.size();
if(stateless_workers[idx].alive) {
next_stateless = (idx + 1) % stateless_workers.size();
co_return co_await stateless_workers[idx].peer->send_request(params, opts);
}
}
co_return kota::outcome_error(kota::ipc::Error{"All stateless workers are down"});
}
template <typename Params>
@@ -115,6 +134,8 @@ void WorkerPool::notify_stateful(std::uint32_t path_id, const Params& params) {
auto it = owner.find(path_id);
if(it == owner.end())
return;
if(!stateful_workers[it->second].alive)
return;
stateful_workers[it->second].peer->send_notification(params);
}

View File

@@ -110,7 +110,11 @@ async def client(
if workspace is not None:
init_options_marker = request.node.get_closest_marker("init_options")
init_options = init_options_marker.args[0] if init_options_marker else None
init_options = dict(init_options_marker.args[0]) if init_options_marker else {}
# Force cache_dir into the workspace so .clice/ cleanup prevents stale PCH.
project = dict(init_options.get("project", {}))
project.setdefault("cache_dir", str(workspace / ".clice"))
init_options["project"] = project
await c.initialize(workspace, initialization_options=init_options)
yield c
@@ -165,12 +169,17 @@ async def _shutdown_client(c: CliceClient) -> None:
try:
server = getattr(c, "_server", None)
if server and server.stderr:
stderr_data = await asyncio.wait_for(server.stderr.read(), timeout=2.0)
if stderr_data:
for line in stderr_data.decode("utf-8", errors="replace").splitlines():
if "[warn]" in line or "[error]" in line:
print(f"[server] {line}", flush=True)
if server:
if server.returncode is not None:
print(f"[server] exit code: {server.returncode}", flush=True)
if server.stderr:
stderr_data = await asyncio.wait_for(server.stderr.read(), timeout=2.0)
if stderr_data:
for line in stderr_data.decode(
"utf-8", errors="replace"
).splitlines():
if "[warn]" in line or "[error]" in line or "Sanitizer" in line:
print(f"[server] {line}", flush=True)
except Exception:
pass

View File

@@ -13,6 +13,9 @@ import re
import signal
import sys
import time
# Force line-buffered stdout so CI sees output immediately.
sys.stdout.reconfigure(line_buffering=True)
from pathlib import Path
from urllib.parse import quote, unquote
@@ -109,7 +112,9 @@ async def write_lsp_message(writer: asyncio.StreamWriter, payload: str):
await writer.drain()
async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool | None:
async def replay_one(
trace_path: Path, clice_bin: Path, timeout: int, wall_timeout: int = 300
) -> bool | None:
"""Replay a single trace. Returns True=PASS, False=FAIL, None=SKIP."""
records = load_trace(trace_path)
if not records:
@@ -179,8 +184,21 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
last_method = None
sent_count = 0
wall_deadline = wall_start + wall_timeout
def remaining_wall():
return max(0, wall_deadline - time.monotonic())
try:
for i, rec in enumerate(records):
if remaining_wall() <= 0:
elapsed = time.monotonic() - wall_start
print(
f" result: TIMEOUT (wall-clock {wall_timeout}s exceeded, {elapsed:.1f}s)"
)
success = False
break
if i > 0:
delay = rec["ts"] - records[i - 1]["ts"]
if delay > 0:
@@ -196,7 +214,7 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
try:
await asyncio.wait_for(
asyncio.gather(*pending.values(), return_exceptions=True),
timeout=timeout,
timeout=min(timeout, remaining_wall()),
)
except asyncio.TimeoutError:
elapsed = time.monotonic() - wall_start
@@ -210,7 +228,19 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
if msg_id is not None and method is not None:
pending[msg_id] = asyncio.get_event_loop().create_future()
await write_lsp_message(proc.stdin, rec["msg"])
try:
await asyncio.wait_for(
write_lsp_message(proc.stdin, rec["msg"]),
timeout=min(30, remaining_wall()),
)
except asyncio.TimeoutError:
elapsed = time.monotonic() - wall_start
print(
f" result: HANG (write blocked at {last_method},"
f" sent={sent_count}/{len(records)}, {elapsed:.1f}s)"
)
success = False
break
sent_count = i + 1
except (ConnectionError, BrokenPipeError):
@@ -231,7 +261,7 @@ async def replay_one(trace_path: Path, clice_bin: Path, timeout: int) -> bool |
try:
await asyncio.wait_for(
asyncio.gather(*pending.values(), return_exceptions=True),
timeout=timeout,
timeout=min(timeout, remaining_wall()),
)
except asyncio.TimeoutError:
elapsed = time.monotonic() - wall_start
@@ -294,7 +324,7 @@ async def async_main(args):
print(f"SKIP: {trace} (not found)")
skipped += 1
continue
result = await replay_one(trace, args.clice, args.timeout)
result = await replay_one(trace, args.clice, args.timeout, args.wall_timeout)
if result is None:
skipped += 1
elif result:
@@ -317,7 +347,16 @@ def main():
p.add_argument("traces", nargs="+", type=Path, help="JSONL trace files")
p.add_argument("--clice", required=True, type=Path, help="Path to clice binary")
p.add_argument(
"--timeout", type=int, default=120, help="Timeout in seconds (default: 120)"
"--timeout",
type=int,
default=120,
help="Per-request timeout in seconds (default: 120)",
)
p.add_argument(
"--wall-timeout",
type=int,
default=300,
help="Max wall-clock time per trace in seconds (default: 300)",
)
args = p.parse_args()
sys.exit(asyncio.run(async_main(args)))

View File

@@ -6,7 +6,7 @@
#include "support/filesystem.h"
#include "kota/codec/json/json.h"
#include "kota/codec/toml.h"
#include "kota/codec/toml/toml.h"
namespace clice::testing {
@@ -148,7 +148,7 @@ TEST_CASE(ApplyDefaults) {
EXPECT_EQ(*config.project.idle_timeout_ms, 3000);
EXPECT_EQ(config.project.max_active_file.value, 8);
EXPECT_EQ(config.project.stateful_worker_count.value, 2u);
EXPECT_EQ(config.project.stateless_worker_count.value, 3u);
EXPECT_GE(config.project.stateless_worker_count.value, 2u);
EXPECT_FALSE(config.project.cache_dir.empty());
EXPECT_FALSE(config.project.index_dir.empty());
EXPECT_FALSE(config.project.logging_dir.empty());

View File

@@ -5,7 +5,7 @@
#include "server/protocol.h"
#include "server/worker_test_helpers.h"
#include "kota/codec/raw_value.h"
#include "kota/codec/json/json.h"
namespace clice::testing {

View File

@@ -6,7 +6,6 @@
#include "server/worker_test_helpers.h"
#include "kota/codec/bincode/bincode.h"
#include "kota/codec/raw_value.h"
namespace clice::testing {