Repository navigation
Reduce multi-Raft control queue latency with a sixteen-frame window - #107
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Heartbeat bursts share the per-peer control queue with acknowledgements. Increase the outstanding control window from eight to sixteen frames so more independent groups can progress before replies arrive. Preserve FIFO authentication, individual reply refill, physical-write completion, deadlines, byte/item reservations, revocation and quorum acknowledgement. Update the real TLS tests to enforce the larger bound and exact failure/shutdown accounting.
Measured against main
5f92e2c7on three pinned 1-vCPU / 3-GiB Cellar nodes, with 500 leaders and 1,000 followers per node, 1,500 three-voter groups and RustFS. Each cohort uses fresh data, a cold Colima VM and two 180-second windows with eight closed-loop counter writers; timed mutations never retry.Observed pooled throughput rises 4.56% (242.58 → 253.65 TPS). The worst window p99 falls 160.83 → 79.99 ms; this is not a pooled percentile. Candidate p99 and mean control-queue delay improve in every matching window. One matching window has 1.17% lower candidate TPS. Shared-host compiler activity and roughly 15 GiB host swap mean the exact gain is not an isolated causal estimate.
All eight windows have zero errors. Every cohort verifies 4,500 exact replica values and 1,500 archive positions, then shuts down cleanly without OOM. Native validation passes 39 transport, 29 host and nine multi-group tests, including crash recovery. Formatting, layout, native-dependency and documentation checks pass, as does strict Clippy for all Cellar targets/features. Full workspace/release qualification is left to CI.
This qualifies the eight-writer workload, not maximum sustainable TPS. Earlier twenty-writer baseline runs still hit archive-admission refusals. Measurement notes and remaining priorities retain those limits and prioritize archive publication, SQLite phase profiling, batch flush reasons and idle-group scheduling.
The rejected manifest-reuse experiment and diagnostic probes are excluded. Raw results remain local; no files under
plans/multi-raft/evidence/are included. This follows the already-merged PR #98.