Repository navigation
Use at least two segments so a single-segment store keeps its data - #18
Open
mclueppers wants to merge 1 commit into
Open
mclueppers wants to merge 1 commit into
mclueppers wants to merge 1 commit into
Conversation
Compaction writes the live records into seg0, and both the rotation and the threshold paths then move new writes to seg1. A store initialised with segment_count 1 therefore writes everything after its first compaction into seg1, outside the range that compaction and index rebuilds scan. That file grows without bound and its records are dropped on the next reload. This predates the recent compaction changes; 0.1.7 behaves the same. init() now raises a segment_count below 2 to 2, which covers counts set through the constructor, init() and USTORE_DEFAULT_SEGMENT_COUNT alike. Adds test_file_store_single_segment_count_keeps_data.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A
FileStoreinitialised withsegment_count == 1loses every record written after its first compaction. Compaction fillsseg0, and the rotation and threshold paths then write toseg1, which lies outside the range that compaction and index rebuilds scan. Fixes #.Solution
init()raises asegment_countbelow 2 to 2, with a log line. That covers a count set through the constructor, throughinit()or throughUSTORE_DEFAULT_SEGMENT_COUNT. Two is the real minimum of the design: one segment holds the compacted data and one takes new writes. I chose raising the value over refusing to init, so existing callers keep working; if you would ratherinit()fail, it's a one-line change.Compatibility
seg1falls inside the scanned range. I haven't tested whether records already stranded there are recovered on upgrade.Testing
pio test -e native, 113/113. The newtest_file_store_single_segment_count_keeps_datafails on master (Expected 10 Was 1) and passes on the branch.Docs
segment_countnoteCloses #17.