Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

### Fixed

- Fixed `FileStore` losing records when configured with a single segment: after the first compaction new writes went to a segment outside the configured range and were dropped on reload. `init()` now uses a minimum of two segments.
- Fixed `FileStore` compaction running with the active segment still open, which LittleFS and FatFS refuse to delete, and fixed the store being left without a writable segment (or with a stray out-of-range segment file) after a failed compaction (#6, #11).
- Fixed compaction recovery paths that could lose data: a power loss after the compacted segment was moved into place but before the journal was cleared deleted all data on the next boot; a failed rename while finalizing deleted all data and still reported success; recovery from a compaction interrupted mid-way kept a stale index, and a later compaction attempt could truncate the only copy of the interrupted compaction's records. A compaction interrupted by a write failure is now recovered immediately rather than at the next boot.

Expand Down
2 changes: 1 addition & 1 deletion docs/microstore_user_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ if (!store.init(fs, "/cfg")) {
Optional `init()` arguments:

- `clearOnInit` — wipe everything before opening (useful for tests).
- `segment_size`, `segment_count` — override the defaults at runtime (still bounded by the compile-time `USTORE_MAX_*` macros).
- `segment_size`, `segment_count` — override the defaults at runtime (still bounded by the compile-time `USTORE_MAX_*` macros). `segment_count` has a minimum of 2: compaction writes into seg0 and new records then go to seg1, so a smaller value is raised to 2.

Always call `store.close()` before destruction if you want a clean flush; the destructor does flush, but `close()` also closes the index and active segment explicitly.

Expand Down
8 changes: 8 additions & 0 deletions include/microStore/FileStore.h
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,14 @@ class BasicFileStore
{
if (segment_size > 0) _segment_size = segment_size;
if (segment_count > 0) _segment_count = segment_count;
// Compaction writes the live records into seg0 and new writes then go to
// seg1, so a store needs at least two segments. With one, every write
// after the first compaction lands in seg1, outside the range that
// compaction and index rebuilds scan, and is lost on the next reload.
if (_segment_count < 2) {
USTORE_LOG("[ustore] init: segment_count %u is below the minimum of 2, using 2\n", _segment_count);
_segment_count = 2;
}
USTORE_LOG("[ustore] init: Initializing FileStore with prefix=%s, segment_size=%u, segment_count=%u\n", prefix, _segment_size, _segment_count);

_filesystem = filesystem;
Expand Down
36 changes: 36 additions & 0 deletions test/test_file_store/test_file_store.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1104,6 +1104,41 @@ void test_dir_prefix_compact() {
void setUp() {}
void tearDown() {}

// A store needs two segments: compaction fills seg0 and new writes go to seg1.
// Asked for one, init() must use two, or every write after the first
// compaction lands outside the scanned range and is lost on reload.
void test_file_store_single_segment_count_keeps_data() {
reset_ram_fs();

uint32_t now = microStore::time() - 10;
char k[8], v[32];
{
microStore::FileStore store;
auto fs = make_ram_fs();
store.init(fs, "/p", /*clearOnInit=*/false, /*segment_size=*/256, /*segment_count=*/1);
// Overwrite ten keys six times: enough to fill a 256-byte segment
// several times over and force repeated compaction.
for (int i = 0; i < 60; i++) {
snprintf(k, sizeof(k), "k%02d", i % 10);
snprintf(v, sizeof(v), "value-%02d-xxxxxxxx", i);
TEST_ASSERT_TRUE(store.put(k, v, /*ttl=*/0, now));
}
}

microStore::FileStore reader;
auto fs = make_ram_fs();
reader.init(fs, "/p", /*clearOnInit=*/false, /*segment_size=*/256, /*segment_count=*/1);
TEST_ASSERT_EQUAL(10u, reader.size());
for (int i = 0; i < 10; i++) {
snprintf(k, sizeof(k), "k%02d", i);
snprintf(v, sizeof(v), "value-%02d-xxxxxxxx", 50 + i);
uint8_t buf[64]; uint16_t sz = sizeof(buf);
TEST_ASSERT_TRUE(reader.get(k, buf, &sz));
TEST_ASSERT_EQUAL(strlen(v), sz);
TEST_ASSERT_EQUAL(0, memcmp(buf, v, sz));
}
}

int runUnityTests(void) {
UNITY_BEGIN();
// TTL tests
Expand All @@ -1127,6 +1162,7 @@ int runUnityTests(void) {
// Additional edge-case tests
RUN_TEST(test_file_store_ttl_exists_expires);
RUN_TEST(test_file_store_compact_basic);
RUN_TEST(test_file_store_single_segment_count_keeps_data);
RUN_TEST(test_file_store_compact_keeps_store_writable);
RUN_TEST(test_file_store_compact_failure_keeps_store_writable);
RUN_TEST(test_file_store_threshold_compact_failure_keeps_store_writable);
Expand Down
Loading