diff --git a/.changes/board-descriptor.md b/.changes/board-descriptor.md index 7e16b8426..86604bd63 100644 --- a/.changes/board-descriptor.md +++ b/.changes/board-descriptor.md @@ -1,5 +1,6 @@ --- kind: changed +pr: 530 --- Stamp a board descriptor into the firmware image instead of compiling each board in @@ -8,7 +9,7 @@ Firmware images now carry a small board descriptor: which board they are for, wh - Add a 256-byte board descriptor at a fixed offset of the app image, written by `scripts/stamp_board.py` after linking and validated on the device with a CRC - Read pins, LED flags, pin bypasses and the OTA board name from the descriptor at startup; the compiled image describes no board and has no pins until it is stamped -- Refuse an OTA download whose descriptor names a different board before it is made bootable, and roll back on first boot if the previous slot disagrees +- Refuse an OTA download whose descriptor is missing or names a different board before it is made bootable, and roll back on first boot if the previous slot disagrees - Boot into the captive portal with no pins, and say so on the serial console, when an image was never stamped or its descriptor is corrupt - Validate the descriptor at boot: chip matches the build, every pin is usable on the chip, no two features share a pin; an unusable LED pin disables that LED instead of failing startup - Move per-board pin bypass lists out of `Chipset.h` into an `OPENSHOCK_GPIO_BYPASS` line in each board fragment diff --git a/.changes/config-nvs.md b/.changes/config-nvs.md index 988cadbc8..1476f07bf 100644 --- a/.changes/config-nvs.md +++ b/.changes/config-nvs.md @@ -1,26 +1,23 @@ --- kind: changed +pr: 548 --- Keep the configuration in NVS instead of a littlefs file ## Release Note Settings are now stored in NVS -The hub keeps its settings in the chip's NVS key-value store instead of a single config file, and saves each setting on its own when it changes. Your WiFi networks, account link and other settings from 1.x are imported automatically on the first boot after updating. +The hub keeps its settings in the chip's NVS key-value store instead of a single config file, and saves each setting on its own when it changes. Hubs on 1.x get here through the migration firmware, which carries your WiFi networks, account link and other settings over. - Each module owns its section of the config (`config/Section.h`); every section type meets the `SectionValue` concept, with its defaults defined in one place and its validation run on every value the section takes in -- On the 2.x partition table the store is the `cfg` partition (the 1.x littlefs `config` partition retyped in place); on the 1.x table it lives in the shared `nvs` partition until the layout is migrated, and then moves into `cfg` -- An empty store is filled once from the 1.x config file, which is never modified, otherwise from the defaults +- The store is the `cfg` partition; an empty one reads as the defaults, and a setting is only written once it changes - WiFi credentials are stored one per key; the WiFi driver's own NVS store is compiled out - A section that can't be read runs on its defaults without erasing what is stored, and the save after a failed one rewrites every key; a single WiFi credential that doesn't read is dropped on its own, so the rest still load -- The store is marked filled only once every section reads back from flash as written; a first fill cut short is redone on the next boot -- A store written by newer firmware is loaded instead of reset, and a `cfg` partition in a newer NVS format is never erased -- A section whose NVS namespace fails to open is a failed load, not a load of the defaults: the defaults are only taken when the namespace was never written. The store likewise never fills or cleans up when its own marker cannot be read +- A `cfg` partition in a newer NVS format is never erased +- A section whose NVS namespace fails to open is a failed load, not a load of the defaults: the defaults are only taken when the namespace was never written - A boolean field of the wrong type in a `jsonconfig set` document rejects the document instead of storing the default, in every section - The config accessors return the value itself (a section always has one); the WiFi section hands a network saved without an id the lowest free one on every path, so `networks` over serial no longer allocates ids itself - An OTA check interval of zero and a hostname esp_netif would refuse become their defaults on every path (NVS, JSON and the setters), not only through the captive portal -- `Config::MigrateToCfgPartition` retypes the `config` partition as `cfg`; nothing calls it yet ## Notices -- warning: **1.x firmware does not see settings changed after this update.** Downgrading to 1.x brings back the settings the hub had before updating, as long as the partition layout has not been migrated yet. Re-enter any WiFi networks or pairing you changed since. -- warning: Until the partition layout is migrated the settings live in the chip's shared NVS partition. If that partition has to be erased (it fills up, or a newer firmware wrote it in a format this one cannot read), the hub imports the 1.x settings again and anything changed since the update is lost. +- important: **Hubs on 1.x must install the migration firmware first.** This release only runs on the new partition table, so a 1.x hub cannot update to it directly. Install the migration firmware, then update as usual. diff --git a/.changes/ota-repository-domain.md b/.changes/ota-repository-domain.md new file mode 100644 index 000000000..6ed3598cf --- /dev/null +++ b/.changes/ota-repository-domain.md @@ -0,0 +1,9 @@ +--- +kind: changed +--- +Move hubs from the 1.x firmware CDN domains to the repository server for OTA updates + +## Release Note +Hubs that stored a 1.x update domain now look for updates on the repository server + +The 1.x domains `firmware.openshock.org` and `fw.openshock.org` serve flat files for 1.x hubs and have no repository API, so a hub that kept one could not update again. A hub storing either now uses `repo.openshock.dev`, which is also the new default. A custom domain is left as it is. diff --git a/.changes/plain-build-partition-table.md b/.changes/plain-build-partition-table.md index 501b0e866..7dd952c7f 100644 --- a/.changes/plain-build-partition-table.md +++ b/.changes/plain-build-partition-table.md @@ -1,7 +1,8 @@ --- kind: fixed +pr: 536 --- -Build without a board fragment with the 4 MB OTA partition table, and skip staticfs cleanly without one +Build without a board fragment with the 4 MB OTA partition table ## Release Note -A build without a board fragment (the VS Code ESP-IDF extension, a plain `idf.py build`) fell back to ESP-IDF's stock 2 MB flash and 1 MB single-app partition table. The app overflowed it, and the staticfs target then tried to size an image from a `static0` partition the table did not have, because its guard compared an undefined CMake variable name to the empty string. `sdkconfig.defaults` now carries the 4 MB flash and the `partitions/ota_4mb.csv` table every board already uses, and a table without `static0` skips the staticfs target with the intended warning. +A build without a board fragment (the VS Code ESP-IDF extension, a plain `idf.py build`) fell back to ESP-IDF's stock 2 MB flash and 1 MB single-app partition table. The app overflowed it, and the staticfs target then tried to size an image from a partition the table did not have, because its guard compared an undefined CMake variable name to the empty string. `sdkconfig.defaults` now carries the 4 MB flash and the `partitions/ota_4mb.csv` table every board already uses, and a table without that partition now stops the build with a clear error. diff --git a/.changes/remove-hubconfig.md b/.changes/remove-hubconfig.md new file mode 100644 index 000000000..84c220d63 --- /dev/null +++ b/.changes/remove-hubconfig.md @@ -0,0 +1,12 @@ +--- +kind: removed +--- +Remove the FlatBuffers form of the config and the rawconfig serial command + +## Release Note +The rawconfig serial command is gone + +Use `jsonconfig` to read or set the whole config over serial. + +- The HubConfig schema, `Config::Wire` and every section's `ToFlatbuffers` are removed +- The portal's Ready message carries only what it shows: the RF TX pin, the E-Stop state, the saved SSIDs and the OTA settings diff --git a/.changes/storage-migration.md b/.changes/storage-migration.md deleted file mode 100644 index 20b482b46..000000000 --- a/.changes/storage-migration.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -kind: changed ---- -Run the 1.x -> 2.x storage migration on hubs still on the 1.x partition table - -## Release Note -Hubs updated from 1.x move to the new storage layout on their own - -On the first boot after updating, a hub still on the 1.x partition table turns its `config` partition into the NVS config store and restarts; once online it downloads this release's portal assets over `static0`, turns it into the `www` partition and restarts again. Nothing moves on flash and the settings carry over. Do not unplug the hub while it says so on the serial console. - -- `components/migration` pipelines `Config::MigrateToCfgPartition` and `Www::MigrateToWwwPartition` in one task started from `app_main` after setup; on the 2.x table it starts nothing -- The OTA manager has one flash-write lock: an install holds it from its start to its restart, a migration step for its whole run, and whichever comes second backs off -- `migrate` over serial reports whether a migration is pending, `migrate now` retries at once, and `migrate www ` runs the www step from an image of your own, for builds with no published release - -## Notices -- warning: **Downgrading to 1.x is no longer possible once the migration has run.** The 1.x config file and portal assets are gone from flash; a downgrade starts from factory defaults. diff --git a/.changes/www-image.md b/.changes/www-image.md index b37b591c5..d0430f7ba 100644 --- a/.changes/www-image.md +++ b/.changes/www-image.md @@ -1,15 +1,13 @@ --- kind: changed +pr: 549 --- -Serve the setup portal from a flat www image where the partition table has a www partition +Serve the setup portal from a flat www image ## Release Note Setup portal assets get their own www partition -The firmware's partition table replaces the 1.x littlefs `static0` partition with a `www` partition (type 0x40, same offset and size) that holds the setup portal's assets as one flat, read-only image. The firmware maps the image once and serves it by pointer, checked against a SHA-256 before anything is served. The image is identical for every board, so an update skips re-flashing it when the one on flash already matches. +The firmware's partition table has a `www` partition (type 0x40) that holds the setup portal's assets as one flat, read-only image. The firmware maps the image once and serves it by pointer, checked against a SHA-256 before anything is served. The image is identical for every board, so an update skips re-flashing it when the one on flash already matches. -Hubs in the field keep their 1.x partition table for now: this release installs over the air like any other, keeps serving the portal from the littlefs `static0`, and keeps updating it the way 1.x firmware does. The in-place move onto the `www` partition ships with the release that runs the storage migrations. -- `partitions/ota_4mb.csv` turns `static0` into `www`; the 1.x table is kept as `partitions/legacy/ota_4mb_v1.csv`. -- `scripts/gen_staticfs.py` packs the frontend build into `www.bin` (`scripts/utils/wwwimage.py`) and, with `--legacy-output`, into the 1.x littlefs `staticfs.bin`. A 1.x-numbered release publishes the littlefs image as its `staticfs` artifact, since 1.x firmware's own update code installs it by mounting it; from 2.0.0 the artifact is `www.bin`. -- The over-the-air update flashes whichever of `www` or `static0` the running table has and verifies it in that partition's format. -- `components/www` adds the in-place migration (`Www::MigrateToWwwPartition`) that retypes `static0` as `www` and downloads the image over it; nothing calls it yet. +- `scripts/gen_staticfs.py` packs the frontend build into `www.bin` (`scripts/utils/wwwimage.py`), published as the release's `staticfs` artifact. +- The over-the-air update flashes the `www` partition and verifies the image there, and refuses a download that is not a www image before erasing anything. diff --git a/.github/actions/build-staticfs/action.yml b/.github/actions/build-staticfs/action.yml index c73aed974..3e741ff2c 100644 --- a/.github/actions/build-staticfs/action.yml +++ b/.github/actions/build-staticfs/action.yml @@ -1,20 +1,18 @@ name: build-staticfs -description: Pack the frontend build into the board-independent www asset image, and the 1.x littlefs staticfs image +description: Pack the frontend build into the board-independent www asset image runs: using: composite steps: - uses: ./.github/actions/install-python-deps - # One flat image serves every board; checked to fit the www partition of partitions/ota_4mb.csv. The littlefs - # staticfs.bin is what a 1.x hub's OTA installs, published for 1.x-numbered releases only. - - name: Build asset images + # One flat image serves every board; checked to fit the www partition of partitions/ota_4mb.csv. + - name: Build asset image shell: bash run: | python scripts/gen_staticfs.py \ --frontend-dir frontend \ --output www.bin \ - --legacy-output staticfs.bin \ --partition-csv partitions/ota_4mb.csv \ --skip-frontend-build @@ -22,8 +20,6 @@ runs: uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 with: name: firmware_staticfs - path: | - www.bin - staticfs.bin + path: www.bin retention-days: 7 if-no-files-found: error diff --git a/.github/actions/repo-server-publish/action.yml b/.github/actions/repo-server-publish/action.yml index 3a30f045b..7dd3ea3c9 100644 --- a/.github/actions/repo-server-publish/action.yml +++ b/.github/actions/repo-server-publish/action.yml @@ -12,7 +12,7 @@ inputs: description: 'init opens the release, publish uploads and promotes it, abort closes one the build never reached' version: required: false - description: 'Firmware version, strict semver (init; publish, where it picks the staticfs image)' + description: 'Firmware version, strict semver (init)' channel: required: false description: 'Release channel: stable, beta or develop (init)' diff --git a/.github/workflows/ci-build.yml b/.github/workflows/ci-build.yml index 390111f81..55f26a2f7 100644 --- a/.github/workflows/ci-build.yml +++ b/.github/workflows/ci-build.yml @@ -295,7 +295,6 @@ jobs: token: ${{ steps.token.outputs.token }} release-id: ${{ needs.stage.outputs.release-id }} boards: ${{ needs.plan.outputs.board-list }} - version: ${{ needs.plan.outputs.version }} mode: publish # Hands the staged version back when the run fails, is cancelled or never publishes; the server's TTL is only a backstop. diff --git a/.github/workflows/cpp-linter.yml b/.github/workflows/cpp-linter.yml index 6c556fc91..c644b5370 100644 --- a/.github/workflows/cpp-linter.yml +++ b/.github/workflows/cpp-linter.yml @@ -42,7 +42,7 @@ jobs: version: 21 tidy-checks: '-*' # clang-tidy rejects the gcc-only flags of the ESP-IDF compile DB, CodeQL covers static analysis files-changed-only: false # GitHub returns 406 for very large PR diffs - ignore: '.github|build|managed_components|components/littlefs|components/flatbuffers|components/serialization/include/serialization/_fbs' + ignore: '.github|build|managed_components|components/flatbuffers|components/serialization/include/serialization/_fbs' thread-comments: ${{ github.event_name == 'pull_request' && 'update' || 'false' }} # Commit comments on push need contents:write # Advisory on beta and master (pushes and PRs targeting them): findings are still reported, but they don't fail the job diff --git a/.gitmodules b/.gitmodules index 153edf041..4db97c5d2 100644 --- a/.gitmodules +++ b/.gitmodules @@ -1,7 +1,7 @@ [submodule "schemas"] path = schemas url = https://github.com/OpenShock/flatbuffers-schemas - branch = local-comms-revamp + branch = master [submodule "frontend/packages/svelte-core"] path = frontend/packages/svelte-core url = git@github.com:OpenShock/svelte-core.git diff --git a/CMakeLists.txt b/CMakeLists.txt index 00d754846..e7df23693 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -187,27 +187,25 @@ if(NOT CMAKE_BINARY_DIR STREQUAL _os_compile_db_dir) endif() # --- Captive-portal asset image (www) ---------------------------------------- -# gen_staticfs.py packs the frontend into the flat `www` image; skipped when the partition table has no such partition. +# gen_staticfs.py packs the frontend into the flat `www` image, which the firmware needs for its captive portal. partition_table_get_partition_info(_www_offset "--partition-name www" "offset") -partition_table_get_partition_info(_www_size "--partition-name www" "size") -if("${_www_offset}" STREQUAL "" OR "${_www_size}" STREQUAL "") - message(WARNING "No `www` partition in the active partition table; " - "skipping asset image generation.") -else() - set(_www_image ${CMAKE_BINARY_DIR}/www.bin) - - # CI never builds this target: its staticfs job runs gen_staticfs.py --skip-frontend-build once for every variant. - add_custom_target(www ALL - BYPRODUCTS ${_www_image} - COMMAND ${_os_python} ${CMAKE_CURRENT_LIST_DIR}/scripts/gen_staticfs.py - --frontend-dir ${CMAKE_CURRENT_LIST_DIR}/frontend - --output ${_www_image} - --max-size ${_www_size} - COMMENT "Building captive-portal asset image (www.bin) for www" - VERBATIM USES_TERMINAL) - - # Register the image with the flasher so `idf.py flash` writes it to the - # www offset alongside the app and partition table. - esptool_py_flash_target_image(flash "www" "${_www_offset}" "${_www_image}") +if("${_www_offset}" STREQUAL "") + message(FATAL_ERROR "No `www` partition in the active partition table.") endif() + +set(_www_image ${CMAKE_BINARY_DIR}/www.bin) + +# CI never builds this target: its staticfs job runs gen_staticfs.py --skip-frontend-build once for every variant. +add_custom_target(www ALL + BYPRODUCTS ${_www_image} + COMMAND ${_os_python} ${CMAKE_CURRENT_LIST_DIR}/scripts/gen_staticfs.py + --frontend-dir ${CMAKE_CURRENT_LIST_DIR}/frontend + --output ${_www_image} + --partition-csv ${CMAKE_CURRENT_LIST_DIR}/${CONFIG_PARTITION_TABLE_CUSTOM_FILENAME} + COMMENT "Building captive-portal asset image (www.bin) for www" + VERBATIM USES_TERMINAL) + +# Register the image with the flasher so `idf.py flash` writes it to the +# www offset alongside the app and partition table. +esptool_py_flash_target_image(flash "www" "${_www_offset}" "${_www_image}") diff --git a/boards/DFRobot-Firebeetle2-ESP32E.defaults b/boards/DFRobot-Firebeetle2-ESP32E.defaults index fb9a28619..63f0342ad 100644 --- a/boards/DFRobot-Firebeetle2-ESP32E.defaults +++ b/boards/DFRobot-Firebeetle2-ESP32E.defaults @@ -4,10 +4,6 @@ CONFIG_IDF_TARGET="esp32" CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y -# OpenShock uses a single 4 MB OTA partition layout (fits every board's flash). -CONFIG_PARTITION_TABLE_CUSTOM=y -CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions/ota_4mb.csv" - # Hardware pins. Not Kconfig: bare OPENSHOCK_* lines, stripped from the sdkconfig fragment by # scripts/build.py and stamped into the app image after linking by scripts/stamp_board.py, # so every board on a chip compiles identical objects and one image can serve them all. diff --git a/boards/NodeMCU-32S.defaults b/boards/NodeMCU-32S.defaults index 3aa0ac044..b144b5f22 100644 --- a/boards/NodeMCU-32S.defaults +++ b/boards/NodeMCU-32S.defaults @@ -4,10 +4,6 @@ CONFIG_IDF_TARGET="esp32" CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y -# OpenShock uses a single 4 MB OTA partition layout (fits every board's flash). -CONFIG_PARTITION_TABLE_CUSTOM=y -CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions/ota_4mb.csv" - # Hardware pins. Not Kconfig: bare OPENSHOCK_* lines, stripped from the sdkconfig fragment by # scripts/build.py and stamped into the app image after linking by scripts/stamp_board.py, # so every board on a chip compiles identical objects and one image can serve them all. diff --git a/boards/OpenShock-Core-V1.defaults b/boards/OpenShock-Core-V1.defaults index 1cac0ee28..c4611ada2 100644 --- a/boards/OpenShock-Core-V1.defaults +++ b/boards/OpenShock-Core-V1.defaults @@ -4,10 +4,6 @@ CONFIG_IDF_TARGET="esp32s3" CONFIG_ESPTOOLPY_FLASHSIZE_8MB=y -# OpenShock uses a single 4 MB OTA partition layout (fits every board's flash). -CONFIG_PARTITION_TABLE_CUSTOM=y -CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions/ota_4mb.csv" - # Hardware pins. Not Kconfig: bare OPENSHOCK_* lines, stripped from the sdkconfig fragment by # scripts/build.py and stamped into the app image after linking by scripts/stamp_board.py, # so every board on a chip compiles identical objects and one image can serve them all. diff --git a/boards/OpenShock-Core-V2.defaults b/boards/OpenShock-Core-V2.defaults index 9c4d2d3da..2fede312c 100644 --- a/boards/OpenShock-Core-V2.defaults +++ b/boards/OpenShock-Core-V2.defaults @@ -4,11 +4,6 @@ CONFIG_IDF_TARGET="esp32s3" CONFIG_ESPTOOLPY_FLASHSIZE_8MB=y -# OpenShock custom OTA partition layout. There is no dedicated 8 MB table yet, so -# reuse the 4 MB OTA layout; it fits within 8 MB (upper flash currently unused). -CONFIG_PARTITION_TABLE_CUSTOM=y -CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions/ota_4mb.csv" - # Native USB only (no USB-UART bridge): console on the built-in USB-Serial-JTAG, so the # serial console reads commands from the port the host actually sees. CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y diff --git a/boards/Pishock-2023.defaults b/boards/Pishock-2023.defaults index 6b41aeebb..1ad66e4a3 100644 --- a/boards/Pishock-2023.defaults +++ b/boards/Pishock-2023.defaults @@ -4,10 +4,6 @@ CONFIG_IDF_TARGET="esp32" CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y -# OpenShock uses a single 4 MB OTA partition layout (fits every board's flash). -CONFIG_PARTITION_TABLE_CUSTOM=y -CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions/ota_4mb.csv" - # Hardware pins. Not Kconfig: bare OPENSHOCK_* lines, stripped from the sdkconfig fragment by # scripts/build.py and stamped into the app image after linking by scripts/stamp_board.py, # so every board on a chip compiles identical objects and one image can serve them all. diff --git a/boards/Pishock-Lite-2021.defaults b/boards/Pishock-Lite-2021.defaults index 653a3ef6e..c661161b1 100644 --- a/boards/Pishock-Lite-2021.defaults +++ b/boards/Pishock-Lite-2021.defaults @@ -4,10 +4,6 @@ CONFIG_IDF_TARGET="esp32" CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y -# OpenShock uses a single 4 MB OTA partition layout (fits every board's flash). -CONFIG_PARTITION_TABLE_CUSTOM=y -CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions/ota_4mb.csv" - # Hardware pins. Not Kconfig: bare OPENSHOCK_* lines, stripped from the sdkconfig fragment by # scripts/build.py and stamped into the app image after linking by scripts/stamp_board.py, # so every board on a chip compiles identical objects and one image can serve them all. diff --git a/boards/Seeed-Xiao-ESP32C3.defaults b/boards/Seeed-Xiao-ESP32C3.defaults index ace0df26a..4da70d612 100644 --- a/boards/Seeed-Xiao-ESP32C3.defaults +++ b/boards/Seeed-Xiao-ESP32C3.defaults @@ -4,10 +4,6 @@ CONFIG_IDF_TARGET="esp32c3" CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y -# OpenShock uses a single 4 MB OTA partition layout (fits every board's flash). -CONFIG_PARTITION_TABLE_CUSTOM=y -CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions/ota_4mb.csv" - # Native USB only (no USB-UART bridge): console on the built-in USB-Serial-JTAG, so the # serial console reads commands from the port the host actually sees. CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y diff --git a/boards/Seeed-Xiao-ESP32S3.defaults b/boards/Seeed-Xiao-ESP32S3.defaults index 79ebb5751..d9cdea1cc 100644 --- a/boards/Seeed-Xiao-ESP32S3.defaults +++ b/boards/Seeed-Xiao-ESP32S3.defaults @@ -4,10 +4,6 @@ CONFIG_IDF_TARGET="esp32s3" CONFIG_ESPTOOLPY_FLASHSIZE_8MB=y -# OpenShock uses a single 4 MB OTA partition layout (fits every board's flash). -CONFIG_PARTITION_TABLE_CUSTOM=y -CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions/ota_4mb.csv" - # Native USB only (no USB-UART bridge): console on the built-in USB-Serial-JTAG, so the # serial console reads commands from the port the host actually sees. CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y diff --git a/boards/Waveshare_esp32_s3_zero.defaults b/boards/Waveshare_esp32_s3_zero.defaults index 02c483206..23efb0564 100644 --- a/boards/Waveshare_esp32_s3_zero.defaults +++ b/boards/Waveshare_esp32_s3_zero.defaults @@ -4,10 +4,6 @@ CONFIG_IDF_TARGET="esp32s3" CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y -# OpenShock uses a single 4 MB OTA partition layout (fits every board's flash). -CONFIG_PARTITION_TABLE_CUSTOM=y -CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions/ota_4mb.csv" - # Native USB only (no USB-UART bridge): console on the built-in USB-Serial-JTAG, so the # serial console reads commands from the port the host actually sees. CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y diff --git a/boards/Wemos-D1-Mini-ESP32.defaults b/boards/Wemos-D1-Mini-ESP32.defaults index 20dce2c7c..e1c58316d 100644 --- a/boards/Wemos-D1-Mini-ESP32.defaults +++ b/boards/Wemos-D1-Mini-ESP32.defaults @@ -4,10 +4,6 @@ CONFIG_IDF_TARGET="esp32" CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y -# OpenShock uses a single 4 MB OTA partition layout (fits every board's flash). -CONFIG_PARTITION_TABLE_CUSTOM=y -CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions/ota_4mb.csv" - # Hardware pins. Not Kconfig: bare OPENSHOCK_* lines, stripped from the sdkconfig fragment by # scripts/build.py and stamped into the app image after linking by scripts/stamp_board.py, # so every board on a chip compiles identical objects and one image can serve them all. diff --git a/boards/Wemos-Lolin-S2-Mini.defaults b/boards/Wemos-Lolin-S2-Mini.defaults index dc7c1801f..51d21652c 100644 --- a/boards/Wemos-Lolin-S2-Mini.defaults +++ b/boards/Wemos-Lolin-S2-Mini.defaults @@ -4,10 +4,6 @@ CONFIG_IDF_TARGET="esp32s2" CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y -# OpenShock uses a single 4 MB OTA partition layout (fits every board's flash). -CONFIG_PARTITION_TABLE_CUSTOM=y -CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions/ota_4mb.csv" - # Native USB only (no USB-UART bridge, and the S2 has no USB-Serial-JTAG): console on the # ROM USB-OTG CDC, so the serial console reads commands from the port the host sees. CONFIG_ESP_CONSOLE_USB_CDC=y diff --git a/boards/Wemos-Lolin-S3-Mini.defaults b/boards/Wemos-Lolin-S3-Mini.defaults index a75258a7a..ed35fada3 100644 --- a/boards/Wemos-Lolin-S3-Mini.defaults +++ b/boards/Wemos-Lolin-S3-Mini.defaults @@ -4,10 +4,6 @@ CONFIG_IDF_TARGET="esp32s3" CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y -# OpenShock uses a single 4 MB OTA partition layout (fits every board's flash). -CONFIG_PARTITION_TABLE_CUSTOM=y -CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions/ota_4mb.csv" - # Native USB only (no USB-UART bridge): console on the built-in USB-Serial-JTAG, so the # serial console reads commands from the port the host actually sees. CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y diff --git a/boards/Wemos-Lolin-S3.defaults b/boards/Wemos-Lolin-S3.defaults index d0cd3c136..5a0ae49ea 100644 --- a/boards/Wemos-Lolin-S3.defaults +++ b/boards/Wemos-Lolin-S3.defaults @@ -4,10 +4,6 @@ CONFIG_IDF_TARGET="esp32s3" CONFIG_ESPTOOLPY_FLASHSIZE_16MB=y -# OpenShock uses a single 4 MB OTA partition layout (fits every board's flash). -CONFIG_PARTITION_TABLE_CUSTOM=y -CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions/ota_4mb.csv" - # Hardware pins. Not Kconfig: bare OPENSHOCK_* lines, stripped from the sdkconfig fragment by # scripts/build.py and stamped into the app image after linking by scripts/stamp_board.py, # so every board on a chip compiles identical objects and one image can serve them all. diff --git a/components/captive_portal/CMakeLists.txt b/components/captive_portal/CMakeLists.txt index c4fd914b7..6158ad918 100644 --- a/components/captive_portal/CMakeLists.txt +++ b/components/captive_portal/CMakeLists.txt @@ -17,10 +17,10 @@ idf_component_register( "src/captiveportal/WebSocketEndpoint.cpp" INCLUDE_DIRS "include" PRIV_INCLUDE_DIRS "src" - REQUIRES config serialization # its config section (captiveportal/CaptivePortalConfig.h) + REQUIRES config # its config section (captiveportal/CaptivePortalConfig.h) PRIV_REQUIRES core device_control # wifi/gateway/message handlers/AppHooks; Config::; CommandHandler/EStopManager common chipset logging temporal osjson serialization # foundation utilities, json, HubToLocalMessage fbs - dns_server fs http hwutil rfc8908 www # DNSServer, LfsPartition (1.x layout), ContentTypes, PartitionUtils, RFC8908Handler, the www image + dns_server http rfc8908 www # DNSServer, ContentTypes, RFC8908Handler, the www image esp_http_server esp_netif esp_timer esp_wifi ) diff --git a/components/captive_portal/include/captiveportal/CaptivePortalConfig.h b/components/captive_portal/include/captiveportal/CaptivePortalConfig.h index 8a5de62df..bcb35c833 100644 --- a/components/captive_portal/include/captiveportal/CaptivePortalConfig.h +++ b/components/captive_portal/include/captiveportal/CaptivePortalConfig.h @@ -4,7 +4,6 @@ // (config/Section.h); namespace "portal", key always (u8). #include "config/Section.h" -#include "serialization/_fbs/HubConfig_generated.h" #include @@ -22,8 +21,6 @@ namespace OpenShock::Config { /// @brief No rules to apply yet. The section runs it on every value it takes in. void Normalize() { } - [[nodiscard]] flatbuffers::Offset ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const; - /// @brief Reads this section from `partition`; a missing key is its default, a read error fails. bool LoadNvs(const char* partition); /// @brief Writes the keys that differ from `previous` (every key when null) to `partition`. diff --git a/components/captive_portal/src/captiveportal/CaptivePortalConfig.cpp b/components/captive_portal/src/captiveportal/CaptivePortalConfig.cpp index d3bd4f2d7..5bbd9b854 100644 --- a/components/captive_portal/src/captiveportal/CaptivePortalConfig.cpp +++ b/components/captive_portal/src/captiveportal/CaptivePortalConfig.cpp @@ -11,13 +11,12 @@ const char* const TAG = "Config::CaptivePortalConfig"; #include "json/Json.h" using namespace OpenShock::Config; -namespace Fbs = OpenShock::Serialization::Configuration; static constexpr const char* kNamespace = "portal"; static constexpr const char* kKeyAlways = "always"; static constexpr const char* kNamespaces[] = {kNamespace}; -static Section s_section({.name = "captivePortal", .order = 2, .hubConfigField = Fbs::HubConfig::VT_CAPTIVE_PORTAL, .namespaces = kNamespaces}); +static Section s_section({.name = "captivePortal", .order = 2, .namespaces = kNamespaces}); CaptivePortalConfig::CaptivePortalConfig() : alwaysEnabled(false) @@ -29,11 +28,6 @@ CaptivePortalConfig::CaptivePortalConfig(bool alwaysEnabled) { } -flatbuffers::Offset CaptivePortalConfig::ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const -{ - return Serialization::Configuration::CreateCaptivePortalConfig(builder, alwaysEnabled); -} - bool CaptivePortalConfig::FromJSON(JSON::JsonView json) { if (auto done = Parse::ObjectOrDefault(*this, json, TAG)) { @@ -65,7 +59,7 @@ bool CaptivePortalConfig::LoadNvs(const char* partition) bool CaptivePortalConfig::SaveNvs(const char* partition, const CaptivePortalConfig* previous) const { - auto handle = Nvs::Handle::Open(partition, kNamespace, true); + auto handle = Nvs::Handle::OpenForWrite(partition, kNamespace); if (!handle) { return false; } diff --git a/components/captive_portal/src/captiveportal/Manager.cpp b/components/captive_portal/src/captiveportal/Manager.cpp index 7cf9986c4..a9da284aa 100644 --- a/components/captive_portal/src/captiveportal/Manager.cpp +++ b/components/captive_portal/src/captiveportal/Manager.cpp @@ -50,7 +50,7 @@ enum : uint32_t { struct ManagerState { bool alwaysEnabled = false; bool closeWhenOnline = false; // Setup finished or the backend disabled the portal: close it once online - bool forceClosed = false; // OTA is flashing the static filesystem; cleared by ReleaseForceClose() + bool forceClosed = false; // OTA is flashing the www image; cleared by ReleaseForceClose() bool forceCloseAckOwed = false; // ForceClose() is waiting for s_forceCloseDone int64_t startupGraceEnd = 0; // esp_timer time until which the portal must not open; 0 = no grace period int64_t autoCloseAt = 0; // esp_timer time at which an idle portal closes; 0 = not armed @@ -257,7 +257,7 @@ static void applyCommands(ManagerState& state, uint32_t commands) } // Runs the captive-portal state machine once. This does heavy work (starting the portal constructs -// CaptivePortalInstance, which mounts LittleFS and spins up the HTTP/WS/DNS servers), so it runs on the dedicated +// CaptivePortalInstance, which maps the www image and spins up the HTTP/WS/DNS servers), so it runs on the dedicated // manager task, never on the esp_timer task, whose 3.5KB stack would overflow. static void captiveportal_tick(ManagerState& state) { @@ -323,8 +323,8 @@ static void captiveportal_tick(ManagerState& state) state.autoCloseAt = 0; } - // Open portal if not running and device needs setup. Not while a retired instance still holds the static - // filesystem and ports 80/53: the new one would fail to start and flap the AP on every tick. + // Open portal if not running and device needs setup. Not while a retired instance still holds the www + // image and ports 80/53: the new one would fail to start and flap the AP on every tick. if (!running && allDestroyed && (state.alwaysEnabled || !CommandHandler::Ok() || !isDeviceFullyConfigured())) { captiveportal_start(); } @@ -435,7 +435,7 @@ bool CaptivePortal::ForceClose(uint32_t timeoutMs) xSemaphoreTake(s_forceCloseDone, 0); // Drop an acknowledgement left over from an earlier, timed-out call sendCommand(kCmdForceClose); - // The manager task stops the portal and answers once it is fully torn down (filesystem unmounted). It stays + // The manager task stops the portal and answers once it is fully torn down (www image unmounted). It stays // force-closed even if this times out. return xSemaphoreTake(s_forceCloseDone, pdMS_TO_TICKS(timeoutMs)) == pdTRUE; } diff --git a/components/captive_portal/src/captiveportal/StaticFileServer.cpp b/components/captive_portal/src/captiveportal/StaticFileServer.cpp index 6f978f40b..7d61c1a58 100644 --- a/components/captive_portal/src/captiveportal/StaticFileServer.cpp +++ b/components/captive_portal/src/captiveportal/StaticFileServer.cpp @@ -6,88 +6,20 @@ const char* const TAG = "CaptivePortalStatic"; #include "captiveportal/HttpUtils.h" #include "http/ContentTypes.h" -#include "hwutil/PartitionUtils.h" #include "Logging.h" #include "rfc8908/RFC8908Handler.h" -#include "SimpleMutex.h" #include "StringHelpers.h" #include "util/HexUtils.h" #include #include -#include -#include #include #include using namespace OpenShock; -namespace { - // Hashing a whole partition takes a noticeable time and its content only changes through OTA, which restarts the - // device, so each partition is hashed once per boot. Entries are only ever appended, under the mutex, so a pointer - // into the table stays valid. - struct PartitionHash { - const esp_partition_t* partition; - char hash[65]; - }; - constexpr std::size_t kMaxHashedPartitions = 2; - PartitionHash s_partitionHashes[kMaxHashedPartitions]; - std::size_t s_partitionHashCount = 0; - SimpleMutex s_partitionHashesMutex; - - // Caller holds s_partitionHashesMutex - const char* findPartitionHash(const esp_partition_t* partition) - { - for (std::size_t i = 0; i < s_partitionHashCount; ++i) { - if (s_partitionHashes[i].partition == partition) { - return s_partitionHashes[i].hash; - } - } - return nullptr; - } -} // namespace - -/// @brief The partition's SHA-256 as hex, computed on first use. Null for a null partition, if hashing failed, or if -/// the table is full. -static const char* getPartitionHash(const esp_partition_t* partition) -{ - if (partition == nullptr) { - return nullptr; - } - { - ScopedLock lock__(&s_partitionHashesMutex); - if (const char* hash = findPartitionHash(partition)) { - return hash; - } - } - - // Hash with the mutex released: a retired instance's httpd task may be looking up its own tag meanwhile - char hash[65]; - if (!TryGetPartitionHash(partition, hash)) { - return nullptr; - } - - ScopedLock lock__(&s_partitionHashesMutex); - if (const char* existing = findPartitionHash(partition)) { - return existing; // Another task hashed the same partition meanwhile - } - if (s_partitionHashCount == kMaxHashedPartitions) { - OS_LOGW(TAG, "Partition hash table full, not caching"); - return nullptr; - } - PartitionHash& entry = s_partitionHashes[s_partitionHashCount++]; - entry.partition = partition; - std::memcpy(entry.hash, hash, sizeof(hash)); - return entry.hash; -} - static const char* contentTypeForPath(std::string_view path) { - // Assets are stored pre-compressed as .gz: the type is that of the content inside, not of the wrapper - if (path.size() >= 3 && path[path.size() - 3] == '.' && CharToLower(path[path.size() - 2]) == 'g' && CharToLower(path[path.size() - 1]) == 'z') { - path.remove_suffix(3); - } - // Walk back to the dot. The longest known extension is 5 characters, so give up after that many, or at a '/'. static constexpr std::size_t kMaxExtensionLen = 5; std::size_t i = path.size(); @@ -119,10 +51,7 @@ static const char* contentTypeForPath(std::string_view path) }; for (const auto& entry : kContentTypes) { - if (entry.extension.size() != ext.size()) continue; - std::size_t j = 0; - while (j < ext.size() && CharToLower(ext[j]) == entry.extension[j]) ++j; - if (j == ext.size()) { + if (StringIEquals(entry.extension, ext)) { return entry.contentType; } } @@ -188,17 +117,13 @@ static constexpr const char* kCacheControlImmutable = "public, max-age=31536000, // What a static response says about itself, shared by the full response, the 304 and the HEAD response. struct Representation { const char* contentType; - bool gzip; const char* cacheControl; - const char* etag; // quoted, or null when unknown + const char* etag; // quoted std::size_t size; }; static bool matchesIfNoneMatch(httpd_req_t* req, const char* etag) { - if (etag == nullptr) { - return false; - } char inm[80]; return httpd_req_get_hdr_value_str(req, "If-None-Match", inm, sizeof(inm)) == ESP_OK && std::strcmp(etag, inm) == 0; } @@ -209,9 +134,7 @@ static void setHeaders(httpd_req_t* req, const Representation& rep) httpd_resp_set_type(req, rep.contentType); httpd_resp_set_hdr(req, "Vary", "Accept-Encoding"); httpd_resp_set_hdr(req, "Cache-Control", rep.cacheControl); - if (rep.etag != nullptr) { - httpd_resp_set_hdr(req, "ETag", rep.etag); - } + httpd_resp_set_hdr(req, "ETag", rep.etag); } static esp_err_t sendNotModified(httpd_req_t* req, const Representation& rep) @@ -241,15 +164,10 @@ static esp_err_t sendHeadResponse(httpd_req_t* req, const Representation& rep) head += rep.contentType; head += "\r\nContent-Length: "; head += std::to_string(rep.size); - if (rep.gzip) { - head += "\r\nContent-Encoding: gzip"; - } - head += "\r\nVary: Accept-Encoding\r\nCache-Control: "; + head += "\r\nContent-Encoding: gzip\r\nVary: Accept-Encoding\r\nCache-Control: "; head += rep.cacheControl; - if (rep.etag != nullptr) { - head += "\r\nETag: "; - head += rep.etag; - } + head += "\r\nETag: "; + head += rep.etag; head += "\r\n\r\n"; std::size_t sent = 0; @@ -265,52 +183,33 @@ static esp_err_t sendHeadResponse(httpd_req_t* req, const Representation& rep) CaptivePortal::StaticFileServer::StaticFileServer() { - // The portal assets: the www image where the partition table has one, else (1.x layout) littlefs static0. - if (const esp_partition_t* www = Www::FindPartition(); www != nullptr) { - if (!m_www.mount(www)) { - OS_LOGE(TAG, "The portal has no assets until a www image is flashed, serving the error page"); // mount() logged why - return; - } - if (!m_www.find("index.html")) { - OS_LOGE(TAG, "The www image has no index.html, serving the error page"); - m_www.unmount(); - return; - } - char hex[Www::kBodyHashSize * 2 + 1]; - HexUtils::ToHex(m_www.image().BodyHash(), hex, false); - m_etag = std::string("\"") + hex + "\""; - OS_LOGI(TAG, "Serving files from the www image (%s)", m_etag.c_str()); + if (!m_www.mount(Www::FindPartition())) { + OS_LOGE(TAG, "The portal has no assets until a www image is flashed, serving the error page"); // mount() logged why return; } - - // Mount the static filesystem (gzipped portal assets) read-only via raw littlefs. - const esp_partition_t* partition = FindStaticPartition(); - if (partition == nullptr) { - OS_LOGE(TAG, "Failed to find a www or static filesystem partition"); - } else if (!m_partition.mount(partition, false)) { - OS_LOGE(TAG, "Failed to mount static filesystem"); - } else if (!m_partition.fileSize("/www/index.html.gz") && !m_partition.fileSize("/www/index.html")) { - OS_LOGE(TAG, "/www/index.html(.gz) not found, serving error page"); - m_partition.unmount(); - } else { - // Hash now, so the first request doesn't pay for it - const char* hash = getPartitionHash(partition); - OS_LOGI(TAG, "Serving files from littlefs (hash: %s)", hash != nullptr ? hash : "?"); + if (!m_www.find("index.html")) { + OS_LOGE(TAG, "The www image has no index.html, serving the error page"); + m_www.unmount(); + return; } + char hex[Www::kBodyHashSize * 2 + 1]; + HexUtils::ToHex(m_www.image().BodyHash(), hex, false); + m_etag = std::string("\"") + hex + "\""; + OS_LOGI(TAG, "Serving files from the www image (%s)", m_etag.c_str()); } void CaptivePortal::StaticFileServer::registerHandler(httpd_handle_t server) { // Decided once: the assets are mounted in the constructor and stay that way until destruction - if (m_www.isMounted() || m_partition.isMounted()) { + if (m_www.isMounted()) { CaptivePortal::registerHandler(server, "/*", HTTP_GET, &StaticFileServer::staticFileHandler, this); CaptivePortal::registerHandler(server, "/*", HTTP_HEAD, &StaticFileServer::staticFileHandler, this); } else { - CaptivePortal::registerHandler(server, "/*", HTTP_GET, &StaticFileServer::missingFsHandler, nullptr); + CaptivePortal::registerHandler(server, "/*", HTTP_GET, &StaticFileServer::missingAssetsHandler, nullptr); } } -esp_err_t CaptivePortal::StaticFileServer::missingFsHandler(httpd_req_t* req) +esp_err_t CaptivePortal::StaticFileServer::missingAssetsHandler(httpd_req_t* req) { // No usable assets on the device; serve the help page for any request. return sendResp( @@ -334,7 +233,7 @@ esp_err_t CaptivePortal::StaticFileServer::staticFileHandler(httpd_req_t* req) std::string_view uri(req->uri); uri = uri.substr(0, uri.find('?')); - // A ".." segment is never a valid asset path (littlefs would resolve it). Browsers normalize these away before + // A ".." segment is never a valid asset path. Browsers normalize these away before // sending, so this only comes from hand-built requests: refuse it rather than answer like a captive probe. if (uri.find("..") != std::string_view::npos) { return sendResp(req, S400, HTTP::ContentType::TextPlain, "Bad Request\n"); @@ -344,28 +243,20 @@ esp_err_t CaptivePortal::StaticFileServer::staticFileHandler(httpd_req_t* req) uri = "/index.html"; } - bool wantsGzip = clientAcceptsGzip(req); - - return self->m_www.isMounted() ? self->serveFromImage(req, uri, wantsGzip) : self->serveFromLittlefs(req, uri, wantsGzip); -} - -esp_err_t CaptivePortal::StaticFileServer::serveFromImage(httpd_req_t* req, std::string_view uri, bool wantsGzip) -{ // Stored without the leading slash. Every body in the image is gzipped. - auto asset = m_www.find(uri.substr(1)); + auto asset = self->m_www.find(uri.substr(1)); if (!asset) { // Unknown path -> captive redirect (the index fallback of a static server). return RFC8908::EmitRedirect(req); } - if (!wantsGzip) { + if (!clientAcceptsGzip(req)) { return sendNotAcceptable(req); } Representation rep { .contentType = contentTypeForPath(uri), - .gzip = true, .cacheControl = asset->immutable() ? kCacheControlImmutable : kCacheControl, - .etag = m_etag.empty() ? nullptr : m_etag.c_str(), + .etag = self->m_etag.c_str(), .size = asset->body.size(), }; @@ -381,88 +272,3 @@ esp_err_t CaptivePortal::StaticFileServer::serveFromImage(httpd_req_t* req, std: // Straight from the flash mapping: no copy, no filesystem return httpd_resp_send(req, reinterpret_cast(asset->body.data()), static_cast(asset->body.size())); } - -esp_err_t CaptivePortal::StaticFileServer::serveFromLittlefs(httpd_req_t* req, std::string_view uri, bool wantsGzip) -{ - // Assets live at /www/, usually pre-compressed as /www/.gz. A client that accepts gzip is offered the - // compressed file first; anyone else, or a path with no .gz, gets the plain file if the image has one. - std::string path; - path.reserve(4 + uri.size() + 3); - path += "/www"; - path += uri; - - // fileSize only answers for regular files: littlefs collapses "//" and "." segments, so a path can land on a - // directory, which must not be opened as a file. - std::optional size; - bool gzip = false; - if (wantsGzip) { - path += ".gz"; - size = m_partition.fileSize(path.c_str()); - gzip = size.has_value(); - if (!gzip) { - path.resize(path.size() - 3); - } - } - if (!gzip) { - size = m_partition.fileSize(path.c_str()); - if (!size) { - if (!wantsGzip) { - path += ".gz"; - if (m_partition.fileSize(path.c_str())) { - return sendNotAcceptable(req); - } - } - // Unknown path -> captive redirect (matches serveStatic default-file fallthrough). - return RFC8908::EmitRedirect(req); - } - } - - // The two encodings are different representations, so they get different tags. Keep the quoted tag alive until - // after send. - std::string etag; - if (const char* fsHash = getPartitionHash(m_partition.partition())) { - etag = std::string("\"") + fsHash + (gzip ? "-gz\"" : "\""); - } - - Representation rep { - .contentType = contentTypeForPath(path), - .gzip = gzip, - .cacheControl = kCacheControl, - .etag = etag.empty() ? nullptr : etag.c_str(), - .size = *size, - }; - - if (matchesIfNoneMatch(req, rep.etag)) { - return sendNotModified(req, rep); - } - if (req->method == HTTP_HEAD) { - return sendHeadResponse(req, rep); - } - - // Headers are only committed with the first chunk, so a failure before anything is on the wire (open or buffer - // allocation) can still be answered with a clean 500 instead of a dropped connection. - bool started = false; - auto begin = [&]() { - setHeaders(req, rep); - if (rep.gzip) { - httpd_resp_set_hdr(req, "Content-Encoding", "gzip"); - } - started = true; - }; - bool ok = m_partition.readStream(path.c_str(), [&](std::span chunk) { - if (!started) { - begin(); - } - return httpd_resp_send_chunk(req, reinterpret_cast(chunk.data()), chunk.size()) == ESP_OK; - }); - if (!ok) { - if (started) { - return ESP_FAIL; // Mid-stream: the client sees a truncated chunked body - } - return sendResp(req, S500, HTTP::ContentType::TextPlain, "Internal Server Error\n"); - } - if (!started) { - begin(); // Empty file - } - return httpd_resp_send_chunk(req, nullptr, 0); -} diff --git a/components/captive_portal/src/captiveportal/StaticFileServer.h b/components/captive_portal/src/captiveportal/StaticFileServer.h index a85223122..3ec136de0 100644 --- a/components/captive_portal/src/captiveportal/StaticFileServer.h +++ b/components/captive_portal/src/captiveportal/StaticFileServer.h @@ -1,18 +1,14 @@ #pragma once -#include "fs/LfsPartition.h" #include "OpenShock.h" #include "www/WwwPartition.h" #include #include -#include namespace OpenShock::CaptivePortal { - /// @brief Serves the portal UI from the flat www image (mapped, served by pointer) or, on the 1.x partition layout - /// that has no www partition, from gzipped assets on the littlefs static0 partition. At most one is mounted, - /// for this object's lifetime. + /// @brief Serves the portal UI from the flat www image (mapped, served by pointer), mounted for this object's lifetime. class StaticFileServer { DISABLE_COPY(StaticFileServer); DISABLE_MOVE(StaticFileServer); @@ -24,13 +20,10 @@ namespace OpenShock::CaptivePortal { void registerHandler(httpd_handle_t server); private: - static esp_err_t missingFsHandler(httpd_req_t* req); + static esp_err_t missingAssetsHandler(httpd_req_t* req); static esp_err_t staticFileHandler(httpd_req_t* req); - esp_err_t serveFromImage(httpd_req_t* req, std::string_view uri, bool wantsGzip); - esp_err_t serveFromLittlefs(httpd_req_t* req, std::string_view uri, bool wantsGzip); Www::Partition m_www; - LfsPartition m_partition; // TODO(remove with 1.x): the littlefs path is transitional, for hubs the www migration has not reached yet - std::string m_etag; // the www image's hash, quoted; empty on the 1.x layout, which tags by partition hash + std::string m_etag; // the www image's hash, quoted }; } // namespace OpenShock::CaptivePortal diff --git a/components/common/CMakeLists.txt b/components/common/CMakeLists.txt index c607bbb87..52e2967c7 100644 --- a/components/common/CMakeLists.txt +++ b/components/common/CMakeLists.txt @@ -7,7 +7,7 @@ # Common.h / FormatHelpers.h / OpenShock.h: basic defines and macros # BoardDescriptor.h: the board descriptor stamped into the app image, Board::Get() # enums/*.h: shared enums (consumers include with the enums/ prefix) -# Convert.h, SemVer.h, Checksum.h, RateLimiter.h, TinyVec.h, +# Convert.h, SemVer.h, Checksum.h, RateLimiter.h, # StringHelpers.h, SimpleMutex.h, util/*.h idf_component_register( @@ -23,7 +23,7 @@ idf_component_register( # this file rather than the whole project. "src/Version.cpp" INCLUDE_DIRS "include" - REQUIRES logging # Logging.h / OS_PANIC* (TinyVec.h) in public headers + REQUIRES logging # Logging.h (the sources in src/) temporal # Temporal.h (RateLimiter.h) in public header freertos # semphr.h in SimpleMutex.h public header PRIV_REQUIRES esp_system # esp_task_wdt.h (TaskUtils watchdog helpers) diff --git a/components/common/Kconfig b/components/common/Kconfig index 5a3e40f0d..a2f947e2b 100644 --- a/components/common/Kconfig +++ b/components/common/Kconfig @@ -20,10 +20,11 @@ menu "OpenShock defaults" Default backend API domain the hub connects to. Overridable at runtime. config OPENSHOCK_FW_CDN_DOMAIN - string "Firmware CDN domain" - default "firmware.openshock.org" + string "Firmware repository server domain" + default "repo.openshock.dev" help - Default CDN domain used for OTA firmware/version lookups. Overridable at runtime. + Default repository server domain used for OTA firmware/version lookups. Overridable at runtime. + Named CDN for its 1.x origin, when OTA read flat files off the CDN. config OPENSHOCK_FW_HOSTNAME string "Default hostname" diff --git a/components/common/host_test/main/CMakeLists.txt b/components/common/host_test/main/CMakeLists.txt index 4dc82d847..e47cd46b6 100644 --- a/components/common/host_test/main/CMakeLists.txt +++ b/components/common/host_test/main/CMakeLists.txt @@ -13,7 +13,6 @@ idf_component_register( "test_semver.cpp" "test_checksum.cpp" "test_boarddescriptor.cpp" - "test_tinyvec.cpp" "test_fnproxy.cpp" "test_functionref.cpp" "../../src/StringHelpers.cpp" # StringSplit/StringIContains/FormatToString (out-of-line) diff --git a/components/common/host_test/main/test_enums.cpp b/components/common/host_test/main/test_enums.cpp index 9df1d309d..e55c051ab 100644 --- a/components/common/host_test/main/test_enums.cpp +++ b/components/common/host_test/main/test_enums.cpp @@ -78,6 +78,17 @@ TEST_CASE("OtaUpdateChannel: stable/beta/develop/dev + case", "[common][enums]") TEST_ASSERT_FALSE(TryParseOtaUpdateChannel("stabl").has_value()); // prefix, not full } +TEST_CASE("OtaUpdateChannel: every name parses back", "[common][enums]") +{ + TEST_ASSERT_EQUAL_STRING("stable", OtaUpdateChannelName(OtaUpdateChannel::Stable)); + TEST_ASSERT_EQUAL_STRING("beta", OtaUpdateChannelName(OtaUpdateChannel::Beta)); + TEST_ASSERT_EQUAL_STRING("develop", OtaUpdateChannelName(OtaUpdateChannel::Develop)); + + for (OtaUpdateChannel channel : {OtaUpdateChannel::Stable, OtaUpdateChannel::Beta, OtaUpdateChannel::Develop}) { + TEST_ASSERT_TRUE(TryParseOtaUpdateChannel(OtaUpdateChannelName(channel)) == channel); + } +} + // ---- OtaUpdateStep --------------------------------------------------------- TEST_CASE("OtaUpdateStep: every step + rejects unknown", "[common][enums]") @@ -93,6 +104,15 @@ TEST_CASE("OtaUpdateStep: every step + rejects unknown", "[common][enums]") TEST_ASSERT_FALSE(TryParseOtaUpdateStep("done").has_value()); } +TEST_CASE("OtaUpdateStep: every name parses back", "[common][enums]") +{ + TEST_ASSERT_EQUAL_STRING("RollingBack", OtaUpdateStepName(OtaUpdateStep::RollingBack)); + + for (OtaUpdateStep step : {OtaUpdateStep::None, OtaUpdateStep::Updating, OtaUpdateStep::Updated, OtaUpdateStep::Validating, OtaUpdateStep::Validated, OtaUpdateStep::RollingBack}) { + TEST_ASSERT_TRUE(TryParseOtaUpdateStep(OtaUpdateStepName(step)) == step); + } +} + // ---- FirmwareBootType (string_view API) ------------------------------------ TEST_CASE("FirmwareBootType: normal/newfirmware/new_firmware/rollback", "[common][enums]") diff --git a/components/common/host_test/main/test_tinyvec.cpp b/components/common/host_test/main/test_tinyvec.cpp deleted file mode 100644 index a6c526a1e..000000000 --- a/components/common/host_test/main/test_tinyvec.cpp +++ /dev/null @@ -1,82 +0,0 @@ -// TinyVec is the firmware's POD-only dynamic array (manual malloc/realloc growth). -// It underpins buffer assembly across the codebase, so exercise its growth, -// zero-fill, replace and move semantics. -#include "unity.h" - -#include "TinyVec.h" - -#include -#include -#include - -TEST_CASE("TinyVec append grows and preserves data", "[util][tinyvec]") -{ - TinyVec v; - TEST_ASSERT_TRUE(v.empty()); - - const uint16_t a[] = {1, 2, 3}; - v.append(a, 3); - TEST_ASSERT_EQUAL_UINT32(3, v.size()); - TEST_ASSERT_GREATER_OR_EQUAL_UINT32(3, v.capacity()); - - const uint16_t b[] = {4, 5}; - v.append(b, 2); - TEST_ASSERT_EQUAL_UINT32(5, v.size()); - for (uint16_t i = 0; i < 5; i++) { - TEST_ASSERT_EQUAL_UINT16(i + 1, v[i]); - } -} - -TEST_CASE("TinyVec resize zero-fills the grown region", "[util][tinyvec]") -{ - TinyVec v; - const uint8_t a[] = {0xAA, 0xBB}; - v.append(a, 2); - - v.resize(5); - TEST_ASSERT_EQUAL_UINT32(5, v.size()); - TEST_ASSERT_EQUAL_UINT8(0xAA, v[0]); - TEST_ASSERT_EQUAL_UINT8(0xBB, v[1]); - TEST_ASSERT_EQUAL_UINT8(0, v[2]); - TEST_ASSERT_EQUAL_UINT8(0, v[4]); -} - -TEST_CASE("TinyVec assign replaces the contents", "[util][tinyvec]") -{ - TinyVec v; - const int a[] = {7, 8, 9, 10}; - v.append(a, 4); - - const int b[] = {1, 2}; - v.assign(b, 2); - TEST_ASSERT_EQUAL_UINT32(2, v.size()); - TEST_ASSERT_EQUAL_INT(1, v[0]); - TEST_ASSERT_EQUAL_INT(2, v[1]); - - v.clear(); - TEST_ASSERT_TRUE(v.empty()); -} - -TEST_CASE("TinyVec move transfers ownership", "[util][tinyvec]") -{ - TinyVec v; - const uint32_t a[] = {100, 200}; - v.append(a, 2); - - TinyVec w(std::move(v)); - TEST_ASSERT_EQUAL_UINT32(2, w.size()); - TEST_ASSERT_EQUAL_UINT32(100, w[0]); - TEST_ASSERT_EQUAL_UINT32(0, v.size()); // moved-from is emptied -} - -TEST_CASE("TinyVec constructs from a buffer and converts to span", "[util][tinyvec]") -{ - const uint8_t a[] = {5, 6, 7}; - TinyVec v(a, 3); - TEST_ASSERT_EQUAL_UINT32(3, v.size()); - TEST_ASSERT_EQUAL_UINT8(6, v[1]); - - std::span s = v; - TEST_ASSERT_EQUAL_UINT32(3, s.size()); - TEST_ASSERT_EQUAL_UINT8(7, s[2]); -} diff --git a/components/common/include/TinyVec.h b/components/common/include/TinyVec.h deleted file mode 100644 index d5cf4572e..000000000 --- a/components/common/include/TinyVec.h +++ /dev/null @@ -1,144 +0,0 @@ -#pragma once - -#include "Logging.h" -#include "OpenShock.h" - -#include -#include -#include -#include -#include -#include -#include -#include - -template -class TinyVec { - static_assert(std::is_trivially_copyable_v && std::is_trivially_destructible_v, "TinyVec requires a trivially copyable & destructible type (POD)."); - - DISABLE_COPY(TinyVec); - -public: - TinyVec() noexcept = default; - - // Zero-initialized, like resize() - TinyVec(SizeType count) - { - if (count == 0) return; - reserve(count); - memset(_data, 0, size_t(count) * sizeof(T)); - _len = count; - } - TinyVec(const T* src, SizeType count) - { - if (!src || count == 0) return; - reserve(count); - memcpy(_data, src, size_t(count) * sizeof(T)); - _len = count; - } - - ~TinyVec() { free(_data); } - - TinyVec(TinyVec&& other) noexcept - { - _data = other._data; - _len = other._len; - _cap = other._cap; - other._data = nullptr; - other._len = other._cap = 0; - } - - TinyVec& operator=(TinyVec&& other) noexcept - { - if (this != &other) { - free(_data); - _data = other._data; - _len = other._len; - _cap = other._cap; - other._data = nullptr; - other._len = other._cap = 0; - } - return *this; - } - - [[nodiscard]] SizeType size() const noexcept { return _len; } - [[nodiscard]] SizeType capacity() const noexcept { return _cap; } - [[nodiscard]] bool empty() const noexcept { return _len == 0; } - [[nodiscard]] T* data() noexcept { return _data; } - [[nodiscard]] const T* data() const noexcept { return _data; } - - T& operator[](SizeType i) noexcept { return _data[i]; } - const T& operator[](SizeType i) const noexcept { return _data[i]; } - - T* begin() noexcept { return _data; } - const T* begin() const noexcept { return _data; } - T* end() noexcept { return _data + _len; } - const T* end() const noexcept { return _data + _len; } - - operator std::span() noexcept { return {_data, static_cast(_len)}; } - operator std::span() const noexcept { return {_data, static_cast(_len)}; } - - void reserve(SizeType new_cap) - { - if (new_cap <= _cap) return; - if (sizeof(T) && new_cap > std::numeric_limits::max() / sizeof(T)) OS_PANIC_INSTANT("TVEC", "Cannot allocate a buffer of given size!"); - - void* newbuf = malloc(size_t(new_cap) * sizeof(T)); - if (!newbuf) OS_PANIC_INSTANT("TVEC", "Could not allocate a buffer of given size!"); - if (_data) { - memcpy(newbuf, _data, size_t(_len) * sizeof(T)); - free(_data); - } - - _data = static_cast(newbuf); - _cap = new_cap; - } - - void resize(SizeType new_len) - { - if (new_len > _cap) reserve(next_cap(new_len)); - - if (new_len > _len) { - memset(_data + _len, 0, size_t(new_len - _len) * sizeof(T)); - } - - _len = new_len; - } - - void append(const T* src, SizeType count) - { - if (!src || count == 0) return; - if (count > std::numeric_limits::max() - _len) OS_PANIC_INSTANT("TVEC", "Length overflow!"); - SizeType new_len = _len + count; - if (new_len > _cap) reserve(next_cap(new_len)); - memcpy(_data + _len, src, size_t(count) * sizeof(T)); - _len = new_len; - } - - void assign(const T* src, SizeType count) - { - if (!src || count == 0) { - _len = 0; - return; - } - if (count > _cap) reserve(next_cap(count)); - memcpy(_data, src, size_t(count) * sizeof(T)); - _len = count; - } - - void clear() noexcept { _len = 0; } - -private: - SizeType next_cap(SizeType min_needed) const noexcept - { - // Grow by 1.5x, saturating instead of wrapping around near the SizeType limit - constexpr SizeType kMax = std::numeric_limits::max(); - SizeType newcap = _cap == 0 ? 4 : (_cap > (kMax - 1) / 3 * 2 ? kMax : SizeType(_cap + _cap / 2 + 1)); - if (newcap < min_needed) newcap = min_needed; - return newcap; - } - - T* _data = nullptr; - SizeType _len = 0; - SizeType _cap = 0; -}; diff --git a/components/common/include/enums/OtaUpdateChannel.h b/components/common/include/enums/OtaUpdateChannel.h index 4105a0fb1..8db2f9286 100644 --- a/components/common/include/enums/OtaUpdateChannel.h +++ b/components/common/include/enums/OtaUpdateChannel.h @@ -13,6 +13,20 @@ namespace OpenShock { Develop }; + // The name jsonconfig renders; TryParseOtaUpdateChannel reads it back. + [[nodiscard]] constexpr const char* OtaUpdateChannelName(OtaUpdateChannel channel) + { + switch (channel) { + case OtaUpdateChannel::Stable: + return "stable"; + case OtaUpdateChannel::Beta: + return "beta"; + case OtaUpdateChannel::Develop: + return "develop"; + } + return ""; + } + [[nodiscard]] inline std::optional TryParseOtaUpdateChannel(std::string_view str) { if (StringIEquals(str, "stable")) { diff --git a/components/common/include/enums/OtaUpdateStep.h b/components/common/include/enums/OtaUpdateStep.h index 28e0fe0a9..243d837f9 100644 --- a/components/common/include/enums/OtaUpdateStep.h +++ b/components/common/include/enums/OtaUpdateStep.h @@ -16,6 +16,26 @@ namespace OpenShock { RollingBack }; + // The name jsonconfig renders; TryParseOtaUpdateStep reads it back. + [[nodiscard]] constexpr const char* OtaUpdateStepName(OtaUpdateStep step) + { + switch (step) { + case OtaUpdateStep::None: + return "None"; + case OtaUpdateStep::Updating: + return "Updating"; + case OtaUpdateStep::Updated: + return "Updated"; + case OtaUpdateStep::Validating: + return "Validating"; + case OtaUpdateStep::Validated: + return "Validated"; + case OtaUpdateStep::RollingBack: + return "RollingBack"; + } + return ""; + } + [[nodiscard]] inline std::optional TryParseOtaUpdateStep(std::string_view str) { if (StringIEquals(str, "none")) { diff --git a/components/config/CMakeLists.txt b/components/config/CMakeLists.txt index ad7c0ddfa..5869a1d64 100644 --- a/components/config/CMakeLists.txt +++ b/components/config/CMakeLists.txt @@ -3,29 +3,19 @@ # # config/Nvs.h typed access to one NVS namespace, the primitive every section's LoadNvs/SaveNvs uses # config/Section.h the section template: RAM cache, serialized writers, saving only what changed, the registry -# config/Store.h picking the store (`cfg`, or the shared `nvs` on the 1.x layout), the move between them, -# filling an empty store (the 1.x file, else defaults), factory reset, the whole-config JSON -# config/Wire.h the whole config as the FlatBuffers HubConfig (local protocol Ready message, rawconfig), and -# HubConfig -> JSON, which is also how the 1.x config file is imported -# config/Migration.h retyping the 1.x littlefs `config` partition as the `cfg` store (run by components/migration) +# config/Store.h opening the `cfg` store, filling an empty one with the defaults, factory reset, the +# whole-config JSON # config/Parse.h field parsing helpers for sections' FromJSON -# -# The 1.x import (Store.cpp's readLegacyConfig, littlefs through fs) is 1.x compatibility code: it goes away on -# master once the release/1.9 branch is cut. idf_component_register( - SRCS "src/Migration.cpp" - "src/Nvs.cpp" + SRCS "src/Nvs.cpp" "src/Parse.cpp" "src/Section.cpp" "src/Store.cpp" - "src/Wire.cpp" INCLUDE_DIRS "include" - REQUIRES common osjson # OpenShock.h, SimpleMutex.h, TinyVec.h, util/FunctionRef.h; json/Json.h and json_generator.h + REQUIRES common osjson # OpenShock.h, SimpleMutex.h, util/FunctionRef.h; json/Json.h and json_generator.h nvs_flash # nvs.h in Nvs.h - serialization flatbuffers # HubConfig in Wire.h esp_hal_gpio # hal/gpio_types.h in Parse.h - PRIV_REQUIRES fs logging esp_partition # LfsPartition (the 1.x import), OS_LOG*/OS_PANIC, the partition lookups - partition_layout esp_system # the config partition retype, esp_restart + PRIV_REQUIRES logging # OS_LOG*/OS_PANIC ) target_compile_options(${COMPONENT_LIB} PRIVATE -Wno-error) diff --git a/components/config/host_test/legacy/gen_legacy_config.cpp b/components/config/host_test/legacy/gen_legacy_config.cpp deleted file mode 100644 index 35a934e74..000000000 --- a/components/config/host_test/legacy/gen_legacy_config.cpp +++ /dev/null @@ -1,116 +0,0 @@ -// Generator for main/legacy_fixtures.h. NOT part of the host test build. -// -// Serializes a hub config exactly the way the Arduino firmware did: the bodies of its -// RootConfig / *Config::ToFlatbuffers (same Create* calls in the same order), then -// FinishHubConfigBuffer as in its Config.cpp trySaveConfig. Build it against the old -// firmware's generated schema header and pinned flatbuffers, e.g. from the firmware -// repo root (any host g++): -// -// mkdir -p /tmp/gen/v152/serialization/_fbs /tmp/gen/dev/serialization/_fbs /tmp/gen/fb152 /tmp/gen/fbdev -// git show 1.5.2:include/serialization/_fbs/HubConfig_generated.h > /tmp/gen/v152/serialization/_fbs/HubConfig_generated.h -// git show develop:include/serialization/_fbs/HubConfig_generated.h > /tmp/gen/dev/serialization/_fbs/HubConfig_generated.h -// git show develop:include/serialization/_fbs/WifiAuthMode_generated.h > /tmp/gen/dev/serialization/_fbs/WifiAuthMode_generated.h -// git -C components/flatbuffers/flatbuffers archive 8b02fe6178427b96aea25396b53ea4ae8cadd7d8 include | tar -x -C /tmp/gen/fb152 -// git -C components/flatbuffers/flatbuffers archive 81edeb17d9118143f2c81caf27edfb0df401279e include | tar -x -C /tmp/gen/fbdev -// g++ -std=c++17 -I/tmp/gen/v152 -I/tmp/gen/fb152/include gen_legacy_config.cpp -o /tmp/gen/v152.out && /tmp/gen/v152.out -// g++ -std=c++17 -DLEGACY_DEVELOP -I/tmp/gen/dev -I/tmp/gen/dev/serialization/_fbs -I/tmp/gen/fbdev/include gen_legacy_config.cpp -o /tmp/gen/dev.out && /tmp/gen/dev.out -// -// The flatbuffers commits are the ones each tag's platformio.ini pinned. -#include "serialization/_fbs/HubConfig_generated.h" - -#include -#include -#include - -using namespace OpenShock::Serialization; -namespace C = OpenShock::Serialization::Configuration; - -struct Creds { - uint8_t id; - std::string ssid, password; -#ifdef LEGACY_DEVELOP - Types::WifiAuthMode authMode; - bool pinned; - uint8_t bssid[6]; -#endif -}; - -static void emit(const char* name, bool keepAlive) -{ - flatbuffers::FlatBufferBuilder builder; - const bool withSensitiveData = true; - - // RootConfig::ToFlatbuffers order: rf, wifi, captivePortal, backend, serialInput, otaUpdate, estop - auto rfOffset = C::CreateRFConfig(builder, 15, keepAlive); - - std::vector credentialsList = { -#ifdef LEGACY_DEVELOP - {1, "HomeNet", "hunter22", Types::WifiAuthMode::WPA2_PSK, true, {0x12, 0x34, 0x56, 0x78, 0x9a, 0xbc}}, - {2, "Guest", "", Types::WifiAuthMode::Open, false, {}}, - {3, "Office", "s3cret!", Types::WifiAuthMode::UNKNOWN, false, {}}, -#else - {1, "HomeNet", "hunter22"}, - {2, "Guest", ""}, - {3, "Office", "s3cret!"}, -#endif - }; - std::vector> fbsCredentialsList; - fbsCredentialsList.reserve(credentialsList.size()); - for (auto& c : credentialsList) { - auto ssidOffset = builder.CreateString(c.ssid); - flatbuffers::Offset passwordOffset; - if (withSensitiveData) { - passwordOffset = builder.CreateString(c.password); - } else { - passwordOffset = 0; - } -#ifdef LEGACY_DEVELOP - const C::MacAddress* bssidPtr = nullptr; - C::MacAddress bssidStruct; - if (c.pinned) { - bssidStruct = C::MacAddress(flatbuffers::span(c.bssid, 6)); - bssidPtr = &bssidStruct; - } - fbsCredentialsList.push_back(C::CreateWiFiCredentials(builder, c.id, ssidOffset, passwordOffset, c.authMode, bssidPtr)); -#else - fbsCredentialsList.push_back(C::CreateWiFiCredentials(builder, c.id, ssidOffset, passwordOffset)); -#endif - } - std::string accessPointSSID = "MyHubAP", hostname = "my-hub"; - auto wifiOffset = C::CreateWiFiConfig(builder, builder.CreateString(accessPointSSID), builder.CreateString(hostname), builder.CreateVector(fbsCredentialsList)); - - auto captivePortalOffset = C::CreateCaptivePortalConfig(builder, true); - - auto domainOffset = builder.CreateString(std::string("api.example.org")); - auto authTokenOffset = builder.CreateString(std::string("tok_0123456789abcdef")); - auto backendOffset = C::CreateBackendConfig(builder, domainOffset, authTokenOffset); - - auto serialInputOffset = C::CreateSerialInputConfig(builder, false); - - auto otaUpdateOffset = C::CreateOtaUpdateConfig(builder, false, builder.CreateString(std::string("fw.example.org")), C::OtaUpdateChannel::Beta, true, true, 120, false, true, 4242, C::OtaUpdateStep::Validating); - - auto estopOffset = C::CreateEStopConfig(builder, true, 13); - - auto root = C::CreateHubConfig(builder, rfOffset, wifiOffset, captivePortalOffset, backendOffset, serialInputOffset, otaUpdateOffset, estopOffset); - C::FinishHubConfigBuffer(builder, root); - - const uint8_t* p = builder.GetBufferPointer(); - std::size_t n = builder.GetSize(); - printf(" // %s (%zu bytes)\n static const uint8_t %s[] = {\n", name, n, name); - for (std::size_t i = 0; i < n; i += 16) { - printf(" "); - for (std::size_t j = i; j < n && j < i + 16; ++j) printf(" 0x%02x,", p[j]); - printf("\n"); - } - printf(" };\n"); -} - -int main() -{ -#ifdef LEGACY_DEVELOP - emit("kDevelopConfig", false); -#else - emit("kV152Config", true); - emit("kV152ConfigKeepAliveOff", false); -#endif -} diff --git a/components/config/host_test/main/CMakeLists.txt b/components/config/host_test/main/CMakeLists.txt index bc807fff7..7c0595da7 100644 --- a/components/config/host_test/main/CMakeLists.txt +++ b/components/config/host_test/main/CMakeLists.txt @@ -1,25 +1,22 @@ # The config framework (components/config) and every module's config section, compiled directly (not via their # components, whose logging dependency pulls in app_update, which has no linux port). So the tests drive the store -# as the firmware assembles it: Config::Init, the modules' accessors, jsonconfig, rawconfig and factory reset, under +# as the firmware assembles it: Config::Init, the modules' accessors, jsonconfig and factory reset, under # the real SimpleMutexes, against ESP-IDF's real NVS over the linux flash emulation and the partition table in -# ../partitions_test.csv. Only the 1.x littlefs reader is faked: stubs/fs/LfsPartition.h is an in-memory file store -# the tests seed with the config file a 1.x hub leaves behind. osjson's Json.cpp and the common sources needed are -# compiled in as well (jsmn / json_generator come from idf_component.yml). +# ../partitions_test.csv. osjson's Json.cpp and the common sources needed are compiled in as well (jsmn / +# json_generator come from idf_component.yml). set(_components ../../..) idf_component_register( SRCS "test_main.cpp" "config_test_helpers.cpp" - "test_roundtrip.cpp" # API + jsonconfig + rawconfig round trips through the NVS store + "test_roundtrip.cpp" # API + jsonconfig round trips through the NVS store "test_defaults.cpp" # empty store / missing namespaces / missing keys - "test_legacy.cpp" # the 1.x config file (legacy_fixtures.h) imported on first boot - "test_corrupt.cpp" # truncated / garbage / tampered input - "test_storage.cpp" # store selection, the move from the shared nvs into cfg, the registry + "test_corrupt.cpp" # malformed blobs and JSON + "test_storage.cpp" # the store is cfg, the registry, mistyped keys # The framework "${_components}/config/src/Nvs.cpp" "${_components}/config/src/Parse.cpp" "${_components}/config/src/Section.cpp" "${_components}/config/src/Store.cpp" - "${_components}/config/src/Wire.cpp" # Each module's section "${_components}/device_control/src/radio/RFConfig.cpp" "${_components}/device_control/src/estop/EStopConfig.cpp" @@ -35,17 +32,14 @@ idf_component_register( "${_components}/common/src/SimpleMutex.cpp" # the section and store locks "${_components}/common/src/StringHelpers.cpp" INCLUDE_DIRS "." - "stubs" # fs/LfsPartition.h (in-memory) "${_components}/config/include" # config/*.h "${_components}/device_control/include" # radio/RFConfig.h, estop/EStopConfig.h "${_components}/core/include" # wifi/WiFiConfig.h, BackendConfig.h, OtaUpdateConfig.h "${_components}/captive_portal/include" # captiveportal/CaptivePortalConfig.h "${_components}/serial_console/include" # serial_console/SerialInputConfig.h "${_components}/chipset/include" # Chipset.h - "${_components}/common/include" # OpenShock.h, SimpleMutex.h, TinyVec.h, enums/*, util/HexUtils.h + "${_components}/common/include" # OpenShock.h, SimpleMutex.h, enums/*, util/HexUtils.h "${_components}/osjson/include" # json/Json.h - "${_components}/serialization/include" # serialization/_fbs/HubConfig_generated.h, VerifiedRoot.h - "${_components}/flatbuffers/flatbuffers/include" REQUIRES unity host_support # Logging.h, openshock_board.h, ESP32 GPIO caps (test/host_support) freertos # SimpleMutex (real POSIX port) diff --git a/components/config/host_test/main/config_test_helpers.cpp b/components/config/host_test/main/config_test_helpers.cpp index d76d19c7f..bdd1400a9 100644 --- a/components/config/host_test/main/config_test_helpers.cpp +++ b/components/config/host_test/main/config_test_helpers.cpp @@ -4,8 +4,6 @@ #include "unity.h" -#include "json/Json.h" - #include #include @@ -17,29 +15,8 @@ using namespace OpenShock; namespace ConfigTest { bool Full::SaveNvs(const char* partition) const { - if (!rf.SaveNvs(partition, nullptr) || !wifi.SaveNvs(partition, nullptr) || !captivePortal.SaveNvs(partition, nullptr) || !backend.SaveNvs(partition, nullptr) || !serialInput.SaveNvs(partition, nullptr) || !otaUpdate.SaveNvs(partition, nullptr) - || !estop.SaveNvs(partition, nullptr)) - { - return false; - } - auto root = Config::Nvs::Handle::Open(partition, Keys::Ns::kRoot, true); - return root && root->SetU8(Keys::Root::kSchema, Keys::kSchemaVersion) && root->Commit(); - } - - std::string Full::ToJSON(bool withSensitiveData) const - { - JSON::StringWriter writer; - json_gen_str_t* gen = writer.gen(); - json_gen_start_object(gen); - rf.ToJSON(gen, "rf", withSensitiveData); - wifi.ToJSON(gen, "wifi", withSensitiveData); - captivePortal.ToJSON(gen, "captivePortal", withSensitiveData); - backend.ToJSON(gen, "backend", withSensitiveData); - serialInput.ToJSON(gen, "serialInput", withSensitiveData); - otaUpdate.ToJSON(gen, "otaUpdate", withSensitiveData); - estop.ToJSON(gen, "estop", withSensitiveData); - json_gen_end_object(gen); - return writer.finish(); + return rf.SaveNvs(partition, nullptr) && wifi.SaveNvs(partition, nullptr) && captivePortal.SaveNvs(partition, nullptr) && backend.SaveNvs(partition, nullptr) && serialInput.SaveNvs(partition, nullptr) && otaUpdate.SaveNvs(partition, nullptr) + && estop.SaveNvs(partition, nullptr); } Full MakeSampleConfig() @@ -63,52 +40,12 @@ namespace ConfigTest { return config; } - std::vector Serialize(const Full& config, bool withSensitiveData) - { - // ForceDefaults like the firmware's own writes, so the bytes match what Config stores and GetRaw returns - flatbuffers::FlatBufferBuilder builder; - builder.ForceDefaults(true); - auto rf = config.rf.ToFlatbuffers(builder, withSensitiveData); - auto wifi = config.wifi.ToFlatbuffers(builder, withSensitiveData); - auto captivePortal = config.captivePortal.ToFlatbuffers(builder, withSensitiveData); - auto backend = config.backend.ToFlatbuffers(builder, withSensitiveData); - auto serialInput = config.serialInput.ToFlatbuffers(builder, withSensitiveData); - auto otaUpdate = config.otaUpdate.ToFlatbuffers(builder, withSensitiveData); - auto estop = config.estop.ToFlatbuffers(builder, withSensitiveData); - auto root = Serialization::Configuration::CreateHubConfig(builder, rf, wifi, captivePortal, backend, serialInput, otaUpdate, estop); - Serialization::Configuration::FinishHubConfigBuffer(builder, root); - return std::vector(builder.GetBufferPointer(), builder.GetBufferPointer() + builder.GetSize()); - } - - bool Deserialize(const uint8_t* data, std::size_t size, Full& out) - { - auto json = Config::HubConfigToJSON(std::span(data, size)); - if (!json) { - return false; - } - JSON::JsonDocument doc; - if (!doc.parse(*json)) { - return false; - } - JSON::JsonView root = doc.root(); - return out.rf.FromJSON(root["rf"]) && out.wifi.FromJSON(root["wifi"]) && out.captivePortal.FromJSON(root["captivePortal"]) && out.backend.FromJSON(root["backend"]) && out.serialInput.FromJSON(root["serialInput"]) - && out.otaUpdate.FromJSON(root["otaUpdate"]) && out.estop.FromJSON(root["estop"]); - } - - static void wipePartition(const char* label) - { - nvs_flash_deinit_partition(label); // ESP_ERR_NVS_NOT_INITIALIZED when it is not up, which is fine - TEST_ASSERT_EQUAL(ESP_OK, nvs_flash_erase_partition(label)); - } - void WipeStore() { - wipePartition(Keys::kPartitionCfg); - wipePartition(Keys::kPartitionShared); - // Back up right away, so a test can write straight into either store; Config::Init's own init is then a no-op + nvs_flash_deinit_partition(Keys::kPartitionCfg); // ESP_ERR_NVS_NOT_INITIALIZED when it is not up, which is fine + TEST_ASSERT_EQUAL(ESP_OK, nvs_flash_erase_partition(Keys::kPartitionCfg)); + // Back up right away, so a test can write straight into the store; Config::Init's own init is then a no-op TEST_ASSERT_EQUAL(ESP_OK, nvs_flash_init_partition(Keys::kPartitionCfg)); - TEST_ASSERT_EQUAL(ESP_OK, nvs_flash_init_partition(Keys::kPartitionShared)); - HostLfs::reset(); } void InitDefault() @@ -117,23 +54,6 @@ namespace ConfigTest { Config::Init(); } - bool InitFromLegacy(const uint8_t* data, std::size_t size) - { - WipeStore(); - Full expected; - const bool parses = data != nullptr && Deserialize(data, size, expected); - if (data != nullptr) { - HostLfs::files()[kLegacyPath].assign(data, data + size); - } - Config::Init(); - TEST_ASSERT_EQUAL_INT_MESSAGE(0, HostLfs::writeCount, "the 1.x file must be left untouched"); - if (!parses) { - return false; - } - // Imported when what the hub runs with is what the file held (as this firmware decodes it) - return expected.ToJSON() == Snapshot().ToJSON(); - } - void Reload() { Config::Init(); @@ -155,18 +75,8 @@ namespace ConfigTest { bool NamespaceExists(const char* partition, const char* ns) { - return OpenShock::Config::Nvs::Handle::Open(partition, ns, false).has_value(); - } - - std::size_t KeyCount(const char* partition, const char* ns) - { - auto handle = OpenShock::Config::Nvs::Handle::Open(partition, ns, false); - if (!handle) { - return 0; - } - std::size_t count = 0; - TEST_ASSERT_TRUE(handle->ForEachKey(NVS_TYPE_ANY, [&](const char*) { ++count; })); - return count; + auto handle = OpenShock::Config::Nvs::Handle::OpenForLoad(partition, ns); + return handle && handle->Exists(); } Full Snapshot() diff --git a/components/config/host_test/main/config_test_helpers.h b/components/config/host_test/main/config_test_helpers.h index db0636db0..1f980bf68 100644 --- a/components/config/host_test/main/config_test_helpers.h +++ b/components/config/host_test/main/config_test_helpers.h @@ -3,10 +3,9 @@ // Shared helpers for the config host tests. The suite runs the config framework (components/config) together with // every module's section (device_control's rf and estop, core's wifi, backend and otaUpdate, captive_portal's // captivePortal, serial_console's serialInput), so it checks the store as the firmware assembles it: over ESP-IDF's -// real NVS on the linux flash emulation, with the 1.x littlefs file faked by stubs/fs/LfsPartition.h. +// real NVS on the linux flash emulation. #include "config/Nvs.h" #include "config/Store.h" -#include "config/Wire.h" #include "BackendConfig.h" #include "captiveportal/CaptivePortalConfig.h" @@ -16,24 +15,17 @@ #include "serial_console/SerialInputConfig.h" #include "wifi/WiFiConfig.h" -#include "fs/LfsPartition.h" - #include #include #include #include namespace ConfigTest { - // Path the store's 1.x import reads inside the 1.x config partition (the Arduino firmware's "/config"). - constexpr const char* kLegacyPath = "/config"; - // The storage format the tests check against: the store's own names and each module's. Written out here on purpose, // so a renamed key or namespace (which would orphan what devices hold) fails a test. namespace Keys { - inline constexpr const char* kPartitionCfg = "cfg"; - inline constexpr const char* kPartitionShared = "nvs"; + inline constexpr const char* kPartitionCfg = "cfg"; namespace Ns { - inline constexpr const char* kRoot = "root"; inline constexpr const char* kBackend = "backend"; inline constexpr const char* kWifi = "wifi"; inline constexpr const char* kWifiCfg = "wifi.cfg"; @@ -42,12 +34,8 @@ namespace ConfigTest { inline constexpr const char* kPortal = "portal"; inline constexpr const char* kSerial = "serial"; inline constexpr const char* kOta = "ota"; - inline constexpr const char* kAll[] = {kRoot, kBackend, kWifi, kWifiCfg, kRf, kEstop, kPortal, kSerial, kOta}; + inline constexpr const char* kAll[] = {kBackend, kWifi, kWifiCfg, kRf, kEstop, kPortal, kSerial, kOta}; } // namespace Ns - namespace Root { - inline constexpr const char* kSchema = "schema"; - } - inline constexpr uint8_t kSchemaVersion = 1; namespace Backend { inline constexpr const char* kDomain = "domain"; inline constexpr const char* kToken = "token"; @@ -69,8 +57,7 @@ namespace ConfigTest { } } // namespace Keys - // Every section's value at once: what a hub's whole config is (the role RootConfig had before the sections moved - // into their modules). Default-constructed, every member holds its defaults. + // Every section's value at once: what a hub's whole config is. Default-constructed, every member holds its defaults. struct Full { OpenShock::Config::RFConfig rf; OpenShock::Config::WiFiConfig wifi; @@ -80,35 +67,19 @@ namespace ConfigTest { OpenShock::Config::OtaUpdateConfig otaUpdate; OpenShock::Config::EStopConfig estop; - // Every section's keys, plus the store's schema marker, so the store loads rather than fills it + // Every section's keys bool SaveNvs(const char* partition) const; - // As the jsonconfig document renders it - std::string ToJSON(bool withSensitiveData = true) const; }; // Every field differs from the defaults. Full MakeSampleConfig(); - // Serializes as the 1.x firmware stored it (each section's FlatBuffers form in a HubConfig): a 1.x config file. - std::vector Serialize(const Full& config, bool withSensitiveData = true); - - // Decodes a HubConfig the way the store imports one (HubConfig -> jsonconfig document -> each section's FromJSON). - bool Deserialize(const uint8_t* data, std::size_t size, Full& out); - - // Erases the `cfg` and shared `nvs` partitions and the fake 1.x file: a hub with nothing stored. + // Erases the `cfg` partition: a hub with nothing stored. void WipeStore(); - // WipeStore + Config::Init: a hub booting on the defaults, which Init writes to the store. + // WipeStore + Config::Init: a hub booting on the defaults. void InitDefault(); - // Boots as a 1.x hub would after installing this firmware: nothing in NVS, `data` as its config file - // (nullptr = no file). Returns whether Init imported the file (as opposed to falling back to defaults). - bool InitFromLegacy(const uint8_t* data, std::size_t size); - inline bool InitFromLegacy(const std::vector& data) - { - return InitFromLegacy(data.data(), data.size()); - } - // A reboot: Config::Init again on whatever the store holds. void Reload(); @@ -122,9 +93,6 @@ namespace ConfigTest { // been erased. bool NamespaceExists(const char* partition, const char* ns); - // The number of keys in `ns` of `partition`; 0 for a namespace that does not exist. - std::size_t KeyCount(const char* partition, const char* ns); - // Reads back every section through the modules' public accessors. Full Snapshot(); diff --git a/components/config/host_test/main/legacy_fixtures.h b/components/config/host_test/main/legacy_fixtures.h deleted file mode 100644 index 592e0b862..000000000 --- a/components/config/host_test/main/legacy_fixtures.h +++ /dev/null @@ -1,133 +0,0 @@ -#pragma once - -// Config files exactly as the Arduino firmware wrote them to the "config" littlefs -// partition, used to check that an upgraded hub keeps its settings. -// -// How these bytes were produced: ../legacy/gen_legacy_config.cpp (host_test/legacy) -// contains the RootConfig / *Config::ToFlatbuffers bodies of the old firmware, -// transcribed verbatim, followed by FinishHubConfigBuffer as in its Config.cpp -// trySaveConfig. It was compiled against the old firmware's own generated schema -// header and the flatbuffers commit that firmware pinned in platformio.ini: -// -// git show :include/serialization/_fbs/HubConfig_generated.h (and WifiAuthMode_generated.h for develop) -// git -C components/flatbuffers/flatbuffers archive include -// -// 1.5.2 (latest stable release): flatbuffers 8b02fe6178427b96aea25396b53ea4ae8cadd7d8 -// develop (d9a47e53, same schema as 1.6.0-rc.1): flatbuffers 81edeb17d9118143f2c81caf27edfb0df401279e, built with -DLEGACY_DEVELOP -// -// All three hold the same settings (see test_legacy.cpp for the expected values): -// rf txPin 15, keepAlive true (kV152ConfigKeepAliveOff / kDevelopConfig: false) -// wifi AP SSID "MyHubAP", hostname "my-hub", credentials -// 1 "HomeNet" / "hunter22" (develop: WPA2_PSK, BSSID 12:34:56:78:9a:bc) -// 2 "Guest" / "" (develop: Open) -// 3 "Office" / "s3cret!" (develop: auth mode UNKNOWN) -// captive alwaysEnabled true -// backend domain "api.example.org", authToken "tok_0123456789abcdef" -// serialInput echo false -// otaUpdate disabled, cdn "fw.example.org", channel Beta, checkOnStartup, checkPeriodically, -// interval 120, no backend management, manual approval, updateId 4242, step Validating -// estop enabled, GPIO13 -// 1.5.2's schema has no WiFiCredentials auth_mode / bssid, and declares RFConfig -// keepalive_enabled with default false (develop: true). - -#include - -namespace ConfigTest::Legacy { - // clang-format off - // kV152Config (436 bytes) - inline constexpr uint8_t kV152Config[] = { - 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x12, 0x00, 0x20, 0x00, 0x04, 0x00, 0x08, 0x00, 0x0c, 0x00, - 0x10, 0x00, 0x14, 0x00, 0x18, 0x00, 0x1c, 0x00, 0x12, 0x00, 0x00, 0x00, 0x90, 0x01, 0x00, 0x00, - 0xcc, 0x00, 0x00, 0x00, 0xb8, 0x00, 0x00, 0x00, 0x70, 0x00, 0x00, 0x00, 0x5c, 0x00, 0x00, 0x00, - 0x28, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x94, 0xfe, 0xff, 0xff, 0x00, 0x00, 0x01, 0x0d, - 0x18, 0x00, 0x14, 0x00, 0x00, 0x00, 0x0c, 0x00, 0x05, 0x00, 0x06, 0x00, 0x07, 0x00, 0x0a, 0x00, - 0x00, 0x00, 0x08, 0x00, 0x10, 0x00, 0x09, 0x00, 0x18, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01, 0x01, - 0x01, 0x03, 0x78, 0x00, 0x08, 0x00, 0x00, 0x00, 0x92, 0x10, 0x00, 0x00, 0x0e, 0x00, 0x00, 0x00, - 0x66, 0x77, 0x2e, 0x65, 0x78, 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x2e, 0x6f, 0x72, 0x67, 0x00, 0x00, - 0x00, 0x00, 0x06, 0x00, 0x08, 0x00, 0x07, 0x00, 0x06, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x08, 0x00, 0x0c, 0x00, 0x04, 0x00, 0x08, 0x00, 0x08, 0x00, 0x00, 0x00, 0x24, 0x00, 0x00, 0x00, - 0x04, 0x00, 0x00, 0x00, 0x14, 0x00, 0x00, 0x00, 0x74, 0x6f, 0x6b, 0x5f, 0x30, 0x31, 0x32, 0x33, - 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x00, 0x00, 0x00, 0x00, - 0x0f, 0x00, 0x00, 0x00, 0x61, 0x70, 0x69, 0x2e, 0x65, 0x78, 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x2e, - 0x6f, 0x72, 0x67, 0x00, 0x00, 0x00, 0x06, 0x00, 0x06, 0x00, 0x05, 0x00, 0x06, 0x00, 0x00, 0x00, - 0x00, 0x01, 0x0a, 0x00, 0x10, 0x00, 0x04, 0x00, 0x08, 0x00, 0x0c, 0x00, 0x0a, 0x00, 0x00, 0x00, - 0x0c, 0x00, 0x00, 0x00, 0x14, 0x00, 0x00, 0x00, 0x1c, 0x00, 0x00, 0x00, 0x07, 0x00, 0x00, 0x00, - 0x4d, 0x79, 0x48, 0x75, 0x62, 0x41, 0x50, 0x00, 0x06, 0x00, 0x00, 0x00, 0x6d, 0x79, 0x2d, 0x68, - 0x75, 0x62, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x60, 0x00, 0x00, 0x00, 0x30, 0x00, 0x00, 0x00, - 0x04, 0x00, 0x00, 0x00, 0xb6, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x03, 0x14, 0x00, 0x00, 0x00, - 0x04, 0x00, 0x00, 0x00, 0x07, 0x00, 0x00, 0x00, 0x73, 0x33, 0x63, 0x72, 0x65, 0x74, 0x21, 0x00, - 0x06, 0x00, 0x00, 0x00, 0x4f, 0x66, 0x66, 0x69, 0x63, 0x65, 0x00, 0x00, 0xde, 0xff, 0xff, 0xff, - 0x00, 0x00, 0x00, 0x02, 0x10, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x47, 0x75, 0x65, 0x73, 0x74, 0x00, 0x0a, 0x00, - 0x10, 0x00, 0x07, 0x00, 0x08, 0x00, 0x0c, 0x00, 0x0a, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, - 0x18, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x68, 0x75, 0x6e, 0x74, - 0x65, 0x72, 0x32, 0x32, 0x00, 0x00, 0x00, 0x00, 0x07, 0x00, 0x00, 0x00, 0x48, 0x6f, 0x6d, 0x65, - 0x4e, 0x65, 0x74, 0x00, 0x08, 0x00, 0x08, 0x00, 0x06, 0x00, 0x07, 0x00, 0x08, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x0f, 0x01, - }; - // kV152ConfigKeepAliveOff (436 bytes) - inline constexpr uint8_t kV152ConfigKeepAliveOff[] = { - 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x12, 0x00, 0x20, 0x00, 0x04, 0x00, 0x08, 0x00, 0x0c, 0x00, - 0x10, 0x00, 0x14, 0x00, 0x18, 0x00, 0x1c, 0x00, 0x12, 0x00, 0x00, 0x00, 0x90, 0x01, 0x00, 0x00, - 0xcc, 0x00, 0x00, 0x00, 0xb8, 0x00, 0x00, 0x00, 0x70, 0x00, 0x00, 0x00, 0x5c, 0x00, 0x00, 0x00, - 0x30, 0x00, 0x00, 0x00, 0x0c, 0x00, 0x00, 0x00, 0x08, 0x00, 0x08, 0x00, 0x06, 0x00, 0x07, 0x00, - 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x0d, 0x18, 0x00, 0x14, 0x00, 0x00, 0x00, 0x0c, 0x00, - 0x05, 0x00, 0x06, 0x00, 0x07, 0x00, 0x0a, 0x00, 0x00, 0x00, 0x08, 0x00, 0x10, 0x00, 0x09, 0x00, - 0x18, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01, 0x01, 0x01, 0x03, 0x78, 0x00, 0x08, 0x00, 0x00, 0x00, - 0x92, 0x10, 0x00, 0x00, 0x0e, 0x00, 0x00, 0x00, 0x66, 0x77, 0x2e, 0x65, 0x78, 0x61, 0x6d, 0x70, - 0x6c, 0x65, 0x2e, 0x6f, 0x72, 0x67, 0x00, 0x00, 0xe2, 0xfe, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, - 0x08, 0x00, 0x0c, 0x00, 0x04, 0x00, 0x08, 0x00, 0x08, 0x00, 0x00, 0x00, 0x24, 0x00, 0x00, 0x00, - 0x04, 0x00, 0x00, 0x00, 0x14, 0x00, 0x00, 0x00, 0x74, 0x6f, 0x6b, 0x5f, 0x30, 0x31, 0x32, 0x33, - 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x00, 0x00, 0x00, 0x00, - 0x0f, 0x00, 0x00, 0x00, 0x61, 0x70, 0x69, 0x2e, 0x65, 0x78, 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x2e, - 0x6f, 0x72, 0x67, 0x00, 0x00, 0x00, 0x06, 0x00, 0x06, 0x00, 0x05, 0x00, 0x06, 0x00, 0x00, 0x00, - 0x00, 0x01, 0x0a, 0x00, 0x10, 0x00, 0x04, 0x00, 0x08, 0x00, 0x0c, 0x00, 0x0a, 0x00, 0x00, 0x00, - 0x0c, 0x00, 0x00, 0x00, 0x14, 0x00, 0x00, 0x00, 0x1c, 0x00, 0x00, 0x00, 0x07, 0x00, 0x00, 0x00, - 0x4d, 0x79, 0x48, 0x75, 0x62, 0x41, 0x50, 0x00, 0x06, 0x00, 0x00, 0x00, 0x6d, 0x79, 0x2d, 0x68, - 0x75, 0x62, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x60, 0x00, 0x00, 0x00, 0x30, 0x00, 0x00, 0x00, - 0x04, 0x00, 0x00, 0x00, 0xb6, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x03, 0x14, 0x00, 0x00, 0x00, - 0x04, 0x00, 0x00, 0x00, 0x07, 0x00, 0x00, 0x00, 0x73, 0x33, 0x63, 0x72, 0x65, 0x74, 0x21, 0x00, - 0x06, 0x00, 0x00, 0x00, 0x4f, 0x66, 0x66, 0x69, 0x63, 0x65, 0x00, 0x00, 0xde, 0xff, 0xff, 0xff, - 0x00, 0x00, 0x00, 0x02, 0x10, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x47, 0x75, 0x65, 0x73, 0x74, 0x00, 0x0a, 0x00, - 0x10, 0x00, 0x07, 0x00, 0x08, 0x00, 0x0c, 0x00, 0x0a, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, - 0x18, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x68, 0x75, 0x6e, 0x74, - 0x65, 0x72, 0x32, 0x32, 0x00, 0x00, 0x00, 0x00, 0x07, 0x00, 0x00, 0x00, 0x48, 0x6f, 0x6d, 0x65, - 0x4e, 0x65, 0x74, 0x00, 0x00, 0x00, 0x06, 0x00, 0x08, 0x00, 0x07, 0x00, 0x06, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x0f, - }; - // kDevelopConfig (476 bytes) - inline constexpr uint8_t kDevelopConfig[] = { - 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x12, 0x00, 0x20, 0x00, 0x04, 0x00, 0x08, 0x00, 0x0c, 0x00, - 0x10, 0x00, 0x14, 0x00, 0x18, 0x00, 0x1c, 0x00, 0x12, 0x00, 0x00, 0x00, 0xb8, 0x01, 0x00, 0x00, - 0xd0, 0x00, 0x00, 0x00, 0xbc, 0x00, 0x00, 0x00, 0x74, 0x00, 0x00, 0x00, 0x60, 0x00, 0x00, 0x00, - 0x28, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x6c, 0xfe, 0xff, 0xff, 0x00, 0x00, 0x01, 0x0d, - 0x18, 0x00, 0x18, 0x00, 0x07, 0x00, 0x10, 0x00, 0x08, 0x00, 0x09, 0x00, 0x0a, 0x00, 0x0e, 0x00, - 0x0b, 0x00, 0x0c, 0x00, 0x14, 0x00, 0x0d, 0x00, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x01, 0x01, 0x01, 0x00, 0x01, 0x03, 0x78, 0x00, 0x08, 0x00, 0x00, 0x00, 0x92, 0x10, 0x00, 0x00, - 0x0e, 0x00, 0x00, 0x00, 0x66, 0x77, 0x2e, 0x65, 0x78, 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x2e, 0x6f, - 0x72, 0x67, 0x00, 0x00, 0x00, 0x00, 0x06, 0x00, 0x08, 0x00, 0x07, 0x00, 0x06, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x08, 0x00, 0x0c, 0x00, 0x04, 0x00, 0x08, 0x00, 0x08, 0x00, 0x00, 0x00, - 0x24, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x14, 0x00, 0x00, 0x00, 0x74, 0x6f, 0x6b, 0x5f, - 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x61, 0x62, 0x63, 0x64, 0x65, 0x66, - 0x00, 0x00, 0x00, 0x00, 0x0f, 0x00, 0x00, 0x00, 0x61, 0x70, 0x69, 0x2e, 0x65, 0x78, 0x61, 0x6d, - 0x70, 0x6c, 0x65, 0x2e, 0x6f, 0x72, 0x67, 0x00, 0x00, 0x00, 0x06, 0x00, 0x06, 0x00, 0x05, 0x00, - 0x06, 0x00, 0x00, 0x00, 0x00, 0x01, 0x0a, 0x00, 0x10, 0x00, 0x04, 0x00, 0x08, 0x00, 0x0c, 0x00, - 0x0a, 0x00, 0x00, 0x00, 0x0c, 0x00, 0x00, 0x00, 0x14, 0x00, 0x00, 0x00, 0x1c, 0x00, 0x00, 0x00, - 0x07, 0x00, 0x00, 0x00, 0x4d, 0x79, 0x48, 0x75, 0x62, 0x41, 0x50, 0x00, 0x06, 0x00, 0x00, 0x00, - 0x6d, 0x79, 0x2d, 0x68, 0x75, 0x62, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x7c, 0x00, 0x00, 0x00, - 0x48, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0a, 0x00, 0x10, 0x00, 0x07, 0x00, - 0x08, 0x00, 0x0c, 0x00, 0x0a, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0x14, 0x00, 0x00, 0x00, - 0x04, 0x00, 0x00, 0x00, 0x07, 0x00, 0x00, 0x00, 0x73, 0x33, 0x63, 0x72, 0x65, 0x74, 0x21, 0x00, - 0x06, 0x00, 0x00, 0x00, 0x4f, 0x66, 0x66, 0x69, 0x63, 0x65, 0x00, 0x00, 0x0c, 0x00, 0x10, 0x00, - 0x06, 0x00, 0x08, 0x00, 0x0c, 0x00, 0x07, 0x00, 0x0c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x00, - 0x10, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x05, 0x00, 0x00, 0x00, 0x47, 0x75, 0x65, 0x73, 0x74, 0x00, 0x0e, 0x00, 0x18, 0x00, 0x06, 0x00, - 0x08, 0x00, 0x0c, 0x00, 0x07, 0x00, 0x12, 0x00, 0x0e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x03, - 0x20, 0x00, 0x00, 0x00, 0x0c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x12, 0x34, 0x56, 0x78, 0x9a, 0xbc, - 0x08, 0x00, 0x00, 0x00, 0x68, 0x75, 0x6e, 0x74, 0x65, 0x72, 0x32, 0x32, 0x00, 0x00, 0x00, 0x00, - 0x07, 0x00, 0x00, 0x00, 0x48, 0x6f, 0x6d, 0x65, 0x4e, 0x65, 0x74, 0x00, 0x08, 0x00, 0x08, 0x00, - 0x06, 0x00, 0x07, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0f, 0x00, - }; - // clang-format on -} // namespace ConfigTest::Legacy diff --git a/components/config/host_test/main/stubs/fs/LfsPartition.h b/components/config/host_test/main/stubs/fs/LfsPartition.h deleted file mode 100644 index 10ce14234..000000000 --- a/components/config/host_test/main/stubs/fs/LfsPartition.h +++ /dev/null @@ -1,83 +0,0 @@ -#pragma once - -// Host-test stub for fs/LfsPartition.h: an in-memory file store instead of littlefs on a flash partition, so the -// tests can stand in the config file a 1.x hub left behind (HostLfs::files()) and see that the import never writes -// to it. Every LfsPartition instance shares one store. -#include "OpenShock.h" -#include "util/FunctionRef.h" - -#include - -#include -#include -#include -#include -#include - -namespace OpenShock { - struct HostLfs { - static std::map>& files() - { - static std::map> s_files; - return s_files; - } - static inline bool mountable = true; // false: the partition holds no mountable image - static inline int writeCount = 0; // writes through the stub: the legacy import must never make one - - static void reset() - { - files().clear(); - mountable = true; - writeCount = 0; - } - }; - - class LfsPartition { - DISABLE_COPY(LfsPartition); - DISABLE_MOVE(LfsPartition); - - public: - LfsPartition() = default; - - bool mount(const esp_partition_t* partition, bool writable, bool formatOnFail = false) - { - (void)writable; - (void)formatOnFail; - m_mounted = partition != nullptr && HostLfs::mountable; - return m_mounted; - } - void unmount() { m_mounted = false; } - bool isMounted() const { return m_mounted; } - - bool exists(const char* path) { return m_mounted && HostLfs::files().contains(path); } - bool readStream(const char* path, Util::FunctionRef)> sink) - { - std::vector data; - return readAll(path, data) && sink(data); - } - bool readAll(const char* path, std::vector& out) - { - auto it = HostLfs::files().find(path); - if (!m_mounted || it == HostLfs::files().end()) { - out.clear(); - return false; - } - out = it->second; - return true; - } - bool writeAll(const char* path, std::span data) - { - ++HostLfs::writeCount; - HostLfs::files()[path].assign(data.begin(), data.end()); - return true; - } - bool remove(const char* path) - { - ++HostLfs::writeCount; - return HostLfs::files().erase(path) != 0; - } - - private: - bool m_mounted = false; - }; -} // namespace OpenShock diff --git a/components/config/host_test/main/test_corrupt.cpp b/components/config/host_test/main/test_corrupt.cpp index 245314984..c9af874d6 100644 --- a/components/config/host_test/main/test_corrupt.cpp +++ b/components/config/host_test/main/test_corrupt.cpp @@ -1,11 +1,10 @@ -// Corrupt input: a truncated, garbage or tampered 1.x config file, a malformed credential blob, or bad JSON over -// serial must be rejected without crashing, and must not replace the stored config. +// Corrupt input: a malformed credential blob or bad JSON over serial must +// be rejected without crashing, and must not replace the stored config. #include #include "unity.h" #include "config_test_helpers.h" -#include "legacy_fixtures.h" #include #include @@ -26,116 +25,8 @@ namespace { state ^= state << 5; return state; } - - // Runs a 1.x config blob through the import: the parser must reject it, and booting with it as the file must - // fall back to the defaults. Returns whether the parser accepted it; never crashes (that is the point). - bool TryImport(const std::vector& data) - { - Full parsed; - const bool accepted = Deserialize(data.data(), data.size(), parsed); - const bool imported = InitFromLegacy(data); - TEST_ASSERT_EQUAL_MESSAGE(accepted, imported, "the import and the parser must agree"); - Snapshot(); - return accepted; - } } // namespace -TEST_CASE("Every truncation of a legacy config is rejected", "[config][corrupt]") -{ - for (std::size_t len = 0; len < sizeof(Legacy::kDevelopConfig); ++len) { - std::vector data(Legacy::kDevelopConfig, Legacy::kDevelopConfig + len); - Full parsed; - char msg[48]; - snprintf(msg, sizeof(msg), "truncated to %zu bytes", len); - TEST_ASSERT_FALSE_MESSAGE(Deserialize(data.data(), data.size(), parsed), msg); - } -} - -TEST_CASE("A truncated 1.x file boots with defaults", "[config][corrupt]") -{ - const std::vector truncated(Legacy::kDevelopConfig, Legacy::kDevelopConfig + sizeof(Legacy::kDevelopConfig) / 2); - TEST_ASSERT_FALSE(InitFromLegacy(truncated)); // fell back - AssertConfigEqual(Full(), Snapshot()); - - // The defaults were written to NVS: the next boot loads them and leaves the bad file alone. - Reload(); - AssertConfigEqual(Full(), Snapshot()); - TEST_ASSERT_EQUAL_INT(0, HostLfs::writeCount); -} - -TEST_CASE("An oversized 1.x file is rejected, not aborted on", "[config][corrupt]") -{ - // A well-formed flatbuffer, just larger than the 1.x partition could ever hold: the Verifier's constructor - // asserts (size < max_size), so the size is checked first. - Full sample = MakeSampleConfig(); - sample.wifi.hostname.assign(5000, 'h'); - TEST_ASSERT_FALSE(InitFromLegacy(Serialize(sample))); - AssertConfigEqual(Full(), Snapshot()); -} - -TEST_CASE("Random garbage is rejected", "[config][corrupt]") -{ - uint32_t state = 0x12345678; - for (int round = 0; round < 300; ++round) { - std::vector data(4 + NextRandom(state) % 600); - for (auto& b : data) b = static_cast(NextRandom(state)); - Full parsed; - char msg[48]; - snprintf(msg, sizeof(msg), "garbage round %d", round); - TEST_ASSERT_FALSE_MESSAGE(Deserialize(data.data(), data.size(), parsed), msg); - } - TryImport(std::vector(64, 0xFF)); -} - -TEST_CASE("Bit flips in a legacy config never crash the loader", "[config][corrupt]") -{ - // A flipped byte may still form a valid (different) config; what matters is that - // verification keeps every read in bounds (run under ASan/UBSan to see it). - for (std::size_t i = 0; i < sizeof(Legacy::kDevelopConfig); ++i) { - for (uint8_t mask : {0x01, 0x80, 0xFF}) { - std::vector data(Legacy::kDevelopConfig, Legacy::kDevelopConfig + sizeof(Legacy::kDevelopConfig)); - data[i] ^= mask; - Full parsed; - Deserialize(data.data(), data.size(), parsed); - } - } -} - -TEST_CASE("Out-of-range offsets in the root table are rejected", "[config][corrupt]") -{ - std::vector data(Legacy::kDevelopConfig, Legacy::kDevelopConfig + sizeof(Legacy::kDevelopConfig)); - - // Root offset pointing past the end of the buffer. - data[0] = 0xF0; - data[1] = 0xFF; - TEST_ASSERT_FALSE(TryImport(data)); - - // Root offset pointing at the last byte. - const uint32_t last = static_cast(data.size() - 1); - data[0] = static_cast(last); - data[1] = static_cast(last >> 8); - TEST_ASSERT_FALSE(TryImport(data)); -} - -TEST_CASE("Unterminated and oversized strings are rejected", "[config][corrupt]") -{ - // Make the length prefix of the backend domain string ("api.example.org") absurd. - std::vector data(Legacy::kDevelopConfig, Legacy::kDevelopConfig + sizeof(Legacy::kDevelopConfig)); - const std::string needle = "api.example.org"; - auto it = std::search(data.begin(), data.end(), needle.begin(), needle.end()); - TEST_ASSERT_TRUE(it != data.end()); - const std::size_t lenPos = static_cast(it - data.begin()) - 4; - TEST_ASSERT_EQUAL_UINT8(needle.size(), data[lenPos]); - - data[lenPos + 1] = 0x7F; // ~32 KiB long - TEST_ASSERT_FALSE(TryImport(data)); - - // Correct length, but the terminating NUL overwritten. - data[lenPos + 1] = 0x00; - data[lenPos + 4 + needle.size()] = 'X'; - TEST_ASSERT_FALSE(TryImport(data)); -} - TEST_CASE("Every truncation and corruption of a credential blob is rejected without crashing", "[config][corrupt]") { Config::WiFiCredentials sample(7, "Office", "s3cret!", WIFI_AUTH_WPA2_PSK); @@ -335,51 +226,12 @@ TEST_CASE("An invalid hostname is normalized to the default on every path", "[co TEST_ASSERT_EQUAL_STRING("hub-1", Snapshot().wifi.hostname.c_str()); } -TEST_CASE("Schema fields the firmware does not model are dropped on import", "[config][corrupt]") -{ - // estop.active / estop.latching (and wifi AP/LAN fields) exist in the 1.x schema but nothing in the firmware - // reads them, so they are not imported and never stored. This pins that behaviour. - namespace Fbs = Serialization::Configuration; - flatbuffers::FlatBufferBuilder builder; - auto estop = Fbs::CreateEStopConfig(builder, false, -1, /*active=*/true, /*latching=*/true); - Fbs::FinishHubConfigBuffer(builder, Fbs::CreateHubConfig(builder, 0, 0, 0, 0, 0, 0, estop)); - TEST_ASSERT_TRUE(InitFromLegacy(builder.GetBufferPointer(), builder.GetSize())); - - flatbuffers::FlatBufferBuilder out; - Fbs::FinishHubConfigBuffer(out, Config::GetAsFlatBuffer(out, true)); - const auto* root = Fbs::GetHubConfig(out.GetBufferPointer()); - TEST_ASSERT_NOT_NULL(root->estop()); - TEST_ASSERT_FALSE(root->estop()->active()); - TEST_ASSERT_FALSE(root->estop()->latching()); -} - TEST_CASE("RemoveWiFiCredentials reports unknown IDs", "[config]") { InitDefault(); TEST_ASSERT_FALSE(Config::RemoveWiFiCredentials(42)); } -TEST_CASE("Out-of-range E-Stop GPIO in a 1.x config falls back to the default", "[config][corrupt]") -{ - // gpio_pin is an int8 in the file; values outside gpio_num_t must not be cast to it - // (undefined behaviour, flagged by UBSan) and mean a corrupt section. - namespace Fbs = Serialization::Configuration; - for (int8_t pin : {static_cast(-128), static_cast(-2), static_cast(GPIO_NUM_MAX), static_cast(127)}) { - flatbuffers::FlatBufferBuilder builder; - auto estop = Fbs::CreateEStopConfig(builder, true, pin); - auto root = Fbs::CreateHubConfig(builder, 0, 0, 0, 0, 0, 0, estop); - Fbs::FinishHubConfigBuffer(builder, root); - - char msg[32]; - snprintf(msg, sizeof(msg), "gpio_pin %d", pin); - TEST_ASSERT_TRUE_MESSAGE(InitFromLegacy(builder.GetBufferPointer(), builder.GetSize()), msg); - - auto estopConfig = Config::GetEStopConfig(); - TEST_ASSERT_EQUAL_INT_MESSAGE(Config::EStopConfig().gpioPin, estopConfig.gpioPin, msg); - TEST_ASSERT_EQUAL_MESSAGE(Config::EStopConfig().enabled, estopConfig.enabled, msg); - } -} - TEST_CASE("jsonconfig set survives deeply nested and huge input", "[config][corrupt][json]") { InitDefault(); diff --git a/components/config/host_test/main/test_defaults.cpp b/components/config/host_test/main/test_defaults.cpp index c929e05ea..e8fc8114e 100644 --- a/components/config/host_test/main/test_defaults.cpp +++ b/components/config/host_test/main/test_defaults.cpp @@ -19,21 +19,13 @@ namespace Keys = ConfigTest::Keys; namespace Nvs = OpenShock::Config::Nvs; namespace { - // Writes a few keys straight into the store, as a different firmware version might have, then marks the schema - // so Init loads rather than imports. + // Writes a few keys straight into the store, as a different firmware version might have. Nvs::Handle Open(const char* ns) { - auto handle = Nvs::Handle::Open(Keys::kPartitionCfg, ns, true); + auto handle = Nvs::Handle::OpenForWrite(Keys::kPartitionCfg, ns); TEST_ASSERT_TRUE(handle.has_value()); return std::move(*handle); } - - void MarkSchema() - { - auto root = Open(Keys::Ns::kRoot); - TEST_ASSERT_TRUE(root.SetU8(Keys::Root::kSchema, Keys::kSchemaVersion)); - TEST_ASSERT_TRUE(root.Commit()); - } } // namespace TEST_CASE("Default values come from Kconfig and the board descriptor", "[config][defaults]") @@ -66,20 +58,25 @@ TEST_CASE("Default values come from Kconfig and the board descriptor", "[config] TEST_ASSERT_FALSE(config.estop.enabled); } -TEST_CASE("An empty store: Init writes a default config that reloads", "[config][defaults]") +TEST_CASE("An empty store reads as the defaults and Init writes nothing", "[config][defaults]") { - InitDefault(); + WipeStore(); + const std::size_t before = UsedEntries(); + Config::Init(); TEST_ASSERT_EQUAL_STRING("cfg", Config::StorageLabel()); AssertConfigEqual(Full(), Snapshot()); + TEST_ASSERT_EQUAL_size_t(before, UsedEntries()); + + Reload(); + AssertConfigEqual(Full(), Snapshot()); +} - // Every section namespace and the schema marker are now on flash +TEST_CASE("Every section is saved under its own namespaces", "[config][defaults]") +{ + InitWith(MakeSampleConfig()); for (const char* ns : Keys::Ns::kAll) { TEST_ASSERT_TRUE_MESSAGE(NamespaceExists(Keys::kPartitionCfg, ns), ns); } - TEST_ASSERT_TRUE(UsedEntries() > 0); - - Reload(); - AssertConfigEqual(Full(), Snapshot()); } TEST_CASE("Missing sections default while present ones are kept", "[config][defaults]") @@ -95,7 +92,6 @@ TEST_CASE("Missing sections default while present ones are kept", "[config][defa TEST_ASSERT_TRUE(estop.SetI8(Keys::Estop::kGpio, 13)); TEST_ASSERT_TRUE(estop.Commit()); } - MarkSchema(); Reload(); Full expected; @@ -121,7 +117,6 @@ TEST_CASE("Missing keys inside a section take their defaults", "[config][default TEST_ASSERT_TRUE(ota.SetI32(Keys::Ota::kUpdateId, 77)); // the rest at their defaults TEST_ASSERT_TRUE(ota.Commit()); } - MarkSchema(); Reload(); const Full loaded = Snapshot(); @@ -189,7 +184,6 @@ TEST_CASE("Invalid stored GPIO pins fall back safely", "[config][defaults]") TEST_ASSERT_TRUE(estop.SetI8(Keys::Estop::kGpio, 0)); // strapping pin: not a valid E-Stop input TEST_ASSERT_TRUE(estop.Commit()); } - MarkSchema(); Reload(); TEST_ASSERT_EQUAL_INT(OPENSHOCK_RF_TX_GPIO, Config::GetRFConfigTxPin()); @@ -202,7 +196,6 @@ TEST_CASE("Invalid stored GPIO pins fall back safely", "[config][defaults]") TEST_ASSERT_TRUE(rf.SetI8(Keys::Rf::kTxPin, -1)); TEST_ASSERT_TRUE(rf.Commit()); } - MarkSchema(); Reload(); TEST_ASSERT_EQUAL_INT(OPENSHOCK_RF_TX_GPIO, Config::GetRFConfigTxPin()); } @@ -230,9 +223,31 @@ TEST_CASE("jsonconfig set with partial sections fills in defaults", "[config][de AssertConfigEqual(expected, Snapshot()); } +TEST_CASE("The 1.x CDN domains already in NVS move to the repository server; a custom one stays", "[config][defaults]") +{ + InitWith(MakeSampleConfig()); + for (const char* legacy : {"firmware.openshock.org", "fw.openshock.org"}) { + { + auto ota = Open(Keys::Ns::kOta); + TEST_ASSERT_TRUE(ota.SetStr("domain", legacy)); + TEST_ASSERT_TRUE(ota.Commit()); + } + Reload(); + TEST_ASSERT_EQUAL_STRING(CONFIG_OPENSHOCK_FW_CDN_DOMAIN, Snapshot().otaUpdate.cdnDomain.c_str()); + } + + { + auto ota = Open(Keys::Ns::kOta); + TEST_ASSERT_TRUE(ota.SetStr("domain", "fw.example.org")); + TEST_ASSERT_TRUE(ota.Commit()); + } + Reload(); + TEST_ASSERT_EQUAL_STRING("fw.example.org", Snapshot().otaUpdate.cdnDomain.c_str()); +} + TEST_CASE("jsonconfig set requires the wifi credentials array when wifi is given", "[config][defaults][json]") { - // Same rule as the develop firmware: a wifi object without "credentials" is rejected. + // A wifi object without "credentials" is rejected. InitDefault(); const std::size_t before = UsedEntries(); TEST_ASSERT_FALSE(Config::SaveFromJSON(R"({"wifi":{"hostname":"my-hub"}})")); diff --git a/components/config/host_test/main/test_legacy.cpp b/components/config/host_test/main/test_legacy.cpp deleted file mode 100644 index a3dda6e2a..000000000 --- a/components/config/host_test/main/test_legacy.cpp +++ /dev/null @@ -1,255 +0,0 @@ -// Upgrade safety: hubs flashed from the Arduino firmware (1.5.x releases and the develop branch) must keep their -// config. The fixtures are the exact bytes those firmwares stored in the "config" partition (see -// legacy_fixtures.h); the 1.9.9 pivot imports that file into NVS on the first boot that finds no NVS config. -#include - -#include "unity.h" - -#include "config_test_helpers.h" -#include "legacy_fixtures.h" - -#include -#include -#include - -using namespace OpenShock; -using namespace ConfigTest; - -namespace { - // The settings every legacy fixture holds (see legacy_fixtures.h). - Full ExpectedLegacyConfig(bool hasAuthModeAndBssid, bool keepAlive) - { - Full config; - config.rf = Config::RFConfig(GPIO_NUM_15, keepAlive); - - Config::WiFiCredentials home(1, "HomeNet", "hunter22"); - Config::WiFiCredentials guest(2, "Guest", ""); - Config::WiFiCredentials office(3, "Office", "s3cret!"); - if (hasAuthModeAndBssid) { - home.authMode = WIFI_AUTH_WPA2_PSK; - home.bssid = {0x12, 0x34, 0x56, 0x78, 0x9a, 0xbc}; - guest.authMode = WIFI_AUTH_OPEN; - } - config.wifi = Config::WiFiConfig("MyHubAP", "my-hub", {home, guest, office}); - - config.captivePortal = Config::CaptivePortalConfig(true); - config.backend = Config::BackendConfig("api.example.org", "tok_0123456789abcdef"); - config.serialInput = Config::SerialInputConfig(false); - config.otaUpdate = Config::OtaUpdateConfig(false, "fw.example.org", OtaUpdateChannel::Beta, true, true, 120, false, true, 4242, OtaUpdateStep::Validating); - config.estop = Config::EStopConfig(true, GPIO_NUM_13); - return config; - } - - // develop's `jsonconfig` output (cJSON_PrintUnformatted of RootConfig::ToJSON) for - // kDevelopConfig: same key order and number/bool/bssid formatting as its ToJSON bodies. - constexpr const char* kDevelopJson = R"({"rf":{"txPin":15,"keepAliveEnabled":false},)" - R"("wifi":{"accessPointSSID":"MyHubAP","hostname":"my-hub","credentials":[)" - R"({"id":1,"ssid":"HomeNet","password":"hunter22","authMode":3,"bssid":"123456789ABC"},)" - R"({"id":2,"ssid":"Guest","password":"","authMode":0},)" - R"({"id":3,"ssid":"Office","password":"s3cret!","authMode":9}]},)" - R"("captivePortal":{"alwaysEnabled":true},)" - R"("backend":{"domain":"api.example.org","authToken":"tok_0123456789abcdef"},)" - R"("serialInput":{"echoEnabled":false},)" - R"("otaUpdate":{"isEnabled":false,"cdnDomain":"fw.example.org","updateChannel":"Beta","checkOnStartup":true,"checkPeriodically":true,)" - R"("checkInterval":120,"allowBackendManagement":false,"requireManualApproval":true,"updateId":4242,"updateStep":"Validating"},)" - R"("estop":{"enabled":true,"gpioPin":13}})"; - - // 1.5.2's `jsonconfig` output for kV152Config (its credentials had no authMode/bssid). - constexpr const char* kV152Json = R"({"rf":{"txPin":15,"keepAliveEnabled":true},)" - R"("wifi":{"accessPointSSID":"MyHubAP","hostname":"my-hub","credentials":[)" - R"({"id":1,"ssid":"HomeNet","password":"hunter22"},)" - R"({"id":2,"ssid":"Guest","password":""},)" - R"({"id":3,"ssid":"Office","password":"s3cret!"}]},)" - R"("captivePortal":{"alwaysEnabled":true},)" - R"("backend":{"domain":"api.example.org","authToken":"tok_0123456789abcdef"},)" - R"("serialInput":{"echoEnabled":false},)" - R"("otaUpdate":{"isEnabled":false,"cdnDomain":"fw.example.org","updateChannel":"Beta","checkOnStartup":true,"checkPeriodically":true,)" - R"("checkInterval":120,"allowBackendManagement":false,"requireManualApproval":true,"updateId":4242,"updateStep":"Validating"},)" - R"("estop":{"enabled":true,"gpioPin":13}})"; -} // namespace - -TEST_CASE("Config stored by develop (Arduino) firmware is imported unchanged", "[config][legacy]") -{ - TEST_ASSERT_TRUE_MESSAGE(InitFromLegacy(Legacy::kDevelopConfig, sizeof(Legacy::kDevelopConfig)), "Init fell back to defaults"); - AssertConfigEqual(ExpectedLegacyConfig(true, false), Snapshot()); - - // Imported into NVS: a reboot loads it from there, with or without the file. - Reload(); - AssertConfigEqual(ExpectedLegacyConfig(true, false), Snapshot()); - HostLfs::files().clear(); - Reload(); - AssertConfigEqual(ExpectedLegacyConfig(true, false), Snapshot()); -} - -TEST_CASE("Config stored by 1.5.2 release firmware is imported unchanged", "[config][legacy]") -{ - TEST_ASSERT_TRUE_MESSAGE(InitFromLegacy(Legacy::kV152Config, sizeof(Legacy::kV152Config)), "Init fell back to defaults"); - - // 1.5.2 stored no auth mode / BSSID: they read back as unknown / not pinned. - // OTA updates and backend management are off in this file, which 1.5.2 encoded - // by omitting the fields (its schema default was false); the stored keepalive - // true shows the file uses that convention, so they must not read as true. - AssertConfigEqual(ExpectedLegacyConfig(false, true), Snapshot()); -} - -TEST_CASE("The 1.x file is read once: a store that has a config ignores it", "[config][legacy]") -{ - InitWith(MakeSampleConfig()); - HostLfs::files()[kLegacyPath].assign(Legacy::kDevelopConfig, Legacy::kDevelopConfig + sizeof(Legacy::kDevelopConfig)); - Reload(); - AssertConfigEqual(MakeSampleConfig(), Snapshot()); - TEST_ASSERT_EQUAL_INT(0, HostLfs::writeCount); -} - -TEST_CASE("A 1.x partition that does not mount, or has no file, gives the defaults", "[config][legacy]") -{ - TEST_ASSERT_FALSE(InitFromLegacy(nullptr, 0)); - AssertConfigEqual(Full(), Snapshot()); - - WipeStore(); - HostLfs::mountable = false; - HostLfs::files()[kLegacyPath].assign(Legacy::kDevelopConfig, Legacy::kDevelopConfig + sizeof(Legacy::kDevelopConfig)); - Config::Init(); - AssertConfigEqual(Full(), Snapshot()); - HostLfs::mountable = true; -} - -TEST_CASE("1.5.2 booleans stored as false keep their value", "[config][legacy]") -{ - // kV152ConfigKeepAliveOff has rf.keepalive_enabled, ota.is_enabled and - // ota.allow_backend_management all false. - TEST_ASSERT_TRUE(InitFromLegacy(Legacy::kV152ConfigKeepAliveOff, sizeof(Legacy::kV152ConfigKeepAliveOff))); - - bool allOn; - { - // Scoped: TEST_IGNORE longjmps out, skipping destructors. - const Full loaded = Snapshot(); - allOn = loaded.rf.keepAliveEnabled && loaded.otaUpdate.isEnabled && loaded.otaUpdate.allowBackendManagement; - } - if (allOn) { - // 1.5.2's schema gave these three fields default false, so flatbuffers never - // wrote a false value; the current schema defaults them to true. With all three - // false, 1.5.2 omitted all three, which is byte for byte how 1.6.0-beta/rc and - // develop store all three true (they omit their default), and no other field - // differs between those writers. Without a version marker this file cannot be - // told apart, so it keeps the current default. Files where any of the three is - // stored as true are resolved (see the tests above and below). Pinned so other - // changes still fail. - TEST_IGNORE_MESSAGE("KNOWN ISSUE: 1.5.x with keepalive, OTA and backend management all off reads back all on (ambiguous with 1.6 defaults)"); - } - AssertConfigEqual(ExpectedLegacyConfig(false, false), Snapshot()); -} - -namespace { - namespace Fbs = Serialization::Configuration; - - // An rf + ota_update config where each of the three flipped-default booleans is - // stored with the given value, or omitted (std::nullopt), as a flatbuffer writer - // under either schema would leave it. - std::vector FlippedDefaultsConfig(std::optional keepAlive, std::optional otaEnabled, std::optional backendManagement) - { - flatbuffers::FlatBufferBuilder builder; - - Fbs::RFConfigBuilder rf(builder); - rf.add_tx_pin(15); - if (keepAlive) builder.AddElement(Fbs::RFConfig::VT_KEEPALIVE_ENABLED, *keepAlive); - auto rfOffset = rf.Finish(); - - Fbs::OtaUpdateConfigBuilder ota(builder); - ota.add_check_interval(30); - if (otaEnabled) builder.AddElement(Fbs::OtaUpdateConfig::VT_IS_ENABLED, *otaEnabled); - if (backendManagement) builder.AddElement(Fbs::OtaUpdateConfig::VT_ALLOW_BACKEND_MANAGEMENT, *backendManagement); - auto otaOffset = ota.Finish(); - - Fbs::FinishHubConfigBuffer(builder, Fbs::CreateHubConfig(builder, rfOffset, 0, 0, 0, 0, otaOffset)); - return std::vector(builder.GetBufferPointer(), builder.GetBufferPointer() + builder.GetSize()); - } - - void AssertFlippedDefaults(const std::vector& file, bool keepAlive, bool otaEnabled, bool backendManagement) - { - TEST_ASSERT_TRUE(InitFromLegacy(file)); - const Full loaded = Snapshot(); - TEST_ASSERT_EQUAL_MESSAGE(keepAlive, loaded.rf.keepAliveEnabled, "rf.keepAliveEnabled"); - TEST_ASSERT_EQUAL_MESSAGE(otaEnabled, loaded.otaUpdate.isEnabled, "otaUpdate.isEnabled"); - TEST_ASSERT_EQUAL_MESSAGE(backendManagement, loaded.otaUpdate.allowBackendManagement, "otaUpdate.allowBackendManagement"); - } -} // namespace - -TEST_CASE("Omitted flipped-default booleans follow the convention the file shows", "[config][legacy]") -{ - // 1.5.x convention (default false): only true is stored, so an omitted field is false. - AssertFlippedDefaults(FlippedDefaultsConfig(true, std::nullopt, std::nullopt), true, false, false); - AssertFlippedDefaults(FlippedDefaultsConfig(std::nullopt, true, std::nullopt), false, true, false); - AssertFlippedDefaults(FlippedDefaultsConfig(std::nullopt, std::nullopt, true), false, false, true); - AssertFlippedDefaults(FlippedDefaultsConfig(true, true, std::nullopt), true, true, false); - - // 1.6+ convention (default true): only false is stored, so an omitted field is true. - AssertFlippedDefaults(FlippedDefaultsConfig(false, std::nullopt, std::nullopt), false, true, true); - AssertFlippedDefaults(FlippedDefaultsConfig(std::nullopt, false, false), true, false, false); - - // Explicit values are always taken as stored. - AssertFlippedDefaults(FlippedDefaultsConfig(true, false, true), true, false, true); - AssertFlippedDefaults(FlippedDefaultsConfig(false, true, false), false, true, false); - - // All omitted: ambiguous, keeps the current schema default. - AssertFlippedDefaults(FlippedDefaultsConfig(std::nullopt, std::nullopt, std::nullopt), true, true, true); -} - -TEST_CASE("The imported config survives the first save by the new firmware", "[config][legacy]") -{ - TEST_ASSERT_TRUE(InitFromLegacy(Legacy::kDevelopConfig, sizeof(Legacy::kDevelopConfig))); - - TEST_ASSERT_TRUE(Config::SetOtaUpdateStep(OtaUpdateStep::Validated)); - Full expected = ExpectedLegacyConfig(true, false); - expected.otaUpdate.updateStep = OtaUpdateStep::Validated; - - Reload(); - AssertConfigEqual(expected, Snapshot()); - TEST_ASSERT_EQUAL_INT(0, HostLfs::writeCount); // the 1.x file is never written to, so a rollback still finds it -} - -TEST_CASE("jsonconfig set accepts JSON exported by develop firmware", "[config][legacy][json]") -{ - InitDefault(); - TEST_ASSERT_TRUE(Config::SaveFromJSON(kDevelopJson)); - AssertConfigEqual(ExpectedLegacyConfig(true, false), Snapshot()); - - Reload(); - AssertConfigEqual(ExpectedLegacyConfig(true, false), Snapshot()); -} - -TEST_CASE("jsonconfig set accepts JSON exported by 1.5.2 firmware", "[config][legacy][json]") -{ - InitDefault(); - TEST_ASSERT_TRUE(Config::SaveFromJSON(kV152Json)); - AssertConfigEqual(ExpectedLegacyConfig(false, true), Snapshot()); -} - -TEST_CASE("jsonconfig get emits the same document develop did", "[config][legacy][json]") -{ - TEST_ASSERT_TRUE(InitFromLegacy(Legacy::kDevelopConfig, sizeof(Legacy::kDevelopConfig))); - TEST_ASSERT_EQUAL_STRING(kDevelopJson, Config::GetAsJSON(true).c_str()); -} - -TEST_CASE("Fields from a newer 1.x schema are tolerated on import", "[config][legacy]") -{ - // The schema has grown fields this firmware does not read (wifi AP password / disable flags, LAN config). A - // file carrying them must still import. - namespace Fbs = Serialization::Configuration; - flatbuffers::FlatBufferBuilder builder; - auto creds = builder.CreateVector(std::vector> {Fbs::CreateWiFiCredentialsDirect(builder, 1, "HomeNet", "hunter22")}); - auto wifi = Fbs::CreateWiFiConfig(builder, builder.CreateString("MyHubAP"), builder.CreateString("my-hub"), creds, builder.CreateString("ap-secret"), true, true, true); - auto lan = Fbs::CreateLanConfigDirect(builder, true, "lan-key"); - auto backend = Fbs::CreateBackendConfigDirect(builder, "api.example.org", "tok"); - auto root = Fbs::CreateHubConfig(builder, 0, wifi, 0, backend, 0, 0, 0, lan); - Fbs::FinishHubConfigBuffer(builder, root); - - TEST_ASSERT_TRUE(InitFromLegacy(builder.GetBufferPointer(), builder.GetSize())); - - auto wifiConfig = Config::GetWiFiConfig(); - TEST_ASSERT_EQUAL_STRING("my-hub", wifiConfig.hostname.c_str()); - TEST_ASSERT_EQUAL_size_t(1, wifiConfig.credentialsList.size()); - TEST_ASSERT_EQUAL_STRING("hunter22", wifiConfig.credentialsList[0].password.c_str()); - - TEST_ASSERT_EQUAL_STRING("api.example.org", Config::GetBackendDomain().c_str()); -} diff --git a/components/config/host_test/main/test_main.cpp b/components/config/host_test/main/test_main.cpp index f5e30a433..0b2c3742d 100644 --- a/components/config/host_test/main/test_main.cpp +++ b/components/config/host_test/main/test_main.cpp @@ -1,6 +1,6 @@ // Host-only (linux target) test binary for the config component: the typed config -// sections, their flatbuffer/JSON (de)serialization, and the Config:: store on top -// of an in-memory ConfigFs. Runs on ESP-IDF's FreeRTOS POSIX port, which starts the +// sections, their JSON (de)serialization, and the Config:: store on top +// of ESP-IDF's real NVS. Runs on ESP-IDF's FreeRTOS POSIX port, which starts the // scheduler and calls app_main from its main task, so the store's SimpleMutexes are // the real one. The exit code is the Unity failure count. #include "unity.h" diff --git a/components/config/host_test/main/test_roundtrip.cpp b/components/config/host_test/main/test_roundtrip.cpp index 6bde677ac..c97b2e1a5 100644 --- a/components/config/host_test/main/test_roundtrip.cpp +++ b/components/config/host_test/main/test_roundtrip.cpp @@ -121,7 +121,7 @@ TEST_CASE("One credential that does not read is dropped; the others load and sur const Full sample = MakeSampleConfig(); InitWith(sample); { - auto wifi = Config::Nvs::Handle::Open(Keys::kPartitionCfg, Keys::Ns::kWifi, true); + auto wifi = Config::Nvs::Handle::OpenForWrite(Keys::kPartitionCfg, Keys::Ns::kWifi); TEST_ASSERT_TRUE(wifi.has_value()); const uint8_t garbage[] = {0xFF, 0x02, 0x00}; // unknown blob version, id 2 TEST_ASSERT_TRUE(wifi->SetBlob("2", garbage)); @@ -143,23 +143,6 @@ TEST_CASE("One credential that does not read is dropped; the others load and sur TEST_ASSERT_EQUAL_UINT8(7, creds[2].id); } -TEST_CASE("GetRaw writes fields that equal their schema default", "[config][roundtrip]") -{ - // 1.5.x reads a missing is_enabled/keepalive_enabled as false, so omitting them breaks loading the dump there. - InitDefault(); - TEST_ASSERT_TRUE(Config::SetRFConfigKeepAliveEnabled(true)); - - TinyVec raw; - TEST_ASSERT_TRUE(Config::GetRaw(raw)); - const auto* root = Serialization::Configuration::GetHubConfig(raw.data()); - // The generated tables derive from flatbuffers::Table privately; every table is one at the same address - auto hasField = [](const void* table, flatbuffers::voffset_t field) { - return static_cast(table)->CheckField(field); - }; - TEST_ASSERT_TRUE(hasField(root->rf(), Serialization::Configuration::RFConfig::VT_KEEPALIVE_ENABLED)); - TEST_ASSERT_TRUE(hasField(root->ota_update(), Serialization::Configuration::OtaUpdateConfig::VT_IS_ENABLED)); -} - TEST_CASE("A save writes only what changed", "[config][roundtrip]") { const Full sample = MakeSampleConfig(); @@ -194,11 +177,10 @@ TEST_CASE("Many max-length WiFi credentials survive a reload", "[config][roundtr AssertConfigEqual(before, Snapshot()); } -TEST_CASE("The store holds more than the 1.x config file could", "[config][roundtrip]") +TEST_CASE("The store holds 32 long WiFi credentials", "[config][roundtrip]") { - // The 1.x firmware capped its config file at 4 KiB, some 30 credentials like these. cfg is that file's 12 KiB - // partition retyped as NVS (three pages, one of them always kept free for compaction): 32 credentials with long - // names and passphrases fit with room for the rest of the config. The store fills up at about 36. + // cfg is 12 KiB of NVS (three pages, one of them always kept free for compaction): 32 credentials with long names and + // passphrases fit with room for the rest of the config. The store fills up at about 36. InitDefault(); for (int i = 0; i < 32; ++i) { std::string ssid = "Network-" + std::to_string(i) + std::string(20, 'x'); @@ -240,7 +222,7 @@ TEST_CASE("jsonconfig round trip keeps strings that need escaping", "[config][ro InitDefault(); TEST_ASSERT_TRUE(Config::SaveFromJSON(json)); - // Escaped exactly once on the way out (as the cJSON firmware did), decoded on the way in. + // Escaped exactly once on the way out (not double-escaped), decoded on the way in. TEST_ASSERT_NOT_EQUAL(std::string::npos, json.find(R"("Office \"5G\"")")); TEST_ASSERT_NOT_EQUAL(std::string::npos, json.find(R"("p@ss w0rd/\\")")); @@ -284,20 +266,6 @@ TEST_CASE("GetAsJSON without sensitive data omits secrets", "[config][roundtrip] TEST_ASSERT_NOT_EQUAL(std::string::npos, redacted.find("\"api.example.org\"")); } -TEST_CASE("The wire form for the portal omits secrets and carries every section", "[config][roundtrip]") -{ - const Full sample = MakeSampleConfig(); - InitWith(sample); - - flatbuffers::FlatBufferBuilder builder; - Serialization::Configuration::FinishHubConfigBuffer(builder, Config::GetAsFlatBuffer(builder, false)); - std::vector exported(builder.GetBufferPointer(), builder.GetBufferPointer() + builder.GetSize()); - - Full decoded; - TEST_ASSERT_TRUE(Deserialize(exported.data(), exported.size(), decoded)); - AssertConfigEqual(sample, decoded, false); -} - TEST_CASE("UpdateOtaUpdateConfig edits only what the update touches and saves only on change", "[config][roundtrip]") { const Full sample = MakeSampleConfig(); @@ -344,24 +312,3 @@ TEST_CASE("Factory reset wipes every setting and survives a reload", "[config][r Reload(); AssertConfigEqual(Full(), Snapshot()); } - -TEST_CASE("rawconfig get and set round trip the whole config", "[config][roundtrip]") -{ - InitWith(MakeSampleConfig()); - TinyVec raw; - TEST_ASSERT_TRUE(Config::GetRaw(raw)); - const std::vector exported(raw.data(), raw.data() + raw.size()); - - InitDefault(); - TEST_ASSERT_TRUE(Config::SetRaw(exported.data(), exported.size())); - AssertConfigEqual(MakeSampleConfig(), Snapshot()); - Reload(); - AssertConfigEqual(MakeSampleConfig(), Snapshot()); - - // A buffer that does not verify changes nothing - const std::size_t before = UsedEntries(); - TEST_ASSERT_FALSE(Config::SetRaw(exported.data(), exported.size() / 2)); - TEST_ASSERT_FALSE(Config::SetRaw(nullptr, 0)); - TEST_ASSERT_EQUAL_size_t(before, UsedEntries()); - AssertConfigEqual(MakeSampleConfig(), Snapshot()); -} diff --git a/components/config/host_test/main/test_storage.cpp b/components/config/host_test/main/test_storage.cpp index 6b28f90fa..d53857906 100644 --- a/components/config/host_test/main/test_storage.cpp +++ b/components/config/host_test/main/test_storage.cpp @@ -1,15 +1,10 @@ -// Where the config lives: the `cfg` partition, the shared `nvs` only until the layout migration creates `cfg`, -// and the one-time move between them on the first boot of the new layout. +// Where the config lives: the `cfg` partition. #include #include "unity.h" #include "config_test_helpers.h" -#include -#include - -#include #include using namespace OpenShock; @@ -17,135 +12,10 @@ using namespace ConfigTest; namespace Keys = ConfigTest::Keys; namespace Nvs = OpenShock::Config::Nvs; -namespace { - bool HasSchema(const char* partition) - { - auto root = Nvs::Handle::Open(partition, Keys::Ns::kRoot, false); - if (!root) return false; - uint8_t schema = 0; - return root->GetU8(Keys::Root::kSchema, schema) == Nvs::Result::Found && schema == Keys::kSchemaVersion; - } - - // A hub on the 1.x layout kept its config in the shared nvs; this stands that in on a table that has cfg too. - void StoreInShared(const Full& config) - { - TEST_ASSERT_EQUAL(ESP_OK, nvs_flash_init_partition(Keys::kPartitionShared)); - TEST_ASSERT_TRUE(config.SaveNvs(Keys::kPartitionShared)); - TEST_ASSERT_TRUE(HasSchema(Keys::kPartitionShared)); - } -} // namespace - TEST_CASE("The store is the cfg partition", "[config][storage]") { InitDefault(); TEST_ASSERT_EQUAL_STRING(Keys::kPartitionCfg, Config::StorageLabel()); - TEST_ASSERT_TRUE(HasSchema(Keys::kPartitionCfg)); - TEST_ASSERT_FALSE(HasSchema(Keys::kPartitionShared)); // nothing of ours goes in the shared partition -} - -TEST_CASE("A config kept in the shared nvs moves into cfg on the first boot that has cfg", "[config][storage]") -{ - WipeStore(); - const Full sample = MakeSampleConfig(); - StoreInShared(sample); - - Config::Init(); - TEST_ASSERT_EQUAL_STRING(Keys::kPartitionCfg, Config::StorageLabel()); - AssertConfigEqual(sample, Snapshot()); - TEST_ASSERT_TRUE(HasSchema(Keys::kPartitionCfg)); - - // The shared copy is gone: every key of every namespace of ours (NVS keeps an emptied namespace's name) - TEST_ASSERT_FALSE(HasSchema(Keys::kPartitionShared)); - for (const char* ns : Keys::Ns::kAll) { - TEST_ASSERT_EQUAL_size_t_MESSAGE(0, KeyCount(Keys::kPartitionShared, ns), ns); - } - - // And a later boot is a plain load from cfg - Reload(); - AssertConfigEqual(sample, Snapshot()); -} - -namespace { - // cfg as the table rewrite leaves it: the 1.x littlefs `config` partition, retyped. A littlefs metadata block - // (revision count, superblock tag, "littlefs", then more of the block) in `block`, the rest left as littlefs left it. - void FillCfgWithLittlefs(uint32_t block) - { - TEST_ASSERT_EQUAL(ESP_OK, nvs_flash_deinit_partition(Keys::kPartitionCfg)); - const esp_partition_t* cfg = esp_partition_find_first(ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_NVS, Keys::kPartitionCfg); - TEST_ASSERT_NOT_NULL(cfg); - TEST_ASSERT_EQUAL(ESP_OK, esp_partition_erase_range(cfg, 0, cfg->size)); - std::vector metadata(0x1000); - for (std::size_t i = 0; i < metadata.size(); ++i) metadata[i] = static_cast(i * 7 + 3); - const uint8_t head[] = {0x01, 0x00, 0x00, 0x00, 0x0F, 0xF0, 0x00, 0x08, 'l', 'i', 't', 't', 'l', 'e', 'f', 's'}; - memcpy(metadata.data(), head, sizeof(head)); - TEST_ASSERT_EQUAL(ESP_OK, esp_partition_write(cfg, block * 0x1000, metadata.data(), metadata.size())); - } -} // namespace - -TEST_CASE("The 1.x littlefs left in cfg by the table rewrite is erased before NVS opens it", "[config][storage]") -{ - for (uint32_t block = 0; block < 2; ++block) { - WipeStore(); - const Full sample = MakeSampleConfig(); - StoreInShared(sample); - FillCfgWithLittlefs(block); - - Config::Init(); - TEST_ASSERT_EQUAL_STRING(Keys::kPartitionCfg, Config::StorageLabel()); - AssertConfigEqual(sample, Snapshot()); - TEST_ASSERT_TRUE(HasSchema(Keys::kPartitionCfg)); - // Nothing of the littlefs is left - const esp_partition_t* cfg = esp_partition_find_first(ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_NVS, Keys::kPartitionCfg); - for (uint32_t b = 0; b < cfg->size / 0x1000; ++b) { - char magic[8]; - TEST_ASSERT_EQUAL(ESP_OK, esp_partition_read(cfg, b * 0x1000 + 8, magic, sizeof(magic))); - TEST_ASSERT_NOT_EQUAL(0, memcmp(magic, "littlefs", sizeof(magic))); - } - - Reload(); - AssertConfigEqual(sample, Snapshot()); - } -} - -TEST_CASE("A config in cfg wins over a stale copy in the shared nvs, which is cleaned up", "[config][storage]") -{ - InitWith(MakeSampleConfig()); - Full stale = MakeSampleConfig(); - stale.wifi.hostname = "stale-hub"; - StoreInShared(stale); - - Reload(); - AssertConfigEqual(MakeSampleConfig(), Snapshot()); - TEST_ASSERT_FALSE(HasSchema(Keys::kPartitionShared)); -} - -TEST_CASE("A cleanup of the shared copy cut short is finished on the next boot", "[config][storage]") -{ - // The shared copy is erased section by section with `root` last: a power cut mid-way leaves the schema marker, - // which is what makes the next boot look at the shared partition again. Here the cut came after two sections. - InitWith(MakeSampleConfig()); - StoreInShared(MakeSampleConfig()); - TEST_ASSERT_TRUE(Nvs::EraseNamespace(Keys::kPartitionShared, Keys::Ns::kOta)); - TEST_ASSERT_TRUE(Nvs::EraseNamespace(Keys::kPartitionShared, Keys::Ns::kSerial)); - TEST_ASSERT_TRUE(HasSchema(Keys::kPartitionShared)); - - Reload(); - AssertConfigEqual(MakeSampleConfig(), Snapshot()); - TEST_ASSERT_FALSE(HasSchema(Keys::kPartitionShared)); - for (const char* ns : Keys::Ns::kAll) { - TEST_ASSERT_EQUAL_size_t_MESSAGE(0, KeyCount(Keys::kPartitionShared, ns), ns); - } -} - -TEST_CASE("The move is idempotent and verified", "[config][storage]") -{ - WipeStore(); - StoreInShared(MakeSampleConfig()); - Config::Init(); - Reload(); - Reload(); - AssertConfigEqual(MakeSampleConfig(), Snapshot()); - TEST_ASSERT_TRUE(HasSchema(Keys::kPartitionCfg)); } TEST_CASE("Sections link themselves into the registry in document order", "[config][storage]") @@ -164,7 +34,7 @@ TEST_CASE("A key stored with another type reads as missing, so as its default", InitWith(MakeSampleConfig()); { // NVS looks keys up by name and type: a string where the rf section reads an i8 is not its pin - auto rf = Nvs::Handle::Open(Keys::kPartitionCfg, Keys::Ns::kRf, true); + auto rf = Nvs::Handle::OpenForWrite(Keys::kPartitionCfg, Keys::Ns::kRf); TEST_ASSERT_TRUE(rf.has_value()); TEST_ASSERT_TRUE(rf->Erase(Keys::Rf::kTxPin)); TEST_ASSERT_TRUE(rf->SetStr(Keys::Rf::kTxPin, "not a number")); diff --git a/components/config/host_test/partitions_test.csv b/components/config/host_test/partitions_test.csv index 7ea4c63b6..2c6100719 100644 --- a/components/config/host_test/partitions_test.csv +++ b/components/config/host_test/partitions_test.csv @@ -1,10 +1,6 @@ # The partition table the config host test runs on (ESP-IDF's linux esp_partition emulates the flash in a file). -# Both a 2.x `cfg` NVS partition and the 1.x littlefs `config` partition, so Config::Init can be tested picking the -# store (cfg), moving a config out of the shared `nvs`, and importing the 1.x file (through the in-memory -# LfsPartition stub; the `config` partition only has to exist for the import to be attempted). # Name, Type, SubType, Offset, Size, Flags nvs, data, nvs, 0x009000, 0x005000, otadata, data, ota, 0x00E000, 0x002000, app0, app, ota_0, 0x010000, 0x100000, cfg, data, nvs, 0x110000, 0x003000, -config, data, spiffs, 0x113000, 0x003000, diff --git a/components/config/host_test/sdkconfig.defaults b/components/config/host_test/sdkconfig.defaults index 2d2c265b6..8a3db0d51 100644 --- a/components/config/host_test/sdkconfig.defaults +++ b/components/config/host_test/sdkconfig.defaults @@ -6,9 +6,8 @@ CONFIG_UNITY_ENABLE_IDF_TEST_RUNNER=y # Same tick rate as the firmware (sdkconfig.defaults at the repo root). CONFIG_FREERTOS_HZ=1000 -# A table with both the 2.x `cfg` store and the 1.x `config` partition (partitions_test.csv), so Config::Init's -# esp_partition_find_first() finds what each test needs. ESP-IDF's linux esp_partition emulates the flash in a -# file, and its nvs_flash runs on that, so the store under test is the real NVS. +# A table with the `cfg` store (partitions_test.csv). ESP-IDF's linux esp_partition emulates the flash in a file, and +# its nvs_flash runs on that, so the store under test is the real NVS. CONFIG_PARTITION_TABLE_CUSTOM=y CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions_test.csv" diff --git a/components/config/include/config/Migration.h b/components/config/include/config/Migration.h deleted file mode 100644 index dbf39e83c..000000000 --- a/components/config/include/config/Migration.h +++ /dev/null @@ -1,23 +0,0 @@ -#pragma once - -// The config half of the 1.x -> 2.x storage migration (docs/STORAGE.md): retype the 1.x littlefs `config` partition -// as the `cfg` NVS store, in place. Run by the storage migration task (migration/StorageMigration.h). -// -// What happens around it is Store's: before it, a 1.x hub keeps its config (imported from the 1.x file) in the shared -// `nvs`; on the first boot after it, Store erases the littlefs left in `cfg` and moves the config over. - -namespace OpenShock::Config { - enum class MigrationResult { - NotNeeded, // the table already has `cfg` - Refused, // no 1.x `config` partition, the table could not be read, or the config is not in the shared nvs - // (retyping would lose it) - }; - - /// @brief Retypes the 1.x `config` partition as `cfg` and restarts, whether the table write went through or not - /// (PartitionTable::Replace). Returns only when it touched nothing. - /// - /// Requires everything that uses NVS or the partition API outside the config store to be stopped: it closes NVS - /// for the table write. A power cut between the erase and the write of the table sector bricks the hub until the - /// table is reflashed over USB (docs/STORAGE.md, Recovery). - MigrationResult MigrateToCfgPartition(); -} // namespace OpenShock::Config diff --git a/components/config/include/config/Nvs.h b/components/config/include/config/Nvs.h index bec8e5e0c..76889d129 100644 --- a/components/config/include/config/Nvs.h +++ b/components/config/include/config/Nvs.h @@ -37,9 +37,8 @@ namespace OpenShock::Config::Nvs { class Handle { public: - /// @brief Opens `ns` in `partition`. A read-only open of a namespace that does not exist yet fails with nullopt - /// and no error logged; a load cannot tell that from a failed open, so a load uses OpenForLoad instead. - [[nodiscard]] static std::optional Open(const char* partition, const char* ns, bool write); + /// @brief Opens `ns` in `partition` read-write, creating it if it does not exist yet. Nullopt when the open failed. + [[nodiscard]] static std::optional OpenForWrite(const char* partition, const char* ns); /// @brief Opens `ns` in `partition` read-only for a load. A namespace that was never written is an empty handle /// (Exists() false) on which every read is Missing: every key is its default. Nullopt only when the open @@ -54,8 +53,6 @@ namespace OpenShock::Config::Nvs { /// @brief False for the empty handle OpenForLoad gives a namespace that was never written. [[nodiscard]] bool Exists() const { return m_handle != 0; } - /// @brief The raw handle, for the untyped key copy in Nvs.cpp; sections never need it. - [[nodiscard]] nvs_handle_t get() const { return m_handle; } [[nodiscard]] Result GetU8(const char* key, uint8_t& out) const; [[nodiscard]] Result GetI8(const char* key, int8_t& out) const; @@ -175,10 +172,4 @@ namespace OpenShock::Config::Nvs { /// @brief Erases every key of `ns` in `partition`; a namespace that does not exist counts as erased. [[nodiscard]] bool EraseNamespace(const char* partition, const char* ns); - - /// @brief Copies every key of `ns` from partition `from` into partition `to`, whatever its type, and commits. - [[nodiscard]] bool CopyNamespace(const char* from, const char* to, const char* ns); - - /// @brief Whether `ns` holds the same keys with the same types and values in both partitions. - [[nodiscard]] bool NamespacesEqual(const char* a, const char* b, const char* ns); } // namespace OpenShock::Config::Nvs diff --git a/components/config/include/config/Section.h b/components/config/include/config/Section.h index 511589366..cd2ff4b99 100644 --- a/components/config/include/config/Section.h +++ b/components/config/include/config/Section.h @@ -12,10 +12,9 @@ // bool SaveNvs(const char* partition, const RFConfig* previous) const; // writes only what differs // bool FromJSON(JSON::JsonView json); // invalid view: defaults // void ToJSON(json_gen_str_t* gen, const char* name, bool withSensitiveData) const; -// flatbuffers::Offset<...> ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const; // }; // static constexpr const char* kNamespaces[] = {"rf"}; -// static Config::Section s_rf({.name = "rf", .order = 0, .hubConfigField = VT_RF, .namespaces = kNamespaces}); +// static Config::Section s_rf({.name = "rf", .order = 0, .namespaces = kNamespaces}); // // and wraps it in whatever accessors it wants (Config::GetRFConfigTxPin() and so on). Nothing outside the module // needs to know T's fields or keys; this framework only sees the section through SectionBase. @@ -45,18 +44,12 @@ namespace OpenShock::JSON { class JsonView; } -namespace flatbuffers { - template - class FlatBufferBuilderImpl; - using FlatBufferBuilder = FlatBufferBuilderImpl; -} // namespace flatbuffers namespace OpenShock::Config { struct SectionInfo { const char* name; // member name of the section in the jsonconfig document, e.g. "rf" - uint8_t order; // position in that document and in HubConfig; unique across sections - uint16_t hubConfigField; // the HubConfig field (VT_*) its wire form goes into for the local protocol; 0 = none - std::span namespaces; // the NVS namespaces it owns (erased by a reset, moved with the store) + uint8_t order; // position in that document; unique across sections + std::span namespaces; // the NVS namespaces it owns (erased by a reset) }; class SectionBase { @@ -71,14 +64,12 @@ namespace OpenShock::Config { [[nodiscard]] SectionBase* Next() const { return m_next; } // The interface the store (Store.cpp) drives. All of these run under the write lock. - virtual bool Load(const char* partition) = 0; // from NVS into the cache; false on a read error - virtual bool Reset(const char* partition) = 0; // erase its namespaces, save and publish the defaults - virtual bool VerifyStored(const char* partition) const = 0; // whether NVS reads back as the cache - virtual void ToJSON(json_gen_str_t* gen, bool withSensitiveData) const = 0; // as the member `name` - virtual bool Stage(const JSON::JsonView& json) = 0; // parse into a staging copy - virtual bool CommitStaged(const char* partition, bool full) = 0; // save it (every key when full) and publish - virtual void DiscardStaged() = 0; - virtual uint32_t ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const = 0; // the sub-table offset + virtual bool Load() = 0; // from NVS into the cache; false on a read error + virtual bool Reset() = 0; // erase its namespaces and publish the defaults + virtual void ToJSON(json_gen_str_t* gen, bool withSensitiveData) const = 0; // as the member `name` + virtual bool Stage(const JSON::JsonView& json) = 0; // parse into a staging copy + virtual bool CommitStaged() = 0; // save it and publish + virtual void DiscardStaged() = 0; protected: explicit SectionBase(const SectionInfo& info); @@ -93,21 +84,18 @@ namespace OpenShock::Config { /// constructed value is the defaults. Normalize() is the type's validation: the section runs it on every value /// it takes in (load, update, import, reset), so no setter can skip it. template - concept SectionValue = std::default_initializable && std::copyable && requires(T& value, const T& constValue, const char* partition, json_gen_str_t* gen, flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) { + concept SectionValue = std::default_initializable && std::copyable && requires(T& value, const T& constValue, const char* partition, json_gen_str_t* gen, bool withSensitiveData) { { value.Normalize() } -> std::same_as; { value.LoadNvs(partition) } -> std::same_as; { constValue.SaveNvs(partition, &constValue) } -> std::same_as; { &T::FromJSON } -> std::same_as; // JsonView may be incomplete here { constValue.ToJSON(gen, "name", withSensitiveData) } -> std::same_as; - { constValue.ToFlatbuffers(builder, withSensitiveData).o } -> std::convertible_to; }; namespace Internal { /// @brief Serializes every writer of every section, the store's own operations included. SimpleMutex& WriteLock(); bool EraseNamespaces(const char* partition, std::span namespaces); - /// @brief Whether two values render the same JSON, secrets included: the complete, canonical comparison. - bool SameJson(Util::FunctionRef renderA, Util::FunctionRef renderB); } // namespace Internal template @@ -164,36 +152,31 @@ namespace OpenShock::Config { return true; } - bool Load(const char* partition) override + bool Load() override { + const char* partition = StorageLabel(); T loaded; - bool ok = loadNvs(loaded, partition); + bool ok = partition == nullptr || loaded.LoadNvs(partition); if (!ok) { - loaded = defaults(); + loaded = T(); } + loaded.Normalize(); m_unsynced = !ok; publish(std::move(loaded)); return ok; } - bool Reset(const char* partition) override + bool Reset() override { - T value = defaults(); - bool ok = true; - if (partition != nullptr) { - ok = Internal::EraseNamespaces(partition, Info().namespaces) && value.SaveNvs(partition, nullptr); - } + const char* partition = StorageLabel(); + T value; + value.Normalize(); + bool ok = partition == nullptr || Internal::EraseNamespaces(partition, Info().namespaces); m_unsynced = !ok; - publish(std::move(value)); // What the hub runs with either way; a failed save only means it isn't kept + publish(std::move(value)); // What the hub runs with either way; a failed erase only means flash still differs return ok; } - bool VerifyStored(const char* partition) const override - { - T stored; - return loadNvs(stored, partition) && sameValue(stored, Get()); - } - void ToJSON(json_gen_str_t* gen, bool withSensitiveData) const override { ScopedLock lock__(&m_dataMutex); @@ -211,13 +194,12 @@ namespace OpenShock::Config { return true; } - bool CommitStaged(const char* partition, bool full) override + bool CommitStaged() override { if (!m_staged) { return true; } - bool ok = partition == nullptr || m_staged->SaveNvs(partition, full || m_unsynced ? nullptr : &m_value); - m_unsynced = !ok; + bool ok = save(*m_staged); if (ok) { publish(std::move(*m_staged)); } @@ -227,30 +209,7 @@ namespace OpenShock::Config { void DiscardStaged() override { m_staged.reset(); } - uint32_t ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const override - { - ScopedLock lock__(&m_dataMutex); - return m_value.ToFlatbuffers(builder, withSensitiveData).o; - } - private: - /// @brief The defaults as the section runs them: default constructed, then normalized like every other value. - static T defaults() - { - T value; - value.Normalize(); - return value; - } - - static bool loadNvs(T& value, const char* partition) - { - if (!value.LoadNvs(partition)) { - return false; - } - value.Normalize(); - return true; - } - // Only the keys that differ from the cache, or every key when flash may not match it bool save(const T& value) { @@ -268,11 +227,6 @@ namespace OpenShock::Config { m_value = std::move(value); } - static bool sameValue(const T& a, const T& b) - { - return Internal::SameJson([&](json_gen_str_t* gen) { a.ToJSON(gen, nullptr, true); }, [&](json_gen_str_t* gen) { b.ToJSON(gen, nullptr, true); }); - } - mutable SimpleMutex m_dataMutex; T m_value; std::optional m_staged; // Only touched under the write lock diff --git a/components/config/include/config/Store.h b/components/config/include/config/Store.h index 72dff90c5..ee124eb16 100644 --- a/components/config/include/config/Store.h +++ b/components/config/include/config/Store.h @@ -2,27 +2,22 @@ // The config store: where every module's section (config/Section.h) lives, and the operations over all of them. // -// The store is the NVS partition labelled `cfg` (2.x partition layouts). On the 1.x layout, which has no `cfg`, -// the same namespaces live in the shared `nvs` partition until the layout migration creates `cfg`; the first boot -// that finds both moves them over, verifies the copy and deletes the shared one. A hub without any usable NVS runs -// on defaults in RAM and says so. -// -// The `root` namespace is the store's own: `schema` (u8) marks a store that has been filled. An unmarked store is -// filled once, from the 1.x littlefs config file when the hub has one, else from every section's defaults. +// The store is the NVS partition labelled `cfg`. A hub whose `cfg` cannot be used runs on defaults in RAM and says so. +// An empty store reads as the defaults, so nothing is written until a value changes. #include #include namespace OpenShock::Config { - /// @brief Picks and opens the store, moves or imports what needs it, then loads every section. Call once, after - /// Board::Init (section defaults read the board) and before anything reads a section. + /// @brief Opens the store and loads every section. Call once, after Board::Init (section + /// defaults read the board) and before anything reads a section. void Init(); - /// @brief The partition the config is stored in ("cfg", "nvs" on the 1.x layout), or null when it only lives in RAM. + /// @brief The partition the config is stored in ("cfg"), or null when it only lives in RAM. [[nodiscard]] const char* StorageLabel(); /** - * @brief Resets every section to its defaults: erases its namespaces and writes the defaults. + * @brief Resets every section to its defaults by erasing its namespaces. * * @note A restart after calling this function is HIGHLY recommended. */ diff --git a/components/config/include/config/Wire.h b/components/config/include/config/Wire.h deleted file mode 100644 index 0a085c164..000000000 --- a/components/config/include/config/Wire.h +++ /dev/null @@ -1,34 +0,0 @@ -#pragma once - -// The whole config in the FlatBuffers HubConfig form. It is never stored: it is the shape the local (captive -// portal) protocol's Ready message carries, and the binary form of the rawconfig serial command. Assembled from -// every section's own ToFlatbuffers; parsed back through the JSON importer (HubConfig -> jsonconfig document). - -#include "serialization/_fbs/HubConfig_generated.h" -#include "TinyVec.h" - -#include -#include -#include -#include -#include - -namespace OpenShock::Config { - /// @brief Builds a HubConfig table from every section that has a HubConfig field. - [[nodiscard]] flatbuffers::Offset GetAsFlatBuffer(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData); - - /// @brief The finished HubConfig buffer, secrets included (rawconfig get). - bool GetRaw(TinyVec& buffer); - - /// @brief Applies a HubConfig buffer as SaveFromJSON applies a document (rawconfig set). False when it does not - /// verify or a section refuses its part. - bool SetRaw(const uint8_t* buffer, std::size_t size); - - /// @brief The jsonconfig document a HubConfig buffer stands for: what SetRaw applies and how the 1.x config file - /// (the same HubConfig, written by the Arduino firmware) is imported. Nullopt when the buffer does not verify. - [[nodiscard]] std::optional HubConfigToJSON(std::span buffer); - - /// @brief The largest HubConfig accepted: what the 1.x config partition could hold, which is also far more than - /// any real config renders to. - inline constexpr std::size_t kMaxHubConfigSize = 4096; -} // namespace OpenShock::Config diff --git a/components/config/src/Migration.cpp b/components/config/src/Migration.cpp deleted file mode 100644 index 6a3dfd820..000000000 --- a/components/config/src/Migration.cpp +++ /dev/null @@ -1,94 +0,0 @@ -#include - -#include "config/Migration.h" - -const char* const TAG = "ConfigMigration"; - -#include "config/Nvs.h" -#include "config/Section.h" -#include "config/Store.h" -#include "Logging.h" -#include "partition_layout/LiveTable.h" -#include "partition_layout/PartitionTable.h" -#include "SimpleMutex.h" - -#include -#include -#include - -#include -#include - -using namespace OpenShock; -namespace PT = OpenShock::PartitionTable; - -namespace { - // The WiFi driver's own store from the Arduino builds: one STA profile, with the password in plain text. Nothing - // reads it any more (CONFIG_ESP_WIFI_NVS_ENABLED=n). - constexpr const char* kWifiDriverNamespace = "nvs.net80211"; - - void dropWifiDriverStore() - { - if (Config::Nvs::EraseNamespace("nvs", kWifiDriverNamespace)) { - OS_LOGI(TAG, "Dropped the WiFi driver's stale NVS store (%s), if it had one", kWifiDriverNamespace); - } - } -} // namespace - -Config::MigrationResult Config::MigrateToCfgPartition() -{ - // A raw table and a parsed one come to some 6.5 KiB: on the heap, not the caller's stack - struct Tables { - PT::Bytes live; - PT::Bytes next; - PT::Table parsed; - }; - auto tables = std::make_unique(); - - if (!PT::ReadLiveBytes(tables->live) || !PT::Parse(tables->live, tables->parsed)) { - OS_LOGE(TAG, "Cannot read the partition table; not migrating"); - return MigrationResult::Refused; - } - if (tables->parsed.Find("cfg") != nullptr) { - return MigrationResult::NotNeeded; - } - const PT::Entry* config = tables->parsed.Find("config"); - if (config == nullptr || config->type != PT::kTypeData || config->subtype != PT::kSubtypeDataSpiffs) { - OS_LOGE(TAG, "No 1.x littlefs config partition; not migrating"); - return MigrationResult::Refused; - } - // The retype hands the partition to NVS, and the next boot erases what littlefs left in it - const char* store = StorageLabel(); - if (store == nullptr || strcmp(store, "nvs") != 0) { - OS_LOGE(TAG, "The config is not in the shared nvs partition; retyping config would lose it"); - return MigrationResult::Refused; - } - - tables->next = tables->live; - if (auto retyped = PT::Retype(tables->next, "config", PT::kTypeData, PT::kSubtypeDataNvs, "cfg"); !retyped) { - OS_LOGE(TAG, "Cannot retype the config partition: %s", PT::RetypeErrorName(retyped.error())); - return MigrationResult::Refused; - } - - // Quiesce: no config write can start and NVS is closed - OS_LOGW(TAG, "Retyping the config partition as cfg. Do not unplug the hub."); - ScopedLock configLock(&Internal::WriteLock()); - dropWifiDriverStore(); - nvs_flash_deinit(); - vTaskDelay(pdMS_TO_TICKS(100)); - - switch (PT::Replace(tables->next, tables->live)) { - case PT::ReplaceResult::Written: - OS_LOGI(TAG, "Partition table rewritten; restarting into the cfg config store"); - break; - case PT::ReplaceResult::Restored: - OS_LOGE(TAG, "The partition table is the old one again; restarting on it"); - break; - case PT::ReplaceResult::Untouched: - OS_LOGE(TAG, "The partition table was not touched; restarting on it"); - break; - } - // NVS is closed: a restart is the one clean way back to a running hub - esp_partition_unload_all(); - esp_restart(); -} diff --git a/components/config/src/Nvs.cpp b/components/config/src/Nvs.cpp index a3cef16ad..9ea282826 100644 --- a/components/config/src/Nvs.cpp +++ b/components/config/src/Nvs.cpp @@ -6,7 +6,6 @@ const char* const TAG = "Config::Nvs"; #include -#include #include #include #include @@ -35,16 +34,13 @@ bool Nvs::InitPartition(const char* label, bool eraseNewer) return true; } -std::optional Nvs::Handle::Open(const char* partition, const char* ns, bool write) +std::optional Nvs::Handle::OpenForWrite(const char* partition, const char* ns) { nvs_handle_t handle = 0; - esp_err_t err = nvs_open_from_partition(partition, ns, write ? NVS_READWRITE : NVS_READONLY, &handle); + esp_err_t err = nvs_open_from_partition(partition, ns, NVS_READWRITE, &handle); if (err == ESP_OK) { return Handle(handle); } - if (err == ESP_ERR_NVS_NOT_FOUND && !write) { - return std::nullopt; // The namespace was never written: every key is a default - } OS_LOGE(TAG, "Failed to open %s/%s: %s", partition, ns, esp_err_to_name(err)); return std::nullopt; } @@ -296,181 +292,6 @@ bool Nvs::EraseNamespace(const char* partition, const char* ns) return true; } existing.reset(); - auto handle = Handle::Open(partition, ns, true); + auto handle = Handle::OpenForWrite(partition, ns); return handle && handle->EraseAll() && handle->Commit(); } - -namespace { - // One entry of any type, read into a byte image so two can be compared and one can be written back. - struct RawEntry { - nvs_type_t type; - std::vector data; // integers little-endian, strings with their NUL, blobs as they are - }; - - bool readRaw(nvs_handle_t handle, const char* key, nvs_type_t type, RawEntry& out) - { - out.type = type; - esp_err_t err = ESP_OK; - switch (type) { - case NVS_TYPE_U8: - case NVS_TYPE_I8: - out.data.resize(1); - err = type == NVS_TYPE_U8 ? nvs_get_u8(handle, key, out.data.data()) : nvs_get_i8(handle, key, reinterpret_cast(out.data.data())); - break; - case NVS_TYPE_U16: - case NVS_TYPE_I16: - out.data.resize(2); - err = type == NVS_TYPE_U16 ? nvs_get_u16(handle, key, reinterpret_cast(out.data.data())) : nvs_get_i16(handle, key, reinterpret_cast(out.data.data())); - break; - case NVS_TYPE_U32: - case NVS_TYPE_I32: - out.data.resize(4); - err = type == NVS_TYPE_U32 ? nvs_get_u32(handle, key, reinterpret_cast(out.data.data())) : nvs_get_i32(handle, key, reinterpret_cast(out.data.data())); - break; - case NVS_TYPE_U64: - case NVS_TYPE_I64: - out.data.resize(8); - err = type == NVS_TYPE_U64 ? nvs_get_u64(handle, key, reinterpret_cast(out.data.data())) : nvs_get_i64(handle, key, reinterpret_cast(out.data.data())); - break; - case NVS_TYPE_STR: - { - size_t length = 0; - err = nvs_get_str(handle, key, nullptr, &length); - if (err == ESP_OK) { - out.data.resize(length); - err = nvs_get_str(handle, key, reinterpret_cast(out.data.data()), &length); - } - break; - } - case NVS_TYPE_BLOB: - { - size_t length = 0; - err = nvs_get_blob(handle, key, nullptr, &length); - if (err == ESP_OK) { - out.data.resize(length); - err = length == 0 ? ESP_OK : nvs_get_blob(handle, key, out.data.data(), &length); - } - break; - } - default: - OS_LOGE(TAG, "Key %s has an unknown type 0x%02x", key, type); - return false; - } - if (err != ESP_OK) { - OS_LOGE(TAG, "Failed to read %s: %s", key, esp_err_to_name(err)); - return false; - } - return true; - } - - bool writeRaw(nvs_handle_t handle, const char* key, const RawEntry& entry) - { - const void* p = entry.data.data(); - esp_err_t err = ESP_OK; - switch (entry.type) { - case NVS_TYPE_U8: - err = nvs_set_u8(handle, key, *static_cast(p)); - break; - case NVS_TYPE_I8: - err = nvs_set_i8(handle, key, *static_cast(p)); - break; - case NVS_TYPE_U16: - err = nvs_set_u16(handle, key, *static_cast(p)); - break; - case NVS_TYPE_I16: - err = nvs_set_i16(handle, key, *static_cast(p)); - break; - case NVS_TYPE_U32: - err = nvs_set_u32(handle, key, *static_cast(p)); - break; - case NVS_TYPE_I32: - err = nvs_set_i32(handle, key, *static_cast(p)); - break; - case NVS_TYPE_U64: - err = nvs_set_u64(handle, key, *static_cast(p)); - break; - case NVS_TYPE_I64: - err = nvs_set_i64(handle, key, *static_cast(p)); - break; - case NVS_TYPE_STR: - err = nvs_set_str(handle, key, static_cast(p)); - break; - case NVS_TYPE_BLOB: - err = nvs_set_blob(handle, key, p, entry.data.size()); - break; - default: - return false; - } - return checkWrite(err, key); - } - - // Every key of `ns` in `partition` with its type. False when the namespace cannot be enumerated. - bool listKeys(const char* partition, const char* ns, std::vector>& out) - { - out.clear(); - nvs_iterator_t it = nullptr; - esp_err_t err = nvs_entry_find(partition, ns, NVS_TYPE_ANY, &it); - while (err == ESP_OK) { - nvs_entry_info_t info; - nvs_entry_info(it, &info); - out.emplace_back(info.key, info.type); - err = nvs_entry_next(&it); - } - nvs_release_iterator(it); - return err == ESP_ERR_NVS_NOT_FOUND; - } -} // namespace - -bool Nvs::CopyNamespace(const char* from, const char* to, const char* ns) -{ - std::vector> keys; - if (!listKeys(from, ns, keys)) { - OS_LOGE(TAG, "Failed to enumerate %s/%s", from, ns); - return false; - } - if (keys.empty()) { - return true; - } - - auto src = Handle::Open(from, ns, false); - auto dst = Handle::Open(to, ns, true); - if (!src || !dst) { - return false; - } - for (const auto& [key, type] : keys) { - RawEntry entry; - if (!readRaw(src->get(), key.c_str(), type, entry) || !writeRaw(dst->get(), key.c_str(), entry)) { - return false; - } - } - return dst->Commit(); -} - -bool Nvs::NamespacesEqual(const char* a, const char* b, const char* ns) -{ - std::vector> keysA, keysB; - if (!listKeys(a, ns, keysA) || !listKeys(b, ns, keysB)) { - return false; - } - std::sort(keysA.begin(), keysA.end()); - std::sort(keysB.begin(), keysB.end()); - if (keysA != keysB) { - return false; - } - if (keysA.empty()) { - return true; - } - - auto ha = Handle::Open(a, ns, false); - auto hb = Handle::Open(b, ns, false); - if (!ha || !hb) { - return false; - } - for (const auto& [key, type] : keysA) { - RawEntry ea, eb; - if (!readRaw(ha->get(), key.c_str(), type, ea) || !readRaw(hb->get(), key.c_str(), type, eb) || ea.data != eb.data) { - return false; - } - } - return true; -} diff --git a/components/config/src/Section.cpp b/components/config/src/Section.cpp index 2fe0b8a89..dcdad9ee8 100644 --- a/components/config/src/Section.cpp +++ b/components/config/src/Section.cpp @@ -4,8 +4,6 @@ #include "config/Nvs.h" -#include "json/Json.h" - using namespace OpenShock; // Constant-initialized, so it is valid before any section's constructor runs, whatever the static init order. @@ -43,15 +41,3 @@ bool Config::Internal::EraseNamespaces(const char* partition, std::span renderA, Util::FunctionRef renderB) -{ - JSON::StringWriter a, b; - json_gen_start_object(a.gen()); - renderA(a.gen()); - json_gen_end_object(a.gen()); - json_gen_start_object(b.gen()); - renderB(b.gen()); - json_gen_end_object(b.gen()); - return a.finish() == b.finish(); -} diff --git a/components/config/src/Store.cpp b/components/config/src/Store.cpp index 0d4cd74b2..29c6f5aa0 100644 --- a/components/config/src/Store.cpp +++ b/components/config/src/Store.cpp @@ -6,32 +6,16 @@ const char* const TAG = "Config"; #include "config/Nvs.h" #include "config/Section.h" -#include "config/Wire.h" #include "Logging.h" #include "SimpleMutex.h" -#include "fs/LfsPartition.h" #include "json/Json.h" -#include - -#include -#include #include -#include using namespace OpenShock; -static constexpr const char* kPartitionCfg = "cfg"; -static constexpr const char* kPartitionShared = "nvs"; -static constexpr const char* kRootNamespace = "root"; -static constexpr const char* kSchemaKey = "schema"; -static constexpr uint8_t kSchemaVersion = 1; - -// The 1.x config file: the HubConfig the Arduino firmware stored as "config" at the root of the littlefs partition -// labelled "config" (partitions/legacy/ota_4mb_v1.csv). Read once, never written, so a rollback to 1.x finds it. -static constexpr const char* kLegacyPartitionLabel = "config"; -static constexpr const char* kLegacyFilePath = "/config"; +static constexpr const char* kPartitionCfg = "cfg"; // Written by Init() alone, under the write lock, before anything else reads a section. static const char* s_partition = nullptr; @@ -43,174 +27,8 @@ const char* Config::StorageLabel() #define CONFIG_LOCK_WRITE() ScopedLock lock__(&Config::Internal::WriteLock()) -// Whether `partition` holds a filled store: true, false, or nullopt when NVS would not say (a failed open or read). -// A store marked by a newer firmware (a rollback) counts as filled: it is loaded best-effort, never refilled. -static std::optional hasSchema(const char* partition) -{ - auto handle = Config::Nvs::Handle::OpenForLoad(partition, kRootNamespace); - uint8_t schema = 0; - if (!handle || !Config::Nvs::Load(*handle, kSchemaKey, schema, 0)) { - return std::nullopt; - } - if (schema < kSchemaVersion) { - return false; - } - if (schema > kSchemaVersion) { - OS_LOGW(TAG, "The config in %s has schema %u, newer than this firmware's %u; loading what this firmware knows", partition, schema, kSchemaVersion); - } - return true; -} - -static bool writeSchema(const char* partition) -{ - auto handle = Config::Nvs::Handle::Open(partition, kRootNamespace, true); - return handle && handle->SetU8(kSchemaKey, kSchemaVersion) && handle->Commit(); -} - -// Every namespace the config owns: the store's own and each section's. -static std::vector allNamespaces() -{ - std::vector namespaces {kRootNamespace}; - for (auto* section = Config::SectionBase::First(); section != nullptr; section = section->Next()) { - for (const char* ns : section->Info().namespaces) { - namespaces.push_back(ns); - } - } - return namespaces; -} - -static bool nvsPartitionExists(const char* label) -{ - return esp_partition_find_first(ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_NVS, label) != nullptr; -} - -// The 2.x cfg is the 1.x littlefs `config` partition retyped in place, so on the first boot after the table rewrite it -// still holds that littlefs. NVS must never try to make sense of it: it is erased. littlefs names itself in the -// superblock entry at the start of either of its first two blocks (revision count, tag, then "littlefs"). -static void clearLegacyLittlefs(const char* label) -{ - const esp_partition_t* partition = esp_partition_find_first(ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_NVS, label); - if (partition == nullptr) { - return; - } - static constexpr uint32_t kBlockSize = 0x1000; - static constexpr uint32_t kMagicOffset = 8; - static constexpr char kMagic[] = {'l', 'i', 't', 't', 'l', 'e', 'f', 's'}; - for (uint32_t block = 0; block < 2 && (block + 1) * kBlockSize <= partition->size; ++block) { - char magic[sizeof(kMagic)]; - if (esp_partition_read(partition, block * kBlockSize + kMagicOffset, magic, sizeof(magic)) != ESP_OK || memcmp(magic, kMagic, sizeof(kMagic)) != 0) { - continue; - } - OS_LOGI(TAG, "Partition %s still holds the 1.x littlefs; erasing it", label); - if (esp_partition_erase_range(partition, 0, partition->size) != ESP_OK) { - OS_LOGE(TAG, "Failed to erase partition %s", label); - } - return; - } -} - -static const char* openStorage() -{ - if (nvsPartitionExists(kPartitionCfg)) { - clearLegacyLittlefs(kPartitionCfg); - if (Config::Nvs::InitPartition(kPartitionCfg, false)) { - return kPartitionCfg; - } - // Never the shared nvs instead: what is saved there is discarded once cfg opens again - OS_LOGE(TAG, "The cfg partition is unusable; running on defaults in RAM, nothing will be saved"); - return nullptr; - } - if (nvsPartitionExists(kPartitionShared) && Config::Nvs::InitPartition(kPartitionShared, true)) { - OS_LOGW(TAG, "No cfg partition (1.x partition layout): the config is kept in the shared nvs partition until the layout is migrated"); - return kPartitionShared; - } - OS_LOGE(TAG, "No NVS partition for the config; running on defaults in RAM, nothing will be saved"); - return nullptr; -} - -// First boot on a 2.x layout after the table rewrite: what the 1.9.9 pivot kept in the shared `nvs` is copied into -// `cfg` key by key, compared, and only then deleted from the shared partition. The schema marker (`root`) is copied -// last, so a power cut mid-copy leaves `cfg` unmarked and the copy is redone. Idempotent: once `cfg` has a schema the -// shared copy is only ever cleaned up. Returns the partition the config now lives in: `cfg`, or the shared `nvs` when -// the copy failed. -static const char* moveSharedIntoCfg() -{ - if (!nvsPartitionExists(kPartitionShared) || !Config::Nvs::InitPartition(kPartitionShared, true)) { - return kPartitionCfg; - } - // Nothing is moved or erased on a read that failed: a shared copy that cannot be read is left for the next boot, and - // so is one whose cfg cannot be checked, since erasing it then could erase the only copy - const std::optional sharedFilled = hasSchema(kPartitionShared); - const std::optional cfgFilled = hasSchema(kPartitionCfg); - if (!sharedFilled.has_value() || !cfgFilled.has_value()) { - OS_LOGE(TAG, "Could not tell whether the config lives in the shared nvs partition; leaving both partitions as they are"); - return kPartitionCfg; - } - if (!*sharedFilled) { - return kPartitionCfg; - } - const auto namespaces = allNamespaces(); // kRootNamespace first, so walked backwards - - if (!*cfgFilled) { - bool ok = true; - for (std::size_t i = namespaces.size(); ok && i-- > 0;) { - ok = Config::Nvs::CopyNamespace(kPartitionShared, kPartitionCfg, namespaces[i]) && Config::Nvs::NamespacesEqual(kPartitionShared, kPartitionCfg, namespaces[i]); - } - if (!ok) { - OS_LOGE(TAG, "Failed to copy the config into cfg; it stays in the shared nvs partition"); - for (const char* ns : namespaces) { - (void)Config::Nvs::EraseNamespace(kPartitionCfg, ns); - } - return kPartitionShared; - } - OS_LOGI(TAG, "Moved the config from the shared nvs partition into cfg"); - } - - // Walked backwards like the copy, so `root` goes last: a power cut mid-erase leaves the schema marker, and the - // cleanup is redone on the next boot instead of stranding the rest (WiFi passwords included) in the shared nvs - bool erased = true; - for (std::size_t i = namespaces.size(); i-- > 0;) { - erased &= Config::Nvs::EraseNamespace(kPartitionShared, namespaces[i]); - } - if (erased) { - OS_LOGI(TAG, "Removed the config copy from the shared nvs partition"); - } - return kPartitionCfg; -} - -// The config file a 1.x hub left behind, as a jsonconfig document. Nullopt when the hub has none. -static std::optional readLegacyConfig() -{ - const esp_partition_t* partition = esp_partition_find_first(ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_SPIFFS, kLegacyPartitionLabel); - if (partition == nullptr) { - return std::nullopt; // Not a 1.x layout: nothing to import - } - - // Read-only and never formatted: the file must survive a rollback to 1.x - LfsPartition fs; - if (!fs.mount(partition, /*writable=*/false, /*formatOnFail=*/false)) { - OS_LOGW(TAG, "The 1.x config partition does not mount; nothing to import"); - return std::nullopt; - } - std::vector blob; - bool read = fs.readAll(kLegacyFilePath, blob); - fs.unmount(); - if (!read) { - OS_LOGW(TAG, "The 1.x config partition holds no config file; nothing to import"); - return std::nullopt; - } - - auto json = Config::HubConfigToJSON(blob); - if (!json) { - OS_LOGE(TAG, "The 1.x config file (%zu bytes) is not a valid config; nothing to import", blob.size()); - return std::nullopt; - } - OS_LOGI(TAG, "Importing the 1.x config (%zu bytes)", blob.size()); - return json; -} - -// Requires the write lock. Stages `root` in every section; on success commits each (every key when `full`). -static bool applyDocument(JSON::JsonView root, bool full) +// Requires the write lock. Stages `root` in every section; on success commits each. +static bool applyDocument(JSON::JsonView root) { if (!root.isObject()) { OS_LOGE(TAG, "The config is not a JSON object"); @@ -233,7 +51,7 @@ static bool applyDocument(JSON::JsonView root, bool full) section->DiscardStaged(); continue; } - if (!section->CommitStaged(s_partition, full)) { + if (!section->CommitStaged()) { OS_LOGE(TAG, "Failed to save the %s config", section->Info().name); ok = false; } @@ -241,71 +59,25 @@ static bool applyDocument(JSON::JsonView root, bool full) return ok; } -// Requires the write lock. Fills an unmarked store: the 1.x file if there is one, else the defaults. The store is -// only marked once every section is on flash as the hub runs it; a write that failed leaves it unmarked, so the fill -// (the 1.x import included) is redone on the next boot. -static void fillStore() -{ - bool filled = false; - bool written = true; - if (auto legacy = readLegacyConfig()) { - JSON::JsonDocument doc; - if (doc.parse(*legacy) && applyDocument(doc.root(), true)) { - filled = true; - } else { - OS_LOGE(TAG, "Failed to apply the imported 1.x config; writing the defaults"); - } - } - if (!filled) { - OS_LOGW(TAG, "No stored config, writing the defaults"); - for (auto* section = Config::SectionBase::First(); section != nullptr; section = section->Next()) { - written &= section->Reset(s_partition); - } - } - - if (s_partition == nullptr) { - return; - } - // Read back: what the hub runs with must be what is on flash, as every later save assumes - for (auto* section = Config::SectionBase::First(); section != nullptr; section = section->Next()) { - if (!section->VerifyStored(s_partition)) { - OS_LOGE(TAG, "The %s config does not read back as written to %s", section->Info().name, s_partition); - written = false; - } - } - if (!written) { - OS_LOGE(TAG, "The config store is incomplete; it will be filled again on the next boot"); - return; - } - if (!writeSchema(s_partition)) { - OS_LOGE(TAG, "Failed to mark the config store; it will be filled again on the next boot"); - } -} - void Config::Init() { CONFIG_LOCK_WRITE(); - s_partition = openStorage(); - if (s_partition != nullptr && std::strcmp(s_partition, kPartitionCfg) == 0) { - s_partition = moveSharedIntoCfg(); - } - - // A store whose marker cannot be read is loaded as if marked: filling it would write the defaults over whatever is - // there, while a failed load leaves flash alone and runs the section on its defaults - if (s_partition == nullptr || hasSchema(s_partition) == false) { - fillStore(); - return; + s_partition = Config::Nvs::InitPartition(kPartitionCfg, false) ? kPartitionCfg : nullptr; + if (s_partition == nullptr) { + OS_LOGE(TAG, "The cfg partition is unusable; running on defaults in RAM, nothing will be saved"); } for (auto* section = SectionBase::First(); section != nullptr; section = section->Next()) { - if (!section->Load(s_partition)) { + if (!section->Load()) { // The section runs on its defaults, but flash is left alone: erasing it would take every key with it (all WiFi // networks). Its next save rewrites every key. OS_LOGE(TAG, "Failed to load the %s config from %s, running it on the defaults", section->Info().name, s_partition); } } - OS_LOGI(TAG, "Loaded config from %s", s_partition); + if (s_partition != nullptr) { + OS_LOGI(TAG, "Loaded config from %s", s_partition); + } } void Config::FactoryReset() @@ -314,7 +86,7 @@ void Config::FactoryReset() bool ok = true; for (auto* section = SectionBase::First(); section != nullptr; section = section->Next()) { - ok &= section->Reset(s_partition); + ok &= section->Reset(); } if (!ok) { OS_PANIC(TAG, "Failed to reset the config. Recommend formatting microcontroller and re-flashing firmware"); @@ -346,5 +118,5 @@ bool Config::SaveFromJSON(std::string_view json) } CONFIG_LOCK_WRITE(); - return applyDocument(doc.root(), false); + return applyDocument(doc.root()); } diff --git a/components/config/src/Wire.cpp b/components/config/src/Wire.cpp deleted file mode 100644 index 4225e3d36..000000000 --- a/components/config/src/Wire.cpp +++ /dev/null @@ -1,272 +0,0 @@ -#include - -#include "config/Wire.h" - -const char* const TAG = "Config::Wire"; - -#include "config/Parse.h" -#include "config/Section.h" -#include "config/Store.h" -#include "Logging.h" -#include "serialization/VerifiedRoot.h" -#include "util/HexUtils.h" - -#include "json/Json.h" - -#include -#include -#include -#include -#include - -using namespace OpenShock; -namespace Fbs = OpenShock::Serialization::Configuration; - -flatbuffers::Offset Config::GetAsFlatBuffer(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) -{ - // Every sub-table is finished before the HubConfig table starts, as flatbuffers requires - std::vector> fields; - for (auto* section = SectionBase::First(); section != nullptr; section = section->Next()) { - if (section->Info().hubConfigField != 0) { - fields.emplace_back(section->Info().hubConfigField, section->ToFlatbuffers(builder, withSensitiveData)); - } - } - - const flatbuffers::uoffset_t start = builder.StartTable(); - for (const auto& [field, offset] : fields) { - builder.AddOffset(field, flatbuffers::Offset(offset)); - } - return flatbuffers::Offset(builder.EndTable(start)); -} - -bool Config::GetRaw(TinyVec& buffer) -{ - // Write every field, even ones equal to the schema default. Older firmware may use different defaults (1.5.x reads a - // missing is_enabled/keepalive_enabled as false), so an omitted field changes meaning when the dump is loaded there. - flatbuffers::FlatBufferBuilder builder; - builder.ForceDefaults(true); - Fbs::FinishHubConfigBuffer(builder, GetAsFlatBuffer(builder, true)); - - buffer.resize(builder.GetSize()); - memcpy(buffer.data(), builder.GetBufferPointer(), builder.GetSize()); - return true; -} - -bool Config::SetRaw(const uint8_t* buffer, std::size_t size) -{ - if (buffer == nullptr) { - return false; - } - auto json = HubConfigToJSON(std::span(buffer, size)); - if (!json) { - OS_LOGE(TAG, "Not a valid HubConfig (%zu bytes)", size); - return false; - } - return SaveFromJSON(*json); -} - -// --------------------------------------------------------------------------- -// HubConfig -> jsonconfig document. Emits, per section, exactly what the section's own FromJSON reads, so a -// HubConfig and the JSON it converts to configure a hub the same way. -// --------------------------------------------------------------------------- - -namespace { - // Whether `field` is physically present in `table` (flatbuffers omits a scalar equal to its schema default). The - // generated tables derive privately from flatbuffers::Table, which is only a view of the same bytes. - bool isStored(const void* table, flatbuffers::voffset_t field) - { - return table != nullptr && static_cast(table)->CheckField(field); - } - - // Whether `field` is present in `table` with a non-zero value, ignoring the schema default. - bool storedTrue(const void* table, flatbuffers::voffset_t field) - { - return isStored(table, field) && static_cast(table)->GetField(field, 0) != 0; - } - - void setString(json_gen_str_t* gen, const char* name, const flatbuffers::String* value) - { - if (value != nullptr) { - JSON::objSetString(gen, name, std::string_view(value->c_str(), value->size())); - } - } - - struct LegacyBools { - bool keepAliveEnabled; - bool otaEnabled; - bool allowBackendManagement; - }; - - // Up to 1.5.x the schema declared rf.keepalive_enabled, ota_update.is_enabled and ota_update.allow_backend_management - // with default false; since 1.6 they default to true. Flatbuffers omits a field equal to its default, so a field - // omitted by a 1.5.x hub (false) reads back as true under the current schema. - // - // Which convention wrote a buffer shows only when at least one of the three is stored: a stored true can only come - // from the old schema (1.6+ omits true, and the 2.x wire form stores all three, so it omits none). When that is the - // case, the omitted ones meant false. A buffer that omits all three is ambiguous (1.5.x with all three off, or 1.6+ - // with all three on) and keeps the current schema's default (true). - LegacyBools resolveFlippedDefaults(const Fbs::HubConfig* config) - { - const Fbs::RFConfig* rf = config->rf(); - const Fbs::OtaUpdateConfig* ota = config->ota_update(); - - LegacyBools out { - .keepAliveEnabled = rf != nullptr ? rf->keepalive_enabled() : true, - .otaEnabled = ota != nullptr ? ota->is_enabled() : true, - .allowBackendManagement = ota != nullptr ? ota->allow_backend_management() : true, - }; - - const bool legacyWriter = storedTrue(rf, Fbs::RFConfig::VT_KEEPALIVE_ENABLED) || storedTrue(ota, Fbs::OtaUpdateConfig::VT_IS_ENABLED) || storedTrue(ota, Fbs::OtaUpdateConfig::VT_ALLOW_BACKEND_MANAGEMENT); - if (legacyWriter) { - if (rf != nullptr && !isStored(rf, Fbs::RFConfig::VT_KEEPALIVE_ENABLED)) out.keepAliveEnabled = false; - if (ota != nullptr && !isStored(ota, Fbs::OtaUpdateConfig::VT_IS_ENABLED)) out.otaEnabled = false; - if (ota != nullptr && !isStored(ota, Fbs::OtaUpdateConfig::VT_ALLOW_BACKEND_MANAGEMENT)) out.allowBackendManagement = false; - } - return out; - } - - void writeRf(json_gen_str_t* gen, const Fbs::RFConfig* rf, const LegacyBools& bools) - { - if (rf == nullptr) { - return; - } - json_gen_push_object(gen, "rf"); - json_gen_obj_set_int(gen, "txPin", rf->tx_pin()); // RFConfig::FromJSON maps "no pin" and invalid pins to the board's - json_gen_obj_set_bool(gen, "keepAliveEnabled", bools.keepAliveEnabled); - json_gen_pop_object(gen); - } - - void writeWifi(json_gen_str_t* gen, const Fbs::WiFiConfig* wifi) - { - if (wifi == nullptr) { - return; - } - json_gen_push_object(gen, "wifi"); - setString(gen, "accessPointSSID", wifi->ap_ssid()); - setString(gen, "hostname", wifi->hostname()); - json_gen_push_array(gen, "credentials"); // Required by WiFiConfig::FromJSON whenever "wifi" is given - if (const auto* list = wifi->credentials()) { - for (const Fbs::WiFiCredentials* creds : *list) { - if (creds == nullptr) { - continue; - } - json_gen_start_object(gen); - json_gen_obj_set_int(gen, "id", creds->id()); - setString(gen, "ssid", creds->ssid()); // A credential without an SSID is dropped by WiFiCredentials::FromJSON - setString(gen, "password", creds->password()); - json_gen_obj_set_int(gen, "authMode", std::to_underlying(creds->auth_mode())); - if (const auto* bssid = creds->bssid()) { - JSON::objSetString(gen, "bssid", HexUtils::ToHex(std::span(bssid->bytes()->data(), 6)).data()); - } - json_gen_end_object(gen); - } - } - json_gen_pop_array(gen); - json_gen_pop_object(gen); - } - - void writeCaptivePortal(json_gen_str_t* gen, const Fbs::CaptivePortalConfig* portal) - { - if (portal == nullptr) { - return; - } - json_gen_push_object(gen, "captivePortal"); - json_gen_obj_set_bool(gen, "alwaysEnabled", portal->always_enabled()); - json_gen_pop_object(gen); - } - - void writeBackend(json_gen_str_t* gen, const Fbs::BackendConfig* backend) - { - if (backend == nullptr) { - return; - } - json_gen_push_object(gen, "backend"); - setString(gen, "domain", backend->domain()); - setString(gen, "authToken", backend->auth_token()); - json_gen_pop_object(gen); - } - - void writeSerialInput(json_gen_str_t* gen, const Fbs::SerialInputConfig* serial) - { - if (serial == nullptr) { - return; - } - json_gen_push_object(gen, "serialInput"); - json_gen_obj_set_bool(gen, "echoEnabled", serial->echo_enabled()); - json_gen_pop_object(gen); - } - - void writeOtaUpdate(json_gen_str_t* gen, const Fbs::OtaUpdateConfig* ota, const LegacyBools& bools) - { - if (ota == nullptr) { - return; - } - json_gen_push_object(gen, "otaUpdate"); - json_gen_obj_set_bool(gen, "isEnabled", bools.otaEnabled); - setString(gen, "cdnDomain", ota->cdn_domain()); - // Out-of-range values (bit flips, configs from newer firmware) have no name and are left out: the default - if (const char* channel = Fbs::EnumNameOtaUpdateChannel(ota->update_channel()); channel != nullptr && channel[0] != '\0') { - JSON::objSetString(gen, "updateChannel", channel); - } - json_gen_obj_set_bool(gen, "checkOnStartup", ota->check_on_startup()); - json_gen_obj_set_bool(gen, "checkPeriodically", ota->check_periodically()); - json_gen_obj_set_int(gen, "checkInterval", ota->check_interval()); - json_gen_obj_set_bool(gen, "allowBackendManagement", bools.allowBackendManagement); - json_gen_obj_set_bool(gen, "requireManualApproval", ota->require_manual_approval()); - json_gen_obj_set_int(gen, "updateId", ota->update_id()); - if (const char* step = Fbs::EnumNameOtaUpdateStep(ota->update_step()); step != nullptr && step[0] != '\0') { - JSON::objSetString(gen, "updateStep", step); - } - json_gen_pop_object(gen); - } - - void writeEStop(json_gen_str_t* gen, const Fbs::EStopConfig* estop) - { - if (estop == nullptr) { - return; - } - // A pin that is neither "no pin" nor a real GPIO means a corrupt section: left out, so it takes its defaults - if (!Config::Parse::GpioNum(estop->gpio_pin())) { - OS_LOGW(TAG, "Invalid E-Stop GPIO pin %d, using the default E-Stop config", estop->gpio_pin()); - return; - } - json_gen_push_object(gen, "estop"); - json_gen_obj_set_bool(gen, "enabled", estop->enabled()); - json_gen_obj_set_int(gen, "gpioPin", estop->gpio_pin()); - json_gen_pop_object(gen); - } -} // namespace - -std::optional Config::HubConfigToJSON(std::span buffer) -{ - if (buffer.empty() || buffer.size() > kMaxHubConfigSize) { - OS_LOGE(TAG, "HubConfig has an impossible size (%zu bytes)", buffer.size()); - return std::nullopt; - } - const Fbs::HubConfig* config = Serialization::GetVerifiedRoot(buffer, kMaxHubConfigSize); - if (config == nullptr) { - OS_LOGE(TAG, "HubConfig failed verification (%zu bytes)", buffer.size()); - return std::nullopt; - } - - const LegacyBools bools = resolveFlippedDefaults(config); - - JSON::StringWriter writer; - json_gen_str_t* gen = writer.gen(); - json_gen_start_object(gen); - writeRf(gen, config->rf(), bools); - writeWifi(gen, config->wifi()); - writeCaptivePortal(gen, config->captive_portal()); - writeBackend(gen, config->backend()); - writeSerialInput(gen, config->serial_input()); - writeOtaUpdate(gen, config->ota_update(), bools); - writeEStop(gen, config->estop()); - json_gen_end_object(gen); - - std::string json = writer.finish(); - if (writer.failed()) { - OS_LOGE(TAG, "HubConfig holds a string that is not valid UTF-8"); - return std::nullopt; - } - return json; -} diff --git a/components/core/CMakeLists.txt b/components/core/CMakeLists.txt index 0812a8271..a2e7fe992 100644 --- a/components/core/CMakeLists.txt +++ b/components/core/CMakeLists.txt @@ -1,9 +1,8 @@ # OpenShock application core. # -# The tightly-interconnected heart of the firmware: wifi, the gateway link and the -# websocket message handlers, plus the JSON/flatbuffer codecs they share. These form a -# single dependency cycle (wifi <-> gateway <-> message handlers) and therefore live -# here together. +# The heart of the firmware: wifi, the gateway link and the websocket message handlers, +# plus the JSON/flatbuffer codecs they share. The gateway uses wifi one way; wifi reaches +# up only through AppHooks and the portal messages it builds. # # The captive portal and the OTA update manager were split out into captive_portal and # ota, which sit above core. core calls back into them only through AppHooks.h. diff --git a/components/core/include/BackendConfig.h b/components/core/include/BackendConfig.h index e2e718639..b6bcaef05 100644 --- a/components/core/include/BackendConfig.h +++ b/components/core/include/BackendConfig.h @@ -4,7 +4,6 @@ // and kept in NVS through the config framework (config/Section.h); namespace "backend", keys domain and token (str). #include "config/Section.h" -#include "serialization/_fbs/HubConfig_generated.h" #include #include @@ -25,8 +24,6 @@ namespace OpenShock::Config { /// @brief No rules to apply yet. The section runs it on every value it takes in. void Normalize() { } - [[nodiscard]] flatbuffers::Offset ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const; - /// @brief Reads this section from `partition`; a missing key is its default, a read error fails. bool LoadNvs(const char* partition); /// @brief Writes the keys that differ from `previous` (every key when null) to `partition`. diff --git a/components/core/include/OtaUpdateConfig.h b/components/core/include/OtaUpdateConfig.h index 6be5faa1c..c42629709 100644 --- a/components/core/include/OtaUpdateConfig.h +++ b/components/core/include/OtaUpdateConfig.h @@ -10,7 +10,6 @@ #include "enums/FirmwareBootType.h" #include "enums/OtaUpdateChannel.h" #include "enums/OtaUpdateStep.h" -#include "serialization/_fbs/HubConfig_generated.h" #include "util/FunctionRef.h" @@ -41,11 +40,10 @@ namespace OpenShock::Config { OtaUpdateStep updateStep; /// @brief The section runs it on every value it takes in: a zero check interval would check on every wake-up - /// of the update task, so it becomes the default (the captive portal refuses it, JSON and NVS do not). + /// of the update task, so it becomes the default (the captive portal refuses it, JSON and NVS do not). The + /// 1.x CDN domains, which serve no repository API, become the default too. void Normalize(); - [[nodiscard]] flatbuffers::Offset ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const; - /// @brief Reads this section from `partition`; a missing key is its default, a read error fails. bool LoadNvs(const char* partition); /// @brief Writes the keys that differ from `previous` (every key when null) to `partition`. diff --git a/components/core/include/wifi/WiFiAuthMode.h b/components/core/include/wifi/WiFiAuthMode.h deleted file mode 100644 index 190f204e9..000000000 --- a/components/core/include/wifi/WiFiAuthMode.h +++ /dev/null @@ -1,66 +0,0 @@ -#pragma once - -#include "serialization/_fbs/WifiAuthMode_generated.h" - -#include - -namespace OpenShock::Config { - // The one mapping between ESP-IDF's wifi_auth_mode_t and the FlatBuffers WifiAuthMode used by the stored credential - // blobs, the jsonconfig document and the local (captive portal) protocol. Modes without a FlatBuffers counterpart map to UNKNOWN / WIFI_AUTH_MAX. - constexpr Serialization::Types::WifiAuthMode ToFbsAuthMode(wifi_auth_mode_t mode) - { - using FbsAuthMode = Serialization::Types::WifiAuthMode; - switch (mode) { - case WIFI_AUTH_OPEN: - return FbsAuthMode::Open; - case WIFI_AUTH_WEP: - return FbsAuthMode::WEP; - case WIFI_AUTH_WPA_PSK: - return FbsAuthMode::WPA_PSK; - case WIFI_AUTH_WPA2_PSK: - return FbsAuthMode::WPA2_PSK; - case WIFI_AUTH_WPA_WPA2_PSK: - return FbsAuthMode::WPA_WPA2_PSK; - case WIFI_AUTH_WPA2_ENTERPRISE: - return FbsAuthMode::WPA2_ENTERPRISE; - case WIFI_AUTH_WPA3_PSK: - return FbsAuthMode::WPA3_PSK; - case WIFI_AUTH_WPA2_WPA3_PSK: - return FbsAuthMode::WPA2_WPA3_PSK; - case WIFI_AUTH_WAPI_PSK: - return FbsAuthMode::WAPI_PSK; - default: - return FbsAuthMode::UNKNOWN; - } - } - - constexpr wifi_auth_mode_t FromFbsAuthMode(Serialization::Types::WifiAuthMode mode) - { - using FbsAuthMode = Serialization::Types::WifiAuthMode; - switch (mode) { - case FbsAuthMode::Open: - return WIFI_AUTH_OPEN; - case FbsAuthMode::WEP: - return WIFI_AUTH_WEP; - case FbsAuthMode::WPA_PSK: - return WIFI_AUTH_WPA_PSK; - case FbsAuthMode::WPA2_PSK: - return WIFI_AUTH_WPA2_PSK; - case FbsAuthMode::WPA_WPA2_PSK: - return WIFI_AUTH_WPA_WPA2_PSK; - case FbsAuthMode::WPA2_ENTERPRISE: - return WIFI_AUTH_WPA2_ENTERPRISE; - case FbsAuthMode::WPA3_PSK: - return WIFI_AUTH_WPA3_PSK; - case FbsAuthMode::WPA2_WPA3_PSK: - return WIFI_AUTH_WPA2_WPA3_PSK; - case FbsAuthMode::WAPI_PSK: - return WIFI_AUTH_WAPI_PSK; - case FbsAuthMode::UNKNOWN: - return WIFI_AUTH_MAX; - } - - // Not unreachable: the value comes from stored config / the local protocol and can hold any byte - return WIFI_AUTH_MAX; - } -} // namespace OpenShock::Config diff --git a/components/core/include/wifi/WiFiConfig.h b/components/core/include/wifi/WiFiConfig.h index 0027ee663..c277bf28a 100644 --- a/components/core/include/wifi/WiFiConfig.h +++ b/components/core/include/wifi/WiFiConfig.h @@ -6,7 +6,6 @@ // namespace "wifi": one blob per credential, key = its id in decimal ("1".."255"); enumerated, never counted #include "config/Section.h" -#include "serialization/_fbs/HubConfig_generated.h" #include "util/FunctionRef.h" #include "wifi/WiFiCredentials.h" @@ -35,8 +34,6 @@ namespace OpenShock::Config { bool FromJSON(JSON::JsonView json); void ToJSON(json_gen_str_t* gen, const char* name, bool withSensitiveData) const; - - [[nodiscard]] flatbuffers::Offset ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const; }; [[nodiscard]] WiFiConfig GetWiFiConfig(); diff --git a/components/core/include/wifi/WiFiCredentials.h b/components/core/include/wifi/WiFiCredentials.h index c3db60dbf..d56f627cf 100644 --- a/components/core/include/wifi/WiFiCredentials.h +++ b/components/core/include/wifi/WiFiCredentials.h @@ -1,7 +1,5 @@ #pragma once -#include "serialization/_fbs/HubConfig_generated.h" - #include #include @@ -37,8 +35,6 @@ namespace OpenShock::Config { /// @brief Decodes a blob written by ToBlob (any version it ever wrote). False on a malformed blob. bool FromBlob(std::span blob); - [[nodiscard]] flatbuffers::Offset ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const; - bool FromJSON(JSON::JsonView json); void ToJSON(json_gen_str_t* gen, const char* name, bool withSensitiveData) const; }; diff --git a/components/core/src/BackendConfig.cpp b/components/core/src/BackendConfig.cpp index 8ab287175..3c4977bfe 100644 --- a/components/core/src/BackendConfig.cpp +++ b/components/core/src/BackendConfig.cpp @@ -11,14 +11,13 @@ const char* const TAG = "Config::BackendConfig"; #include "json/Json.h" using namespace OpenShock::Config; -namespace Fbs = OpenShock::Serialization::Configuration; static constexpr const char* kNamespace = "backend"; static constexpr const char* kKeyDomain = "domain"; static constexpr const char* kKeyToken = "token"; static constexpr const char* kNamespaces[] = {kNamespace}; -static Section s_section({.name = "backend", .order = 3, .hubConfigField = Fbs::HubConfig::VT_BACKEND, .namespaces = kNamespaces}); +static Section s_section({.name = "backend", .order = 3, .namespaces = kNamespaces}); BackendConfig::BackendConfig() : domain(CONFIG_OPENSHOCK_API_DOMAIN) @@ -32,20 +31,6 @@ BackendConfig::BackendConfig(std::string_view domain, std::string_view authToken { } -flatbuffers::Offset BackendConfig::ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const -{ - auto domainOffset = builder.CreateString(domain); - - flatbuffers::Offset authTokenOffset; - if (withSensitiveData) { - authTokenOffset = builder.CreateString(authToken); - } else { - authTokenOffset = 0; - } - - return Serialization::Configuration::CreateBackendConfig(builder, domainOffset, authTokenOffset); -} - bool BackendConfig::FromJSON(JSON::JsonView json) { if (auto done = Parse::ObjectOrDefault(*this, json, TAG)) { @@ -81,7 +66,7 @@ bool BackendConfig::LoadNvs(const char* partition) bool BackendConfig::SaveNvs(const char* partition, const BackendConfig* previous) const { - auto handle = Nvs::Handle::Open(partition, kNamespace, true); + auto handle = Nvs::Handle::OpenForWrite(partition, kNamespace); if (!handle) { return false; } diff --git a/components/core/src/GatewayClient.cpp b/components/core/src/GatewayClient.cpp index 96c204a1a..acd084b71 100644 --- a/components/core/src/GatewayClient.cpp +++ b/components/core/src/GatewayClient.cpp @@ -13,7 +13,6 @@ const char* const TAG = "GatewayClient"; #include "OtaUpdateConfig.h" #include "serialization/WSGateway.h" #include "Temporal.h" -#include "visual/VisualStateManager.h" #include diff --git a/components/core/src/GatewayConnectionManager.cpp b/components/core/src/GatewayConnectionManager.cpp index aa3559054..f65cb94ec 100644 --- a/components/core/src/GatewayConnectionManager.cpp +++ b/components/core/src/GatewayConnectionManager.cpp @@ -4,8 +4,6 @@ const char* const TAG = "GatewayConnectionManager"; -#include "visual/VisualStateManager.h" - #include "AppHooks.h" #include "BackendConfig.h" #include "events/Events.h" @@ -28,9 +26,6 @@ const char* const TAG = "GatewayConnectionManager"; #include #include -const char* const AUTH_TOKEN_FILE = "/authToken"; - -const uint8_t FLAG_NONE = 0; const uint8_t FLAG_HAS_IP = 1 << 0; const uint8_t FLAG_LINKED = 1 << 1; diff --git a/components/core/src/OtaUpdateConfig.cpp b/components/core/src/OtaUpdateConfig.cpp index fd9279d49..aea81d3fa 100644 --- a/components/core/src/OtaUpdateConfig.cpp +++ b/components/core/src/OtaUpdateConfig.cpp @@ -10,10 +10,12 @@ const char* const TAG = "Config::OtaUpdateConfig"; #include "json/Json.h" +#include +#include +#include #include using namespace OpenShock::Config; -namespace Fbs = OpenShock::Serialization::Configuration; static constexpr const char* kNamespace = "ota"; static constexpr const char* kKeyEnabled = "enabled"; @@ -29,8 +31,11 @@ static constexpr const char* kKeyStep = "step"; static constexpr uint16_t kDefaultCheckInterval = 30; // minutes +// The domains of the 1.x firmware CDN: they serve flat files for 1.x hubs, not the repository API this firmware asks for. +static constexpr std::string_view kLegacyCdnDomains[] = {"firmware.openshock.org", "fw.openshock.org"}; + static constexpr const char* kNamespaces[] = {kNamespace}; -static Section s_section({.name = "otaUpdate", .order = 5, .hubConfigField = Fbs::HubConfig::VT_OTA_UPDATE, .namespaces = kNamespaces}); +static Section s_section({.name = "otaUpdate", .order = 5, .namespaces = kNamespaces}); using namespace std::string_view_literals; @@ -64,30 +69,6 @@ OtaUpdateConfig::OtaUpdateConfig( { } -flatbuffers::Offset OtaUpdateConfig::ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const -{ - namespace Fbs = Serialization::Configuration; - - auto cdnDomainOffset = builder.CreateString(cdnDomain); - - // Same field order as CreateOtaUpdateConfig, except that is_enabled and - // allow_backend_management are stored even when they equal the schema default, so - // they read back the same under every schema version (up to 1.5.x both defaulted - // to false) and RootConfig::FromFlatbuffers never has to infer them. - Fbs::OtaUpdateConfigBuilder otaBuilder(builder); - otaBuilder.add_update_id(updateId); - otaBuilder.add_cdn_domain(cdnDomainOffset); - otaBuilder.add_check_interval(checkInterval); - otaBuilder.add_update_step(static_cast(updateStep)); - otaBuilder.add_require_manual_approval(requireManualApproval); - builder.AddElement(Fbs::OtaUpdateConfig::VT_ALLOW_BACKEND_MANAGEMENT, static_cast(allowBackendManagement)); - otaBuilder.add_check_periodically(checkPeriodically); - otaBuilder.add_check_on_startup(checkOnStartup); - otaBuilder.add_update_channel(static_cast(updateChannel)); - builder.AddElement(Fbs::OtaUpdateConfig::VT_IS_ENABLED, static_cast(isEnabled)); - return otaBuilder.Finish(); -} - bool OtaUpdateConfig::FromJSON(JSON::JsonView json) { if (auto done = Parse::ObjectOrDefault(*this, json, TAG)) { @@ -115,14 +96,14 @@ void OtaUpdateConfig::ToJSON(json_gen_str_t* gen, const char* name, bool withSen JSON::objBegin(gen, name); json_gen_obj_set_bool(gen, "isEnabled", isEnabled); JSON::objSetString(gen, "cdnDomain", cdnDomain); - JSON::objSetString(gen, "updateChannel", OpenShock::Serialization::Configuration::EnumNameOtaUpdateChannel(static_cast(updateChannel))); + JSON::objSetString(gen, "updateChannel", OpenShock::OtaUpdateChannelName(updateChannel)); json_gen_obj_set_bool(gen, "checkOnStartup", checkOnStartup); json_gen_obj_set_bool(gen, "checkPeriodically", checkPeriodically); json_gen_obj_set_int(gen, "checkInterval", checkInterval); json_gen_obj_set_bool(gen, "allowBackendManagement", allowBackendManagement); json_gen_obj_set_bool(gen, "requireManualApproval", requireManualApproval); json_gen_obj_set_int(gen, "updateId", updateId); - JSON::objSetString(gen, "updateStep", OpenShock::Serialization::Configuration::EnumNameOtaUpdateStep(static_cast(updateStep))); + JSON::objSetString(gen, "updateStep", OpenShock::OtaUpdateStepName(updateStep)); JSON::objEnd(gen, name); } @@ -132,6 +113,12 @@ void OtaUpdateConfig::Normalize() OS_LOGW(TAG, "A zero update check interval is not allowed, using the default"); checkInterval = kDefaultCheckInterval; } + // Rewritten on every load rather than once: it needs no migration state, and a hub that restores an old config still + // lands here. Remove it once no hub or config backup can hold the 1.x domain. + if (std::ranges::find(kLegacyCdnDomains, cdnDomain) != std::end(kLegacyCdnDomains)) { + OS_LOGI(TAG, "Moving OTA updates from the 1.x CDN to %s", CONFIG_OPENSHOCK_FW_CDN_DOMAIN); + cdnDomain = CONFIG_OPENSHOCK_FW_CDN_DOMAIN; + } } bool OtaUpdateConfig::LoadNvs(const char* partition) @@ -157,7 +144,7 @@ bool OtaUpdateConfig::LoadNvs(const char* partition) bool OtaUpdateConfig::SaveNvs(const char* partition, const OtaUpdateConfig* previous) const { - auto handle = Nvs::Handle::Open(partition, kNamespace, true); + auto handle = Nvs::Handle::OpenForWrite(partition, kNamespace); if (!handle) { return false; } diff --git a/components/core/src/serialization/WSGateway.cpp b/components/core/src/serialization/WSGateway.cpp index fe5078f2f..02890a91e 100644 --- a/components/core/src/serialization/WSGateway.cpp +++ b/components/core/src/serialization/WSGateway.cpp @@ -3,6 +3,7 @@ const char* const TAG = "WSGateway"; #include +#include #include "Logging.h" #include "OpenShock.h" @@ -43,6 +44,11 @@ bool Gateway::SerializePongMessage(Common::SerializationCallbackFn callback) return finishMessage(builder, Gateway::HubToGatewayMessagePayload::Pong, pong, callback); } +// SerializeBootStatusMessage casts between these by value, and the gateway protocol must stay compatible +static_assert(std::to_underlying(OpenShock::FirmwareBootType::Normal) == std::to_underlying(Types::FirmwareBootType::Normal)); +static_assert(std::to_underlying(OpenShock::FirmwareBootType::NewFirmware) == std::to_underlying(Types::FirmwareBootType::NewFirmware)); +static_assert(std::to_underlying(OpenShock::FirmwareBootType::Rollback) == std::to_underlying(Types::FirmwareBootType::Rollback)); + bool Gateway::SerializeBootStatusMessage(int32_t updateId, OpenShock::FirmwareBootType bootType, Common::SerializationCallbackFn callback) { flatbuffers::FlatBufferBuilder builder(128); diff --git a/components/core/src/serialization/WSLocal.cpp b/components/core/src/serialization/WSLocal.cpp index e68ca2ad0..9b6529df5 100644 --- a/components/core/src/serialization/WSLocal.cpp +++ b/components/core/src/serialization/WSLocal.cpp @@ -1,16 +1,20 @@ -#include "serialization/WSLocal.h" +#include -const char* const TAG = "WSLocal"; +#include "serialization/WSLocal.h" #include "Chipset.h" -#include "config/Wire.h" -#include "Logging.h" +#include "estop/EStopConfig.h" +#include "OtaUpdateConfig.h" +#include "radio/RFConfig.h" #include "util/HexUtils.h" -#include "wifi/WiFiAuthMode.h" +#include "wifi/WiFiConfig.h" #include "wifi/WiFiNetwork.h" #include "serialization/_fbs/HubToLocalMessage_generated.h" +#include +#include + using namespace OpenShock::Serialization; // Wraps a payload in a HubToLocalMessage, finishes the buffer and hands it to the callback. @@ -22,14 +26,55 @@ static bool finishMessage(flatbuffers::FlatBufferBuilder& builder, Local::HubToL return callback(builder.GetBufferSpan()); } +// Modes without a counterpart in the schema are UNKNOWN. +static constexpr Types::WifiAuthMode toFbsAuthMode(wifi_auth_mode_t mode) +{ + switch (mode) { + case WIFI_AUTH_OPEN: + return Types::WifiAuthMode::Open; + case WIFI_AUTH_WEP: + return Types::WifiAuthMode::WEP; + case WIFI_AUTH_WPA_PSK: + return Types::WifiAuthMode::WPA_PSK; + case WIFI_AUTH_WPA2_PSK: + return Types::WifiAuthMode::WPA2_PSK; + case WIFI_AUTH_WPA_WPA2_PSK: + return Types::WifiAuthMode::WPA_WPA2_PSK; + case WIFI_AUTH_WPA2_ENTERPRISE: + return Types::WifiAuthMode::WPA2_ENTERPRISE; + case WIFI_AUTH_WPA3_PSK: + return Types::WifiAuthMode::WPA3_PSK; + case WIFI_AUTH_WPA2_WPA3_PSK: + return Types::WifiAuthMode::WPA2_WPA3_PSK; + case WIFI_AUTH_WAPI_PSK: + return Types::WifiAuthMode::WAPI_PSK; + default: + return Types::WifiAuthMode::UNKNOWN; + } +} + static flatbuffers::Offset createWiFiNetwork(flatbuffers::FlatBufferBuilder& builder, const OpenShock::WiFiNetwork& network) { auto bssid = network.GetHexBSSID(); - auto authMode = OpenShock::Config::ToFbsAuthMode(network.authMode); + auto authMode = toFbsAuthMode(network.authMode); return Types::CreateWifiNetworkDirect(builder, network.ssid, bssid.data(), network.channel, network.rssi, authMode, network.IsSaved()); } +// createOtaSettings casts between these by value +static_assert(std::to_underlying(OpenShock::OtaUpdateChannel::Stable) == std::to_underlying(Local::OtaChannel::Stable)); +static_assert(std::to_underlying(OpenShock::OtaUpdateChannel::Beta) == std::to_underlying(Local::OtaChannel::Beta)); +static_assert(std::to_underlying(OpenShock::OtaUpdateChannel::Develop) == std::to_underlying(Local::OtaChannel::Develop)); + +static flatbuffers::Offset createOtaSettings(flatbuffers::FlatBufferBuilder& builder) +{ + const auto ota = OpenShock::Config::GetOtaUpdateConfig(); + + auto cdnDomain = builder.CreateString(ota.cdnDomain); + + return Local::CreateOtaSettings(builder, ota.isEnabled, cdnDomain, static_cast(ota.updateChannel), ota.checkInterval, ota.allowBackendManagement, ota.requireManualApproval); +} + bool Local::SerializeErrorMessage(std::string_view message, Common::SerializationCallbackFn callback) { flatbuffers::FlatBufferBuilder builder(64 + message.length()); @@ -49,19 +94,25 @@ bool Local::SerializeReadyMessage(const WiFiNetwork* connectedNetwork, bool acco fbsNetwork = createWiFiNetwork(builder, *connectedNetwork); } - auto configOffset = OpenShock::Config::GetAsFlatBuffer(builder, false); - if (configOffset.IsNull()) { - OS_LOGE(TAG, "Failed to serialize config"); - return false; - } - auto inputPins = OpenShock::GetValidInputPinsVector(); auto inputPinsOffset = builder.CreateVector(inputPins); auto outputPins = OpenShock::GetValidOutputPinsVector(); auto outputPinsOffset = builder.CreateVector(outputPins); - auto readyMessageOffset = Serialization::Local::CreateReadyMessage(builder, true, fbsNetwork, accountLinked, configOffset, inputPinsOffset, outputPinsOffset); + // Only what the portal shows. The whole config has no wire form, and secrets never go into this message. + std::vector> savedSsids; + for (const auto& credentials : OpenShock::Config::GetWiFiCredentials()) { + savedSsids.push_back(builder.CreateString(credentials.ssid)); + } + auto savedSsidsOffset = builder.CreateVector(savedSsids); + + auto otaOffset = createOtaSettings(builder); + + const auto rfTxPin = static_cast(OpenShock::Config::GetRFConfigTxPin()); + const auto estop = OpenShock::Config::GetEStopConfig(); + + auto readyMessageOffset = Serialization::Local::CreateReadyMessage(builder, true, fbsNetwork, accountLinked, inputPinsOffset, outputPinsOffset, false, rfTxPin, estop.enabled, static_cast(estop.gpioPin), savedSsidsOffset, otaOffset); return finishMessage(builder, Local::HubToLocalMessagePayload::ReadyMessage, readyMessageOffset, callback); } diff --git a/components/core/src/wifi/WiFiConfig.cpp b/components/core/src/wifi/WiFiConfig.cpp index 2a242ad84..28699452b 100644 --- a/components/core/src/wifi/WiFiConfig.cpp +++ b/components/core/src/wifi/WiFiConfig.cpp @@ -18,7 +18,6 @@ const char* const TAG = "Config::WiFiConfig"; #include using namespace OpenShock::Config; -namespace Fbs = OpenShock::Serialization::Configuration; static_assert(OpenShock::HostnameUtils::IsValid(CONFIG_OPENSHOCK_FW_HOSTNAME), "CONFIG_OPENSHOCK_FW_HOSTNAME must be 1-32 letters, digits or hyphens, not starting or ending with a hyphen"); @@ -28,7 +27,7 @@ static constexpr const char* kKeyApSsid = "ap_ssid"; static constexpr const char* kKeyHostname = "hostname"; static constexpr const char* kNamespaces[] = {kNamespaceCfg, kNamespaceCreds}; -static Section s_section({.name = "wifi", .order = 1, .hubConfigField = Fbs::HubConfig::VT_WIFI, .namespaces = kNamespaces}); +static Section s_section({.name = "wifi", .order = 1, .namespaces = kNamespaces}); WiFiConfig::WiFiConfig() : accessPointSSID(CONFIG_OPENSHOCK_FW_AP_PREFIX) @@ -44,18 +43,6 @@ WiFiConfig::WiFiConfig(std::string_view accessPointSSID, std::string_view hostna { } -flatbuffers::Offset WiFiConfig::ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const -{ - std::vector> fbsCredentialsList; - fbsCredentialsList.reserve(credentialsList.size()); - - for (auto& credentials : credentialsList) { - fbsCredentialsList.push_back(credentials.ToFlatbuffers(builder, withSensitiveData)); - } - - return Serialization::Configuration::CreateWiFiConfig(builder, builder.CreateString(accessPointSSID), builder.CreateString(hostname), builder.CreateVector(fbsCredentialsList)); -} - bool WiFiConfig::FromJSON(JSON::JsonView json) { if (auto done = Parse::ObjectOrDefault(*this, json, TAG)) { @@ -207,7 +194,7 @@ bool WiFiConfig::LoadNvs(const char* partition) bool WiFiConfig::SaveNvs(const char* partition, const WiFiConfig* previous) const { { - auto handle = Nvs::Handle::Open(partition, kNamespaceCfg, true); + auto handle = Nvs::Handle::OpenForWrite(partition, kNamespaceCfg); if (!handle) { return false; } @@ -216,7 +203,7 @@ bool WiFiConfig::SaveNvs(const char* partition, const WiFiConfig* previous) cons } } - auto handle = Nvs::Handle::Open(partition, kNamespaceCreds, true); + auto handle = Nvs::Handle::OpenForWrite(partition, kNamespaceCreds); if (!handle) { return false; } diff --git a/components/core/src/wifi/WiFiCredentials.cpp b/components/core/src/wifi/WiFiCredentials.cpp index cc57c024a..c6c816b8f 100644 --- a/components/core/src/wifi/WiFiCredentials.cpp +++ b/components/core/src/wifi/WiFiCredentials.cpp @@ -4,7 +4,6 @@ const char* const TAG = "Config::WiFiCredentials"; #include "config/Parse.h" #include "Logging.h" -#include "wifi/WiFiAuthMode.h" #include "util/HexUtils.h" #include "json/Json.h" @@ -14,7 +13,41 @@ const char* const TAG = "Config::WiFiCredentials"; #include using namespace OpenShock::Config; -using FbsAuthMode = OpenShock::Serialization::Types::WifiAuthMode; + +// The auth mode codes the blob and the jsonconfig document store. Owned by this file alone: neither ESP-IDF's +// wifi_auth_mode_t nor the FlatBuffers schema may renumber what is on flash. Never reuse or renumber a code; append new ones. +static constexpr std::pair kBlobAuthModes[] = { + {0, WIFI_AUTH_OPEN}, + {1, WIFI_AUTH_WEP}, + {2, WIFI_AUTH_WPA_PSK}, + {3, WIFI_AUTH_WPA2_PSK}, + {4, WIFI_AUTH_WPA_WPA2_PSK}, + {5, WIFI_AUTH_WPA2_ENTERPRISE}, + {6, WIFI_AUTH_WPA3_PSK}, + {7, WIFI_AUTH_WPA2_WPA3_PSK}, + {8, WIFI_AUTH_WAPI_PSK}, +}; +static constexpr uint8_t kBlobAuthUnknown = 9; + +static uint8_t toBlobAuthMode(wifi_auth_mode_t mode) +{ + for (const auto& [code, known] : kBlobAuthModes) { + if (known == mode) { + return code; + } + } + return kBlobAuthUnknown; +} + +static wifi_auth_mode_t fromBlobAuthMode(uint8_t code) +{ + for (const auto& [known, mode] : kBlobAuthModes) { + if (known == code) { + return mode; + } + } + return WIFI_AUTH_MAX; +} WiFiCredentials::WiFiCredentials() : id(0) @@ -42,27 +75,6 @@ bool WiFiCredentials::HasPinnedBSSID() const return false; } -flatbuffers::Offset WiFiCredentials::ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const -{ - auto ssidOffset = builder.CreateString(ssid); - - flatbuffers::Offset passwordOffset; - if (withSensitiveData) { - passwordOffset = builder.CreateString(password); - } else { - passwordOffset = 0; - } - - const Serialization::Configuration::MacAddress* bssidPtr = nullptr; - Serialization::Configuration::MacAddress bssidStruct; - if (HasPinnedBSSID()) { - bssidStruct = Serialization::Configuration::MacAddress(flatbuffers::span(bssid.data(), 6)); - bssidPtr = &bssidStruct; - } - - return Serialization::Configuration::CreateWiFiCredentials(builder, id, ssidOffset, passwordOffset, ToFbsAuthMode(authMode), bssidPtr); -} - bool WiFiCredentials::FromJSON(JSON::JsonView json) { if (auto done = Parse::ObjectOrDefault(*this, json, TAG)) { @@ -75,8 +87,7 @@ bool WiFiCredentials::FromJSON(JSON::JsonView json) password = json["password"].tryGetStr().value_or(std::string()); - uint8_t authModeVal = json["authMode"].tryGetU8().value_or(std::to_underlying(FbsAuthMode::UNKNOWN)); - authMode = FromFbsAuthMode(static_cast(authModeVal)); + authMode = fromBlobAuthMode(json["authMode"].tryGetU8().value_or(kBlobAuthUnknown)); bssid.fill(0); if (auto bssidStr = json["bssid"].tryGetStr(); bssidStr && bssidStr->size() == 12) { @@ -104,7 +115,7 @@ void WiFiCredentials::ToJSON(json_gen_str_t* gen, const char* name, bool withSen if (withSensitiveData) { JSON::objSetString(gen, "password", password); } - json_gen_obj_set_int(gen, "authMode", std::to_underlying(ToFbsAuthMode(authMode))); + json_gen_obj_set_int(gen, "authMode", toBlobAuthMode(authMode)); if (HasPinnedBSSID()) { JSON::objSetString(gen, "bssid", HexUtils::ToHex(std::span(bssid)).data()); } @@ -119,41 +130,6 @@ void WiFiCredentials::ToJSON(json_gen_str_t* gen, const char* name, bool withSen // change an older reader must not misread, and that costs a rolled-back hub its saved networks. static constexpr uint8_t kBlobVersion = 1; -// The auth mode codes the blob stores. Owned by this format alone: neither ESP-IDF's wifi_auth_mode_t nor the -// FlatBuffers schema may renumber what is on flash. Never reuse or renumber a code; append new ones. -static constexpr std::pair kBlobAuthModes[] = { - {0, WIFI_AUTH_OPEN}, - {1, WIFI_AUTH_WEP}, - {2, WIFI_AUTH_WPA_PSK}, - {3, WIFI_AUTH_WPA2_PSK}, - {4, WIFI_AUTH_WPA_WPA2_PSK}, - {5, WIFI_AUTH_WPA2_ENTERPRISE}, - {6, WIFI_AUTH_WPA3_PSK}, - {7, WIFI_AUTH_WPA2_WPA3_PSK}, - {8, WIFI_AUTH_WAPI_PSK}, -}; -static constexpr uint8_t kBlobAuthUnknown = 9; - -static uint8_t toBlobAuthMode(wifi_auth_mode_t mode) -{ - for (const auto& [code, known] : kBlobAuthModes) { - if (known == mode) { - return code; - } - } - return kBlobAuthUnknown; -} - -static wifi_auth_mode_t fromBlobAuthMode(uint8_t code) -{ - for (const auto& [known, mode] : kBlobAuthModes) { - if (known == code) { - return mode; - } - } - return WIFI_AUTH_MAX; -} - std::vector WiFiCredentials::ToBlob() const { std::vector blob; diff --git a/components/crypto/CMakeLists.txt b/components/crypto/CMakeLists.txt index 3ca443e35..0d0afa9ea 100644 --- a/components/crypto/CMakeLists.txt +++ b/components/crypto/CMakeLists.txt @@ -8,7 +8,7 @@ idf_component_register( SRCS "src/Base64.cpp" INCLUDE_DIRS "include" - REQUIRES common # OpenShock.h (Hashing.h), TinyVec.h (Base64.h) in public headers + REQUIRES common # OpenShock.h (Hashing.h) in public headers mbedtls # mbedtls/md.h in Hashing.h public header PRIV_REQUIRES logging # Logging.h (Base64.cpp) ) diff --git a/components/crypto/host_test/main/CMakeLists.txt b/components/crypto/host_test/main/CMakeLists.txt index cab6a8bbe..da7d9f93b 100644 --- a/components/crypto/host_test/main/CMakeLists.txt +++ b/components/crypto/host_test/main/CMakeLists.txt @@ -1,6 +1,6 @@ -# Compile crypto's PURE sources directly. Base64 pulls in common headers -# (TinyVec.h -> OpenShock.h, Logging.h), included by path below; Logging.h and the -# generated openshock_board.h come from test/host_support. +# Compile crypto's PURE sources directly. Hashing.h pulls in common's OpenShock.h, +# included by path below; Logging.h and the generated openshock_board.h come from +# test/host_support. idf_component_register( SRCS "test_main.cpp" "test_base64.cpp" @@ -8,7 +8,7 @@ idf_component_register( "../../src/Base64.cpp" INCLUDE_DIRS "." "../../include" # crypto headers (Hashing.h, Base64.h) - "../../../common/include" # TinyVec.h, OpenShock.h + "../../../common/include" # OpenShock.h REQUIRES unity host_support # Logging.h, openshock_board.h (test/host_support) mbedtls # Base64.cpp -> mbedtls/base64.h diff --git a/components/crypto/host_test/main/test_base64.cpp b/components/crypto/host_test/main/test_base64.cpp index 7a667385a..022f15997 100644 --- a/components/crypto/host_test/main/test_base64.cpp +++ b/components/crypto/host_test/main/test_base64.cpp @@ -2,11 +2,11 @@ #include "unity.h" #include "Base64.h" -#include "TinyVec.h" #include #include #include +#include using namespace OpenShock; @@ -29,7 +29,7 @@ TEST_CASE("Base64 encode known vectors", "[util][base64]") TEST_CASE("Base64 decode known vector", "[util][base64]") { - TinyVec out; + std::vector out; TEST_ASSERT_TRUE(Base64::Decode("TWFu", out)); TEST_ASSERT_EQUAL_size_t(3, out.size()); TEST_ASSERT_EQUAL_UINT8('M', out[0]); @@ -39,7 +39,7 @@ TEST_CASE("Base64 decode known vector", "[util][base64]") TEST_CASE("Base64 decode rejects invalid input", "[util][base64]") { - TinyVec out; + std::vector out; TEST_ASSERT_FALSE(Base64::Decode("!!!!", out)); // invalid chars } @@ -49,7 +49,7 @@ TEST_CASE("Base64 round-trip", "[util][base64]") std::string encoded; TEST_ASSERT_TRUE(Base64::Encode(std::span(data, 5), encoded)); - TinyVec decoded; + std::vector decoded; TEST_ASSERT_TRUE(Base64::Decode(encoded, decoded)); TEST_ASSERT_EQUAL_size_t(5, decoded.size()); for (int i = 0; i < 5; ++i) { @@ -59,7 +59,7 @@ TEST_CASE("Base64 round-trip", "[util][base64]") TEST_CASE("Base64 decode sizes the output exactly, including padded input", "[util][base64]") { - TinyVec out; + std::vector out; TEST_ASSERT_TRUE(Base64::Decode("TQ==", out)); TEST_ASSERT_EQUAL_size_t(1, out.size()); @@ -74,7 +74,7 @@ TEST_CASE("Base64 decode sizes the output exactly, including padded input", "[ut TEST_CASE("Base64 failed decode leaves the output empty", "[util][base64]") { - TinyVec out; + std::vector out; TEST_ASSERT_TRUE(Base64::Decode("TWFu", out)); TEST_ASSERT_FALSE(Base64::Decode("TW!u", out)); TEST_ASSERT_EQUAL_size_t(0, out.size()); diff --git a/components/crypto/include/Base64.h b/components/crypto/include/Base64.h index d4d34f01e..d43f4f093 100644 --- a/components/crypto/include/Base64.h +++ b/components/crypto/include/Base64.h @@ -5,8 +5,7 @@ #include #include #include - -#include "TinyVec.h" +#include namespace OpenShock::Base64 { /// @brief Encodes a byte array to base64. @@ -19,5 +18,5 @@ namespace OpenShock::Base64 { /// @param data The base64 text to decode. /// @param output Receives the decoded bytes (sized to exactly what was decoded); cleared on failure. /// @return True on success, false on invalid input. - bool Decode(std::string_view data, TinyVec& output) noexcept; + bool Decode(std::string_view data, std::vector& output); } // namespace OpenShock::Base64 diff --git a/components/crypto/src/Base64.cpp b/components/crypto/src/Base64.cpp index 7d2b0448e..61871b110 100644 --- a/components/crypto/src/Base64.cpp +++ b/components/crypto/src/Base64.cpp @@ -23,7 +23,7 @@ bool Base64::Encode(std::span data, std::string& output) std::size_t requiredLen = CalculateEncodedSize(data.size()); // Allocate buffer for base64 string - TinyVec buffer(requiredLen); + std::vector buffer(requiredLen); std::size_t written = 0; @@ -42,7 +42,7 @@ bool Base64::Encode(std::span data, std::string& output) return true; } -bool Base64::Decode(std::string_view data, TinyVec& output) noexcept +bool Base64::Decode(std::string_view data, std::vector& output) { std::size_t requiredLen = CalculateDecodedSize(data.size()); diff --git a/components/device_control/CMakeLists.txt b/components/device_control/CMakeLists.txt index 19b63b478..cc8933283 100644 --- a/components/device_control/CMakeLists.txt +++ b/components/device_control/CMakeLists.txt @@ -12,7 +12,7 @@ idf_component_register( SRCS ${device_control_sources} INCLUDE_DIRS "include" REQUIRES common esp_hal_gpio esp_driver_rmt freertos # public headers (enums, gpio_types, rmt_tx/encoder, TaskHandle/QueueHandle) - config serialization # its config sections (radio/RFConfig.h, estop/EStopConfig.h): config/Section.h, HubConfig wire types + config # its config sections (radio/RFConfig.h, estop/EStopConfig.h): config/Section.h PRIV_REQUIRES chipset events led_drivers logging temporal protocols # concurrency/util folded into common (public REQUIRE) esp_driver_gpio esp_wifi esp_event esp_netif soc ) diff --git a/components/device_control/include/estop/EStopConfig.h b/components/device_control/include/estop/EStopConfig.h index 7106199a6..96101ef21 100644 --- a/components/device_control/include/estop/EStopConfig.h +++ b/components/device_control/include/estop/EStopConfig.h @@ -4,7 +4,6 @@ // NVS through the config framework (config/Section.h); NVS namespace "estop", keys enabled (u8) and gpio (i8). #include "config/Section.h" -#include "serialization/_fbs/HubConfig_generated.h" #include @@ -32,8 +31,6 @@ namespace OpenShock::Config { bool FromJSON(JSON::JsonView json); void ToJSON(json_gen_str_t* gen, const char* name, bool withSensitiveData) const; - - [[nodiscard]] flatbuffers::Offset ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const; }; [[nodiscard]] EStopConfig GetEStopConfig(); diff --git a/components/device_control/include/radio/RFConfig.h b/components/device_control/include/radio/RFConfig.h index 4610fcf65..157b6d10e 100644 --- a/components/device_control/include/radio/RFConfig.h +++ b/components/device_control/include/radio/RFConfig.h @@ -4,7 +4,6 @@ // through the config framework (config/Section.h); NVS namespace "rf", keys tx_pin (i8) and keepalive (u8). #include "config/Section.h" -#include "serialization/_fbs/HubConfig_generated.h" #include @@ -30,8 +29,6 @@ namespace OpenShock::Config { bool FromJSON(JSON::JsonView json); void ToJSON(json_gen_str_t* gen, const char* name, bool withSensitiveData) const; - - [[nodiscard]] flatbuffers::Offset ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const; }; /// @brief The section itself, for whole-value reads and read-modify-writes. diff --git a/components/device_control/src/estop/EStopConfig.cpp b/components/device_control/src/estop/EStopConfig.cpp index c77401c84..fa6bb13e6 100644 --- a/components/device_control/src/estop/EStopConfig.cpp +++ b/components/device_control/src/estop/EStopConfig.cpp @@ -13,14 +13,13 @@ const char* const TAG = "Config::EStopConfig"; #include "json/Json.h" using namespace OpenShock::Config; -namespace Fbs = OpenShock::Serialization::Configuration; static constexpr const char* kNamespace = "estop"; static constexpr const char* kKeyEnabled = "enabled"; static constexpr const char* kKeyGpio = "gpio"; static constexpr const char* kNamespaces[] = {kNamespace}; -static Section s_section({.name = "estop", .order = 6, .hubConfigField = Fbs::HubConfig::VT_ESTOP, .namespaces = kNamespaces}); +static Section s_section({.name = "estop", .order = 6, .namespaces = kNamespaces}); EStopConfig::EStopConfig() : enabled(OpenShock::IsValidInputPin(Board::Get().estopGpio)) @@ -68,7 +67,7 @@ bool EStopConfig::LoadNvs(const char* partition) bool EStopConfig::SaveNvs(const char* partition, const EStopConfig* previous) const { - auto handle = Nvs::Handle::Open(partition, kNamespace, true); + auto handle = Nvs::Handle::OpenForWrite(partition, kNamespace); if (!handle) { return false; } @@ -101,12 +100,6 @@ void EStopConfig::ToJSON(json_gen_str_t* gen, const char* name, bool withSensiti JSON::objEnd(gen, name); } -flatbuffers::Offset EStopConfig::ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const -{ - (void)withSensitiveData; - return Fbs::CreateEStopConfig(builder, enabled, gpioPin); -} - EStopConfig OpenShock::Config::GetEStopConfig() { return s_section.Get(); diff --git a/components/device_control/src/radio/RFConfig.cpp b/components/device_control/src/radio/RFConfig.cpp index 33468d5f5..b78e68900 100644 --- a/components/device_control/src/radio/RFConfig.cpp +++ b/components/device_control/src/radio/RFConfig.cpp @@ -12,14 +12,13 @@ const char* const TAG = "Config::RFConfig"; #include "json/Json.h" using namespace OpenShock::Config; -namespace Fbs = OpenShock::Serialization::Configuration; static constexpr const char* kNamespace = "rf"; static constexpr const char* kKeyTxPin = "tx_pin"; static constexpr const char* kKeyKeepAlive = "keepalive"; static constexpr const char* kNamespaces[] = {kNamespace}; -static Section s_section({.name = "rf", .order = 0, .hubConfigField = Fbs::HubConfig::VT_RF, .namespaces = kNamespaces}); +static Section s_section({.name = "rf", .order = 0, .namespaces = kNamespaces}); RFConfig::RFConfig() : txPin(static_cast(Board::Get().rfTxGpio)) @@ -56,7 +55,7 @@ bool RFConfig::LoadNvs(const char* partition) bool RFConfig::SaveNvs(const char* partition, const RFConfig* previous) const { - auto handle = Nvs::Handle::Open(partition, kNamespace, true); + auto handle = Nvs::Handle::OpenForWrite(partition, kNamespace); if (!handle) { return false; } @@ -90,17 +89,6 @@ void RFConfig::ToJSON(json_gen_str_t* gen, const char* name, bool withSensitiveD JSON::objEnd(gen, name); } -flatbuffers::Offset RFConfig::ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const -{ - (void)withSensitiveData; - // keepalive_enabled is stored even when it equals the schema default, so the value reads back the same under every - // schema version (up to 1.5.x it defaulted to false): Config::HubConfigToJSON never has to infer it. - Fbs::RFConfigBuilder rfBuilder(builder); - rfBuilder.add_tx_pin(txPin); - builder.AddElement(Fbs::RFConfig::VT_KEEPALIVE_ENABLED, static_cast(keepAliveEnabled)); - return rfBuilder.Finish(); -} - RFConfig OpenShock::Config::GetRFConfig() { return s_section.Get(); diff --git a/components/fs/CMakeLists.txt b/components/fs/CMakeLists.txt deleted file mode 100644 index 372e18884..000000000 --- a/components/fs/CMakeLists.txt +++ /dev/null @@ -1,15 +0,0 @@ -# littlefs-over-esp_partition filesystem layer (no VFS), built on the vendored littlefs -# core. Provides the shared LfsPartition mount kernel (the captive portal streams its -# pre-built image assets straight from it), the ConfigFs facade (read/write blob store -# for configuration: atomic writes, wear leveling) and FsCheck: non-destructive helpers -# to validate a partition holds a mountable image. -# -# littlefs is a PRIVATE dependency: lfs.h is confined to LfsPartition.cpp so it never -# leaks into consumers of this component's headers. -idf_component_register( - SRCS "src/LfsPartition.cpp" - "src/FsCheck.cpp" - INCLUDE_DIRS "include" - REQUIRES common esp_partition - PRIV_REQUIRES littlefs logging -) diff --git a/components/fs/host_test/CMakeLists.txt b/components/fs/host_test/CMakeLists.txt deleted file mode 100644 index 79e00ce0c..000000000 --- a/components/fs/host_test/CMakeLists.txt +++ /dev/null @@ -1,14 +0,0 @@ -cmake_minimum_required(VERSION 3.16) - -include($ENV{IDF_PATH}/tools/cmake/project.cmake) - -# Only build 'main'; it compiles the vendored littlefs core directly against a RAM block -# device (see main/CMakeLists.txt), so no firmware component needs to be registered. -set(COMPONENTS main) - -# linux-target freertos mock, so unity's IDF integration has its dependencies. -list(APPEND EXTRA_COMPONENT_DIRS - "$ENV{IDF_PATH}/tools/mocks/freertos/") - -project(fs_host_test) -include(${CMAKE_CURRENT_LIST_DIR}/../../../cmake/cxx_standard.cmake) diff --git a/components/fs/host_test/main/CMakeLists.txt b/components/fs/host_test/main/CMakeLists.txt deleted file mode 100644 index 1c7cc876b..000000000 --- a/components/fs/host_test/main/CMakeLists.txt +++ /dev/null @@ -1,14 +0,0 @@ -# Runs the vendored littlefs core over a RAM block device with the exact geometry and -# mount parameters LfsPartition uses, to check how the small config partition behaves. -idf_component_register( - SRCS "test_main.cpp" - "test_config_partition.cpp" - "../../../littlefs/src/lfs.c" - "../../../littlefs/src/lfs_util.c" - INCLUDE_DIRS "." - "../../../littlefs/include" - REQUIRES unity - WHOLE_ARCHIVE # keep the auto-registered TEST_CASEs from being stripped -) - -target_compile_definitions(${COMPONENT_LIB} PRIVATE LFS_NO_DEBUG LFS_NO_WARN LFS_NO_ERROR LFS_NO_ASSERT) diff --git a/components/fs/host_test/main/test_config_partition.cpp b/components/fs/host_test/main/test_config_partition.cpp deleted file mode 100644 index 6c0dc219e..000000000 --- a/components/fs/host_test/main/test_config_partition.cpp +++ /dev/null @@ -1,173 +0,0 @@ -// The 1.x config partition is 3 x 4 KiB (partitions/legacy/ota_4mb_v1.csv). littlefs keeps its root metadata pair in two of those -// blocks, and ConfigFs::writeAll writes ".tmp" and renames it over the old file. A file too large to be -// inlined in the metadata needs its own data block while the old copy still occupies the only free one, so the -// review suspected that larger configs can't be rewritten at all (REVIEW_ACTIONS.md, X01). -// -// These tests run the real littlefs core with LfsPartition's exact mount parameters over a RAM block device and -// replay ConfigFs's write sequence, so they answer that directly. The "large config" cases describe the behaviour -// the firmware needs. X01 is real: anything over the 512-byte inline limit saves once and then fails with -// LFS_ERR_NOSPC, so those cases are ignored as KNOWN until the config storage is fixed. Remove the -// TEST_IGNORE_MESSAGE lines to check a fix. -#include "unity.h" - -#include "lfs.h" - -#include -#include -#include -#include -#include - -namespace { - // Must match components/fs/src/LfsPartition.cpp and the config partition size. - constexpr lfs_size_t kBlockSize = 4096; - constexpr lfs_size_t kBlockCount = 0x3000 / kBlockSize; - constexpr lfs_size_t kReadSize = 128; - constexpr lfs_size_t kProgSize = 128; - constexpr lfs_size_t kCacheSize = 512; - constexpr lfs_size_t kLookahead = 128; - constexpr int32_t kBlockCyclesRw = 512; - constexpr const char* kConfigPath = "/config"; - - std::vector s_flash; - - int bdRead(const struct lfs_config* c, lfs_block_t block, lfs_off_t off, void* buffer, lfs_size_t size) - { - std::memcpy(buffer, &s_flash[block * c->block_size + off], size); - return 0; - } - int bdProg(const struct lfs_config* c, lfs_block_t block, lfs_off_t off, const void* buffer, lfs_size_t size) - { - std::memcpy(&s_flash[block * c->block_size + off], buffer, size); - return 0; - } - int bdErase(const struct lfs_config* c, lfs_block_t block) - { - std::memset(&s_flash[block * c->block_size], 0xFF, c->block_size); - return 0; - } - int bdSync(const struct lfs_config*) - { - return 0; - } - - struct Fs { - lfs_t lfs {}; - lfs_config cfg {}; - - Fs() - { - s_flash.assign(kBlockSize * kBlockCount, 0xFF); - - cfg.read = bdRead; - cfg.prog = bdProg; - cfg.erase = bdErase; - cfg.sync = bdSync; - cfg.read_size = kReadSize; - cfg.prog_size = kProgSize; - cfg.block_size = kBlockSize; - cfg.block_count = kBlockCount; - cfg.block_cycles = kBlockCyclesRw; - cfg.cache_size = kCacheSize; - cfg.lookahead_size = kLookahead; - - TEST_ASSERT_EQUAL(0, lfs_format(&lfs, &cfg)); - TEST_ASSERT_EQUAL(0, lfs_mount(&lfs, &cfg)); - } - ~Fs() { lfs_unmount(&lfs); } - - // The same sequence as LfsPartition::writeAll: write .tmp, then rename it over . - int writeAll(const std::vector& data) - { - std::string tmp = std::string(kConfigPath) + ".tmp"; - - lfs_file_t file; - int err = lfs_file_open(&lfs, &file, tmp.c_str(), LFS_O_WRONLY | LFS_O_CREAT | LFS_O_TRUNC); - if (err < 0) return err; - - lfs_ssize_t written = lfs_file_write(&lfs, &file, data.data(), data.size()); - int closeErr = lfs_file_close(&lfs, &file); - if (written < 0 || closeErr < 0 || static_cast(written) != data.size()) { - lfs_remove(&lfs, tmp.c_str()); - return written < 0 ? static_cast(written) : (closeErr < 0 ? closeErr : LFS_ERR_IO); - } - - err = lfs_rename(&lfs, tmp.c_str(), kConfigPath); - if (err < 0) lfs_remove(&lfs, tmp.c_str()); - return err; - } - - bool readBack(std::vector& out) - { - lfs_file_t file; - if (lfs_file_open(&lfs, &file, kConfigPath, LFS_O_RDONLY) < 0) return false; - lfs_soff_t size = lfs_file_size(&lfs, &file); - out.resize(size < 0 ? 0 : static_cast(size)); - lfs_ssize_t r = lfs_file_read(&lfs, &file, out.data(), out.size()); - lfs_file_close(&lfs, &file); - return r == static_cast(out.size()); - } - }; - - std::vector blob(size_t size, uint8_t seed) - { - std::vector v(size); - for (size_t i = 0; i < size; i++) v[i] = static_cast(seed + i * 31); - return v; - } - - // Writes `size` bytes `times` times (as repeated config saves would), checking each write reads back. - void assertRewritable(size_t size, int times) - { - Fs fs; - for (int i = 0; i < times; i++) { - auto data = blob(size, static_cast(i)); - char msg[96]; - int err = fs.writeAll(data); - std::snprintf(msg, sizeof(msg), "write #%d of a %zu-byte config failed (lfs error %d)", i + 1, size, err); - TEST_ASSERT_EQUAL_MESSAGE(0, err, msg); - - std::vector back; - TEST_ASSERT_TRUE(fs.readBack(back)); - TEST_ASSERT_TRUE_MESSAGE(back == data, "config read back differs from what was written"); - } - } -} // namespace - -TEST_CASE("A small (inlined) config can be rewritten repeatedly", "[fs][config]") -{ - assertRewritable(256, 50); -} - -TEST_CASE("A config just over the inline limit can be rewritten repeatedly (X01)", "[fs][config]") -{ - TEST_IGNORE_MESSAGE("KNOWN ISSUE (X01): configs over the 512-byte inline limit can't be rewritten on the 3-block partition"); - assertRewritable(600, 50); -} - -TEST_CASE("A config of a few saved networks plus a token (~1.5 KiB) can be rewritten repeatedly (X01)", "[fs][config]") -{ - TEST_IGNORE_MESSAGE("KNOWN ISSUE (X01): configs over the 512-byte inline limit can't be rewritten on the 3-block partition"); - assertRewritable(1536, 50); -} - -TEST_CASE("Reports the largest config size that survives repeated rewrites", "[fs][config][info]") -{ - // Not a pass/fail check: prints where the partition stops accepting rewrites, to size MaxConfigSize / the - // partition. Binary search over sizes, each candidate written 20 times on a fresh filesystem. - size_t lo = 0, hi = 4096; - while (lo < hi) { - size_t mid = (lo + hi + 1) / 2; - Fs fs; - bool ok = true; - for (int i = 0; i < 20 && ok; i++) { - ok = fs.writeAll(blob(mid, static_cast(i))) == 0; - } - if (ok) { - lo = mid; - } else { - hi = mid - 1; - } - } - std::printf("[fs] largest config rewritable 20 times on the 3-block partition: %zu bytes\n", lo); -} diff --git a/components/fs/host_test/main/test_main.cpp b/components/fs/host_test/main/test_main.cpp deleted file mode 100644 index 1b5161df6..000000000 --- a/components/fs/host_test/main/test_main.cpp +++ /dev/null @@ -1,10 +0,0 @@ -// Host-only (linux target) test binary for the fs component. FreeRTOS is -// mocked, so there is no ESP startup - drive Unity directly. -#include "unity.h" - -extern "C" int main(void) -{ - UNITY_BEGIN(); - unity_run_all_tests(); - return UNITY_END(); -} diff --git a/components/fs/host_test/sdkconfig.defaults b/components/fs/host_test/sdkconfig.defaults deleted file mode 100644 index c3d7cf2ce..000000000 --- a/components/fs/host_test/sdkconfig.defaults +++ /dev/null @@ -1,4 +0,0 @@ -CONFIG_IDF_TARGET="linux" -CONFIG_IDF_TARGET_LINUX=y -CONFIG_COMPILER_CXX_EXCEPTIONS=y -CONFIG_UNITY_ENABLE_IDF_TEST_RUNNER=y diff --git a/components/fs/include/fs/ConfigFs.h b/components/fs/include/fs/ConfigFs.h deleted file mode 100644 index 30e102461..000000000 --- a/components/fs/include/fs/ConfigFs.h +++ /dev/null @@ -1,39 +0,0 @@ -#pragma once - -#include "fs/LfsPartition.h" - -#include "OpenShock.h" - -#include - -#include -#include -#include - -namespace OpenShock { - // Read/write view of a littlefs partition tailored for a configuration store: whole - // small blobs, atomic replace (temp file + rename), wear leveling enabled, formats a - // fresh partition on first mount. - // - // Deals only in raw bytes; it has no knowledge of the config serialization format - // (flatbuffers). Serialization and any cross-task locking belong to the caller. - class ConfigFs { - DISABLE_COPY(ConfigFs); - DISABLE_MOVE(ConfigFs); - - public: - ConfigFs() = default; - - bool mount(const esp_partition_t* partition) { return m_fs.mount(partition, true, /*formatOnFail=*/true); } - void unmount() { m_fs.unmount(); } - bool isMounted() const { return m_fs.isMounted(); } - - bool exists(const char* path) { return m_fs.exists(path); } - bool read(const char* path, std::vector& out) { return m_fs.readAll(path, out); } - bool write(const char* path, std::span data) { return m_fs.writeAll(path, data); } - bool remove(const char* path) { return m_fs.remove(path); } - - private: - LfsPartition m_fs; - }; -} // namespace OpenShock diff --git a/components/fs/include/fs/FsCheck.h b/components/fs/include/fs/FsCheck.h deleted file mode 100644 index 3ba27801b..000000000 --- a/components/fs/include/fs/FsCheck.h +++ /dev/null @@ -1,18 +0,0 @@ -#pragma once - -#include - -namespace OpenShock { - // Verify that `partition` holds a valid, mountable littlefs image by mounting it and - // immediately unmounting. Returns false for a null partition or any mount failure. - // - // Non-destructive: `writable` is passed through to the mount (a read-only mount can - // never write), and a corrupt image is never reformatted (formatOnFail=false), so - // this stays a pure check regardless of `writable`. Use it to validate a freshly - // flashed filesystem image before committing to it. - bool TestFilesystem(const esp_partition_t* partition, bool writable = false); - - // Convenience overload: find a data partition by label (any subtype) and test it. - // Returns false if no such partition exists or it fails to mount. - bool TestFilesystem(const char* partitionLabel, bool writable = false); -} // namespace OpenShock diff --git a/components/fs/include/fs/LfsPartition.h b/components/fs/include/fs/LfsPartition.h deleted file mode 100644 index 17505c9e9..000000000 --- a/components/fs/include/fs/LfsPartition.h +++ /dev/null @@ -1,66 +0,0 @@ -#pragma once - -#include "OpenShock.h" -#include "util/FunctionRef.h" - -#include - -#include -#include -#include -#include -#include - -namespace OpenShock { - namespace detail { - struct LfsState; // defined in LfsPartition.cpp; keeps lfs.h out of public headers - } - - // Mounts a littlefs filesystem directly over a raw esp_partition using the vendored - // littlefs core, no VFS registration. This is the shared kernel behind ConfigFs - // (read/write) and the captive portal's StaticFileServer (read-only). - // - // Not thread-safe: callers that share a partition across tasks must serialize access - // themselves. - class LfsPartition { - DISABLE_COPY(LfsPartition); - DISABLE_MOVE(LfsPartition); - - public: - LfsPartition(); - ~LfsPartition(); - - // Mount `partition`. When `writable` is false, prog/erase are refused and wear - // leveling is disabled (correct for a pre-built, read-only image). When `writable` - // is true and the mount fails, the partition is formatted first if `formatOnFail`. - bool mount(const esp_partition_t* partition, bool writable, bool formatOnFail = false); - void unmount(); - bool isMounted() const { return m_mounted; } - // The partition this is mounted on, or null when not mounted. - const esp_partition_t* partition() const; - - // True if `path` names anything, file or directory. - bool exists(const char* path); - - // The size of the regular file at `path`; empty if there is none (a directory does not count). - std::optional fileSize(const char* path); - - // Stream a file to `sink` in chunks. Returns false if the file is missing/unreadable - // or if `sink` returns false (which aborts the stream). - bool readStream(const char* path, Util::FunctionRef)> sink); - - // Read a whole file into `out`. Returns false (and clears `out`) on any error. - bool readAll(const char* path, std::vector& out); - - // Atomically replace `path` with `data` (write to a temp file, then rename). - // Writable mounts only. - bool writeAll(const char* path, std::span data); - - // Delete `path`. Writable mounts only. - bool remove(const char* path); - - private: - detail::LfsState* m_state; - bool m_mounted; - }; -} // namespace OpenShock diff --git a/components/fs/src/FsCheck.cpp b/components/fs/src/FsCheck.cpp deleted file mode 100644 index 59265c58b..000000000 --- a/components/fs/src/FsCheck.cpp +++ /dev/null @@ -1,26 +0,0 @@ -#include "fs/FsCheck.h" - -#include "fs/LfsPartition.h" - -using namespace OpenShock; - -bool OpenShock::TestFilesystem(const esp_partition_t* partition, bool writable) -{ - if (partition == nullptr) { - return false; - } - - LfsPartition fs; - if (!fs.mount(partition, writable, /*formatOnFail=*/false)) { - return false; - } - - fs.unmount(); - return true; -} - -bool OpenShock::TestFilesystem(const char* partitionLabel, bool writable) -{ - const esp_partition_t* partition = esp_partition_find_first(ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_ANY, partitionLabel); - return TestFilesystem(partition, writable); -} diff --git a/components/fs/src/LfsPartition.cpp b/components/fs/src/LfsPartition.cpp deleted file mode 100644 index 49d529fd6..000000000 --- a/components/fs/src/LfsPartition.cpp +++ /dev/null @@ -1,287 +0,0 @@ -#include "fs/LfsPartition.h" - -const char* const TAG = "LfsPartition"; - -#include "Logging.h" - -#include "lfs.h" - -#include -#include -#include -#include - -using namespace OpenShock; - -// Mount parameters must be compatible with how the image is built (see the -// CONFIG_LITTLEFS_* defaults / mklittlefs): 4 KB erase blocks, 128-byte read/prog. -// littlefs only fixes block_size + block_count on disk (validated against the -// superblock at mount); the rest are runtime buffer sizes. -static constexpr lfs_size_t LFS_BLOCK_SIZE = 4096; -static constexpr lfs_size_t LFS_READ_SIZE = 128; -static constexpr lfs_size_t LFS_PROG_SIZE = 128; -static constexpr lfs_size_t LFS_CACHE_SIZE = 512; -static constexpr lfs_size_t LFS_LOOKAHEAD_SIZE = 128; -static constexpr int32_t LFS_BLOCK_CYCLES_RW = 512; // wear leveling for read/write mounts - -// All littlefs state lives here so lfs.h stays out of the public headers. The block -// device callbacks recover it via lfs_config::context. -struct OpenShock::detail::LfsState { - lfs_t lfs; - struct lfs_config cfg; - const esp_partition_t* partition; - bool writable; -}; - -using OpenShock::detail::LfsState; - -static int bdRead(const struct lfs_config* c, lfs_block_t block, lfs_off_t off, void* buffer, lfs_size_t size) -{ - auto* st = static_cast(c->context); - size_t addr = static_cast(block) * c->block_size + off; - return esp_partition_read(st->partition, addr, buffer, size) == ESP_OK ? 0 : LFS_ERR_IO; -} - -static int bdProg(const struct lfs_config* c, lfs_block_t block, lfs_off_t off, const void* buffer, lfs_size_t size) -{ - auto* st = static_cast(c->context); - if (!st->writable) { - return LFS_ERR_IO; - } - size_t addr = static_cast(block) * c->block_size + off; - return esp_partition_write(st->partition, addr, buffer, size) == ESP_OK ? 0 : LFS_ERR_IO; -} - -static int bdErase(const struct lfs_config* c, lfs_block_t block) -{ - auto* st = static_cast(c->context); - if (!st->writable) { - return LFS_ERR_IO; - } - size_t addr = static_cast(block) * c->block_size; - return esp_partition_erase_range(st->partition, addr, c->block_size) == ESP_OK ? 0 : LFS_ERR_IO; -} - -static int bdSync(const struct lfs_config*) -{ - return 0; // esp_partition writes are synchronous -} - -LfsPartition::LfsPartition() - : m_state(new LfsState {}) - , m_mounted(false) -{ -} - -LfsPartition::~LfsPartition() -{ - unmount(); - delete m_state; -} - -bool LfsPartition::mount(const esp_partition_t* partition, bool writable, bool formatOnFail) -{ - if (m_mounted) { - return true; - } - if (partition == nullptr) { - return false; - } - - m_state->partition = partition; - m_state->writable = writable; - - struct lfs_config& cfg = m_state->cfg; - memset(&cfg, 0, sizeof(cfg)); - cfg.context = m_state; - cfg.read = &bdRead; - cfg.prog = &bdProg; - cfg.erase = &bdErase; - cfg.sync = &bdSync; - cfg.read_size = LFS_READ_SIZE; - cfg.prog_size = LFS_PROG_SIZE; - cfg.block_size = LFS_BLOCK_SIZE; - cfg.block_count = partition->size / LFS_BLOCK_SIZE; - cfg.block_cycles = writable ? LFS_BLOCK_CYCLES_RW : -1; // -1 disables wear leveling - cfg.cache_size = LFS_CACHE_SIZE; - cfg.lookahead_size = LFS_LOOKAHEAD_SIZE; - - int err = lfs_mount(&m_state->lfs, &cfg); - if (err < 0) { - if (writable && formatOnFail) { - OS_LOGW(TAG, "Mount failed (lfs error %d), formatting partition", err); - if (lfs_format(&m_state->lfs, &cfg) < 0 || lfs_mount(&m_state->lfs, &cfg) < 0) { - OS_LOGE(TAG, "Failed to format and mount partition"); - m_state->partition = nullptr; - return false; - } - } else { - OS_LOGE(TAG, "Failed to mount partition (lfs error %d)", err); - m_state->partition = nullptr; - return false; - } - } - - m_mounted = true; - return true; -} - -void LfsPartition::unmount() -{ - if (m_mounted) { - lfs_unmount(&m_state->lfs); - m_mounted = false; - } - m_state->partition = nullptr; -} - -const esp_partition_t* LfsPartition::partition() const -{ - return m_mounted ? m_state->partition : nullptr; -} - -bool LfsPartition::exists(const char* path) -{ - if (!m_mounted) { - return false; - } - struct lfs_info info; - return lfs_stat(&m_state->lfs, path, &info) >= 0; -} - -std::optional LfsPartition::fileSize(const char* path) -{ - if (!m_mounted) { - return std::nullopt; - } - struct lfs_info info; - if (lfs_stat(&m_state->lfs, path, &info) < 0 || info.type != LFS_TYPE_REG) { - return std::nullopt; - } - return info.size; -} - -bool LfsPartition::readStream(const char* path, Util::FunctionRef)> sink) -{ - if (!m_mounted) { - return false; - } - - lfs_file_t file; - if (lfs_file_open(&m_state->lfs, &file, path, LFS_O_RDONLY) < 0) { - return false; - } - - // Larger chunks mean far fewer downstream writes (an httpd chunk costs several socket sends and an event post); - // heap-allocated so callers on small task stacks are not affected. - constexpr std::size_t kChunkSize = 4096; - std::unique_ptr buf(new (std::nothrow) uint8_t[kChunkSize]); - if (buf == nullptr) { - lfs_file_close(&m_state->lfs, &file); - return false; - } - - bool ok = true; - for (;;) { - lfs_ssize_t r = lfs_file_read(&m_state->lfs, &file, buf.get(), kChunkSize); - if (r < 0) { - ok = false; - break; - } - if (r == 0) { - break; // EOF - } - if (!sink(std::span(buf.get(), static_cast(r)))) { - ok = false; - break; - } - } - - lfs_file_close(&m_state->lfs, &file); - return ok; -} - -bool LfsPartition::readAll(const char* path, std::vector& out) -{ - out.clear(); - if (!m_mounted) { - return false; - } - - lfs_file_t file; - if (lfs_file_open(&m_state->lfs, &file, path, LFS_O_RDONLY) < 0) { - return false; - } - - bool ok = true; - lfs_soff_t size = lfs_file_size(&m_state->lfs, &file); - if (size < 0) { - ok = false; - } else if (size > 0) { - out.resize(static_cast(size)); - lfs_ssize_t r = lfs_file_read(&m_state->lfs, &file, out.data(), static_cast(size)); - if (r < 0 || static_cast(r) != static_cast(size)) { - ok = false; - } - } - - lfs_file_close(&m_state->lfs, &file); - if (!ok) { - out.clear(); - } - return ok; -} - -bool LfsPartition::writeAll(const char* path, std::span data) -{ - if (!m_mounted || !m_state->writable) { - OS_LOGE(TAG, "Can't write %s: partition %s", path, !m_mounted ? "not mounted" : "read-only"); - return false; - } - - // Write to a temp file then rename, so a power loss can't leave a half-written file. - std::string tmp = std::string(path) + ".tmp"; - - lfs_file_t file; - int err = lfs_file_open(&m_state->lfs, &file, tmp.c_str(), LFS_O_WRONLY | LFS_O_CREAT | LFS_O_TRUNC); - if (err < 0) { - OS_LOGE(TAG, "Failed to open %s for writing (lfs error %d)", tmp.c_str(), err); - return false; - } - - bool ok = true; - if (!data.empty()) { - lfs_ssize_t w = lfs_file_write(&m_state->lfs, &file, data.data(), data.size()); - if (w < 0 || static_cast(w) != data.size()) { - OS_LOGE(TAG, "Failed to write %zu bytes to %s (lfs result %ld)", data.size(), tmp.c_str(), static_cast(w)); - ok = false; - } - } - err = lfs_file_close(&m_state->lfs, &file); - if (err < 0) { - OS_LOGE(TAG, "Failed to close %s (lfs error %d)", tmp.c_str(), err); - ok = false; - } - - if (!ok) { - lfs_remove(&m_state->lfs, tmp.c_str()); - return false; - } - - err = lfs_rename(&m_state->lfs, tmp.c_str(), path); - if (err < 0) { - OS_LOGE(TAG, "Failed to rename %s to %s (lfs error %d)", tmp.c_str(), path, err); - lfs_remove(&m_state->lfs, tmp.c_str()); - return false; - } - - return true; -} - -bool LfsPartition::remove(const char* path) -{ - if (!m_mounted || !m_state->writable) { - return false; - } - return lfs_remove(&m_state->lfs, path) >= 0; -} diff --git a/components/hwutil/CMakeLists.txt b/components/hwutil/CMakeLists.txt deleted file mode 100644 index 136a5cccf..000000000 --- a/components/hwutil/CMakeLists.txt +++ /dev/null @@ -1,17 +0,0 @@ -# Hardware-dependent utilities split out of `util` so that `util` stays -# pure std-C++ and host-testable. These pull in firmware (esp_partition, HTTP -# OTA) types that do not exist on the linux host target. -# -# Public surface: -# include/hwutil/PartitionUtils.h: hash / flash an esp_partition from a URL - -idf_component_register( - SRCS "src/PartitionUtils.cpp" - INCLUDE_DIRS "include" - REQUIRES esp_partition # esp_partition_t in public header - common # util/FunctionRef.h in public header; Convert, StringHelpers, HexUtils - PRIV_REQUIRES crypto # Hashing - temporal # Temporal.h - http # HTTPRequestManager (URL flashing) - logging # Logging.h -) diff --git a/components/led_drivers/CMakeLists.txt b/components/led_drivers/CMakeLists.txt index 4c3acddae..4206ee587 100644 --- a/components/led_drivers/CMakeLists.txt +++ b/components/led_drivers/CMakeLists.txt @@ -6,7 +6,7 @@ # led_drivers/RgbLedDriver.h # # Note: this is driver-level only. The VisualStateManager that picks patterns -# based on estop/gateway/wifi state is a separate concern and lives in main. +# based on estop/gateway/wifi state is a separate concern and lives in device_control. idf_component_register( SRCS "src/MonoLedDriver.cpp" diff --git a/components/littlefs/CMakeLists.txt b/components/littlefs/CMakeLists.txt deleted file mode 100644 index 0210da35a..000000000 --- a/components/littlefs/CMakeLists.txt +++ /dev/null @@ -1,17 +0,0 @@ -# Vendored littlefs core (upstream ARM littlefs, BSD-3-Clause). This is the plain -# filesystem library only, no VFS layer. OpenShock mounts partitions against it -# directly via an esp_partition-backed block device (see the fs component's -# LfsPartition). Sources copied from github.com/littlefs-project/littlefs. -idf_component_register( - SRCS "src/lfs.c" - "src/lfs_util.c" - INCLUDE_DIRS "include" -) - -# Quiet the library: it otherwise emits printf-based trace/debug/warn/error output. -target_compile_definitions(${COMPONENT_LIB} PUBLIC - LFS_NO_DEBUG - LFS_NO_WARN - LFS_NO_ERROR - LFS_NO_ASSERT -) diff --git a/components/littlefs/LICENSE.md b/components/littlefs/LICENSE.md deleted file mode 100644 index a4c01b288..000000000 --- a/components/littlefs/LICENSE.md +++ /dev/null @@ -1,25 +0,0 @@ -Copyright (c) 2022, The littlefs authors. -Copyright (c) 2017, Arm Limited. All rights reserved. - -Redistribution and use in source and binary forms, with or without modification, -are permitted provided that the following conditions are met: - -- Redistributions of source code must retain the above copyright notice, this - list of conditions and the following disclaimer. -- Redistributions in binary form must reproduce the above copyright notice, this - list of conditions and the following disclaimer in the documentation and/or - other materials provided with the distribution. -- Neither the name of ARM nor the names of its contributors may be used to - endorse or promote products derived from this software without specific prior - written permission. - -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND -ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR -ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES -(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; -LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON -ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS -SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. \ No newline at end of file diff --git a/components/littlefs/include/lfs.h b/components/littlefs/include/lfs.h deleted file mode 100644 index 8fbd5b479..000000000 --- a/components/littlefs/include/lfs.h +++ /dev/null @@ -1,802 +0,0 @@ -/* - * The little filesystem - * - * Copyright (c) 2022, The littlefs authors. - * Copyright (c) 2017, Arm Limited. All rights reserved. - * SPDX-License-Identifier: BSD-3-Clause - */ -#ifndef LFS_H -#define LFS_H - -#include "lfs_util.h" - -#ifdef __cplusplus -extern "C" -{ -#endif - - -/// Version info /// - -// Software library version -// Major (top-nibble), incremented on backwards incompatible changes -// Minor (bottom-nibble), incremented on feature additions -#define LFS_VERSION 0x0002000b -#define LFS_VERSION_MAJOR (0xffff & (LFS_VERSION >> 16)) -#define LFS_VERSION_MINOR (0xffff & (LFS_VERSION >> 0)) - -// Version of On-disk data structures -// Major (top-nibble), incremented on backwards incompatible changes -// Minor (bottom-nibble), incremented on feature additions -#define LFS_DISK_VERSION 0x00020001 -#define LFS_DISK_VERSION_MAJOR (0xffff & (LFS_DISK_VERSION >> 16)) -#define LFS_DISK_VERSION_MINOR (0xffff & (LFS_DISK_VERSION >> 0)) - - -/// Definitions /// - -// Type definitions -typedef uint32_t lfs_size_t; -typedef uint32_t lfs_off_t; - -typedef int32_t lfs_ssize_t; -typedef int32_t lfs_soff_t; - -typedef uint32_t lfs_block_t; - -// Maximum name size in bytes, may be redefined to reduce the size of the -// info struct. Limited to <= 1022. Stored in superblock and must be -// respected by other littlefs drivers. -#ifndef LFS_NAME_MAX -#define LFS_NAME_MAX 255 -#endif - -// Maximum size of a file in bytes, may be redefined to limit to support other -// drivers. Limited on disk to <= 2147483647. Stored in superblock and must be -// respected by other littlefs drivers. -#ifndef LFS_FILE_MAX -#define LFS_FILE_MAX 2147483647 -#endif - -// Maximum size of custom attributes in bytes, may be redefined, but there is -// no real benefit to using a smaller LFS_ATTR_MAX. Limited to <= 1022. Stored -// in superblock and must be respected by other littlefs drivers. -#ifndef LFS_ATTR_MAX -#define LFS_ATTR_MAX 1022 -#endif - -// Possible error codes, these are negative to allow -// valid positive return values -enum lfs_error { - LFS_ERR_OK = 0, // No error - LFS_ERR_IO = -5, // Error during device operation - LFS_ERR_CORRUPT = -84, // Corrupted - LFS_ERR_NOENT = -2, // No directory entry - LFS_ERR_EXIST = -17, // Entry already exists - LFS_ERR_NOTDIR = -20, // Entry is not a dir - LFS_ERR_ISDIR = -21, // Entry is a dir - LFS_ERR_NOTEMPTY = -39, // Dir is not empty - LFS_ERR_BADF = -9, // Bad file number - LFS_ERR_FBIG = -27, // File too large - LFS_ERR_INVAL = -22, // Invalid parameter - LFS_ERR_NOSPC = -28, // No space left on device - LFS_ERR_NOMEM = -12, // No more memory available - LFS_ERR_NOATTR = -61, // No data/attr available - LFS_ERR_NAMETOOLONG = -36, // File name too long -}; - -// File types -enum lfs_type { - // file types - LFS_TYPE_REG = 0x001, - LFS_TYPE_DIR = 0x002, - - // internally used types - LFS_TYPE_SPLICE = 0x400, - LFS_TYPE_NAME = 0x000, - LFS_TYPE_STRUCT = 0x200, - LFS_TYPE_USERATTR = 0x300, - LFS_TYPE_FROM = 0x100, - LFS_TYPE_TAIL = 0x600, - LFS_TYPE_GLOBALS = 0x700, - LFS_TYPE_CRC = 0x500, - - // internally used type specializations - LFS_TYPE_CREATE = 0x401, - LFS_TYPE_DELETE = 0x4ff, - LFS_TYPE_SUPERBLOCK = 0x0ff, - LFS_TYPE_DIRSTRUCT = 0x200, - LFS_TYPE_CTZSTRUCT = 0x202, - LFS_TYPE_INLINESTRUCT = 0x201, - LFS_TYPE_SOFTTAIL = 0x600, - LFS_TYPE_HARDTAIL = 0x601, - LFS_TYPE_MOVESTATE = 0x7ff, - LFS_TYPE_CCRC = 0x500, - LFS_TYPE_FCRC = 0x5ff, - - // internal chip sources - LFS_FROM_NOOP = 0x000, - LFS_FROM_MOVE = 0x101, - LFS_FROM_USERATTRS = 0x102, -}; - -// File open flags -enum lfs_open_flags { - // open flags - LFS_O_RDONLY = 1, // Open a file as read only -#ifndef LFS_READONLY - LFS_O_WRONLY = 2, // Open a file as write only - LFS_O_RDWR = 3, // Open a file as read and write - LFS_O_CREAT = 0x0100, // Create a file if it does not exist - LFS_O_EXCL = 0x0200, // Fail if a file already exists - LFS_O_TRUNC = 0x0400, // Truncate the existing file to zero size - LFS_O_APPEND = 0x0800, // Move to end of file on every write -#endif - - // internally used flags -#ifndef LFS_READONLY - LFS_F_DIRTY = 0x00010000, // File does not match storage due to write - LFS_F_DUSTY = 0x00020000, // File does not match storage due to desync - LFS_F_WRITING = 0x00040000, // File has been written since last flush -#endif - LFS_F_READING = 0x00080000, // File has been read since last flush -#ifndef LFS_READONLY - LFS_F_ERRED = 0x00100000, // An error occurred during write -#endif - LFS_F_INLINE = 0x01000000, // Currently inlined in directory entry -}; - -// File seek flags -enum lfs_whence_flags { - LFS_SEEK_SET = 0, // Seek relative to an absolute position - LFS_SEEK_CUR = 1, // Seek relative to the current file position - LFS_SEEK_END = 2, // Seek relative to the end of the file -}; - - -// Configuration provided during initialization of the littlefs -struct lfs_config { - // Opaque user provided context that can be used to pass - // information to the block device operations - void *context; - - // Read a region in a block. Negative error codes are propagated - // to the user. - int (*read)(const struct lfs_config *c, lfs_block_t block, - lfs_off_t off, void *buffer, lfs_size_t size); - - // Program a region in a block. The block must have previously - // been erased. Negative error codes are propagated to the user. - // May return LFS_ERR_CORRUPT if the block should be considered bad. - int (*prog)(const struct lfs_config *c, lfs_block_t block, - lfs_off_t off, const void *buffer, lfs_size_t size); - - // Erase a block. A block must be erased before being programmed. - // The state of an erased block is undefined. Negative error codes - // are propagated to the user. - // May return LFS_ERR_CORRUPT if the block should be considered bad. - int (*erase)(const struct lfs_config *c, lfs_block_t block); - - // Sync the state of the underlying block device. Negative error codes - // are propagated to the user. - int (*sync)(const struct lfs_config *c); - -#ifdef LFS_THREADSAFE - // Lock the underlying block device. Negative error codes - // are propagated to the user. - int (*lock)(const struct lfs_config *c); - - // Unlock the underlying block device. Negative error codes - // are propagated to the user. - int (*unlock)(const struct lfs_config *c); -#endif - - // Minimum size of a block read in bytes. All read operations will be a - // multiple of this value. - lfs_size_t read_size; - - // Minimum size of a block program in bytes. All program operations will be - // a multiple of this value. - lfs_size_t prog_size; - - // Size of an erasable block in bytes. This does not impact ram consumption - // and may be larger than the physical erase size. However, non-inlined - // files take up at minimum one block. Must be a multiple of the read and - // program sizes. - lfs_size_t block_size; - - // Number of erasable blocks on the device. Defaults to block_count stored - // on disk when zero. - lfs_size_t block_count; - - // Number of erase cycles before littlefs evicts metadata logs and moves - // the metadata to another block. Suggested values are in the - // range 100-1000, with large values having better performance at the cost - // of less consistent wear distribution. - // - // Set to -1 to disable block-level wear-leveling. - int32_t block_cycles; - - // Size of block caches in bytes. Each cache buffers a portion of a block in - // RAM. The littlefs needs a read cache, a program cache, and one additional - // cache per file. Larger caches can improve performance by storing more - // data and reducing the number of disk accesses. Must be a multiple of the - // read and program sizes, and a factor of the block size. - lfs_size_t cache_size; - - // Size of the lookahead buffer in bytes. A larger lookahead buffer - // increases the number of blocks found during an allocation pass. The - // lookahead buffer is stored as a compact bitmap, so each byte of RAM - // can track 8 blocks. - lfs_size_t lookahead_size; - - // Threshold for metadata compaction during lfs_fs_gc in bytes. Metadata - // pairs that exceed this threshold will be compacted during lfs_fs_gc. - // Defaults to ~88% block_size when zero, though the default may change - // in the future. - // - // Note this only affects lfs_fs_gc. Normal compactions still only occur - // when full. - // - // Set to -1 to disable metadata compaction during lfs_fs_gc. - lfs_size_t compact_thresh; - - // Optional statically allocated read buffer. Must be cache_size. - // By default lfs_malloc is used to allocate this buffer. - void *read_buffer; - - // Optional statically allocated program buffer. Must be cache_size. - // By default lfs_malloc is used to allocate this buffer. - void *prog_buffer; - - // Optional statically allocated lookahead buffer. Must be lookahead_size. - // By default lfs_malloc is used to allocate this buffer. - void *lookahead_buffer; - - // Optional upper limit on length of file names in bytes. No downside for - // larger names except the size of the info struct which is controlled by - // the LFS_NAME_MAX define. Defaults to LFS_NAME_MAX or name_max stored on - // disk when zero. - lfs_size_t name_max; - - // Optional upper limit on files in bytes. No downside for larger files - // but must be <= LFS_FILE_MAX. Defaults to LFS_FILE_MAX or file_max stored - // on disk when zero. - lfs_size_t file_max; - - // Optional upper limit on custom attributes in bytes. No downside for - // larger attributes size but must be <= LFS_ATTR_MAX. Defaults to - // LFS_ATTR_MAX or attr_max stored on disk when zero. - lfs_size_t attr_max; - - // Optional upper limit on total space given to metadata pairs in bytes. On - // devices with large blocks (e.g. 128kB) setting this to a low size (2-8kB) - // can help bound the metadata compaction time. Must be <= block_size. - // Defaults to block_size when zero. - lfs_size_t metadata_max; - - // Optional upper limit on inlined files in bytes. Inlined files live in - // metadata and decrease storage requirements, but may be limited to - // improve metadata-related performance. Must be <= cache_size, <= - // attr_max, and <= block_size/8. Defaults to the largest possible - // inline_max when zero. - // - // Set to -1 to disable inlined files. - lfs_size_t inline_max; - -#ifdef LFS_MULTIVERSION - // On-disk version to use when writing in the form of 16-bit major version - // + 16-bit minor version. This limiting metadata to what is supported by - // older minor versions. Note that some features will be lost. Defaults to - // to the most recent minor version when zero. - uint32_t disk_version; -#endif -}; - -// File info structure -struct lfs_info { - // Type of the file, either LFS_TYPE_REG or LFS_TYPE_DIR - uint8_t type; - - // Size of the file, only valid for REG files. Limited to 32-bits. - lfs_size_t size; - - // Name of the file stored as a null-terminated string. Limited to - // LFS_NAME_MAX+1, which can be changed by redefining LFS_NAME_MAX to - // reduce RAM. LFS_NAME_MAX is stored in superblock and must be - // respected by other littlefs drivers. - char name[LFS_NAME_MAX+1]; -}; - -// Filesystem info structure -struct lfs_fsinfo { - // On-disk version. - uint32_t disk_version; - - // Size of a logical block in bytes. - lfs_size_t block_size; - - // Number of logical blocks in filesystem. - lfs_size_t block_count; - - // Upper limit on the length of file names in bytes. - lfs_size_t name_max; - - // Upper limit on the size of files in bytes. - lfs_size_t file_max; - - // Upper limit on the size of custom attributes in bytes. - lfs_size_t attr_max; -}; - -// Custom attribute structure, used to describe custom attributes -// committed atomically during file writes. -struct lfs_attr { - // 8-bit type of attribute, provided by user and used to - // identify the attribute - uint8_t type; - - // Pointer to buffer containing the attribute - void *buffer; - - // Size of attribute in bytes, limited to LFS_ATTR_MAX - lfs_size_t size; -}; - -// Optional configuration provided during lfs_file_opencfg -struct lfs_file_config { - // Optional statically allocated file buffer. Must be cache_size. - // By default lfs_malloc is used to allocate this buffer. - void *buffer; - - // Optional list of custom attributes related to the file. If the file - // is opened with read access, these attributes will be read from disk - // during the open call. If the file is opened with write access, the - // attributes will be written to disk every file sync or close. This - // write occurs atomically with update to the file's contents. - // - // Custom attributes are uniquely identified by an 8-bit type and limited - // to LFS_ATTR_MAX bytes. When read, if the stored attribute is smaller - // than the buffer, it will be padded with zeros. If the stored attribute - // is larger, then it will be silently truncated. If the attribute is not - // found, it will be created implicitly. - struct lfs_attr *attrs; - - // Number of custom attributes in the list - lfs_size_t attr_count; -}; - - -/// internal littlefs data structures /// -typedef struct lfs_cache { - lfs_block_t block; - lfs_off_t off; - lfs_size_t size; - uint8_t *buffer; -} lfs_cache_t; - -typedef struct lfs_mdir { - lfs_block_t pair[2]; - uint32_t rev; - lfs_off_t off; - uint32_t etag; - uint16_t count; - bool erased; - bool split; - lfs_block_t tail[2]; -} lfs_mdir_t; - -// littlefs directory type -typedef struct lfs_dir { - struct lfs_dir *next; - uint16_t id; - uint8_t type; - lfs_mdir_t m; - - lfs_off_t pos; - lfs_block_t head[2]; -} lfs_dir_t; - -// littlefs file type -typedef struct lfs_file { - struct lfs_file *next; - uint16_t id; - uint8_t type; - lfs_mdir_t m; - - struct lfs_ctz { - lfs_block_t head; - lfs_size_t size; - } ctz; - - uint32_t flags; - lfs_off_t pos; - lfs_block_t block; - lfs_off_t off; - lfs_cache_t cache; - - const struct lfs_file_config *cfg; -} lfs_file_t; - -typedef struct lfs_superblock { - uint32_t version; - lfs_size_t block_size; - lfs_size_t block_count; - lfs_size_t name_max; - lfs_size_t file_max; - lfs_size_t attr_max; -} lfs_superblock_t; - -typedef struct lfs_gstate { - uint32_t tag; - lfs_block_t pair[2]; -} lfs_gstate_t; - -// The littlefs filesystem type -typedef struct lfs { - lfs_cache_t rcache; - lfs_cache_t pcache; - - lfs_block_t root[2]; - struct lfs_mlist { - struct lfs_mlist *next; - uint16_t id; - uint8_t type; - lfs_mdir_t m; - } *mlist; - uint32_t seed; - - lfs_gstate_t gstate; - lfs_gstate_t gdisk; - lfs_gstate_t gdelta; - - struct lfs_lookahead { - lfs_block_t start; - lfs_block_t size; - lfs_block_t next; - lfs_block_t ckpoint; - uint8_t *buffer; - } lookahead; - - const struct lfs_config *cfg; - lfs_size_t block_count; - lfs_size_t name_max; - lfs_size_t file_max; - lfs_size_t attr_max; - lfs_size_t inline_max; - -#ifdef LFS_MIGRATE - struct lfs1 *lfs1; -#endif -} lfs_t; - - -/// Filesystem functions /// - -#ifndef LFS_READONLY -// Format a block device with the littlefs -// -// Requires a littlefs object and config struct. This clobbers the littlefs -// object, and does not leave the filesystem mounted. The config struct must -// be zeroed for defaults and backwards compatibility. -// -// Returns a negative error code on failure. -int lfs_format(lfs_t *lfs, const struct lfs_config *config); -#endif - -// Mounts a littlefs -// -// Requires a littlefs object and config struct. Multiple filesystems -// may be mounted simultaneously with multiple littlefs objects. Both -// lfs and config must be allocated while mounted. The config struct must -// be zeroed for defaults and backwards compatibility. -// -// Returns a negative error code on failure. -int lfs_mount(lfs_t *lfs, const struct lfs_config *config); - -// Unmounts a littlefs -// -// Does nothing besides releasing any allocated resources. -// Returns a negative error code on failure. -int lfs_unmount(lfs_t *lfs); - -/// General operations /// - -#ifndef LFS_READONLY -// Removes a file or directory -// -// If removing a directory, the directory must be empty. -// Returns a negative error code on failure. -int lfs_remove(lfs_t *lfs, const char *path); -#endif - -#ifndef LFS_READONLY -// Rename or move a file or directory -// -// If the destination exists, it must match the source in type. -// If the destination is a directory, the directory must be empty. -// -// Returns a negative error code on failure. -int lfs_rename(lfs_t *lfs, const char *oldpath, const char *newpath); -#endif - -// Find info about a file or directory -// -// Fills out the info structure, based on the specified file or directory. -// Returns a negative error code on failure. -int lfs_stat(lfs_t *lfs, const char *path, struct lfs_info *info); - -// Get a custom attribute -// -// Custom attributes are uniquely identified by an 8-bit type and limited -// to LFS_ATTR_MAX bytes. When read, if the stored attribute is smaller than -// the buffer, it will be padded with zeros. If the stored attribute is larger, -// then it will be silently truncated. If no attribute is found, the error -// LFS_ERR_NOATTR is returned and the buffer is filled with zeros. -// -// Returns the size of the attribute, or a negative error code on failure. -// Note, the returned size is the size of the attribute on disk, irrespective -// of the size of the buffer. This can be used to dynamically allocate a buffer -// or check for existence. -lfs_ssize_t lfs_getattr(lfs_t *lfs, const char *path, - uint8_t type, void *buffer, lfs_size_t size); - -#ifndef LFS_READONLY -// Set custom attributes -// -// Custom attributes are uniquely identified by an 8-bit type and limited -// to LFS_ATTR_MAX bytes. If an attribute is not found, it will be -// implicitly created. -// -// Returns a negative error code on failure. -int lfs_setattr(lfs_t *lfs, const char *path, - uint8_t type, const void *buffer, lfs_size_t size); -#endif - -#ifndef LFS_READONLY -// Removes a custom attribute -// -// If an attribute is not found, nothing happens. -// -// Returns a negative error code on failure. -int lfs_removeattr(lfs_t *lfs, const char *path, uint8_t type); -#endif - - -/// File operations /// - -#ifndef LFS_NO_MALLOC -// Open a file -// -// The mode that the file is opened in is determined by the flags, which -// are values from the enum lfs_open_flags that are bitwise-ored together. -// -// Returns a negative error code on failure. -int lfs_file_open(lfs_t *lfs, lfs_file_t *file, - const char *path, int flags); - -// if LFS_NO_MALLOC is defined, lfs_file_open() will fail with LFS_ERR_NOMEM -// thus use lfs_file_opencfg() with config.buffer set. -#endif - -// Open a file with extra configuration -// -// The mode that the file is opened in is determined by the flags, which -// are values from the enum lfs_open_flags that are bitwise-ored together. -// -// The config struct provides additional config options per file as described -// above. The config struct must remain allocated while the file is open, and -// the config struct must be zeroed for defaults and backwards compatibility. -// -// Returns a negative error code on failure. -int lfs_file_opencfg(lfs_t *lfs, lfs_file_t *file, - const char *path, int flags, - const struct lfs_file_config *config); - -// Close a file -// -// Any pending writes are written out to storage as though -// sync had been called and releases any allocated resources. -// -// Returns a negative error code on failure. -int lfs_file_close(lfs_t *lfs, lfs_file_t *file); - -// Synchronize a file on storage -// -// Any pending writes are written out to storage. -// Returns a negative error code on failure. -int lfs_file_sync(lfs_t *lfs, lfs_file_t *file); - -// Read data from file -// -// Takes a buffer and size indicating where to store the read data. -// Returns the number of bytes read, or a negative error code on failure. -lfs_ssize_t lfs_file_read(lfs_t *lfs, lfs_file_t *file, - void *buffer, lfs_size_t size); - -#ifndef LFS_READONLY -// Write data to file -// -// Takes a buffer and size indicating the data to write. The file will not -// actually be updated on the storage until either sync or close is called. -// -// Returns the number of bytes written, or a negative error code on failure. -lfs_ssize_t lfs_file_write(lfs_t *lfs, lfs_file_t *file, - const void *buffer, lfs_size_t size); -#endif - -// Change the position of the file -// -// The change in position is determined by the offset and whence flag. -// Returns the new position of the file, or a negative error code on failure. -lfs_soff_t lfs_file_seek(lfs_t *lfs, lfs_file_t *file, - lfs_soff_t off, int whence); - -#ifndef LFS_READONLY -// Truncates the size of the file to the specified size -// -// Returns a negative error code on failure. -int lfs_file_truncate(lfs_t *lfs, lfs_file_t *file, lfs_off_t size); -#endif - -// Return the position of the file -// -// Equivalent to lfs_file_seek(lfs, file, 0, LFS_SEEK_CUR) -// Returns the position of the file, or a negative error code on failure. -lfs_soff_t lfs_file_tell(lfs_t *lfs, lfs_file_t *file); - -// Change the position of the file to the beginning of the file -// -// Equivalent to lfs_file_seek(lfs, file, 0, LFS_SEEK_SET) -// Returns a negative error code on failure. -int lfs_file_rewind(lfs_t *lfs, lfs_file_t *file); - -// Return the size of the file -// -// Similar to lfs_file_seek(lfs, file, 0, LFS_SEEK_END) -// Returns the size of the file, or a negative error code on failure. -lfs_soff_t lfs_file_size(lfs_t *lfs, lfs_file_t *file); - - -/// Directory operations /// - -#ifndef LFS_READONLY -// Create a directory -// -// Returns a negative error code on failure. -int lfs_mkdir(lfs_t *lfs, const char *path); -#endif - -// Open a directory -// -// Once open a directory can be used with read to iterate over files. -// Returns a negative error code on failure. -int lfs_dir_open(lfs_t *lfs, lfs_dir_t *dir, const char *path); - -// Close a directory -// -// Releases any allocated resources. -// Returns a negative error code on failure. -int lfs_dir_close(lfs_t *lfs, lfs_dir_t *dir); - -// Read an entry in the directory -// -// Fills out the info structure, based on the specified file or directory. -// Returns a positive value on success, 0 at the end of directory, -// or a negative error code on failure. -int lfs_dir_read(lfs_t *lfs, lfs_dir_t *dir, struct lfs_info *info); - -// Change the position of the directory -// -// The new off must be a value previous returned from tell and specifies -// an absolute offset in the directory seek. -// -// Returns a negative error code on failure. -int lfs_dir_seek(lfs_t *lfs, lfs_dir_t *dir, lfs_off_t off); - -// Return the position of the directory -// -// The returned offset is only meant to be consumed by seek and may not make -// sense, but does indicate the current position in the directory iteration. -// -// Returns the position of the directory, or a negative error code on failure. -lfs_soff_t lfs_dir_tell(lfs_t *lfs, lfs_dir_t *dir); - -// Change the position of the directory to the beginning of the directory -// -// Returns a negative error code on failure. -int lfs_dir_rewind(lfs_t *lfs, lfs_dir_t *dir); - - -/// Filesystem-level filesystem operations - -// Find on-disk info about the filesystem -// -// Fills out the fsinfo structure based on the filesystem found on-disk. -// Returns a negative error code on failure. -int lfs_fs_stat(lfs_t *lfs, struct lfs_fsinfo *fsinfo); - -// Finds the current size of the filesystem -// -// Note: Result is best effort. If files share COW structures, the returned -// size may be larger than the filesystem actually is. -// -// Returns the number of allocated blocks, or a negative error code on failure. -lfs_ssize_t lfs_fs_size(lfs_t *lfs); - -// Traverse through all blocks in use by the filesystem -// -// The provided callback will be called with each block address that is -// currently in use by the filesystem. This can be used to determine which -// blocks are in use or how much of the storage is available. -// -// Returns a negative error code on failure. -int lfs_fs_traverse(lfs_t *lfs, int (*cb)(void*, lfs_block_t), void *data); - -#ifndef LFS_READONLY -// Attempt to make the filesystem consistent and ready for writing -// -// Calling this function is not required, consistency will be implicitly -// enforced on the first operation that writes to the filesystem, but this -// function allows the work to be performed earlier and without other -// filesystem changes. -// -// Returns a negative error code on failure. -int lfs_fs_mkconsistent(lfs_t *lfs); -#endif - -#ifndef LFS_READONLY -// Attempt any janitorial work -// -// This currently: -// 1. Calls mkconsistent if not already consistent -// 2. Compacts metadata > compact_thresh -// 3. Populates the block allocator -// -// Though additional janitorial work may be added in the future. -// -// Calling this function is not required, but may allow the offloading of -// expensive janitorial work to a less time-critical code path. -// -// Returns a negative error code on failure. Accomplishing nothing is not -// an error. -int lfs_fs_gc(lfs_t *lfs); -#endif - -#ifndef LFS_READONLY -// Grows the filesystem to a new size, updating the superblock with the new -// block count. -// -// If LFS_SHRINKNONRELOCATING is defined, this function will also accept -// block_counts smaller than the current configuration, after checking -// that none of the blocks that are being removed are in use. -// Note that littlefs's pseudorandom block allocation means that -// this is very unlikely to work in the general case. -// -// Returns a negative error code on failure. -int lfs_fs_grow(lfs_t *lfs, lfs_size_t block_count); -#endif - -#ifndef LFS_READONLY -#ifdef LFS_MIGRATE -// Attempts to migrate a previous version of littlefs -// -// Behaves similarly to the lfs_format function. Attempts to mount -// the previous version of littlefs and update the filesystem so it can be -// mounted with the current version of littlefs. -// -// Requires a littlefs object and config struct. This clobbers the littlefs -// object, and does not leave the filesystem mounted. The config struct must -// be zeroed for defaults and backwards compatibility. -// -// Returns a negative error code on failure. -int lfs_migrate(lfs_t *lfs, const struct lfs_config *cfg); -#endif -#endif - - -#ifdef __cplusplus -} /* extern "C" */ -#endif - -#endif diff --git a/components/littlefs/include/lfs_util.h b/components/littlefs/include/lfs_util.h deleted file mode 100644 index c1999fa96..000000000 --- a/components/littlefs/include/lfs_util.h +++ /dev/null @@ -1,273 +0,0 @@ -/* - * lfs utility functions - * - * Copyright (c) 2022, The littlefs authors. - * Copyright (c) 2017, Arm Limited. All rights reserved. - * SPDX-License-Identifier: BSD-3-Clause - */ -#ifndef LFS_UTIL_H -#define LFS_UTIL_H - -#define LFS_STRINGIZE(x) LFS_STRINGIZE2(x) -#define LFS_STRINGIZE2(x) #x - -// Users can override lfs_util.h with their own configuration by defining -// LFS_CONFIG as a header file to include (-DLFS_CONFIG=lfs_config.h). -// -// If LFS_CONFIG is used, none of the default utils will be emitted and must be -// provided by the config file. To start, I would suggest copying lfs_util.h -// and modifying as needed. -#ifdef LFS_CONFIG -#include LFS_STRINGIZE(LFS_CONFIG) -#else - -// Alternatively, users can provide a header file which defines -// macros and other things consumed by littlefs. -// -// For example, provide my_defines.h, which contains -// something like: -// -// #include -// extern void *my_malloc(size_t sz); -// #define LFS_MALLOC(sz) my_malloc(sz) -// -// And build littlefs with the header by defining LFS_DEFINES. -// (-DLFS_DEFINES=my_defines.h) - -#ifdef LFS_DEFINES -#include LFS_STRINGIZE(LFS_DEFINES) -#endif - -// System includes -#include -#include -#include -#include - -#ifndef LFS_NO_MALLOC -#include -#endif -#ifndef LFS_NO_ASSERT -#include -#endif -#if !defined(LFS_NO_DEBUG) || \ - !defined(LFS_NO_WARN) || \ - !defined(LFS_NO_ERROR) || \ - defined(LFS_YES_TRACE) -#include -#endif - -#ifdef __cplusplus -extern "C" -{ -#endif - - -// Macros, may be replaced by system specific wrappers. Arguments to these -// macros must not have side-effects as the macros can be removed for a smaller -// code footprint - -// Logging functions -#ifndef LFS_TRACE -#ifdef LFS_YES_TRACE -#define LFS_TRACE_(fmt, ...) \ - printf("%s:%d:trace: " fmt "%s\n", __FILE__, __LINE__, __VA_ARGS__) -#define LFS_TRACE(...) LFS_TRACE_(__VA_ARGS__, "") -#else -#define LFS_TRACE(...) -#endif -#endif - -#ifndef LFS_DEBUG -#ifndef LFS_NO_DEBUG -#define LFS_DEBUG_(fmt, ...) \ - printf("%s:%d:debug: " fmt "%s\n", __FILE__, __LINE__, __VA_ARGS__) -#define LFS_DEBUG(...) LFS_DEBUG_(__VA_ARGS__, "") -#else -#define LFS_DEBUG(...) -#endif -#endif - -#ifndef LFS_WARN -#ifndef LFS_NO_WARN -#define LFS_WARN_(fmt, ...) \ - printf("%s:%d:warn: " fmt "%s\n", __FILE__, __LINE__, __VA_ARGS__) -#define LFS_WARN(...) LFS_WARN_(__VA_ARGS__, "") -#else -#define LFS_WARN(...) -#endif -#endif - -#ifndef LFS_ERROR -#ifndef LFS_NO_ERROR -#define LFS_ERROR_(fmt, ...) \ - printf("%s:%d:error: " fmt "%s\n", __FILE__, __LINE__, __VA_ARGS__) -#define LFS_ERROR(...) LFS_ERROR_(__VA_ARGS__, "") -#else -#define LFS_ERROR(...) -#endif -#endif - -// Runtime assertions -#ifndef LFS_ASSERT -#ifndef LFS_NO_ASSERT -#define LFS_ASSERT(test) assert(test) -#else -#define LFS_ASSERT(test) -#endif -#endif - - -// Builtin functions, these may be replaced by more efficient -// toolchain-specific implementations. LFS_NO_INTRINSICS falls back to a more -// expensive basic C implementation for debugging purposes - -// Min/max functions for unsigned 32-bit numbers -static inline uint32_t lfs_max(uint32_t a, uint32_t b) { - return (a > b) ? a : b; -} - -static inline uint32_t lfs_min(uint32_t a, uint32_t b) { - return (a < b) ? a : b; -} - -// Align to nearest multiple of a size -static inline uint32_t lfs_aligndown(uint32_t a, uint32_t alignment) { - return a - (a % alignment); -} - -static inline uint32_t lfs_alignup(uint32_t a, uint32_t alignment) { - return lfs_aligndown(a + alignment-1, alignment); -} - -// Find the smallest power of 2 greater than or equal to a -static inline uint32_t lfs_npw2(uint32_t a) { -#if !defined(LFS_NO_INTRINSICS) && (defined(__GNUC__) || defined(__CC_ARM)) - return 32 - __builtin_clz(a-1); -#else - uint32_t r = 0; - uint32_t s; - a -= 1; - s = (a > 0xffff) << 4; a >>= s; r |= s; - s = (a > 0xff ) << 3; a >>= s; r |= s; - s = (a > 0xf ) << 2; a >>= s; r |= s; - s = (a > 0x3 ) << 1; a >>= s; r |= s; - return (r | (a >> 1)) + 1; -#endif -} - -// Count the number of trailing binary zeros in a -// lfs_ctz(0) may be undefined -static inline uint32_t lfs_ctz(uint32_t a) { -#if !defined(LFS_NO_INTRINSICS) && defined(__GNUC__) - return __builtin_ctz(a); -#else - return lfs_npw2((a & -a) + 1) - 1; -#endif -} - -// Count the number of binary ones in a -static inline uint32_t lfs_popc(uint32_t a) { -#if !defined(LFS_NO_INTRINSICS) && (defined(__GNUC__) || defined(__CC_ARM)) - return __builtin_popcount(a); -#else - a = a - ((a >> 1) & 0x55555555); - a = (a & 0x33333333) + ((a >> 2) & 0x33333333); - return (((a + (a >> 4)) & 0xf0f0f0f) * 0x1010101) >> 24; -#endif -} - -// Find the sequence comparison of a and b, this is the distance -// between a and b ignoring overflow -static inline int lfs_scmp(uint32_t a, uint32_t b) { - return (int)(unsigned)(a - b); -} - -// Convert between 32-bit little-endian and native order -static inline uint32_t lfs_fromle32(uint32_t a) { -#if (defined( BYTE_ORDER ) && defined( ORDER_LITTLE_ENDIAN ) && BYTE_ORDER == ORDER_LITTLE_ENDIAN ) || \ - (defined(__BYTE_ORDER ) && defined(__ORDER_LITTLE_ENDIAN ) && __BYTE_ORDER == __ORDER_LITTLE_ENDIAN ) || \ - (defined(__BYTE_ORDER__) && defined(__ORDER_LITTLE_ENDIAN__) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__) - return a; -#elif !defined(LFS_NO_INTRINSICS) && ( \ - (defined( BYTE_ORDER ) && defined( ORDER_BIG_ENDIAN ) && BYTE_ORDER == ORDER_BIG_ENDIAN ) || \ - (defined(__BYTE_ORDER ) && defined(__ORDER_BIG_ENDIAN ) && __BYTE_ORDER == __ORDER_BIG_ENDIAN ) || \ - (defined(__BYTE_ORDER__) && defined(__ORDER_BIG_ENDIAN__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__)) - return __builtin_bswap32(a); -#else - return ((uint32_t)((uint8_t*)&a)[0] << 0) | - ((uint32_t)((uint8_t*)&a)[1] << 8) | - ((uint32_t)((uint8_t*)&a)[2] << 16) | - ((uint32_t)((uint8_t*)&a)[3] << 24); -#endif -} - -static inline uint32_t lfs_tole32(uint32_t a) { - return lfs_fromle32(a); -} - -// Convert between 32-bit big-endian and native order -static inline uint32_t lfs_frombe32(uint32_t a) { -#if !defined(LFS_NO_INTRINSICS) && ( \ - (defined( BYTE_ORDER ) && defined( ORDER_LITTLE_ENDIAN ) && BYTE_ORDER == ORDER_LITTLE_ENDIAN ) || \ - (defined(__BYTE_ORDER ) && defined(__ORDER_LITTLE_ENDIAN ) && __BYTE_ORDER == __ORDER_LITTLE_ENDIAN ) || \ - (defined(__BYTE_ORDER__) && defined(__ORDER_LITTLE_ENDIAN__) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__)) - return __builtin_bswap32(a); -#elif (defined( BYTE_ORDER ) && defined( ORDER_BIG_ENDIAN ) && BYTE_ORDER == ORDER_BIG_ENDIAN ) || \ - (defined(__BYTE_ORDER ) && defined(__ORDER_BIG_ENDIAN ) && __BYTE_ORDER == __ORDER_BIG_ENDIAN ) || \ - (defined(__BYTE_ORDER__) && defined(__ORDER_BIG_ENDIAN__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__) - return a; -#else - return ((uint32_t)((uint8_t*)&a)[0] << 24) | - ((uint32_t)((uint8_t*)&a)[1] << 16) | - ((uint32_t)((uint8_t*)&a)[2] << 8) | - ((uint32_t)((uint8_t*)&a)[3] << 0); -#endif -} - -static inline uint32_t lfs_tobe32(uint32_t a) { - return lfs_frombe32(a); -} - -// Calculate CRC-32 with polynomial = 0x04c11db7 -#ifdef LFS_CRC -static inline uint32_t lfs_crc(uint32_t crc, const void *buffer, size_t size) { - return LFS_CRC(crc, buffer, size); -} -#else -uint32_t lfs_crc(uint32_t crc, const void *buffer, size_t size); -#endif - -// Allocate memory, only used if buffers are not provided to littlefs -// -// littlefs current has no alignment requirements, as it only allocates -// byte-level buffers. -static inline void *lfs_malloc(size_t size) { -#if defined(LFS_MALLOC) - return LFS_MALLOC(size); -#elif !defined(LFS_NO_MALLOC) - return malloc(size); -#else - (void)size; - return NULL; -#endif -} - -// Deallocate memory, only used if buffers are not provided to littlefs -static inline void lfs_free(void *p) { -#if defined(LFS_FREE) - LFS_FREE(p); -#elif !defined(LFS_NO_MALLOC) - free(p); -#else - (void)p; -#endif -} - - -#ifdef __cplusplus -} /* extern "C" */ -#endif - -#endif -#endif diff --git a/components/littlefs/src/lfs.c b/components/littlefs/src/lfs.c deleted file mode 100644 index c83c7e0b0..000000000 --- a/components/littlefs/src/lfs.c +++ /dev/null @@ -1,6558 +0,0 @@ -/* - * The little filesystem - * - * Copyright (c) 2022, The littlefs authors. - * Copyright (c) 2017, Arm Limited. All rights reserved. - * SPDX-License-Identifier: BSD-3-Clause - */ -#include "lfs.h" -#include "lfs_util.h" - - -// some constants used throughout the code -#define LFS_BLOCK_NULL ((lfs_block_t)-1) -#define LFS_BLOCK_INLINE ((lfs_block_t)-2) - -enum { - LFS_OK_RELOCATED = 1, - LFS_OK_DROPPED = 2, - LFS_OK_ORPHANED = 3, -}; - -enum { - LFS_CMP_EQ = 0, - LFS_CMP_LT = 1, - LFS_CMP_GT = 2, -}; - - -/// Caching block device operations /// - -static inline void lfs_cache_drop(lfs_t *lfs, lfs_cache_t *rcache) { - // do not zero, cheaper if cache is readonly or only going to be - // written with identical data (during relocates) - (void)lfs; - rcache->block = LFS_BLOCK_NULL; -} - -static inline void lfs_cache_zero(lfs_t *lfs, lfs_cache_t *pcache) { - // zero to avoid information leak - memset(pcache->buffer, 0xff, lfs->cfg->cache_size); - pcache->block = LFS_BLOCK_NULL; -} - -static int lfs_bd_read(lfs_t *lfs, - const lfs_cache_t *pcache, lfs_cache_t *rcache, lfs_size_t hint, - lfs_block_t block, lfs_off_t off, - void *buffer, lfs_size_t size) { - uint8_t *data = buffer; - if (off+size > lfs->cfg->block_size - || (lfs->block_count && block >= lfs->block_count)) { - return LFS_ERR_CORRUPT; - } - - while (size > 0) { - lfs_size_t diff = size; - - if (pcache && block == pcache->block && - off < pcache->off + pcache->size) { - if (off >= pcache->off) { - // is already in pcache? - diff = lfs_min(diff, pcache->size - (off-pcache->off)); - memcpy(data, &pcache->buffer[off-pcache->off], diff); - - data += diff; - off += diff; - size -= diff; - continue; - } - - // pcache takes priority - diff = lfs_min(diff, pcache->off-off); - } - - if (block == rcache->block && - off < rcache->off + rcache->size) { - if (off >= rcache->off) { - // is already in rcache? - diff = lfs_min(diff, rcache->size - (off-rcache->off)); - memcpy(data, &rcache->buffer[off-rcache->off], diff); - - data += diff; - off += diff; - size -= diff; - continue; - } - - // rcache takes priority - diff = lfs_min(diff, rcache->off-off); - } - - if (size >= hint && off % lfs->cfg->read_size == 0 && - size >= lfs->cfg->read_size) { - // bypass cache? - diff = lfs_aligndown(diff, lfs->cfg->read_size); - int err = lfs->cfg->read(lfs->cfg, block, off, data, diff); - LFS_ASSERT(err <= 0); - if (err) { - return err; - } - - data += diff; - off += diff; - size -= diff; - continue; - } - - // load to cache, first condition can no longer fail - LFS_ASSERT(!lfs->block_count || block < lfs->block_count); - rcache->block = block; - rcache->off = lfs_aligndown(off, lfs->cfg->read_size); - rcache->size = lfs_min( - lfs_min( - lfs_alignup(off+hint, lfs->cfg->read_size), - lfs->cfg->block_size) - - rcache->off, - lfs->cfg->cache_size); - int err = lfs->cfg->read(lfs->cfg, rcache->block, - rcache->off, rcache->buffer, rcache->size); - LFS_ASSERT(err <= 0); - if (err) { - return err; - } - } - - return 0; -} - -static int lfs_bd_cmp(lfs_t *lfs, - const lfs_cache_t *pcache, lfs_cache_t *rcache, lfs_size_t hint, - lfs_block_t block, lfs_off_t off, - const void *buffer, lfs_size_t size) { - const uint8_t *data = buffer; - lfs_size_t diff = 0; - - for (lfs_off_t i = 0; i < size; i += diff) { - uint8_t dat[8]; - - diff = lfs_min(size-i, sizeof(dat)); - int err = lfs_bd_read(lfs, - pcache, rcache, hint-i, - block, off+i, &dat, diff); - if (err) { - return err; - } - - int res = memcmp(dat, data + i, diff); - if (res) { - return res < 0 ? LFS_CMP_LT : LFS_CMP_GT; - } - } - - return LFS_CMP_EQ; -} - -static int lfs_bd_crc(lfs_t *lfs, - const lfs_cache_t *pcache, lfs_cache_t *rcache, lfs_size_t hint, - lfs_block_t block, lfs_off_t off, lfs_size_t size, uint32_t *crc) { - lfs_size_t diff = 0; - - for (lfs_off_t i = 0; i < size; i += diff) { - uint8_t dat[8]; - diff = lfs_min(size-i, sizeof(dat)); - int err = lfs_bd_read(lfs, - pcache, rcache, hint-i, - block, off+i, &dat, diff); - if (err) { - return err; - } - - *crc = lfs_crc(*crc, &dat, diff); - } - - return 0; -} - -#ifndef LFS_READONLY -static int lfs_bd_flush(lfs_t *lfs, - lfs_cache_t *pcache, lfs_cache_t *rcache, bool validate) { - if (pcache->block != LFS_BLOCK_NULL && pcache->block != LFS_BLOCK_INLINE) { - LFS_ASSERT(pcache->block < lfs->block_count); - lfs_size_t diff = lfs_alignup(pcache->size, lfs->cfg->prog_size); - int err = lfs->cfg->prog(lfs->cfg, pcache->block, - pcache->off, pcache->buffer, diff); - LFS_ASSERT(err <= 0); - if (err) { - return err; - } - - if (validate) { - // check data on disk - lfs_cache_drop(lfs, rcache); - int res = lfs_bd_cmp(lfs, - NULL, rcache, diff, - pcache->block, pcache->off, pcache->buffer, diff); - if (res < 0) { - return res; - } - - if (res != LFS_CMP_EQ) { - return LFS_ERR_CORRUPT; - } - } - - lfs_cache_zero(lfs, pcache); - } - - return 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_bd_sync(lfs_t *lfs, - lfs_cache_t *pcache, lfs_cache_t *rcache, bool validate) { - lfs_cache_drop(lfs, rcache); - - int err = lfs_bd_flush(lfs, pcache, rcache, validate); - if (err) { - return err; - } - - err = lfs->cfg->sync(lfs->cfg); - LFS_ASSERT(err <= 0); - return err; -} -#endif - -#ifndef LFS_READONLY -static int lfs_bd_prog(lfs_t *lfs, - lfs_cache_t *pcache, lfs_cache_t *rcache, bool validate, - lfs_block_t block, lfs_off_t off, - const void *buffer, lfs_size_t size) { - const uint8_t *data = buffer; - LFS_ASSERT(block == LFS_BLOCK_INLINE || block < lfs->block_count); - LFS_ASSERT(off + size <= lfs->cfg->block_size); - - while (size > 0) { - if (block == pcache->block && - off >= pcache->off && - off < pcache->off + lfs->cfg->cache_size) { - // already fits in pcache? - lfs_size_t diff = lfs_min(size, - lfs->cfg->cache_size - (off-pcache->off)); - memcpy(&pcache->buffer[off-pcache->off], data, diff); - - data += diff; - off += diff; - size -= diff; - - pcache->size = lfs_max(pcache->size, off - pcache->off); - if (pcache->size == lfs->cfg->cache_size) { - // eagerly flush out pcache if we fill up - int err = lfs_bd_flush(lfs, pcache, rcache, validate); - if (err) { - return err; - } - } - - continue; - } - - // pcache must have been flushed, either by programming an - // entire block or manually flushing the pcache - LFS_ASSERT(pcache->block == LFS_BLOCK_NULL); - - // prepare pcache, first condition can no longer fail - pcache->block = block; - pcache->off = lfs_aligndown(off, lfs->cfg->prog_size); - pcache->size = 0; - } - - return 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_bd_erase(lfs_t *lfs, lfs_block_t block) { - LFS_ASSERT(block < lfs->block_count); - int err = lfs->cfg->erase(lfs->cfg, block); - LFS_ASSERT(err <= 0); - return err; -} -#endif - - -/// Small type-level utilities /// - -// some operations on paths -static inline lfs_size_t lfs_path_namelen(const char *path) { - return (lfs_size_t)strcspn(path, "/"); -} - -static inline bool lfs_path_islast(const char *path) { - lfs_size_t namelen = lfs_path_namelen(path); - return path[namelen + strspn(path + namelen, "/")] == '\0'; -} - -static inline bool lfs_path_isdir(const char *path) { - return path[lfs_path_namelen(path)] != '\0'; -} - -// operations on block pairs -static inline void lfs_pair_swap(lfs_block_t pair[2]) { - lfs_block_t t = pair[0]; - pair[0] = pair[1]; - pair[1] = t; -} - -static inline bool lfs_pair_isnull(const lfs_block_t pair[2]) { - return pair[0] == LFS_BLOCK_NULL || pair[1] == LFS_BLOCK_NULL; -} - -static inline int lfs_pair_cmp( - const lfs_block_t paira[2], - const lfs_block_t pairb[2]) { - return !(paira[0] == pairb[0] || paira[1] == pairb[1] || - paira[0] == pairb[1] || paira[1] == pairb[0]); -} - -static inline bool lfs_pair_issync( - const lfs_block_t paira[2], - const lfs_block_t pairb[2]) { - return (paira[0] == pairb[0] && paira[1] == pairb[1]) || - (paira[0] == pairb[1] && paira[1] == pairb[0]); -} - -static inline void lfs_pair_fromle32(lfs_block_t pair[2]) { - pair[0] = lfs_fromle32(pair[0]); - pair[1] = lfs_fromle32(pair[1]); -} - -#ifndef LFS_READONLY -static inline void lfs_pair_tole32(lfs_block_t pair[2]) { - pair[0] = lfs_tole32(pair[0]); - pair[1] = lfs_tole32(pair[1]); -} -#endif - -// operations on 32-bit entry tags -typedef uint32_t lfs_tag_t; -typedef int32_t lfs_stag_t; - -#define LFS_MKTAG(type, id, size) \ - (((lfs_tag_t)(type) << 20) | ((lfs_tag_t)(id) << 10) | (lfs_tag_t)(size)) - -#define LFS_MKTAG_IF(cond, type, id, size) \ - ((cond) ? LFS_MKTAG(type, id, size) : LFS_MKTAG(LFS_FROM_NOOP, 0, 0)) - -#define LFS_MKTAG_IF_ELSE(cond, type1, id1, size1, type2, id2, size2) \ - ((cond) ? LFS_MKTAG(type1, id1, size1) : LFS_MKTAG(type2, id2, size2)) - -static inline bool lfs_tag_isvalid(lfs_tag_t tag) { - return !(tag & 0x80000000); -} - -static inline bool lfs_tag_isdelete(lfs_tag_t tag) { - return ((int32_t)(tag << 22) >> 22) == -1; -} - -static inline uint16_t lfs_tag_type1(lfs_tag_t tag) { - return (tag & 0x70000000) >> 20; -} - -static inline uint16_t lfs_tag_type2(lfs_tag_t tag) { - return (tag & 0x78000000) >> 20; -} - -static inline uint16_t lfs_tag_type3(lfs_tag_t tag) { - return (tag & 0x7ff00000) >> 20; -} - -static inline uint8_t lfs_tag_chunk(lfs_tag_t tag) { - return (tag & 0x0ff00000) >> 20; -} - -static inline int8_t lfs_tag_splice(lfs_tag_t tag) { - return (int8_t)lfs_tag_chunk(tag); -} - -static inline uint16_t lfs_tag_id(lfs_tag_t tag) { - return (tag & 0x000ffc00) >> 10; -} - -static inline lfs_size_t lfs_tag_size(lfs_tag_t tag) { - return tag & 0x000003ff; -} - -static inline lfs_size_t lfs_tag_dsize(lfs_tag_t tag) { - return sizeof(tag) + lfs_tag_size(tag + lfs_tag_isdelete(tag)); -} - -// operations on attributes in attribute lists -struct lfs_mattr { - lfs_tag_t tag; - const void *buffer; -}; - -struct lfs_diskoff { - lfs_block_t block; - lfs_off_t off; -}; - -#define LFS_MKATTRS(...) \ - (struct lfs_mattr[]){__VA_ARGS__}, \ - sizeof((struct lfs_mattr[]){__VA_ARGS__}) / sizeof(struct lfs_mattr) - -// operations on global state -static inline void lfs_gstate_xor(lfs_gstate_t *a, const lfs_gstate_t *b) { - a->tag ^= b->tag; - a->pair[0] ^= b->pair[0]; - a->pair[1] ^= b->pair[1]; -} - -static inline bool lfs_gstate_iszero(const lfs_gstate_t *a) { - return a->tag == 0 - && a->pair[0] == 0 - && a->pair[1] == 0; -} - -#ifndef LFS_READONLY -static inline bool lfs_gstate_hasorphans(const lfs_gstate_t *a) { - return lfs_tag_size(a->tag); -} - -static inline uint8_t lfs_gstate_getorphans(const lfs_gstate_t *a) { - return lfs_tag_size(a->tag) & 0x1ff; -} - -static inline bool lfs_gstate_hasmove(const lfs_gstate_t *a) { - return lfs_tag_type1(a->tag); -} -#endif - -static inline bool lfs_gstate_needssuperblock(const lfs_gstate_t *a) { - return lfs_tag_size(a->tag) >> 9; -} - -static inline bool lfs_gstate_hasmovehere(const lfs_gstate_t *a, - const lfs_block_t *pair) { - return lfs_tag_type1(a->tag) && lfs_pair_cmp(a->pair, pair) == 0; -} - -static inline void lfs_gstate_fromle32(lfs_gstate_t *a) { - a->tag = lfs_fromle32(a->tag); - a->pair[0] = lfs_fromle32(a->pair[0]); - a->pair[1] = lfs_fromle32(a->pair[1]); -} - -#ifndef LFS_READONLY -static inline void lfs_gstate_tole32(lfs_gstate_t *a) { - a->tag = lfs_tole32(a->tag); - a->pair[0] = lfs_tole32(a->pair[0]); - a->pair[1] = lfs_tole32(a->pair[1]); -} -#endif - -// operations on forward-CRCs used to track erased state -struct lfs_fcrc { - lfs_size_t size; - uint32_t crc; -}; - -static void lfs_fcrc_fromle32(struct lfs_fcrc *fcrc) { - fcrc->size = lfs_fromle32(fcrc->size); - fcrc->crc = lfs_fromle32(fcrc->crc); -} - -#ifndef LFS_READONLY -static void lfs_fcrc_tole32(struct lfs_fcrc *fcrc) { - fcrc->size = lfs_tole32(fcrc->size); - fcrc->crc = lfs_tole32(fcrc->crc); -} -#endif - -// other endianness operations -static void lfs_ctz_fromle32(struct lfs_ctz *ctz) { - ctz->head = lfs_fromle32(ctz->head); - ctz->size = lfs_fromle32(ctz->size); -} - -#ifndef LFS_READONLY -static void lfs_ctz_tole32(struct lfs_ctz *ctz) { - ctz->head = lfs_tole32(ctz->head); - ctz->size = lfs_tole32(ctz->size); -} -#endif - -static inline void lfs_superblock_fromle32(lfs_superblock_t *superblock) { - superblock->version = lfs_fromle32(superblock->version); - superblock->block_size = lfs_fromle32(superblock->block_size); - superblock->block_count = lfs_fromle32(superblock->block_count); - superblock->name_max = lfs_fromle32(superblock->name_max); - superblock->file_max = lfs_fromle32(superblock->file_max); - superblock->attr_max = lfs_fromle32(superblock->attr_max); -} - -#ifndef LFS_READONLY -static inline void lfs_superblock_tole32(lfs_superblock_t *superblock) { - superblock->version = lfs_tole32(superblock->version); - superblock->block_size = lfs_tole32(superblock->block_size); - superblock->block_count = lfs_tole32(superblock->block_count); - superblock->name_max = lfs_tole32(superblock->name_max); - superblock->file_max = lfs_tole32(superblock->file_max); - superblock->attr_max = lfs_tole32(superblock->attr_max); -} -#endif - -#ifndef LFS_NO_ASSERT -static bool lfs_mlist_isopen(struct lfs_mlist *head, - struct lfs_mlist *node) { - for (struct lfs_mlist **p = &head; *p; p = &(*p)->next) { - if (*p == (struct lfs_mlist*)node) { - return true; - } - } - - return false; -} -#endif - -static void lfs_mlist_remove(lfs_t *lfs, struct lfs_mlist *mlist) { - for (struct lfs_mlist **p = &lfs->mlist; *p; p = &(*p)->next) { - if (*p == mlist) { - *p = (*p)->next; - break; - } - } -} - -static void lfs_mlist_append(lfs_t *lfs, struct lfs_mlist *mlist) { - mlist->next = lfs->mlist; - lfs->mlist = mlist; -} - -// some other filesystem operations -static uint32_t lfs_fs_disk_version(lfs_t *lfs) { - (void)lfs; -#ifdef LFS_MULTIVERSION - if (lfs->cfg->disk_version) { - return lfs->cfg->disk_version; - } else -#endif - { - return LFS_DISK_VERSION; - } -} - -static uint16_t lfs_fs_disk_version_major(lfs_t *lfs) { - return 0xffff & (lfs_fs_disk_version(lfs) >> 16); - -} - -static uint16_t lfs_fs_disk_version_minor(lfs_t *lfs) { - return 0xffff & (lfs_fs_disk_version(lfs) >> 0); -} - - -/// Internal operations predeclared here /// -#ifndef LFS_READONLY -static int lfs_dir_commit(lfs_t *lfs, lfs_mdir_t *dir, - const struct lfs_mattr *attrs, int attrcount); -static int lfs_dir_compact(lfs_t *lfs, - lfs_mdir_t *dir, const struct lfs_mattr *attrs, int attrcount, - lfs_mdir_t *source, uint16_t begin, uint16_t end); -static lfs_ssize_t lfs_file_flushedwrite(lfs_t *lfs, lfs_file_t *file, - const void *buffer, lfs_size_t size); -static lfs_ssize_t lfs_file_write_(lfs_t *lfs, lfs_file_t *file, - const void *buffer, lfs_size_t size); -static int lfs_file_sync_(lfs_t *lfs, lfs_file_t *file); -static int lfs_file_outline(lfs_t *lfs, lfs_file_t *file); -static int lfs_file_flush(lfs_t *lfs, lfs_file_t *file); - -static int lfs_fs_deorphan(lfs_t *lfs, bool powerloss); -static int lfs_fs_preporphans(lfs_t *lfs, int8_t orphans); -static void lfs_fs_prepmove(lfs_t *lfs, - uint16_t id, const lfs_block_t pair[2]); -static int lfs_fs_pred(lfs_t *lfs, const lfs_block_t dir[2], - lfs_mdir_t *pdir); -static lfs_stag_t lfs_fs_parent(lfs_t *lfs, const lfs_block_t dir[2], - lfs_mdir_t *parent); -static int lfs_fs_forceconsistency(lfs_t *lfs); -#endif - -static void lfs_fs_prepsuperblock(lfs_t *lfs, bool needssuperblock); - -#ifdef LFS_MIGRATE -static int lfs1_traverse(lfs_t *lfs, - int (*cb)(void*, lfs_block_t), void *data); -#endif - -static int lfs_dir_rewind_(lfs_t *lfs, lfs_dir_t *dir); - -static lfs_ssize_t lfs_file_flushedread(lfs_t *lfs, lfs_file_t *file, - void *buffer, lfs_size_t size); -static lfs_ssize_t lfs_file_read_(lfs_t *lfs, lfs_file_t *file, - void *buffer, lfs_size_t size); -static int lfs_file_close_(lfs_t *lfs, lfs_file_t *file); -static lfs_soff_t lfs_file_size_(lfs_t *lfs, lfs_file_t *file); - -static lfs_ssize_t lfs_fs_size_(lfs_t *lfs); -static int lfs_fs_traverse_(lfs_t *lfs, - int (*cb)(void *data, lfs_block_t block), void *data, - bool includeorphans); - -static int lfs_deinit(lfs_t *lfs); -static int lfs_unmount_(lfs_t *lfs); - - -/// Block allocator /// - -// allocations should call this when all allocated blocks are committed to -// the filesystem -// -// after a checkpoint, the block allocator may realloc any untracked blocks -static void lfs_alloc_ckpoint(lfs_t *lfs) { - lfs->lookahead.ckpoint = lfs->block_count; -} - -// drop the lookahead buffer, this is done during mounting and failed -// traversals in order to avoid invalid lookahead state -static void lfs_alloc_drop(lfs_t *lfs) { - lfs->lookahead.size = 0; - lfs->lookahead.next = 0; - lfs_alloc_ckpoint(lfs); -} - -#ifndef LFS_READONLY -static int lfs_alloc_lookahead(void *p, lfs_block_t block) { - lfs_t *lfs = (lfs_t*)p; - lfs_block_t off = ((block - lfs->lookahead.start) - + lfs->block_count) % lfs->block_count; - - if (off < lfs->lookahead.size) { - lfs->lookahead.buffer[off / 8] |= 1U << (off % 8); - } - - return 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_alloc_scan(lfs_t *lfs) { - // move lookahead buffer to the first unused block - // - // note we limit the lookahead buffer to at most the amount of blocks - // checkpointed, this prevents the math in lfs_alloc from underflowing - lfs->lookahead.start = (lfs->lookahead.start + lfs->lookahead.next) - % lfs->block_count; - lfs->lookahead.next = 0; - lfs->lookahead.size = lfs_min( - 8*lfs->cfg->lookahead_size, - lfs->lookahead.ckpoint); - - // find mask of free blocks from tree - memset(lfs->lookahead.buffer, 0, lfs->cfg->lookahead_size); - int err = lfs_fs_traverse_(lfs, lfs_alloc_lookahead, lfs, true); - if (err) { - lfs_alloc_drop(lfs); - return err; - } - - return 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_alloc(lfs_t *lfs, lfs_block_t *block) { - while (true) { - // scan our lookahead buffer for free blocks - while (lfs->lookahead.next < lfs->lookahead.size) { - if (!(lfs->lookahead.buffer[lfs->lookahead.next / 8] - & (1U << (lfs->lookahead.next % 8)))) { - // found a free block - *block = (lfs->lookahead.start + lfs->lookahead.next) - % lfs->block_count; - - // eagerly find next free block to maximize how many blocks - // lfs_alloc_ckpoint makes available for scanning - while (true) { - lfs->lookahead.next += 1; - lfs->lookahead.ckpoint -= 1; - - if (lfs->lookahead.next >= lfs->lookahead.size - || !(lfs->lookahead.buffer[lfs->lookahead.next / 8] - & (1U << (lfs->lookahead.next % 8)))) { - return 0; - } - } - } - - lfs->lookahead.next += 1; - lfs->lookahead.ckpoint -= 1; - } - - // In order to keep our block allocator from spinning forever when our - // filesystem is full, we mark points where there are no in-flight - // allocations with a checkpoint before starting a set of allocations. - // - // If we've looked at all blocks since the last checkpoint, we report - // the filesystem as out of storage. - // - if (lfs->lookahead.ckpoint <= 0) { - LFS_ERROR("No more free space 0x%"PRIx32, - (lfs->lookahead.start + lfs->lookahead.next) - % lfs->block_count); - return LFS_ERR_NOSPC; - } - - // No blocks in our lookahead buffer, we need to scan the filesystem for - // unused blocks in the next lookahead window. - int err = lfs_alloc_scan(lfs); - if(err) { - return err; - } - } -} -#endif - -/// Metadata pair and directory operations /// -static lfs_stag_t lfs_dir_getslice(lfs_t *lfs, const lfs_mdir_t *dir, - lfs_tag_t gmask, lfs_tag_t gtag, - lfs_off_t goff, void *gbuffer, lfs_size_t gsize) { - lfs_off_t off = dir->off; - lfs_tag_t ntag = dir->etag; - lfs_stag_t gdiff = 0; - - // synthetic moves - if (lfs_gstate_hasmovehere(&lfs->gdisk, dir->pair) && - lfs_tag_id(gmask) != 0) { - if (lfs_tag_id(lfs->gdisk.tag) == lfs_tag_id(gtag)) { - return LFS_ERR_NOENT; - } else if (lfs_tag_id(lfs->gdisk.tag) < lfs_tag_id(gtag)) { - gdiff -= LFS_MKTAG(0, 1, 0); - } - } - - // iterate over dir block backwards (for faster lookups) - while (off >= sizeof(lfs_tag_t) + lfs_tag_dsize(ntag)) { - off -= lfs_tag_dsize(ntag); - lfs_tag_t tag = ntag; - int err = lfs_bd_read(lfs, - NULL, &lfs->rcache, sizeof(ntag), - dir->pair[0], off, &ntag, sizeof(ntag)); - LFS_ASSERT(err <= 0); - if (err) { - return err; - } - - ntag = (lfs_frombe32(ntag) ^ tag) & 0x7fffffff; - - if (lfs_tag_id(gmask) != 0 && - lfs_tag_type1(tag) == LFS_TYPE_SPLICE && - lfs_tag_id(tag) <= lfs_tag_id(gtag - gdiff)) { - if (tag == (LFS_MKTAG(LFS_TYPE_CREATE, 0, 0) | - (LFS_MKTAG(0, 0x3ff, 0) & (gtag - gdiff)))) { - // found where we were created - return LFS_ERR_NOENT; - } - - // move around splices - gdiff += LFS_MKTAG(0, lfs_tag_splice(tag), 0); - } - - if ((gmask & tag) == (gmask & (gtag - gdiff))) { - if (lfs_tag_isdelete(tag)) { - return LFS_ERR_NOENT; - } - - lfs_size_t diff = lfs_min(lfs_tag_size(tag), gsize); - err = lfs_bd_read(lfs, - NULL, &lfs->rcache, diff, - dir->pair[0], off+sizeof(tag)+goff, gbuffer, diff); - LFS_ASSERT(err <= 0); - if (err) { - return err; - } - - memset((uint8_t*)gbuffer + diff, 0, gsize - diff); - - return tag + gdiff; - } - } - - return LFS_ERR_NOENT; -} - -static lfs_stag_t lfs_dir_get(lfs_t *lfs, const lfs_mdir_t *dir, - lfs_tag_t gmask, lfs_tag_t gtag, void *buffer) { - return lfs_dir_getslice(lfs, dir, - gmask, gtag, - 0, buffer, lfs_tag_size(gtag)); -} - -static int lfs_dir_getread(lfs_t *lfs, const lfs_mdir_t *dir, - const lfs_cache_t *pcache, lfs_cache_t *rcache, lfs_size_t hint, - lfs_tag_t gmask, lfs_tag_t gtag, - lfs_off_t off, void *buffer, lfs_size_t size) { - uint8_t *data = buffer; - if (off+size > lfs->cfg->block_size) { - return LFS_ERR_CORRUPT; - } - - while (size > 0) { - lfs_size_t diff = size; - - if (pcache && pcache->block == LFS_BLOCK_INLINE && - off < pcache->off + pcache->size) { - if (off >= pcache->off) { - // is already in pcache? - diff = lfs_min(diff, pcache->size - (off-pcache->off)); - memcpy(data, &pcache->buffer[off-pcache->off], diff); - - data += diff; - off += diff; - size -= diff; - continue; - } - - // pcache takes priority - diff = lfs_min(diff, pcache->off-off); - } - - if (rcache->block == LFS_BLOCK_INLINE && - off < rcache->off + rcache->size) { - if (off >= rcache->off) { - // is already in rcache? - diff = lfs_min(diff, rcache->size - (off-rcache->off)); - memcpy(data, &rcache->buffer[off-rcache->off], diff); - - data += diff; - off += diff; - size -= diff; - continue; - } - } - - // load to cache, first condition can no longer fail - rcache->block = LFS_BLOCK_INLINE; - rcache->off = lfs_aligndown(off, lfs->cfg->read_size); - rcache->size = lfs_min(lfs_alignup(off+hint, lfs->cfg->read_size), - lfs->cfg->cache_size); - int err = lfs_dir_getslice(lfs, dir, gmask, gtag, - rcache->off, rcache->buffer, rcache->size); - if (err < 0) { - return err; - } - } - - return 0; -} - -#ifndef LFS_READONLY -static int lfs_dir_traverse_filter(void *p, - lfs_tag_t tag, const void *buffer) { - lfs_tag_t *filtertag = p; - (void)buffer; - - // which mask depends on unique bit in tag structure - uint32_t mask = (tag & LFS_MKTAG(0x100, 0, 0)) - ? LFS_MKTAG(0x7ff, 0x3ff, 0) - : LFS_MKTAG(0x700, 0x3ff, 0); - - // check for redundancy - if ((mask & tag) == (mask & *filtertag) || - lfs_tag_isdelete(*filtertag) || - (LFS_MKTAG(0x7ff, 0x3ff, 0) & tag) == ( - LFS_MKTAG(LFS_TYPE_DELETE, 0, 0) | - (LFS_MKTAG(0, 0x3ff, 0) & *filtertag))) { - *filtertag = LFS_MKTAG(LFS_FROM_NOOP, 0, 0); - return true; - } - - // check if we need to adjust for created/deleted tags - if (lfs_tag_type1(tag) == LFS_TYPE_SPLICE && - lfs_tag_id(tag) <= lfs_tag_id(*filtertag)) { - *filtertag += LFS_MKTAG(0, lfs_tag_splice(tag), 0); - } - - return false; -} -#endif - -#ifndef LFS_READONLY -// maximum recursive depth of lfs_dir_traverse, the deepest call: -// -// traverse with commit -// '-> traverse with move -// '-> traverse with filter -// -#define LFS_DIR_TRAVERSE_DEPTH 3 - -struct lfs_dir_traverse { - const lfs_mdir_t *dir; - lfs_off_t off; - lfs_tag_t ptag; - const struct lfs_mattr *attrs; - int attrcount; - - lfs_tag_t tmask; - lfs_tag_t ttag; - uint16_t begin; - uint16_t end; - int16_t diff; - - int (*cb)(void *data, lfs_tag_t tag, const void *buffer); - void *data; - - lfs_tag_t tag; - const void *buffer; - struct lfs_diskoff disk; -}; - -static int lfs_dir_traverse(lfs_t *lfs, - const lfs_mdir_t *dir, lfs_off_t off, lfs_tag_t ptag, - const struct lfs_mattr *attrs, int attrcount, - lfs_tag_t tmask, lfs_tag_t ttag, - uint16_t begin, uint16_t end, int16_t diff, - int (*cb)(void *data, lfs_tag_t tag, const void *buffer), void *data) { - // This function in inherently recursive, but bounded. To allow tool-based - // analysis without unnecessary code-cost we use an explicit stack - struct lfs_dir_traverse stack[LFS_DIR_TRAVERSE_DEPTH-1]; - unsigned sp = 0; - int res; - - // iterate over directory and attrs - lfs_tag_t tag; - const void *buffer; - struct lfs_diskoff disk = {0}; - while (true) { - { - if (off+lfs_tag_dsize(ptag) < dir->off) { - off += lfs_tag_dsize(ptag); - int err = lfs_bd_read(lfs, - NULL, &lfs->rcache, sizeof(tag), - dir->pair[0], off, &tag, sizeof(tag)); - if (err) { - return err; - } - - tag = (lfs_frombe32(tag) ^ ptag) | 0x80000000; - disk.block = dir->pair[0]; - disk.off = off+sizeof(lfs_tag_t); - buffer = &disk; - ptag = tag; - } else if (attrcount > 0) { - tag = attrs[0].tag; - buffer = attrs[0].buffer; - attrs += 1; - attrcount -= 1; - } else { - // finished traversal, pop from stack? - res = 0; - break; - } - - // do we need to filter? - lfs_tag_t mask = LFS_MKTAG(0x7ff, 0, 0); - if ((mask & tmask & tag) != (mask & tmask & ttag)) { - continue; - } - - if (lfs_tag_id(tmask) != 0) { - LFS_ASSERT(sp < LFS_DIR_TRAVERSE_DEPTH); - // recurse, scan for duplicates, and update tag based on - // creates/deletes - stack[sp] = (struct lfs_dir_traverse){ - .dir = dir, - .off = off, - .ptag = ptag, - .attrs = attrs, - .attrcount = attrcount, - .tmask = tmask, - .ttag = ttag, - .begin = begin, - .end = end, - .diff = diff, - .cb = cb, - .data = data, - .tag = tag, - .buffer = buffer, - .disk = disk, - }; - sp += 1; - - tmask = 0; - ttag = 0; - begin = 0; - end = 0; - diff = 0; - cb = lfs_dir_traverse_filter; - data = &stack[sp-1].tag; - continue; - } - } - -popped: - // in filter range? - if (lfs_tag_id(tmask) != 0 && - !(lfs_tag_id(tag) >= begin && lfs_tag_id(tag) < end)) { - continue; - } - - // handle special cases for mcu-side operations - if (lfs_tag_type3(tag) == LFS_FROM_NOOP) { - // do nothing - } else if (lfs_tag_type3(tag) == LFS_FROM_MOVE) { - // Without this condition, lfs_dir_traverse can exhibit an - // extremely expensive O(n^3) of nested loops when renaming. - // This happens because lfs_dir_traverse tries to filter tags by - // the tags in the source directory, triggering a second - // lfs_dir_traverse with its own filter operation. - // - // traverse with commit - // '-> traverse with filter - // '-> traverse with move - // '-> traverse with filter - // - // However we don't actually care about filtering the second set of - // tags, since duplicate tags have no effect when filtering. - // - // This check skips this unnecessary recursive filtering explicitly, - // reducing this runtime from O(n^3) to O(n^2). - if (cb == lfs_dir_traverse_filter) { - continue; - } - - // recurse into move - stack[sp] = (struct lfs_dir_traverse){ - .dir = dir, - .off = off, - .ptag = ptag, - .attrs = attrs, - .attrcount = attrcount, - .tmask = tmask, - .ttag = ttag, - .begin = begin, - .end = end, - .diff = diff, - .cb = cb, - .data = data, - .tag = LFS_MKTAG(LFS_FROM_NOOP, 0, 0), - }; - sp += 1; - - uint16_t fromid = lfs_tag_size(tag); - uint16_t toid = lfs_tag_id(tag); - dir = buffer; - off = 0; - ptag = 0xffffffff; - attrs = NULL; - attrcount = 0; - tmask = LFS_MKTAG(0x600, 0x3ff, 0); - ttag = LFS_MKTAG(LFS_TYPE_STRUCT, 0, 0); - begin = fromid; - end = fromid+1; - diff = toid-fromid+diff; - } else if (lfs_tag_type3(tag) == LFS_FROM_USERATTRS) { - for (unsigned i = 0; i < lfs_tag_size(tag); i++) { - const struct lfs_attr *a = buffer; - res = cb(data, LFS_MKTAG(LFS_TYPE_USERATTR + a[i].type, - lfs_tag_id(tag) + diff, a[i].size), a[i].buffer); - if (res < 0) { - return res; - } - - if (res) { - break; - } - } - } else { - res = cb(data, tag + LFS_MKTAG(0, diff, 0), buffer); - if (res < 0) { - return res; - } - - if (res) { - break; - } - } - } - - if (sp > 0) { - // pop from the stack and return, fortunately all pops share - // a destination - dir = stack[sp-1].dir; - off = stack[sp-1].off; - ptag = stack[sp-1].ptag; - attrs = stack[sp-1].attrs; - attrcount = stack[sp-1].attrcount; - tmask = stack[sp-1].tmask; - ttag = stack[sp-1].ttag; - begin = stack[sp-1].begin; - end = stack[sp-1].end; - diff = stack[sp-1].diff; - cb = stack[sp-1].cb; - data = stack[sp-1].data; - tag = stack[sp-1].tag; - buffer = stack[sp-1].buffer; - disk = stack[sp-1].disk; - sp -= 1; - goto popped; - } else { - return res; - } -} -#endif - -static lfs_stag_t lfs_dir_fetchmatch(lfs_t *lfs, - lfs_mdir_t *dir, const lfs_block_t pair[2], - lfs_tag_t fmask, lfs_tag_t ftag, uint16_t *id, - int (*cb)(void *data, lfs_tag_t tag, const void *buffer), void *data) { - // we can find tag very efficiently during a fetch, since we're already - // scanning the entire directory - lfs_stag_t besttag = -1; - - // if either block address is invalid we return LFS_ERR_CORRUPT here, - // otherwise later writes to the pair could fail - if (lfs->block_count - && (pair[0] >= lfs->block_count || pair[1] >= lfs->block_count)) { - return LFS_ERR_CORRUPT; - } - - // find the block with the most recent revision - uint32_t revs[2] = {0, 0}; - int r = 0; - for (int i = 0; i < 2; i++) { - int err = lfs_bd_read(lfs, - NULL, &lfs->rcache, sizeof(revs[i]), - pair[i], 0, &revs[i], sizeof(revs[i])); - revs[i] = lfs_fromle32(revs[i]); - if (err && err != LFS_ERR_CORRUPT) { - return err; - } - - if (err != LFS_ERR_CORRUPT && - lfs_scmp(revs[i], revs[(i+1)%2]) > 0) { - r = i; - } - } - - dir->pair[0] = pair[(r+0)%2]; - dir->pair[1] = pair[(r+1)%2]; - dir->rev = revs[(r+0)%2]; - dir->off = 0; // nonzero = found some commits - - // now scan tags to fetch the actual dir and find possible match - for (int i = 0; i < 2; i++) { - lfs_off_t off = 0; - lfs_tag_t ptag = 0xffffffff; - - uint16_t tempcount = 0; - lfs_block_t temptail[2] = {LFS_BLOCK_NULL, LFS_BLOCK_NULL}; - bool tempsplit = false; - lfs_stag_t tempbesttag = besttag; - - // assume not erased until proven otherwise - bool maybeerased = false; - bool hasfcrc = false; - struct lfs_fcrc fcrc; - - dir->rev = lfs_tole32(dir->rev); - uint32_t crc = lfs_crc(0xffffffff, &dir->rev, sizeof(dir->rev)); - dir->rev = lfs_fromle32(dir->rev); - - while (true) { - // extract next tag - lfs_tag_t tag; - off += lfs_tag_dsize(ptag); - int err = lfs_bd_read(lfs, - NULL, &lfs->rcache, lfs->cfg->block_size, - dir->pair[0], off, &tag, sizeof(tag)); - if (err) { - if (err == LFS_ERR_CORRUPT) { - // can't continue? - break; - } - return err; - } - - crc = lfs_crc(crc, &tag, sizeof(tag)); - tag = lfs_frombe32(tag) ^ ptag; - - // next commit not yet programmed? - if (!lfs_tag_isvalid(tag)) { - // we only might be erased if the last tag was a crc - maybeerased = (lfs_tag_type2(ptag) == LFS_TYPE_CCRC); - break; - // out of range? - } else if (off + lfs_tag_dsize(tag) > lfs->cfg->block_size) { - break; - } - - ptag = tag; - - if (lfs_tag_type2(tag) == LFS_TYPE_CCRC) { - // check the crc attr - uint32_t dcrc; - err = lfs_bd_read(lfs, - NULL, &lfs->rcache, lfs->cfg->block_size, - dir->pair[0], off+sizeof(tag), &dcrc, sizeof(dcrc)); - if (err) { - if (err == LFS_ERR_CORRUPT) { - break; - } - return err; - } - dcrc = lfs_fromle32(dcrc); - - if (crc != dcrc) { - break; - } - - // reset the next bit if we need to - ptag ^= (lfs_tag_t)(lfs_tag_chunk(tag) & 1U) << 31; - - // toss our crc into the filesystem seed for - // pseudorandom numbers, note we use another crc here - // as a collection function because it is sufficiently - // random and convenient - lfs->seed = lfs_crc(lfs->seed, &crc, sizeof(crc)); - - // update with what's found so far - besttag = tempbesttag; - dir->off = off + lfs_tag_dsize(tag); - dir->etag = ptag; - dir->count = tempcount; - dir->tail[0] = temptail[0]; - dir->tail[1] = temptail[1]; - dir->split = tempsplit; - - // reset crc, hasfcrc - crc = 0xffffffff; - continue; - } - - // crc the entry first, hopefully leaving it in the cache - err = lfs_bd_crc(lfs, - NULL, &lfs->rcache, lfs->cfg->block_size, - dir->pair[0], off+sizeof(tag), - lfs_tag_dsize(tag)-sizeof(tag), &crc); - if (err) { - if (err == LFS_ERR_CORRUPT) { - break; - } - return err; - } - - // directory modification tags? - if (lfs_tag_type1(tag) == LFS_TYPE_NAME) { - // increase count of files if necessary - if (lfs_tag_id(tag) >= tempcount) { - tempcount = lfs_tag_id(tag) + 1; - } - } else if (lfs_tag_type1(tag) == LFS_TYPE_SPLICE) { - tempcount += lfs_tag_splice(tag); - - if (tag == (LFS_MKTAG(LFS_TYPE_DELETE, 0, 0) | - (LFS_MKTAG(0, 0x3ff, 0) & tempbesttag))) { - tempbesttag |= 0x80000000; - } else if (tempbesttag != -1 && - lfs_tag_id(tag) <= lfs_tag_id(tempbesttag)) { - tempbesttag += LFS_MKTAG(0, lfs_tag_splice(tag), 0); - } - } else if (lfs_tag_type1(tag) == LFS_TYPE_TAIL) { - tempsplit = (lfs_tag_chunk(tag) & 1); - - err = lfs_bd_read(lfs, - NULL, &lfs->rcache, lfs->cfg->block_size, - dir->pair[0], off+sizeof(tag), &temptail, 8); - if (err) { - if (err == LFS_ERR_CORRUPT) { - break; - } - return err; - } - lfs_pair_fromle32(temptail); - } else if (lfs_tag_type3(tag) == LFS_TYPE_FCRC) { - err = lfs_bd_read(lfs, - NULL, &lfs->rcache, lfs->cfg->block_size, - dir->pair[0], off+sizeof(tag), - &fcrc, sizeof(fcrc)); - if (err) { - if (err == LFS_ERR_CORRUPT) { - break; - } - return err; - } - - lfs_fcrc_fromle32(&fcrc); - hasfcrc = true; - } - - // found a match for our fetcher? - if ((fmask & tag) == (fmask & ftag)) { - LFS_ASSERT(cb != NULL); - int res = cb(data, tag, &(struct lfs_diskoff){ - dir->pair[0], off+sizeof(tag)}); - if (res < 0) { - if (res == LFS_ERR_CORRUPT) { - break; - } - return res; - } - - if (res == LFS_CMP_EQ) { - // found a match - tempbesttag = tag; - } else if ((LFS_MKTAG(0x7ff, 0x3ff, 0) & tag) == - (LFS_MKTAG(0x7ff, 0x3ff, 0) & tempbesttag)) { - // found an identical tag, but contents didn't match - // this must mean that our besttag has been overwritten - tempbesttag = -1; - } else if (res == LFS_CMP_GT && - lfs_tag_id(tag) <= lfs_tag_id(tempbesttag)) { - // found a greater match, keep track to keep things sorted - tempbesttag = tag | 0x80000000; - } - } - } - - // found no valid commits? - if (dir->off == 0) { - // try the other block? - lfs_pair_swap(dir->pair); - dir->rev = revs[(r+1)%2]; - continue; - } - - // did we end on a valid commit? we may have an erased block - dir->erased = false; - if (maybeerased && dir->off % lfs->cfg->prog_size == 0) { - #ifdef LFS_MULTIVERSION - // note versions < lfs2.1 did not have fcrc tags, if - // we're < lfs2.1 treat missing fcrc as erased data - // - // we don't strictly need to do this, but otherwise writing - // to lfs2.0 disks becomes very inefficient - if (lfs_fs_disk_version(lfs) < 0x00020001) { - dir->erased = true; - - } else - #endif - if (hasfcrc) { - // check for an fcrc matching the next prog's erased state, if - // this failed most likely a previous prog was interrupted, we - // need a new erase - uint32_t fcrc_ = 0xffffffff; - int err = lfs_bd_crc(lfs, - NULL, &lfs->rcache, lfs->cfg->block_size, - dir->pair[0], dir->off, fcrc.size, &fcrc_); - if (err && err != LFS_ERR_CORRUPT) { - return err; - } - - // found beginning of erased part? - dir->erased = (fcrc_ == fcrc.crc); - } - } - - // synthetic move - if (lfs_gstate_hasmovehere(&lfs->gdisk, dir->pair)) { - if (lfs_tag_id(lfs->gdisk.tag) == lfs_tag_id(besttag)) { - besttag |= 0x80000000; - } else if (besttag != -1 && - lfs_tag_id(lfs->gdisk.tag) < lfs_tag_id(besttag)) { - besttag -= LFS_MKTAG(0, 1, 0); - } - } - - // found tag? or found best id? - if (id) { - *id = lfs_min(lfs_tag_id(besttag), dir->count); - } - - if (lfs_tag_isvalid(besttag)) { - return besttag; - } else if (lfs_tag_id(besttag) < dir->count) { - return LFS_ERR_NOENT; - } else { - return 0; - } - } - - LFS_ERROR("Corrupted dir pair at {0x%"PRIx32", 0x%"PRIx32"}", - dir->pair[0], dir->pair[1]); - return LFS_ERR_CORRUPT; -} - -static int lfs_dir_fetch(lfs_t *lfs, - lfs_mdir_t *dir, const lfs_block_t pair[2]) { - // note, mask=-1, tag=-1 can never match a tag since this - // pattern has the invalid bit set - return (int)lfs_dir_fetchmatch(lfs, dir, pair, - (lfs_tag_t)-1, (lfs_tag_t)-1, NULL, NULL, NULL); -} - -static int lfs_dir_getgstate(lfs_t *lfs, const lfs_mdir_t *dir, - lfs_gstate_t *gstate) { - lfs_gstate_t temp; - lfs_stag_t res = lfs_dir_get(lfs, dir, LFS_MKTAG(0x7ff, 0, 0), - LFS_MKTAG(LFS_TYPE_MOVESTATE, 0, sizeof(temp)), &temp); - if (res < 0 && res != LFS_ERR_NOENT) { - return res; - } - - if (res != LFS_ERR_NOENT) { - // xor together to find resulting gstate - lfs_gstate_fromle32(&temp); - lfs_gstate_xor(gstate, &temp); - } - - return 0; -} - -static int lfs_dir_getinfo(lfs_t *lfs, lfs_mdir_t *dir, - uint16_t id, struct lfs_info *info) { - if (id == 0x3ff) { - // special case for root - strcpy(info->name, "/"); - info->type = LFS_TYPE_DIR; - return 0; - } - - lfs_stag_t tag = lfs_dir_get(lfs, dir, LFS_MKTAG(0x780, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_NAME, id, lfs->name_max+1), info->name); - if (tag < 0) { - return (int)tag; - } - - info->type = lfs_tag_type3(tag); - - struct lfs_ctz ctz; - tag = lfs_dir_get(lfs, dir, LFS_MKTAG(0x700, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_STRUCT, id, sizeof(ctz)), &ctz); - if (tag < 0) { - return (int)tag; - } - lfs_ctz_fromle32(&ctz); - - if (lfs_tag_type3(tag) == LFS_TYPE_CTZSTRUCT) { - info->size = ctz.size; - } else if (lfs_tag_type3(tag) == LFS_TYPE_INLINESTRUCT) { - info->size = lfs_tag_size(tag); - } - - return 0; -} - -struct lfs_dir_find_match { - lfs_t *lfs; - const void *name; - lfs_size_t size; -}; - -static int lfs_dir_find_match(void *data, - lfs_tag_t tag, const void *buffer) { - struct lfs_dir_find_match *name = data; - lfs_t *lfs = name->lfs; - const struct lfs_diskoff *disk = buffer; - - // compare with disk - lfs_size_t diff = lfs_min(name->size, lfs_tag_size(tag)); - int res = lfs_bd_cmp(lfs, - NULL, &lfs->rcache, diff, - disk->block, disk->off, name->name, diff); - if (res != LFS_CMP_EQ) { - return res; - } - - // only equal if our size is still the same - if (name->size != lfs_tag_size(tag)) { - return (name->size < lfs_tag_size(tag)) ? LFS_CMP_LT : LFS_CMP_GT; - } - - // found a match! - return LFS_CMP_EQ; -} - -// lfs_dir_find tries to set path and id even if file is not found -// -// returns: -// - 0 if file is found -// - LFS_ERR_NOENT if file or parent is not found -// - LFS_ERR_NOTDIR if parent is not a dir -static lfs_stag_t lfs_dir_find(lfs_t *lfs, lfs_mdir_t *dir, - const char **path, uint16_t *id) { - // we reduce path to a single name if we can find it - const char *name = *path; - - // default to root dir - lfs_stag_t tag = LFS_MKTAG(LFS_TYPE_DIR, 0x3ff, 0); - dir->tail[0] = lfs->root[0]; - dir->tail[1] = lfs->root[1]; - - // empty paths are not allowed - if (*name == '\0') { - return LFS_ERR_INVAL; - } - - while (true) { -nextname: - // skip slashes if we're a directory - if (lfs_tag_type3(tag) == LFS_TYPE_DIR) { - name += strspn(name, "/"); - } - lfs_size_t namelen = (lfs_size_t)strcspn(name, "/"); - - // skip '.' - if (namelen == 1 && memcmp(name, ".", 1) == 0) { - name += namelen; - goto nextname; - } - - // error on unmatched '..', trying to go above root? - if (namelen == 2 && memcmp(name, "..", 2) == 0) { - return LFS_ERR_INVAL; - } - - // skip if matched by '..' in name - const char *suffix = name + namelen; - lfs_size_t sufflen; - int depth = 1; - while (true) { - suffix += strspn(suffix, "/"); - sufflen = (lfs_size_t)strcspn(suffix, "/"); - if (sufflen == 0) { - break; - } - - if (sufflen == 1 && memcmp(suffix, ".", 1) == 0) { - // noop - } else if (sufflen == 2 && memcmp(suffix, "..", 2) == 0) { - depth -= 1; - if (depth == 0) { - name = suffix + sufflen; - goto nextname; - } - } else { - depth += 1; - } - - suffix += sufflen; - } - - // found path - if (*name == '\0') { - return tag; - } - - // update what we've found so far - *path = name; - - // only continue if we're a directory - if (lfs_tag_type3(tag) != LFS_TYPE_DIR) { - return LFS_ERR_NOTDIR; - } - - // grab the entry data - if (lfs_tag_id(tag) != 0x3ff) { - lfs_stag_t res = lfs_dir_get(lfs, dir, LFS_MKTAG(0x700, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_STRUCT, lfs_tag_id(tag), 8), dir->tail); - if (res < 0) { - return res; - } - lfs_pair_fromle32(dir->tail); - } - - // find entry matching name - while (true) { - tag = lfs_dir_fetchmatch(lfs, dir, dir->tail, - LFS_MKTAG(0x780, 0, 0), - LFS_MKTAG(LFS_TYPE_NAME, 0, namelen), - id, - lfs_dir_find_match, &(struct lfs_dir_find_match){ - lfs, name, namelen}); - if (tag < 0) { - return tag; - } - - if (tag) { - break; - } - - if (!dir->split) { - return LFS_ERR_NOENT; - } - } - - // to next name - name += namelen; - } -} - -// commit logic -struct lfs_commit { - lfs_block_t block; - lfs_off_t off; - lfs_tag_t ptag; - uint32_t crc; - - lfs_off_t begin; - lfs_off_t end; -}; - -#ifndef LFS_READONLY -static int lfs_dir_commitprog(lfs_t *lfs, struct lfs_commit *commit, - const void *buffer, lfs_size_t size) { - int err = lfs_bd_prog(lfs, - &lfs->pcache, &lfs->rcache, false, - commit->block, commit->off , - (const uint8_t*)buffer, size); - if (err) { - return err; - } - - commit->crc = lfs_crc(commit->crc, buffer, size); - commit->off += size; - return 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_dir_commitattr(lfs_t *lfs, struct lfs_commit *commit, - lfs_tag_t tag, const void *buffer) { - // check if we fit - lfs_size_t dsize = lfs_tag_dsize(tag); - if (commit->off + dsize > commit->end) { - return LFS_ERR_NOSPC; - } - - // write out tag - lfs_tag_t ntag = lfs_tobe32((tag & 0x7fffffff) ^ commit->ptag); - int err = lfs_dir_commitprog(lfs, commit, &ntag, sizeof(ntag)); - if (err) { - return err; - } - - if (!(tag & 0x80000000)) { - // from memory - err = lfs_dir_commitprog(lfs, commit, buffer, dsize-sizeof(tag)); - if (err) { - return err; - } - } else { - // from disk - const struct lfs_diskoff *disk = buffer; - for (lfs_off_t i = 0; i < dsize-sizeof(tag); i++) { - // rely on caching to make this efficient - uint8_t dat; - err = lfs_bd_read(lfs, - NULL, &lfs->rcache, dsize-sizeof(tag)-i, - disk->block, disk->off+i, &dat, 1); - if (err) { - return err; - } - - err = lfs_dir_commitprog(lfs, commit, &dat, 1); - if (err) { - return err; - } - } - } - - commit->ptag = tag & 0x7fffffff; - return 0; -} -#endif - -#ifndef LFS_READONLY - -static int lfs_dir_commitcrc(lfs_t *lfs, struct lfs_commit *commit) { - // align to program units - // - // this gets a bit complex as we have two types of crcs: - // - 5-word crc with fcrc to check following prog (middle of block) - // - 2-word crc with no following prog (end of block) - const lfs_off_t end = lfs_alignup( - lfs_min(commit->off + 5*sizeof(uint32_t), lfs->cfg->block_size), - lfs->cfg->prog_size); - - lfs_off_t off1 = 0; - uint32_t crc1 = 0; - - // create crc tags to fill up remainder of commit, note that - // padding is not crced, which lets fetches skip padding but - // makes committing a bit more complicated - while (commit->off < end) { - lfs_off_t noff = ( - lfs_min(end - (commit->off+sizeof(lfs_tag_t)), 0x3fe) - + (commit->off+sizeof(lfs_tag_t))); - // too large for crc tag? need padding commits - if (noff < end) { - noff = lfs_min(noff, end - 5*sizeof(uint32_t)); - } - - // space for fcrc? - uint8_t eperturb = (uint8_t)-1; - if (noff >= end && noff <= lfs->cfg->block_size - lfs->cfg->prog_size) { - // first read the leading byte, this always contains a bit - // we can perturb to avoid writes that don't change the fcrc - int err = lfs_bd_read(lfs, - NULL, &lfs->rcache, lfs->cfg->prog_size, - commit->block, noff, &eperturb, 1); - if (err && err != LFS_ERR_CORRUPT) { - return err; - } - - #ifdef LFS_MULTIVERSION - // unfortunately fcrcs break mdir fetching < lfs2.1, so only write - // these if we're a >= lfs2.1 filesystem - if (lfs_fs_disk_version(lfs) <= 0x00020000) { - // don't write fcrc - } else - #endif - { - // find the expected fcrc, don't bother avoiding a reread - // of the eperturb, it should still be in our cache - struct lfs_fcrc fcrc = { - .size = lfs->cfg->prog_size, - .crc = 0xffffffff - }; - err = lfs_bd_crc(lfs, - NULL, &lfs->rcache, lfs->cfg->prog_size, - commit->block, noff, fcrc.size, &fcrc.crc); - if (err && err != LFS_ERR_CORRUPT) { - return err; - } - - lfs_fcrc_tole32(&fcrc); - err = lfs_dir_commitattr(lfs, commit, - LFS_MKTAG(LFS_TYPE_FCRC, 0x3ff, sizeof(struct lfs_fcrc)), - &fcrc); - if (err) { - return err; - } - } - } - - // build commit crc - struct { - lfs_tag_t tag; - uint32_t crc; - } ccrc; - lfs_tag_t ntag = LFS_MKTAG( - LFS_TYPE_CCRC + (((uint8_t)~eperturb) >> 7), 0x3ff, - noff - (commit->off+sizeof(lfs_tag_t))); - ccrc.tag = lfs_tobe32(ntag ^ commit->ptag); - commit->crc = lfs_crc(commit->crc, &ccrc.tag, sizeof(lfs_tag_t)); - ccrc.crc = lfs_tole32(commit->crc); - - int err = lfs_bd_prog(lfs, - &lfs->pcache, &lfs->rcache, false, - commit->block, commit->off, &ccrc, sizeof(ccrc)); - if (err) { - return err; - } - - // keep track of non-padding checksum to verify - if (off1 == 0) { - off1 = commit->off + sizeof(lfs_tag_t); - crc1 = commit->crc; - } - - commit->off = noff; - // perturb valid bit? - commit->ptag = ntag ^ ((lfs_tag_t)(0x80 & ~eperturb) << 24); - // reset crc for next commit - commit->crc = 0xffffffff; - - // manually flush here since we don't prog the padding, this confuses - // the caching layer - if (noff >= end || noff >= lfs->pcache.off + lfs->cfg->cache_size) { - // flush buffers - int err = lfs_bd_sync(lfs, &lfs->pcache, &lfs->rcache, false); - if (err) { - return err; - } - } - } - - // successful commit, check checksums to make sure - // - // note that we don't need to check padding commits, worst - // case if they are corrupted we would have had to compact anyways - lfs_off_t off = commit->begin; - uint32_t crc = 0xffffffff; - int err = lfs_bd_crc(lfs, - NULL, &lfs->rcache, off1+sizeof(uint32_t), - commit->block, off, off1-off, &crc); - if (err) { - return err; - } - - // check non-padding commits against known crc - if (crc != crc1) { - return LFS_ERR_CORRUPT; - } - - // make sure to check crc in case we happen to pick - // up an unrelated crc (frozen block?) - err = lfs_bd_crc(lfs, - NULL, &lfs->rcache, sizeof(uint32_t), - commit->block, off1, sizeof(uint32_t), &crc); - if (err) { - return err; - } - - if (crc != 0) { - return LFS_ERR_CORRUPT; - } - - return 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_dir_alloc(lfs_t *lfs, lfs_mdir_t *dir) { - // allocate pair of dir blocks (backwards, so we write block 1 first) - for (int i = 0; i < 2; i++) { - int err = lfs_alloc(lfs, &dir->pair[(i+1)%2]); - if (err) { - return err; - } - } - - // zero for reproducibility in case initial block is unreadable - dir->rev = 0; - - // rather than clobbering one of the blocks we just pretend - // the revision may be valid - int err = lfs_bd_read(lfs, - NULL, &lfs->rcache, sizeof(dir->rev), - dir->pair[0], 0, &dir->rev, sizeof(dir->rev)); - dir->rev = lfs_fromle32(dir->rev); - if (err && err != LFS_ERR_CORRUPT) { - return err; - } - - // to make sure we don't immediately evict, align the new revision count - // to our block_cycles modulus, see lfs_dir_compact for why our modulus - // is tweaked this way - if (lfs->cfg->block_cycles > 0) { - dir->rev = lfs_alignup(dir->rev, ((lfs->cfg->block_cycles+1)|1)); - } - - // set defaults - dir->off = sizeof(dir->rev); - dir->etag = 0xffffffff; - dir->count = 0; - dir->tail[0] = LFS_BLOCK_NULL; - dir->tail[1] = LFS_BLOCK_NULL; - dir->erased = false; - dir->split = false; - - // don't write out yet, let caller take care of that - return 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_dir_drop(lfs_t *lfs, lfs_mdir_t *dir, lfs_mdir_t *tail) { - // steal state - int err = lfs_dir_getgstate(lfs, tail, &lfs->gdelta); - if (err) { - return err; - } - - // steal tail - lfs_pair_tole32(tail->tail); - err = lfs_dir_commit(lfs, dir, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_TAIL + tail->split, 0x3ff, 8), tail->tail})); - lfs_pair_fromle32(tail->tail); - if (err) { - return err; - } - - return 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_dir_split(lfs_t *lfs, - lfs_mdir_t *dir, const struct lfs_mattr *attrs, int attrcount, - lfs_mdir_t *source, uint16_t split, uint16_t end) { - // create tail metadata pair - lfs_mdir_t tail; - int err = lfs_dir_alloc(lfs, &tail); - if (err) { - return err; - } - - tail.split = dir->split; - tail.tail[0] = dir->tail[0]; - tail.tail[1] = dir->tail[1]; - - // note we don't care about LFS_OK_RELOCATED - int res = lfs_dir_compact(lfs, &tail, attrs, attrcount, source, split, end); - if (res < 0) { - return res; - } - - dir->tail[0] = tail.pair[0]; - dir->tail[1] = tail.pair[1]; - dir->split = true; - - // update root if needed - if (lfs_pair_cmp(dir->pair, lfs->root) == 0 && split == 0) { - lfs->root[0] = tail.pair[0]; - lfs->root[1] = tail.pair[1]; - } - - return 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_dir_commit_size(void *p, lfs_tag_t tag, const void *buffer) { - lfs_size_t *size = p; - (void)buffer; - - *size += lfs_tag_dsize(tag); - return 0; -} -#endif - -#ifndef LFS_READONLY -struct lfs_dir_commit_commit { - lfs_t *lfs; - struct lfs_commit *commit; -}; -#endif - -#ifndef LFS_READONLY -static int lfs_dir_commit_commit(void *p, lfs_tag_t tag, const void *buffer) { - struct lfs_dir_commit_commit *commit = p; - return lfs_dir_commitattr(commit->lfs, commit->commit, tag, buffer); -} -#endif - -#ifndef LFS_READONLY -static bool lfs_dir_needsrelocation(lfs_t *lfs, lfs_mdir_t *dir) { - // If our revision count == n * block_cycles, we should force a relocation, - // this is how littlefs wear-levels at the metadata-pair level. Note that we - // actually use (block_cycles+1)|1, this is to avoid two corner cases: - // 1. block_cycles = 1, which would prevent relocations from terminating - // 2. block_cycles = 2n, which, due to aliasing, would only ever relocate - // one metadata block in the pair, effectively making this useless - return (lfs->cfg->block_cycles > 0 - && ((dir->rev + 1) % ((lfs->cfg->block_cycles+1)|1) == 0)); -} -#endif - -#ifndef LFS_READONLY -static int lfs_dir_compact(lfs_t *lfs, - lfs_mdir_t *dir, const struct lfs_mattr *attrs, int attrcount, - lfs_mdir_t *source, uint16_t begin, uint16_t end) { - // save some state in case block is bad - bool relocated = false; - bool tired = lfs_dir_needsrelocation(lfs, dir); - - // increment revision count - dir->rev += 1; - - // do not proactively relocate blocks during migrations, this - // can cause a number of failure states such: clobbering the - // v1 superblock if we relocate root, and invalidating directory - // pointers if we relocate the head of a directory. On top of - // this, relocations increase the overall complexity of - // lfs_migration, which is already a delicate operation. -#ifdef LFS_MIGRATE - if (lfs->lfs1) { - tired = false; - } -#endif - - if (tired && lfs_pair_cmp(dir->pair, (const lfs_block_t[2]){0, 1}) != 0) { - // we're writing too much, time to relocate - goto relocate; - } - - // begin loop to commit compaction to blocks until a compact sticks - while (true) { - { - // setup commit state - struct lfs_commit commit = { - .block = dir->pair[1], - .off = 0, - .ptag = 0xffffffff, - .crc = 0xffffffff, - - .begin = 0, - .end = (lfs->cfg->metadata_max ? - lfs->cfg->metadata_max : lfs->cfg->block_size) - 8, - }; - - // erase block to write to - int err = lfs_bd_erase(lfs, dir->pair[1]); - if (err) { - if (err == LFS_ERR_CORRUPT) { - goto relocate; - } - return err; - } - - // write out header - dir->rev = lfs_tole32(dir->rev); - err = lfs_dir_commitprog(lfs, &commit, - &dir->rev, sizeof(dir->rev)); - dir->rev = lfs_fromle32(dir->rev); - if (err) { - if (err == LFS_ERR_CORRUPT) { - goto relocate; - } - return err; - } - - // traverse the directory, this time writing out all unique tags - err = lfs_dir_traverse(lfs, - source, 0, 0xffffffff, attrs, attrcount, - LFS_MKTAG(0x400, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_NAME, 0, 0), - begin, end, -begin, - lfs_dir_commit_commit, &(struct lfs_dir_commit_commit){ - lfs, &commit}); - if (err) { - if (err == LFS_ERR_CORRUPT) { - goto relocate; - } - return err; - } - - // commit tail, which may be new after last size check - if (!lfs_pair_isnull(dir->tail)) { - lfs_pair_tole32(dir->tail); - err = lfs_dir_commitattr(lfs, &commit, - LFS_MKTAG(LFS_TYPE_TAIL + dir->split, 0x3ff, 8), - dir->tail); - lfs_pair_fromle32(dir->tail); - if (err) { - if (err == LFS_ERR_CORRUPT) { - goto relocate; - } - return err; - } - } - - // bring over gstate? - lfs_gstate_t delta = {0}; - if (!relocated) { - lfs_gstate_xor(&delta, &lfs->gdisk); - lfs_gstate_xor(&delta, &lfs->gstate); - } - lfs_gstate_xor(&delta, &lfs->gdelta); - delta.tag &= ~LFS_MKTAG(0, 0, 0x3ff); - - err = lfs_dir_getgstate(lfs, dir, &delta); - if (err) { - return err; - } - - if (!lfs_gstate_iszero(&delta)) { - lfs_gstate_tole32(&delta); - err = lfs_dir_commitattr(lfs, &commit, - LFS_MKTAG(LFS_TYPE_MOVESTATE, 0x3ff, - sizeof(delta)), &delta); - if (err) { - if (err == LFS_ERR_CORRUPT) { - goto relocate; - } - return err; - } - } - - // complete commit with crc - err = lfs_dir_commitcrc(lfs, &commit); - if (err) { - if (err == LFS_ERR_CORRUPT) { - goto relocate; - } - return err; - } - - // successful compaction, swap dir pair to indicate most recent - LFS_ASSERT(commit.off % lfs->cfg->prog_size == 0); - lfs_pair_swap(dir->pair); - dir->count = end - begin; - dir->off = commit.off; - dir->etag = commit.ptag; - // update gstate - lfs->gdelta = (lfs_gstate_t){0}; - if (!relocated) { - lfs->gdisk = lfs->gstate; - } - } - break; - -relocate: - // commit was corrupted, drop caches and prepare to relocate block - relocated = true; - lfs_cache_drop(lfs, &lfs->pcache); - if (!tired) { - LFS_DEBUG("Bad block at 0x%"PRIx32, dir->pair[1]); - } - - // can't relocate superblock, filesystem is now frozen - if (lfs_pair_cmp(dir->pair, (const lfs_block_t[2]){0, 1}) == 0) { - LFS_WARN("Superblock 0x%"PRIx32" has become unwritable", - dir->pair[1]); - return LFS_ERR_NOSPC; - } - - // relocate half of pair - int err = lfs_alloc(lfs, &dir->pair[1]); - if (err && (err != LFS_ERR_NOSPC || !tired)) { - return err; - } - - tired = false; - continue; - } - - return relocated ? LFS_OK_RELOCATED : 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_dir_splittingcompact(lfs_t *lfs, lfs_mdir_t *dir, - const struct lfs_mattr *attrs, int attrcount, - lfs_mdir_t *source, uint16_t begin, uint16_t end) { - while (true) { - // find size of first split, we do this by halving the split until - // the metadata is guaranteed to fit - // - // Note that this isn't a true binary search, we never increase the - // split size. This may result in poorly distributed metadata but isn't - // worth the extra code size or performance hit to fix. - lfs_size_t split = begin; - while (end - split > 1) { - lfs_size_t size = 0; - int err = lfs_dir_traverse(lfs, - source, 0, 0xffffffff, attrs, attrcount, - LFS_MKTAG(0x400, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_NAME, 0, 0), - split, end, -split, - lfs_dir_commit_size, &size); - if (err) { - return err; - } - - // space is complicated, we need room for: - // - // - tail: 4+2*4 = 12 bytes - // - gstate: 4+3*4 = 16 bytes - // - move delete: 4 = 4 bytes - // - crc: 4+4 = 8 bytes - // total = 40 bytes - // - // And we cap at half a block to avoid degenerate cases with - // nearly-full metadata blocks. - // - lfs_size_t metadata_max = (lfs->cfg->metadata_max) - ? lfs->cfg->metadata_max - : lfs->cfg->block_size; - if (end - split < 0xff - && size <= lfs_min( - metadata_max - 40, - lfs_alignup( - metadata_max/2, - lfs->cfg->prog_size))) { - break; - } - - split = split + ((end - split) / 2); - } - - if (split == begin) { - // no split needed - break; - } - - // split into two metadata pairs and continue - int err = lfs_dir_split(lfs, dir, attrs, attrcount, - source, split, end); - if (err && err != LFS_ERR_NOSPC) { - return err; - } - - if (err) { - // we can't allocate a new block, try to compact with degraded - // performance - LFS_WARN("Unable to split {0x%"PRIx32", 0x%"PRIx32"}", - dir->pair[0], dir->pair[1]); - break; - } else { - end = split; - } - } - - if (lfs_dir_needsrelocation(lfs, dir) - && lfs_pair_cmp(dir->pair, (const lfs_block_t[2]){0, 1}) == 0) { - // oh no! we're writing too much to the superblock, - // should we expand? - lfs_ssize_t size = lfs_fs_size_(lfs); - if (size < 0) { - return size; - } - - // littlefs cannot reclaim expanded superblocks, so expand cautiously - // - // if our filesystem is more than ~88% full, don't expand, this is - // somewhat arbitrary - if (lfs->block_count - size > lfs->block_count/8) { - LFS_DEBUG("Expanding superblock at rev %"PRIu32, dir->rev); - int err = lfs_dir_split(lfs, dir, attrs, attrcount, - source, begin, end); - if (err && err != LFS_ERR_NOSPC) { - return err; - } - - if (err) { - // welp, we tried, if we ran out of space there's not much - // we can do, we'll error later if we've become frozen - LFS_WARN("Unable to expand superblock"); - } else { - // duplicate the superblock entry into the new superblock - end = 1; - } - } - } - - return lfs_dir_compact(lfs, dir, attrs, attrcount, source, begin, end); -} -#endif - -#ifndef LFS_READONLY -static int lfs_dir_relocatingcommit(lfs_t *lfs, lfs_mdir_t *dir, - const lfs_block_t pair[2], - const struct lfs_mattr *attrs, int attrcount, - lfs_mdir_t *pdir) { - int state = 0; - - // calculate changes to the directory - bool hasdelete = false; - for (int i = 0; i < attrcount; i++) { - if (lfs_tag_type3(attrs[i].tag) == LFS_TYPE_CREATE) { - dir->count += 1; - } else if (lfs_tag_type3(attrs[i].tag) == LFS_TYPE_DELETE) { - LFS_ASSERT(dir->count > 0); - dir->count -= 1; - hasdelete = true; - } else if (lfs_tag_type1(attrs[i].tag) == LFS_TYPE_TAIL) { - dir->tail[0] = ((lfs_block_t*)attrs[i].buffer)[0]; - dir->tail[1] = ((lfs_block_t*)attrs[i].buffer)[1]; - dir->split = (lfs_tag_chunk(attrs[i].tag) & 1); - lfs_pair_fromle32(dir->tail); - } - } - - // should we actually drop the directory block? - if (hasdelete && dir->count == 0) { - LFS_ASSERT(pdir); - int err = lfs_fs_pred(lfs, dir->pair, pdir); - if (err && err != LFS_ERR_NOENT) { - return err; - } - - if (err != LFS_ERR_NOENT && pdir->split) { - state = LFS_OK_DROPPED; - goto fixmlist; - } - } - - if (dir->erased && dir->count < 0xff) { - // try to commit - struct lfs_commit commit = { - .block = dir->pair[0], - .off = dir->off, - .ptag = dir->etag, - .crc = 0xffffffff, - - .begin = dir->off, - .end = (lfs->cfg->metadata_max ? - lfs->cfg->metadata_max : lfs->cfg->block_size) - 8, - }; - - // traverse attrs that need to be written out - lfs_pair_tole32(dir->tail); - int err = lfs_dir_traverse(lfs, - dir, dir->off, dir->etag, attrs, attrcount, - 0, 0, 0, 0, 0, - lfs_dir_commit_commit, &(struct lfs_dir_commit_commit){ - lfs, &commit}); - lfs_pair_fromle32(dir->tail); - if (err) { - if (err == LFS_ERR_NOSPC || err == LFS_ERR_CORRUPT) { - goto compact; - } - return err; - } - - // commit any global diffs if we have any - lfs_gstate_t delta = {0}; - lfs_gstate_xor(&delta, &lfs->gstate); - lfs_gstate_xor(&delta, &lfs->gdisk); - lfs_gstate_xor(&delta, &lfs->gdelta); - delta.tag &= ~LFS_MKTAG(0, 0, 0x3ff); - if (!lfs_gstate_iszero(&delta)) { - err = lfs_dir_getgstate(lfs, dir, &delta); - if (err) { - return err; - } - - lfs_gstate_tole32(&delta); - err = lfs_dir_commitattr(lfs, &commit, - LFS_MKTAG(LFS_TYPE_MOVESTATE, 0x3ff, - sizeof(delta)), &delta); - if (err) { - if (err == LFS_ERR_NOSPC || err == LFS_ERR_CORRUPT) { - goto compact; - } - return err; - } - } - - // finalize commit with the crc - err = lfs_dir_commitcrc(lfs, &commit); - if (err) { - if (err == LFS_ERR_NOSPC || err == LFS_ERR_CORRUPT) { - goto compact; - } - return err; - } - - // successful commit, update dir - LFS_ASSERT(commit.off % lfs->cfg->prog_size == 0); - dir->off = commit.off; - dir->etag = commit.ptag; - // and update gstate - lfs->gdisk = lfs->gstate; - lfs->gdelta = (lfs_gstate_t){0}; - - goto fixmlist; - } - -compact: - // fall back to compaction - lfs_cache_drop(lfs, &lfs->pcache); - - state = lfs_dir_splittingcompact(lfs, dir, attrs, attrcount, - dir, 0, dir->count); - if (state < 0) { - return state; - } - - goto fixmlist; - -fixmlist:; - // this complicated bit of logic is for fixing up any active - // metadata-pairs that we may have affected - // - // note we have to make two passes since the mdir passed to - // lfs_dir_commit could also be in this list, and even then - // we need to copy the pair so they don't get clobbered if we refetch - // our mdir. - lfs_block_t oldpair[2] = {pair[0], pair[1]}; - for (struct lfs_mlist *d = lfs->mlist; d; d = d->next) { - if (lfs_pair_cmp(d->m.pair, oldpair) == 0) { - d->m = *dir; - if (d->m.pair != pair) { - for (int i = 0; i < attrcount; i++) { - if (lfs_tag_type3(attrs[i].tag) == LFS_TYPE_DELETE && - d->id == lfs_tag_id(attrs[i].tag) && - d->type != LFS_TYPE_DIR) { - d->m.pair[0] = LFS_BLOCK_NULL; - d->m.pair[1] = LFS_BLOCK_NULL; - } else if (lfs_tag_type3(attrs[i].tag) == LFS_TYPE_DELETE && - d->id > lfs_tag_id(attrs[i].tag)) { - d->id -= 1; - if (d->type == LFS_TYPE_DIR) { - ((lfs_dir_t*)d)->pos -= 1; - } - } else if (lfs_tag_type3(attrs[i].tag) == LFS_TYPE_CREATE && - d->id >= lfs_tag_id(attrs[i].tag)) { - d->id += 1; - if (d->type == LFS_TYPE_DIR) { - ((lfs_dir_t*)d)->pos += 1; - } - } - } - } - - while (d->id >= d->m.count && d->m.split) { - // we split and id is on tail now - if (lfs_pair_cmp(d->m.tail, lfs->root) != 0) { - d->id -= d->m.count; - } - int err = lfs_dir_fetch(lfs, &d->m, d->m.tail); - if (err) { - return err; - } - } - } - } - - return state; -} -#endif - -#ifndef LFS_READONLY -static int lfs_dir_orphaningcommit(lfs_t *lfs, lfs_mdir_t *dir, - const struct lfs_mattr *attrs, int attrcount) { - // check for any inline files that aren't RAM backed and - // forcefully evict them, needed for filesystem consistency - for (lfs_file_t *f = (lfs_file_t*)lfs->mlist; f; f = f->next) { - if (dir != &f->m && lfs_pair_cmp(f->m.pair, dir->pair) == 0 && - f->type == LFS_TYPE_REG && (f->flags & LFS_F_INLINE) && - f->ctz.size > lfs->cfg->cache_size) { - int err = lfs_file_outline(lfs, f); - if (err) { - return err; - } - - err = lfs_file_flush(lfs, f); - if (err) { - return err; - } - } - } - - lfs_block_t lpair[2] = {dir->pair[0], dir->pair[1]}; - lfs_mdir_t ldir = *dir; - lfs_mdir_t pdir; - int state = lfs_dir_relocatingcommit(lfs, &ldir, dir->pair, - attrs, attrcount, &pdir); - if (state < 0) { - return state; - } - - // update if we're not in mlist, note we may have already been - // updated if we are in mlist - if (lfs_pair_cmp(dir->pair, lpair) == 0) { - *dir = ldir; - } - - // commit was successful, but may require other changes in the - // filesystem, these would normally be tail recursive, but we have - // flattened them here avoid unbounded stack usage - - // need to drop? - if (state == LFS_OK_DROPPED) { - // steal state - int err = lfs_dir_getgstate(lfs, dir, &lfs->gdelta); - if (err) { - return err; - } - - // steal tail, note that this can't create a recursive drop - lpair[0] = pdir.pair[0]; - lpair[1] = pdir.pair[1]; - lfs_pair_tole32(dir->tail); - state = lfs_dir_relocatingcommit(lfs, &pdir, lpair, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_TAIL + dir->split, 0x3ff, 8), - dir->tail}), - NULL); - lfs_pair_fromle32(dir->tail); - if (state < 0) { - return state; - } - - ldir = pdir; - } - - // need to relocate? - bool orphans = false; - while (state == LFS_OK_RELOCATED) { - LFS_DEBUG("Relocating {0x%"PRIx32", 0x%"PRIx32"} " - "-> {0x%"PRIx32", 0x%"PRIx32"}", - lpair[0], lpair[1], ldir.pair[0], ldir.pair[1]); - state = 0; - - // update internal root - if (lfs_pair_cmp(lpair, lfs->root) == 0) { - lfs->root[0] = ldir.pair[0]; - lfs->root[1] = ldir.pair[1]; - } - - // update internally tracked dirs - for (struct lfs_mlist *d = lfs->mlist; d; d = d->next) { - if (lfs_pair_cmp(lpair, d->m.pair) == 0) { - d->m.pair[0] = ldir.pair[0]; - d->m.pair[1] = ldir.pair[1]; - } - - if (d->type == LFS_TYPE_DIR && - lfs_pair_cmp(lpair, ((lfs_dir_t*)d)->head) == 0) { - ((lfs_dir_t*)d)->head[0] = ldir.pair[0]; - ((lfs_dir_t*)d)->head[1] = ldir.pair[1]; - } - } - - // find parent - lfs_stag_t tag = lfs_fs_parent(lfs, lpair, &pdir); - if (tag < 0 && tag != LFS_ERR_NOENT) { - return tag; - } - - bool hasparent = (tag != LFS_ERR_NOENT); - if (tag != LFS_ERR_NOENT) { - // note that if we have a parent, we must have a pred, so this will - // always create an orphan - int err = lfs_fs_preporphans(lfs, +1); - if (err) { - return err; - } - - // fix pending move in this pair? this looks like an optimization but - // is in fact _required_ since relocating may outdate the move. - uint16_t moveid = 0x3ff; - if (lfs_gstate_hasmovehere(&lfs->gstate, pdir.pair)) { - moveid = lfs_tag_id(lfs->gstate.tag); - LFS_DEBUG("Fixing move while relocating " - "{0x%"PRIx32", 0x%"PRIx32"} 0x%"PRIx16"\n", - pdir.pair[0], pdir.pair[1], moveid); - lfs_fs_prepmove(lfs, 0x3ff, NULL); - if (moveid < lfs_tag_id(tag)) { - tag -= LFS_MKTAG(0, 1, 0); - } - } - - lfs_block_t ppair[2] = {pdir.pair[0], pdir.pair[1]}; - lfs_pair_tole32(ldir.pair); - state = lfs_dir_relocatingcommit(lfs, &pdir, ppair, LFS_MKATTRS( - {LFS_MKTAG_IF(moveid != 0x3ff, - LFS_TYPE_DELETE, moveid, 0), NULL}, - {tag, ldir.pair}), - NULL); - lfs_pair_fromle32(ldir.pair); - if (state < 0) { - return state; - } - - if (state == LFS_OK_RELOCATED) { - lpair[0] = ppair[0]; - lpair[1] = ppair[1]; - ldir = pdir; - orphans = true; - continue; - } - } - - // find pred - int err = lfs_fs_pred(lfs, lpair, &pdir); - if (err && err != LFS_ERR_NOENT) { - return err; - } - LFS_ASSERT(!(hasparent && err == LFS_ERR_NOENT)); - - // if we can't find dir, it must be new - if (err != LFS_ERR_NOENT) { - if (lfs_gstate_hasorphans(&lfs->gstate)) { - // next step, clean up orphans - err = lfs_fs_preporphans(lfs, -(int8_t)hasparent); - if (err) { - return err; - } - } - - // fix pending move in this pair? this looks like an optimization - // but is in fact _required_ since relocating may outdate the move. - uint16_t moveid = 0x3ff; - if (lfs_gstate_hasmovehere(&lfs->gstate, pdir.pair)) { - moveid = lfs_tag_id(lfs->gstate.tag); - LFS_DEBUG("Fixing move while relocating " - "{0x%"PRIx32", 0x%"PRIx32"} 0x%"PRIx16"\n", - pdir.pair[0], pdir.pair[1], moveid); - lfs_fs_prepmove(lfs, 0x3ff, NULL); - } - - // replace bad pair, either we clean up desync, or no desync occured - lpair[0] = pdir.pair[0]; - lpair[1] = pdir.pair[1]; - lfs_pair_tole32(ldir.pair); - state = lfs_dir_relocatingcommit(lfs, &pdir, lpair, LFS_MKATTRS( - {LFS_MKTAG_IF(moveid != 0x3ff, - LFS_TYPE_DELETE, moveid, 0), NULL}, - {LFS_MKTAG(LFS_TYPE_TAIL + pdir.split, 0x3ff, 8), - ldir.pair}), - NULL); - lfs_pair_fromle32(ldir.pair); - if (state < 0) { - return state; - } - - ldir = pdir; - } - } - - return orphans ? LFS_OK_ORPHANED : 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_dir_commit(lfs_t *lfs, lfs_mdir_t *dir, - const struct lfs_mattr *attrs, int attrcount) { - int orphans = lfs_dir_orphaningcommit(lfs, dir, attrs, attrcount); - if (orphans < 0) { - return orphans; - } - - if (orphans) { - // make sure we've removed all orphans, this is a noop if there - // are none, but if we had nested blocks failures we may have - // created some - int err = lfs_fs_deorphan(lfs, false); - if (err) { - return err; - } - } - - return 0; -} -#endif - - -/// Top level directory operations /// -#ifndef LFS_READONLY -static int lfs_mkdir_(lfs_t *lfs, const char *path) { - // deorphan if we haven't yet, needed at most once after poweron - int err = lfs_fs_forceconsistency(lfs); - if (err) { - return err; - } - - struct lfs_mlist cwd; - cwd.next = lfs->mlist; - uint16_t id; - err = lfs_dir_find(lfs, &cwd.m, &path, &id); - if (!(err == LFS_ERR_NOENT && lfs_path_islast(path))) { - return (err < 0) ? err : LFS_ERR_EXIST; - } - - // check that name fits - lfs_size_t nlen = lfs_path_namelen(path); - if (nlen > lfs->name_max) { - return LFS_ERR_NAMETOOLONG; - } - - // build up new directory - lfs_alloc_ckpoint(lfs); - lfs_mdir_t dir; - err = lfs_dir_alloc(lfs, &dir); - if (err) { - return err; - } - - // find end of list - lfs_mdir_t pred = cwd.m; - while (pred.split) { - err = lfs_dir_fetch(lfs, &pred, pred.tail); - if (err) { - return err; - } - } - - // setup dir - lfs_pair_tole32(pred.tail); - err = lfs_dir_commit(lfs, &dir, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_SOFTTAIL, 0x3ff, 8), pred.tail})); - lfs_pair_fromle32(pred.tail); - if (err) { - return err; - } - - // current block not end of list? - if (cwd.m.split) { - // update tails, this creates a desync - err = lfs_fs_preporphans(lfs, +1); - if (err) { - return err; - } - - // it's possible our predecessor has to be relocated, and if - // our parent is our predecessor's predecessor, this could have - // caused our parent to go out of date, fortunately we can hook - // ourselves into littlefs to catch this - cwd.type = 0; - cwd.id = 0; - lfs->mlist = &cwd; - - lfs_pair_tole32(dir.pair); - err = lfs_dir_commit(lfs, &pred, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_SOFTTAIL, 0x3ff, 8), dir.pair})); - lfs_pair_fromle32(dir.pair); - if (err) { - lfs->mlist = cwd.next; - return err; - } - - lfs->mlist = cwd.next; - err = lfs_fs_preporphans(lfs, -1); - if (err) { - return err; - } - } - - // now insert into our parent block - lfs_pair_tole32(dir.pair); - err = lfs_dir_commit(lfs, &cwd.m, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_CREATE, id, 0), NULL}, - {LFS_MKTAG(LFS_TYPE_DIR, id, nlen), path}, - {LFS_MKTAG(LFS_TYPE_DIRSTRUCT, id, 8), dir.pair}, - {LFS_MKTAG_IF(!cwd.m.split, - LFS_TYPE_SOFTTAIL, 0x3ff, 8), dir.pair})); - lfs_pair_fromle32(dir.pair); - if (err) { - return err; - } - - return 0; -} -#endif - -static int lfs_dir_open_(lfs_t *lfs, lfs_dir_t *dir, const char *path) { - lfs_stag_t tag = lfs_dir_find(lfs, &dir->m, &path, NULL); - if (tag < 0) { - return tag; - } - - if (lfs_tag_type3(tag) != LFS_TYPE_DIR) { - return LFS_ERR_NOTDIR; - } - - lfs_block_t pair[2]; - if (lfs_tag_id(tag) == 0x3ff) { - // handle root dir separately - pair[0] = lfs->root[0]; - pair[1] = lfs->root[1]; - } else { - // get dir pair from parent - lfs_stag_t res = lfs_dir_get(lfs, &dir->m, LFS_MKTAG(0x700, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_STRUCT, lfs_tag_id(tag), 8), pair); - if (res < 0) { - return res; - } - lfs_pair_fromle32(pair); - } - - // fetch first pair - int err = lfs_dir_fetch(lfs, &dir->m, pair); - if (err) { - return err; - } - - // setup entry - dir->head[0] = dir->m.pair[0]; - dir->head[1] = dir->m.pair[1]; - dir->id = 0; - dir->pos = 0; - - // add to list of mdirs - dir->type = LFS_TYPE_DIR; - lfs_mlist_append(lfs, (struct lfs_mlist *)dir); - - return 0; -} - -static int lfs_dir_close_(lfs_t *lfs, lfs_dir_t *dir) { - // remove from list of mdirs - lfs_mlist_remove(lfs, (struct lfs_mlist *)dir); - - return 0; -} - -static int lfs_dir_read_(lfs_t *lfs, lfs_dir_t *dir, struct lfs_info *info) { - memset(info, 0, sizeof(*info)); - - // special offset for '.' and '..' - if (dir->pos == 0) { - info->type = LFS_TYPE_DIR; - strcpy(info->name, "."); - dir->pos += 1; - return true; - } else if (dir->pos == 1) { - info->type = LFS_TYPE_DIR; - strcpy(info->name, ".."); - dir->pos += 1; - return true; - } - - while (true) { - if (dir->id == dir->m.count) { - if (!dir->m.split) { - return false; - } - - int err = lfs_dir_fetch(lfs, &dir->m, dir->m.tail); - if (err) { - return err; - } - - dir->id = 0; - } - - int err = lfs_dir_getinfo(lfs, &dir->m, dir->id, info); - if (err && err != LFS_ERR_NOENT) { - return err; - } - - dir->id += 1; - if (err != LFS_ERR_NOENT) { - break; - } - } - - dir->pos += 1; - return true; -} - -static int lfs_dir_seek_(lfs_t *lfs, lfs_dir_t *dir, lfs_off_t off) { - // simply walk from head dir - int err = lfs_dir_rewind_(lfs, dir); - if (err) { - return err; - } - - // first two for ./.. - dir->pos = lfs_min(2, off); - off -= dir->pos; - - // skip superblock entry - dir->id = (off > 0 && lfs_pair_cmp(dir->head, lfs->root) == 0); - - while (off > 0) { - if (dir->id == dir->m.count) { - if (!dir->m.split) { - return LFS_ERR_INVAL; - } - - err = lfs_dir_fetch(lfs, &dir->m, dir->m.tail); - if (err) { - return err; - } - - dir->id = 0; - } - - int diff = lfs_min(dir->m.count - dir->id, off); - dir->id += diff; - dir->pos += diff; - off -= diff; - } - - return 0; -} - -static lfs_soff_t lfs_dir_tell_(lfs_t *lfs, lfs_dir_t *dir) { - (void)lfs; - return dir->pos; -} - -static int lfs_dir_rewind_(lfs_t *lfs, lfs_dir_t *dir) { - // reload the head dir - int err = lfs_dir_fetch(lfs, &dir->m, dir->head); - if (err) { - return err; - } - - dir->id = 0; - dir->pos = 0; - return 0; -} - - -/// File index list operations /// -static int lfs_ctz_index(lfs_t *lfs, lfs_off_t *off) { - lfs_off_t size = *off; - lfs_off_t b = lfs->cfg->block_size - 2*4; - lfs_off_t i = size / b; - if (i == 0) { - return 0; - } - - i = (size - 4*(lfs_popc(i-1)+2)) / b; - *off = size - b*i - 4*lfs_popc(i); - return i; -} - -static int lfs_ctz_find(lfs_t *lfs, - const lfs_cache_t *pcache, lfs_cache_t *rcache, - lfs_block_t head, lfs_size_t size, - lfs_size_t pos, lfs_block_t *block, lfs_off_t *off) { - if (size == 0) { - *block = LFS_BLOCK_NULL; - *off = 0; - return 0; - } - - lfs_off_t current = lfs_ctz_index(lfs, &(lfs_off_t){size-1}); - lfs_off_t target = lfs_ctz_index(lfs, &pos); - - while (current > target) { - lfs_size_t skip = lfs_min( - lfs_npw2(current-target+1) - 1, - lfs_ctz(current)); - - int err = lfs_bd_read(lfs, - pcache, rcache, sizeof(head), - head, 4*skip, &head, sizeof(head)); - head = lfs_fromle32(head); - if (err) { - return err; - } - - current -= 1 << skip; - } - - *block = head; - *off = pos; - return 0; -} - -#ifndef LFS_READONLY -static int lfs_ctz_extend(lfs_t *lfs, - lfs_cache_t *pcache, lfs_cache_t *rcache, - lfs_block_t head, lfs_size_t size, - lfs_block_t *block, lfs_off_t *off) { - while (true) { - // go ahead and grab a block - lfs_block_t nblock; - int err = lfs_alloc(lfs, &nblock); - if (err) { - return err; - } - - { - err = lfs_bd_erase(lfs, nblock); - if (err) { - if (err == LFS_ERR_CORRUPT) { - goto relocate; - } - return err; - } - - if (size == 0) { - *block = nblock; - *off = 0; - return 0; - } - - lfs_size_t noff = size - 1; - lfs_off_t index = lfs_ctz_index(lfs, &noff); - noff = noff + 1; - - // just copy out the last block if it is incomplete - if (noff != lfs->cfg->block_size) { - for (lfs_off_t i = 0; i < noff; i++) { - uint8_t data; - err = lfs_bd_read(lfs, - NULL, rcache, noff-i, - head, i, &data, 1); - if (err) { - return err; - } - - err = lfs_bd_prog(lfs, - pcache, rcache, true, - nblock, i, &data, 1); - if (err) { - if (err == LFS_ERR_CORRUPT) { - goto relocate; - } - return err; - } - } - - *block = nblock; - *off = noff; - return 0; - } - - // append block - index += 1; - lfs_size_t skips = lfs_ctz(index) + 1; - lfs_block_t nhead = head; - for (lfs_off_t i = 0; i < skips; i++) { - nhead = lfs_tole32(nhead); - err = lfs_bd_prog(lfs, pcache, rcache, true, - nblock, 4*i, &nhead, 4); - nhead = lfs_fromle32(nhead); - if (err) { - if (err == LFS_ERR_CORRUPT) { - goto relocate; - } - return err; - } - - if (i != skips-1) { - err = lfs_bd_read(lfs, - NULL, rcache, sizeof(nhead), - nhead, 4*i, &nhead, sizeof(nhead)); - nhead = lfs_fromle32(nhead); - if (err) { - return err; - } - } - } - - *block = nblock; - *off = 4*skips; - return 0; - } - -relocate: - LFS_DEBUG("Bad block at 0x%"PRIx32, nblock); - - // just clear cache and try a new block - lfs_cache_drop(lfs, pcache); - } -} -#endif - -static int lfs_ctz_traverse(lfs_t *lfs, - const lfs_cache_t *pcache, lfs_cache_t *rcache, - lfs_block_t head, lfs_size_t size, - int (*cb)(void*, lfs_block_t), void *data) { - if (size == 0) { - return 0; - } - - lfs_off_t index = lfs_ctz_index(lfs, &(lfs_off_t){size-1}); - - while (true) { - int err = cb(data, head); - if (err) { - return err; - } - - if (index == 0) { - return 0; - } - - lfs_block_t heads[2]; - int count = 2 - (index & 1); - err = lfs_bd_read(lfs, - pcache, rcache, count*sizeof(head), - head, 0, &heads, count*sizeof(head)); - heads[0] = lfs_fromle32(heads[0]); - heads[1] = lfs_fromle32(heads[1]); - if (err) { - return err; - } - - for (int i = 0; i < count-1; i++) { - err = cb(data, heads[i]); - if (err) { - return err; - } - } - - head = heads[count-1]; - index -= count; - } -} - - -/// Top level file operations /// -static int lfs_file_opencfg_(lfs_t *lfs, lfs_file_t *file, - const char *path, int flags, - const struct lfs_file_config *cfg) { -#ifndef LFS_READONLY - // deorphan if we haven't yet, needed at most once after poweron - if ((flags & LFS_O_WRONLY) == LFS_O_WRONLY) { - int err = lfs_fs_forceconsistency(lfs); - if (err) { - return err; - } - } -#else - LFS_ASSERT((flags & LFS_O_RDONLY) == LFS_O_RDONLY); -#endif - - // setup simple file details - int err; - file->cfg = cfg; - file->flags = flags; - file->pos = 0; - file->off = 0; - file->cache.buffer = NULL; - - // allocate entry for file if it doesn't exist - lfs_stag_t tag = lfs_dir_find(lfs, &file->m, &path, &file->id); - if (tag < 0 && !(tag == LFS_ERR_NOENT && lfs_path_islast(path))) { - err = tag; - goto cleanup; - } - - // get id, add to list of mdirs to catch update changes - file->type = LFS_TYPE_REG; - lfs_mlist_append(lfs, (struct lfs_mlist *)file); - -#ifdef LFS_READONLY - if (tag == LFS_ERR_NOENT) { - err = LFS_ERR_NOENT; - goto cleanup; -#else - if (tag == LFS_ERR_NOENT) { - if (!(flags & LFS_O_CREAT)) { - err = LFS_ERR_NOENT; - goto cleanup; - } - - // don't allow trailing slashes - if (lfs_path_isdir(path)) { - err = LFS_ERR_NOTDIR; - goto cleanup; - } - - // check that name fits - lfs_size_t nlen = lfs_path_namelen(path); - if (nlen > lfs->name_max) { - err = LFS_ERR_NAMETOOLONG; - goto cleanup; - } - - // get next slot and create entry to remember name - err = lfs_dir_commit(lfs, &file->m, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_CREATE, file->id, 0), NULL}, - {LFS_MKTAG(LFS_TYPE_REG, file->id, nlen), path}, - {LFS_MKTAG(LFS_TYPE_INLINESTRUCT, file->id, 0), NULL})); - - // it may happen that the file name doesn't fit in the metadata blocks, e.g., a 256 byte file name will - // not fit in a 128 byte block. - err = (err == LFS_ERR_NOSPC) ? LFS_ERR_NAMETOOLONG : err; - if (err) { - goto cleanup; - } - - tag = LFS_MKTAG(LFS_TYPE_INLINESTRUCT, 0, 0); - } else if (flags & LFS_O_EXCL) { - err = LFS_ERR_EXIST; - goto cleanup; -#endif - } else if (lfs_tag_type3(tag) != LFS_TYPE_REG) { - err = LFS_ERR_ISDIR; - goto cleanup; -#ifndef LFS_READONLY - } else if (flags & LFS_O_TRUNC) { - // truncate if requested - tag = LFS_MKTAG(LFS_TYPE_INLINESTRUCT, file->id, 0); - file->flags |= LFS_F_DIRTY; -#endif - } else { - // try to load what's on disk, if it's inlined we'll fix it later - tag = lfs_dir_get(lfs, &file->m, LFS_MKTAG(0x700, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_STRUCT, file->id, 8), &file->ctz); - if (tag < 0) { - err = tag; - goto cleanup; - } - lfs_ctz_fromle32(&file->ctz); - } - - // fetch attrs - for (unsigned i = 0; i < file->cfg->attr_count; i++) { - // if opened for read / read-write operations - if ((file->flags & LFS_O_RDONLY) == LFS_O_RDONLY) { - lfs_stag_t res = lfs_dir_get(lfs, &file->m, - LFS_MKTAG(0x7ff, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_USERATTR + file->cfg->attrs[i].type, - file->id, file->cfg->attrs[i].size), - file->cfg->attrs[i].buffer); - if (res < 0 && res != LFS_ERR_NOENT) { - err = res; - goto cleanup; - } - } - -#ifndef LFS_READONLY - // if opened for write / read-write operations - if ((file->flags & LFS_O_WRONLY) == LFS_O_WRONLY) { - if (file->cfg->attrs[i].size > lfs->attr_max) { - err = LFS_ERR_NOSPC; - goto cleanup; - } - - file->flags |= LFS_F_DIRTY; - } -#endif - } - - // allocate buffer if needed - if (file->cfg->buffer) { - file->cache.buffer = file->cfg->buffer; - } else { - file->cache.buffer = lfs_malloc(lfs->cfg->cache_size); - if (!file->cache.buffer) { - err = LFS_ERR_NOMEM; - goto cleanup; - } - } - - // zero to avoid information leak - lfs_cache_zero(lfs, &file->cache); - - if (lfs_tag_type3(tag) == LFS_TYPE_INLINESTRUCT) { - // load inline files - file->ctz.head = LFS_BLOCK_INLINE; - file->ctz.size = lfs_tag_size(tag); - file->flags |= LFS_F_INLINE; - file->cache.block = file->ctz.head; - file->cache.off = 0; - file->cache.size = lfs->cfg->cache_size; - - // don't always read (may be new/trunc file) - if (file->ctz.size > 0) { - lfs_stag_t res = lfs_dir_get(lfs, &file->m, - LFS_MKTAG(0x700, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_STRUCT, file->id, - lfs_min(file->cache.size, 0x3fe)), - file->cache.buffer); - if (res < 0) { - err = res; - goto cleanup; - } - } - } - - return 0; - -cleanup: - // clean up lingering resources -#ifndef LFS_READONLY - file->flags |= LFS_F_ERRED; -#endif - lfs_file_close_(lfs, file); - return err; -} - -#ifndef LFS_NO_MALLOC -static int lfs_file_open_(lfs_t *lfs, lfs_file_t *file, - const char *path, int flags) { - static const struct lfs_file_config defaults = {0}; - int err = lfs_file_opencfg_(lfs, file, path, flags, &defaults); - return err; -} -#endif - -static int lfs_file_close_(lfs_t *lfs, lfs_file_t *file) { - int err = 0; -#ifndef LFS_READONLY - // it's not safe to do anything if our file errored - if (!(file->flags & LFS_F_ERRED)) { - err = lfs_file_sync_(lfs, file); - } -#endif - - // remove from list of mdirs - lfs_mlist_remove(lfs, (struct lfs_mlist*)file); - - // clean up memory - if (!file->cfg->buffer) { - lfs_free(file->cache.buffer); - } - - return err; -} - - -#ifndef LFS_READONLY -static int lfs_file_relocate(lfs_t *lfs, lfs_file_t *file) { - while (true) { - // just relocate what exists into new block - lfs_block_t nblock; - int err = lfs_alloc(lfs, &nblock); - if (err) { - return err; - } - - err = lfs_bd_erase(lfs, nblock); - if (err) { - if (err == LFS_ERR_CORRUPT) { - goto relocate; - } - return err; - } - - // either read from dirty cache or disk - for (lfs_off_t i = 0; i < file->off; i++) { - uint8_t data; - if (file->flags & LFS_F_INLINE) { - err = lfs_dir_getread(lfs, &file->m, - // note we evict inline files before they can be dirty - NULL, &file->cache, file->off-i, - LFS_MKTAG(0xfff, 0x1ff, 0), - LFS_MKTAG(LFS_TYPE_INLINESTRUCT, file->id, 0), - i, &data, 1); - if (err) { - return err; - } - } else { - err = lfs_bd_read(lfs, - &file->cache, &lfs->rcache, file->off-i, - file->block, i, &data, 1); - if (err) { - return err; - } - } - - err = lfs_bd_prog(lfs, - &lfs->pcache, &lfs->rcache, true, - nblock, i, &data, 1); - if (err) { - if (err == LFS_ERR_CORRUPT) { - goto relocate; - } - return err; - } - } - - // copy over new state of file - memcpy(file->cache.buffer, lfs->pcache.buffer, lfs->cfg->cache_size); - file->cache.block = lfs->pcache.block; - file->cache.off = lfs->pcache.off; - file->cache.size = lfs->pcache.size; - lfs_cache_zero(lfs, &lfs->pcache); - - file->block = nblock; - file->flags |= LFS_F_WRITING; - return 0; - -relocate: - LFS_DEBUG("Bad block at 0x%"PRIx32, nblock); - - // just clear cache and try a new block - lfs_cache_drop(lfs, &lfs->pcache); - } -} -#endif - -#ifndef LFS_READONLY -static int lfs_file_outline(lfs_t *lfs, lfs_file_t *file) { - file->off = file->pos; - lfs_alloc_ckpoint(lfs); - int err = lfs_file_relocate(lfs, file); - if (err) { - return err; - } - - file->flags &= ~LFS_F_INLINE; - return 0; -} -#endif - -static int lfs_file_flush(lfs_t *lfs, lfs_file_t *file) { - if (file->flags & LFS_F_READING) { - if (!(file->flags & LFS_F_INLINE)) { - lfs_cache_drop(lfs, &file->cache); - } - file->flags &= ~LFS_F_READING; - } - -#ifndef LFS_READONLY - if (file->flags & LFS_F_WRITING) { - lfs_off_t pos = file->pos; - - if (!(file->flags & LFS_F_INLINE)) { - // copy over anything after current branch - lfs_file_t orig = { - .ctz.head = file->ctz.head, - .ctz.size = file->ctz.size, - .flags = LFS_O_RDONLY, - .pos = file->pos, - .cache = lfs->rcache, - }; - lfs_cache_drop(lfs, &lfs->rcache); - - while (file->pos < file->ctz.size) { - // copy over a byte at a time, leave it up to caching - // to make this efficient - uint8_t data; - lfs_ssize_t res = lfs_file_flushedread(lfs, &orig, &data, 1); - if (res < 0) { - return res; - } - - res = lfs_file_flushedwrite(lfs, file, &data, 1); - if (res < 0) { - return res; - } - - // keep our reference to the rcache in sync - if (lfs->rcache.block != LFS_BLOCK_NULL) { - lfs_cache_drop(lfs, &orig.cache); - lfs_cache_drop(lfs, &lfs->rcache); - } - } - - // write out what we have - while (true) { - int err = lfs_bd_flush(lfs, &file->cache, &lfs->rcache, true); - if (err) { - if (err == LFS_ERR_CORRUPT) { - goto relocate; - } - return err; - } - - break; - -relocate: - LFS_DEBUG("Bad block at 0x%"PRIx32, file->block); - err = lfs_file_relocate(lfs, file); - if (err) { - return err; - } - } - } else { - file->pos = lfs_max(file->pos, file->ctz.size); - } - - // actual file updates - file->ctz.head = file->block; - file->ctz.size = file->pos; - file->flags &= ~LFS_F_WRITING; - file->flags |= LFS_F_DIRTY; - - file->pos = pos; - } -#endif - - return 0; -} - -#ifndef LFS_READONLY -static int lfs_file_sync_(lfs_t *lfs, lfs_file_t *file) { - int err = lfs_file_flush(lfs, file); - if (err) { - file->flags |= LFS_F_ERRED; - return err; - } - - if ((file->flags & LFS_F_DIRTY) && - !lfs_pair_isnull(file->m.pair)) { - // before we commit metadata, we need sync the disk to make sure - // data writes don't complete after metadata writes - if (!(file->flags & LFS_F_INLINE)) { - err = lfs_bd_sync(lfs, &lfs->pcache, &lfs->rcache, false); - if (err) { - return err; - } - } - - // update dir entry - uint16_t type; - const void *buffer; - lfs_size_t size; - struct lfs_ctz ctz; - if (file->flags & LFS_F_INLINE) { - // inline the whole file - type = LFS_TYPE_INLINESTRUCT; - buffer = file->cache.buffer; - size = file->ctz.size; - } else { - // update the ctz reference - type = LFS_TYPE_CTZSTRUCT; - // copy ctz so alloc will work during a relocate - ctz = file->ctz; - lfs_ctz_tole32(&ctz); - buffer = &ctz; - size = sizeof(ctz); - } - - // commit file data and attributes - err = lfs_dir_commit(lfs, &file->m, LFS_MKATTRS( - {LFS_MKTAG(type, file->id, size), buffer}, - {LFS_MKTAG(LFS_FROM_USERATTRS, file->id, - file->cfg->attr_count), file->cfg->attrs})); - if (err) { - file->flags |= LFS_F_ERRED; - return err; - } - - file->flags &= ~LFS_F_DIRTY; - } - - // mark any other file handles as dirty + desync - for (lfs_file_t *f = (lfs_file_t*)lfs->mlist; f; f = f->next) { - if (file != f - && f->type == LFS_TYPE_REG - && lfs_pair_cmp(f->m.pair, file->m.pair) == 0 - && f->id == file->id) { - f->flags |= LFS_F_DUSTY; - } - } - - file->flags &= ~LFS_F_ERRED & ~LFS_F_DUSTY; - return 0; -} -#endif - -static lfs_ssize_t lfs_file_flushedread(lfs_t *lfs, lfs_file_t *file, - void *buffer, lfs_size_t size) { - uint8_t *data = buffer; - lfs_size_t nsize = size; - - if (file->pos >= file->ctz.size) { - // eof if past end - return 0; - } - - size = lfs_min(size, file->ctz.size - file->pos); - nsize = size; - - while (nsize > 0) { - // check if we need a new block - if (!(file->flags & LFS_F_READING) || - file->off == lfs->cfg->block_size) { - if (!(file->flags & LFS_F_INLINE)) { - int err = lfs_ctz_find(lfs, NULL, &file->cache, - file->ctz.head, file->ctz.size, - file->pos, &file->block, &file->off); - if (err) { - return err; - } - } else { - file->block = LFS_BLOCK_INLINE; - file->off = file->pos; - } - - file->flags |= LFS_F_READING; - } - - // read as much as we can in current block - lfs_size_t diff = lfs_min(nsize, lfs->cfg->block_size - file->off); - if (file->flags & LFS_F_INLINE) { - int err = lfs_dir_getread(lfs, &file->m, - NULL, &file->cache, lfs->cfg->block_size, - LFS_MKTAG(0xfff, 0x1ff, 0), - LFS_MKTAG(LFS_TYPE_INLINESTRUCT, file->id, 0), - file->off, data, diff); - if (err) { - return err; - } - } else { - int err = lfs_bd_read(lfs, - NULL, &file->cache, lfs->cfg->block_size, - file->block, file->off, data, diff); - if (err) { - return err; - } - } - - file->pos += diff; - file->off += diff; - data += diff; - nsize -= diff; - } - - return size; -} - -static lfs_ssize_t lfs_file_read_(lfs_t *lfs, lfs_file_t *file, - void *buffer, lfs_size_t size) { - LFS_ASSERT((file->flags & LFS_O_RDONLY) == LFS_O_RDONLY); - -#ifndef LFS_READONLY - if (file->flags & LFS_F_WRITING) { - // flush out any writes - int err = lfs_file_flush(lfs, file); - if (err) { - return err; - } - } -#endif - - return lfs_file_flushedread(lfs, file, buffer, size); -} - - -#ifndef LFS_READONLY -static lfs_ssize_t lfs_file_flushedwrite(lfs_t *lfs, lfs_file_t *file, - const void *buffer, lfs_size_t size) { - const uint8_t *data = buffer; - lfs_size_t nsize = size; - - if ((file->flags & LFS_F_INLINE) && - lfs_max(file->pos+nsize, file->ctz.size) > lfs->inline_max) { - // inline file doesn't fit anymore - int err = lfs_file_outline(lfs, file); - if (err) { - file->flags |= LFS_F_ERRED; - return err; - } - } - - while (nsize > 0) { - // check if we need a new block - if (!(file->flags & LFS_F_WRITING) || - file->off == lfs->cfg->block_size) { - if (!(file->flags & LFS_F_INLINE)) { - if (!(file->flags & LFS_F_WRITING) && file->pos > 0) { - // find out which block we're extending from - int err = lfs_ctz_find(lfs, NULL, &file->cache, - file->ctz.head, file->ctz.size, - file->pos-1, &file->block, &(lfs_off_t){0}); - if (err) { - file->flags |= LFS_F_ERRED; - return err; - } - - // mark cache as dirty since we may have read data into it - lfs_cache_zero(lfs, &file->cache); - } - - // extend file with new blocks - lfs_alloc_ckpoint(lfs); - int err = lfs_ctz_extend(lfs, &file->cache, &lfs->rcache, - file->block, file->pos, - &file->block, &file->off); - if (err) { - file->flags |= LFS_F_ERRED; - return err; - } - } else { - file->block = LFS_BLOCK_INLINE; - file->off = file->pos; - } - - file->flags |= LFS_F_WRITING; - } - - // program as much as we can in current block - lfs_size_t diff = lfs_min(nsize, lfs->cfg->block_size - file->off); - while (true) { - int err = lfs_bd_prog(lfs, &file->cache, &lfs->rcache, true, - file->block, file->off, data, diff); - if (err) { - if (err == LFS_ERR_CORRUPT) { - goto relocate; - } - file->flags |= LFS_F_ERRED; - return err; - } - - break; -relocate: - err = lfs_file_relocate(lfs, file); - if (err) { - file->flags |= LFS_F_ERRED; - return err; - } - } - - file->pos += diff; - file->off += diff; - data += diff; - nsize -= diff; - - lfs_alloc_ckpoint(lfs); - } - - return size; -} - -static lfs_ssize_t lfs_file_write_(lfs_t *lfs, lfs_file_t *file, - const void *buffer, lfs_size_t size) { - LFS_ASSERT((file->flags & LFS_O_WRONLY) == LFS_O_WRONLY); - - if (file->flags & LFS_F_READING) { - // drop any reads - int err = lfs_file_flush(lfs, file); - if (err) { - return err; - } - } - - if ((file->flags & LFS_O_APPEND) && file->pos < file->ctz.size) { - file->pos = file->ctz.size; - } - - if (file->pos + size > lfs->file_max) { - // Larger than file limit? - return LFS_ERR_FBIG; - } - - if (!(file->flags & LFS_F_WRITING) && file->pos > file->ctz.size) { - // fill with zeros - lfs_off_t pos = file->pos; - file->pos = file->ctz.size; - - while (file->pos < pos) { - lfs_ssize_t res = lfs_file_flushedwrite(lfs, file, &(uint8_t){0}, 1); - if (res < 0) { - return res; - } - } - } - - lfs_ssize_t nsize = lfs_file_flushedwrite(lfs, file, buffer, size); - if (nsize < 0) { - return nsize; - } - - file->flags &= ~LFS_F_ERRED & ~LFS_F_DUSTY; - return nsize; -} -#endif - -static lfs_soff_t lfs_file_seek_(lfs_t *lfs, lfs_file_t *file, - lfs_soff_t off, int whence) { - // find new pos - // - // fortunately for us, littlefs is limited to 31-bit file sizes, so we - // don't have to worry too much about integer overflow - lfs_off_t npos = file->pos; - if (whence == LFS_SEEK_SET) { - npos = off; - } else if (whence == LFS_SEEK_CUR) { - npos = file->pos + (lfs_off_t)off; - } else if (whence == LFS_SEEK_END) { - npos = (lfs_off_t)lfs_file_size_(lfs, file) + (lfs_off_t)off; - } - - if (npos > lfs->file_max) { - // file position out of range - return LFS_ERR_INVAL; - } - - if (file->pos == npos) { - // noop - position has not changed - return npos; - } - - // if we're only reading and our new offset is still in the file's cache - // we can avoid flushing and needing to reread the data - if ((file->flags & LFS_F_READING) - && file->off != lfs->cfg->block_size) { - int oindex = lfs_ctz_index(lfs, &(lfs_off_t){file->pos}); - lfs_off_t noff = npos; - int nindex = lfs_ctz_index(lfs, &noff); - if (oindex == nindex - && noff >= file->cache.off - && noff < file->cache.off + file->cache.size) { - file->pos = npos; - file->off = noff; - return npos; - } - } - - // write out everything beforehand, may be noop if rdonly - int err = lfs_file_flush(lfs, file); - if (err) { - return err; - } - - // update pos - file->pos = npos; - return npos; -} - -#ifndef LFS_READONLY -static int lfs_file_truncate_(lfs_t *lfs, lfs_file_t *file, lfs_off_t size) { - LFS_ASSERT((file->flags & LFS_O_WRONLY) == LFS_O_WRONLY); - - if (size > LFS_FILE_MAX) { - return LFS_ERR_INVAL; - } - - lfs_off_t pos = file->pos; - lfs_off_t oldsize = lfs_file_size_(lfs, file); - if (size < oldsize) { - // revert to inline file? - if (size <= lfs->inline_max) { - // flush+seek to head - lfs_soff_t res = lfs_file_seek_(lfs, file, 0, LFS_SEEK_SET); - if (res < 0) { - return (int)res; - } - - // read our data into rcache temporarily - lfs_cache_drop(lfs, &lfs->rcache); - res = lfs_file_flushedread(lfs, file, - lfs->rcache.buffer, size); - if (res < 0) { - return (int)res; - } - - file->ctz.head = LFS_BLOCK_INLINE; - file->ctz.size = size; - file->flags |= LFS_F_DIRTY | LFS_F_READING | LFS_F_INLINE; - file->cache.block = file->ctz.head; - file->cache.off = 0; - file->cache.size = lfs->cfg->cache_size; - memcpy(file->cache.buffer, lfs->rcache.buffer, size); - - } else { - // need to flush since directly changing metadata - int err = lfs_file_flush(lfs, file); - if (err) { - return err; - } - - // lookup new head in ctz skip list - err = lfs_ctz_find(lfs, NULL, &file->cache, - file->ctz.head, file->ctz.size, - size-1, &file->block, &(lfs_off_t){0}); - if (err) { - return err; - } - - // need to set pos/block/off consistently so seeking back to - // the old position does not get confused - file->pos = size; - file->ctz.head = file->block; - file->ctz.size = size; - file->flags |= LFS_F_DIRTY | LFS_F_READING; - } - } else if (size > oldsize) { - // flush+seek if not already at end - lfs_soff_t res = lfs_file_seek_(lfs, file, 0, LFS_SEEK_END); - if (res < 0) { - return (int)res; - } - - // fill with zeros - while (file->pos < size) { - res = lfs_file_write_(lfs, file, &(uint8_t){0}, 1); - if (res < 0) { - return (int)res; - } - } - } - - // restore pos - lfs_soff_t res = lfs_file_seek_(lfs, file, pos, LFS_SEEK_SET); - if (res < 0) { - return (int)res; - } - - return 0; -} -#endif - -static lfs_soff_t lfs_file_tell_(lfs_t *lfs, lfs_file_t *file) { - (void)lfs; - return file->pos; -} - -static int lfs_file_rewind_(lfs_t *lfs, lfs_file_t *file) { - lfs_soff_t res = lfs_file_seek_(lfs, file, 0, LFS_SEEK_SET); - if (res < 0) { - return (int)res; - } - - return 0; -} - -static lfs_soff_t lfs_file_size_(lfs_t *lfs, lfs_file_t *file) { - (void)lfs; - -#ifndef LFS_READONLY - if (file->flags & LFS_F_WRITING) { - return lfs_max(file->pos, file->ctz.size); - } -#endif - - return file->ctz.size; -} - - -/// General fs operations /// -static int lfs_stat_(lfs_t *lfs, const char *path, struct lfs_info *info) { - lfs_mdir_t cwd; - lfs_stag_t tag = lfs_dir_find(lfs, &cwd, &path, NULL); - if (tag < 0) { - return (int)tag; - } - - // only allow trailing slashes on dirs - if (strchr(path, '/') != NULL - && lfs_tag_type3(tag) != LFS_TYPE_DIR) { - return LFS_ERR_NOTDIR; - } - - return lfs_dir_getinfo(lfs, &cwd, lfs_tag_id(tag), info); -} - -#ifndef LFS_READONLY -static int lfs_remove_(lfs_t *lfs, const char *path) { - // deorphan if we haven't yet, needed at most once after poweron - int err = lfs_fs_forceconsistency(lfs); - if (err) { - return err; - } - - lfs_mdir_t cwd; - lfs_stag_t tag = lfs_dir_find(lfs, &cwd, &path, NULL); - if (tag < 0 || lfs_tag_id(tag) == 0x3ff) { - return (tag < 0) ? (int)tag : LFS_ERR_INVAL; - } - - struct lfs_mlist dir; - dir.next = lfs->mlist; - if (lfs_tag_type3(tag) == LFS_TYPE_DIR) { - // must be empty before removal - lfs_block_t pair[2]; - lfs_stag_t res = lfs_dir_get(lfs, &cwd, LFS_MKTAG(0x700, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_STRUCT, lfs_tag_id(tag), 8), pair); - if (res < 0) { - return (int)res; - } - lfs_pair_fromle32(pair); - - err = lfs_dir_fetch(lfs, &dir.m, pair); - if (err) { - return err; - } - - if (dir.m.count > 0 || dir.m.split) { - return LFS_ERR_NOTEMPTY; - } - - // mark fs as orphaned - err = lfs_fs_preporphans(lfs, +1); - if (err) { - return err; - } - - // I know it's crazy but yes, dir can be changed by our parent's - // commit (if predecessor is child) - dir.type = 0; - dir.id = 0; - lfs->mlist = &dir; - } - - // delete the entry - err = lfs_dir_commit(lfs, &cwd, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_DELETE, lfs_tag_id(tag), 0), NULL})); - if (err) { - lfs->mlist = dir.next; - return err; - } - - lfs->mlist = dir.next; - if (lfs_gstate_hasorphans(&lfs->gstate)) { - LFS_ASSERT(lfs_tag_type3(tag) == LFS_TYPE_DIR); - - // fix orphan - err = lfs_fs_preporphans(lfs, -1); - if (err) { - return err; - } - - err = lfs_fs_pred(lfs, dir.m.pair, &cwd); - if (err) { - return err; - } - - err = lfs_dir_drop(lfs, &cwd, &dir.m); - if (err) { - return err; - } - } - - return 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_rename_(lfs_t *lfs, const char *oldpath, const char *newpath) { - // deorphan if we haven't yet, needed at most once after poweron - int err = lfs_fs_forceconsistency(lfs); - if (err) { - return err; - } - - // find old entry - lfs_mdir_t oldcwd; - lfs_stag_t oldtag = lfs_dir_find(lfs, &oldcwd, &oldpath, NULL); - if (oldtag < 0 || lfs_tag_id(oldtag) == 0x3ff) { - return (oldtag < 0) ? (int)oldtag : LFS_ERR_INVAL; - } - - // find new entry - lfs_mdir_t newcwd; - uint16_t newid; - lfs_stag_t prevtag = lfs_dir_find(lfs, &newcwd, &newpath, &newid); - if ((prevtag < 0 || lfs_tag_id(prevtag) == 0x3ff) && - !(prevtag == LFS_ERR_NOENT && lfs_path_islast(newpath))) { - return (prevtag < 0) ? (int)prevtag : LFS_ERR_INVAL; - } - - // if we're in the same pair there's a few special cases... - bool samepair = (lfs_pair_cmp(oldcwd.pair, newcwd.pair) == 0); - uint16_t newoldid = lfs_tag_id(oldtag); - - struct lfs_mlist prevdir; - prevdir.next = lfs->mlist; - if (prevtag == LFS_ERR_NOENT) { - // if we're a file, don't allow trailing slashes - if (lfs_path_isdir(newpath) - && lfs_tag_type3(oldtag) != LFS_TYPE_DIR) { - return LFS_ERR_NOTDIR; - } - - // check that name fits - lfs_size_t nlen = lfs_path_namelen(newpath); - if (nlen > lfs->name_max) { - return LFS_ERR_NAMETOOLONG; - } - - // there is a small chance we are being renamed in the same - // directory/ to an id less than our old id, the global update - // to handle this is a bit messy - if (samepair && newid <= newoldid) { - newoldid += 1; - } - } else if (lfs_tag_type3(prevtag) != lfs_tag_type3(oldtag)) { - return (lfs_tag_type3(prevtag) == LFS_TYPE_DIR) - ? LFS_ERR_ISDIR - : LFS_ERR_NOTDIR; - } else if (samepair && newid == newoldid) { - // we're renaming to ourselves?? - return 0; - } else if (lfs_tag_type3(prevtag) == LFS_TYPE_DIR) { - // must be empty before removal - lfs_block_t prevpair[2]; - lfs_stag_t res = lfs_dir_get(lfs, &newcwd, LFS_MKTAG(0x700, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_STRUCT, newid, 8), prevpair); - if (res < 0) { - return (int)res; - } - lfs_pair_fromle32(prevpair); - - // must be empty before removal - err = lfs_dir_fetch(lfs, &prevdir.m, prevpair); - if (err) { - return err; - } - - if (prevdir.m.count > 0 || prevdir.m.split) { - return LFS_ERR_NOTEMPTY; - } - - // mark fs as orphaned - err = lfs_fs_preporphans(lfs, +1); - if (err) { - return err; - } - - // I know it's crazy but yes, dir can be changed by our parent's - // commit (if predecessor is child) - prevdir.type = 0; - prevdir.id = 0; - lfs->mlist = &prevdir; - } - - if (!samepair) { - lfs_fs_prepmove(lfs, newoldid, oldcwd.pair); - } - - // move over all attributes - err = lfs_dir_commit(lfs, &newcwd, LFS_MKATTRS( - {LFS_MKTAG_IF(prevtag != LFS_ERR_NOENT, - LFS_TYPE_DELETE, newid, 0), NULL}, - {LFS_MKTAG(LFS_TYPE_CREATE, newid, 0), NULL}, - {LFS_MKTAG(lfs_tag_type3(oldtag), - newid, lfs_path_namelen(newpath)), newpath}, - {LFS_MKTAG(LFS_FROM_MOVE, newid, lfs_tag_id(oldtag)), &oldcwd}, - {LFS_MKTAG_IF(samepair, - LFS_TYPE_DELETE, newoldid, 0), NULL})); - if (err) { - lfs->mlist = prevdir.next; - return err; - } - - // let commit clean up after move (if we're different! otherwise move - // logic already fixed it for us) - if (!samepair && lfs_gstate_hasmove(&lfs->gstate)) { - // prep gstate and delete move id - lfs_fs_prepmove(lfs, 0x3ff, NULL); - err = lfs_dir_commit(lfs, &oldcwd, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_DELETE, lfs_tag_id(oldtag), 0), NULL})); - if (err) { - lfs->mlist = prevdir.next; - return err; - } - } - - lfs->mlist = prevdir.next; - if (lfs_gstate_hasorphans(&lfs->gstate)) { - LFS_ASSERT(prevtag != LFS_ERR_NOENT - && lfs_tag_type3(prevtag) == LFS_TYPE_DIR); - - // fix orphan - err = lfs_fs_preporphans(lfs, -1); - if (err) { - return err; - } - - err = lfs_fs_pred(lfs, prevdir.m.pair, &newcwd); - if (err) { - return err; - } - - err = lfs_dir_drop(lfs, &newcwd, &prevdir.m); - if (err) { - return err; - } - } - - return 0; -} -#endif - -static lfs_ssize_t lfs_getattr_(lfs_t *lfs, const char *path, - uint8_t type, void *buffer, lfs_size_t size) { - lfs_mdir_t cwd; - lfs_stag_t tag = lfs_dir_find(lfs, &cwd, &path, NULL); - if (tag < 0) { - return tag; - } - - uint16_t id = lfs_tag_id(tag); - if (id == 0x3ff) { - // special case for root - id = 0; - int err = lfs_dir_fetch(lfs, &cwd, lfs->root); - if (err) { - return err; - } - } - - tag = lfs_dir_get(lfs, &cwd, LFS_MKTAG(0x7ff, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_USERATTR + type, - id, lfs_min(size, lfs->attr_max)), - buffer); - if (tag < 0) { - if (tag == LFS_ERR_NOENT) { - return LFS_ERR_NOATTR; - } - - return tag; - } - - return lfs_tag_size(tag); -} - -#ifndef LFS_READONLY -static int lfs_commitattr(lfs_t *lfs, const char *path, - uint8_t type, const void *buffer, lfs_size_t size) { - lfs_mdir_t cwd; - lfs_stag_t tag = lfs_dir_find(lfs, &cwd, &path, NULL); - if (tag < 0) { - return tag; - } - - uint16_t id = lfs_tag_id(tag); - if (id == 0x3ff) { - // special case for root - id = 0; - int err = lfs_dir_fetch(lfs, &cwd, lfs->root); - if (err) { - return err; - } - } - - return lfs_dir_commit(lfs, &cwd, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_USERATTR + type, id, size), buffer})); -} -#endif - -#ifndef LFS_READONLY -static int lfs_setattr_(lfs_t *lfs, const char *path, - uint8_t type, const void *buffer, lfs_size_t size) { - if (size > lfs->attr_max) { - return LFS_ERR_NOSPC; - } - - return lfs_commitattr(lfs, path, type, buffer, size); -} -#endif - -#ifndef LFS_READONLY -static int lfs_removeattr_(lfs_t *lfs, const char *path, uint8_t type) { - return lfs_commitattr(lfs, path, type, NULL, 0x3ff); -} -#endif - - -/// Filesystem operations /// - -// compile time checks, see lfs.h for why these limits exist -#if LFS_NAME_MAX > 1022 -#error "Invalid LFS_NAME_MAX, must be <= 1022" -#endif - -#if LFS_FILE_MAX > 2147483647 -#error "Invalid LFS_FILE_MAX, must be <= 2147483647" -#endif - -#if LFS_ATTR_MAX > 1022 -#error "Invalid LFS_ATTR_MAX, must be <= 1022" -#endif - -// common filesystem initialization -static int lfs_init(lfs_t *lfs, const struct lfs_config *cfg) { - lfs->cfg = cfg; - lfs->block_count = cfg->block_count; // May be 0 - int err = 0; - -#ifdef LFS_MULTIVERSION - // this driver only supports minor version < current minor version - LFS_ASSERT(!lfs->cfg->disk_version || ( - (0xffff & (lfs->cfg->disk_version >> 16)) - == LFS_DISK_VERSION_MAJOR - && (0xffff & (lfs->cfg->disk_version >> 0)) - <= LFS_DISK_VERSION_MINOR)); -#endif - - // check that bool is a truthy-preserving type - // - // note the most common reason for this failure is a before-c99 compiler, - // which littlefs currently does not support - LFS_ASSERT((bool)0x80000000); - - // check that the required io functions are provided - LFS_ASSERT(lfs->cfg->read != NULL); -#ifndef LFS_READONLY - LFS_ASSERT(lfs->cfg->prog != NULL); - LFS_ASSERT(lfs->cfg->erase != NULL); - LFS_ASSERT(lfs->cfg->sync != NULL); -#endif - - // validate that the lfs-cfg sizes were initiated properly before - // performing any arithmetic logics with them - LFS_ASSERT(lfs->cfg->read_size != 0); - LFS_ASSERT(lfs->cfg->prog_size != 0); - LFS_ASSERT(lfs->cfg->cache_size != 0); - - // check that block size is a multiple of cache size is a multiple - // of prog and read sizes - LFS_ASSERT(lfs->cfg->cache_size % lfs->cfg->read_size == 0); - LFS_ASSERT(lfs->cfg->cache_size % lfs->cfg->prog_size == 0); - LFS_ASSERT(lfs->cfg->block_size % lfs->cfg->cache_size == 0); - - // check that the block size is large enough to fit all ctz pointers - LFS_ASSERT(lfs->cfg->block_size >= 128); - // this is the exact calculation for all ctz pointers, if this fails - // and the simpler assert above does not, math must be broken - LFS_ASSERT(4*lfs_npw2(0xffffffff / (lfs->cfg->block_size-2*4)) - <= lfs->cfg->block_size); - - // block_cycles = 0 is no longer supported. - // - // block_cycles is the number of erase cycles before littlefs evicts - // metadata logs as a part of wear leveling. Suggested values are in the - // range of 100-1000, or set block_cycles to -1 to disable block-level - // wear-leveling. - LFS_ASSERT(lfs->cfg->block_cycles != 0); - - // check that compact_thresh makes sense - // - // metadata can't be compacted below block_size/2, and metadata can't - // exceed a block_size - LFS_ASSERT(lfs->cfg->compact_thresh == 0 - || lfs->cfg->compact_thresh >= lfs->cfg->block_size/2); - LFS_ASSERT(lfs->cfg->compact_thresh == (lfs_size_t)-1 - || lfs->cfg->compact_thresh <= lfs->cfg->block_size); - - // check that metadata_max is a multiple of read_size and prog_size, - // and a factor of the block_size - LFS_ASSERT(!lfs->cfg->metadata_max - || lfs->cfg->metadata_max % lfs->cfg->read_size == 0); - LFS_ASSERT(!lfs->cfg->metadata_max - || lfs->cfg->metadata_max % lfs->cfg->prog_size == 0); - LFS_ASSERT(!lfs->cfg->metadata_max - || lfs->cfg->block_size % lfs->cfg->metadata_max == 0); - - // setup read cache - if (lfs->cfg->read_buffer) { - lfs->rcache.buffer = lfs->cfg->read_buffer; - } else { - lfs->rcache.buffer = lfs_malloc(lfs->cfg->cache_size); - if (!lfs->rcache.buffer) { - err = LFS_ERR_NOMEM; - goto cleanup; - } - } - - // setup program cache - if (lfs->cfg->prog_buffer) { - lfs->pcache.buffer = lfs->cfg->prog_buffer; - } else { - lfs->pcache.buffer = lfs_malloc(lfs->cfg->cache_size); - if (!lfs->pcache.buffer) { - err = LFS_ERR_NOMEM; - goto cleanup; - } - } - - // zero to avoid information leaks - lfs_cache_zero(lfs, &lfs->rcache); - lfs_cache_zero(lfs, &lfs->pcache); - - // setup lookahead buffer, note mount finishes initializing this after - // we establish a decent pseudo-random seed - LFS_ASSERT(lfs->cfg->lookahead_size > 0); - if (lfs->cfg->lookahead_buffer) { - lfs->lookahead.buffer = lfs->cfg->lookahead_buffer; - } else { - lfs->lookahead.buffer = lfs_malloc(lfs->cfg->lookahead_size); - if (!lfs->lookahead.buffer) { - err = LFS_ERR_NOMEM; - goto cleanup; - } - } - - // check that the size limits are sane - LFS_ASSERT(lfs->cfg->name_max <= LFS_NAME_MAX); - lfs->name_max = lfs->cfg->name_max; - if (!lfs->name_max) { - lfs->name_max = LFS_NAME_MAX; - } - - LFS_ASSERT(lfs->cfg->file_max <= LFS_FILE_MAX); - lfs->file_max = lfs->cfg->file_max; - if (!lfs->file_max) { - lfs->file_max = LFS_FILE_MAX; - } - - LFS_ASSERT(lfs->cfg->attr_max <= LFS_ATTR_MAX); - lfs->attr_max = lfs->cfg->attr_max; - if (!lfs->attr_max) { - lfs->attr_max = LFS_ATTR_MAX; - } - - LFS_ASSERT(lfs->cfg->metadata_max <= lfs->cfg->block_size); - - LFS_ASSERT(lfs->cfg->inline_max == (lfs_size_t)-1 - || lfs->cfg->inline_max <= lfs->cfg->cache_size); - LFS_ASSERT(lfs->cfg->inline_max == (lfs_size_t)-1 - || lfs->cfg->inline_max <= lfs->attr_max); - LFS_ASSERT(lfs->cfg->inline_max == (lfs_size_t)-1 - || lfs->cfg->inline_max <= ((lfs->cfg->metadata_max) - ? lfs->cfg->metadata_max - : lfs->cfg->block_size)/8); - lfs->inline_max = lfs->cfg->inline_max; - if (lfs->inline_max == (lfs_size_t)-1) { - lfs->inline_max = 0; - } else if (lfs->inline_max == 0) { - lfs->inline_max = lfs_min( - lfs->cfg->cache_size, - lfs_min( - lfs->attr_max, - ((lfs->cfg->metadata_max) - ? lfs->cfg->metadata_max - : lfs->cfg->block_size)/8)); - } - - // setup default state - lfs->root[0] = LFS_BLOCK_NULL; - lfs->root[1] = LFS_BLOCK_NULL; - lfs->mlist = NULL; - lfs->seed = 0; - lfs->gdisk = (lfs_gstate_t){0}; - lfs->gstate = (lfs_gstate_t){0}; - lfs->gdelta = (lfs_gstate_t){0}; -#ifdef LFS_MIGRATE - lfs->lfs1 = NULL; -#endif - - return 0; - -cleanup: - lfs_deinit(lfs); - return err; -} - -static int lfs_deinit(lfs_t *lfs) { - // free allocated memory - if (!lfs->cfg->read_buffer) { - lfs_free(lfs->rcache.buffer); - } - - if (!lfs->cfg->prog_buffer) { - lfs_free(lfs->pcache.buffer); - } - - if (!lfs->cfg->lookahead_buffer) { - lfs_free(lfs->lookahead.buffer); - } - - return 0; -} - - - -#ifndef LFS_READONLY -static int lfs_format_(lfs_t *lfs, const struct lfs_config *cfg) { - int err = 0; - { - err = lfs_init(lfs, cfg); - if (err) { - return err; - } - - LFS_ASSERT(cfg->block_count != 0); - - // create free lookahead - memset(lfs->lookahead.buffer, 0, lfs->cfg->lookahead_size); - lfs->lookahead.start = 0; - lfs->lookahead.size = lfs_min(8*lfs->cfg->lookahead_size, - lfs->block_count); - lfs->lookahead.next = 0; - lfs_alloc_ckpoint(lfs); - - // create root dir - lfs_mdir_t root; - err = lfs_dir_alloc(lfs, &root); - if (err) { - goto cleanup; - } - - // write one superblock - lfs_superblock_t superblock = { - .version = lfs_fs_disk_version(lfs), - .block_size = lfs->cfg->block_size, - .block_count = lfs->block_count, - .name_max = lfs->name_max, - .file_max = lfs->file_max, - .attr_max = lfs->attr_max, - }; - - lfs_superblock_tole32(&superblock); - err = lfs_dir_commit(lfs, &root, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_CREATE, 0, 0), NULL}, - {LFS_MKTAG(LFS_TYPE_SUPERBLOCK, 0, 8), "littlefs"}, - {LFS_MKTAG(LFS_TYPE_INLINESTRUCT, 0, sizeof(superblock)), - &superblock})); - if (err) { - goto cleanup; - } - - // force compaction to prevent accidentally mounting any - // older version of littlefs that may live on disk - root.erased = false; - err = lfs_dir_commit(lfs, &root, NULL, 0); - if (err) { - goto cleanup; - } - - // sanity check that fetch works - err = lfs_dir_fetch(lfs, &root, (const lfs_block_t[2]){0, 1}); - if (err) { - goto cleanup; - } - } - -cleanup: - lfs_deinit(lfs); - return err; - -} -#endif - -struct lfs_tortoise_t { - lfs_block_t pair[2]; - lfs_size_t i; - lfs_size_t period; -}; - -static int lfs_tortoise_detectcycles( - const lfs_mdir_t *dir, struct lfs_tortoise_t *tortoise) { - // detect cycles with Brent's algorithm - if (lfs_pair_issync(dir->tail, tortoise->pair)) { - LFS_WARN("Cycle detected in tail list"); - return LFS_ERR_CORRUPT; - } - if (tortoise->i == tortoise->period) { - tortoise->pair[0] = dir->tail[0]; - tortoise->pair[1] = dir->tail[1]; - tortoise->i = 0; - tortoise->period *= 2; - } - tortoise->i += 1; - - return LFS_ERR_OK; -} - -static int lfs_mount_(lfs_t *lfs, const struct lfs_config *cfg) { - int err = lfs_init(lfs, cfg); - if (err) { - return err; - } - - // scan directory blocks for superblock and any global updates - lfs_mdir_t dir = {.tail = {0, 1}}; - struct lfs_tortoise_t tortoise = { - .pair = {LFS_BLOCK_NULL, LFS_BLOCK_NULL}, - .i = 1, - .period = 1, - }; - while (!lfs_pair_isnull(dir.tail)) { - err = lfs_tortoise_detectcycles(&dir, &tortoise); - if (err < 0) { - goto cleanup; - } - - // fetch next block in tail list - lfs_stag_t tag = lfs_dir_fetchmatch(lfs, &dir, dir.tail, - LFS_MKTAG(0x7ff, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_SUPERBLOCK, 0, 8), - NULL, - lfs_dir_find_match, &(struct lfs_dir_find_match){ - lfs, "littlefs", 8}); - if (tag < 0) { - err = tag; - goto cleanup; - } - - // has superblock? - if (tag && !lfs_tag_isdelete(tag)) { - // update root - lfs->root[0] = dir.pair[0]; - lfs->root[1] = dir.pair[1]; - - // grab superblock - lfs_superblock_t superblock; - tag = lfs_dir_get(lfs, &dir, LFS_MKTAG(0x7ff, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_INLINESTRUCT, 0, sizeof(superblock)), - &superblock); - if (tag < 0) { - err = tag; - goto cleanup; - } - lfs_superblock_fromle32(&superblock); - - // check version - uint16_t major_version = (0xffff & (superblock.version >> 16)); - uint16_t minor_version = (0xffff & (superblock.version >> 0)); - if (major_version != lfs_fs_disk_version_major(lfs) - || minor_version > lfs_fs_disk_version_minor(lfs)) { - LFS_ERROR("Invalid version " - "v%"PRIu16".%"PRIu16" != v%"PRIu16".%"PRIu16, - major_version, - minor_version, - lfs_fs_disk_version_major(lfs), - lfs_fs_disk_version_minor(lfs)); - err = LFS_ERR_INVAL; - goto cleanup; - } - - // found older minor version? set an in-device only bit in the - // gstate so we know we need to rewrite the superblock before - // the first write - bool needssuperblock = false; - if (minor_version < lfs_fs_disk_version_minor(lfs)) { - LFS_DEBUG("Found older minor version " - "v%"PRIu16".%"PRIu16" < v%"PRIu16".%"PRIu16, - major_version, - minor_version, - lfs_fs_disk_version_major(lfs), - lfs_fs_disk_version_minor(lfs)); - needssuperblock = true; - } - // note this bit is reserved on disk, so fetching more gstate - // will not interfere here - lfs_fs_prepsuperblock(lfs, needssuperblock); - - // check superblock configuration - if (superblock.name_max) { - if (superblock.name_max > lfs->name_max) { - LFS_ERROR("Unsupported name_max (%"PRIu32" > %"PRIu32")", - superblock.name_max, lfs->name_max); - err = LFS_ERR_INVAL; - goto cleanup; - } - - lfs->name_max = superblock.name_max; - } - - if (superblock.file_max) { - if (superblock.file_max > lfs->file_max) { - LFS_ERROR("Unsupported file_max (%"PRIu32" > %"PRIu32")", - superblock.file_max, lfs->file_max); - err = LFS_ERR_INVAL; - goto cleanup; - } - - lfs->file_max = superblock.file_max; - } - - if (superblock.attr_max) { - if (superblock.attr_max > lfs->attr_max) { - LFS_ERROR("Unsupported attr_max (%"PRIu32" > %"PRIu32")", - superblock.attr_max, lfs->attr_max); - err = LFS_ERR_INVAL; - goto cleanup; - } - - lfs->attr_max = superblock.attr_max; - - // we also need to update inline_max in case attr_max changed - lfs->inline_max = lfs_min(lfs->inline_max, lfs->attr_max); - } - - // this is where we get the block_count from disk if block_count=0 - if (lfs->cfg->block_count - && superblock.block_count != lfs->cfg->block_count) { - LFS_ERROR("Invalid block count (%"PRIu32" != %"PRIu32")", - superblock.block_count, lfs->cfg->block_count); - err = LFS_ERR_INVAL; - goto cleanup; - } - - lfs->block_count = superblock.block_count; - - if (superblock.block_size != lfs->cfg->block_size) { - LFS_ERROR("Invalid block size (%"PRIu32" != %"PRIu32")", - superblock.block_size, lfs->cfg->block_size); - err = LFS_ERR_INVAL; - goto cleanup; - } - } - - // has gstate? - err = lfs_dir_getgstate(lfs, &dir, &lfs->gstate); - if (err) { - goto cleanup; - } - } - - // update littlefs with gstate - if (!lfs_gstate_iszero(&lfs->gstate)) { - LFS_DEBUG("Found pending gstate 0x%08"PRIx32"%08"PRIx32"%08"PRIx32, - lfs->gstate.tag, - lfs->gstate.pair[0], - lfs->gstate.pair[1]); - } - lfs->gstate.tag += !lfs_tag_isvalid(lfs->gstate.tag); - lfs->gdisk = lfs->gstate; - - // setup free lookahead, to distribute allocations uniformly across - // boots, we start the allocator at a random location - lfs->lookahead.start = lfs->seed % lfs->block_count; - lfs_alloc_drop(lfs); - - return 0; - -cleanup: - lfs_unmount_(lfs); - return err; -} - -static int lfs_unmount_(lfs_t *lfs) { - return lfs_deinit(lfs); -} - - -/// Filesystem filesystem operations /// -static int lfs_fs_stat_(lfs_t *lfs, struct lfs_fsinfo *fsinfo) { - // if the superblock is up-to-date, we must be on the most recent - // minor version of littlefs - if (!lfs_gstate_needssuperblock(&lfs->gstate)) { - fsinfo->disk_version = lfs_fs_disk_version(lfs); - - // otherwise we need to read the minor version on disk - } else { - // fetch the superblock - lfs_mdir_t dir; - int err = lfs_dir_fetch(lfs, &dir, lfs->root); - if (err) { - return err; - } - - lfs_superblock_t superblock; - lfs_stag_t tag = lfs_dir_get(lfs, &dir, LFS_MKTAG(0x7ff, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_INLINESTRUCT, 0, sizeof(superblock)), - &superblock); - if (tag < 0) { - return tag; - } - lfs_superblock_fromle32(&superblock); - - // read the on-disk version - fsinfo->disk_version = superblock.version; - } - - // filesystem geometry - fsinfo->block_size = lfs->cfg->block_size; - fsinfo->block_count = lfs->block_count; - - // other on-disk configuration, we cache all of these for internal use - fsinfo->name_max = lfs->name_max; - fsinfo->file_max = lfs->file_max; - fsinfo->attr_max = lfs->attr_max; - - return 0; -} - -int lfs_fs_traverse_(lfs_t *lfs, - int (*cb)(void *data, lfs_block_t block), void *data, - bool includeorphans) { - // iterate over metadata pairs - lfs_mdir_t dir = {.tail = {0, 1}}; - -#ifdef LFS_MIGRATE - // also consider v1 blocks during migration - if (lfs->lfs1) { - int err = lfs1_traverse(lfs, cb, data); - if (err) { - return err; - } - - dir.tail[0] = lfs->root[0]; - dir.tail[1] = lfs->root[1]; - } -#endif - - struct lfs_tortoise_t tortoise = { - .pair = {LFS_BLOCK_NULL, LFS_BLOCK_NULL}, - .i = 1, - .period = 1, - }; - int err = LFS_ERR_OK; - while (!lfs_pair_isnull(dir.tail)) { - err = lfs_tortoise_detectcycles(&dir, &tortoise); - if (err < 0) { - return LFS_ERR_CORRUPT; - } - - for (int i = 0; i < 2; i++) { - int err = cb(data, dir.tail[i]); - if (err) { - return err; - } - } - - // iterate through ids in directory - int err = lfs_dir_fetch(lfs, &dir, dir.tail); - if (err) { - return err; - } - - for (uint16_t id = 0; id < dir.count; id++) { - struct lfs_ctz ctz; - lfs_stag_t tag = lfs_dir_get(lfs, &dir, LFS_MKTAG(0x700, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_STRUCT, id, sizeof(ctz)), &ctz); - if (tag < 0) { - if (tag == LFS_ERR_NOENT) { - continue; - } - return tag; - } - lfs_ctz_fromle32(&ctz); - - if (lfs_tag_type3(tag) == LFS_TYPE_CTZSTRUCT) { - err = lfs_ctz_traverse(lfs, NULL, &lfs->rcache, - ctz.head, ctz.size, cb, data); - if (err) { - return err; - } - } else if (includeorphans && - lfs_tag_type3(tag) == LFS_TYPE_DIRSTRUCT) { - for (int i = 0; i < 2; i++) { - err = cb(data, (&ctz.head)[i]); - if (err) { - return err; - } - } - } - } - } - -#ifndef LFS_READONLY - // iterate over any open files - for (lfs_file_t *f = (lfs_file_t*)lfs->mlist; f; f = f->next) { - if (f->type != LFS_TYPE_REG) { - continue; - } - - if (((f->flags & LFS_F_DIRTY) || (f->flags & LFS_F_DUSTY)) - && !(f->flags & LFS_F_INLINE)) { - int err = lfs_ctz_traverse(lfs, &f->cache, &lfs->rcache, - f->ctz.head, f->ctz.size, cb, data); - if (err) { - return err; - } - } - - if ((f->flags & LFS_F_WRITING) && !(f->flags & LFS_F_INLINE)) { - int err = lfs_ctz_traverse(lfs, &f->cache, &lfs->rcache, - f->block, f->pos, cb, data); - if (err) { - return err; - } - } - } -#endif - - return 0; -} - -#ifndef LFS_READONLY -static int lfs_fs_pred(lfs_t *lfs, - const lfs_block_t pair[2], lfs_mdir_t *pdir) { - // iterate over all directory directory entries - pdir->tail[0] = 0; - pdir->tail[1] = 1; - struct lfs_tortoise_t tortoise = { - .pair = {LFS_BLOCK_NULL, LFS_BLOCK_NULL}, - .i = 1, - .period = 1, - }; - int err = LFS_ERR_OK; - while (!lfs_pair_isnull(pdir->tail)) { - err = lfs_tortoise_detectcycles(pdir, &tortoise); - if (err < 0) { - return LFS_ERR_CORRUPT; - } - - if (lfs_pair_cmp(pdir->tail, pair) == 0) { - return 0; - } - - int err = lfs_dir_fetch(lfs, pdir, pdir->tail); - if (err) { - return err; - } - } - - return LFS_ERR_NOENT; -} -#endif - -#ifndef LFS_READONLY -struct lfs_fs_parent_match { - lfs_t *lfs; - const lfs_block_t pair[2]; -}; -#endif - -#ifndef LFS_READONLY -static int lfs_fs_parent_match(void *data, - lfs_tag_t tag, const void *buffer) { - struct lfs_fs_parent_match *find = data; - lfs_t *lfs = find->lfs; - const struct lfs_diskoff *disk = buffer; - (void)tag; - - lfs_block_t child[2]; - int err = lfs_bd_read(lfs, - &lfs->pcache, &lfs->rcache, lfs->cfg->block_size, - disk->block, disk->off, &child, sizeof(child)); - if (err) { - return err; - } - - lfs_pair_fromle32(child); - return (lfs_pair_cmp(child, find->pair) == 0) ? LFS_CMP_EQ : LFS_CMP_LT; -} -#endif - -#ifndef LFS_READONLY -static lfs_stag_t lfs_fs_parent(lfs_t *lfs, const lfs_block_t pair[2], - lfs_mdir_t *parent) { - // use fetchmatch with callback to find pairs - parent->tail[0] = 0; - parent->tail[1] = 1; - struct lfs_tortoise_t tortoise = { - .pair = {LFS_BLOCK_NULL, LFS_BLOCK_NULL}, - .i = 1, - .period = 1, - }; - int err = LFS_ERR_OK; - while (!lfs_pair_isnull(parent->tail)) { - err = lfs_tortoise_detectcycles(parent, &tortoise); - if (err < 0) { - return err; - } - - lfs_stag_t tag = lfs_dir_fetchmatch(lfs, parent, parent->tail, - LFS_MKTAG(0x7ff, 0, 0x3ff), - LFS_MKTAG(LFS_TYPE_DIRSTRUCT, 0, 8), - NULL, - lfs_fs_parent_match, &(struct lfs_fs_parent_match){ - lfs, {pair[0], pair[1]}}); - if (tag && tag != LFS_ERR_NOENT) { - return tag; - } - } - - return LFS_ERR_NOENT; -} -#endif - -static void lfs_fs_prepsuperblock(lfs_t *lfs, bool needssuperblock) { - lfs->gstate.tag = (lfs->gstate.tag & ~LFS_MKTAG(0, 0, 0x200)) - | (uint32_t)needssuperblock << 9; -} - -#ifndef LFS_READONLY -static int lfs_fs_preporphans(lfs_t *lfs, int8_t orphans) { - LFS_ASSERT(lfs_tag_size(lfs->gstate.tag) > 0x000 || orphans >= 0); - LFS_ASSERT(lfs_tag_size(lfs->gstate.tag) < 0x1ff || orphans <= 0); - lfs->gstate.tag += orphans; - lfs->gstate.tag = ((lfs->gstate.tag & ~LFS_MKTAG(0x800, 0, 0)) | - ((uint32_t)lfs_gstate_hasorphans(&lfs->gstate) << 31)); - - return 0; -} -#endif - -#ifndef LFS_READONLY -static void lfs_fs_prepmove(lfs_t *lfs, - uint16_t id, const lfs_block_t pair[2]) { - lfs->gstate.tag = ((lfs->gstate.tag & ~LFS_MKTAG(0x7ff, 0x3ff, 0)) | - ((id != 0x3ff) ? LFS_MKTAG(LFS_TYPE_DELETE, id, 0) : 0)); - lfs->gstate.pair[0] = (id != 0x3ff) ? pair[0] : 0; - lfs->gstate.pair[1] = (id != 0x3ff) ? pair[1] : 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_fs_desuperblock(lfs_t *lfs) { - if (!lfs_gstate_needssuperblock(&lfs->gstate)) { - return 0; - } - - LFS_DEBUG("Rewriting superblock {0x%"PRIx32", 0x%"PRIx32"}", - lfs->root[0], - lfs->root[1]); - - lfs_mdir_t root; - int err = lfs_dir_fetch(lfs, &root, lfs->root); - if (err) { - return err; - } - - // write a new superblock - lfs_superblock_t superblock = { - .version = lfs_fs_disk_version(lfs), - .block_size = lfs->cfg->block_size, - .block_count = lfs->block_count, - .name_max = lfs->name_max, - .file_max = lfs->file_max, - .attr_max = lfs->attr_max, - }; - - lfs_superblock_tole32(&superblock); - err = lfs_dir_commit(lfs, &root, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_INLINESTRUCT, 0, sizeof(superblock)), - &superblock})); - if (err) { - return err; - } - - lfs_fs_prepsuperblock(lfs, false); - return 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_fs_demove(lfs_t *lfs) { - if (!lfs_gstate_hasmove(&lfs->gdisk)) { - return 0; - } - - // Fix bad moves - LFS_DEBUG("Fixing move {0x%"PRIx32", 0x%"PRIx32"} 0x%"PRIx16, - lfs->gdisk.pair[0], - lfs->gdisk.pair[1], - lfs_tag_id(lfs->gdisk.tag)); - - // no other gstate is supported at this time, so if we found something else - // something most likely went wrong in gstate calculation - LFS_ASSERT(lfs_tag_type3(lfs->gdisk.tag) == LFS_TYPE_DELETE); - - // fetch and delete the moved entry - lfs_mdir_t movedir; - int err = lfs_dir_fetch(lfs, &movedir, lfs->gdisk.pair); - if (err) { - return err; - } - - // prep gstate and delete move id - uint16_t moveid = lfs_tag_id(lfs->gdisk.tag); - lfs_fs_prepmove(lfs, 0x3ff, NULL); - err = lfs_dir_commit(lfs, &movedir, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_DELETE, moveid, 0), NULL})); - if (err) { - return err; - } - - return 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_fs_deorphan(lfs_t *lfs, bool powerloss) { - if (!lfs_gstate_hasorphans(&lfs->gstate)) { - return 0; - } - - // Check for orphans in two separate passes: - // - 1 for half-orphans (relocations) - // - 2 for full-orphans (removes/renames) - // - // Two separate passes are needed as half-orphans can contain outdated - // references to full-orphans, effectively hiding them from the deorphan - // search. - // - int pass = 0; - while (pass < 2) { - // Fix any orphans - lfs_mdir_t pdir = {.split = true, .tail = {0, 1}}; - lfs_mdir_t dir; - bool moreorphans = false; - - // iterate over all directory directory entries - while (!lfs_pair_isnull(pdir.tail)) { - int err = lfs_dir_fetch(lfs, &dir, pdir.tail); - if (err) { - return err; - } - - // check head blocks for orphans - if (!pdir.split) { - // check if we have a parent - lfs_mdir_t parent; - lfs_stag_t tag = lfs_fs_parent(lfs, pdir.tail, &parent); - if (tag < 0 && tag != LFS_ERR_NOENT) { - return tag; - } - - if (pass == 0 && tag != LFS_ERR_NOENT) { - lfs_block_t pair[2]; - lfs_stag_t state = lfs_dir_get(lfs, &parent, - LFS_MKTAG(0x7ff, 0x3ff, 0), tag, pair); - if (state < 0) { - return state; - } - lfs_pair_fromle32(pair); - - if (!lfs_pair_issync(pair, pdir.tail)) { - // we have desynced - LFS_DEBUG("Fixing half-orphan " - "{0x%"PRIx32", 0x%"PRIx32"} " - "-> {0x%"PRIx32", 0x%"PRIx32"}", - pdir.tail[0], pdir.tail[1], pair[0], pair[1]); - - // fix pending move in this pair? this looks like an - // optimization but is in fact _required_ since - // relocating may outdate the move. - uint16_t moveid = 0x3ff; - if (lfs_gstate_hasmovehere(&lfs->gstate, pdir.pair)) { - moveid = lfs_tag_id(lfs->gstate.tag); - LFS_DEBUG("Fixing move while fixing orphans " - "{0x%"PRIx32", 0x%"PRIx32"} 0x%"PRIx16"\n", - pdir.pair[0], pdir.pair[1], moveid); - lfs_fs_prepmove(lfs, 0x3ff, NULL); - } - - lfs_pair_tole32(pair); - state = lfs_dir_orphaningcommit(lfs, &pdir, LFS_MKATTRS( - {LFS_MKTAG_IF(moveid != 0x3ff, - LFS_TYPE_DELETE, moveid, 0), NULL}, - {LFS_MKTAG(LFS_TYPE_SOFTTAIL, 0x3ff, 8), - pair})); - lfs_pair_fromle32(pair); - if (state < 0) { - return state; - } - - // did our commit create more orphans? - if (state == LFS_OK_ORPHANED) { - moreorphans = true; - } - - // refetch tail - continue; - } - } - - // note we only check for full orphans if we may have had a - // power-loss, otherwise orphans are created intentionally - // during operations such as lfs_mkdir - if (pass == 1 && tag == LFS_ERR_NOENT && powerloss) { - // we are an orphan - LFS_DEBUG("Fixing orphan {0x%"PRIx32", 0x%"PRIx32"}", - pdir.tail[0], pdir.tail[1]); - - // steal state - err = lfs_dir_getgstate(lfs, &dir, &lfs->gdelta); - if (err) { - return err; - } - - // steal tail - lfs_pair_tole32(dir.tail); - int state = lfs_dir_orphaningcommit(lfs, &pdir, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_TAIL + dir.split, 0x3ff, 8), - dir.tail})); - lfs_pair_fromle32(dir.tail); - if (state < 0) { - return state; - } - - // did our commit create more orphans? - if (state == LFS_OK_ORPHANED) { - moreorphans = true; - } - - // refetch tail - continue; - } - } - - pdir = dir; - } - - pass = moreorphans ? 0 : pass+1; - } - - // mark orphans as fixed - return lfs_fs_preporphans(lfs, -lfs_gstate_getorphans(&lfs->gstate)); -} -#endif - -#ifndef LFS_READONLY -static int lfs_fs_forceconsistency(lfs_t *lfs) { - int err = lfs_fs_desuperblock(lfs); - if (err) { - return err; - } - - err = lfs_fs_demove(lfs); - if (err) { - return err; - } - - err = lfs_fs_deorphan(lfs, true); - if (err) { - return err; - } - - return 0; -} -#endif - -#ifndef LFS_READONLY -static int lfs_fs_mkconsistent_(lfs_t *lfs) { - // lfs_fs_forceconsistency does most of the work here - int err = lfs_fs_forceconsistency(lfs); - if (err) { - return err; - } - - // do we have any pending gstate? - lfs_gstate_t delta = {0}; - lfs_gstate_xor(&delta, &lfs->gdisk); - lfs_gstate_xor(&delta, &lfs->gstate); - if (!lfs_gstate_iszero(&delta)) { - // lfs_dir_commit will implicitly write out any pending gstate - lfs_mdir_t root; - err = lfs_dir_fetch(lfs, &root, lfs->root); - if (err) { - return err; - } - - err = lfs_dir_commit(lfs, &root, NULL, 0); - if (err) { - return err; - } - } - - return 0; -} -#endif - -static int lfs_fs_size_count(void *p, lfs_block_t block) { - (void)block; - lfs_size_t *size = p; - *size += 1; - return 0; -} - -static lfs_ssize_t lfs_fs_size_(lfs_t *lfs) { - lfs_size_t size = 0; - int err = lfs_fs_traverse_(lfs, lfs_fs_size_count, &size, false); - if (err) { - return err; - } - - return size; -} - -// explicit garbage collection -#ifndef LFS_READONLY -static int lfs_fs_gc_(lfs_t *lfs) { - // force consistency, even if we're not necessarily going to write, - // because this function is supposed to take care of janitorial work - // isn't it? - int err = lfs_fs_forceconsistency(lfs); - if (err) { - return err; - } - - // try to compact metadata pairs, note we can't really accomplish - // anything if compact_thresh doesn't at least leave a prog_size - // available - if (lfs->cfg->compact_thresh - < lfs->cfg->block_size - lfs->cfg->prog_size) { - // iterate over all mdirs - lfs_mdir_t mdir = {.tail = {0, 1}}; - while (!lfs_pair_isnull(mdir.tail)) { - err = lfs_dir_fetch(lfs, &mdir, mdir.tail); - if (err) { - return err; - } - - // not erased? exceeds our compaction threshold? - if (!mdir.erased || ((lfs->cfg->compact_thresh == 0) - ? mdir.off > lfs->cfg->block_size - lfs->cfg->block_size/8 - : mdir.off > lfs->cfg->compact_thresh)) { - // the easiest way to trigger a compaction is to mark - // the mdir as unerased and add an empty commit - mdir.erased = false; - err = lfs_dir_commit(lfs, &mdir, NULL, 0); - if (err) { - return err; - } - } - } - } - - // try to populate the lookahead buffer, unless it's already full - if (lfs->lookahead.size < lfs_min( - 8 * lfs->cfg->lookahead_size, - lfs->block_count)) { - err = lfs_alloc_scan(lfs); - if (err) { - return err; - } - } - - return 0; -} -#endif - -#ifndef LFS_READONLY -#ifdef LFS_SHRINKNONRELOCATING -static int lfs_shrink_checkblock(void *data, lfs_block_t block) { - lfs_size_t threshold = *((lfs_size_t*)data); - if (block >= threshold) { - return LFS_ERR_NOTEMPTY; - } - return 0; -} -#endif - -static int lfs_fs_grow_(lfs_t *lfs, lfs_size_t block_count) { - int err; - - if (block_count == lfs->block_count) { - return 0; - } - - -#ifndef LFS_SHRINKNONRELOCATING - // shrinking is not supported - LFS_ASSERT(block_count >= lfs->block_count); -#endif -#ifdef LFS_SHRINKNONRELOCATING - if (block_count < lfs->block_count) { - err = lfs_fs_traverse_(lfs, lfs_shrink_checkblock, &block_count, true); - if (err) { - return err; - } - } -#endif - - lfs->block_count = block_count; - - // fetch the root - lfs_mdir_t root; - err = lfs_dir_fetch(lfs, &root, lfs->root); - if (err) { - return err; - } - - // update the superblock - lfs_superblock_t superblock; - lfs_stag_t tag = lfs_dir_get(lfs, &root, LFS_MKTAG(0x7ff, 0x3ff, 0), - LFS_MKTAG(LFS_TYPE_INLINESTRUCT, 0, sizeof(superblock)), - &superblock); - if (tag < 0) { - return tag; - } - lfs_superblock_fromle32(&superblock); - - superblock.block_count = lfs->block_count; - - lfs_superblock_tole32(&superblock); - err = lfs_dir_commit(lfs, &root, LFS_MKATTRS( - {tag, &superblock})); - if (err) { - return err; - } - return 0; -} -#endif - -#ifdef LFS_MIGRATE -////// Migration from littelfs v1 below this ////// - -/// Version info /// - -// Software library version -// Major (top-nibble), incremented on backwards incompatible changes -// Minor (bottom-nibble), incremented on feature additions -#define LFS1_VERSION 0x00010007 -#define LFS1_VERSION_MAJOR (0xffff & (LFS1_VERSION >> 16)) -#define LFS1_VERSION_MINOR (0xffff & (LFS1_VERSION >> 0)) - -// Version of On-disk data structures -// Major (top-nibble), incremented on backwards incompatible changes -// Minor (bottom-nibble), incremented on feature additions -#define LFS1_DISK_VERSION 0x00010001 -#define LFS1_DISK_VERSION_MAJOR (0xffff & (LFS1_DISK_VERSION >> 16)) -#define LFS1_DISK_VERSION_MINOR (0xffff & (LFS1_DISK_VERSION >> 0)) - - -/// v1 Definitions /// - -// File types -enum lfs1_type { - LFS1_TYPE_REG = 0x11, - LFS1_TYPE_DIR = 0x22, - LFS1_TYPE_SUPERBLOCK = 0x2e, -}; - -typedef struct lfs1 { - lfs_block_t root[2]; -} lfs1_t; - -typedef struct lfs1_entry { - lfs_off_t off; - - struct lfs1_disk_entry { - uint8_t type; - uint8_t elen; - uint8_t alen; - uint8_t nlen; - union { - struct { - lfs_block_t head; - lfs_size_t size; - } file; - lfs_block_t dir[2]; - } u; - } d; -} lfs1_entry_t; - -typedef struct lfs1_dir { - struct lfs1_dir *next; - lfs_block_t pair[2]; - lfs_off_t off; - - lfs_block_t head[2]; - lfs_off_t pos; - - struct lfs1_disk_dir { - uint32_t rev; - lfs_size_t size; - lfs_block_t tail[2]; - } d; -} lfs1_dir_t; - -typedef struct lfs1_superblock { - lfs_off_t off; - - struct lfs1_disk_superblock { - uint8_t type; - uint8_t elen; - uint8_t alen; - uint8_t nlen; - lfs_block_t root[2]; - uint32_t block_size; - uint32_t block_count; - uint32_t version; - char magic[8]; - } d; -} lfs1_superblock_t; - - -/// Low-level wrappers v1->v2 /// -static void lfs1_crc(uint32_t *crc, const void *buffer, size_t size) { - *crc = lfs_crc(*crc, buffer, size); -} - -static int lfs1_bd_read(lfs_t *lfs, lfs_block_t block, - lfs_off_t off, void *buffer, lfs_size_t size) { - // if we ever do more than writes to alternating pairs, - // this may need to consider pcache - return lfs_bd_read(lfs, &lfs->pcache, &lfs->rcache, size, - block, off, buffer, size); -} - -static int lfs1_bd_crc(lfs_t *lfs, lfs_block_t block, - lfs_off_t off, lfs_size_t size, uint32_t *crc) { - for (lfs_off_t i = 0; i < size; i++) { - uint8_t c; - int err = lfs1_bd_read(lfs, block, off+i, &c, 1); - if (err) { - return err; - } - - lfs1_crc(crc, &c, 1); - } - - return 0; -} - - -/// Endian swapping functions /// -static void lfs1_dir_fromle32(struct lfs1_disk_dir *d) { - d->rev = lfs_fromle32(d->rev); - d->size = lfs_fromle32(d->size); - d->tail[0] = lfs_fromle32(d->tail[0]); - d->tail[1] = lfs_fromle32(d->tail[1]); -} - -static void lfs1_dir_tole32(struct lfs1_disk_dir *d) { - d->rev = lfs_tole32(d->rev); - d->size = lfs_tole32(d->size); - d->tail[0] = lfs_tole32(d->tail[0]); - d->tail[1] = lfs_tole32(d->tail[1]); -} - -static void lfs1_entry_fromle32(struct lfs1_disk_entry *d) { - d->u.dir[0] = lfs_fromle32(d->u.dir[0]); - d->u.dir[1] = lfs_fromle32(d->u.dir[1]); -} - -static void lfs1_entry_tole32(struct lfs1_disk_entry *d) { - d->u.dir[0] = lfs_tole32(d->u.dir[0]); - d->u.dir[1] = lfs_tole32(d->u.dir[1]); -} - -static void lfs1_superblock_fromle32(struct lfs1_disk_superblock *d) { - d->root[0] = lfs_fromle32(d->root[0]); - d->root[1] = lfs_fromle32(d->root[1]); - d->block_size = lfs_fromle32(d->block_size); - d->block_count = lfs_fromle32(d->block_count); - d->version = lfs_fromle32(d->version); -} - - -///// Metadata pair and directory operations /// -static inline lfs_size_t lfs1_entry_size(const lfs1_entry_t *entry) { - return 4 + entry->d.elen + entry->d.alen + entry->d.nlen; -} - -static int lfs1_dir_fetch(lfs_t *lfs, - lfs1_dir_t *dir, const lfs_block_t pair[2]) { - // copy out pair, otherwise may be aliasing dir - const lfs_block_t tpair[2] = {pair[0], pair[1]}; - bool valid = false; - - // check both blocks for the most recent revision - for (int i = 0; i < 2; i++) { - struct lfs1_disk_dir test; - int err = lfs1_bd_read(lfs, tpair[i], 0, &test, sizeof(test)); - lfs1_dir_fromle32(&test); - if (err) { - if (err == LFS_ERR_CORRUPT) { - continue; - } - return err; - } - - if (valid && lfs_scmp(test.rev, dir->d.rev) < 0) { - continue; - } - - if ((0x7fffffff & test.size) < sizeof(test)+4 || - (0x7fffffff & test.size) > lfs->cfg->block_size) { - continue; - } - - uint32_t crc = 0xffffffff; - lfs1_dir_tole32(&test); - lfs1_crc(&crc, &test, sizeof(test)); - lfs1_dir_fromle32(&test); - err = lfs1_bd_crc(lfs, tpair[i], sizeof(test), - (0x7fffffff & test.size) - sizeof(test), &crc); - if (err) { - if (err == LFS_ERR_CORRUPT) { - continue; - } - return err; - } - - if (crc != 0) { - continue; - } - - valid = true; - - // setup dir in case it's valid - dir->pair[0] = tpair[(i+0) % 2]; - dir->pair[1] = tpair[(i+1) % 2]; - dir->off = sizeof(dir->d); - dir->d = test; - } - - if (!valid) { - LFS_ERROR("Corrupted dir pair at {0x%"PRIx32", 0x%"PRIx32"}", - tpair[0], tpair[1]); - return LFS_ERR_CORRUPT; - } - - return 0; -} - -static int lfs1_dir_next(lfs_t *lfs, lfs1_dir_t *dir, lfs1_entry_t *entry) { - while (dir->off + sizeof(entry->d) > (0x7fffffff & dir->d.size)-4) { - if (!(0x80000000 & dir->d.size)) { - entry->off = dir->off; - return LFS_ERR_NOENT; - } - - int err = lfs1_dir_fetch(lfs, dir, dir->d.tail); - if (err) { - return err; - } - - dir->off = sizeof(dir->d); - dir->pos += sizeof(dir->d) + 4; - } - - int err = lfs1_bd_read(lfs, dir->pair[0], dir->off, - &entry->d, sizeof(entry->d)); - lfs1_entry_fromle32(&entry->d); - if (err) { - return err; - } - - entry->off = dir->off; - dir->off += lfs1_entry_size(entry); - dir->pos += lfs1_entry_size(entry); - return 0; -} - -/// littlefs v1 specific operations /// -int lfs1_traverse(lfs_t *lfs, int (*cb)(void*, lfs_block_t), void *data) { - if (lfs_pair_isnull(lfs->lfs1->root)) { - return 0; - } - - // iterate over metadata pairs - lfs1_dir_t dir; - lfs1_entry_t entry; - lfs_block_t cwd[2] = {0, 1}; - - while (true) { - for (int i = 0; i < 2; i++) { - int err = cb(data, cwd[i]); - if (err) { - return err; - } - } - - int err = lfs1_dir_fetch(lfs, &dir, cwd); - if (err) { - return err; - } - - // iterate over contents - while (dir.off + sizeof(entry.d) <= (0x7fffffff & dir.d.size)-4) { - err = lfs1_bd_read(lfs, dir.pair[0], dir.off, - &entry.d, sizeof(entry.d)); - lfs1_entry_fromle32(&entry.d); - if (err) { - return err; - } - - dir.off += lfs1_entry_size(&entry); - if ((0x70 & entry.d.type) == (0x70 & LFS1_TYPE_REG)) { - err = lfs_ctz_traverse(lfs, NULL, &lfs->rcache, - entry.d.u.file.head, entry.d.u.file.size, cb, data); - if (err) { - return err; - } - } - } - - // we also need to check if we contain a threaded v2 directory - lfs_mdir_t dir2 = {.split=true, .tail={cwd[0], cwd[1]}}; - while (dir2.split) { - err = lfs_dir_fetch(lfs, &dir2, dir2.tail); - if (err) { - break; - } - - for (int i = 0; i < 2; i++) { - err = cb(data, dir2.pair[i]); - if (err) { - return err; - } - } - } - - cwd[0] = dir.d.tail[0]; - cwd[1] = dir.d.tail[1]; - - if (lfs_pair_isnull(cwd)) { - break; - } - } - - return 0; -} - -static int lfs1_moved(lfs_t *lfs, const void *e) { - if (lfs_pair_isnull(lfs->lfs1->root)) { - return 0; - } - - // skip superblock - lfs1_dir_t cwd; - int err = lfs1_dir_fetch(lfs, &cwd, (const lfs_block_t[2]){0, 1}); - if (err) { - return err; - } - - // iterate over all directory directory entries - lfs1_entry_t entry; - while (!lfs_pair_isnull(cwd.d.tail)) { - err = lfs1_dir_fetch(lfs, &cwd, cwd.d.tail); - if (err) { - return err; - } - - while (true) { - err = lfs1_dir_next(lfs, &cwd, &entry); - if (err && err != LFS_ERR_NOENT) { - return err; - } - - if (err == LFS_ERR_NOENT) { - break; - } - - if (!(0x80 & entry.d.type) && - memcmp(&entry.d.u, e, sizeof(entry.d.u)) == 0) { - return true; - } - } - } - - return false; -} - -/// Filesystem operations /// -static int lfs1_mount(lfs_t *lfs, struct lfs1 *lfs1, - const struct lfs_config *cfg) { - int err = 0; - { - err = lfs_init(lfs, cfg); - if (err) { - return err; - } - - lfs->lfs1 = lfs1; - lfs->lfs1->root[0] = LFS_BLOCK_NULL; - lfs->lfs1->root[1] = LFS_BLOCK_NULL; - - // setup free lookahead - lfs->lookahead.start = 0; - lfs->lookahead.size = 0; - lfs->lookahead.next = 0; - lfs_alloc_ckpoint(lfs); - - // load superblock - lfs1_dir_t dir; - lfs1_superblock_t superblock; - err = lfs1_dir_fetch(lfs, &dir, (const lfs_block_t[2]){0, 1}); - if (err && err != LFS_ERR_CORRUPT) { - goto cleanup; - } - - if (!err) { - err = lfs1_bd_read(lfs, dir.pair[0], sizeof(dir.d), - &superblock.d, sizeof(superblock.d)); - lfs1_superblock_fromle32(&superblock.d); - if (err) { - goto cleanup; - } - - lfs->lfs1->root[0] = superblock.d.root[0]; - lfs->lfs1->root[1] = superblock.d.root[1]; - } - - if (err || memcmp(superblock.d.magic, "littlefs", 8) != 0) { - LFS_ERROR("Invalid superblock at {0x%"PRIx32", 0x%"PRIx32"}", - 0, 1); - err = LFS_ERR_CORRUPT; - goto cleanup; - } - - uint16_t major_version = (0xffff & (superblock.d.version >> 16)); - uint16_t minor_version = (0xffff & (superblock.d.version >> 0)); - if ((major_version != LFS1_DISK_VERSION_MAJOR || - minor_version > LFS1_DISK_VERSION_MINOR)) { - LFS_ERROR("Invalid version v%d.%d", major_version, minor_version); - err = LFS_ERR_INVAL; - goto cleanup; - } - - return 0; - } - -cleanup: - lfs_deinit(lfs); - return err; -} - -static int lfs1_unmount(lfs_t *lfs) { - return lfs_deinit(lfs); -} - -/// v1 migration /// -static int lfs_migrate_(lfs_t *lfs, const struct lfs_config *cfg) { - struct lfs1 lfs1; - - // Indeterminate filesystem size not allowed for migration. - LFS_ASSERT(cfg->block_count != 0); - - int err = lfs1_mount(lfs, &lfs1, cfg); - if (err) { - return err; - } - - { - // iterate through each directory, copying over entries - // into new directory - lfs1_dir_t dir1; - lfs_mdir_t dir2; - dir1.d.tail[0] = lfs->lfs1->root[0]; - dir1.d.tail[1] = lfs->lfs1->root[1]; - while (!lfs_pair_isnull(dir1.d.tail)) { - // iterate old dir - err = lfs1_dir_fetch(lfs, &dir1, dir1.d.tail); - if (err) { - goto cleanup; - } - - // create new dir and bind as temporary pretend root - err = lfs_dir_alloc(lfs, &dir2); - if (err) { - goto cleanup; - } - - dir2.rev = dir1.d.rev; - dir1.head[0] = dir1.pair[0]; - dir1.head[1] = dir1.pair[1]; - lfs->root[0] = dir2.pair[0]; - lfs->root[1] = dir2.pair[1]; - - err = lfs_dir_commit(lfs, &dir2, NULL, 0); - if (err) { - goto cleanup; - } - - while (true) { - lfs1_entry_t entry1; - err = lfs1_dir_next(lfs, &dir1, &entry1); - if (err && err != LFS_ERR_NOENT) { - goto cleanup; - } - - if (err == LFS_ERR_NOENT) { - break; - } - - // check that entry has not been moved - if (entry1.d.type & 0x80) { - int moved = lfs1_moved(lfs, &entry1.d.u); - if (moved < 0) { - err = moved; - goto cleanup; - } - - if (moved) { - continue; - } - - entry1.d.type &= ~0x80; - } - - // also fetch name - char name[LFS_NAME_MAX+1]; - memset(name, 0, sizeof(name)); - err = lfs1_bd_read(lfs, dir1.pair[0], - entry1.off + 4+entry1.d.elen+entry1.d.alen, - name, entry1.d.nlen); - if (err) { - goto cleanup; - } - - bool isdir = (entry1.d.type == LFS1_TYPE_DIR); - - // create entry in new dir - err = lfs_dir_fetch(lfs, &dir2, lfs->root); - if (err) { - goto cleanup; - } - - uint16_t id; - err = lfs_dir_find(lfs, &dir2, &(const char*){name}, &id); - if (!(err == LFS_ERR_NOENT && id != 0x3ff)) { - err = (err < 0) ? err : LFS_ERR_EXIST; - goto cleanup; - } - - lfs1_entry_tole32(&entry1.d); - err = lfs_dir_commit(lfs, &dir2, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_CREATE, id, 0), NULL}, - {LFS_MKTAG_IF_ELSE(isdir, - LFS_TYPE_DIR, id, entry1.d.nlen, - LFS_TYPE_REG, id, entry1.d.nlen), - name}, - {LFS_MKTAG_IF_ELSE(isdir, - LFS_TYPE_DIRSTRUCT, id, sizeof(entry1.d.u), - LFS_TYPE_CTZSTRUCT, id, sizeof(entry1.d.u)), - &entry1.d.u})); - lfs1_entry_fromle32(&entry1.d); - if (err) { - goto cleanup; - } - } - - if (!lfs_pair_isnull(dir1.d.tail)) { - // find last block and update tail to thread into fs - err = lfs_dir_fetch(lfs, &dir2, lfs->root); - if (err) { - goto cleanup; - } - - while (dir2.split) { - err = lfs_dir_fetch(lfs, &dir2, dir2.tail); - if (err) { - goto cleanup; - } - } - - lfs_pair_tole32(dir2.pair); - err = lfs_dir_commit(lfs, &dir2, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_SOFTTAIL, 0x3ff, 8), dir1.d.tail})); - lfs_pair_fromle32(dir2.pair); - if (err) { - goto cleanup; - } - } - - // Copy over first block to thread into fs. Unfortunately - // if this fails there is not much we can do. - LFS_DEBUG("Migrating {0x%"PRIx32", 0x%"PRIx32"} " - "-> {0x%"PRIx32", 0x%"PRIx32"}", - lfs->root[0], lfs->root[1], dir1.head[0], dir1.head[1]); - - err = lfs_bd_erase(lfs, dir1.head[1]); - if (err) { - goto cleanup; - } - - err = lfs_dir_fetch(lfs, &dir2, lfs->root); - if (err) { - goto cleanup; - } - - for (lfs_off_t i = 0; i < dir2.off; i++) { - uint8_t dat; - err = lfs_bd_read(lfs, - NULL, &lfs->rcache, dir2.off, - dir2.pair[0], i, &dat, 1); - if (err) { - goto cleanup; - } - - err = lfs_bd_prog(lfs, - &lfs->pcache, &lfs->rcache, true, - dir1.head[1], i, &dat, 1); - if (err) { - goto cleanup; - } - } - - err = lfs_bd_flush(lfs, &lfs->pcache, &lfs->rcache, true); - if (err) { - goto cleanup; - } - } - - // Create new superblock. This marks a successful migration! - err = lfs1_dir_fetch(lfs, &dir1, (const lfs_block_t[2]){0, 1}); - if (err) { - goto cleanup; - } - - dir2.pair[0] = dir1.pair[0]; - dir2.pair[1] = dir1.pair[1]; - dir2.rev = dir1.d.rev; - dir2.off = sizeof(dir2.rev); - dir2.etag = 0xffffffff; - dir2.count = 0; - dir2.tail[0] = lfs->lfs1->root[0]; - dir2.tail[1] = lfs->lfs1->root[1]; - dir2.erased = false; - dir2.split = true; - - lfs_superblock_t superblock = { - .version = LFS_DISK_VERSION, - .block_size = lfs->cfg->block_size, - .block_count = lfs->cfg->block_count, - .name_max = lfs->name_max, - .file_max = lfs->file_max, - .attr_max = lfs->attr_max, - }; - - lfs_superblock_tole32(&superblock); - err = lfs_dir_commit(lfs, &dir2, LFS_MKATTRS( - {LFS_MKTAG(LFS_TYPE_CREATE, 0, 0), NULL}, - {LFS_MKTAG(LFS_TYPE_SUPERBLOCK, 0, 8), "littlefs"}, - {LFS_MKTAG(LFS_TYPE_INLINESTRUCT, 0, sizeof(superblock)), - &superblock})); - if (err) { - goto cleanup; - } - - // sanity check that fetch works - err = lfs_dir_fetch(lfs, &dir2, (const lfs_block_t[2]){0, 1}); - if (err) { - goto cleanup; - } - - // force compaction to prevent accidentally mounting v1 - dir2.erased = false; - err = lfs_dir_commit(lfs, &dir2, NULL, 0); - if (err) { - goto cleanup; - } - } - -cleanup: - lfs1_unmount(lfs); - return err; -} - -#endif - - -/// Public API wrappers /// - -// Here we can add tracing/thread safety easily - -// Thread-safe wrappers if enabled -#ifdef LFS_THREADSAFE -#define LFS_LOCK(cfg) cfg->lock(cfg) -#define LFS_UNLOCK(cfg) cfg->unlock(cfg) -#else -#define LFS_LOCK(cfg) ((void)cfg, 0) -#define LFS_UNLOCK(cfg) ((void)cfg) -#endif - -// Public API -#ifndef LFS_READONLY -int lfs_format(lfs_t *lfs, const struct lfs_config *cfg) { - int err = LFS_LOCK(cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_format(%p, %p {.context=%p, " - ".read=%p, .prog=%p, .erase=%p, .sync=%p, " - ".read_size=%"PRIu32", .prog_size=%"PRIu32", " - ".block_size=%"PRIu32", .block_count=%"PRIu32", " - ".block_cycles=%"PRId32", .cache_size=%"PRIu32", " - ".lookahead_size=%"PRIu32", .read_buffer=%p, " - ".prog_buffer=%p, .lookahead_buffer=%p, " - ".name_max=%"PRIu32", .file_max=%"PRIu32", " - ".attr_max=%"PRIu32"})", - (void*)lfs, (void*)cfg, cfg->context, - (void*)(uintptr_t)cfg->read, (void*)(uintptr_t)cfg->prog, - (void*)(uintptr_t)cfg->erase, (void*)(uintptr_t)cfg->sync, - cfg->read_size, cfg->prog_size, cfg->block_size, cfg->block_count, - cfg->block_cycles, cfg->cache_size, cfg->lookahead_size, - cfg->read_buffer, cfg->prog_buffer, cfg->lookahead_buffer, - cfg->name_max, cfg->file_max, cfg->attr_max); - - err = lfs_format_(lfs, cfg); - - LFS_TRACE("lfs_format -> %d", err); - LFS_UNLOCK(cfg); - return err; -} -#endif - -int lfs_mount(lfs_t *lfs, const struct lfs_config *cfg) { - int err = LFS_LOCK(cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_mount(%p, %p {.context=%p, " - ".read=%p, .prog=%p, .erase=%p, .sync=%p, " - ".read_size=%"PRIu32", .prog_size=%"PRIu32", " - ".block_size=%"PRIu32", .block_count=%"PRIu32", " - ".block_cycles=%"PRId32", .cache_size=%"PRIu32", " - ".lookahead_size=%"PRIu32", .read_buffer=%p, " - ".prog_buffer=%p, .lookahead_buffer=%p, " - ".name_max=%"PRIu32", .file_max=%"PRIu32", " - ".attr_max=%"PRIu32"})", - (void*)lfs, (void*)cfg, cfg->context, - (void*)(uintptr_t)cfg->read, (void*)(uintptr_t)cfg->prog, - (void*)(uintptr_t)cfg->erase, (void*)(uintptr_t)cfg->sync, - cfg->read_size, cfg->prog_size, cfg->block_size, cfg->block_count, - cfg->block_cycles, cfg->cache_size, cfg->lookahead_size, - cfg->read_buffer, cfg->prog_buffer, cfg->lookahead_buffer, - cfg->name_max, cfg->file_max, cfg->attr_max); - - err = lfs_mount_(lfs, cfg); - - LFS_TRACE("lfs_mount -> %d", err); - LFS_UNLOCK(cfg); - return err; -} - -int lfs_unmount(lfs_t *lfs) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_unmount(%p)", (void*)lfs); - - err = lfs_unmount_(lfs); - - LFS_TRACE("lfs_unmount -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} - -#ifndef LFS_READONLY -int lfs_remove(lfs_t *lfs, const char *path) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_remove(%p, \"%s\")", (void*)lfs, path); - - err = lfs_remove_(lfs, path); - - LFS_TRACE("lfs_remove -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} -#endif - -#ifndef LFS_READONLY -int lfs_rename(lfs_t *lfs, const char *oldpath, const char *newpath) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_rename(%p, \"%s\", \"%s\")", (void*)lfs, oldpath, newpath); - - err = lfs_rename_(lfs, oldpath, newpath); - - LFS_TRACE("lfs_rename -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} -#endif - -int lfs_stat(lfs_t *lfs, const char *path, struct lfs_info *info) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_stat(%p, \"%s\", %p)", (void*)lfs, path, (void*)info); - - err = lfs_stat_(lfs, path, info); - - LFS_TRACE("lfs_stat -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} - -lfs_ssize_t lfs_getattr(lfs_t *lfs, const char *path, - uint8_t type, void *buffer, lfs_size_t size) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_getattr(%p, \"%s\", %"PRIu8", %p, %"PRIu32")", - (void*)lfs, path, type, buffer, size); - - lfs_ssize_t res = lfs_getattr_(lfs, path, type, buffer, size); - - LFS_TRACE("lfs_getattr -> %"PRId32, res); - LFS_UNLOCK(lfs->cfg); - return res; -} - -#ifndef LFS_READONLY -int lfs_setattr(lfs_t *lfs, const char *path, - uint8_t type, const void *buffer, lfs_size_t size) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_setattr(%p, \"%s\", %"PRIu8", %p, %"PRIu32")", - (void*)lfs, path, type, buffer, size); - - err = lfs_setattr_(lfs, path, type, buffer, size); - - LFS_TRACE("lfs_setattr -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} -#endif - -#ifndef LFS_READONLY -int lfs_removeattr(lfs_t *lfs, const char *path, uint8_t type) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_removeattr(%p, \"%s\", %"PRIu8")", (void*)lfs, path, type); - - err = lfs_removeattr_(lfs, path, type); - - LFS_TRACE("lfs_removeattr -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} -#endif - -#ifndef LFS_NO_MALLOC -int lfs_file_open(lfs_t *lfs, lfs_file_t *file, const char *path, int flags) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_file_open(%p, %p, \"%s\", %x)", - (void*)lfs, (void*)file, path, (unsigned)flags); - LFS_ASSERT(!lfs_mlist_isopen(lfs->mlist, (struct lfs_mlist*)file)); - - err = lfs_file_open_(lfs, file, path, flags); - - LFS_TRACE("lfs_file_open -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} -#endif - -int lfs_file_opencfg(lfs_t *lfs, lfs_file_t *file, - const char *path, int flags, - const struct lfs_file_config *cfg) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_file_opencfg(%p, %p, \"%s\", %x, %p {" - ".buffer=%p, .attrs=%p, .attr_count=%"PRIu32"})", - (void*)lfs, (void*)file, path, (unsigned)flags, - (void*)cfg, cfg->buffer, (void*)cfg->attrs, cfg->attr_count); - LFS_ASSERT(!lfs_mlist_isopen(lfs->mlist, (struct lfs_mlist*)file)); - - err = lfs_file_opencfg_(lfs, file, path, flags, cfg); - - LFS_TRACE("lfs_file_opencfg -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} - -int lfs_file_close(lfs_t *lfs, lfs_file_t *file) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_file_close(%p, %p)", (void*)lfs, (void*)file); - LFS_ASSERT(lfs_mlist_isopen(lfs->mlist, (struct lfs_mlist*)file)); - - err = lfs_file_close_(lfs, file); - - LFS_TRACE("lfs_file_close -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} - -#ifndef LFS_READONLY -int lfs_file_sync(lfs_t *lfs, lfs_file_t *file) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_file_sync(%p, %p)", (void*)lfs, (void*)file); - LFS_ASSERT(lfs_mlist_isopen(lfs->mlist, (struct lfs_mlist*)file)); - - err = lfs_file_sync_(lfs, file); - - LFS_TRACE("lfs_file_sync -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} -#endif - -lfs_ssize_t lfs_file_read(lfs_t *lfs, lfs_file_t *file, - void *buffer, lfs_size_t size) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_file_read(%p, %p, %p, %"PRIu32")", - (void*)lfs, (void*)file, buffer, size); - LFS_ASSERT(lfs_mlist_isopen(lfs->mlist, (struct lfs_mlist*)file)); - - lfs_ssize_t res = lfs_file_read_(lfs, file, buffer, size); - - LFS_TRACE("lfs_file_read -> %"PRId32, res); - LFS_UNLOCK(lfs->cfg); - return res; -} - -#ifndef LFS_READONLY -lfs_ssize_t lfs_file_write(lfs_t *lfs, lfs_file_t *file, - const void *buffer, lfs_size_t size) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_file_write(%p, %p, %p, %"PRIu32")", - (void*)lfs, (void*)file, buffer, size); - LFS_ASSERT(lfs_mlist_isopen(lfs->mlist, (struct lfs_mlist*)file)); - - lfs_ssize_t res = lfs_file_write_(lfs, file, buffer, size); - - LFS_TRACE("lfs_file_write -> %"PRId32, res); - LFS_UNLOCK(lfs->cfg); - return res; -} -#endif - -lfs_soff_t lfs_file_seek(lfs_t *lfs, lfs_file_t *file, - lfs_soff_t off, int whence) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_file_seek(%p, %p, %"PRId32", %d)", - (void*)lfs, (void*)file, off, whence); - LFS_ASSERT(lfs_mlist_isopen(lfs->mlist, (struct lfs_mlist*)file)); - - lfs_soff_t res = lfs_file_seek_(lfs, file, off, whence); - - LFS_TRACE("lfs_file_seek -> %"PRId32, res); - LFS_UNLOCK(lfs->cfg); - return res; -} - -#ifndef LFS_READONLY -int lfs_file_truncate(lfs_t *lfs, lfs_file_t *file, lfs_off_t size) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_file_truncate(%p, %p, %"PRIu32")", - (void*)lfs, (void*)file, size); - LFS_ASSERT(lfs_mlist_isopen(lfs->mlist, (struct lfs_mlist*)file)); - - err = lfs_file_truncate_(lfs, file, size); - - LFS_TRACE("lfs_file_truncate -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} -#endif - -lfs_soff_t lfs_file_tell(lfs_t *lfs, lfs_file_t *file) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_file_tell(%p, %p)", (void*)lfs, (void*)file); - LFS_ASSERT(lfs_mlist_isopen(lfs->mlist, (struct lfs_mlist*)file)); - - lfs_soff_t res = lfs_file_tell_(lfs, file); - - LFS_TRACE("lfs_file_tell -> %"PRId32, res); - LFS_UNLOCK(lfs->cfg); - return res; -} - -int lfs_file_rewind(lfs_t *lfs, lfs_file_t *file) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_file_rewind(%p, %p)", (void*)lfs, (void*)file); - - err = lfs_file_rewind_(lfs, file); - - LFS_TRACE("lfs_file_rewind -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} - -lfs_soff_t lfs_file_size(lfs_t *lfs, lfs_file_t *file) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_file_size(%p, %p)", (void*)lfs, (void*)file); - LFS_ASSERT(lfs_mlist_isopen(lfs->mlist, (struct lfs_mlist*)file)); - - lfs_soff_t res = lfs_file_size_(lfs, file); - - LFS_TRACE("lfs_file_size -> %"PRIu32, res); - LFS_UNLOCK(lfs->cfg); - return res; -} - -#ifndef LFS_READONLY -int lfs_mkdir(lfs_t *lfs, const char *path) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_mkdir(%p, \"%s\")", (void*)lfs, path); - - err = lfs_mkdir_(lfs, path); - - LFS_TRACE("lfs_mkdir -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} -#endif - -int lfs_dir_open(lfs_t *lfs, lfs_dir_t *dir, const char *path) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_dir_open(%p, %p, \"%s\")", (void*)lfs, (void*)dir, path); - LFS_ASSERT(!lfs_mlist_isopen(lfs->mlist, (struct lfs_mlist*)dir)); - - err = lfs_dir_open_(lfs, dir, path); - - LFS_TRACE("lfs_dir_open -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} - -int lfs_dir_close(lfs_t *lfs, lfs_dir_t *dir) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_dir_close(%p, %p)", (void*)lfs, (void*)dir); - - err = lfs_dir_close_(lfs, dir); - - LFS_TRACE("lfs_dir_close -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} - -int lfs_dir_read(lfs_t *lfs, lfs_dir_t *dir, struct lfs_info *info) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_dir_read(%p, %p, %p)", - (void*)lfs, (void*)dir, (void*)info); - - err = lfs_dir_read_(lfs, dir, info); - - LFS_TRACE("lfs_dir_read -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} - -int lfs_dir_seek(lfs_t *lfs, lfs_dir_t *dir, lfs_off_t off) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_dir_seek(%p, %p, %"PRIu32")", - (void*)lfs, (void*)dir, off); - - err = lfs_dir_seek_(lfs, dir, off); - - LFS_TRACE("lfs_dir_seek -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} - -lfs_soff_t lfs_dir_tell(lfs_t *lfs, lfs_dir_t *dir) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_dir_tell(%p, %p)", (void*)lfs, (void*)dir); - - lfs_soff_t res = lfs_dir_tell_(lfs, dir); - - LFS_TRACE("lfs_dir_tell -> %"PRId32, res); - LFS_UNLOCK(lfs->cfg); - return res; -} - -int lfs_dir_rewind(lfs_t *lfs, lfs_dir_t *dir) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_dir_rewind(%p, %p)", (void*)lfs, (void*)dir); - - err = lfs_dir_rewind_(lfs, dir); - - LFS_TRACE("lfs_dir_rewind -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} - -int lfs_fs_stat(lfs_t *lfs, struct lfs_fsinfo *fsinfo) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_fs_stat(%p, %p)", (void*)lfs, (void*)fsinfo); - - err = lfs_fs_stat_(lfs, fsinfo); - - LFS_TRACE("lfs_fs_stat -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} - -lfs_ssize_t lfs_fs_size(lfs_t *lfs) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_fs_size(%p)", (void*)lfs); - - lfs_ssize_t res = lfs_fs_size_(lfs); - - LFS_TRACE("lfs_fs_size -> %"PRId32, res); - LFS_UNLOCK(lfs->cfg); - return res; -} - -int lfs_fs_traverse(lfs_t *lfs, int (*cb)(void *, lfs_block_t), void *data) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_fs_traverse(%p, %p, %p)", - (void*)lfs, (void*)(uintptr_t)cb, data); - - err = lfs_fs_traverse_(lfs, cb, data, true); - - LFS_TRACE("lfs_fs_traverse -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} - -#ifndef LFS_READONLY -int lfs_fs_mkconsistent(lfs_t *lfs) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_fs_mkconsistent(%p)", (void*)lfs); - - err = lfs_fs_mkconsistent_(lfs); - - LFS_TRACE("lfs_fs_mkconsistent -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} -#endif - -#ifndef LFS_READONLY -int lfs_fs_gc(lfs_t *lfs) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_fs_gc(%p)", (void*)lfs); - - err = lfs_fs_gc_(lfs); - - LFS_TRACE("lfs_fs_gc -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} -#endif - -#ifndef LFS_READONLY -int lfs_fs_grow(lfs_t *lfs, lfs_size_t block_count) { - int err = LFS_LOCK(lfs->cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_fs_grow(%p, %"PRIu32")", (void*)lfs, block_count); - - err = lfs_fs_grow_(lfs, block_count); - - LFS_TRACE("lfs_fs_grow -> %d", err); - LFS_UNLOCK(lfs->cfg); - return err; -} -#endif - -#ifdef LFS_MIGRATE -int lfs_migrate(lfs_t *lfs, const struct lfs_config *cfg) { - int err = LFS_LOCK(cfg); - if (err) { - return err; - } - LFS_TRACE("lfs_migrate(%p, %p {.context=%p, " - ".read=%p, .prog=%p, .erase=%p, .sync=%p, " - ".read_size=%"PRIu32", .prog_size=%"PRIu32", " - ".block_size=%"PRIu32", .block_count=%"PRIu32", " - ".block_cycles=%"PRId32", .cache_size=%"PRIu32", " - ".lookahead_size=%"PRIu32", .read_buffer=%p, " - ".prog_buffer=%p, .lookahead_buffer=%p, " - ".name_max=%"PRIu32", .file_max=%"PRIu32", " - ".attr_max=%"PRIu32"})", - (void*)lfs, (void*)cfg, cfg->context, - (void*)(uintptr_t)cfg->read, (void*)(uintptr_t)cfg->prog, - (void*)(uintptr_t)cfg->erase, (void*)(uintptr_t)cfg->sync, - cfg->read_size, cfg->prog_size, cfg->block_size, cfg->block_count, - cfg->block_cycles, cfg->cache_size, cfg->lookahead_size, - cfg->read_buffer, cfg->prog_buffer, cfg->lookahead_buffer, - cfg->name_max, cfg->file_max, cfg->attr_max); - - err = lfs_migrate_(lfs, cfg); - - LFS_TRACE("lfs_migrate -> %d", err); - LFS_UNLOCK(cfg); - return err; -} -#endif - diff --git a/components/littlefs/src/lfs_util.c b/components/littlefs/src/lfs_util.c deleted file mode 100644 index dac72abc3..000000000 --- a/components/littlefs/src/lfs_util.c +++ /dev/null @@ -1,37 +0,0 @@ -/* - * lfs util functions - * - * Copyright (c) 2022, The littlefs authors. - * Copyright (c) 2017, Arm Limited. All rights reserved. - * SPDX-License-Identifier: BSD-3-Clause - */ -#include "lfs_util.h" - -// Only compile if user does not provide custom config -#ifndef LFS_CONFIG - - -// If user provides their own CRC impl we don't need this -#ifndef LFS_CRC -// Software CRC implementation with small lookup table -uint32_t lfs_crc(uint32_t crc, const void *buffer, size_t size) { - static const uint32_t rtable[16] = { - 0x00000000, 0x1db71064, 0x3b6e20c8, 0x26d930ac, - 0x76dc4190, 0x6b6b51f4, 0x4db26158, 0x5005713c, - 0xedb88320, 0xf00f9344, 0xd6d6a3e8, 0xcb61b38c, - 0x9b64c2b0, 0x86d3d2d4, 0xa00ae278, 0xbdbdf21c, - }; - - const uint8_t *data = buffer; - - for (size_t i = 0; i < size; i++) { - crc = (crc >> 4) ^ rtable[(crc ^ (data[i] >> 0)) & 0xf]; - crc = (crc >> 4) ^ rtable[(crc ^ (data[i] >> 4)) & 0xf]; - } - - return crc; -} -#endif - - -#endif diff --git a/components/migration/CMakeLists.txt b/components/migration/CMakeLists.txt deleted file mode 100644 index 6dc98bd46..000000000 --- a/components/migration/CMakeLists.txt +++ /dev/null @@ -1,15 +0,0 @@ -# The 1.x -> 2.x storage migration, pipelined (docs/STORAGE.md): on a hub still on the 1.x partition table, retype -# `config` as the `cfg` NVS store (config/Migration.h), then write the www image over `static0` and retype it as -# `www` (www/Migration.h). It sits at the top of the component graph: it holds the OTA flash-write lock, closes the -# captive portal and asks the OTA manager for the running release's www image. On a hub already on the 2.x table it -# does nothing and starts no task. -idf_component_register( - SRCS "src/StorageMigration.cpp" - INCLUDE_DIRS "include" - REQUIRES common # SemVer.h, SimpleMutex.h - PRIV_REQUIRES config www partition_layout # the two halves, the live table checks - ota captive_portal core # the flash-write lock and release lookup, ForceClose, WiFiManager - http logging # HTTP::Client, OS_LOG* -) - -target_compile_options(${COMPONENT_LIB} PRIVATE -Wno-error) diff --git a/components/migration/include/migration/StorageMigration.h b/components/migration/include/migration/StorageMigration.h deleted file mode 100644 index 252c2e9a4..000000000 --- a/components/migration/include/migration/StorageMigration.h +++ /dev/null @@ -1,34 +0,0 @@ -#pragma once - -// Pipelines the two halves of the 1.x -> 2.x storage migration (docs/STORAGE.md) on a hub that still boots the 1.x -// partition table. Each boot starts by looking at the live table, so a restart at any point resumes at the first -// unfinished step: -// -// 1. `config` is still littlefs: Config::MigrateToCfgPartition retypes it as `cfg` and restarts; the next boot's -// Config::Init erases the littlefs and moves the config in from the shared nvs. -// 2. `static0` is still littlefs: once online, Www::MigrateToWwwPartition writes the www image over it, retypes it -// as `www` and restarts. The image is the running release's `staticfs` artifact (a 2.x release publishes the -// www image there); a build with no such release (a dev build, a 1.x-numbered one) waits for `migrate www -// ` over the serial console. -// -// Each step takes the OTA flash-write lock for its whole run, so an install never writes static0 or runs while the -// table changes, and backs off while an install is running. - -#include -#include - -namespace OpenShock::Migration { - /// @brief Starts the migration task when the live table still has a 1.x partition to migrate; a no-op otherwise. - /// Call after setup, so a migration only ever starts from a firmware that came up (and validated itself). - bool Start(); - - /// @brief Whether the migration task is running: the live table still has something to migrate. - [[nodiscard]] bool IsPending(); - - /// @brief Hands the www step an image to use instead of the running release's: the task downloads it on its next - /// round. False when no migration is pending. - bool RequestWww(std::string url, const uint8_t (&sha256)[32]); - - /// @brief Asks the task to run its next round now instead of after its retry delay. False when none is pending. - bool RequestNow(); -} // namespace OpenShock::Migration diff --git a/components/migration/src/StorageMigration.cpp b/components/migration/src/StorageMigration.cpp deleted file mode 100644 index 267349f3a..000000000 --- a/components/migration/src/StorageMigration.cpp +++ /dev/null @@ -1,235 +0,0 @@ -#include - -#include "migration/StorageMigration.h" - -const char* const TAG = "StorageMigration"; - -#include "captiveportal/Manager.h" -#include "config/Migration.h" -#include "http/HTTPRequestManager.h" -#include "Logging.h" -#include "OpenShock.h" -#include "OtaUpdateManager.h" -#include "partition_layout/LiveTable.h" -#include "partition_layout/PartitionTable.h" -#include "SimpleMutex.h" -#include "util/TaskUtils.h" -#include "wifi/WiFiManager.h" -#include "www/Migration.h" - -#include - -#include -#include -#include - -using namespace OpenShock; -namespace PT = OpenShock::PartitionTable; - -namespace { - constexpr TickType_t kWifiPoll = pdMS_TO_TICKS(5'000); - constexpr TickType_t kRetryDelay = pdMS_TO_TICKS(10 * 60'000); // a network, server or busy-hub problem - - struct WwwSource { - std::string url; - uint8_t sha256[32]; - }; - - TaskHandle_t s_task = nullptr; - SimpleMutex s_requestMutex {}; - std::optional s_requestedWww; // Guarded by s_requestMutex; kept until a migration succeeds or a new request replaces it - bool s_saidNoRelease = false; - - struct Pending { - bool cfg; // `config` is still the 1.x littlefs and there is no `cfg` - bool www; // `static0` is still the 1.x littlefs and there is no `www` - }; - - // What the live table still has to migrate; nullopt when it cannot be read. - std::optional readPending() - { - struct Tables { - PT::Bytes live; - PT::Table parsed; - }; - auto tables = std::make_unique(); - if (!PT::ReadLiveBytes(tables->live) || !PT::Parse(tables->live, tables->parsed)) { - OS_LOGE(TAG, "Cannot read the partition table"); - return std::nullopt; - } - auto isLittlefs = [](const PT::Entry* entry) { - return entry != nullptr && entry->type == PT::kTypeData && entry->subtype == PT::kSubtypeDataSpiffs; - }; - return Pending { - .cfg = tables->parsed.Find("cfg") == nullptr && isLittlefs(tables->parsed.Find("config")), - .www = tables->parsed.Find("www") == nullptr && isLittlefs(tables->parsed.Find("static0")), - }; - } - - // A copy of the serial-supplied source, so a failed attempt (a busy hub, a download that did not go through) - // retries from it instead of falling back to the release lookup - std::optional requestedWww() - { - ScopedLock lock(&s_requestMutex); - return s_requestedWww; - } - - // The running release's www image, for a build that has one published. - std::optional releaseWww(HTTP::Client& client) - { - const SemVer& version = Constants::FirmwareVersion(); - if (version.major < 2) { - if (!s_saidNoRelease) { - OS_LOGW(TAG, "No www image is published for firmware %s; waiting for `migrate www ` over serial", Constants::FW_VERSION); - s_saidNoRelease = true; - } - return std::nullopt; - } - auto release = OtaUpdateManager::TryGetFirmwareRelease(client, version); - if (!release) { - OS_LOGW(TAG, "The release for firmware %s is not available; retrying later", Constants::FW_VERSION); - return std::nullopt; - } - WwwSource source; - source.url = std::move(release->filesystemBinaryUrl); - memcpy(source.sha256, release->filesystemBinaryHash, sizeof(source.sha256)); - return source; - } - - // Both steps hold the OTA flash-write lock from before their first flash write to the restart, and give it back on - // every return (every return is a failure: success ends in esp_restart). - struct FlashWriteLock { - bool held = OtaUpdateManager::TryLockFlashWrites(); - ~FlashWriteLock() - { - if (held) OtaUpdateManager::UnlockFlashWrites(); - } - }; - - // Returns only when nothing was migrated (the step restarts the hub when it was) - void migrateCfg() - { - FlashWriteLock lock; - if (!lock.held) { - OS_LOGI(TAG, "An update is being installed; migrating the config partition later"); - return; - } - switch (Config::MigrateToCfgPartition()) { - case Config::MigrationResult::NotNeeded: - break; - case Config::MigrationResult::Refused: - OS_LOGE(TAG, "The config partition migration was refused; the config stays in the shared nvs"); - break; - } - } - - void migrateWww() - { - HTTP::Client client; - auto source = requestedWww(); - if (!source) { - source = releaseWww(client); - } - if (!source) { - return; - } - - FlashWriteLock lock; - if (!lock.held) { - OS_LOGI(TAG, "An update is being installed; migrating the www partition later"); - return; - } - // The portal stays closed until the restart. A close that timed out still leaves the portal force-closed, so every - // return from here releases it. - struct ReleasePortalOnReturn { - ~ReleasePortalOnReturn() { CaptivePortal::ReleaseForceClose(); } - } releasePortal; - if (!CaptivePortal::ForceClose(5000)) { - OS_LOGE(TAG, "Failed to close the captive portal; migrating the www partition later"); - return; - } - OS_LOGI(TAG, "Migrating the www partition from %s", source->url.c_str()); - switch (Www::MigrateToWwwPartition(client, source->url, source->sha256)) { - case Www::MigrationResult::NotNeeded: - break; - case Www::MigrationResult::Refused: - OS_LOGE(TAG, "The www partition migration was refused; the portal keeps serving from static0"); - break; - case Www::MigrationResult::Failed: - OS_LOGE(TAG, "The www image did not migrate; retrying later"); - break; - } - } - - void migrationTask(void*) - { - while (true) { - auto pending = readPending(); - if (!pending) { - // Unreadable: nothing can be decided; look again later - } else if (pending->cfg) { - migrateCfg(); - } else if (pending->www) { - if (!WiFiManager::IsConnected()) { - TaskUtils::WaitForNotify(kWifiPoll); // Consumes a request made meanwhile, so it does not cut the retry delay short later - continue; - } - migrateWww(); - } else { - OS_LOGI(TAG, "The storage migration is complete"); - break; - } - // Each step restarts the hub on success; being here means it did not happen this round - TaskUtils::WaitForNotify(kRetryDelay); - } - s_task = nullptr; - vTaskDelete(nullptr); - } -} // namespace - -bool Migration::Start() -{ - auto pending = readPending(); - if (!pending) { - return false; - } - if (!pending->cfg && !pending->www) { - OS_LOGD(TAG, "The partition table is the 2.x layout; nothing to migrate"); - return true; - } - OS_LOGI(TAG, "This hub still has the 1.x %s%s%s partition; the storage migration is pending", pending->cfg ? "config" : "", pending->cfg && pending->www ? " and " : "", pending->www ? "static0" : ""); - if (TaskUtils::TaskCreateExpensive(migrationTask, "StorageMigration", 8'192, nullptr, 1, &s_task) != pdPASS) { - OS_LOGE(TAG, "Failed to create the migration task"); - return false; - } - return true; -} - -bool Migration::IsPending() -{ - return s_task != nullptr; -} - -bool Migration::RequestWww(std::string url, const uint8_t (&sha256)[32]) -{ - if (s_task == nullptr) { - return false; - } - { - ScopedLock lock(&s_requestMutex); - s_requestedWww.emplace(); - s_requestedWww->url = std::move(url); - memcpy(s_requestedWww->sha256, sha256, sizeof(s_requestedWww->sha256)); - } - return RequestNow(); -} - -bool Migration::RequestNow() -{ - TaskHandle_t task = s_task; - if (task == nullptr) { - return false; - } - TaskUtils::Notify(task); - return true; -} diff --git a/components/ota/CMakeLists.txt b/components/ota/CMakeLists.txt index c6d72598c..313d12ea5 100644 --- a/components/ota/CMakeLists.txt +++ b/components/ota/CMakeLists.txt @@ -7,13 +7,14 @@ # registered in OtaUpdateManager::Init(). idf_component_register( SRCS "src/OtaUpdateManager.cpp" + "src/PartitionUtils.cpp" INCLUDE_DIRS "include" REQUIRES common # enums/FirmwareBootType.h, enums/OtaUpdateChannel.h, SemVer.h in public header PRIV_REQUIRES core captive_portal config # gateway/wifi/WSGateway/AppHooks; CaptivePortal::ForceClose; Config:: - fs # TestFilesystem: a littlefs static0 on the 1.x layout - crypto http hwutil logging osjson temporal # Hashing, HTTPRequestManager, PartitionUtils, json - www # the www partition the staticfs artifact goes into (2.x layout) + crypto http logging osjson temporal # Hashing, HTTPRequestManager, json + www # the www partition the staticfs artifact goes into app_update esp_event esp_netif esp_wifi # esp_ota_ops.h, WiFi/IP event handlers + esp_partition # flashing partitions from a URL ) # Match core: IDF compiles application components with -Werror, the legacy build did not. diff --git a/components/ota/include/OtaUpdateManager.h b/components/ota/include/OtaUpdateManager.h index 100030ad3..a896ae39c 100644 --- a/components/ota/include/OtaUpdateManager.h +++ b/components/ota/include/OtaUpdateManager.h @@ -20,8 +20,8 @@ namespace OpenShock::OtaUpdateManager { struct FirmwareRelease { std::string appBinaryUrl; uint8_t appBinaryHash[32]; - std::string filesystemBinaryUrl; // the `staticfs` artifact: the www image (www.bin) - uint8_t filesystemBinaryHash[32]; + std::string wwwBinaryUrl; // the `staticfs` artifact: the www image (www.bin) + uint8_t wwwBinaryHash[32]; }; /// @brief Asks the repository server for the head of a channel for this board. @@ -34,12 +34,6 @@ namespace OpenShock::OtaUpdateManager { bool TryStartFirmwareUpdate(const OpenShock::SemVer& version); - /// @brief Keeps installs out while the caller writes flash an install also writes (the storage migration's www image - /// and partition table). False while an install is running. Held by the calling task until - /// UnlockFlashWrites() or a restart; an install that comes up meanwhile is dropped, not queued. - [[nodiscard]] bool TryLockFlashWrites(); - void UnlockFlashWrites(); - /// @brief Asks the update task to check the configured channel for a new version as soon as it can. bool RequestUpdateCheck(); diff --git a/components/ota/src/OtaUpdateManager.cpp b/components/ota/src/OtaUpdateManager.cpp index 6ce2f9d11..ae2396144 100644 --- a/components/ota/src/OtaUpdateManager.cpp +++ b/components/ota/src/OtaUpdateManager.cpp @@ -7,14 +7,13 @@ const char* const TAG = "OtaUpdateManager"; #include "AppHooks.h" #include "BoardDescriptor.h" #include "captiveportal/Manager.h" -#include "fs/FsCheck.h" #include "GatewayConnectionManager.h" #include "Hashing.h" #include "http/HTTPRequestManager.h" -#include "hwutil/PartitionUtils.h" #include "Logging.h" #include "OpenShock.h" #include "OtaUpdateConfig.h" +#include "PartitionUtils.h" #include "SemVer.h" #include "serialization/WSGateway.h" #include "SimpleMutex.h" @@ -75,9 +74,6 @@ static OpenShock::FirmwareBootType _bootType; static TaskHandle_t _taskHandle = nullptr; static OpenShock::SemVer _requestedVersion; static OpenShock::SimpleMutex _requestedVersionMutex = {}; -// Held by whoever writes the flash an install writes: the update task for the whole install (until the restart on -// success), the storage migration from before its first flash write to its restart (TryLockFlashWrites). -static OpenShock::SimpleMutex _flashWriteMutex = {}; using namespace OpenShock; @@ -176,12 +172,10 @@ static bool _sendFailureMessage(std::string_view message, bool fatal = false) /// @brief Whether the descriptor in `slot` names the board this hub is, logging the verdict. /// -/// A slot with no descriptor at all passes: the repository server keyed pre-descriptor builds by board, and -/// refusing them would block every downgrade to one. A descriptor that is present but corrupt, or a placeholder -/// from an image nobody stamped, is not that case: such an image would run with no pins, so it is refused when -/// strict. Running unstamped ourselves passes, since there is nothing to compare to. -/// @param strict True for an image about to be made bootable: an unreadable, corrupt or unstamped descriptor is a -/// refusal. False for the first-boot check against the slot we came from, where a problem reading +/// An image with no descriptor at all, a corrupt one, or a placeholder nobody stamped would run with no pins, so it is +/// refused when strict. Running unstamped ourselves passes, since there is nothing to compare to. +/// @param strict True for an image about to be made bootable: an unreadable, missing, corrupt or unstamped descriptor +/// is a refusal. False for the first-boot check against the slot we came from, where a problem reading /// the old slot must not roll back a good image. static bool otaum_slot_matches_this_board(const esp_partition_t* slot, const char* what, bool strict) { @@ -195,8 +189,8 @@ static bool otaum_slot_matches_this_board(const esp_partition_t* slot, const cha BoardDescriptor other; switch (Board::Parse(bytes, other)) { case Board::ParseResult::Absent: - OS_LOGW(TAG, "%s carries no board descriptor (built before descriptors existed), cannot check it against this board", what); - return true; + OS_LOGE(TAG, "%s carries no board descriptor", what); + return !strict; case Board::ParseResult::Invalid: OS_LOGE(TAG, "%s carries a corrupt board descriptor", what); return !strict; @@ -265,43 +259,37 @@ static bool otaum_flash_app_partition(OpenShock::HTTP::Client& client, const esp return true; } -static bool otaum_flash_fs_partition(OpenShock::HTTP::Client& client, const esp_partition_t* parition, std::string_view remoteUrl, const uint8_t (&remoteHash)[32]) +static bool otaum_flash_www_partition(OpenShock::HTTP::Client& client, const esp_partition_t* partition, std::string_view remoteUrl, const uint8_t (&remoteHash)[32]) { - OS_LOGD(TAG, "Flashing filesystem partition"); + OS_LOGD(TAG, "Flashing www partition"); if (!otaum_send_progress_msg(Serialization::Types::OtaUpdateProgressTask::FlashingFilesystem, 0.0f)) { return false; } auto onProgress = [](std::size_t current, std::size_t total, float progress) -> bool { - OS_LOGD(TAG, "Flashing filesystem partition: %u / %u (%.2f%%)", current, total, progress * 100.0f); + OS_LOGD(TAG, "Flashing www partition: %u / %u (%.2f%%)", current, total, progress * 100.0f); otaum_send_progress_msg(Serialization::Types::OtaUpdateProgressTask::FlashingFilesystem, progress); return true; }; - // Refuse an image in the other format before the first erase: a www image written over littlefs static0 (a 2.x - // release on the 1.x table, before the www migration has run) or a littlefs image over www would fail the check - // below only after the portal assets are gone. - const bool isWwwPartition = parition->type == Www::kPartitionType; - bool formatMismatch = false; - auto matchesPartition = [isWwwPartition, &formatMismatch](std::span chunk) -> bool { - if (Www::StartsLikeImage(chunk) != isWwwPartition) { - if (isWwwPartition) { - OS_LOGE(TAG, "The filesystem image is not a www image"); - } else { - OS_LOGE(TAG, "The filesystem image is a www image, but this partition table has no www partition"); - } + // Refuse anything that is not a www image before the first erase: it would fail the check below only after the + // portal assets are gone. + bool formatMismatch = false; + auto matchesPartition = [&formatMismatch](std::span chunk) -> bool { + if (!Www::StartsLikeImage(chunk)) { + OS_LOGE(TAG, "The downloaded image is not a www image"); formatMismatch = true; return false; } return true; }; - if (!OpenShock::FlashPartitionFromUrl(client, parition, remoteUrl, remoteHash, onProgress, matchesPartition)) { - OS_LOGE(TAG, "Failed to flash filesystem partition"); - _sendFailureMessage(formatMismatch ? "The filesystem image does not match this hub's partition table"sv : "Failed to flash filesystem partition"sv); + if (!OpenShock::FlashPartitionFromUrl(client, partition, remoteUrl, remoteHash, onProgress, matchesPartition)) { + OS_LOGE(TAG, "Failed to flash www partition"); + _sendFailureMessage(formatMismatch ? "The downloaded image is not a www image"sv : "Failed to flash www partition"sv); return false; } @@ -309,12 +297,10 @@ static bool otaum_flash_fs_partition(OpenShock::HTTP::Client& client, const esp_ return false; } - // Verify the freshly flashed image: the www format where the table has a www partition, else (1.x layout) that the - // littlefs image mounts. - const bool valid = isWwwPartition ? Www::VerifyPartition(parition) : OpenShock::TestFilesystem(parition); - if (!valid) { - OS_LOGE(TAG, "The flashed filesystem image is not valid"); - _sendFailureMessage("The flashed filesystem image is not valid"sv); + // Verify the freshly flashed image + if (!Www::VerifyPartition(partition)) { + OS_LOGE(TAG, "The flashed www image is not valid"); + _sendFailureMessage("The flashed www image is not valid"sv); return false; } @@ -403,7 +389,7 @@ static void otaum_updatetask(void* arg) } // Reuse a single connection for this update's sequential CDN requests - // (version -> release -> filesystem binary -> app binary). + // (version -> release -> www image -> app binary). OpenShock::HTTP::Client client; OpenShock::SemVer version; @@ -435,13 +421,6 @@ static void otaum_updatetask(void* arg) continue; } - // Never alongside the storage migration's www and partition table writes; it restarts the hub when it is done - ScopedTryLock flashWrites(&_flashWriteMutex, 0); - if (!flashWrites.isLocked()) { - OS_LOGW(TAG, "The storage migration is writing flash; not updating now"); - continue; - } - OS_LOGD(TAG, "Updating to version: %.*s", static_cast(versionStr.length()), versionStr.data()); // Generate random int32_t for this update. @@ -488,8 +467,8 @@ static void otaum_updatetask(void* arg) OS_LOGD(TAG, " Version: %.*s", static_cast(versionStr.length()), versionStr.data()); OS_LOGD(TAG, " App binary URL: %.*s", static_cast(release.appBinaryUrl.length()), release.appBinaryUrl.data()); OS_LOGD(TAG, " App binary hash: %s", HexUtils::ToHex<32>(release.appBinaryHash).data()); - OS_LOGD(TAG, " Filesystem binary URL: %.*s", static_cast(release.filesystemBinaryUrl.length()), release.filesystemBinaryUrl.data()); - OS_LOGD(TAG, " Filesystem binary hash: %s", HexUtils::ToHex<32>(release.filesystemBinaryHash).data()); + OS_LOGD(TAG, " www image URL: %.*s", static_cast(release.wwwBinaryUrl.length()), release.wwwBinaryUrl.data()); + OS_LOGD(TAG, " www image hash: %s", HexUtils::ToHex<32>(release.wwwBinaryHash).data()); // Get available app update partition. const esp_partition_t* appPartition = esp_ota_get_next_update_partition(nullptr); @@ -499,16 +478,11 @@ static void otaum_updatetask(void* arg) continue; } - // The portal assets go into the www partition; on the 1.x layout, which has none, into littlefs static0. The - // staticfs artifact of a 1.x-numbered release is the littlefs image (scripts/repo_server_publish.py), which is - // what every hub still on the 1.x layout installs, by this code or by the 1.x firmware's own. - const esp_partition_t* filesystemPartition = Www::FindPartition(); - if (filesystemPartition == nullptr) { - filesystemPartition = FindStaticPartition(); - } - if (filesystemPartition == nullptr) { - OS_LOGE(TAG, "Failed to find a www or static filesystem partition"); - _sendFailureMessage("Failed to find filesystem partition"sv); + // The portal assets go into the www partition + const esp_partition_t* wwwPartition = Www::FindPartition(); + if (wwwPartition == nullptr) { + OS_LOGE(TAG, "Failed to find the www partition"); + _sendFailureMessage("Failed to find www partition"sv); continue; } @@ -534,13 +508,12 @@ static void otaum_updatetask(void* arg) continue; } - // Flash app and filesystem partitions. An unchanged www image (the same hash) is not written again; a littlefs - // static0 always is, as the 1.x firmware does. + // Flash the app and www partitions. An unchanged www image (the same hash) is not written again. std::array currentWwwHash {}; - const bool wwwCurrent = filesystemPartition->type == Www::kPartitionType && Www::ImageSha256(filesystemPartition, currentWwwHash) && memcmp(currentWwwHash.data(), release.filesystemBinaryHash, 32) == 0; + const bool wwwCurrent = Www::ImageSha256(wwwPartition, currentWwwHash) && memcmp(currentWwwHash.data(), release.wwwBinaryHash, 32) == 0; if (wwwCurrent) { OS_LOGI(TAG, "The www image is already current, not flashing it"); - } else if (!otaum_flash_fs_partition(client, filesystemPartition, release.filesystemBinaryUrl, release.filesystemBinaryHash)) { + } else if (!otaum_flash_www_partition(client, wwwPartition, release.wwwBinaryUrl, release.wwwBinaryHash)) { continue; } if (!otaum_flash_app_partition(client, appPartition, release.appBinaryUrl, release.appBinaryHash)) { @@ -569,20 +542,6 @@ static void otaum_updatetask(void* arg) esp_restart(); } -static const char* otaum_channel_name(OpenShock::OtaUpdateChannel channel) -{ - switch (channel) { - case OtaUpdateChannel::Stable: - return "stable"; - case OtaUpdateChannel::Beta: - return "beta"; - case OtaUpdateChannel::Develop: - return "develop"; - } - - return nullptr; // Channel comes from config and may be out of range -} - /// @brief Fetches the repository server domain from runtime config. /// /// Historically named cdnDomain, from when OTA metadata was flat files served straight off the CDN. @@ -694,11 +653,7 @@ bool OtaUpdateManager::Init() std::optional OtaUpdateManager::TryGetFirmwareVersion(HTTP::Client& client, OtaUpdateChannel channel) { - const char* channelName = otaum_channel_name(channel); - if (channelName == nullptr) { - OS_LOGE(TAG, "Unknown channel: %u", std::to_underlying(channel)); - return std::nullopt; - } + const char* channelName = OtaUpdateChannelName(channel); auto domain = otaum_try_get_repo_domain(); if (!domain) { @@ -762,7 +717,7 @@ static bool _tryParseIntoHash(std::string_view hash, uint8_t (&hashBytes)[32]) /// /// OTA is per-partition, so the hub wants exactly these two. The 'merged' artifact is deliberately /// ignored: it is a full-flash esptool image (bootloader at 0x1000, partition table at 0x8000, app at -/// 0x10000, filesystem at its offset) meant for USB flashing a blank device. Writing it into an OTA app +/// 0x10000, www image at its offset) meant for USB flashing a blank device. Writing it into an OTA app /// slot would put a bootloader image where the app belongs. See firmware-api-spec.md §9.2. [[nodiscard]] static std::optional otaum_parse_board_artifacts(JSON::JsonView root) { @@ -774,7 +729,7 @@ static bool _tryParseIntoHash(std::string_view hash, uint8_t (&hashBytes)[32]) OtaUpdateManager::FirmwareRelease release {}; - bool foundApp = false, foundFilesystem = false; + bool foundApp = false, foundWww = false; int count = artifacts.count(); for (int i = 0; i < count; i++) { @@ -804,15 +759,15 @@ static bool _tryParseIntoHash(std::string_view hash, uint8_t (&hashBytes)[32]) release.appBinaryUrl = std::move(*urlStr); foundApp = true; } else if (*typeStr == "staticfs"sv) { - if (foundFilesystem) { + if (foundWww) { OS_LOGE(TAG, "Duplicate 'staticfs' artifact"); return std::nullopt; } - if (!_tryParseIntoHash(*hashStr, release.filesystemBinaryHash)) { + if (!_tryParseIntoHash(*hashStr, release.wwwBinaryHash)) { return std::nullopt; } - release.filesystemBinaryUrl = std::move(*urlStr); - foundFilesystem = true; + release.wwwBinaryUrl = std::move(*urlStr); + foundWww = true; } } @@ -821,7 +776,7 @@ static bool _tryParseIntoHash(std::string_view hash, uint8_t (&hashBytes)[32]) return std::nullopt; } - if (!foundFilesystem) { + if (!foundWww) { OS_LOGE(TAG, "Release is missing a 'staticfs' artifact"); return std::nullopt; } @@ -863,16 +818,6 @@ bool OtaUpdateManager::TryStartFirmwareUpdate(const OpenShock::SemVer& version) return otaum_try_queue_update_request(version); } -bool OtaUpdateManager::TryLockFlashWrites() -{ - return _flashWriteMutex.lock(0); -} - -void OtaUpdateManager::UnlockFlashWrites() -{ - _flashWriteMutex.unlock(); -} - bool OtaUpdateManager::RequestUpdateCheck() { return otaum_try_notify_task(OTA_TASK_EVENT_CHECK_REQUESTED); diff --git a/components/hwutil/src/PartitionUtils.cpp b/components/ota/src/PartitionUtils.cpp similarity index 86% rename from components/hwutil/src/PartitionUtils.cpp rename to components/ota/src/PartitionUtils.cpp index 6ffad8157..0247db83d 100644 --- a/components/hwutil/src/PartitionUtils.cpp +++ b/components/ota/src/PartitionUtils.cpp @@ -1,4 +1,4 @@ -#include "hwutil/PartitionUtils.h" +#include "PartitionUtils.h" const char* const TAG = "PartitionUtils"; @@ -7,7 +7,6 @@ const char* const TAG = "PartitionUtils"; #include "Logging.h" #include "Temporal.h" -#include "util/HexUtils.h" #include #include @@ -16,26 +15,6 @@ const char* const TAG = "PartitionUtils"; // seconds, starving the idle tasks into the task watchdog; one step of this size takes well under a second. static constexpr std::size_t kEraseStep = 64 * 1024; -const esp_partition_t* OpenShock::FindStaticPartition() -{ - return esp_partition_find_first(ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_SPIFFS, "static0"); -} - -bool OpenShock::TryGetPartitionHash(const esp_partition_t* partition, char (&hash)[65]) -{ - uint8_t buffer[32]; - esp_err_t err = esp_partition_get_sha256(partition, buffer); - if (err != ESP_OK) { - OS_LOGE(TAG, "Failed to get partition hash: %s", esp_err_to_name(err)); - return false; - } - - // Copy the hash to the output buffer - HexUtils::ToHex<32>(buffer, hash, false); - - return true; -} - bool OpenShock::FlashPartitionFromUrl( HTTP::Client& client, const esp_partition_t* partition, diff --git a/components/hwutil/include/hwutil/PartitionUtils.h b/components/ota/src/PartitionUtils.h similarity index 66% rename from components/hwutil/include/hwutil/PartitionUtils.h rename to components/ota/src/PartitionUtils.h index b54109447..9ad26a3c7 100644 --- a/components/hwutil/include/hwutil/PartitionUtils.h +++ b/components/ota/src/PartitionUtils.h @@ -15,17 +15,12 @@ namespace OpenShock { class Client; } - /// @brief The littlefs `static0` partition that holds the captive portal's assets on the 1.x partition layout, or null - /// on the www layout (which has the flat `www` image instead, www/WwwPartition.h). 1.x compatibility. - const esp_partition_t* FindStaticPartition(); - - bool TryGetPartitionHash(const esp_partition_t* partition, char (&hash)[65]); /// @brief Streams `remoteUrl` into `partition`, erasing just ahead of the data, and checks the SHA-256 of what was /// downloaded against `remoteHash`. /// @param progressCallback Invoked with (written, total, fraction) while flashing; only used during this call, never stored. /// @param firstChunkCheck Invoked with the first chunk of the body before anything is erased: returning false aborts - /// the flash with the partition untouched, so a download that is not the kind of image the partition takes (a - /// littlefs image for a www partition, say) never destroys what is there. Only used during this call, never stored. + /// the flash with the partition untouched, so a download that is not the kind of image the partition takes + /// never destroys what is there. Only used during this call, never stored. bool FlashPartitionFromUrl( HTTP::Client& client, const esp_partition_t* partition, diff --git a/components/partition_layout/CMakeLists.txt b/components/partition_layout/CMakeLists.txt deleted file mode 100644 index 80c78dd1b..000000000 --- a/components/partition_layout/CMakeLists.txt +++ /dev/null @@ -1,11 +0,0 @@ -# (Named partition_layout: ESP-IDF owns the component name partition_table.) -# The ESP-IDF partition table as data: parse the binary table and retype one entry in place, re-sealing its MD5, -# for the in-place table migrations (docs/STORAGE.md). PartitionTable.cpp is pure C++ over bytes plus OpenShock::MD5 -# (Hashing.h), so the host tests run it unchanged. LiveTable.cpp is the device side: reading the table on flash and -# the guarded write that replaces it. -idf_component_register( - SRCS "src/PartitionTable.cpp" - "src/LiveTable.cpp" - INCLUDE_DIRS "include" - PRIV_REQUIRES crypto logging freertos spi_flash bootloader_support esp_partition app_update # MD5, OS_LOG*, vTaskDelay, esp_flash_read, the table offset, the external partition, esp_ota_* -) diff --git a/components/partition_layout/host_test/CMakeLists.txt b/components/partition_layout/host_test/CMakeLists.txt deleted file mode 100644 index 2cb8b5bad..000000000 --- a/components/partition_layout/host_test/CMakeLists.txt +++ /dev/null @@ -1,15 +0,0 @@ -cmake_minimum_required(VERSION 3.16) - -include($ENV{IDF_PATH}/tools/cmake/project.cmake) - -# Only build 'main'; the component is pure C++ over byte spans plus Hashing.h, compiled directly (see main/CMakeLists.txt). -set(COMPONENTS main) - -# linux-target freertos mock, so unity's IDF integration has its dependencies, and -# the shared host-test stand-ins (openshock_board.h, CONFIG_OPENSHOCK_*). -list(APPEND EXTRA_COMPONENT_DIRS - "$ENV{IDF_PATH}/tools/mocks/freertos/" - "${CMAKE_CURRENT_LIST_DIR}/../../../test/host_support") - -project(partition_layout_host_test) -include(${CMAKE_CURRENT_LIST_DIR}/../../../cmake/cxx_standard.cmake) diff --git a/components/partition_layout/host_test/main/CMakeLists.txt b/components/partition_layout/host_test/main/CMakeLists.txt deleted file mode 100644 index 9bf904d4f..000000000 --- a/components/partition_layout/host_test/main/CMakeLists.txt +++ /dev/null @@ -1,18 +0,0 @@ -# Runs the parser and the in-place retype against the 1.x table every fielded hub carries (built in the test from -# its entries) and against deliberately broken copies of it. -idf_component_register( - SRCS "test_main.cpp" - "test_partition_table.cpp" - "../../src/PartitionTable.cpp" - INCLUDE_DIRS "." - "../../include" # partition_layout/PartitionTable.h - "../../../crypto/include" # Hashing.h - "../../../common/include" # OpenShock.h - REQUIRES unity - host_support # openshock_board.h (test/host_support) - mbedtls # Hashing.h -> mbedtls/md.h - WHOLE_ARCHIVE # keep the auto-registered TEST_CASEs from being stripped -) - -# See components/crypto/host_test/main/CMakeLists.txt: esp_mem.o is otherwise not pulled in by the link order. -target_link_libraries(${COMPONENT_LIB} INTERFACE "-u esp_mbedtls_mem_calloc" "-u esp_mbedtls_mem_free") diff --git a/components/partition_layout/host_test/main/test_main.cpp b/components/partition_layout/host_test/main/test_main.cpp deleted file mode 100644 index fab0f6457..000000000 --- a/components/partition_layout/host_test/main/test_main.cpp +++ /dev/null @@ -1,10 +0,0 @@ -// Host-only (linux target) test binary for the partition_table component. FreeRTOS is -// mocked, so there is no ESP startup - drive Unity directly. -#include "unity.h" - -extern "C" int main(void) -{ - UNITY_BEGIN(); - unity_run_all_tests(); - return UNITY_END(); -} diff --git a/components/partition_layout/host_test/main/test_partition_table.cpp b/components/partition_layout/host_test/main/test_partition_table.cpp deleted file mode 100644 index 3129c99b8..000000000 --- a/components/partition_layout/host_test/main/test_partition_table.cpp +++ /dev/null @@ -1,239 +0,0 @@ -// The parser and the in-place retype against the 1.x table every fielded hub carries and a corpus of broken copies. -#include "unity.h" - -#include "Hashing.h" -#include "partition_layout/PartitionTable.h" - -#include -#include - -using namespace OpenShock; -namespace PT = OpenShock::PartitionTable; - -namespace { - struct Row { - const char* label; - uint8_t type; - uint8_t subtype; - uint32_t offset; - uint32_t size; - }; - - // partitions/legacy/ota_4mb_v1.csv - const std::vector kV1 = { - { "nvs", PT::kTypeData, PT::kSubtypeDataNvs, 0x009000, 0x005000}, - { "otadata", PT::kTypeData, 0x00, 0x00E000, 0x002000}, - { "app0", PT::kTypeApp, 0x10, 0x010000, 0x1A0000}, - { "app1", PT::kTypeApp, 0x11, 0x1B0000, 0x1A0000}, - { "config", PT::kTypeData, PT::kSubtypeDataSpiffs, 0x350000, 0x003000}, - { "static0", PT::kTypeData, PT::kSubtypeDataSpiffs, 0x353000, 0x09D000}, - {"coredump", PT::kTypeData, 0x03, 0x3F0000, 0x010000}, - }; - - void Put32(uint8_t* p, uint32_t value) - { - for (int i = 0; i < 4; ++i) p[i] = static_cast(value >> (8 * i)); - } - - // The bytes gen_esp32part.py writes for `rows`: the entries, the MD5 entry, 0xFF to the end. - PT::Bytes Build(const std::vector& rows) - { - PT::Bytes bytes; - bytes.fill(0xFF); - std::size_t pos = 0; - for (const Row& row : rows) { - uint8_t* p = bytes.data() + pos; - p[0] = PT::kMagic & 0xFF; - p[1] = PT::kMagic >> 8; - p[2] = row.type; - p[3] = row.subtype; - Put32(p + 4, row.offset); - Put32(p + 8, row.size); - memset(p + 12, 0, PT::kLabelSize); - memcpy(p + 12, row.label, strlen(row.label)); - Put32(p + 28, 0); - pos += PT::kEntrySize; - } - bytes[pos] = PT::kMd5Magic & 0xFF; - bytes[pos + 1] = PT::kMd5Magic >> 8; - MD5::Digest digest {}; - TEST_ASSERT_TRUE(MD5::compute(bytes.data(), pos, digest)); - memcpy(bytes.data() + pos + 16, digest.data(), digest.size()); - return bytes; - } - - PT::Table ParseOk(std::span bytes) - { - PT::Table table {}; - auto result = PT::Parse(bytes, table); - TEST_ASSERT_TRUE_MESSAGE(result.has_value(), result ? "" : PT::ParseErrorName(result.error())); - return table; - } - - void ExpectParseError(std::span bytes, PT::ParseError expected) - { - PT::Table table {}; - auto result = PT::Parse(bytes, table); - TEST_ASSERT_FALSE_MESSAGE(result.has_value(), "parsed a broken table"); - TEST_ASSERT_EQUAL_STRING(PT::ParseErrorName(expected), PT::ParseErrorName(result.error())); - } - - void ExpectRetypeError(PT::Bytes bytes, std::string_view label, std::string_view newLabel, PT::RetypeError expected) - { - const PT::Bytes before = bytes; - auto result = PT::Retype(bytes, label, PT::kTypeData, PT::kSubtypeDataNvs, newLabel); - TEST_ASSERT_FALSE_MESSAGE(result.has_value(), "retyped what it should not have"); - TEST_ASSERT_EQUAL_STRING(PT::RetypeErrorName(expected), PT::RetypeErrorName(result.error())); - TEST_ASSERT_TRUE_MESSAGE(bytes == before, "a refused retype changed the table"); - } -} // namespace - -TEST_CASE("The 1.x table parses with its MD5", "[partition_table]") -{ - PT::Table v1 = ParseOk(Build(kV1)); - TEST_ASSERT_EQUAL_size_t(7, v1.count); - TEST_ASSERT_EQUAL_STRING("nvs", v1.entries[0].label); - TEST_ASSERT_EQUAL_HEX32(0x9000, v1.entries[0].offset); - TEST_ASSERT_EQUAL_HEX32(0x5000, v1.entries[0].size); - TEST_ASSERT_EQUAL_UINT8(PT::kTypeData, v1.entries[0].type); - TEST_ASSERT_EQUAL_UINT8(PT::kSubtypeDataNvs, v1.entries[0].subtype); - const PT::Entry* config = v1.Find("config"); - TEST_ASSERT_NOT_NULL(config); - TEST_ASSERT_EQUAL_UINT8(PT::kSubtypeDataSpiffs, config->subtype); - TEST_ASSERT_EQUAL_HEX32(0x350000, config->offset); - TEST_ASSERT_NULL(v1.Find("cfg")); -} - -TEST_CASE("A table without an MD5 entry, or with a wrong one, is refused", "[partition_table][corrupt]") -{ - const PT::Bytes good = Build(kV1); - const std::size_t md5Pos = kV1.size() * PT::kEntrySize; - - PT::Bytes bytes = good; - bytes[16] ^= 0x01; // a byte inside the first entry: the MD5 no longer matches - ExpectParseError(bytes, PT::ParseError::Md5Mismatch); - - bytes = good; - memset(bytes.data() + md5Pos, 0xFF, PT::kEntrySize); // the MD5 entry erased - ExpectParseError(bytes, PT::ParseError::NoMd5); - - bytes = good; - bytes[md5Pos + 16] ^= 0x01; // the digest itself - ExpectParseError(bytes, PT::ParseError::Md5Mismatch); - - bytes = good; - bytes[md5Pos + PT::kEntrySize + 5] = 0x00; // garbage after the MD5 entry - ExpectParseError(bytes, PT::ParseError::Trailing); - - bytes = good; - memcpy(bytes.data() + md5Pos + PT::kEntrySize, bytes.data(), PT::kEntrySize); // an entry after the MD5 entry - ExpectParseError(bytes, PT::ParseError::Trailing); - - PT::Bytes erased; - erased.fill(0xFF); - ExpectParseError(erased, PT::ParseError::NoEntries); - ExpectParseError(std::vector(10, 0xFF), PT::ParseError::TooShort); - ExpectParseError(std::span(good).first(3 * PT::kEntrySize), PT::ParseError::NoMd5); -} - -TEST_CASE("Only the first 0xC00 bytes are the table; whatever follows is ignored", "[partition_table]") -{ - // A read that takes more than the table (the next sector, say): the bytes past kMaxLen are not even looked at, - // so they need not be erased - const PT::Bytes good = Build(kV1); - std::vector longer(PT::kMaxLen + 0x400, 0x5A); - memcpy(longer.data(), good.data(), good.size()); - PT::Table table = ParseOk(longer); - TEST_ASSERT_EQUAL_size_t(kV1.size(), table.count); - TEST_ASSERT_NOT_NULL(table.Find("static0")); -} - -TEST_CASE("A table with no room left for the MD5 entry is refused", "[partition_table][corrupt]") -{ - // kMaxEntries entries fill every slot but the last; one more leaves the MD5 entry nowhere to go - PT::Bytes bytes; - bytes.fill(0xFF); - for (std::size_t i = 0; i <= PT::kMaxEntries; ++i) { - uint8_t* p = bytes.data() + i * PT::kEntrySize; - memset(p, 0, PT::kEntrySize); - p[0] = PT::kMagic & 0xFF; - p[1] = PT::kMagic >> 8; - p[2] = PT::kTypeData; - p[3] = PT::kSubtypeDataNvs; - Put32(p + 4, 0x9000 + static_cast(i) * 0x1000); - Put32(p + 8, 0x1000); - p[12] = 'p'; - } - ExpectParseError(bytes, PT::ParseError::TooManyEntries); -} - -TEST_CASE("Retyping static0 as www is byte for byte what the packer would write", "[partition_table][retype]") -{ - // The one retype the www migration does: static0 (data/spiffs) becomes www (type 0x40, subtype 0x00), in place - PT::Bytes bytes = Build(kV1); - TEST_ASSERT_TRUE(PT::Retype(bytes, "static0", 0x40, 0x00, "www").has_value()); - - std::vector expected = kV1; - expected[5] = {"www", 0x40, 0x00, 0x353000, 0x09D000}; - TEST_ASSERT_EQUAL_UINT8_ARRAY(Build(expected).data(), bytes.data(), PT::kMaxLen); - - PT::Table table = ParseOk(bytes); - TEST_ASSERT_NULL(table.Find("static0")); - const PT::Entry* www = table.Find("www"); - TEST_ASSERT_NOT_NULL(www); - TEST_ASSERT_EQUAL_UINT8(0x40, www->type); - TEST_ASSERT_EQUAL_UINT8(0x00, www->subtype); - TEST_ASSERT_EQUAL_HEX32(0x353000, www->offset); - TEST_ASSERT_EQUAL_HEX32(0x09D000, www->size); -} - -TEST_CASE("Retype keeps the label when the new one is the same", "[partition_table][retype]") -{ - // The entry's own label is not "taken": only the type and subtype change - PT::Bytes bytes = Build(kV1); - TEST_ASSERT_TRUE(PT::Retype(bytes, "config", PT::kTypeData, PT::kSubtypeDataNvs, "config").has_value()); - - std::vector expected = kV1; - expected[4] = {"config", PT::kTypeData, PT::kSubtypeDataNvs, 0x350000, 0x003000}; - TEST_ASSERT_EQUAL_UINT8_ARRAY(Build(expected).data(), bytes.data(), PT::kMaxLen); - TEST_ASSERT_EQUAL_UINT8(PT::kSubtypeDataNvs, ParseOk(bytes).Find("config")->subtype); -} - -TEST_CASE("Retype changes one entry's type, subtype and label and re-seals the MD5", "[partition_table][retype]") -{ - PT::Bytes bytes = Build(kV1); - TEST_ASSERT_TRUE(PT::Retype(bytes, "config", PT::kTypeData, PT::kSubtypeDataNvs, "cfg").has_value()); - - // What gen_esp32part.py would write for the retyped table, byte for byte - std::vector expected = kV1; - expected[4] = {"cfg", PT::kTypeData, PT::kSubtypeDataNvs, 0x350000, 0x003000}; - TEST_ASSERT_EQUAL_UINT8_ARRAY(Build(expected).data(), bytes.data(), PT::kMaxLen); - - PT::Table table = ParseOk(bytes); - TEST_ASSERT_NULL(table.Find("config")); - const PT::Entry* cfg = table.Find("cfg"); - TEST_ASSERT_NOT_NULL(cfg); - TEST_ASSERT_EQUAL_UINT8(PT::kSubtypeDataNvs, cfg->subtype); - TEST_ASSERT_EQUAL_HEX32(0x350000, cfg->offset); - TEST_ASSERT_EQUAL_HEX32(0x3000, cfg->size); -} - -TEST_CASE("Retype refuses what it cannot do and leaves the table alone", "[partition_table][retype]") -{ - const PT::Bytes good = Build(kV1); - ExpectRetypeError(good, "nope", "cfg", PT::RetypeError::NoSuchEntry); - ExpectRetypeError(good, "config", "static0", PT::RetypeError::LabelTaken); - - PT::Bytes broken = good; - broken[16] ^= 0x01; - ExpectRetypeError(broken, "config", "cfg", PT::RetypeError::Unparsable); -} - -TEST_CASE("Retype takes a label of the full 16 bytes and refuses a longer one", "[partition_table][retype]") -{ - PT::Bytes bytes = Build(kV1); - TEST_ASSERT_TRUE(PT::Retype(bytes, "config", PT::kTypeData, PT::kSubtypeDataNvs, "sixteen-chars-xx").has_value()); - TEST_ASSERT_NOT_NULL(ParseOk(bytes).Find("sixteen-chars-xx")); - - ExpectRetypeError(Build(kV1), "config", "seventeen-chars-x", PT::RetypeError::LabelTooLong); -} diff --git a/components/partition_layout/host_test/sdkconfig.defaults b/components/partition_layout/host_test/sdkconfig.defaults deleted file mode 100644 index c3d7cf2ce..000000000 --- a/components/partition_layout/host_test/sdkconfig.defaults +++ /dev/null @@ -1,4 +0,0 @@ -CONFIG_IDF_TARGET="linux" -CONFIG_IDF_TARGET_LINUX=y -CONFIG_COMPILER_CXX_EXCEPTIONS=y -CONFIG_UNITY_ENABLE_IDF_TEST_RUNNER=y diff --git a/components/partition_layout/include/partition_layout/LiveTable.h b/components/partition_layout/include/partition_layout/LiveTable.h deleted file mode 100644 index 54290a1d0..000000000 --- a/components/partition_layout/include/partition_layout/LiveTable.h +++ /dev/null @@ -1,28 +0,0 @@ -#pragma once - -// The device side of PartitionTable.h: the table this hub's flash holds right now, and the one dangerous write the -// firmware ever makes. Not built for the host tests. - -#include "partition_layout/PartitionTable.h" - -#include - -namespace OpenShock::PartitionTable { - /// @brief The raw table at 0x8000; false when the flash read fails. - bool ReadLiveBytes(Bytes& out); - - enum class ReplaceResult : uint8_t { - Written, // `next` is on flash - Restored, // `next` would not stick; `live` is back - Untouched, // could not even start: the table on flash was never touched - }; - - /// @brief Writes `next` over the table at 0x8000, whose exact bytes are `live`, and reads it back. - /// - /// The caller quiesces first (no NVS, nothing else using the partition API) and restarts afterwards whatever the - /// result: the running firmware keeps the table it loaded at boot until then. A write that fails (an error, a - /// read-back mismatch) is retried, then `live` is written back, and that is retried for as long as it takes: this - /// never returns while no table that parses is on flash. A power cut between the erase and the write of that one - /// sector is the only thing it cannot recover from. - [[nodiscard]] ReplaceResult Replace(const Bytes& next, const Bytes& live); -} // namespace OpenShock::PartitionTable diff --git a/components/partition_layout/include/partition_layout/PartitionTable.h b/components/partition_layout/include/partition_layout/PartitionTable.h deleted file mode 100644 index 75f722726..000000000 --- a/components/partition_layout/include/partition_layout/PartitionTable.h +++ /dev/null @@ -1,81 +0,0 @@ -#pragma once - -// The binary partition table the bootloader reads at 0x8000 (components/bootloader_support/include/ -// esp_flash_partitions.h): 32-byte entries - magic 0x50AA, type, subtype, offset u32, size u32, label[16], -// flags u32 - followed by an MD5 entry (magic 0xEBEB, 14 bytes 0xFF, the MD5 of everything before it) and -// 0xFF up to 0xC00. -// -// What the in-place table migrations need (docs/STORAGE.md): read the table, and retype one entry - its type, -// subtype and label - without moving anything, re-sealing the MD5. Pure C++ over bytes plus OpenShock::MD5, so the -// host tests run it. - -#include -#include -#include -#include -#include -#include - -namespace OpenShock::PartitionTable { - inline constexpr std::size_t kMaxLen = 0xC00; - inline constexpr std::size_t kEntrySize = 32; - inline constexpr std::size_t kMaxEntries = kMaxLen / kEntrySize - 1; // minus the MD5 entry - inline constexpr std::size_t kLabelSize = 16; - inline constexpr uint16_t kMagic = 0x50AA; - inline constexpr uint16_t kMd5Magic = 0xEBEB; - - inline constexpr uint8_t kTypeApp = 0x00; - inline constexpr uint8_t kTypeData = 0x01; - inline constexpr uint8_t kSubtypeDataNvs = 0x02; - inline constexpr uint8_t kSubtypeDataSpiffs = 0x82; - - /// @brief The raw table, as it is on flash. - using Bytes = std::array; - - struct Entry { - uint8_t type; - uint8_t subtype; - uint32_t offset; - uint32_t size; - uint32_t flags; - char label[kLabelSize + 1]; // NUL-terminated - - [[nodiscard]] std::string_view Label() const { return label; } - }; - - struct Table { - std::array entries; - std::size_t count; - - [[nodiscard]] std::span Entries() const { return std::span(entries).first(count); } - [[nodiscard]] const Entry* Find(std::string_view label) const; - }; - - enum class ParseError : uint8_t { - TooShort, - NoEntries, - TooManyEntries, - NoMd5, // the table ended without an MD5 entry: the bootloader refuses it - Md5Mismatch, // the MD5 entry does not match the entries - Trailing, // something other than 0xFF after the MD5 entry - HashFailed, // computing the MD5 failed: nothing is known about the table - }; - [[nodiscard]] const char* ParseErrorName(ParseError error); - - /// @brief Decodes `bytes` (the first 0xC00 of a table; longer is fine) into `out` and verifies its MD5. A Table is - /// some 3.4 KiB, so on the device `out` belongs on the heap, not on a task's stack. - [[nodiscard]] std::expected Parse(std::span bytes, Table& out); - - enum class RetypeError : uint8_t { - Unparsable, // `bytes` is not a table that parses - NoSuchEntry, // no entry is labelled `label` - LabelTooLong, // the new label does not fit the 16-byte field - LabelTaken, // another entry already has the new label - HashFailed, // computing the MD5 failed - }; - [[nodiscard]] const char* RetypeErrorName(RetypeError error); - - /// @brief Gives the entry labelled `label` a new type, subtype and label in place - offset, size and flags stay - - /// and re-seals the MD5. On failure `bytes` is unchanged. - [[nodiscard]] std::expected Retype(Bytes& bytes, std::string_view label, uint8_t type, uint8_t subtype, std::string_view newLabel); -} // namespace OpenShock::PartitionTable diff --git a/components/partition_layout/src/LiveTable.cpp b/components/partition_layout/src/LiveTable.cpp deleted file mode 100644 index c7f5e65ca..000000000 --- a/components/partition_layout/src/LiveTable.cpp +++ /dev/null @@ -1,86 +0,0 @@ -#include -#include - -#include "partition_layout/LiveTable.h" - -const char* const TAG = "PartitionTable"; - -#include "Logging.h" - -#include -#include -#include -#include - -#include - -using namespace OpenShock; - -namespace { - constexpr int kWriteAttempts = 3; - constexpr TickType_t kWriteRetryDelay = pdMS_TO_TICKS(1'000); - - // Writes `table` through the OTA path, which lifts the dangerous-write protection for that one sector, erases it - // and verifies the written table; then reads it back. False unless `table` is what is on flash. - bool write(const esp_partition_t* pt, const PartitionTable::Bytes& table) - { - esp_err_t err = ESP_OK; - esp_ota_handle_t handle = 0; - // ---- From the erase in esp_ota_begin until esp_ota_write completes, a power cut leaves no partition table ---- - err = esp_ota_begin(pt, ESP_PARTITION_TABLE_MAX_LEN, &handle); - if (err == ESP_OK) { - err = esp_ota_write(handle, table.data(), ESP_PARTITION_TABLE_MAX_LEN); - // ---- The table is whole again from here ---- - esp_err_t endErr = esp_ota_end(handle); - if (err == ESP_OK) { - err = endErr; - } - } - if (err != ESP_OK) { - OS_LOGE(TAG, "Failed to write the partition table: %s", esp_err_to_name(err)); - return false; - } - - auto readBack = std::make_unique(); - if (!PartitionTable::ReadLiveBytes(*readBack) || *readBack != table) { - OS_LOGE(TAG, "The partition table does not read back as written"); - return false; - } - return true; - } -} // namespace - -bool PartitionTable::ReadLiveBytes(Bytes& out) -{ - esp_err_t err = esp_flash_read(nullptr, out.data(), ESP_PRIMARY_PARTITION_TABLE_OFFSET, out.size()); - if (err != ESP_OK) { - OS_LOGE(TAG, "Failed to read the partition table: %s", esp_err_to_name(err)); - return false; - } - return true; -} - -PartitionTable::ReplaceResult PartitionTable::Replace(const Bytes& next, const Bytes& live) -{ - const esp_partition_t* pt = nullptr; - esp_err_t err = esp_partition_register_external(nullptr, ESP_PRIMARY_PARTITION_TABLE_OFFSET, ESP_PARTITION_TABLE_SIZE, "pt", ESP_PARTITION_TYPE_PARTITION_TABLE, ESP_PARTITION_SUBTYPE_PARTITION_TABLE_PRIMARY, &pt); - if (err != ESP_OK) { - OS_LOGE(TAG, "Failed to register the partition table region: %s", esp_err_to_name(err)); - return ReplaceResult::Untouched; - } - - for (int attempt = 0; attempt < kWriteAttempts; ++attempt) { - if (write(pt, next)) { - return ReplaceResult::Written; - } - } - OS_LOGE(TAG, "The new partition table does not stick; writing the old one back"); - for (uint32_t attempt = 1;; ++attempt) { - if (write(pt, live)) { - return ReplaceResult::Restored; - } - // Whatever is on flash now may not parse: restarting would brick the hub. Keep trying. - OS_LOGE(TAG, "Failed to write the old partition table back (attempt %u). Do not unplug the hub.", static_cast(attempt)); - vTaskDelay(kWriteRetryDelay); - } -} diff --git a/components/partition_layout/src/PartitionTable.cpp b/components/partition_layout/src/PartitionTable.cpp deleted file mode 100644 index c56fcd481..000000000 --- a/components/partition_layout/src/PartitionTable.cpp +++ /dev/null @@ -1,197 +0,0 @@ -#include "partition_layout/PartitionTable.h" - -#include "Hashing.h" - -#include -#include - -using namespace OpenShock; - -namespace { - uint16_t readU16(const uint8_t* p) - { - return static_cast(p[0] | (p[1] << 8)); - } - uint32_t readU32(const uint8_t* p) - { - return static_cast(p[0]) | (static_cast(p[1]) << 8) | (static_cast(p[2]) << 16) | (static_cast(p[3]) << 24); - } - - std::string_view rawLabel(const uint8_t* entry) - { - const char* label = reinterpret_cast(entry + 12); - return std::string_view(label, strnlen(label, PartitionTable::kLabelSize)); - } - - // Checks the structure and the MD5 without decoding anything: the number of entries, or why the bootloader would - // refuse the table. - std::expected scan(std::span bytes) - { - using namespace PartitionTable; - - if (bytes.size() < kEntrySize) { - return std::unexpected(ParseError::TooShort); - } - const std::size_t limit = std::min(bytes.size(), kMaxLen); - - std::size_t count = 0; - std::size_t pos = 0; - bool md5Seen = false; - for (; pos + kEntrySize <= limit; pos += kEntrySize) { - const uint8_t* p = bytes.data() + pos; - const uint16_t magic = readU16(p); - if (magic == kMagic) { - if (md5Seen) { - return std::unexpected(ParseError::Trailing); - } - if (count >= kMaxEntries) { - return std::unexpected(ParseError::TooManyEntries); - } - ++count; - continue; - } - if (magic == kMd5Magic) { - if (md5Seen) { - return std::unexpected(ParseError::Trailing); - } - MD5::Digest digest {}; - if (!MD5::compute(bytes.data(), pos, digest)) { - return std::unexpected(ParseError::HashFailed); - } - if (memcmp(digest.data(), p + 16, digest.size()) != 0) { - return std::unexpected(ParseError::Md5Mismatch); - } - md5Seen = true; - continue; - } - // 0xFFFF (erased flash) or anything else: the end of the table. What follows must be erased. - for (std::size_t i = pos; i < limit; ++i) { - if (bytes[i] != 0xFF) { - return std::unexpected(md5Seen ? ParseError::Trailing : ParseError::NoMd5); - } - } - break; - } - - if (count == 0) { - return std::unexpected(ParseError::NoEntries); - } - if (!md5Seen) { - return std::unexpected(ParseError::NoMd5); - } - return count; - } -} // namespace - -const PartitionTable::Entry* PartitionTable::Table::Find(std::string_view label) const -{ - for (const Entry& entry : Entries()) { - if (entry.Label() == label) { - return &entry; - } - } - return nullptr; -} - -const char* PartitionTable::ParseErrorName(ParseError error) -{ - switch (error) { - case ParseError::TooShort: - return "too short"; - case ParseError::NoEntries: - return "no entries"; - case ParseError::TooManyEntries: - return "too many entries"; - case ParseError::NoMd5: - return "no MD5 entry"; - case ParseError::Md5Mismatch: - return "MD5 mismatch"; - case ParseError::Trailing: - return "data after the MD5 entry"; - case ParseError::HashFailed: - return "computing the MD5 failed"; - } - return "unknown"; -} - -std::expected PartitionTable::Parse(std::span bytes, Table& out) -{ - auto count = scan(bytes); - if (!count) { - return std::unexpected(count.error()); - } - - out.count = *count; - for (std::size_t i = 0; i < out.count; ++i) { - const uint8_t* p = bytes.data() + i * kEntrySize; - Entry& entry = out.entries[i]; - entry.type = p[2]; - entry.subtype = p[3]; - entry.offset = readU32(p + 4); - entry.size = readU32(p + 8); - memcpy(entry.label, p + 12, kLabelSize); - entry.label[kLabelSize] = '\0'; - entry.flags = readU32(p + 28); - } - return {}; -} - -const char* PartitionTable::RetypeErrorName(RetypeError error) -{ - switch (error) { - case RetypeError::Unparsable: - return "the table does not parse"; - case RetypeError::NoSuchEntry: - return "no entry has that label"; - case RetypeError::LabelTooLong: - return "the new label is too long"; - case RetypeError::LabelTaken: - return "another entry has the new label"; - case RetypeError::HashFailed: - return "computing the MD5 failed"; - } - return "unknown"; -} - -std::expected PartitionTable::Retype(Bytes& bytes, std::string_view label, uint8_t type, uint8_t subtype, std::string_view newLabel) -{ - if (newLabel.size() > kLabelSize) { - return std::unexpected(RetypeError::LabelTooLong); - } - auto count = scan(bytes); - if (!count) { - return std::unexpected(count.error() == ParseError::HashFailed ? RetypeError::HashFailed : RetypeError::Unparsable); - } - - uint8_t* target = nullptr; - for (std::size_t i = 0; i < *count; ++i) { - uint8_t* p = bytes.data() + i * kEntrySize; - const std::string_view own = rawLabel(p); - if (own == label) { - target = p; - } else if (own == newLabel) { - return std::unexpected(RetypeError::LabelTaken); - } - } - if (target == nullptr) { - return std::unexpected(RetypeError::NoSuchEntry); - } - - uint8_t original[kEntrySize]; - memcpy(original, target, kEntrySize); - - target[2] = type; - target[3] = subtype; - memset(target + 12, 0, kLabelSize); - memcpy(target + 12, newLabel.data(), newLabel.size()); - - // The MD5 entry follows the last entry; its digest covers every entry before it - const std::size_t md5Pos = *count * kEntrySize; - MD5::Digest digest {}; - if (!MD5::compute(bytes.data(), md5Pos, digest)) { - memcpy(target, original, kEntrySize); - return std::unexpected(RetypeError::HashFailed); - } - memcpy(bytes.data() + md5Pos + 16, digest.data(), digest.size()); - return {}; -} diff --git a/components/serial_console/CMakeLists.txt b/components/serial_console/CMakeLists.txt index eda9e4c15..4ca64d73a 100644 --- a/components/serial_console/CMakeLists.txt +++ b/components/serial_console/CMakeLists.txt @@ -10,8 +10,8 @@ FILE(GLOB_RECURSE console_sources idf_component_register( SRCS ${console_sources} INCLUDE_DIRS "include" - REQUIRES config serialization # its config section (serial_console/SerialInputConfig.h) - PRIV_REQUIRES core serial device_control crypto migration # core: wifi/gateway/http/serialization; serial: Serial::Init/Read; device_control: estop/CommandHandler/visual/radio; crypto: Base64 + REQUIRES config # its config section (serial_console/SerialInputConfig.h) + PRIV_REQUIRES core serial device_control # core: wifi/gateway/http/serialization; serial: Serial::Init/Read; device_control: estop/CommandHandler/visual/radio common logging temporal osjson chipset # used directly; previously only reachable through core's public REQUIRES ) diff --git a/components/serial_console/Kconfig b/components/serial_console/Kconfig index 8f502d9a3..25cb019cf 100644 --- a/components/serial_console/Kconfig +++ b/components/serial_console/Kconfig @@ -5,7 +5,7 @@ menu "OpenShock serial console" range 256 16384 default 4096 help - A longer line is discarded. rawconfig and jsonconfig take the whole config on - one line, so this bounds the config the console can set. + A longer line is discarded. jsonconfig takes the whole config on one line, so + this bounds the config the console can set. endmenu diff --git a/components/serial_console/include/serial_console/SerialInputConfig.h b/components/serial_console/include/serial_console/SerialInputConfig.h index 1c814ddd1..f4c0747a5 100644 --- a/components/serial_console/include/serial_console/SerialInputConfig.h +++ b/components/serial_console/include/serial_console/SerialInputConfig.h @@ -4,7 +4,6 @@ // (config/Section.h); namespace "serial", key echo (u8). #include "config/Section.h" -#include "serialization/_fbs/HubConfig_generated.h" #include @@ -22,8 +21,6 @@ namespace OpenShock::Config { /// @brief No rules to apply yet. The section runs it on every value it takes in. void Normalize() { } - [[nodiscard]] flatbuffers::Offset ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const; - /// @brief Reads this section from `partition`; a missing key is its default, a read error fails. bool LoadNvs(const char* partition); /// @brief Writes the keys that differ from `previous` (every key when null) to `partition`. diff --git a/components/serial_console/include/serial_console/command_handlers/index.h b/components/serial_console/include/serial_console/command_handlers/index.h index 94abd722a..9096b092c 100644 --- a/components/serial_console/include/serial_console/command_handlers/index.h +++ b/components/serial_console/include/serial_console/command_handlers/index.h @@ -18,11 +18,9 @@ namespace OpenShock::SerialCmds::CommandHandlers { OpenShock::SerialCmds::CommandGroup NetworksHandler(); OpenShock::SerialCmds::CommandGroup KeepAliveHandler(); OpenShock::SerialCmds::CommandGroup JsonConfigHandler(); - OpenShock::SerialCmds::CommandGroup RawConfigHandler(); OpenShock::SerialCmds::CommandGroup RfTransmitHandler(); OpenShock::SerialCmds::CommandGroup FactoryResetHandler(); OpenShock::SerialCmds::CommandGroup LedTestHandler(); - OpenShock::SerialCmds::CommandGroup MigrateHandler(); inline std::vector AllCommandHandlers() { @@ -40,11 +38,9 @@ namespace OpenShock::SerialCmds::CommandHandlers { NetworksHandler(), KeepAliveHandler(), JsonConfigHandler(), - RawConfigHandler(), RfTransmitHandler(), FactoryResetHandler(), LedTestHandler(), - MigrateHandler(), }; } } // namespace OpenShock::SerialCmds::CommandHandlers diff --git a/components/serial_console/src/SerialInputConfig.cpp b/components/serial_console/src/SerialInputConfig.cpp index 5d8ce017a..7d30262cc 100644 --- a/components/serial_console/src/SerialInputConfig.cpp +++ b/components/serial_console/src/SerialInputConfig.cpp @@ -11,13 +11,12 @@ const char* const TAG = "Config::SerialInputConfig"; #include "json/Json.h" using namespace OpenShock::Config; -namespace Fbs = OpenShock::Serialization::Configuration; static constexpr const char* kNamespace = "serial"; static constexpr const char* kKeyEcho = "echo"; static constexpr const char* kNamespaces[] = {kNamespace}; -static Section s_section({.name = "serialInput", .order = 4, .hubConfigField = Fbs::HubConfig::VT_SERIAL_INPUT, .namespaces = kNamespaces}); +static Section s_section({.name = "serialInput", .order = 4, .namespaces = kNamespaces}); SerialInputConfig::SerialInputConfig() : echoEnabled(true) @@ -29,11 +28,6 @@ SerialInputConfig::SerialInputConfig(bool echoEnabled) { } -flatbuffers::Offset SerialInputConfig::ToFlatbuffers(flatbuffers::FlatBufferBuilder& builder, bool withSensitiveData) const -{ - return Serialization::Configuration::CreateSerialInputConfig(builder, echoEnabled); -} - bool SerialInputConfig::FromJSON(JSON::JsonView json) { if (auto done = Parse::ObjectOrDefault(*this, json, TAG)) { @@ -65,7 +59,7 @@ bool SerialInputConfig::LoadNvs(const char* partition) bool SerialInputConfig::SaveNvs(const char* partition, const SerialInputConfig* previous) const { - auto handle = Nvs::Handle::Open(partition, kNamespace, true); + auto handle = Nvs::Handle::OpenForWrite(partition, kNamespace); if (!handle) { return false; } diff --git a/components/serial_console/src/SerialInputHandler.cpp b/components/serial_console/src/SerialInputHandler.cpp index 6b8b52c9f..1e2b73f92 100644 --- a/components/serial_console/src/SerialInputHandler.cpp +++ b/components/serial_console/src/SerialInputHandler.cpp @@ -4,7 +4,6 @@ const char* const TAG = "SerialInputHandler"; -#include "Base64.h" #include "Chipset.h" #include "CommandHandler.h" #include "Convert.h" @@ -25,6 +24,7 @@ const char* const TAG = "SerialInputHandler"; #include "util/TaskUtils.h" #include "wifi/WiFiManager.h" +#include #include #include #include diff --git a/components/serial_console/src/command_handlers/migrate.cpp b/components/serial_console/src/command_handlers/migrate.cpp deleted file mode 100644 index ef48fc187..000000000 --- a/components/serial_console/src/command_handlers/migrate.cpp +++ /dev/null @@ -1,62 +0,0 @@ -#include "serial_console/command_handlers/common.h" - -#include "migration/StorageMigration.h" -#include "StringHelpers.h" -#include "util/HexUtils.h" - -#include - -// `migrate`: the state of the 1.x -> 2.x storage migration (migration/StorageMigration.h) and a way to run its www -// step from an image of your own, for builds that have no release carrying one. - -static void handleMigrateStatus(std::string_view arg, bool isAutomated) -{ - (void)arg; - (void)isAutomated; - SERPR_RESPONSE("Migrate|%s", OpenShock::Migration::IsPending() ? "pending" : "done"); -} - -static void handleMigrateNow(std::string_view arg, bool isAutomated) -{ - (void)arg; - (void)isAutomated; - if (!OpenShock::Migration::RequestNow()) { - SERPR_ERROR("No storage migration is pending"); - return; - } - SERPR_SUCCESS("Running the next migration step now"); -} - -static void handleMigrateWww(std::string_view arg, bool isAutomated) -{ - (void)isAutomated; - auto parts = OpenShock::StringSplit(arg, ' ', 1); - if (parts.size() != 2 || parts[0].empty() || parts[1].size() != 64) { - SERPR_ERROR("Usage: migrate www "); - return; - } - uint8_t sha256[32]; - if (OpenShock::HexUtils::TryParseHex(parts[1], std::span(sha256)) != sizeof(sha256)) { - SERPR_ERROR("The hash is not 64 hex digits"); - return; - } - if (!OpenShock::Migration::RequestWww(std::string(parts[0]), sha256)) { - SERPR_ERROR("No storage migration is pending"); - return; - } - SERPR_SUCCESS("The www partition migration will download that image once WiFi is up; the hub restarts when it is done"); -} - -OpenShock::SerialCmds::CommandGroup OpenShock::SerialCmds::CommandHandlers::MigrateHandler() -{ - auto group = OpenShock::SerialCmds::CommandGroup("migrate"sv); - - group.addCommand("Whether the 1.x -> 2.x storage migration is still pending"sv, handleMigrateStatus); - group.addCommand("now"sv, "Run the next pending migration step now instead of after the retry delay"sv, handleMigrateNow); - - auto& www = group.addCommand("www"sv, "Run the www partition migration from this image instead of the running release's"sv, handleMigrateWww); - www.addArgument("url"sv, "the www image (http or https)"sv, "http://192.168.1.2:8000/www.bin"sv); - www.addArgument("sha256"sv, "its SHA-256, 64 hex digits"sv, "0000000000000000000000000000000000000000000000000000000000000000"sv); - - return group; -} diff --git a/components/serial_console/src/command_handlers/rawconfig.cpp b/components/serial_console/src/command_handlers/rawconfig.cpp deleted file mode 100644 index d1dd6e02c..000000000 --- a/components/serial_console/src/command_handlers/rawconfig.cpp +++ /dev/null @@ -1,49 +0,0 @@ -#include "serial_console/command_handlers/common.h" - -#include "Base64.h" -#include "config/Wire.h" -#include "TinyVec.h" - -#include - -static void handleRawConfigCommand(std::string_view arg, bool isAutomated) -{ - if (arg.empty()) { - TinyVec buffer; - - // Get raw config - if (!OpenShock::Config::GetRaw(buffer)) { - SERPR_ERROR("Failed to get raw config"); - return; - } - - std::string base64; - if (!OpenShock::Base64::Encode(buffer, base64)) { - SERPR_ERROR("Failed to encode raw config to base64"); - return; - } - - SERPR_RESPONSE("RawConfig|%s", base64.c_str()); - return; - } - - TinyVec buffer; - if (!OpenShock::Base64::Decode(arg, buffer)) { - SERPR_ERROR("Failed to decode base64"); - return; - } - - OpenShock::SerialCmds::ReportSavedAndRestart(OpenShock::Config::SetRaw(buffer.data(), buffer.size())); -} - -OpenShock::SerialCmds::CommandGroup OpenShock::SerialCmds::CommandHandlers::RawConfigHandler() -{ - auto group = OpenShock::SerialCmds::CommandGroup("rawconfig"sv); - - group.addCommand("Get the raw binary config"sv, handleRawConfigCommand); - - auto& setCommand = group.addCommand("Set the raw binary config, and restart"sv, handleRawConfigCommand); - setCommand.addArgument("base64"sv, "must be a base64 encoded string"sv, "(base64 encoded binary data)"sv); - - return group; -} diff --git a/components/serialization/CMakeLists.txt b/components/serialization/CMakeLists.txt index ec965f5ab..f1daa6f68 100644 --- a/components/serialization/CMakeLists.txt +++ b/components/serialization/CMakeLists.txt @@ -2,9 +2,9 @@ # # For now this is the generated flatbuffer wire types (serialization/_fbs/*_generated.h, # produced by scripts/generate_schemas.py from schemas/*.fbs), a single flatbuffers-only -# layer that config, wifi, the message codecs and the handlers all depend on one-way. -# Extracting these out of `core` is what lets config (and later the codecs themselves: -# JsonAPI/JsonSerial/WSGateway/WSLocal) leave core without a dependency cycle. +# layer that the message codecs and the handlers depend on one-way. Extracting these out +# of `core` is what lets the codecs (JsonAPI/JsonSerial/WSGateway/WSLocal) leave core +# later without a dependency cycle. idf_component_register( INCLUDE_DIRS "include" REQUIRES flatbuffers # flatbuffers/flatbuffers.h in the generated headers diff --git a/components/serialization/include/serialization/_fbs/HubConfig_generated.h b/components/serialization/include/serialization/_fbs/HubConfig_generated.h deleted file mode 100644 index 5659ca9be..000000000 --- a/components/serialization/include/serialization/_fbs/HubConfig_generated.h +++ /dev/null @@ -1,1174 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - - -#ifndef FLATBUFFERS_GENERATED_HUBCONFIG_OPENSHOCK_SERIALIZATION_CONFIGURATION_H_ -#define FLATBUFFERS_GENERATED_HUBCONFIG_OPENSHOCK_SERIALIZATION_CONFIGURATION_H_ - -#include "flatbuffers/flatbuffers.h" - -// Ensure the included flatbuffers.h is the same version as when this file was -// generated, otherwise it may not be compatible. -static_assert(FLATBUFFERS_VERSION_MAJOR == 25 && - FLATBUFFERS_VERSION_MINOR == 12 && - FLATBUFFERS_VERSION_REVISION == 19, - "Non-compatible flatbuffers version included"); - -#include "WifiAuthMode_generated.h" - -namespace OpenShock { -namespace Serialization { -namespace Configuration { - -struct RFConfig; -struct RFConfigBuilder; - -struct EStopConfig; -struct EStopConfigBuilder; - -struct MacAddress; - -struct WiFiCredentials; -struct WiFiCredentialsBuilder; - -struct WiFiConfig; -struct WiFiConfigBuilder; - -struct CaptivePortalConfig; -struct CaptivePortalConfigBuilder; - -struct BackendConfig; -struct BackendConfigBuilder; - -struct SerialInputConfig; -struct SerialInputConfigBuilder; - -struct OtaUpdateConfig; -struct OtaUpdateConfigBuilder; - -struct LanConfig; -struct LanConfigBuilder; - -struct HubConfig; -struct HubConfigBuilder; - -enum class OtaUpdateChannel : uint8_t { - Stable = 0, - Beta = 1, - Develop = 2, - MIN = Stable, - MAX = Develop -}; - -inline const OtaUpdateChannel (&EnumValuesOtaUpdateChannel())[3] { - static const OtaUpdateChannel values[] = { - OtaUpdateChannel::Stable, - OtaUpdateChannel::Beta, - OtaUpdateChannel::Develop - }; - return values; -} - -inline const char * const *EnumNamesOtaUpdateChannel() { - static const char * const names[4] = { - "Stable", - "Beta", - "Develop", - nullptr - }; - return names; -} - -inline const char *EnumNameOtaUpdateChannel(OtaUpdateChannel e) { - if (::flatbuffers::IsOutRange(e, OtaUpdateChannel::Stable, OtaUpdateChannel::Develop)) return ""; - const size_t index = static_cast(e); - return EnumNamesOtaUpdateChannel()[index]; -} - -enum class OtaUpdateStep : uint8_t { - None = 0, - Updating = 1, - Updated = 2, - Validating = 3, - Validated = 4, - RollingBack = 5, - MIN = None, - MAX = RollingBack -}; - -inline const OtaUpdateStep (&EnumValuesOtaUpdateStep())[6] { - static const OtaUpdateStep values[] = { - OtaUpdateStep::None, - OtaUpdateStep::Updating, - OtaUpdateStep::Updated, - OtaUpdateStep::Validating, - OtaUpdateStep::Validated, - OtaUpdateStep::RollingBack - }; - return values; -} - -inline const char * const *EnumNamesOtaUpdateStep() { - static const char * const names[7] = { - "None", - "Updating", - "Updated", - "Validating", - "Validated", - "RollingBack", - nullptr - }; - return names; -} - -inline const char *EnumNameOtaUpdateStep(OtaUpdateStep e) { - if (::flatbuffers::IsOutRange(e, OtaUpdateStep::None, OtaUpdateStep::RollingBack)) return ""; - const size_t index = static_cast(e); - return EnumNamesOtaUpdateStep()[index]; -} - -/// 6-byte MAC address (BSSID) -FLATBUFFERS_MANUALLY_ALIGNED_STRUCT(1) MacAddress FLATBUFFERS_FINAL_CLASS { - private: - uint8_t bytes_[6]; - - public: - struct Traits; - static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() { - return "OpenShock.Serialization.Configuration.MacAddress"; - } - MacAddress() - : bytes_() { - } - MacAddress(::flatbuffers::span _bytes) { - ::flatbuffers::CastToArray(bytes_).CopyFromSpan(_bytes); - } - const ::flatbuffers::Array *bytes() const { - return &::flatbuffers::CastToArray(bytes_); - } -}; -FLATBUFFERS_STRUCT_END(MacAddress, 6); - -struct MacAddress::Traits { - using type = MacAddress; -}; - -struct RFConfig FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { - typedef RFConfigBuilder Builder; - struct Traits; - static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() { - return "OpenShock.Serialization.Configuration.RFConfig"; - } - enum FlatBuffersVTableOffset FLATBUFFERS_VTABLE_UNDERLYING_TYPE { - VT_TX_PIN = 4, - VT_KEEPALIVE_ENABLED = 6 - }; - /// The GPIO pin connected to the RF modulator's data pin for transmitting (TX) - int8_t tx_pin() const { - return GetField(VT_TX_PIN, 0); - } - /// Whether to transmit keepalive messages to keep the shockers from entering sleep mode - bool keepalive_enabled() const { - return GetField(VT_KEEPALIVE_ENABLED, 1) != 0; - } - template - bool Verify(::flatbuffers::VerifierTemplate &verifier) const { - return VerifyTableStart(verifier) && - VerifyField(verifier, VT_TX_PIN, 1) && - VerifyField(verifier, VT_KEEPALIVE_ENABLED, 1) && - verifier.EndTable(); - } -}; - -struct RFConfigBuilder { - typedef RFConfig Table; - ::flatbuffers::FlatBufferBuilder &fbb_; - ::flatbuffers::uoffset_t start_; - void add_tx_pin(int8_t tx_pin) { - fbb_.AddElement(RFConfig::VT_TX_PIN, tx_pin, 0); - } - void add_keepalive_enabled(bool keepalive_enabled) { - fbb_.AddElement(RFConfig::VT_KEEPALIVE_ENABLED, static_cast(keepalive_enabled), 1); - } - explicit RFConfigBuilder(::flatbuffers::FlatBufferBuilder &_fbb) - : fbb_(_fbb) { - start_ = fbb_.StartTable(); - } - ::flatbuffers::Offset Finish() { - const auto end = fbb_.EndTable(start_); - auto o = ::flatbuffers::Offset(end); - return o; - } -}; - -inline ::flatbuffers::Offset CreateRFConfig( - ::flatbuffers::FlatBufferBuilder &_fbb, - int8_t tx_pin = 0, - bool keepalive_enabled = true) { - RFConfigBuilder builder_(_fbb); - builder_.add_keepalive_enabled(keepalive_enabled); - builder_.add_tx_pin(tx_pin); - return builder_.Finish(); -} - -struct RFConfig::Traits { - using type = RFConfig; - static auto constexpr Create = CreateRFConfig; -}; - -struct EStopConfig FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { - typedef EStopConfigBuilder Builder; - struct Traits; - static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() { - return "OpenShock.Serialization.Configuration.EStopConfig"; - } - enum FlatBuffersVTableOffset FLATBUFFERS_VTABLE_UNDERLYING_TYPE { - VT_ENABLED = 4, - VT_GPIO_PIN = 6, - VT_ACTIVE = 8, - VT_LATCHING = 10 - }; - bool enabled() const { - return GetField(VT_ENABLED, 0) != 0; - } - /// The GPIO pin connected to the E-Stop button - int8_t gpio_pin() const { - return GetField(VT_GPIO_PIN, 0); - } - /// Persistent state of the E-Stop button - bool active() const { - return GetField(VT_ACTIVE, 0) != 0; - } - /// Set When EmergencyStop is a latching switch - bool latching() const { - return GetField(VT_LATCHING, 0) != 0; - } - template - bool Verify(::flatbuffers::VerifierTemplate &verifier) const { - return VerifyTableStart(verifier) && - VerifyField(verifier, VT_ENABLED, 1) && - VerifyField(verifier, VT_GPIO_PIN, 1) && - VerifyField(verifier, VT_ACTIVE, 1) && - VerifyField(verifier, VT_LATCHING, 1) && - verifier.EndTable(); - } -}; - -struct EStopConfigBuilder { - typedef EStopConfig Table; - ::flatbuffers::FlatBufferBuilder &fbb_; - ::flatbuffers::uoffset_t start_; - void add_enabled(bool enabled) { - fbb_.AddElement(EStopConfig::VT_ENABLED, static_cast(enabled), 0); - } - void add_gpio_pin(int8_t gpio_pin) { - fbb_.AddElement(EStopConfig::VT_GPIO_PIN, gpio_pin, 0); - } - void add_active(bool active) { - fbb_.AddElement(EStopConfig::VT_ACTIVE, static_cast(active), 0); - } - void add_latching(bool latching) { - fbb_.AddElement(EStopConfig::VT_LATCHING, static_cast(latching), 0); - } - explicit EStopConfigBuilder(::flatbuffers::FlatBufferBuilder &_fbb) - : fbb_(_fbb) { - start_ = fbb_.StartTable(); - } - ::flatbuffers::Offset Finish() { - const auto end = fbb_.EndTable(start_); - auto o = ::flatbuffers::Offset(end); - return o; - } -}; - -inline ::flatbuffers::Offset CreateEStopConfig( - ::flatbuffers::FlatBufferBuilder &_fbb, - bool enabled = false, - int8_t gpio_pin = 0, - bool active = false, - bool latching = false) { - EStopConfigBuilder builder_(_fbb); - builder_.add_latching(latching); - builder_.add_active(active); - builder_.add_gpio_pin(gpio_pin); - builder_.add_enabled(enabled); - return builder_.Finish(); -} - -struct EStopConfig::Traits { - using type = EStopConfig; - static auto constexpr Create = CreateEStopConfig; -}; - -struct WiFiCredentials FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { - typedef WiFiCredentialsBuilder Builder; - struct Traits; - static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() { - return "OpenShock.Serialization.Configuration.WiFiCredentials"; - } - enum FlatBuffersVTableOffset FLATBUFFERS_VTABLE_UNDERLYING_TYPE { - VT_ID = 4, - VT_SSID = 6, - VT_PASSWORD = 8, - VT_AUTH_MODE = 10, - VT_BSSID = 12 - }; - /// ID of the WiFi network credentials, used for referencing the credentials with a low memory footprint - uint8_t id() const { - return GetField(VT_ID, 0); - } - /// SSID of the WiFi network - const ::flatbuffers::String *ssid() const { - return GetPointer(VT_SSID); - } - /// Password of the WiFi network - const ::flatbuffers::String *password() const { - return GetPointer(VT_PASSWORD); - } - /// Auth mode of the network when credentials were saved. Used to reject evil twins with weaker security. - OpenShock::Serialization::Types::WifiAuthMode auth_mode() const { - return static_cast(GetField(VT_AUTH_MODE, 9)); - } - /// Pinned BSSID of the last successfully connected AP. All zeros means no pin. - const OpenShock::Serialization::Configuration::MacAddress *bssid() const { - return GetStruct(VT_BSSID); - } - template - bool Verify(::flatbuffers::VerifierTemplate &verifier) const { - return VerifyTableStart(verifier) && - VerifyField(verifier, VT_ID, 1) && - VerifyOffset(verifier, VT_SSID) && - verifier.VerifyString(ssid()) && - VerifyOffset(verifier, VT_PASSWORD) && - verifier.VerifyString(password()) && - VerifyField(verifier, VT_AUTH_MODE, 1) && - VerifyField(verifier, VT_BSSID, 1) && - verifier.EndTable(); - } -}; - -struct WiFiCredentialsBuilder { - typedef WiFiCredentials Table; - ::flatbuffers::FlatBufferBuilder &fbb_; - ::flatbuffers::uoffset_t start_; - void add_id(uint8_t id) { - fbb_.AddElement(WiFiCredentials::VT_ID, id, 0); - } - void add_ssid(::flatbuffers::Offset<::flatbuffers::String> ssid) { - fbb_.AddOffset(WiFiCredentials::VT_SSID, ssid); - } - void add_password(::flatbuffers::Offset<::flatbuffers::String> password) { - fbb_.AddOffset(WiFiCredentials::VT_PASSWORD, password); - } - void add_auth_mode(OpenShock::Serialization::Types::WifiAuthMode auth_mode) { - fbb_.AddElement(WiFiCredentials::VT_AUTH_MODE, static_cast(auth_mode), 9); - } - void add_bssid(const OpenShock::Serialization::Configuration::MacAddress *bssid) { - fbb_.AddStruct(WiFiCredentials::VT_BSSID, bssid); - } - explicit WiFiCredentialsBuilder(::flatbuffers::FlatBufferBuilder &_fbb) - : fbb_(_fbb) { - start_ = fbb_.StartTable(); - } - ::flatbuffers::Offset Finish() { - const auto end = fbb_.EndTable(start_); - auto o = ::flatbuffers::Offset(end); - return o; - } -}; - -inline ::flatbuffers::Offset CreateWiFiCredentials( - ::flatbuffers::FlatBufferBuilder &_fbb, - uint8_t id = 0, - ::flatbuffers::Offset<::flatbuffers::String> ssid = 0, - ::flatbuffers::Offset<::flatbuffers::String> password = 0, - OpenShock::Serialization::Types::WifiAuthMode auth_mode = OpenShock::Serialization::Types::WifiAuthMode::UNKNOWN, - const OpenShock::Serialization::Configuration::MacAddress *bssid = nullptr) { - WiFiCredentialsBuilder builder_(_fbb); - builder_.add_bssid(bssid); - builder_.add_password(password); - builder_.add_ssid(ssid); - builder_.add_auth_mode(auth_mode); - builder_.add_id(id); - return builder_.Finish(); -} - -struct WiFiCredentials::Traits { - using type = WiFiCredentials; - static auto constexpr Create = CreateWiFiCredentials; -}; - -inline ::flatbuffers::Offset CreateWiFiCredentialsDirect( - ::flatbuffers::FlatBufferBuilder &_fbb, - uint8_t id = 0, - const char *ssid = nullptr, - const char *password = nullptr, - OpenShock::Serialization::Types::WifiAuthMode auth_mode = OpenShock::Serialization::Types::WifiAuthMode::UNKNOWN, - const OpenShock::Serialization::Configuration::MacAddress *bssid = nullptr) { - auto ssid__ = ssid ? _fbb.CreateString(ssid) : 0; - auto password__ = password ? _fbb.CreateString(password) : 0; - return OpenShock::Serialization::Configuration::CreateWiFiCredentials( - _fbb, - id, - ssid__, - password__, - auth_mode, - bssid); -} - -struct WiFiConfig FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { - typedef WiFiConfigBuilder Builder; - struct Traits; - static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() { - return "OpenShock.Serialization.Configuration.WiFiConfig"; - } - enum FlatBuffersVTableOffset FLATBUFFERS_VTABLE_UNDERLYING_TYPE { - VT_AP_SSID = 4, - VT_HOSTNAME = 6, - VT_CREDENTIALS = 8, - VT_AP_PASSWORD = 10, - VT_AP_DISABLED = 12, - VT_STA_DISABLED = 14, - VT_OFFLINE_MODE = 16 - }; - /// Access point SSID - const ::flatbuffers::String *ap_ssid() const { - return GetPointer(VT_AP_SSID); - } - /// Hub hostname - const ::flatbuffers::String *hostname() const { - return GetPointer(VT_HOSTNAME); - } - /// WiFi network credentials - const ::flatbuffers::Vector<::flatbuffers::Offset> *credentials() const { - return GetPointer> *>(VT_CREDENTIALS); - } - /// Access point password (empty = open network) - const ::flatbuffers::String *ap_password() const { - return GetPointer(VT_AP_PASSWORD); - } - /// Disable WiFi Access Point mode completely - bool ap_disabled() const { - return GetField(VT_AP_DISABLED, 0) != 0; - } - /// Disable WiFi Station mode completely - bool sta_disabled() const { - return GetField(VT_STA_DISABLED, 0) != 0; - } - /// Don't connect to any cloud services - bool offline_mode() const { - return GetField(VT_OFFLINE_MODE, 0) != 0; - } - template - bool Verify(::flatbuffers::VerifierTemplate &verifier) const { - return VerifyTableStart(verifier) && - VerifyOffset(verifier, VT_AP_SSID) && - verifier.VerifyString(ap_ssid()) && - VerifyOffset(verifier, VT_HOSTNAME) && - verifier.VerifyString(hostname()) && - VerifyOffset(verifier, VT_CREDENTIALS) && - verifier.VerifyVector(credentials()) && - verifier.VerifyVectorOfTables(credentials()) && - VerifyOffset(verifier, VT_AP_PASSWORD) && - verifier.VerifyString(ap_password()) && - VerifyField(verifier, VT_AP_DISABLED, 1) && - VerifyField(verifier, VT_STA_DISABLED, 1) && - VerifyField(verifier, VT_OFFLINE_MODE, 1) && - verifier.EndTable(); - } -}; - -struct WiFiConfigBuilder { - typedef WiFiConfig Table; - ::flatbuffers::FlatBufferBuilder &fbb_; - ::flatbuffers::uoffset_t start_; - void add_ap_ssid(::flatbuffers::Offset<::flatbuffers::String> ap_ssid) { - fbb_.AddOffset(WiFiConfig::VT_AP_SSID, ap_ssid); - } - void add_hostname(::flatbuffers::Offset<::flatbuffers::String> hostname) { - fbb_.AddOffset(WiFiConfig::VT_HOSTNAME, hostname); - } - void add_credentials(::flatbuffers::Offset<::flatbuffers::Vector<::flatbuffers::Offset>> credentials) { - fbb_.AddOffset(WiFiConfig::VT_CREDENTIALS, credentials); - } - void add_ap_password(::flatbuffers::Offset<::flatbuffers::String> ap_password) { - fbb_.AddOffset(WiFiConfig::VT_AP_PASSWORD, ap_password); - } - void add_ap_disabled(bool ap_disabled) { - fbb_.AddElement(WiFiConfig::VT_AP_DISABLED, static_cast(ap_disabled), 0); - } - void add_sta_disabled(bool sta_disabled) { - fbb_.AddElement(WiFiConfig::VT_STA_DISABLED, static_cast(sta_disabled), 0); - } - void add_offline_mode(bool offline_mode) { - fbb_.AddElement(WiFiConfig::VT_OFFLINE_MODE, static_cast(offline_mode), 0); - } - explicit WiFiConfigBuilder(::flatbuffers::FlatBufferBuilder &_fbb) - : fbb_(_fbb) { - start_ = fbb_.StartTable(); - } - ::flatbuffers::Offset Finish() { - const auto end = fbb_.EndTable(start_); - auto o = ::flatbuffers::Offset(end); - return o; - } -}; - -inline ::flatbuffers::Offset CreateWiFiConfig( - ::flatbuffers::FlatBufferBuilder &_fbb, - ::flatbuffers::Offset<::flatbuffers::String> ap_ssid = 0, - ::flatbuffers::Offset<::flatbuffers::String> hostname = 0, - ::flatbuffers::Offset<::flatbuffers::Vector<::flatbuffers::Offset>> credentials = 0, - ::flatbuffers::Offset<::flatbuffers::String> ap_password = 0, - bool ap_disabled = false, - bool sta_disabled = false, - bool offline_mode = false) { - WiFiConfigBuilder builder_(_fbb); - builder_.add_ap_password(ap_password); - builder_.add_credentials(credentials); - builder_.add_hostname(hostname); - builder_.add_ap_ssid(ap_ssid); - builder_.add_offline_mode(offline_mode); - builder_.add_sta_disabled(sta_disabled); - builder_.add_ap_disabled(ap_disabled); - return builder_.Finish(); -} - -struct WiFiConfig::Traits { - using type = WiFiConfig; - static auto constexpr Create = CreateWiFiConfig; -}; - -inline ::flatbuffers::Offset CreateWiFiConfigDirect( - ::flatbuffers::FlatBufferBuilder &_fbb, - const char *ap_ssid = nullptr, - const char *hostname = nullptr, - const std::vector<::flatbuffers::Offset> *credentials = nullptr, - const char *ap_password = nullptr, - bool ap_disabled = false, - bool sta_disabled = false, - bool offline_mode = false) { - auto ap_ssid__ = ap_ssid ? _fbb.CreateString(ap_ssid) : 0; - auto hostname__ = hostname ? _fbb.CreateString(hostname) : 0; - auto credentials__ = credentials ? _fbb.CreateVector<::flatbuffers::Offset>(*credentials) : 0; - auto ap_password__ = ap_password ? _fbb.CreateString(ap_password) : 0; - return OpenShock::Serialization::Configuration::CreateWiFiConfig( - _fbb, - ap_ssid__, - hostname__, - credentials__, - ap_password__, - ap_disabled, - sta_disabled, - offline_mode); -} - -struct CaptivePortalConfig FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { - typedef CaptivePortalConfigBuilder Builder; - struct Traits; - static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() { - return "OpenShock.Serialization.Configuration.CaptivePortalConfig"; - } - enum FlatBuffersVTableOffset FLATBUFFERS_VTABLE_UNDERLYING_TYPE { - VT_ALWAYS_ENABLED = 4 - }; - /// Whether the captive portal is forced to be enabled - /// The captive portal will otherwise shut down when a gateway connection is established - bool always_enabled() const { - return GetField(VT_ALWAYS_ENABLED, 0) != 0; - } - template - bool Verify(::flatbuffers::VerifierTemplate &verifier) const { - return VerifyTableStart(verifier) && - VerifyField(verifier, VT_ALWAYS_ENABLED, 1) && - verifier.EndTable(); - } -}; - -struct CaptivePortalConfigBuilder { - typedef CaptivePortalConfig Table; - ::flatbuffers::FlatBufferBuilder &fbb_; - ::flatbuffers::uoffset_t start_; - void add_always_enabled(bool always_enabled) { - fbb_.AddElement(CaptivePortalConfig::VT_ALWAYS_ENABLED, static_cast(always_enabled), 0); - } - explicit CaptivePortalConfigBuilder(::flatbuffers::FlatBufferBuilder &_fbb) - : fbb_(_fbb) { - start_ = fbb_.StartTable(); - } - ::flatbuffers::Offset Finish() { - const auto end = fbb_.EndTable(start_); - auto o = ::flatbuffers::Offset(end); - return o; - } -}; - -inline ::flatbuffers::Offset CreateCaptivePortalConfig( - ::flatbuffers::FlatBufferBuilder &_fbb, - bool always_enabled = false) { - CaptivePortalConfigBuilder builder_(_fbb); - builder_.add_always_enabled(always_enabled); - return builder_.Finish(); -} - -struct CaptivePortalConfig::Traits { - using type = CaptivePortalConfig; - static auto constexpr Create = CreateCaptivePortalConfig; -}; - -struct BackendConfig FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { - typedef BackendConfigBuilder Builder; - struct Traits; - static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() { - return "OpenShock.Serialization.Configuration.BackendConfig"; - } - enum FlatBuffersVTableOffset FLATBUFFERS_VTABLE_UNDERLYING_TYPE { - VT_DOMAIN = 4, - VT_AUTH_TOKEN = 6 - }; - /// Domain name of the backend server, e.g. "api.shocklink.net" - const ::flatbuffers::String *domain() const { - return GetPointer(VT_DOMAIN); - } - /// Authentication token for the backend server - const ::flatbuffers::String *auth_token() const { - return GetPointer(VT_AUTH_TOKEN); - } - template - bool Verify(::flatbuffers::VerifierTemplate &verifier) const { - return VerifyTableStart(verifier) && - VerifyOffset(verifier, VT_DOMAIN) && - verifier.VerifyString(domain()) && - VerifyOffset(verifier, VT_AUTH_TOKEN) && - verifier.VerifyString(auth_token()) && - verifier.EndTable(); - } -}; - -struct BackendConfigBuilder { - typedef BackendConfig Table; - ::flatbuffers::FlatBufferBuilder &fbb_; - ::flatbuffers::uoffset_t start_; - void add_domain(::flatbuffers::Offset<::flatbuffers::String> domain) { - fbb_.AddOffset(BackendConfig::VT_DOMAIN, domain); - } - void add_auth_token(::flatbuffers::Offset<::flatbuffers::String> auth_token) { - fbb_.AddOffset(BackendConfig::VT_AUTH_TOKEN, auth_token); - } - explicit BackendConfigBuilder(::flatbuffers::FlatBufferBuilder &_fbb) - : fbb_(_fbb) { - start_ = fbb_.StartTable(); - } - ::flatbuffers::Offset Finish() { - const auto end = fbb_.EndTable(start_); - auto o = ::flatbuffers::Offset(end); - return o; - } -}; - -inline ::flatbuffers::Offset CreateBackendConfig( - ::flatbuffers::FlatBufferBuilder &_fbb, - ::flatbuffers::Offset<::flatbuffers::String> domain = 0, - ::flatbuffers::Offset<::flatbuffers::String> auth_token = 0) { - BackendConfigBuilder builder_(_fbb); - builder_.add_auth_token(auth_token); - builder_.add_domain(domain); - return builder_.Finish(); -} - -struct BackendConfig::Traits { - using type = BackendConfig; - static auto constexpr Create = CreateBackendConfig; -}; - -inline ::flatbuffers::Offset CreateBackendConfigDirect( - ::flatbuffers::FlatBufferBuilder &_fbb, - const char *domain = nullptr, - const char *auth_token = nullptr) { - auto domain__ = domain ? _fbb.CreateString(domain) : 0; - auto auth_token__ = auth_token ? _fbb.CreateString(auth_token) : 0; - return OpenShock::Serialization::Configuration::CreateBackendConfig( - _fbb, - domain__, - auth_token__); -} - -struct SerialInputConfig FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { - typedef SerialInputConfigBuilder Builder; - struct Traits; - static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() { - return "OpenShock.Serialization.Configuration.SerialInputConfig"; - } - enum FlatBuffersVTableOffset FLATBUFFERS_VTABLE_UNDERLYING_TYPE { - VT_ECHO_ENABLED = 4 - }; - /// Whether to echo typed characters back to the serial console - bool echo_enabled() const { - return GetField(VT_ECHO_ENABLED, 1) != 0; - } - template - bool Verify(::flatbuffers::VerifierTemplate &verifier) const { - return VerifyTableStart(verifier) && - VerifyField(verifier, VT_ECHO_ENABLED, 1) && - verifier.EndTable(); - } -}; - -struct SerialInputConfigBuilder { - typedef SerialInputConfig Table; - ::flatbuffers::FlatBufferBuilder &fbb_; - ::flatbuffers::uoffset_t start_; - void add_echo_enabled(bool echo_enabled) { - fbb_.AddElement(SerialInputConfig::VT_ECHO_ENABLED, static_cast(echo_enabled), 1); - } - explicit SerialInputConfigBuilder(::flatbuffers::FlatBufferBuilder &_fbb) - : fbb_(_fbb) { - start_ = fbb_.StartTable(); - } - ::flatbuffers::Offset Finish() { - const auto end = fbb_.EndTable(start_); - auto o = ::flatbuffers::Offset(end); - return o; - } -}; - -inline ::flatbuffers::Offset CreateSerialInputConfig( - ::flatbuffers::FlatBufferBuilder &_fbb, - bool echo_enabled = true) { - SerialInputConfigBuilder builder_(_fbb); - builder_.add_echo_enabled(echo_enabled); - return builder_.Finish(); -} - -struct SerialInputConfig::Traits { - using type = SerialInputConfig; - static auto constexpr Create = CreateSerialInputConfig; -}; - -struct OtaUpdateConfig FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { - typedef OtaUpdateConfigBuilder Builder; - struct Traits; - static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() { - return "OpenShock.Serialization.Configuration.OtaUpdateConfig"; - } - enum FlatBuffersVTableOffset FLATBUFFERS_VTABLE_UNDERLYING_TYPE { - VT_IS_ENABLED = 4, - VT_CDN_DOMAIN = 6, - VT_UPDATE_CHANNEL = 8, - VT_CHECK_ON_STARTUP = 10, - VT_CHECK_PERIODICALLY = 12, - VT_CHECK_INTERVAL = 14, - VT_ALLOW_BACKEND_MANAGEMENT = 16, - VT_REQUIRE_MANUAL_APPROVAL = 18, - VT_UPDATE_ID = 20, - VT_UPDATE_STEP = 22 - }; - /// Indicates whether OTA updates are enabled. - bool is_enabled() const { - return GetField(VT_IS_ENABLED, 1) != 0; - } - /// The domain name of the OTA Content Delivery Network (CDN). - const ::flatbuffers::String *cdn_domain() const { - return GetPointer(VT_CDN_DOMAIN); - } - /// The update channel to use. - OpenShock::Serialization::Configuration::OtaUpdateChannel update_channel() const { - return static_cast(GetField(VT_UPDATE_CHANNEL, 0)); - } - /// Indicates whether to check for updates on startup. - bool check_on_startup() const { - return GetField(VT_CHECK_ON_STARTUP, 0) != 0; - } - /// Indicates whether to check for updates periodically. - bool check_periodically() const { - return GetField(VT_CHECK_PERIODICALLY, 0) != 0; - } - /// The interval in minutes between periodic update checks. - uint16_t check_interval() const { - return GetField(VT_CHECK_INTERVAL, 0); - } - /// Indicates if the backend is authorized to manage the hub's update version on behalf of the user. - bool allow_backend_management() const { - return GetField(VT_ALLOW_BACKEND_MANAGEMENT, 1) != 0; - } - /// Indicates if manual approval via serial input or captive portal is required before installing updates. - bool require_manual_approval() const { - return GetField(VT_REQUIRE_MANUAL_APPROVAL, 0) != 0; - } - /// Update process ID, used to track the update process server-side across reboots. - int32_t update_id() const { - return GetField(VT_UPDATE_ID, 0); - } - /// Indicates what step of the update process the hub is currently in, used to detect failed updates for status reporting. - OpenShock::Serialization::Configuration::OtaUpdateStep update_step() const { - return static_cast(GetField(VT_UPDATE_STEP, 0)); - } - template - bool Verify(::flatbuffers::VerifierTemplate &verifier) const { - return VerifyTableStart(verifier) && - VerifyField(verifier, VT_IS_ENABLED, 1) && - VerifyOffset(verifier, VT_CDN_DOMAIN) && - verifier.VerifyString(cdn_domain()) && - VerifyField(verifier, VT_UPDATE_CHANNEL, 1) && - VerifyField(verifier, VT_CHECK_ON_STARTUP, 1) && - VerifyField(verifier, VT_CHECK_PERIODICALLY, 1) && - VerifyField(verifier, VT_CHECK_INTERVAL, 2) && - VerifyField(verifier, VT_ALLOW_BACKEND_MANAGEMENT, 1) && - VerifyField(verifier, VT_REQUIRE_MANUAL_APPROVAL, 1) && - VerifyField(verifier, VT_UPDATE_ID, 4) && - VerifyField(verifier, VT_UPDATE_STEP, 1) && - verifier.EndTable(); - } -}; - -struct OtaUpdateConfigBuilder { - typedef OtaUpdateConfig Table; - ::flatbuffers::FlatBufferBuilder &fbb_; - ::flatbuffers::uoffset_t start_; - void add_is_enabled(bool is_enabled) { - fbb_.AddElement(OtaUpdateConfig::VT_IS_ENABLED, static_cast(is_enabled), 1); - } - void add_cdn_domain(::flatbuffers::Offset<::flatbuffers::String> cdn_domain) { - fbb_.AddOffset(OtaUpdateConfig::VT_CDN_DOMAIN, cdn_domain); - } - void add_update_channel(OpenShock::Serialization::Configuration::OtaUpdateChannel update_channel) { - fbb_.AddElement(OtaUpdateConfig::VT_UPDATE_CHANNEL, static_cast(update_channel), 0); - } - void add_check_on_startup(bool check_on_startup) { - fbb_.AddElement(OtaUpdateConfig::VT_CHECK_ON_STARTUP, static_cast(check_on_startup), 0); - } - void add_check_periodically(bool check_periodically) { - fbb_.AddElement(OtaUpdateConfig::VT_CHECK_PERIODICALLY, static_cast(check_periodically), 0); - } - void add_check_interval(uint16_t check_interval) { - fbb_.AddElement(OtaUpdateConfig::VT_CHECK_INTERVAL, check_interval, 0); - } - void add_allow_backend_management(bool allow_backend_management) { - fbb_.AddElement(OtaUpdateConfig::VT_ALLOW_BACKEND_MANAGEMENT, static_cast(allow_backend_management), 1); - } - void add_require_manual_approval(bool require_manual_approval) { - fbb_.AddElement(OtaUpdateConfig::VT_REQUIRE_MANUAL_APPROVAL, static_cast(require_manual_approval), 0); - } - void add_update_id(int32_t update_id) { - fbb_.AddElement(OtaUpdateConfig::VT_UPDATE_ID, update_id, 0); - } - void add_update_step(OpenShock::Serialization::Configuration::OtaUpdateStep update_step) { - fbb_.AddElement(OtaUpdateConfig::VT_UPDATE_STEP, static_cast(update_step), 0); - } - explicit OtaUpdateConfigBuilder(::flatbuffers::FlatBufferBuilder &_fbb) - : fbb_(_fbb) { - start_ = fbb_.StartTable(); - } - ::flatbuffers::Offset Finish() { - const auto end = fbb_.EndTable(start_); - auto o = ::flatbuffers::Offset(end); - return o; - } -}; - -inline ::flatbuffers::Offset CreateOtaUpdateConfig( - ::flatbuffers::FlatBufferBuilder &_fbb, - bool is_enabled = true, - ::flatbuffers::Offset<::flatbuffers::String> cdn_domain = 0, - OpenShock::Serialization::Configuration::OtaUpdateChannel update_channel = OpenShock::Serialization::Configuration::OtaUpdateChannel::Stable, - bool check_on_startup = false, - bool check_periodically = false, - uint16_t check_interval = 0, - bool allow_backend_management = true, - bool require_manual_approval = false, - int32_t update_id = 0, - OpenShock::Serialization::Configuration::OtaUpdateStep update_step = OpenShock::Serialization::Configuration::OtaUpdateStep::None) { - OtaUpdateConfigBuilder builder_(_fbb); - builder_.add_update_id(update_id); - builder_.add_cdn_domain(cdn_domain); - builder_.add_check_interval(check_interval); - builder_.add_update_step(update_step); - builder_.add_require_manual_approval(require_manual_approval); - builder_.add_allow_backend_management(allow_backend_management); - builder_.add_check_periodically(check_periodically); - builder_.add_check_on_startup(check_on_startup); - builder_.add_update_channel(update_channel); - builder_.add_is_enabled(is_enabled); - return builder_.Finish(); -} - -struct OtaUpdateConfig::Traits { - using type = OtaUpdateConfig; - static auto constexpr Create = CreateOtaUpdateConfig; -}; - -inline ::flatbuffers::Offset CreateOtaUpdateConfigDirect( - ::flatbuffers::FlatBufferBuilder &_fbb, - bool is_enabled = true, - const char *cdn_domain = nullptr, - OpenShock::Serialization::Configuration::OtaUpdateChannel update_channel = OpenShock::Serialization::Configuration::OtaUpdateChannel::Stable, - bool check_on_startup = false, - bool check_periodically = false, - uint16_t check_interval = 0, - bool allow_backend_management = true, - bool require_manual_approval = false, - int32_t update_id = 0, - OpenShock::Serialization::Configuration::OtaUpdateStep update_step = OpenShock::Serialization::Configuration::OtaUpdateStep::None) { - auto cdn_domain__ = cdn_domain ? _fbb.CreateString(cdn_domain) : 0; - return OpenShock::Serialization::Configuration::CreateOtaUpdateConfig( - _fbb, - is_enabled, - cdn_domain__, - update_channel, - check_on_startup, - check_periodically, - check_interval, - allow_backend_management, - require_manual_approval, - update_id, - update_step); -} - -struct LanConfig FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { - typedef LanConfigBuilder Builder; - struct Traits; - static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() { - return "OpenShock.Serialization.Configuration.LanConfig"; - } - enum FlatBuffersVTableOffset FLATBUFFERS_VTABLE_UNDERLYING_TYPE { - VT_API_KEY_ENABLED = 4, - VT_API_KEY = 6 - }; - /// Whether an API key is required for LAN endpoints - bool api_key_enabled() const { - return GetField(VT_API_KEY_ENABLED, 0) != 0; - } - /// API key for authenticating LAN requests (generated or user-set) - const ::flatbuffers::String *api_key() const { - return GetPointer(VT_API_KEY); - } - template - bool Verify(::flatbuffers::VerifierTemplate &verifier) const { - return VerifyTableStart(verifier) && - VerifyField(verifier, VT_API_KEY_ENABLED, 1) && - VerifyOffset(verifier, VT_API_KEY) && - verifier.VerifyString(api_key()) && - verifier.EndTable(); - } -}; - -struct LanConfigBuilder { - typedef LanConfig Table; - ::flatbuffers::FlatBufferBuilder &fbb_; - ::flatbuffers::uoffset_t start_; - void add_api_key_enabled(bool api_key_enabled) { - fbb_.AddElement(LanConfig::VT_API_KEY_ENABLED, static_cast(api_key_enabled), 0); - } - void add_api_key(::flatbuffers::Offset<::flatbuffers::String> api_key) { - fbb_.AddOffset(LanConfig::VT_API_KEY, api_key); - } - explicit LanConfigBuilder(::flatbuffers::FlatBufferBuilder &_fbb) - : fbb_(_fbb) { - start_ = fbb_.StartTable(); - } - ::flatbuffers::Offset Finish() { - const auto end = fbb_.EndTable(start_); - auto o = ::flatbuffers::Offset(end); - return o; - } -}; - -inline ::flatbuffers::Offset CreateLanConfig( - ::flatbuffers::FlatBufferBuilder &_fbb, - bool api_key_enabled = false, - ::flatbuffers::Offset<::flatbuffers::String> api_key = 0) { - LanConfigBuilder builder_(_fbb); - builder_.add_api_key(api_key); - builder_.add_api_key_enabled(api_key_enabled); - return builder_.Finish(); -} - -struct LanConfig::Traits { - using type = LanConfig; - static auto constexpr Create = CreateLanConfig; -}; - -inline ::flatbuffers::Offset CreateLanConfigDirect( - ::flatbuffers::FlatBufferBuilder &_fbb, - bool api_key_enabled = false, - const char *api_key = nullptr) { - auto api_key__ = api_key ? _fbb.CreateString(api_key) : 0; - return OpenShock::Serialization::Configuration::CreateLanConfig( - _fbb, - api_key_enabled, - api_key__); -} - -struct HubConfig FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { - typedef HubConfigBuilder Builder; - struct Traits; - static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() { - return "OpenShock.Serialization.Configuration.HubConfig"; - } - enum FlatBuffersVTableOffset FLATBUFFERS_VTABLE_UNDERLYING_TYPE { - VT_RF = 4, - VT_WIFI = 6, - VT_CAPTIVE_PORTAL = 8, - VT_BACKEND = 10, - VT_SERIAL_INPUT = 12, - VT_OTA_UPDATE = 14, - VT_ESTOP = 16, - VT_LAN = 18 - }; - /// RF Transmitter configuration - const OpenShock::Serialization::Configuration::RFConfig *rf() const { - return GetPointer(VT_RF); - } - /// WiFi configuration - const OpenShock::Serialization::Configuration::WiFiConfig *wifi() const { - return GetPointer(VT_WIFI); - } - /// Captive portal configuration - const OpenShock::Serialization::Configuration::CaptivePortalConfig *captive_portal() const { - return GetPointer(VT_CAPTIVE_PORTAL); - } - /// Backend configuration - const OpenShock::Serialization::Configuration::BackendConfig *backend() const { - return GetPointer(VT_BACKEND); - } - /// Serial input configuration - const OpenShock::Serialization::Configuration::SerialInputConfig *serial_input() const { - return GetPointer(VT_SERIAL_INPUT); - } - /// OTA update configuration - const OpenShock::Serialization::Configuration::OtaUpdateConfig *ota_update() const { - return GetPointer(VT_OTA_UPDATE); - } - /// E-Stop configuration - const OpenShock::Serialization::Configuration::EStopConfig *estop() const { - return GetPointer(VT_ESTOP); - } - /// LAN API configuration - const OpenShock::Serialization::Configuration::LanConfig *lan() const { - return GetPointer(VT_LAN); - } - template - bool Verify(::flatbuffers::VerifierTemplate &verifier) const { - return VerifyTableStart(verifier) && - VerifyOffset(verifier, VT_RF) && - verifier.VerifyTable(rf()) && - VerifyOffset(verifier, VT_WIFI) && - verifier.VerifyTable(wifi()) && - VerifyOffset(verifier, VT_CAPTIVE_PORTAL) && - verifier.VerifyTable(captive_portal()) && - VerifyOffset(verifier, VT_BACKEND) && - verifier.VerifyTable(backend()) && - VerifyOffset(verifier, VT_SERIAL_INPUT) && - verifier.VerifyTable(serial_input()) && - VerifyOffset(verifier, VT_OTA_UPDATE) && - verifier.VerifyTable(ota_update()) && - VerifyOffset(verifier, VT_ESTOP) && - verifier.VerifyTable(estop()) && - VerifyOffset(verifier, VT_LAN) && - verifier.VerifyTable(lan()) && - verifier.EndTable(); - } -}; - -struct HubConfigBuilder { - typedef HubConfig Table; - ::flatbuffers::FlatBufferBuilder &fbb_; - ::flatbuffers::uoffset_t start_; - void add_rf(::flatbuffers::Offset rf) { - fbb_.AddOffset(HubConfig::VT_RF, rf); - } - void add_wifi(::flatbuffers::Offset wifi) { - fbb_.AddOffset(HubConfig::VT_WIFI, wifi); - } - void add_captive_portal(::flatbuffers::Offset captive_portal) { - fbb_.AddOffset(HubConfig::VT_CAPTIVE_PORTAL, captive_portal); - } - void add_backend(::flatbuffers::Offset backend) { - fbb_.AddOffset(HubConfig::VT_BACKEND, backend); - } - void add_serial_input(::flatbuffers::Offset serial_input) { - fbb_.AddOffset(HubConfig::VT_SERIAL_INPUT, serial_input); - } - void add_ota_update(::flatbuffers::Offset ota_update) { - fbb_.AddOffset(HubConfig::VT_OTA_UPDATE, ota_update); - } - void add_estop(::flatbuffers::Offset estop) { - fbb_.AddOffset(HubConfig::VT_ESTOP, estop); - } - void add_lan(::flatbuffers::Offset lan) { - fbb_.AddOffset(HubConfig::VT_LAN, lan); - } - explicit HubConfigBuilder(::flatbuffers::FlatBufferBuilder &_fbb) - : fbb_(_fbb) { - start_ = fbb_.StartTable(); - } - ::flatbuffers::Offset Finish() { - const auto end = fbb_.EndTable(start_); - auto o = ::flatbuffers::Offset(end); - return o; - } -}; - -inline ::flatbuffers::Offset CreateHubConfig( - ::flatbuffers::FlatBufferBuilder &_fbb, - ::flatbuffers::Offset rf = 0, - ::flatbuffers::Offset wifi = 0, - ::flatbuffers::Offset captive_portal = 0, - ::flatbuffers::Offset backend = 0, - ::flatbuffers::Offset serial_input = 0, - ::flatbuffers::Offset ota_update = 0, - ::flatbuffers::Offset estop = 0, - ::flatbuffers::Offset lan = 0) { - HubConfigBuilder builder_(_fbb); - builder_.add_lan(lan); - builder_.add_estop(estop); - builder_.add_ota_update(ota_update); - builder_.add_serial_input(serial_input); - builder_.add_backend(backend); - builder_.add_captive_portal(captive_portal); - builder_.add_wifi(wifi); - builder_.add_rf(rf); - return builder_.Finish(); -} - -struct HubConfig::Traits { - using type = HubConfig; - static auto constexpr Create = CreateHubConfig; -}; - -inline const OpenShock::Serialization::Configuration::HubConfig *GetHubConfig(const void *buf) { - return ::flatbuffers::GetRoot(buf); -} - -inline const OpenShock::Serialization::Configuration::HubConfig *GetSizePrefixedHubConfig(const void *buf) { - return ::flatbuffers::GetSizePrefixedRoot(buf); -} - -template -inline bool VerifyHubConfigBuffer( - ::flatbuffers::VerifierTemplate &verifier) { - return verifier.template VerifyBuffer(nullptr); -} - -template -inline bool VerifySizePrefixedHubConfigBuffer( - ::flatbuffers::VerifierTemplate &verifier) { - return verifier.template VerifySizePrefixedBuffer(nullptr); -} - -inline void FinishHubConfigBuffer( - ::flatbuffers::FlatBufferBuilder &fbb, - ::flatbuffers::Offset root) { - fbb.Finish(root); -} - -inline void FinishSizePrefixedHubConfigBuffer( - ::flatbuffers::FlatBufferBuilder &fbb, - ::flatbuffers::Offset root) { - fbb.FinishSizePrefixed(root); -} - -} // namespace Configuration -} // namespace Serialization -} // namespace OpenShock - -#endif // FLATBUFFERS_GENERATED_HUBCONFIG_OPENSHOCK_SERIALIZATION_CONFIGURATION_H_ diff --git a/components/serialization/include/serialization/_fbs/HubToLocalMessage_generated.h b/components/serialization/include/serialization/_fbs/HubToLocalMessage_generated.h index 728cdb8c0..c12eadcef 100644 --- a/components/serialization/include/serialization/_fbs/HubToLocalMessage_generated.h +++ b/components/serialization/include/serialization/_fbs/HubToLocalMessage_generated.h @@ -13,7 +13,6 @@ static_assert(FLATBUFFERS_VERSION_MAJOR == 25 && FLATBUFFERS_VERSION_REVISION == 19, "Non-compatible flatbuffers version included"); -#include "HubConfig_generated.h" #include "WifiNetwork_generated.h" #include "WifiNetworkEventType_generated.h" #include "WifiScanStatus_generated.h" @@ -22,6 +21,9 @@ namespace OpenShock { namespace Serialization { namespace Local { +struct OtaSettings; +struct OtaSettingsBuilder; + struct ReadyMessage; struct ReadyMessageBuilder; @@ -46,6 +48,39 @@ struct AccountLinkStatusEventBuilder; struct HubToLocalMessage; struct HubToLocalMessageBuilder; +enum class OtaChannel : uint8_t { + Stable = 0, + Beta = 1, + Develop = 2, + MIN = Stable, + MAX = Develop +}; + +inline const OtaChannel (&EnumValuesOtaChannel())[3] { + static const OtaChannel values[] = { + OtaChannel::Stable, + OtaChannel::Beta, + OtaChannel::Develop + }; + return values; +} + +inline const char * const *EnumNamesOtaChannel() { + static const char * const names[4] = { + "Stable", + "Beta", + "Develop", + nullptr + }; + return names; +} + +inline const char *EnumNameOtaChannel(OtaChannel e) { + if (::flatbuffers::IsOutRange(e, OtaChannel::Stable, OtaChannel::Develop)) return ""; + const size_t index = static_cast(e); + return EnumNamesOtaChannel()[index]; +} + enum class HubToLocalMessagePayload : uint8_t { NONE = 0, ReadyMessage = 1, @@ -131,6 +166,128 @@ bool VerifyHubToLocalMessagePayload(::flatbuffers::VerifierTemplate &verifier template bool VerifyHubToLocalMessagePayloadVector(::flatbuffers::VerifierTemplate &verifier, const ::flatbuffers::Vector<::flatbuffers::Offset> *values, const ::flatbuffers::Vector *types); +/// The OTA settings the portal shows and edits. +struct OtaSettings FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { + typedef OtaSettingsBuilder Builder; + struct Traits; + static FLATBUFFERS_CONSTEXPR_CPP11 const char *GetFullyQualifiedName() { + return "OpenShock.Serialization.Local.OtaSettings"; + } + enum FlatBuffersVTableOffset FLATBUFFERS_VTABLE_UNDERLYING_TYPE { + VT_ENABLED = 4, + VT_CDN_DOMAIN = 6, + VT_CHANNEL = 8, + VT_CHECK_INTERVAL = 10, + VT_ALLOW_BACKEND_MANAGEMENT = 12, + VT_REQUIRE_MANUAL_APPROVAL = 14 + }; + bool enabled() const { + return GetField(VT_ENABLED, 0) != 0; + } + const ::flatbuffers::String *cdn_domain() const { + return GetPointer(VT_CDN_DOMAIN); + } + OpenShock::Serialization::Local::OtaChannel channel() const { + return static_cast(GetField(VT_CHANNEL, 0)); + } + uint16_t check_interval() const { + return GetField(VT_CHECK_INTERVAL, 0); + } + bool allow_backend_management() const { + return GetField(VT_ALLOW_BACKEND_MANAGEMENT, 0) != 0; + } + bool require_manual_approval() const { + return GetField(VT_REQUIRE_MANUAL_APPROVAL, 0) != 0; + } + template + bool Verify(::flatbuffers::VerifierTemplate &verifier) const { + return VerifyTableStart(verifier) && + VerifyField(verifier, VT_ENABLED, 1) && + VerifyOffset(verifier, VT_CDN_DOMAIN) && + verifier.VerifyString(cdn_domain()) && + VerifyField(verifier, VT_CHANNEL, 1) && + VerifyField(verifier, VT_CHECK_INTERVAL, 2) && + VerifyField(verifier, VT_ALLOW_BACKEND_MANAGEMENT, 1) && + VerifyField(verifier, VT_REQUIRE_MANUAL_APPROVAL, 1) && + verifier.EndTable(); + } +}; + +struct OtaSettingsBuilder { + typedef OtaSettings Table; + ::flatbuffers::FlatBufferBuilder &fbb_; + ::flatbuffers::uoffset_t start_; + void add_enabled(bool enabled) { + fbb_.AddElement(OtaSettings::VT_ENABLED, static_cast(enabled), 0); + } + void add_cdn_domain(::flatbuffers::Offset<::flatbuffers::String> cdn_domain) { + fbb_.AddOffset(OtaSettings::VT_CDN_DOMAIN, cdn_domain); + } + void add_channel(OpenShock::Serialization::Local::OtaChannel channel) { + fbb_.AddElement(OtaSettings::VT_CHANNEL, static_cast(channel), 0); + } + void add_check_interval(uint16_t check_interval) { + fbb_.AddElement(OtaSettings::VT_CHECK_INTERVAL, check_interval, 0); + } + void add_allow_backend_management(bool allow_backend_management) { + fbb_.AddElement(OtaSettings::VT_ALLOW_BACKEND_MANAGEMENT, static_cast(allow_backend_management), 0); + } + void add_require_manual_approval(bool require_manual_approval) { + fbb_.AddElement(OtaSettings::VT_REQUIRE_MANUAL_APPROVAL, static_cast(require_manual_approval), 0); + } + explicit OtaSettingsBuilder(::flatbuffers::FlatBufferBuilder &_fbb) + : fbb_(_fbb) { + start_ = fbb_.StartTable(); + } + ::flatbuffers::Offset Finish() { + const auto end = fbb_.EndTable(start_); + auto o = ::flatbuffers::Offset(end); + return o; + } +}; + +inline ::flatbuffers::Offset CreateOtaSettings( + ::flatbuffers::FlatBufferBuilder &_fbb, + bool enabled = false, + ::flatbuffers::Offset<::flatbuffers::String> cdn_domain = 0, + OpenShock::Serialization::Local::OtaChannel channel = OpenShock::Serialization::Local::OtaChannel::Stable, + uint16_t check_interval = 0, + bool allow_backend_management = false, + bool require_manual_approval = false) { + OtaSettingsBuilder builder_(_fbb); + builder_.add_cdn_domain(cdn_domain); + builder_.add_check_interval(check_interval); + builder_.add_require_manual_approval(require_manual_approval); + builder_.add_allow_backend_management(allow_backend_management); + builder_.add_channel(channel); + builder_.add_enabled(enabled); + return builder_.Finish(); +} + +struct OtaSettings::Traits { + using type = OtaSettings; + static auto constexpr Create = CreateOtaSettings; +}; + +inline ::flatbuffers::Offset CreateOtaSettingsDirect( + ::flatbuffers::FlatBufferBuilder &_fbb, + bool enabled = false, + const char *cdn_domain = nullptr, + OpenShock::Serialization::Local::OtaChannel channel = OpenShock::Serialization::Local::OtaChannel::Stable, + uint16_t check_interval = 0, + bool allow_backend_management = false, + bool require_manual_approval = false) { + auto cdn_domain__ = cdn_domain ? _fbb.CreateString(cdn_domain) : 0; + return OpenShock::Serialization::Local::CreateOtaSettings( + _fbb, + enabled, + cdn_domain__, + channel, + check_interval, + allow_backend_management, + require_manual_approval); +} + struct ReadyMessage FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { typedef ReadyMessageBuilder Builder; struct Traits; @@ -141,10 +298,14 @@ struct ReadyMessage FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { VT_POGGIES = 4, VT_CONNECTED_WIFI = 6, VT_ACCOUNT_LINKED = 8, - VT_CONFIG = 10, - VT_GPIO_VALID_INPUTS = 12, - VT_GPIO_VALID_OUTPUTS = 14, - VT_HAS_STANDARDIZED_PINS = 16 + VT_GPIO_VALID_INPUTS = 10, + VT_GPIO_VALID_OUTPUTS = 12, + VT_HAS_STANDARDIZED_PINS = 14, + VT_RF_TX_PIN = 16, + VT_ESTOP_ENABLED = 18, + VT_ESTOP_GPIO_PIN = 20, + VT_SAVED_SSIDS = 22, + VT_OTA = 24 }; bool poggies() const { return GetField(VT_POGGIES, 0) != 0; @@ -155,9 +316,6 @@ struct ReadyMessage FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { bool account_linked() const { return GetField(VT_ACCOUNT_LINKED, 0) != 0; } - const OpenShock::Serialization::Configuration::HubConfig *config() const { - return GetPointer(VT_CONFIG); - } const ::flatbuffers::Vector *gpio_valid_inputs() const { return GetPointer *>(VT_GPIO_VALID_INPUTS); } @@ -167,6 +325,22 @@ struct ReadyMessage FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { bool has_standardized_pins() const { return GetField(VT_HAS_STANDARDIZED_PINS, 0) != 0; } + int8_t rf_tx_pin() const { + return GetField(VT_RF_TX_PIN, -1); + } + bool estop_enabled() const { + return GetField(VT_ESTOP_ENABLED, 0) != 0; + } + int8_t estop_gpio_pin() const { + return GetField(VT_ESTOP_GPIO_PIN, -1); + } + /// SSIDs of the saved networks. Passwords never leave the hub. + const ::flatbuffers::Vector<::flatbuffers::Offset<::flatbuffers::String>> *saved_ssids() const { + return GetPointer> *>(VT_SAVED_SSIDS); + } + const OpenShock::Serialization::Local::OtaSettings *ota() const { + return GetPointer(VT_OTA); + } template bool Verify(::flatbuffers::VerifierTemplate &verifier) const { return VerifyTableStart(verifier) && @@ -174,13 +348,19 @@ struct ReadyMessage FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { VerifyOffset(verifier, VT_CONNECTED_WIFI) && verifier.VerifyTable(connected_wifi()) && VerifyField(verifier, VT_ACCOUNT_LINKED, 1) && - VerifyOffset(verifier, VT_CONFIG) && - verifier.VerifyTable(config()) && VerifyOffset(verifier, VT_GPIO_VALID_INPUTS) && verifier.VerifyVector(gpio_valid_inputs()) && VerifyOffset(verifier, VT_GPIO_VALID_OUTPUTS) && verifier.VerifyVector(gpio_valid_outputs()) && VerifyField(verifier, VT_HAS_STANDARDIZED_PINS, 1) && + VerifyField(verifier, VT_RF_TX_PIN, 1) && + VerifyField(verifier, VT_ESTOP_ENABLED, 1) && + VerifyField(verifier, VT_ESTOP_GPIO_PIN, 1) && + VerifyOffset(verifier, VT_SAVED_SSIDS) && + verifier.VerifyVector(saved_ssids()) && + verifier.VerifyVectorOfStrings(saved_ssids()) && + VerifyOffset(verifier, VT_OTA) && + verifier.VerifyTable(ota()) && verifier.EndTable(); } }; @@ -198,9 +378,6 @@ struct ReadyMessageBuilder { void add_account_linked(bool account_linked) { fbb_.AddElement(ReadyMessage::VT_ACCOUNT_LINKED, static_cast(account_linked), 0); } - void add_config(::flatbuffers::Offset config) { - fbb_.AddOffset(ReadyMessage::VT_CONFIG, config); - } void add_gpio_valid_inputs(::flatbuffers::Offset<::flatbuffers::Vector> gpio_valid_inputs) { fbb_.AddOffset(ReadyMessage::VT_GPIO_VALID_INPUTS, gpio_valid_inputs); } @@ -210,6 +387,21 @@ struct ReadyMessageBuilder { void add_has_standardized_pins(bool has_standardized_pins) { fbb_.AddElement(ReadyMessage::VT_HAS_STANDARDIZED_PINS, static_cast(has_standardized_pins), 0); } + void add_rf_tx_pin(int8_t rf_tx_pin) { + fbb_.AddElement(ReadyMessage::VT_RF_TX_PIN, rf_tx_pin, -1); + } + void add_estop_enabled(bool estop_enabled) { + fbb_.AddElement(ReadyMessage::VT_ESTOP_ENABLED, static_cast(estop_enabled), 0); + } + void add_estop_gpio_pin(int8_t estop_gpio_pin) { + fbb_.AddElement(ReadyMessage::VT_ESTOP_GPIO_PIN, estop_gpio_pin, -1); + } + void add_saved_ssids(::flatbuffers::Offset<::flatbuffers::Vector<::flatbuffers::Offset<::flatbuffers::String>>> saved_ssids) { + fbb_.AddOffset(ReadyMessage::VT_SAVED_SSIDS, saved_ssids); + } + void add_ota(::flatbuffers::Offset ota) { + fbb_.AddOffset(ReadyMessage::VT_OTA, ota); + } explicit ReadyMessageBuilder(::flatbuffers::FlatBufferBuilder &_fbb) : fbb_(_fbb) { start_ = fbb_.StartTable(); @@ -226,15 +418,23 @@ inline ::flatbuffers::Offset CreateReadyMessage( bool poggies = false, ::flatbuffers::Offset connected_wifi = 0, bool account_linked = false, - ::flatbuffers::Offset config = 0, ::flatbuffers::Offset<::flatbuffers::Vector> gpio_valid_inputs = 0, ::flatbuffers::Offset<::flatbuffers::Vector> gpio_valid_outputs = 0, - bool has_standardized_pins = false) { + bool has_standardized_pins = false, + int8_t rf_tx_pin = -1, + bool estop_enabled = false, + int8_t estop_gpio_pin = -1, + ::flatbuffers::Offset<::flatbuffers::Vector<::flatbuffers::Offset<::flatbuffers::String>>> saved_ssids = 0, + ::flatbuffers::Offset ota = 0) { ReadyMessageBuilder builder_(_fbb); + builder_.add_ota(ota); + builder_.add_saved_ssids(saved_ssids); builder_.add_gpio_valid_outputs(gpio_valid_outputs); builder_.add_gpio_valid_inputs(gpio_valid_inputs); - builder_.add_config(config); builder_.add_connected_wifi(connected_wifi); + builder_.add_estop_gpio_pin(estop_gpio_pin); + builder_.add_estop_enabled(estop_enabled); + builder_.add_rf_tx_pin(rf_tx_pin); builder_.add_has_standardized_pins(has_standardized_pins); builder_.add_account_linked(account_linked); builder_.add_poggies(poggies); @@ -251,21 +451,30 @@ inline ::flatbuffers::Offset CreateReadyMessageDirect( bool poggies = false, ::flatbuffers::Offset connected_wifi = 0, bool account_linked = false, - ::flatbuffers::Offset config = 0, const std::vector *gpio_valid_inputs = nullptr, const std::vector *gpio_valid_outputs = nullptr, - bool has_standardized_pins = false) { + bool has_standardized_pins = false, + int8_t rf_tx_pin = -1, + bool estop_enabled = false, + int8_t estop_gpio_pin = -1, + const std::vector<::flatbuffers::Offset<::flatbuffers::String>> *saved_ssids = nullptr, + ::flatbuffers::Offset ota = 0) { auto gpio_valid_inputs__ = gpio_valid_inputs ? _fbb.CreateVector(*gpio_valid_inputs) : 0; auto gpio_valid_outputs__ = gpio_valid_outputs ? _fbb.CreateVector(*gpio_valid_outputs) : 0; + auto saved_ssids__ = saved_ssids ? _fbb.CreateVector<::flatbuffers::Offset<::flatbuffers::String>>(*saved_ssids) : 0; return OpenShock::Serialization::Local::CreateReadyMessage( _fbb, poggies, connected_wifi, account_linked, - config, gpio_valid_inputs__, gpio_valid_outputs__, - has_standardized_pins); + has_standardized_pins, + rf_tx_pin, + estop_enabled, + estop_gpio_pin, + saved_ssids__, + ota); } struct ErrorMessage FLATBUFFERS_FINAL_CLASS : private ::flatbuffers::Table { diff --git a/components/www/CMakeLists.txt b/components/www/CMakeLists.txt index 1aaf0eb9d..7612e5f67 100644 --- a/components/www/CMakeLists.txt +++ b/components/www/CMakeLists.txt @@ -1,23 +1,17 @@ # The `www` image: the captive-portal assets as one flat, read-only directory, mapped from the `www` -# partition and served by pointer. Replaces the littlefs `static0` image: three files that are written -# whole by OTA and then only read need no filesystem, and a flat image reads straight from the flash -# cache instead of through littlefs' callback buffers. +# partition and served by pointer: files that are written whole by OTA and then only read need no +# filesystem, and a flat image reads straight from the flash cache. # # include/www/WwwImage.h the format: a parser/validator over a byte span, pure C++ (host-tested) # include/www/WwwPartition.h the device side: find the partition, map it, verify, serve -# include/www/Migration.h writing the image over the 1.x static0 and retyping it as www (run by components/migration) # # scripts/utils/wwwimage.py writes the format; change both together. idf_component_register( - SRCS "src/Migration.cpp" - "src/WwwImage.cpp" + SRCS "src/WwwImage.cpp" "src/WwwPartition.cpp" INCLUDE_DIRS "include" REQUIRES common # OpenShock.h esp_partition # esp_partition_t in WwwPartition.h PRIV_REQUIRES crypto logging # SHA256 for the image hash; OS_LOG* spi_flash # esp_partition_mmap - hwutil partition_layout # the migration: FindStaticPartition/FlashPartitionFromUrl, retyping static0 - app_update # esp_ota_get_next_update_partition staging slot - esp_system # esp_restart ) diff --git a/components/www/host_test/main/test_www_image.cpp b/components/www/host_test/main/test_www_image.cpp index 43783775b..dbad45263 100644 --- a/components/www/host_test/main/test_www_image.cpp +++ b/components/www/host_test/main/test_www_image.cpp @@ -191,9 +191,8 @@ TEST_CASE("Only bytes that begin with the magic start like an image", "[www]") TEST_ASSERT_FALSE(Www::StartsLikeImage(std::span(bytes).first(3))); TEST_ASSERT_FALSE(Www::StartsLikeImage({})); - // Shaped like the start of a littlefs image: a revision count, a metadata tag, then the "littlefs" name - const uint8_t littlefs[] = {0x01, 0x00, 0x00, 0x00, 0xF0, 0x0F, 0xFF, 0xF7, 'l', 'i', 't', 't', 'l', 'e', 'f', 's'}; - TEST_ASSERT_FALSE(Www::StartsLikeImage(littlefs)); + const uint8_t other[] = {0x01, 0x00, 0x00, 0x00, 0xF0, 0x0F, 0xFF, 0xF7}; + TEST_ASSERT_FALSE(Www::StartsLikeImage(other)); } TEST_CASE("Every truncation is rejected", "[www]") diff --git a/components/www/include/www/Migration.h b/components/www/include/www/Migration.h deleted file mode 100644 index e381a23ee..000000000 --- a/components/www/include/www/Migration.h +++ /dev/null @@ -1,31 +0,0 @@ -#pragma once - -// The www half of the 1.x -> 2.x storage migration (docs/STORAGE.md): write the www image over the 1.x littlefs -// `static0` partition and retype it as the `www` partition, in place. Run by the storage migration task -// (migration/StorageMigration.h). - -#include -#include - -namespace OpenShock::HTTP { - class Client; -} - -namespace OpenShock::Www { - enum class MigrationResult { - NotNeeded, // the table already has `www` - Refused, // nothing was touched: no 1.x littlefs `static0` partition, or the table could not be read - Failed, // the image did not download, verify, or copy: static0 still has the 1.x assets unless the final copy - // failed - }; - - /// @brief Downloads the www image at `url` into the inactive OTA slot, checks it against `sha256` (the release's - /// `staticfs` artifact hash) and that it parses, then copies it over `static0`, retypes that as `www`, and - /// restarts, whether the table write went through or not (PartitionTable::Replace). Returns only when it did - /// not restart. - /// - /// Requires the captive portal to be closed (it serves from static0) and no OTA install to be running (it writes - /// static0 too). A power cut between the erase and the write of the table sector bricks the hub until the table is - /// reflashed over USB (docs/STORAGE.md, Recovery). - MigrationResult MigrateToWwwPartition(HTTP::Client& client, std::string_view url, const uint8_t (&sha256)[32]); -} // namespace OpenShock::Www diff --git a/components/www/include/www/WwwImage.h b/components/www/include/www/WwwImage.h index 2eb6d2ae3..7a5f7c59c 100644 --- a/components/www/include/www/WwwImage.h +++ b/components/www/include/www/WwwImage.h @@ -40,6 +40,7 @@ namespace OpenShock::Www { inline constexpr std::size_t kHeaderSize = 64; inline constexpr std::size_t kNodeSize = 16; inline constexpr std::size_t kBodyHashSize = 32; + inline constexpr std::size_t kHashOffset = 12; // body_hash in the header inline constexpr std::size_t kAlign = 4; inline constexpr uint16_t kFlagDirectory = 1u << 0; @@ -82,7 +83,7 @@ namespace OpenShock::Www { }; [[nodiscard]] const char* ParseErrorName(ParseError error); - /// @brief Whether `bytes` begins with the image magic: enough to tell a www image from a littlefs one in the first + /// @brief Whether `bytes` begins with the image magic: enough to tell a www image from anything else in the first /// chunk of a download, before anything is erased. Not a validation; Parse() is. [[nodiscard]] bool StartsLikeImage(std::span bytes); @@ -101,7 +102,7 @@ namespace OpenShock::Www { /// @brief Bytes from the header to the end of the last body's padding: what an OTA hash of the image covers. [[nodiscard]] std::size_t ImageSize() const { return m_size; } /// @brief The image's own hash; it changes whenever any byte of the image does. - [[nodiscard]] std::span BodyHash() const { return m_bytes.subspan<12, kBodyHashSize>(); } + [[nodiscard]] std::span BodyHash() const { return m_bytes.subspan(); } [[nodiscard]] std::span Bytes() const { return m_bytes.first(m_size); } private: diff --git a/components/www/include/www/WwwPartition.h b/components/www/include/www/WwwPartition.h index 3a79ed9a1..ac8e4edb3 100644 --- a/components/www/include/www/WwwPartition.h +++ b/components/www/include/www/WwwPartition.h @@ -14,11 +14,7 @@ #include namespace OpenShock::Www { - inline constexpr esp_partition_type_t kPartitionType = static_cast(0x40); - inline constexpr esp_partition_subtype_t kPartitionSubtype = static_cast(0x00); - inline constexpr const char* kPartitionLabel = "www"; - - /// @brief The `www` partition of the running partition table, or null on a table without one (the 1.x layout). + /// @brief The `www` partition of the running partition table, or null on a table without one. [[nodiscard]] const esp_partition_t* FindPartition(); /// @brief Whether `partition` holds a valid image; what the OTA path runs on a freshly flashed partition. diff --git a/components/www/src/Migration.cpp b/components/www/src/Migration.cpp deleted file mode 100644 index 1a92b2803..000000000 --- a/components/www/src/Migration.cpp +++ /dev/null @@ -1,167 +0,0 @@ -#include "www/Migration.h" - -const char* const TAG = "WwwMigration"; - -#include "hwutil/PartitionUtils.h" -#include "Logging.h" -#include "partition_layout/LiveTable.h" -#include "partition_layout/PartitionTable.h" -#include "www/WwwImage.h" -#include "www/WwwPartition.h" - -#include -#include -#include - -#include -#include -#include -#include - -using namespace OpenShock; -namespace PT = OpenShock::PartitionTable; - -namespace { - constexpr const char* kStaticLabel = "static0"; // the 1.x entry the www partition takes over - constexpr std::size_t kCopyBufferSize = 4096; - constexpr std::size_t kEraseStep = 64 * 1024; - - bool copyImage(const esp_partition_t* source, const esp_partition_t* destination, std::size_t size) - { - std::array buffer; - std::size_t erasedTo = 0; - for (std::size_t offset = 0; offset < size;) { - const std::size_t length = std::min(buffer.size(), size - offset); - while (erasedTo < offset + length) { - const std::size_t eraseLength = std::min(kEraseStep, static_cast(destination->size) - erasedTo); - if (eraseLength == 0 || esp_partition_erase_range(destination, erasedTo, eraseLength) != ESP_OK) { - OS_LOGE(TAG, "Failed to erase static0 at offset %zu", erasedTo); - return false; - } - erasedTo += eraseLength; - } - if (esp_partition_read(source, offset, buffer.data(), length) != ESP_OK) { - OS_LOGE(TAG, "Failed to read the staged www image"); - return false; - } - if (esp_partition_write(destination, offset, buffer.data(), length) != ESP_OK) { - OS_LOGE(TAG, "Failed to write static0"); - return false; - } - offset += length; - } - return true; - } - - // Downloads into the inactive app slot first. Its contents are disposable once this firmware has booted, while - // static0 still holds the 1.x portal until the supplied image has passed both integrity checks. - bool downloadInto(HTTP::Client& client, const esp_partition_t* staticPartition, std::string_view url, const uint8_t (&hash)[32]) - { - const esp_partition_t* staging = esp_ota_get_next_update_partition(nullptr); - if (staging == nullptr) { - OS_LOGE(TAG, "No inactive OTA partition is available to stage the www image"); - return false; - } - - auto noProgress = [](std::size_t, std::size_t, float) -> bool { - return true; - }; - auto startsLikeWwwImage = [](std::span chunk) -> bool { - if (!Www::StartsLikeImage(chunk)) { - OS_LOGE(TAG, "The download is not a www image"); - return false; - } - return true; - }; - if (!FlashPartitionFromUrl(client, staging, url, hash, noProgress, startsLikeWwwImage)) { - OS_LOGE(TAG, "Failed to download the www image into the staging partition"); - return false; - } - - Www::Partition staged; - if (!staged.mount(staging)) { - OS_LOGE(TAG, "The staged www image does not parse"); - return false; - } - const std::size_t size = staged.image().ImageSize(); - if (size > staticPartition->size) { - OS_LOGE(TAG, "The staged www image is too large for static0"); - return false; - } - - std::array stagedDigest; - if (!Www::ImageSha256(staging, stagedDigest) || memcmp(stagedDigest.data(), hash, 32) != 0) { - OS_LOGE(TAG, "The staged www image does not match the supplied hash"); - return false; - } - - OS_LOGI(TAG, "Writing the verified www image over static0 (0x%06x)", static_cast(staticPartition->address)); - if (!copyImage(staging, staticPartition, size)) { - return false; - } - - std::array digest; - if (!Www::ImageSha256(staticPartition, digest) || memcmp(digest.data(), hash, 32) != 0) { - OS_LOGE(TAG, "The www image does not read back as written"); - return false; - } - return true; - } -} // namespace - -Www::MigrationResult Www::MigrateToWwwPartition(HTTP::Client& client, std::string_view url, const uint8_t (&sha256)[32]) -{ - // Two raw tables (3 KiB each) and a parsed one (some 3.4 KiB) come to close to 9.5 KiB: on the heap, not the - // caller's stack - struct Tables { - PT::Bytes live; - PT::Bytes next; - PT::Table parsed; - }; - auto tables = std::make_unique(); - - if (!PT::ReadLiveBytes(tables->live) || !PT::Parse(tables->live, tables->parsed)) { - OS_LOGE(TAG, "Cannot read the partition table; not migrating"); - return MigrationResult::Refused; - } - if (tables->parsed.Find(kPartitionLabel) != nullptr) { - return MigrationResult::NotNeeded; - } - const PT::Entry* entry = tables->parsed.Find(kStaticLabel); - if (entry == nullptr || entry->type != PT::kTypeData || entry->subtype != PT::kSubtypeDataSpiffs) { - OS_LOGE(TAG, "No 1.x littlefs static0 partition; not migrating"); - return MigrationResult::Refused; - } - const esp_partition_t* staticPartition = FindStaticPartition(); - if (staticPartition == nullptr || staticPartition->address != entry->offset || staticPartition->size != entry->size) { - OS_LOGE(TAG, "The running partition table does not match the one on flash; not migrating"); - return MigrationResult::Refused; - } - - tables->next = tables->live; - if (auto retyped = PT::Retype(tables->next, kStaticLabel, static_cast(kPartitionType), static_cast(kPartitionSubtype), kPartitionLabel); !retyped) { - OS_LOGE(TAG, "Cannot retype the static0 partition: %s", PT::RetypeErrorName(retyped.error())); - return MigrationResult::Refused; - } - - // static0 is not touched until the image has downloaded and verified in the inactive OTA partition. - if (!downloadInto(client, staticPartition, url, sha256)) { - return MigrationResult::Failed; - } - - OS_LOGW(TAG, "Retyping the static0 partition as www. Do not unplug the hub."); - switch (PT::Replace(tables->next, tables->live)) { - case PT::ReplaceResult::Written: - OS_LOGI(TAG, "Partition table rewritten; restarting into the www partition"); - break; - case PT::ReplaceResult::Restored: - OS_LOGE(TAG, "The partition table is the old one again; restarting on it"); - break; - case PT::ReplaceResult::Untouched: - OS_LOGE(TAG, "The partition table was not touched; restarting on it"); - break; - } - // The running firmware keeps the table it loaded at boot: a restart is the one clean way onto the new one - esp_partition_unload_all(); - esp_restart(); -} diff --git a/components/www/src/WwwImage.cpp b/components/www/src/WwwImage.cpp index 7b1819801..33cf319de 100644 --- a/components/www/src/WwwImage.cpp +++ b/components/www/src/WwwImage.cpp @@ -17,7 +17,6 @@ namespace { { return static_cast(p[0]) | (static_cast(p[1]) << 8) | (static_cast(p[2]) << 16) | (static_cast(p[3]) << 24); } - constexpr std::size_t kHashOffset = 12; // body_hash in the header std::size_t padded(std::size_t size) { diff --git a/components/www/src/WwwPartition.cpp b/components/www/src/WwwPartition.cpp index 84a882efd..86613fc83 100644 --- a/components/www/src/WwwPartition.cpp +++ b/components/www/src/WwwPartition.cpp @@ -9,6 +9,10 @@ const char* const TAG = "WwwPartition"; using namespace OpenShock; +static constexpr esp_partition_type_t kPartitionType = static_cast(0x40); +static constexpr esp_partition_subtype_t kPartitionSubtype = static_cast(0x00); +static constexpr const char* kPartitionLabel = "www"; + const esp_partition_t* Www::FindPartition() { return esp_partition_find_first(kPartitionType, kPartitionSubtype, kPartitionLabel); diff --git a/docs/STORAGE.md b/docs/STORAGE.md index a8d670b17..85aed0f4e 100644 --- a/docs/STORAGE.md +++ b/docs/STORAGE.md @@ -1,32 +1,31 @@ # Storage and partition layouts -How the firmware lays out flash, where it keeps its config and the captive portal's assets, and how a 1.x hub's -littlefs `config` and `static0` partitions become the NVS config store and the `www` image partition in place. The -reasoning behind each choice is in the design document (OpenShock Firmware 2.0: Storage and Partition Migration -Design); this page describes what the code does. +How the firmware lays out flash and where it keeps its config and the captive portal's assets. ## Partition tables One table, `partitions/ota_4mb.csv`, for every board whatever its flash size. Board files never name it. -| Partition | Offset | Size | 1.x (`partitions/legacy/ota_4mb_v1.csv`) | -| --- | --- | --- | --- | -| `nvs` | 0x9000 | 20 KiB | the same | -| `otadata` | 0xE000 | 8 KiB | the same | -| `app0`, `app1` | 0x10000, 0x1B0000 | 1,664 KiB each | the same | -| `cfg` (data/nvs) | 0x350000 | 12 KiB | littlefs `config` (data/spiffs) | -| `www` (type 0x40) | 0x353000 | 628 KiB | littlefs `static0` (data/spiffs) | -| `coredump` | 0x3F0000 | 64 KiB | the same | +| Partition | Offset | Size | +| --- | --- | --- | +| `nvs` | 0x9000 | 20 KiB | +| `otadata` | 0xE000 | 8 KiB | +| `app0`, `app1` | 0x10000, 0x1B0000 | 1,664 KiB each | +| `cfg` (data/nvs) | 0x350000 | 12 KiB | +| `www` (type 0x40) | 0x353000 | 628 KiB | +| `coredump` | 0x3F0000 | 64 KiB | + +The flash past 4 MB on bigger boards stays unused. -It is the 1.x geometry entry for entry: a fielded hub rewrites its table in place and nothing moves. Only `config` and -`static0` change, in type and label, each by its own migration; either can run first. The flash past 4 MB on bigger -boards stays unused. +This firmware only runs on this table. A hub on the 1.x table reaches it through the migration firmware on the +`migration/firmware-2.0` branch, which is the only place that code lives; only after that can it install a release from +here. Going back is not supported: the migration firmware treats a store without its `root/schema` marker as empty, so +a hub set up on this firmware would lose its config. ## Config The config lives in the `cfg` NVS partition as typed keys, one namespace per section. At 12 KiB (three NVS pages, one -always kept free for compaction) it holds the whole config with some 32 long WiFi credentials, more than the 4 KiB -1.x config file did; it fills up at about 36. The config component +always kept free for compaction) it holds the whole config with some 32 long WiFi credentials; it fills up at about 36. The config component (`components/config`) is only the framework; each module owns its section, its keys and its accessors: | Section | Owner | NVS namespaces | @@ -39,52 +38,28 @@ always kept free for compaction) it holds the whole config with some 32 long WiF | `otaUpdate` | core (`OtaUpdateConfig.h`) | `ota` | | `estop` | device_control (`estop/EStopConfig.h`) | `estop` | -The store's own namespace is `root`, with `schema` (u8, 1) marking a filled store. Namespace and key names are at +Namespace and key names are at most 15 characters and are permanent once hubs hold them: renaming one is a migration, not an edit. A missing key -reads as its default and an unknown key is ignored, so adding or removing a field never changes the schema. +reads as its default and an unknown key is ignored, so adding or removing a field needs no migration. To add a section, define a value type whose default constructor is the defaults, with `Normalize` (its validation, run on every value the section takes in), `LoadNvs`, `SaveNvs`, `FromJSON` (opened with `Parse::ObjectOrDefault`, -booleans through `Parse::JsonBool`), `ToJSON` and `ToFlatbuffers`, and one -`Config::Section` for it in the module (the `SectionValue` concept in `config/Section.h` is the contract). It -joins `jsonconfig`, `rawconfig`, the local protocol's Ready message and factory reset through the section registry. +booleans through `Parse::JsonBool`) and `ToJSON`, and one `Config::Section` for it in the module (the +`SectionValue` concept in `config/Section.h` is the contract). It joins `jsonconfig` and factory reset through the +section registry. The config has no FlatBuffers form: the portal's Ready message carries only the settings it shows. A load opens its namespace with `Nvs::Handle::OpenForLoad`: a namespace never written reads every key as its default, an open that failed fails the load, and a failed load runs the section on its defaults without trusting flash. A save goes through `Nvs::Saver`, which writes only the members that differ from the previous value. Where the store is: -- `cfg` once the table has it. On the first boot after the table rewrite `cfg` still holds the 1.x littlefs; it is - erased before NVS opens it. -- On the 1.x table, which has no `cfg`, the same namespaces in the shared `nvs` partition. The first boot that finds - `cfg` copies them over key by key, compares the copy, and deletes the shared one. -- An empty store is filled once: from the 1.x littlefs config file (`/config` on the 1.x `config` partition, read - only, so a rollback to 1.x still finds it until the config partition migration below has run) when the partition - mounts and the file reads as a valid config, else from every section's defaults (a 1.x file that is missing, - unreadable or invalid is logged and the defaults are written). The store is marked filled only once every section - reads back from flash as written; a fill that failed part-way is redone on the next boot. -- Without any usable NVS, the hub runs on defaults in RAM and logs it. +- `cfg`. An empty store reads as the defaults; nothing is written until a value changes, so a value left at its + default follows the firmware's default. +- Without a usable `cfg`, the hub runs on defaults in RAM and logs it. The shared `nvs` belongs to ESP-IDF (PHY calibration cache) and may be erased at any time; nothing of OpenShock's is -kept there once the copy into `cfg` has been made and verified. A copy that fails to write or to read back is undone -and the config stays in the shared `nvs`, to be copied again on the next boot. Until then an erased shared `nvs` is an -empty store: the hub imports the 1.x file again and loses what changed since the update. The WiFi driver's own NVS store is compiled out (`CONFIG_ESP_WIFI_NVS_ENABLED=n`). -`sysinfo` on the serial console prints where the config is. - -## The config partition migration - -`Config::MigrateToCfgPartition()` (`config/Migration.h`) moves a 1.x hub onto `cfg`. The storage migration task (below) -runs it first, right after setup. - -1. It refuses unless the live table has the 1.x littlefs `config` partition and no `cfg`, and the config is in the - shared `nvs` (where a 1.x hub keeps it after its first boot on this firmware, imported from the 1.x file). -2. It retypes `config` as `cfg` (data/nvs) in a copy of the live table and re-seals the MD5 - (`components/partition_layout`). Every offset and size stays, so the bootloader that boots the 1.x table boots it. -3. Config writes are held, the WiFi driver's stale `nvs.net80211` store is dropped, NVS is closed, and the table sector - is erased and written through the OTA path and read back. A write that will not stick puts the old table back; the - hub never restarts without a table that parses. -4. The hub restarts; `Store` erases the littlefs left in `cfg` (the 1.x config file with it: from here a downgrade to - 1.x starts from defaults) and moves the config in. +kept there. The WiFi driver's own NVS store is compiled out (`CONFIG_ESP_WIFI_NVS_ENABLED=n`). `sysinfo` on the serial +console prints where the config is. ## Portal assets @@ -96,54 +71,4 @@ served. The same image serves every board. The OTA update skips flashing it when the hash on flash already matches the release's. -Releases from 2.0.0 publish the image as `www.bin` under the repository server's `staticfs` artifact. A 1.x-numbered -release (the 1.9.x pivot that every fielded hub installs first) publishes the littlefs `staticfs.bin` there instead, -packed from the same assets by `gen_staticfs.py --legacy-output`: a 1.x hub installs an update with its own OTA code, -which writes `staticfs` over its littlefs `static0` and aborts unless that then mounts. `scripts/repo_server_publish.py` -picks the file by the release's major version. - -On the 1.x table, which has no `www`, this firmware serves the portal from the littlefs `static0` and its OTA update -flashes `static0` and verifies it by mounting, as the 1.x firmware does. That holds until the www migration below -runs, whichever of the two migrations goes first: the config migration above only retypes `config` as `cfg` and -leaves `static0` as it is. - -## The www partition migration - -`Www::MigrateToWwwPartition()` (`www/Migration.h`) moves a 1.x hub onto `www`. The storage migration task (below) -runs it once `cfg` exists and the hub is online, with the portal closed and no OTA install running. - -1. It refuses unless the live table has the 1.x littlefs `static0` partition and no `www`. -2. It retypes `static0` as `www` (type 0x40) in a copy of the live table and re-seals the MD5 - (`components/partition_layout`). Every offset and size stays, so the bootloader that boots the 1.x table boots it. -3. It downloads the release's www image into the inactive OTA app slot, checks its SHA-256 against the published one, - and checks that it parses and hashes the same. Only then does it copy the image over `static0` and verify the - copied bytes, so a failed download or invalid image leaves the 1.x assets intact. -4. The table sector is erased and written through the OTA path and read back. A write that will not stick puts the - old table back; the hub never restarts without a table that parses. -5. The hub restarts and the portal serves from `www`. - -## The storage migration task - -`Migration::Start()` (`migration/StorageMigration.h`, called from `app_main` after setup) reads the live table and, -on a hub that still has a 1.x `config` or `static0`, starts one task that runs the two migrations above in order: -the config partition first (no network needed), then, on the next boot and once WiFi is connected, the www -partition. Each step takes the OTA manager's flash-write lock for its whole run, so an install never writes `static0` -or runs while the table changes, and backs off while an install is running. A step that returns without restarting -is retried after ten minutes; `migrate now` over serial retries at once. - -The www image is the running release's `staticfs` artifact, which from 2.0.0 is `www.bin`. A build with no such -release (a dev build, a 1.x-numbered one) logs that and waits for `migrate www ` over the serial -console, which downloads that image instead (plain `http://` works, for a server on the LAN). `migrate` alone says -whether anything is still pending. On the 2.x table nothing starts. - -## Recovery - -The table sector's erase-to-write window (one sector, well under a second, once per hub) is the one step a power cut -cannot recover from: the bootloader then finds no partition table. The app, config and assets are still on flash, so -writing the board's table restores the hub with its config: - -```sh -esptool write_flash 0x8000 partition-table.bin # this board's, from its release -``` - -Flashing the merged image for the board at 0x0 also works but erases the config. +Releases publish the image as `www.bin` under the repository server's `staticfs` artifact. diff --git a/frontend/src/lib/MessageHandlers/WifiNetworkEventHandler.ts b/frontend/src/lib/MessageHandlers/WifiNetworkEventHandler.ts index 1ec784dd8..d01595ee6 100644 --- a/frontend/src/lib/MessageHandlers/WifiNetworkEventHandler.ts +++ b/frontend/src/lib/MessageHandlers/WifiNetworkEventHandler.ts @@ -68,14 +68,11 @@ function handleSavedEvent(fbsNetwork: FbsWifiNetwork) { // Credentials are per SSID, so every scanned BSSID of it is now saved hubState.markWifiNetworksSaved(ssid); - // Update config credentials so savedOnlySSIDs stays in sync - if (hubState.config && !hubState.config.wifi.credentials.some((c) => c.ssid === ssid)) { + // Keep the saved SSIDs in sync for savedOnlySSIDs + if (hubState.config && !hubState.config.savedSSIDs.includes(ssid)) { hubState.config = { ...hubState.config, - wifi: { - ...hubState.config.wifi, - credentials: [...hubState.config.wifi.credentials, { id: 0, ssid, password: null }], - }, + savedSSIDs: [...hubState.config.savedSSIDs, ssid], }; } @@ -91,14 +88,11 @@ function handleRemovedEvent(fbsNetwork: FbsWifiNetwork) { // Credentials are per SSID, and the event may carry a zeroed BSSID if the network wasn't scanned hubState.markWifiNetworksUnsaved(ssid); - // Update config credentials so savedOnlySSIDs stays in sync + // Keep the saved SSIDs in sync for savedOnlySSIDs if (hubState.config) { hubState.config = { ...hubState.config, - wifi: { - ...hubState.config.wifi, - credentials: hubState.config.wifi.credentials.filter((c) => c.ssid !== ssid), - }, + savedSSIDs: hubState.config.savedSSIDs.filter((saved) => saved !== ssid), }; } diff --git a/frontend/src/lib/MessageHandlers/index.ts b/frontend/src/lib/MessageHandlers/index.ts index 64f851989..698351a8f 100644 --- a/frontend/src/lib/MessageHandlers/index.ts +++ b/frontend/src/lib/MessageHandlers/index.ts @@ -41,7 +41,7 @@ PayloadHandlers[HubToLocalMessagePayload.ReadyMessage] = (cli, msg) => { : null ); hubState.accountLinked = payload.accountLinked(); - hubState.config = mapConfig(payload.config()); + hubState.config = mapConfig(payload); const gpioValidInputs = payload.gpioValidInputsArray(); if (gpioValidInputs) { diff --git a/frontend/src/lib/_fbs/open-shock/serialization/configuration.ts b/frontend/src/lib/_fbs/open-shock/serialization/configuration.ts deleted file mode 100644 index 2587fdbca..000000000 --- a/frontend/src/lib/_fbs/open-shock/serialization/configuration.ts +++ /dev/null @@ -1,17 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -/* eslint-disable @typescript-eslint/no-unused-vars, @typescript-eslint/no-explicit-any, @typescript-eslint/no-non-null-assertion */ - -export { BackendConfig } from './configuration/backend-config'; -export { CaptivePortalConfig } from './configuration/captive-portal-config'; -export { EStopConfig } from './configuration/estop-config'; -export { HubConfig } from './configuration/hub-config'; -export { LanConfig } from './configuration/lan-config'; -export { MacAddress } from './configuration/mac-address'; -export { OtaUpdateChannel } from './configuration/ota-update-channel'; -export { OtaUpdateConfig } from './configuration/ota-update-config'; -export { OtaUpdateStep } from './configuration/ota-update-step'; -export { RFConfig } from './configuration/rfconfig'; -export { SerialInputConfig } from './configuration/serial-input-config'; -export { WiFiConfig } from './configuration/wi-fi-config'; -export { WiFiCredentials } from './configuration/wi-fi-credentials'; diff --git a/frontend/src/lib/_fbs/open-shock/serialization/configuration/backend-config.ts b/frontend/src/lib/_fbs/open-shock/serialization/configuration/backend-config.ts deleted file mode 100644 index cb3205c94..000000000 --- a/frontend/src/lib/_fbs/open-shock/serialization/configuration/backend-config.ts +++ /dev/null @@ -1,68 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -/* eslint-disable @typescript-eslint/no-unused-vars, @typescript-eslint/no-explicit-any, @typescript-eslint/no-non-null-assertion */ - -import * as flatbuffers from 'flatbuffers'; - -export class BackendConfig { - bb: flatbuffers.ByteBuffer|null = null; - bb_pos = 0; - __init(i:number, bb:flatbuffers.ByteBuffer):BackendConfig { - this.bb_pos = i; - this.bb = bb; - return this; -} - -static getRootAsBackendConfig(bb:flatbuffers.ByteBuffer, obj?:BackendConfig):BackendConfig { - return (obj || new BackendConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -static getSizePrefixedRootAsBackendConfig(bb:flatbuffers.ByteBuffer, obj?:BackendConfig):BackendConfig { - bb.setPosition(bb.position() + flatbuffers.SIZE_PREFIX_LENGTH); - return (obj || new BackendConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -/** - * Domain name of the backend server, e.g. "api.shocklink.net" - */ -domain():string|null -domain(optionalEncoding:flatbuffers.Encoding):string|Uint8Array|null -domain(optionalEncoding?:any):string|Uint8Array|null { - const offset = this.bb!.__offset(this.bb_pos, 4); - return offset ? this.bb!.__string(this.bb_pos + offset, optionalEncoding) : null; -} - -/** - * Authentication token for the backend server - */ -authToken():string|null -authToken(optionalEncoding:flatbuffers.Encoding):string|Uint8Array|null -authToken(optionalEncoding?:any):string|Uint8Array|null { - const offset = this.bb!.__offset(this.bb_pos, 6); - return offset ? this.bb!.__string(this.bb_pos + offset, optionalEncoding) : null; -} - -static startBackendConfig(builder:flatbuffers.Builder) { - builder.startObject(3); -} - -static addDomain(builder:flatbuffers.Builder, domainOffset:flatbuffers.Offset) { - builder.addFieldOffset(0, domainOffset, 0); -} - -static addAuthToken(builder:flatbuffers.Builder, authTokenOffset:flatbuffers.Offset) { - builder.addFieldOffset(1, authTokenOffset, 0); -} - -static endBackendConfig(builder:flatbuffers.Builder):flatbuffers.Offset { - const offset = builder.endObject(); - return offset; -} - -static createBackendConfig(builder:flatbuffers.Builder, domainOffset:flatbuffers.Offset, authTokenOffset:flatbuffers.Offset):flatbuffers.Offset { - BackendConfig.startBackendConfig(builder); - BackendConfig.addDomain(builder, domainOffset); - BackendConfig.addAuthToken(builder, authTokenOffset); - return BackendConfig.endBackendConfig(builder); -} -} diff --git a/frontend/src/lib/_fbs/open-shock/serialization/configuration/captive-portal-config.ts b/frontend/src/lib/_fbs/open-shock/serialization/configuration/captive-portal-config.ts deleted file mode 100644 index 179cefe07..000000000 --- a/frontend/src/lib/_fbs/open-shock/serialization/configuration/captive-portal-config.ts +++ /dev/null @@ -1,52 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -/* eslint-disable @typescript-eslint/no-unused-vars, @typescript-eslint/no-explicit-any, @typescript-eslint/no-non-null-assertion */ - -import * as flatbuffers from 'flatbuffers'; - -export class CaptivePortalConfig { - bb: flatbuffers.ByteBuffer|null = null; - bb_pos = 0; - __init(i:number, bb:flatbuffers.ByteBuffer):CaptivePortalConfig { - this.bb_pos = i; - this.bb = bb; - return this; -} - -static getRootAsCaptivePortalConfig(bb:flatbuffers.ByteBuffer, obj?:CaptivePortalConfig):CaptivePortalConfig { - return (obj || new CaptivePortalConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -static getSizePrefixedRootAsCaptivePortalConfig(bb:flatbuffers.ByteBuffer, obj?:CaptivePortalConfig):CaptivePortalConfig { - bb.setPosition(bb.position() + flatbuffers.SIZE_PREFIX_LENGTH); - return (obj || new CaptivePortalConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -/** - * Whether the captive portal is forced to be enabled - * The captive portal will otherwise shut down when a gateway connection is established - */ -alwaysEnabled():boolean { - const offset = this.bb!.__offset(this.bb_pos, 4); - return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; -} - -static startCaptivePortalConfig(builder:flatbuffers.Builder) { - builder.startObject(1); -} - -static addAlwaysEnabled(builder:flatbuffers.Builder, alwaysEnabled:boolean) { - builder.addFieldInt8(0, +alwaysEnabled, +false); -} - -static endCaptivePortalConfig(builder:flatbuffers.Builder):flatbuffers.Offset { - const offset = builder.endObject(); - return offset; -} - -static createCaptivePortalConfig(builder:flatbuffers.Builder, alwaysEnabled:boolean):flatbuffers.Offset { - CaptivePortalConfig.startCaptivePortalConfig(builder); - CaptivePortalConfig.addAlwaysEnabled(builder, alwaysEnabled); - return CaptivePortalConfig.endCaptivePortalConfig(builder); -} -} diff --git a/frontend/src/lib/_fbs/open-shock/serialization/configuration/estop-config.ts b/frontend/src/lib/_fbs/open-shock/serialization/configuration/estop-config.ts deleted file mode 100644 index ac41c7bd1..000000000 --- a/frontend/src/lib/_fbs/open-shock/serialization/configuration/estop-config.ts +++ /dev/null @@ -1,87 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -/* eslint-disable @typescript-eslint/no-unused-vars, @typescript-eslint/no-explicit-any, @typescript-eslint/no-non-null-assertion */ - -import * as flatbuffers from 'flatbuffers'; - -export class EStopConfig { - bb: flatbuffers.ByteBuffer|null = null; - bb_pos = 0; - __init(i:number, bb:flatbuffers.ByteBuffer):EStopConfig { - this.bb_pos = i; - this.bb = bb; - return this; -} - -static getRootAsEStopConfig(bb:flatbuffers.ByteBuffer, obj?:EStopConfig):EStopConfig { - return (obj || new EStopConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -static getSizePrefixedRootAsEStopConfig(bb:flatbuffers.ByteBuffer, obj?:EStopConfig):EStopConfig { - bb.setPosition(bb.position() + flatbuffers.SIZE_PREFIX_LENGTH); - return (obj || new EStopConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -enabled():boolean { - const offset = this.bb!.__offset(this.bb_pos, 4); - return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; -} - -/** - * The GPIO pin connected to the E-Stop button - */ -gpioPin():number { - const offset = this.bb!.__offset(this.bb_pos, 6); - return offset ? this.bb!.readInt8(this.bb_pos + offset) : 0; -} - -/** - * Persistent state of the E-Stop button - */ -active():boolean { - const offset = this.bb!.__offset(this.bb_pos, 8); - return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; -} - -/** - * Set When EmergencyStop is a latching switch - */ -latching():boolean { - const offset = this.bb!.__offset(this.bb_pos, 10); - return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; -} - -static startEStopConfig(builder:flatbuffers.Builder) { - builder.startObject(4); -} - -static addEnabled(builder:flatbuffers.Builder, enabled:boolean) { - builder.addFieldInt8(0, +enabled, +false); -} - -static addGpioPin(builder:flatbuffers.Builder, gpioPin:number) { - builder.addFieldInt8(1, gpioPin, 0); -} - -static addActive(builder:flatbuffers.Builder, active:boolean) { - builder.addFieldInt8(2, +active, +false); -} - -static addLatching(builder:flatbuffers.Builder, latching:boolean) { - builder.addFieldInt8(3, +latching, +false); -} - -static endEStopConfig(builder:flatbuffers.Builder):flatbuffers.Offset { - const offset = builder.endObject(); - return offset; -} - -static createEStopConfig(builder:flatbuffers.Builder, enabled:boolean, gpioPin:number, active:boolean, latching:boolean):flatbuffers.Offset { - EStopConfig.startEStopConfig(builder); - EStopConfig.addEnabled(builder, enabled); - EStopConfig.addGpioPin(builder, gpioPin); - EStopConfig.addActive(builder, active); - EStopConfig.addLatching(builder, latching); - return EStopConfig.endEStopConfig(builder); -} -} diff --git a/frontend/src/lib/_fbs/open-shock/serialization/configuration/hub-config.ts b/frontend/src/lib/_fbs/open-shock/serialization/configuration/hub-config.ts deleted file mode 100644 index a850d0718..000000000 --- a/frontend/src/lib/_fbs/open-shock/serialization/configuration/hub-config.ts +++ /dev/null @@ -1,148 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -/* eslint-disable @typescript-eslint/no-unused-vars, @typescript-eslint/no-explicit-any, @typescript-eslint/no-non-null-assertion */ - -import * as flatbuffers from 'flatbuffers'; - -import { BackendConfig } from '../../../open-shock/serialization/configuration/backend-config'; -import { CaptivePortalConfig } from '../../../open-shock/serialization/configuration/captive-portal-config'; -import { EStopConfig } from '../../../open-shock/serialization/configuration/estop-config'; -import { LanConfig } from '../../../open-shock/serialization/configuration/lan-config'; -import { OtaUpdateConfig } from '../../../open-shock/serialization/configuration/ota-update-config'; -import { RFConfig } from '../../../open-shock/serialization/configuration/rfconfig'; -import { SerialInputConfig } from '../../../open-shock/serialization/configuration/serial-input-config'; -import { WiFiConfig } from '../../../open-shock/serialization/configuration/wi-fi-config'; - - -export class HubConfig { - bb: flatbuffers.ByteBuffer|null = null; - bb_pos = 0; - __init(i:number, bb:flatbuffers.ByteBuffer):HubConfig { - this.bb_pos = i; - this.bb = bb; - return this; -} - -static getRootAsHubConfig(bb:flatbuffers.ByteBuffer, obj?:HubConfig):HubConfig { - return (obj || new HubConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -static getSizePrefixedRootAsHubConfig(bb:flatbuffers.ByteBuffer, obj?:HubConfig):HubConfig { - bb.setPosition(bb.position() + flatbuffers.SIZE_PREFIX_LENGTH); - return (obj || new HubConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -/** - * RF Transmitter configuration - */ -rf(obj?:RFConfig):RFConfig|null { - const offset = this.bb!.__offset(this.bb_pos, 4); - return offset ? (obj || new RFConfig()).__init(this.bb!.__indirect(this.bb_pos + offset), this.bb!) : null; -} - -/** - * WiFi configuration - */ -wifi(obj?:WiFiConfig):WiFiConfig|null { - const offset = this.bb!.__offset(this.bb_pos, 6); - return offset ? (obj || new WiFiConfig()).__init(this.bb!.__indirect(this.bb_pos + offset), this.bb!) : null; -} - -/** - * Captive portal configuration - */ -captivePortal(obj?:CaptivePortalConfig):CaptivePortalConfig|null { - const offset = this.bb!.__offset(this.bb_pos, 8); - return offset ? (obj || new CaptivePortalConfig()).__init(this.bb!.__indirect(this.bb_pos + offset), this.bb!) : null; -} - -/** - * Backend configuration - */ -backend(obj?:BackendConfig):BackendConfig|null { - const offset = this.bb!.__offset(this.bb_pos, 10); - return offset ? (obj || new BackendConfig()).__init(this.bb!.__indirect(this.bb_pos + offset), this.bb!) : null; -} - -/** - * Serial input configuration - */ -serialInput(obj?:SerialInputConfig):SerialInputConfig|null { - const offset = this.bb!.__offset(this.bb_pos, 12); - return offset ? (obj || new SerialInputConfig()).__init(this.bb!.__indirect(this.bb_pos + offset), this.bb!) : null; -} - -/** - * OTA update configuration - */ -otaUpdate(obj?:OtaUpdateConfig):OtaUpdateConfig|null { - const offset = this.bb!.__offset(this.bb_pos, 14); - return offset ? (obj || new OtaUpdateConfig()).__init(this.bb!.__indirect(this.bb_pos + offset), this.bb!) : null; -} - -/** - * E-Stop configuration - */ -estop(obj?:EStopConfig):EStopConfig|null { - const offset = this.bb!.__offset(this.bb_pos, 16); - return offset ? (obj || new EStopConfig()).__init(this.bb!.__indirect(this.bb_pos + offset), this.bb!) : null; -} - -/** - * LAN API configuration - */ -lan(obj?:LanConfig):LanConfig|null { - const offset = this.bb!.__offset(this.bb_pos, 18); - return offset ? (obj || new LanConfig()).__init(this.bb!.__indirect(this.bb_pos + offset), this.bb!) : null; -} - -static startHubConfig(builder:flatbuffers.Builder) { - builder.startObject(8); -} - -static addRf(builder:flatbuffers.Builder, rfOffset:flatbuffers.Offset) { - builder.addFieldOffset(0, rfOffset, 0); -} - -static addWifi(builder:flatbuffers.Builder, wifiOffset:flatbuffers.Offset) { - builder.addFieldOffset(1, wifiOffset, 0); -} - -static addCaptivePortal(builder:flatbuffers.Builder, captivePortalOffset:flatbuffers.Offset) { - builder.addFieldOffset(2, captivePortalOffset, 0); -} - -static addBackend(builder:flatbuffers.Builder, backendOffset:flatbuffers.Offset) { - builder.addFieldOffset(3, backendOffset, 0); -} - -static addSerialInput(builder:flatbuffers.Builder, serialInputOffset:flatbuffers.Offset) { - builder.addFieldOffset(4, serialInputOffset, 0); -} - -static addOtaUpdate(builder:flatbuffers.Builder, otaUpdateOffset:flatbuffers.Offset) { - builder.addFieldOffset(5, otaUpdateOffset, 0); -} - -static addEstop(builder:flatbuffers.Builder, estopOffset:flatbuffers.Offset) { - builder.addFieldOffset(6, estopOffset, 0); -} - -static addLan(builder:flatbuffers.Builder, lanOffset:flatbuffers.Offset) { - builder.addFieldOffset(7, lanOffset, 0); -} - -static endHubConfig(builder:flatbuffers.Builder):flatbuffers.Offset { - const offset = builder.endObject(); - return offset; -} - -static finishHubConfigBuffer(builder:flatbuffers.Builder, offset:flatbuffers.Offset) { - builder.finish(offset); -} - -static finishSizePrefixedHubConfigBuffer(builder:flatbuffers.Builder, offset:flatbuffers.Offset) { - builder.finish(offset, undefined, true); -} - -} diff --git a/frontend/src/lib/_fbs/open-shock/serialization/configuration/lan-config.ts b/frontend/src/lib/_fbs/open-shock/serialization/configuration/lan-config.ts deleted file mode 100644 index 662ac03b9..000000000 --- a/frontend/src/lib/_fbs/open-shock/serialization/configuration/lan-config.ts +++ /dev/null @@ -1,66 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -/* eslint-disable @typescript-eslint/no-unused-vars, @typescript-eslint/no-explicit-any, @typescript-eslint/no-non-null-assertion */ - -import * as flatbuffers from 'flatbuffers'; - -export class LanConfig { - bb: flatbuffers.ByteBuffer|null = null; - bb_pos = 0; - __init(i:number, bb:flatbuffers.ByteBuffer):LanConfig { - this.bb_pos = i; - this.bb = bb; - return this; -} - -static getRootAsLanConfig(bb:flatbuffers.ByteBuffer, obj?:LanConfig):LanConfig { - return (obj || new LanConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -static getSizePrefixedRootAsLanConfig(bb:flatbuffers.ByteBuffer, obj?:LanConfig):LanConfig { - bb.setPosition(bb.position() + flatbuffers.SIZE_PREFIX_LENGTH); - return (obj || new LanConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -/** - * Whether an API key is required for LAN endpoints - */ -apiKeyEnabled():boolean { - const offset = this.bb!.__offset(this.bb_pos, 4); - return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; -} - -/** - * API key for authenticating LAN requests (generated or user-set) - */ -apiKey():string|null -apiKey(optionalEncoding:flatbuffers.Encoding):string|Uint8Array|null -apiKey(optionalEncoding?:any):string|Uint8Array|null { - const offset = this.bb!.__offset(this.bb_pos, 6); - return offset ? this.bb!.__string(this.bb_pos + offset, optionalEncoding) : null; -} - -static startLanConfig(builder:flatbuffers.Builder) { - builder.startObject(2); -} - -static addApiKeyEnabled(builder:flatbuffers.Builder, apiKeyEnabled:boolean) { - builder.addFieldInt8(0, +apiKeyEnabled, +false); -} - -static addApiKey(builder:flatbuffers.Builder, apiKeyOffset:flatbuffers.Offset) { - builder.addFieldOffset(1, apiKeyOffset, 0); -} - -static endLanConfig(builder:flatbuffers.Builder):flatbuffers.Offset { - const offset = builder.endObject(); - return offset; -} - -static createLanConfig(builder:flatbuffers.Builder, apiKeyEnabled:boolean, apiKeyOffset:flatbuffers.Offset):flatbuffers.Offset { - LanConfig.startLanConfig(builder); - LanConfig.addApiKeyEnabled(builder, apiKeyEnabled); - LanConfig.addApiKey(builder, apiKeyOffset); - return LanConfig.endLanConfig(builder); -} -} diff --git a/frontend/src/lib/_fbs/open-shock/serialization/configuration/mac-address.ts b/frontend/src/lib/_fbs/open-shock/serialization/configuration/mac-address.ts deleted file mode 100644 index b2be7358d..000000000 --- a/frontend/src/lib/_fbs/open-shock/serialization/configuration/mac-address.ts +++ /dev/null @@ -1,38 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -/* eslint-disable @typescript-eslint/no-unused-vars, @typescript-eslint/no-explicit-any, @typescript-eslint/no-non-null-assertion */ - -import * as flatbuffers from 'flatbuffers'; - -/** - * 6-byte MAC address (BSSID) - */ -export class MacAddress { - bb: flatbuffers.ByteBuffer|null = null; - bb_pos = 0; - __init(i:number, bb:flatbuffers.ByteBuffer):MacAddress { - this.bb_pos = i; - this.bb = bb; - return this; -} - -bytes(index: number):number|null { - return this.bb!.readUint8(this.bb_pos + 0 + index); -} - -static sizeOf():number { - return 6; -} - -static createMacAddress(builder:flatbuffers.Builder, bytes: number[]):flatbuffers.Offset { - builder.prep(1, 6); - - for (let i = 5; i >= 0; --i) { - builder.writeInt8((bytes?.[i] ?? 0)); - - } - - return builder.offset(); -} - -} diff --git a/frontend/src/lib/_fbs/open-shock/serialization/configuration/ota-update-config.ts b/frontend/src/lib/_fbs/open-shock/serialization/configuration/ota-update-config.ts deleted file mode 100644 index 482766fd4..000000000 --- a/frontend/src/lib/_fbs/open-shock/serialization/configuration/ota-update-config.ts +++ /dev/null @@ -1,174 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -/* eslint-disable @typescript-eslint/no-unused-vars, @typescript-eslint/no-explicit-any, @typescript-eslint/no-non-null-assertion */ - -import * as flatbuffers from 'flatbuffers'; - -import { OtaUpdateChannel } from '../../../open-shock/serialization/configuration/ota-update-channel'; -import { OtaUpdateStep } from '../../../open-shock/serialization/configuration/ota-update-step'; - - -export class OtaUpdateConfig { - bb: flatbuffers.ByteBuffer|null = null; - bb_pos = 0; - __init(i:number, bb:flatbuffers.ByteBuffer):OtaUpdateConfig { - this.bb_pos = i; - this.bb = bb; - return this; -} - -static getRootAsOtaUpdateConfig(bb:flatbuffers.ByteBuffer, obj?:OtaUpdateConfig):OtaUpdateConfig { - return (obj || new OtaUpdateConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -static getSizePrefixedRootAsOtaUpdateConfig(bb:flatbuffers.ByteBuffer, obj?:OtaUpdateConfig):OtaUpdateConfig { - bb.setPosition(bb.position() + flatbuffers.SIZE_PREFIX_LENGTH); - return (obj || new OtaUpdateConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -/** - * Indicates whether OTA updates are enabled. - */ -isEnabled():boolean { - const offset = this.bb!.__offset(this.bb_pos, 4); - return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : true; -} - -/** - * The domain name of the OTA Content Delivery Network (CDN). - */ -cdnDomain():string|null -cdnDomain(optionalEncoding:flatbuffers.Encoding):string|Uint8Array|null -cdnDomain(optionalEncoding?:any):string|Uint8Array|null { - const offset = this.bb!.__offset(this.bb_pos, 6); - return offset ? this.bb!.__string(this.bb_pos + offset, optionalEncoding) : null; -} - -/** - * The update channel to use. - */ -updateChannel():OtaUpdateChannel { - const offset = this.bb!.__offset(this.bb_pos, 8); - return offset ? this.bb!.readUint8(this.bb_pos + offset) : OtaUpdateChannel.Stable; -} - -/** - * Indicates whether to check for updates on startup. - */ -checkOnStartup():boolean { - const offset = this.bb!.__offset(this.bb_pos, 10); - return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; -} - -/** - * Indicates whether to check for updates periodically. - */ -checkPeriodically():boolean { - const offset = this.bb!.__offset(this.bb_pos, 12); - return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; -} - -/** - * The interval in minutes between periodic update checks. - */ -checkInterval():number { - const offset = this.bb!.__offset(this.bb_pos, 14); - return offset ? this.bb!.readUint16(this.bb_pos + offset) : 0; -} - -/** - * Indicates if the backend is authorized to manage the hub's update version on behalf of the user. - */ -allowBackendManagement():boolean { - const offset = this.bb!.__offset(this.bb_pos, 16); - return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : true; -} - -/** - * Indicates if manual approval via serial input or captive portal is required before installing updates. - */ -requireManualApproval():boolean { - const offset = this.bb!.__offset(this.bb_pos, 18); - return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; -} - -/** - * Update process ID, used to track the update process server-side across reboots. - */ -updateId():number { - const offset = this.bb!.__offset(this.bb_pos, 20); - return offset ? this.bb!.readInt32(this.bb_pos + offset) : 0; -} - -/** - * Indicates what step of the update process the hub is currently in, used to detect failed updates for status reporting. - */ -updateStep():OtaUpdateStep { - const offset = this.bb!.__offset(this.bb_pos, 22); - return offset ? this.bb!.readUint8(this.bb_pos + offset) : OtaUpdateStep.None; -} - -static startOtaUpdateConfig(builder:flatbuffers.Builder) { - builder.startObject(10); -} - -static addIsEnabled(builder:flatbuffers.Builder, isEnabled:boolean) { - builder.addFieldInt8(0, +isEnabled, +true); -} - -static addCdnDomain(builder:flatbuffers.Builder, cdnDomainOffset:flatbuffers.Offset) { - builder.addFieldOffset(1, cdnDomainOffset, 0); -} - -static addUpdateChannel(builder:flatbuffers.Builder, updateChannel:OtaUpdateChannel) { - builder.addFieldInt8(2, updateChannel, OtaUpdateChannel.Stable); -} - -static addCheckOnStartup(builder:flatbuffers.Builder, checkOnStartup:boolean) { - builder.addFieldInt8(3, +checkOnStartup, +false); -} - -static addCheckPeriodically(builder:flatbuffers.Builder, checkPeriodically:boolean) { - builder.addFieldInt8(4, +checkPeriodically, +false); -} - -static addCheckInterval(builder:flatbuffers.Builder, checkInterval:number) { - builder.addFieldInt16(5, checkInterval, 0); -} - -static addAllowBackendManagement(builder:flatbuffers.Builder, allowBackendManagement:boolean) { - builder.addFieldInt8(6, +allowBackendManagement, +true); -} - -static addRequireManualApproval(builder:flatbuffers.Builder, requireManualApproval:boolean) { - builder.addFieldInt8(7, +requireManualApproval, +false); -} - -static addUpdateId(builder:flatbuffers.Builder, updateId:number) { - builder.addFieldInt32(8, updateId, 0); -} - -static addUpdateStep(builder:flatbuffers.Builder, updateStep:OtaUpdateStep) { - builder.addFieldInt8(9, updateStep, OtaUpdateStep.None); -} - -static endOtaUpdateConfig(builder:flatbuffers.Builder):flatbuffers.Offset { - const offset = builder.endObject(); - return offset; -} - -static createOtaUpdateConfig(builder:flatbuffers.Builder, isEnabled:boolean, cdnDomainOffset:flatbuffers.Offset, updateChannel:OtaUpdateChannel, checkOnStartup:boolean, checkPeriodically:boolean, checkInterval:number, allowBackendManagement:boolean, requireManualApproval:boolean, updateId:number, updateStep:OtaUpdateStep):flatbuffers.Offset { - OtaUpdateConfig.startOtaUpdateConfig(builder); - OtaUpdateConfig.addIsEnabled(builder, isEnabled); - OtaUpdateConfig.addCdnDomain(builder, cdnDomainOffset); - OtaUpdateConfig.addUpdateChannel(builder, updateChannel); - OtaUpdateConfig.addCheckOnStartup(builder, checkOnStartup); - OtaUpdateConfig.addCheckPeriodically(builder, checkPeriodically); - OtaUpdateConfig.addCheckInterval(builder, checkInterval); - OtaUpdateConfig.addAllowBackendManagement(builder, allowBackendManagement); - OtaUpdateConfig.addRequireManualApproval(builder, requireManualApproval); - OtaUpdateConfig.addUpdateId(builder, updateId); - OtaUpdateConfig.addUpdateStep(builder, updateStep); - return OtaUpdateConfig.endOtaUpdateConfig(builder); -} -} diff --git a/frontend/src/lib/_fbs/open-shock/serialization/configuration/ota-update-step.ts b/frontend/src/lib/_fbs/open-shock/serialization/configuration/ota-update-step.ts deleted file mode 100644 index 2e1c4c61a..000000000 --- a/frontend/src/lib/_fbs/open-shock/serialization/configuration/ota-update-step.ts +++ /dev/null @@ -1,12 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -/* eslint-disable @typescript-eslint/no-unused-vars, @typescript-eslint/no-explicit-any, @typescript-eslint/no-non-null-assertion */ - -export enum OtaUpdateStep { - None = 0, - Updating = 1, - Updated = 2, - Validating = 3, - Validated = 4, - RollingBack = 5 -} diff --git a/frontend/src/lib/_fbs/open-shock/serialization/configuration/rfconfig.ts b/frontend/src/lib/_fbs/open-shock/serialization/configuration/rfconfig.ts deleted file mode 100644 index 2a5a4d351..000000000 --- a/frontend/src/lib/_fbs/open-shock/serialization/configuration/rfconfig.ts +++ /dev/null @@ -1,64 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -/* eslint-disable @typescript-eslint/no-unused-vars, @typescript-eslint/no-explicit-any, @typescript-eslint/no-non-null-assertion */ - -import * as flatbuffers from 'flatbuffers'; - -export class RFConfig { - bb: flatbuffers.ByteBuffer|null = null; - bb_pos = 0; - __init(i:number, bb:flatbuffers.ByteBuffer):RFConfig { - this.bb_pos = i; - this.bb = bb; - return this; -} - -static getRootAsRFConfig(bb:flatbuffers.ByteBuffer, obj?:RFConfig):RFConfig { - return (obj || new RFConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -static getSizePrefixedRootAsRFConfig(bb:flatbuffers.ByteBuffer, obj?:RFConfig):RFConfig { - bb.setPosition(bb.position() + flatbuffers.SIZE_PREFIX_LENGTH); - return (obj || new RFConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -/** - * The GPIO pin connected to the RF modulator's data pin for transmitting (TX) - */ -txPin():number { - const offset = this.bb!.__offset(this.bb_pos, 4); - return offset ? this.bb!.readInt8(this.bb_pos + offset) : 0; -} - -/** - * Whether to transmit keepalive messages to keep the shockers from entering sleep mode - */ -keepaliveEnabled():boolean { - const offset = this.bb!.__offset(this.bb_pos, 6); - return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : true; -} - -static startRFConfig(builder:flatbuffers.Builder) { - builder.startObject(2); -} - -static addTxPin(builder:flatbuffers.Builder, txPin:number) { - builder.addFieldInt8(0, txPin, 0); -} - -static addKeepaliveEnabled(builder:flatbuffers.Builder, keepaliveEnabled:boolean) { - builder.addFieldInt8(1, +keepaliveEnabled, +true); -} - -static endRFConfig(builder:flatbuffers.Builder):flatbuffers.Offset { - const offset = builder.endObject(); - return offset; -} - -static createRFConfig(builder:flatbuffers.Builder, txPin:number, keepaliveEnabled:boolean):flatbuffers.Offset { - RFConfig.startRFConfig(builder); - RFConfig.addTxPin(builder, txPin); - RFConfig.addKeepaliveEnabled(builder, keepaliveEnabled); - return RFConfig.endRFConfig(builder); -} -} diff --git a/frontend/src/lib/_fbs/open-shock/serialization/configuration/serial-input-config.ts b/frontend/src/lib/_fbs/open-shock/serialization/configuration/serial-input-config.ts deleted file mode 100644 index c7534fd35..000000000 --- a/frontend/src/lib/_fbs/open-shock/serialization/configuration/serial-input-config.ts +++ /dev/null @@ -1,51 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -/* eslint-disable @typescript-eslint/no-unused-vars, @typescript-eslint/no-explicit-any, @typescript-eslint/no-non-null-assertion */ - -import * as flatbuffers from 'flatbuffers'; - -export class SerialInputConfig { - bb: flatbuffers.ByteBuffer|null = null; - bb_pos = 0; - __init(i:number, bb:flatbuffers.ByteBuffer):SerialInputConfig { - this.bb_pos = i; - this.bb = bb; - return this; -} - -static getRootAsSerialInputConfig(bb:flatbuffers.ByteBuffer, obj?:SerialInputConfig):SerialInputConfig { - return (obj || new SerialInputConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -static getSizePrefixedRootAsSerialInputConfig(bb:flatbuffers.ByteBuffer, obj?:SerialInputConfig):SerialInputConfig { - bb.setPosition(bb.position() + flatbuffers.SIZE_PREFIX_LENGTH); - return (obj || new SerialInputConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -/** - * Whether to echo typed characters back to the serial console - */ -echoEnabled():boolean { - const offset = this.bb!.__offset(this.bb_pos, 4); - return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : true; -} - -static startSerialInputConfig(builder:flatbuffers.Builder) { - builder.startObject(1); -} - -static addEchoEnabled(builder:flatbuffers.Builder, echoEnabled:boolean) { - builder.addFieldInt8(0, +echoEnabled, +true); -} - -static endSerialInputConfig(builder:flatbuffers.Builder):flatbuffers.Offset { - const offset = builder.endObject(); - return offset; -} - -static createSerialInputConfig(builder:flatbuffers.Builder, echoEnabled:boolean):flatbuffers.Offset { - SerialInputConfig.startSerialInputConfig(builder); - SerialInputConfig.addEchoEnabled(builder, echoEnabled); - return SerialInputConfig.endSerialInputConfig(builder); -} -} diff --git a/frontend/src/lib/_fbs/open-shock/serialization/configuration/wi-fi-config.ts b/frontend/src/lib/_fbs/open-shock/serialization/configuration/wi-fi-config.ts deleted file mode 100644 index 53b589fbc..000000000 --- a/frontend/src/lib/_fbs/open-shock/serialization/configuration/wi-fi-config.ts +++ /dev/null @@ -1,155 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -/* eslint-disable @typescript-eslint/no-unused-vars, @typescript-eslint/no-explicit-any, @typescript-eslint/no-non-null-assertion */ - -import * as flatbuffers from 'flatbuffers'; - -import { WiFiCredentials } from '../../../open-shock/serialization/configuration/wi-fi-credentials'; - - -export class WiFiConfig { - bb: flatbuffers.ByteBuffer|null = null; - bb_pos = 0; - __init(i:number, bb:flatbuffers.ByteBuffer):WiFiConfig { - this.bb_pos = i; - this.bb = bb; - return this; -} - -static getRootAsWiFiConfig(bb:flatbuffers.ByteBuffer, obj?:WiFiConfig):WiFiConfig { - return (obj || new WiFiConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -static getSizePrefixedRootAsWiFiConfig(bb:flatbuffers.ByteBuffer, obj?:WiFiConfig):WiFiConfig { - bb.setPosition(bb.position() + flatbuffers.SIZE_PREFIX_LENGTH); - return (obj || new WiFiConfig()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -/** - * Access point SSID - */ -apSsid():string|null -apSsid(optionalEncoding:flatbuffers.Encoding):string|Uint8Array|null -apSsid(optionalEncoding?:any):string|Uint8Array|null { - const offset = this.bb!.__offset(this.bb_pos, 4); - return offset ? this.bb!.__string(this.bb_pos + offset, optionalEncoding) : null; -} - -/** - * Hub hostname - */ -hostname():string|null -hostname(optionalEncoding:flatbuffers.Encoding):string|Uint8Array|null -hostname(optionalEncoding?:any):string|Uint8Array|null { - const offset = this.bb!.__offset(this.bb_pos, 6); - return offset ? this.bb!.__string(this.bb_pos + offset, optionalEncoding) : null; -} - -/** - * WiFi network credentials - */ -credentials(index: number, obj?:WiFiCredentials):WiFiCredentials|null { - const offset = this.bb!.__offset(this.bb_pos, 8); - return offset ? (obj || new WiFiCredentials()).__init(this.bb!.__indirect(this.bb!.__vector(this.bb_pos + offset) + index * 4), this.bb!) : null; -} - -credentialsLength():number { - const offset = this.bb!.__offset(this.bb_pos, 8); - return offset ? this.bb!.__vector_len(this.bb_pos + offset) : 0; -} - -/** - * Access point password (empty = open network) - */ -apPassword():string|null -apPassword(optionalEncoding:flatbuffers.Encoding):string|Uint8Array|null -apPassword(optionalEncoding?:any):string|Uint8Array|null { - const offset = this.bb!.__offset(this.bb_pos, 10); - return offset ? this.bb!.__string(this.bb_pos + offset, optionalEncoding) : null; -} - -/** - * Disable WiFi Access Point mode completely - */ -apDisabled():boolean { - const offset = this.bb!.__offset(this.bb_pos, 12); - return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; -} - -/** - * Disable WiFi Station mode completely - */ -staDisabled():boolean { - const offset = this.bb!.__offset(this.bb_pos, 14); - return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; -} - -/** - * Don't connect to any cloud services - */ -offlineMode():boolean { - const offset = this.bb!.__offset(this.bb_pos, 16); - return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; -} - -static startWiFiConfig(builder:flatbuffers.Builder) { - builder.startObject(7); -} - -static addApSsid(builder:flatbuffers.Builder, apSsidOffset:flatbuffers.Offset) { - builder.addFieldOffset(0, apSsidOffset, 0); -} - -static addHostname(builder:flatbuffers.Builder, hostnameOffset:flatbuffers.Offset) { - builder.addFieldOffset(1, hostnameOffset, 0); -} - -static addCredentials(builder:flatbuffers.Builder, credentialsOffset:flatbuffers.Offset) { - builder.addFieldOffset(2, credentialsOffset, 0); -} - -static createCredentialsVector(builder:flatbuffers.Builder, data:flatbuffers.Offset[]):flatbuffers.Offset { - builder.startVector(4, data.length, 4); - for (let i = data.length - 1; i >= 0; i--) { - builder.addOffset(data[i]!); - } - return builder.endVector(); -} - -static startCredentialsVector(builder:flatbuffers.Builder, numElems:number) { - builder.startVector(4, numElems, 4); -} - -static addApPassword(builder:flatbuffers.Builder, apPasswordOffset:flatbuffers.Offset) { - builder.addFieldOffset(3, apPasswordOffset, 0); -} - -static addApDisabled(builder:flatbuffers.Builder, apDisabled:boolean) { - builder.addFieldInt8(4, +apDisabled, +false); -} - -static addStaDisabled(builder:flatbuffers.Builder, staDisabled:boolean) { - builder.addFieldInt8(5, +staDisabled, +false); -} - -static addOfflineMode(builder:flatbuffers.Builder, offlineMode:boolean) { - builder.addFieldInt8(6, +offlineMode, +false); -} - -static endWiFiConfig(builder:flatbuffers.Builder):flatbuffers.Offset { - const offset = builder.endObject(); - return offset; -} - -static createWiFiConfig(builder:flatbuffers.Builder, apSsidOffset:flatbuffers.Offset, hostnameOffset:flatbuffers.Offset, credentialsOffset:flatbuffers.Offset, apPasswordOffset:flatbuffers.Offset, apDisabled:boolean, staDisabled:boolean, offlineMode:boolean):flatbuffers.Offset { - WiFiConfig.startWiFiConfig(builder); - WiFiConfig.addApSsid(builder, apSsidOffset); - WiFiConfig.addHostname(builder, hostnameOffset); - WiFiConfig.addCredentials(builder, credentialsOffset); - WiFiConfig.addApPassword(builder, apPasswordOffset); - WiFiConfig.addApDisabled(builder, apDisabled); - WiFiConfig.addStaDisabled(builder, staDisabled); - WiFiConfig.addOfflineMode(builder, offlineMode); - return WiFiConfig.endWiFiConfig(builder); -} -} diff --git a/frontend/src/lib/_fbs/open-shock/serialization/configuration/wi-fi-credentials.ts b/frontend/src/lib/_fbs/open-shock/serialization/configuration/wi-fi-credentials.ts deleted file mode 100644 index e1b180d0f..000000000 --- a/frontend/src/lib/_fbs/open-shock/serialization/configuration/wi-fi-credentials.ts +++ /dev/null @@ -1,102 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -/* eslint-disable @typescript-eslint/no-unused-vars, @typescript-eslint/no-explicit-any, @typescript-eslint/no-non-null-assertion */ - -import * as flatbuffers from 'flatbuffers'; - -import { MacAddress } from '../../../open-shock/serialization/configuration/mac-address'; -import { WifiAuthMode } from '../../../open-shock/serialization/types/wifi-auth-mode'; - - -export class WiFiCredentials { - bb: flatbuffers.ByteBuffer|null = null; - bb_pos = 0; - __init(i:number, bb:flatbuffers.ByteBuffer):WiFiCredentials { - this.bb_pos = i; - this.bb = bb; - return this; -} - -static getRootAsWiFiCredentials(bb:flatbuffers.ByteBuffer, obj?:WiFiCredentials):WiFiCredentials { - return (obj || new WiFiCredentials()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -static getSizePrefixedRootAsWiFiCredentials(bb:flatbuffers.ByteBuffer, obj?:WiFiCredentials):WiFiCredentials { - bb.setPosition(bb.position() + flatbuffers.SIZE_PREFIX_LENGTH); - return (obj || new WiFiCredentials()).__init(bb.readInt32(bb.position()) + bb.position(), bb); -} - -/** - * ID of the WiFi network credentials, used for referencing the credentials with a low memory footprint - */ -id():number { - const offset = this.bb!.__offset(this.bb_pos, 4); - return offset ? this.bb!.readUint8(this.bb_pos + offset) : 0; -} - -/** - * SSID of the WiFi network - */ -ssid():string|null -ssid(optionalEncoding:flatbuffers.Encoding):string|Uint8Array|null -ssid(optionalEncoding?:any):string|Uint8Array|null { - const offset = this.bb!.__offset(this.bb_pos, 6); - return offset ? this.bb!.__string(this.bb_pos + offset, optionalEncoding) : null; -} - -/** - * Password of the WiFi network - */ -password():string|null -password(optionalEncoding:flatbuffers.Encoding):string|Uint8Array|null -password(optionalEncoding?:any):string|Uint8Array|null { - const offset = this.bb!.__offset(this.bb_pos, 8); - return offset ? this.bb!.__string(this.bb_pos + offset, optionalEncoding) : null; -} - -/** - * Auth mode of the network when credentials were saved. Used to reject evil twins with weaker security. - */ -authMode():WifiAuthMode { - const offset = this.bb!.__offset(this.bb_pos, 10); - return offset ? this.bb!.readUint8(this.bb_pos + offset) : WifiAuthMode.UNKNOWN; -} - -/** - * Pinned BSSID of the last successfully connected AP. All zeros means no pin. - */ -bssid(obj?:MacAddress):MacAddress|null { - const offset = this.bb!.__offset(this.bb_pos, 12); - return offset ? (obj || new MacAddress()).__init(this.bb_pos + offset, this.bb!) : null; -} - -static startWiFiCredentials(builder:flatbuffers.Builder) { - builder.startObject(5); -} - -static addId(builder:flatbuffers.Builder, id:number) { - builder.addFieldInt8(0, id, 0); -} - -static addSsid(builder:flatbuffers.Builder, ssidOffset:flatbuffers.Offset) { - builder.addFieldOffset(1, ssidOffset, 0); -} - -static addPassword(builder:flatbuffers.Builder, passwordOffset:flatbuffers.Offset) { - builder.addFieldOffset(2, passwordOffset, 0); -} - -static addAuthMode(builder:flatbuffers.Builder, authMode:WifiAuthMode) { - builder.addFieldInt8(3, authMode, WifiAuthMode.UNKNOWN); -} - -static addBssid(builder:flatbuffers.Builder, bssidOffset:flatbuffers.Offset) { - builder.addFieldStruct(4, bssidOffset, 0); -} - -static endWiFiCredentials(builder:flatbuffers.Builder):flatbuffers.Offset { - const offset = builder.endObject(); - return offset; -} - -} diff --git a/frontend/src/lib/_fbs/open-shock/serialization/configuration/ota-update-channel.ts b/frontend/src/lib/_fbs/open-shock/serialization/local/ota-channel.ts similarity index 88% rename from frontend/src/lib/_fbs/open-shock/serialization/configuration/ota-update-channel.ts rename to frontend/src/lib/_fbs/open-shock/serialization/local/ota-channel.ts index 2c49196f0..7b08c0975 100644 --- a/frontend/src/lib/_fbs/open-shock/serialization/configuration/ota-update-channel.ts +++ b/frontend/src/lib/_fbs/open-shock/serialization/local/ota-channel.ts @@ -2,7 +2,7 @@ /* eslint-disable @typescript-eslint/no-unused-vars, @typescript-eslint/no-explicit-any, @typescript-eslint/no-non-null-assertion */ -export enum OtaUpdateChannel { +export enum OtaChannel { Stable = 0, Beta = 1, Develop = 2 diff --git a/frontend/src/lib/_fbs/open-shock/serialization/local/ota-settings.ts b/frontend/src/lib/_fbs/open-shock/serialization/local/ota-settings.ts new file mode 100644 index 000000000..8e4ccf86c --- /dev/null +++ b/frontend/src/lib/_fbs/open-shock/serialization/local/ota-settings.ts @@ -0,0 +1,106 @@ +// automatically generated by the FlatBuffers compiler, do not modify + +/* eslint-disable @typescript-eslint/no-unused-vars, @typescript-eslint/no-explicit-any, @typescript-eslint/no-non-null-assertion */ + +import * as flatbuffers from 'flatbuffers'; + +import { OtaChannel } from '../../../open-shock/serialization/local/ota-channel'; + + +/** + * The OTA settings the portal shows and edits. + */ +export class OtaSettings { + bb: flatbuffers.ByteBuffer|null = null; + bb_pos = 0; + __init(i:number, bb:flatbuffers.ByteBuffer):OtaSettings { + this.bb_pos = i; + this.bb = bb; + return this; +} + +static getRootAsOtaSettings(bb:flatbuffers.ByteBuffer, obj?:OtaSettings):OtaSettings { + return (obj || new OtaSettings()).__init(bb.readInt32(bb.position()) + bb.position(), bb); +} + +static getSizePrefixedRootAsOtaSettings(bb:flatbuffers.ByteBuffer, obj?:OtaSettings):OtaSettings { + bb.setPosition(bb.position() + flatbuffers.SIZE_PREFIX_LENGTH); + return (obj || new OtaSettings()).__init(bb.readInt32(bb.position()) + bb.position(), bb); +} + +enabled():boolean { + const offset = this.bb!.__offset(this.bb_pos, 4); + return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; +} + +cdnDomain():string|null +cdnDomain(optionalEncoding:flatbuffers.Encoding):string|Uint8Array|null +cdnDomain(optionalEncoding?:any):string|Uint8Array|null { + const offset = this.bb!.__offset(this.bb_pos, 6); + return offset ? this.bb!.__string(this.bb_pos + offset, optionalEncoding) : null; +} + +channel():OtaChannel { + const offset = this.bb!.__offset(this.bb_pos, 8); + return offset ? this.bb!.readUint8(this.bb_pos + offset) : OtaChannel.Stable; +} + +checkInterval():number { + const offset = this.bb!.__offset(this.bb_pos, 10); + return offset ? this.bb!.readUint16(this.bb_pos + offset) : 0; +} + +allowBackendManagement():boolean { + const offset = this.bb!.__offset(this.bb_pos, 12); + return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; +} + +requireManualApproval():boolean { + const offset = this.bb!.__offset(this.bb_pos, 14); + return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; +} + +static startOtaSettings(builder:flatbuffers.Builder) { + builder.startObject(6); +} + +static addEnabled(builder:flatbuffers.Builder, enabled:boolean) { + builder.addFieldInt8(0, +enabled, +false); +} + +static addCdnDomain(builder:flatbuffers.Builder, cdnDomainOffset:flatbuffers.Offset) { + builder.addFieldOffset(1, cdnDomainOffset, 0); +} + +static addChannel(builder:flatbuffers.Builder, channel:OtaChannel) { + builder.addFieldInt8(2, channel, OtaChannel.Stable); +} + +static addCheckInterval(builder:flatbuffers.Builder, checkInterval:number) { + builder.addFieldInt16(3, checkInterval, 0); +} + +static addAllowBackendManagement(builder:flatbuffers.Builder, allowBackendManagement:boolean) { + builder.addFieldInt8(4, +allowBackendManagement, +false); +} + +static addRequireManualApproval(builder:flatbuffers.Builder, requireManualApproval:boolean) { + builder.addFieldInt8(5, +requireManualApproval, +false); +} + +static endOtaSettings(builder:flatbuffers.Builder):flatbuffers.Offset { + const offset = builder.endObject(); + return offset; +} + +static createOtaSettings(builder:flatbuffers.Builder, enabled:boolean, cdnDomainOffset:flatbuffers.Offset, channel:OtaChannel, checkInterval:number, allowBackendManagement:boolean, requireManualApproval:boolean):flatbuffers.Offset { + OtaSettings.startOtaSettings(builder); + OtaSettings.addEnabled(builder, enabled); + OtaSettings.addCdnDomain(builder, cdnDomainOffset); + OtaSettings.addChannel(builder, channel); + OtaSettings.addCheckInterval(builder, checkInterval); + OtaSettings.addAllowBackendManagement(builder, allowBackendManagement); + OtaSettings.addRequireManualApproval(builder, requireManualApproval); + return OtaSettings.endOtaSettings(builder); +} +} diff --git a/frontend/src/lib/_fbs/open-shock/serialization/local/ready-message.ts b/frontend/src/lib/_fbs/open-shock/serialization/local/ready-message.ts index cc8819eea..3fe0712ba 100644 --- a/frontend/src/lib/_fbs/open-shock/serialization/local/ready-message.ts +++ b/frontend/src/lib/_fbs/open-shock/serialization/local/ready-message.ts @@ -4,7 +4,7 @@ import * as flatbuffers from 'flatbuffers'; -import { HubConfig } from '../../../open-shock/serialization/configuration/hub-config'; +import { OtaSettings } from '../../../open-shock/serialization/local/ota-settings'; import { WifiNetwork } from '../../../open-shock/serialization/types/wifi-network'; @@ -41,48 +41,78 @@ accountLinked():boolean { return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; } -config(obj?:HubConfig):HubConfig|null { - const offset = this.bb!.__offset(this.bb_pos, 10); - return offset ? (obj || new HubConfig()).__init(this.bb!.__indirect(this.bb_pos + offset), this.bb!) : null; -} - gpioValidInputs(index: number):number|null { - const offset = this.bb!.__offset(this.bb_pos, 12); + const offset = this.bb!.__offset(this.bb_pos, 10); return offset ? this.bb!.readInt8(this.bb!.__vector(this.bb_pos + offset) + index) : 0; } gpioValidInputsLength():number { - const offset = this.bb!.__offset(this.bb_pos, 12); + const offset = this.bb!.__offset(this.bb_pos, 10); return offset ? this.bb!.__vector_len(this.bb_pos + offset) : 0; } gpioValidInputsArray():Int8Array|null { - const offset = this.bb!.__offset(this.bb_pos, 12); + const offset = this.bb!.__offset(this.bb_pos, 10); return offset ? new Int8Array(this.bb!.bytes().buffer, this.bb!.bytes().byteOffset + this.bb!.__vector(this.bb_pos + offset), this.bb!.__vector_len(this.bb_pos + offset)) : null; } gpioValidOutputs(index: number):number|null { - const offset = this.bb!.__offset(this.bb_pos, 14); + const offset = this.bb!.__offset(this.bb_pos, 12); return offset ? this.bb!.readInt8(this.bb!.__vector(this.bb_pos + offset) + index) : 0; } gpioValidOutputsLength():number { - const offset = this.bb!.__offset(this.bb_pos, 14); + const offset = this.bb!.__offset(this.bb_pos, 12); return offset ? this.bb!.__vector_len(this.bb_pos + offset) : 0; } gpioValidOutputsArray():Int8Array|null { - const offset = this.bb!.__offset(this.bb_pos, 14); + const offset = this.bb!.__offset(this.bb_pos, 12); return offset ? new Int8Array(this.bb!.bytes().buffer, this.bb!.bytes().byteOffset + this.bb!.__vector(this.bb_pos + offset), this.bb!.__vector_len(this.bb_pos + offset)) : null; } hasStandardizedPins():boolean { + const offset = this.bb!.__offset(this.bb_pos, 14); + return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; +} + +rfTxPin():number { const offset = this.bb!.__offset(this.bb_pos, 16); + return offset ? this.bb!.readInt8(this.bb_pos + offset) : -1; +} + +estopEnabled():boolean { + const offset = this.bb!.__offset(this.bb_pos, 18); return offset ? !!this.bb!.readInt8(this.bb_pos + offset) : false; } +estopGpioPin():number { + const offset = this.bb!.__offset(this.bb_pos, 20); + return offset ? this.bb!.readInt8(this.bb_pos + offset) : -1; +} + +/** + * SSIDs of the saved networks. Passwords never leave the hub. + */ +savedSsids(index: number):string +savedSsids(index: number,optionalEncoding:flatbuffers.Encoding):string|Uint8Array +savedSsids(index: number,optionalEncoding?:any):string|Uint8Array|null { + const offset = this.bb!.__offset(this.bb_pos, 22); + return offset ? this.bb!.__string(this.bb!.__vector(this.bb_pos + offset) + index * 4, optionalEncoding) : null; +} + +savedSsidsLength():number { + const offset = this.bb!.__offset(this.bb_pos, 22); + return offset ? this.bb!.__vector_len(this.bb_pos + offset) : 0; +} + +ota(obj?:OtaSettings):OtaSettings|null { + const offset = this.bb!.__offset(this.bb_pos, 24); + return offset ? (obj || new OtaSettings()).__init(this.bb!.__indirect(this.bb_pos + offset), this.bb!) : null; +} + static startReadyMessage(builder:flatbuffers.Builder) { - builder.startObject(7); + builder.startObject(11); } static addPoggies(builder:flatbuffers.Builder, poggies:boolean) { @@ -97,12 +127,8 @@ static addAccountLinked(builder:flatbuffers.Builder, accountLinked:boolean) { builder.addFieldInt8(2, +accountLinked, +false); } -static addConfig(builder:flatbuffers.Builder, configOffset:flatbuffers.Offset) { - builder.addFieldOffset(3, configOffset, 0); -} - static addGpioValidInputs(builder:flatbuffers.Builder, gpioValidInputsOffset:flatbuffers.Offset) { - builder.addFieldOffset(4, gpioValidInputsOffset, 0); + builder.addFieldOffset(3, gpioValidInputsOffset, 0); } static createGpioValidInputsVector(builder:flatbuffers.Builder, data:number[]|Int8Array):flatbuffers.Offset; @@ -123,7 +149,7 @@ static startGpioValidInputsVector(builder:flatbuffers.Builder, numElems:number) } static addGpioValidOutputs(builder:flatbuffers.Builder, gpioValidOutputsOffset:flatbuffers.Offset) { - builder.addFieldOffset(5, gpioValidOutputsOffset, 0); + builder.addFieldOffset(4, gpioValidOutputsOffset, 0); } static createGpioValidOutputsVector(builder:flatbuffers.Builder, data:number[]|Int8Array):flatbuffers.Offset; @@ -144,7 +170,39 @@ static startGpioValidOutputsVector(builder:flatbuffers.Builder, numElems:number) } static addHasStandardizedPins(builder:flatbuffers.Builder, hasStandardizedPins:boolean) { - builder.addFieldInt8(6, +hasStandardizedPins, +false); + builder.addFieldInt8(5, +hasStandardizedPins, +false); +} + +static addRfTxPin(builder:flatbuffers.Builder, rfTxPin:number) { + builder.addFieldInt8(6, rfTxPin, -1); +} + +static addEstopEnabled(builder:flatbuffers.Builder, estopEnabled:boolean) { + builder.addFieldInt8(7, +estopEnabled, +false); +} + +static addEstopGpioPin(builder:flatbuffers.Builder, estopGpioPin:number) { + builder.addFieldInt8(8, estopGpioPin, -1); +} + +static addSavedSsids(builder:flatbuffers.Builder, savedSsidsOffset:flatbuffers.Offset) { + builder.addFieldOffset(9, savedSsidsOffset, 0); +} + +static createSavedSsidsVector(builder:flatbuffers.Builder, data:flatbuffers.Offset[]):flatbuffers.Offset { + builder.startVector(4, data.length, 4); + for (let i = data.length - 1; i >= 0; i--) { + builder.addOffset(data[i]!); + } + return builder.endVector(); +} + +static startSavedSsidsVector(builder:flatbuffers.Builder, numElems:number) { + builder.startVector(4, numElems, 4); +} + +static addOta(builder:flatbuffers.Builder, otaOffset:flatbuffers.Offset) { + builder.addFieldOffset(10, otaOffset, 0); } static endReadyMessage(builder:flatbuffers.Builder):flatbuffers.Offset { diff --git a/frontend/src/lib/api.ts b/frontend/src/lib/api.ts index 30776bc32..56a87b919 100644 --- a/frontend/src/lib/api.ts +++ b/frontend/src/lib/api.ts @@ -1,4 +1,4 @@ -import { OtaUpdateChannel } from '#lib/_fbs/open-shock/serialization/configuration/ota-update-channel.js'; +import { OtaChannel } from '#lib/_fbs/open-shock/serialization/local/ota-channel.js'; import type { OtaUpdateConfig } from '#lib/mappers/ConfigMapper.js'; import { hubState } from '#lib/stores/index.js'; import { getApiBaseUrl } from '#lib/utils/localRedirect.js'; @@ -212,10 +212,10 @@ export async function disconnectWifiNetwork(): Promise { // OTA -const _otaChannelNames: Record = { - [OtaUpdateChannel.Stable]: 'stable', - [OtaUpdateChannel.Beta]: 'beta', - [OtaUpdateChannel.Develop]: 'develop', +const _otaChannelNames: Record = { + [OtaChannel.Stable]: 'stable', + [OtaChannel.Beta]: 'beta', + [OtaChannel.Develop]: 'develop', }; // On success the change is mirrored into hubState.config, which is otherwise only sent once on @@ -255,7 +255,7 @@ export function setOtaEnabled(isEnabled: boolean): Promise { }); } -export function setOtaChannel(updateChannel: OtaUpdateChannel): Promise { +export function setOtaChannel(updateChannel: OtaChannel): Promise { return setOtaSettings( { channel: _otaChannelNames[updateChannel] }, 'Failed to update OTA channel', @@ -291,7 +291,7 @@ export function setOtaRequireManualApproval(requireManualApproval: boolean): Pro ); } -export function checkOtaUpdates(channel: OtaUpdateChannel): Promise { +export function checkOtaUpdates(channel: OtaChannel): Promise { return otaRequest( 'check?' + new URLSearchParams({ channel: _otaChannelNames[channel] }), 'POST', diff --git a/frontend/src/lib/components/sections/OtaSection.svelte b/frontend/src/lib/components/sections/OtaSection.svelte index f091f4fea..1ec00afc9 100644 --- a/frontend/src/lib/components/sections/OtaSection.svelte +++ b/frontend/src/lib/components/sections/OtaSection.svelte @@ -2,7 +2,7 @@ import SectionHeader from '#lib/components/SectionHeader.svelte'; import SettingSwitch from '#lib/components/SettingSwitch.svelte'; import { hubState } from '#lib/stores/index.js'; - import { OtaUpdateChannel } from '#lib/_fbs/open-shock/serialization/configuration/ota-update-channel.js'; + import { OtaChannel } from '#lib/_fbs/open-shock/serialization/local/ota-channel.js'; import { setOtaEnabled, setOtaChannel, @@ -16,9 +16,9 @@ import { Label } from '@openshock/svelte-core/components/ui/label'; const channels = [ - { value: OtaUpdateChannel.Stable, label: 'Stable' }, - { value: OtaUpdateChannel.Beta, label: 'Beta' }, - { value: OtaUpdateChannel.Develop, label: 'Develop' }, + { value: OtaChannel.Stable, label: 'Stable' }, + { value: OtaChannel.Beta, label: 'Beta' }, + { value: OtaChannel.Develop, label: 'Develop' }, ]; let otaConfig = $derived(hubState.config?.otaUpdate); @@ -50,7 +50,7 @@ saving = false; } - async function setChannel(channel: OtaUpdateChannel) { + async function setChannel(channel: OtaChannel) { if (saving || channel === otaConfig?.updateChannel) return; saving = true; await setOtaChannel(channel); @@ -60,7 +60,7 @@ async function checkForUpdates() { if (saving) return; saving = true; - await checkOtaUpdates(otaConfig?.updateChannel ?? OtaUpdateChannel.Stable); + await checkOtaUpdates(otaConfig?.updateChannel ?? OtaChannel.Stable); saving = false; } diff --git a/frontend/src/lib/mappers/ConfigMapper.test.ts b/frontend/src/lib/mappers/ConfigMapper.test.ts index 83eb4a4a0..458edf9f3 100644 --- a/frontend/src/lib/mappers/ConfigMapper.test.ts +++ b/frontend/src/lib/mappers/ConfigMapper.test.ts @@ -1,108 +1,76 @@ -import { BackendConfig } from '#lib/_fbs/open-shock/serialization/configuration/backend-config.js'; -import { CaptivePortalConfig } from '#lib/_fbs/open-shock/serialization/configuration/captive-portal-config.js'; -import { EStopConfig } from '#lib/_fbs/open-shock/serialization/configuration/estop-config.js'; -import { HubConfig } from '#lib/_fbs/open-shock/serialization/configuration/hub-config.js'; -import { OtaUpdateChannel } from '#lib/_fbs/open-shock/serialization/configuration/ota-update-channel.js'; -import { OtaUpdateConfig } from '#lib/_fbs/open-shock/serialization/configuration/ota-update-config.js'; -import { OtaUpdateStep } from '#lib/_fbs/open-shock/serialization/configuration/ota-update-step.js'; -import { RFConfig } from '#lib/_fbs/open-shock/serialization/configuration/rfconfig.js'; -import { SerialInputConfig } from '#lib/_fbs/open-shock/serialization/configuration/serial-input-config.js'; -import { WiFiConfig } from '#lib/_fbs/open-shock/serialization/configuration/wi-fi-config.js'; -import { WiFiCredentials } from '#lib/_fbs/open-shock/serialization/configuration/wi-fi-credentials.js'; +import { OtaChannel } from '#lib/_fbs/open-shock/serialization/local/ota-channel.js'; +import { OtaSettings } from '#lib/_fbs/open-shock/serialization/local/ota-settings.js'; +import { ReadyMessage } from '#lib/_fbs/open-shock/serialization/local/ready-message.js'; import { Builder, ByteBuffer } from 'flatbuffers'; import { describe, expect, it } from 'vitest'; import { mapConfig } from './ConfigMapper'; -interface Strings { - apSsid: string; - hostname: string; - domain: string; - cdnDomain: string; +function readBack(b: Builder, root: number): ReadyMessage { + b.finish(root); + return ReadyMessage.getRootAsReadyMessage(new ByteBuffer(b.asUint8Array())); } -// Builds a HubConfig the way the hub serializes it (every section present). -function buildHubConfig(strings: Strings): HubConfig { - const b = new Builder(512); +// Builds a Ready message the way the hub serializes it. +function buildReady(cdnDomain: string, ssids: string[]): ReadyMessage { + const b = new Builder(256); - const ssid = b.createString('HomeNet'); - WiFiCredentials.startWiFiCredentials(b); - WiFiCredentials.addId(b, 1); - WiFiCredentials.addSsid(b, ssid); - const cred = WiFiCredentials.endWiFiCredentials(b); - - const wifi = WiFiConfig.createWiFiConfig( - b, - b.createString(strings.apSsid), - b.createString(strings.hostname), - WiFiConfig.createCredentialsVector(b, [cred]), - 0, - false, - false, - false - ); - const rf = RFConfig.createRFConfig(b, 15, true); - const portal = CaptivePortalConfig.createCaptivePortalConfig(b, false); - const backend = BackendConfig.createBackendConfig(b, b.createString(strings.domain), 0); - const serial = SerialInputConfig.createSerialInputConfig(b, true); - const ota = OtaUpdateConfig.createOtaUpdateConfig( + const domain = b.createString(cdnDomain); + const ota = OtaSettings.createOtaSettings(b, true, domain, OtaChannel.Beta, 120, false, true); + const saved = ReadyMessage.createSavedSsidsVector( b, - true, - b.createString(strings.cdnDomain), - OtaUpdateChannel.Beta, - true, - false, - 30, - true, - false, - 0, - OtaUpdateStep.None + ssids.map((ssid) => b.createString(ssid)) ); - const estop = EStopConfig.createEStopConfig(b, true, 13, false, false); - HubConfig.startHubConfig(b); - HubConfig.addRf(b, rf); - HubConfig.addWifi(b, wifi); - HubConfig.addCaptivePortal(b, portal); - HubConfig.addBackend(b, backend); - HubConfig.addSerialInput(b, serial); - HubConfig.addOtaUpdate(b, ota); - HubConfig.addEstop(b, estop); - HubConfig.finishHubConfigBuffer(b, HubConfig.endHubConfig(b)); - - return HubConfig.getRootAsHubConfig(new ByteBuffer(b.asUint8Array())); + ReadyMessage.startReadyMessage(b); + ReadyMessage.addRfTxPin(b, 15); + ReadyMessage.addEstopEnabled(b, true); + ReadyMessage.addEstopGpioPin(b, 13); + ReadyMessage.addSavedSsids(b, saved); + ReadyMessage.addOta(b, ota); + return readBack(b, ReadyMessage.endReadyMessage(b)); } describe('mapConfig', () => { - it('returns null for a missing config', () => { - expect(mapConfig(null)).toBeNull(); + it('maps every setting', () => { + expect(mapConfig(buildReady('fw.example.org', ['HomeNet', 'Cafe']))).toEqual({ + rf: { txPin: 15 }, + estop: { enabled: true, gpioPin: 13 }, + otaUpdate: { + isEnabled: true, + cdnDomain: 'fw.example.org', + updateChannel: OtaChannel.Beta, + checkInterval: 120, + allowBackendManagement: false, + requireManualApproval: true, + }, + savedSSIDs: ['HomeNet', 'Cafe'], + }); }); - it('maps every section', () => { - const config = mapConfig( - buildHubConfig({ - apSsid: 'OpenShock-', - hostname: 'hub', - domain: 'api.example.org', - cdnDomain: 'fw.example.org', - }) - ); + it('accepts an empty domain and no saved networks', () => { + const config = mapConfig(buildReady('', [])); + + expect(config.otaUpdate.cdnDomain).toBe(''); + expect(config.savedSSIDs).toEqual([]); + }); + + it('reads an unset pin as -1', () => { + const b = new Builder(64); + const domain = b.createString('fw.example.org'); + const ota = OtaSettings.createOtaSettings(b, true, domain, OtaChannel.Stable, 30, true, false); + ReadyMessage.startReadyMessage(b); + ReadyMessage.addOta(b, ota); + + const config = mapConfig(readBack(b, ReadyMessage.endReadyMessage(b))); - expect(config?.rf).toEqual({ txPin: 15, keepaliveEnabled: true }); - expect(config?.wifi.hostname).toBe('hub'); - expect(config?.wifi.credentials).toEqual([{ id: 1, ssid: 'HomeNet', password: null }]); - expect(config?.backend.domain).toBe('api.example.org'); - expect(config?.otaUpdate.cdnDomain).toBe('fw.example.org'); - expect(config?.otaUpdate.updateChannel).toBe(OtaUpdateChannel.Beta); - expect(config?.estop).toEqual({ enabled: true, gpioPin: 13 }); + expect(config.rf.txPin).toBe(-1); + expect(config.estop.gpioPin).toBe(-1); }); - it('accepts empty strings the hub can legitimately store', () => { - const config = mapConfig( - buildHubConfig({ apSsid: '', hostname: '', domain: '', cdnDomain: '' }) - ); + it('rejects a message without the OTA settings', () => { + const b = new Builder(64); + ReadyMessage.startReadyMessage(b); - expect(config?.wifi.apSsid).toBe(''); - expect(config?.backend.domain).toBe(''); - expect(config?.otaUpdate.cdnDomain).toBe(''); + expect(() => mapConfig(readBack(b, ReadyMessage.endReadyMessage(b)))).toThrow('ready.ota'); }); }); diff --git a/frontend/src/lib/mappers/ConfigMapper.ts b/frontend/src/lib/mappers/ConfigMapper.ts index 3e38f3f19..99e9dfe4d 100644 --- a/frontend/src/lib/mappers/ConfigMapper.ts +++ b/frontend/src/lib/mappers/ConfigMapper.ts @@ -1,198 +1,50 @@ -import type { OtaUpdateChannel } from '#lib/_fbs/open-shock/serialization/configuration.js'; -import { HubConfig } from '#lib/_fbs/open-shock/serialization/configuration/hub-config.js'; - -// TODO: Update these configs and ensure that typescript enforces them to be up to date - -export interface RFConfig { - txPin: number; - keepaliveEnabled: boolean; -} - -export interface WifiCredentials { - id: number; - ssid: string; - password: string | null; -} - -export interface WifiConfig { - apSsid: string; - hostname: string; - credentials: WifiCredentials[]; -} - -export interface CaptivePortalConfig { - alwaysEnabled: boolean; -} - -export interface BackendConfig { - domain: string; - authToken: string | null; -} - -export interface SerialInputConfig { - echoEnabled: boolean; -} +import type { OtaChannel } from '#lib/_fbs/open-shock/serialization/local/ota-channel.js'; +import type { ReadyMessage } from '#lib/_fbs/open-shock/serialization/local/ready-message.js'; export interface OtaUpdateConfig { isEnabled: boolean; cdnDomain: string; - updateChannel: OtaUpdateChannel; - checkOnStartup: boolean; + updateChannel: OtaChannel; checkInterval: number; allowBackendManagement: boolean; requireManualApproval: boolean; } -export interface EStopConfig { - enabled: boolean; - gpioPin: number; -} - +// The settings the portal shows, as the hub reports them in its Ready message. export interface Config { - rf: RFConfig; - wifi: WifiConfig; - captivePortal: CaptivePortalConfig; - backend: BackendConfig; - serialInput: SerialInputConfig; + rf: { txPin: number }; + estop: { enabled: boolean; gpioPin: number }; otaUpdate: OtaUpdateConfig; - estop: EStopConfig; -} - -function mapRfConfig(hubConfig: HubConfig): RFConfig { - const rf = hubConfig.rf(); - if (!rf) throw new Error('hubConfig.rf is null'); - - const txPin = rf.txPin(); - const keepaliveEnabled = rf.keepaliveEnabled(); - - return { - txPin, - keepaliveEnabled, - }; -} - -function mapWifiConfig(hubConfig: HubConfig): WifiConfig { - const wifi = hubConfig.wifi(); - if (!wifi) throw new Error('hubConfig.wifi is null'); - - const apSsid = wifi.apSsid(); - const hostname = wifi.hostname(); - - if (apSsid === null) throw new Error('wifi.apSsid is null'); - if (hostname === null) throw new Error('wifi.hostname is null'); - - const credentials: WifiCredentials[] = []; - const credentialsLength = wifi.credentialsLength(); - for (let i = 0; i < credentialsLength; i++) { - const cred = wifi.credentials(i); - if (!cred) throw new Error('wifi.credentials is null'); - - const id = cred.id(); - const ssid = cred.ssid(); - const password = cred.password(); - - if (!id) throw new Error('cred.id is null'); - if (ssid === null) throw new Error('cred.ssid is null'); - - credentials.push({ - id, - ssid, - password, - }); - } - - return { - apSsid, - hostname, - credentials, - }; -} - -function mapCaptivePortalConfig(hubConfig: HubConfig): CaptivePortalConfig { - const captivePortal = hubConfig.captivePortal(); - if (!captivePortal) throw new Error('hubConfig.captivePortal is null'); - - const alwaysEnabled = captivePortal.alwaysEnabled(); - - return { - alwaysEnabled, - }; -} - -function mapBackendConfig(hubConfig: HubConfig): BackendConfig { - const backend = hubConfig.backend(); - if (!backend) throw new Error('hubConfig.backend is null'); - - const domain = backend.domain(); - const authToken = backend.authToken(); - - if (domain === null) throw new Error('backend.domain is null'); - - return { - domain, - authToken, - }; -} - -function mapSerialInputConfig(hubConfig: HubConfig): SerialInputConfig { - const serialInput = hubConfig.serialInput(); - if (!serialInput) throw new Error('hubConfig.serialInput is null'); - - const echoEnabled = serialInput.echoEnabled(); - - return { - echoEnabled, - }; + savedSSIDs: string[]; } -function mapOtaUpdateConfig(hubConfig: HubConfig): OtaUpdateConfig { - const otaUpdate = hubConfig.otaUpdate(); - if (!otaUpdate) throw new Error('hubConfig.otaUpdate is null'); - - const isEnabled = otaUpdate.isEnabled(); - const cdnDomain = otaUpdate.cdnDomain(); - const updateChannel = otaUpdate.updateChannel(); - const checkOnStartup = otaUpdate.checkOnStartup(); - const checkInterval = otaUpdate.checkInterval(); - const allowBackendManagement = otaUpdate.allowBackendManagement(); - const requireManualApproval = otaUpdate.requireManualApproval(); +function mapOtaUpdateConfig(ready: ReadyMessage): OtaUpdateConfig { + const ota = ready.ota(); + if (!ota) throw new Error('ready.ota is null'); - if (cdnDomain === null) throw new Error('otaUpdate.cdnDomain is null'); + const cdnDomain = ota.cdnDomain(); + if (cdnDomain === null) throw new Error('ota.cdnDomain is null'); return { - isEnabled, + isEnabled: ota.enabled(), cdnDomain, - updateChannel, - checkOnStartup, - checkInterval, - allowBackendManagement, - requireManualApproval, + updateChannel: ota.channel(), + checkInterval: ota.checkInterval(), + allowBackendManagement: ota.allowBackendManagement(), + requireManualApproval: ota.requireManualApproval(), }; } -function mapEstopConfig(hubConfig: HubConfig): EStopConfig { - const estop = hubConfig.estop(); - if (!estop) throw new Error('hubConfig.estop is null'); - - const enabled = estop.enabled(); - const gpioPin = estop.gpioPin(); - - return { - enabled, - gpioPin, - }; -} - -export function mapConfig(hubConfig: HubConfig | null): Config | null { - if (!hubConfig) return null; +export function mapConfig(ready: ReadyMessage): Config { + const savedSSIDs: string[] = []; + for (let i = 0; i < ready.savedSsidsLength(); i++) { + savedSSIDs.push(ready.savedSsids(i)); + } return { - rf: mapRfConfig(hubConfig), - wifi: mapWifiConfig(hubConfig), - captivePortal: mapCaptivePortalConfig(hubConfig), - backend: mapBackendConfig(hubConfig), - serialInput: mapSerialInputConfig(hubConfig), - otaUpdate: mapOtaUpdateConfig(hubConfig), - estop: mapEstopConfig(hubConfig), + rf: { txPin: ready.rfTxPin() }, + estop: { enabled: ready.estopEnabled(), gpioPin: ready.estopGpioPin() }, + otaUpdate: mapOtaUpdateConfig(ready), + savedSSIDs, }; } diff --git a/frontend/src/lib/stores/HubStateStore.svelte.ts b/frontend/src/lib/stores/HubStateStore.svelte.ts index 0bef1f21b..3aa07c18f 100644 --- a/frontend/src/lib/stores/HubStateStore.svelte.ts +++ b/frontend/src/lib/stores/HubStateStore.svelte.ts @@ -48,15 +48,15 @@ export class HubStateStore { // eslint-disable-next-line svelte/prefer-svelte-reactivity -- rebuilt on every change, never mutated afterwards Array.from(this.wifiNetworks.entries()).reduce(ssidMapReducer, new Map()) ); - // Saved SSIDs from config that aren't visible in scan results + // Saved SSIDs that aren't visible in scan results savedOnlySSIDs = $derived.by(() => { // eslint-disable-next-line svelte/prefer-svelte-reactivity -- local to this derivation const scannedSavedSSIDs = new Set(); for (const [, group] of this.wifiNetworkGroups) { if (group.saved) scannedSavedSSIDs.add(group.ssid); } - const creds = this.config?.wifi?.credentials ?? []; - return creds.map((c) => c.ssid).filter((ssid) => !scannedSavedSSIDs.has(ssid)); + const saved = this.config?.savedSSIDs ?? []; + return saved.filter((ssid) => !scannedSavedSSIDs.has(ssid)); }); accountLinked = $state(false); diff --git a/main/CMakeLists.txt b/main/CMakeLists.txt index a9df7370f..89bbb6e0a 100644 --- a/main/CMakeLists.txt +++ b/main/CMakeLists.txt @@ -4,7 +4,7 @@ # in app_main. idf_component_register( SRCS "main.cpp" - REQUIRES core captive_portal ota serial serial_console device_control events logging common chipset nvs_flash config migration + REQUIRES core captive_portal ota serial serial_console device_control events logging common chipset nvs_flash config ) target_compile_options(${COMPONENT_LIB} PRIVATE -Wno-error) diff --git a/main/main.cpp b/main/main.cpp index 24fab0eeb..e84853a26 100644 --- a/main/main.cpp +++ b/main/main.cpp @@ -12,7 +12,6 @@ const char* const TAG = "main"; #include "events/Events.h" #include "GatewayConnectionManager.h" #include "Logging.h" -#include "migration/StorageMigration.h" #include "OtaUpdateManager.h" #include "serial/Serial.h" #include "serial_console/SerialInputHandler.h" @@ -146,11 +145,6 @@ extern "C" void app_main() appSetup(); } - // After setup, so a migration only ever starts from a firmware that came up (and, after an OTA, validated itself) - if (!OpenShock::Migration::Start()) { - OS_LOGE(TAG, "Unable to start the storage migration"); - } - while (true) { OpenShock::GatewayConnectionManager::Update(); diff --git a/partitions/legacy/ota_4mb_v1.csv b/partitions/legacy/ota_4mb_v1.csv deleted file mode 100644 index 6d656b613..000000000 --- a/partitions/legacy/ota_4mb_v1.csv +++ /dev/null @@ -1,15 +0,0 @@ -# OpenShock 1.x partition table (every board, whatever its flash size). NOT built any more. -# -# Kept as the reference for what fielded 1.5.x / 1.6.x hubs carry and as a fixture for the -# partition-table tests (components/partition_layout/host_test): the storage migrations recognise -# this layout and retype it into partitions/ota_4mb.csv, keeping every offset and size. The config -# migration retypes its littlefs `config` into the `cfg` NVS partition; the www migration writes the -# www image over its littlefs `static0` and retypes it into the `www` partition. -# Name, Type, SubType, Offset, Size, Flags -nvs, data, nvs, 0x009000, 0x005000, -otadata, data, ota, 0x00E000, 0x002000, -app0, app, ota_0, 0x010000, 0x1A0000, -app1, app, ota_1, 0x1B0000, 0x1A0000, -config, data, spiffs, 0x350000, 0x003000, -static0, data, spiffs, 0x353000, 0x09D000, -coredump, data, coredump, 0x3F0000, 0x010000, diff --git a/partitions/ota_4mb.csv b/partitions/ota_4mb.csv index 9efc6fea0..7b447dc1a 100644 --- a/partitions/ota_4mb.csv +++ b/partitions/ota_4mb.csv @@ -1,10 +1,8 @@ # OpenShock partition table, for every board whatever its flash size. # -# The geometry of the 1.x table (partitions/legacy/ota_4mb_v1.csv), entry for entry: the storage -# migrations rewrite the table in place and move nothing. Two 1.x littlefs partitions change, in type -# and label only: `config` becomes `cfg`, the NVS config store, and `static0` becomes `www`, the flat -# read-only image of the captive-portal assets (type 0x40 so no ESP-IDF component ever tries to mount -# it; the firmware looks it up by type). See docs/STORAGE.md. +# `cfg` is the NVS config store and `www` the flat read-only image of the captive-portal assets (type +# 0x40 so no ESP-IDF component ever tries to mount it; the firmware looks it up by type). See +# docs/STORAGE.md. # Name, Type, SubType, Offset, Size, Flags nvs, data, nvs, 0x009000, 0x005000, otadata, data, ota, 0x00E000, 0x002000, diff --git a/partitions/single_4mb.csv b/partitions/single_4mb.csv deleted file mode 100644 index 42a5ccae1..000000000 --- a/partitions/single_4mb.csv +++ /dev/null @@ -1,9 +0,0 @@ -# CURRENTLY NOT USED - KEPT FOR REFERENCE -# OpenShock 4MB Partition Table - without OTA -# Name, Type, SubType, Offset, Size, Flags -# nvs, data, nvs, 0x009000, 0x005000, -# otadata, data, ota, 0x00e000, 0x002000, -# app0, app, ota_0, 0x010000, 0x340000, -# config, data, spiffs, 0x350000, 0x003000, -# static0, data, spiffs, 0x353000, 0x09D000, -# coredump, data, coredump, 0x3F0000, 0x010000, \ No newline at end of file diff --git a/pyproject.toml b/pyproject.toml index bf216132b..cd888fdb6 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -30,5 +30,5 @@ check_untyped_defs = true exclude = ['/tests/'] [[tool.mypy.overrides]] -module = ['esptool', 'esptool.*', 'littlefs', 'requests'] +module = ['esptool', 'esptool.*', 'requests'] ignore_missing_imports = true diff --git a/requirements.in b/requirements.in index e1a699359..72d3d1ac0 100644 --- a/requirements.in +++ b/requirements.in @@ -1,8 +1,5 @@ cryptography==50.0.2 esptool==5.6.0 -# scripts/gen_staticfs.py --legacy-output: the littlefs staticfs image a 1.x hub's OTA installs and checks by -# mounting it, published for 1.x-numbered releases (the 1.9.x pivot) only. Goes away with release/1.9. -littlefs-python==0.19.0 # .github/workflows run scripts/ci_build_gate.py, get_vars.py and repo_server_*.py against the GitHub and repository-server APIs requests==2.34.2 semver==3.1.0 diff --git a/requirements.txt b/requirements.txt index 60efab423..5d3ebf1c1 100644 --- a/requirements.txt +++ b/requirements.txt @@ -369,77 +369,6 @@ intelhex==2.3.0 \ --hash=sha256:87cc5225657524ec6361354be928adfd56bcf2a3dcc646c40f8f094c39c07db4 \ --hash=sha256:892b7361a719f4945237da8ccf754e9513db32f5628852785aea108dcd250093 # via esptool -littlefs-python==0.19.0 \ - --hash=sha256:012a4117d9e07376100907ae737ee35b48d58e394a17b00f3cbcd83d24d353bb \ - --hash=sha256:020e2cc8b08e4246b41d8413146e8193c84e779e8e10e5ff08d6e60250022331 \ - --hash=sha256:0864448636264eee6e2673118c711ad22e0663b6a6c63d2a95c05ee212d9c1fa \ - --hash=sha256:0965a06332177694681c1485a564d0ede4958258a649f45a4ed409c98ad2ab2c \ - --hash=sha256:0aaae1347d5a05da5e582b0d4c241d70b2736d01e5dbc58f9ae39ac659fc8393 \ - --hash=sha256:0e5ca23e6f1938d92ea75f7ce2409b77e09480e7b523cac35b2bdbf6c957305e \ - --hash=sha256:0fcf1e8c35e63d399a89f55fd2dad370ed312ea8584aba2f5fea691dabe2db72 \ - --hash=sha256:109dbfdd1bd381567d7f831aabcb279a09ffd85c80e9c696434395fa83fb7606 \ - --hash=sha256:140bcd3e34896e90dae524cc4762d97378cbd0e3b473a06829d6144e6ca9dd5b \ - --hash=sha256:15007c78d1bb6b1a06ec0e7c662dd9e7469a06a5134fadd67bcd9812223905a2 \ - --hash=sha256:1586da7b9c6a4508479395322378fb36ca021e586cef2d2fe4b4b3f26c4e3c04 \ - --hash=sha256:19d436537ace9f01d0f43b0453242d3aad3132abf97c5265302f03d84cc56482 \ - --hash=sha256:1bd7dc438f27d6b3d6ec9a9210740653eefa1239745d0c9da230554d8bed9470 \ - --hash=sha256:2c6df3da27680b08350f898c3a66ebddce15f1892f6aa007f9ca5f36d262aa76 \ - --hash=sha256:2c6fc43c47003b8a394d9639b89c82bbe234cf9fdc1f835be4f30b5e4ba05d49 \ - --hash=sha256:30b3c6c8f7e85526747eb8188c2b8ccf572ac1701c7ecf4949b1231de0ced510 \ - --hash=sha256:4058e125fa7b44f91327ecf57db18f3a0d67d9015a67d4579e2e683870d33627 \ - --hash=sha256:4a7be934562a5e3053646ce5bb0a98dea8361b97b8aa06bc830c18d49f79a8aa \ - --hash=sha256:4ce8c6eeeb2703abaf8cb60f2dab624b17af42b13df3dbc6c42d4142bad79a59 \ - --hash=sha256:51425e64a4e37bcbaaa2c99e12ffbbb1d81f96618bb824ac68c938757ed853d3 \ - --hash=sha256:53edd72cbce541939beec404a5509f4afc08ef694759201ffe91aff8adc5f254 \ - --hash=sha256:53f19918ad27c7fec890eb08d2f0d5649cf55c2c179e2777c4ea2ed958ddbe67 \ - --hash=sha256:5421c47b5bdd43d4cb855910263ee8884c2b6b0a779dfab6f31e1e27604a93d0 \ - --hash=sha256:54945e95faaf1edaf7bd554ea13889ad2ab786e213d0fca114385ee16de2c54d \ - --hash=sha256:55343a36ccbf389d927f2e07ed7097b3b65f3e87b0416a670d5cae0f6131b5a8 \ - --hash=sha256:5675b86b67a12a9764924525ddc15ef82b688714d4093ce696e082253ae2e790 \ - --hash=sha256:58c37bc1d4d845af40a2456f1850359dd35325001f52208b04fcc07068e2e5fb \ - --hash=sha256:58e0db33a7481cd85e4e7074959a4df39bc4f76426596fcab7462913ad13f288 \ - --hash=sha256:58e8eb151741747447c953286c35743d82a51527bc500d61678823abdf14b6b5 \ - --hash=sha256:5a9a84ac0a4da2e37f9bfed0ecf70d2a8c57063000a39e93d1e4ab247643e7bd \ - --hash=sha256:5eef1580bc4a1678b5777ace3a646cf064bcb282114eea8c4a251c3704d2affd \ - --hash=sha256:6051ff3ece097e21eb301693ca93e7bceac55cbe378ca14dfdd948a08dfe9866 \ - --hash=sha256:62b7b9ff5ee463bf0359e2bfd04231cc06550481a16a50c79bb3d7af4bd77a4c \ - --hash=sha256:643ffb360cedfde6a310da09ae806d030dd1769ef78bd79afe09eb345c4acccc \ - --hash=sha256:65a415e6efa597091bc292f8e6b7ffb7b96d4e7209cbbf9c3b856df18fb5934a \ - --hash=sha256:6a5a799297570241487bdf42de82fa8d238966f1af502f4f4f5b32bfc4d74676 \ - --hash=sha256:6d59cb8a72912cbf7f10a3f88f881e8cef625945b3259083f69b71816087c617 \ - --hash=sha256:711fa5c5748ba24cf3178ccff632357e4f93fb28901ceab6927e9c03a6b37e43 \ - --hash=sha256:71252f91d871357d2581473367840806c6dc05427e965e100f4b1433ae402c62 \ - --hash=sha256:7eb56074fbef683a923908091cae60fb3554f9bdaf82db92811e5bae8675b2cd \ - --hash=sha256:870f9b7c6d309bbd6d4ad95757c1af39376a5ba3c43800cab8c0933f5ad6da34 \ - --hash=sha256:89905592c4f509375d7d3d1a0187ceda57db74d8371093ccf42d5db6f190b10f \ - --hash=sha256:8abdaa1c78cb665c6cad6c8c5f0893d702af86587aadd7f1bc0442f61c742a19 \ - --hash=sha256:903bea7431e625f05cf76babcdf9bd39fcee7e12ebae43284711af9424c5f3c7 \ - --hash=sha256:947b3590e995d022051509c58c310550478f47074dca58479217b5938d5b7128 \ - --hash=sha256:986360d0fe75a123f156b5d6009cbc100ed2d2b966bada9b180ce7ff0faa43ee \ - --hash=sha256:9a409b071260aa0006b773c17528f8e2c91cf115799bddf6951e0e07b99b359e \ - --hash=sha256:9cefa021649f760979657a414874f88292317effdd38dfbdd7edac7800ea514e \ - --hash=sha256:9e18b5359f24e6189d2869ae40297311d073ebad3be68cc83df6643b3a1089ba \ - --hash=sha256:a27c00ad37a5ad79ad1b7dd72dd18d82b01b1deebe6aba50527424f13f16cc43 \ - --hash=sha256:b64cc280bb53a0d93a8ccc4ab6ec3e90f53bbd06c6d7f318567cdaac26968046 \ - --hash=sha256:c170794e550c18853f2ea1767df526aa879c27c963064cd88920994430cc6ffb \ - --hash=sha256:c197db6623142b33f484709445d04902c4c6af91c686d065f703d6ae94309fa2 \ - --hash=sha256:c8d017ff77bd7435bdacb4d94e7f7c53aceab0bed5ada73940be9ed5ba4e205e \ - --hash=sha256:c99b0ec49e9b840b2b4b143bc60272a9bed24eaf5187d166b2d54b09aac19d39 \ - --hash=sha256:ccfa7e423d55b2645095a8abf56d4f46ae932c04e19ecd9e08d9e1191e5603cf \ - --hash=sha256:d054ac280541afdae5722fb8d7ebc0e121d822c33d95d57889fd8fab2a211486 \ - --hash=sha256:d15020333abda47819d9d11d6fabe13eb64b83abeb9ae2d93c07a142c53f6aa4 \ - --hash=sha256:d63e077eecd9982f0d3816ee0c95dd3b6c3e514e54272a4e80154df7f320e342 \ - --hash=sha256:d643f7d5c4adf601e500b8958deb99d6f8bae914cf6c6be14e3cad047888506f \ - --hash=sha256:e006ad20d0af696967b55ff6d74a5ce1a0ecce5ba5a6a0f8030d56b0a6f6e92f \ - --hash=sha256:ec8d3de4f05d9420d0e1d013d9134cc68b2b6c15656115a7add2f1f9f5a655be \ - --hash=sha256:f155325d95e95690b39ec716e20b87b1c5a1a09a91cde158208aaa6277bea896 \ - --hash=sha256:f5125a8d763129808d52e874501e45100b7fdd619584e558674507202a1f1b38 \ - --hash=sha256:f526d9216cc9dfccb81b9d7d990d9a2e9cee62261ec608bea8465e951006bd56 \ - --hash=sha256:f889f3d1839df23fda7c69a3b4d9a3fa1083330ab03b121448b71f51442c6d63 \ - --hash=sha256:f8ecde73f89f4bd8edf764f5c422a40aef68bc401ce3465ced10cd0525bd23f3 \ - --hash=sha256:fa1280eeb6d32b026dea65e5078d5af0fadccb26dbf709c073d0e5729270a4b7 \ - --hash=sha256:fb9c65c235d33e10b9caa06d3cc37e396cc10f85d0ed7a4de7bfdb35f1c94814 - # via -r requirements.in markdown-it-py==4.2.0 \ --hash=sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49 \ --hash=sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a diff --git a/schemas b/schemas index 94ee44ad5..477013546 160000 --- a/schemas +++ b/schemas @@ -1 +1 @@ -Subproject commit 94ee44ad59c0ec77506a5d17e1603666a5d2e6ff +Subproject commit 4770135465cd342c51dd5cee68004b6321fee40f diff --git a/scripts/gen_dep_graph.py b/scripts/gen_dep_graph.py index 4021f50ea..79573c2f4 100644 --- a/scripts/gen_dep_graph.py +++ b/scripts/gen_dep_graph.py @@ -50,10 +50,8 @@ LABEL_OVERRIDES = { 'osjson': 'OSJson', 'http': 'HTTP', - 'fs': 'FS', 'dns_server': 'DnsServer', 'rfc8908': 'RFC8908', - 'hwutil': 'HwUtil', } diff --git a/scripts/gen_staticfs.py b/scripts/gen_staticfs.py index 9ddfd3853..b9974e0db 100644 --- a/scripts/gen_staticfs.py +++ b/scripts/gen_staticfs.py @@ -4,17 +4,12 @@ Pipeline: 1. Build the SvelteKit frontend (pnpm) when its sources are newer than the last build (unless --skip-frontend-build, which CI passes). 2. Gzip every built asset (the frontend build emits plain files; compression happens here). - 3. Pack them into one read-only image (scripts/utils/wwwimage.py), checked to fit the smallest `www` partition. + 3. Pack them into one read-only image (scripts/utils/wwwimage.py), checked to fit the `www` partition. The output is reproducible: gzip headers carry no timestamp and entries are sorted, so an unchanged frontend gives a byte-identical www.bin (and the same hash, so devices don't re-flash it). One image serves every board: it is not padded to any partition size, only checked to fit. -With --legacy-output it also packs the same assets as a littlefs image of the 1.x `static0` partition -(partitions/legacy/ota_4mb_v1.csv, under /www/.gz). That is what a 1.x hub's OTA installs as the `staticfs` -artifact and verifies by mounting it, so it is published for 1.x-numbered releases (the 1.9.x pivot) only; it goes -away with release/1.9. - Invoked by CMake (see the root CMakeLists.txt), and by CI with --skip-frontend-build. """ @@ -24,22 +19,11 @@ import os import shutil import subprocess -import sys from collections.abc import Iterable from pathlib import Path from utils import partitions, wwwimage from utils.partitions import WWW_PARTITION -from utils.paths import ROOT - -# The littlefs geometry the 1.x firmware (and this firmware on the 1.x layout) mounts static0 with (components/fs/src/ -# LfsPartition.cpp). Only the block size is on disk and checked at mount; the rest are runtime buffer sizes. -LEGACY_PARTITION = 'static0' -LEGACY_TABLE = ROOT / 'partitions' / 'legacy' / 'ota_4mb_v1.csv' -LFS_BLOCK_SIZE = 4096 -LFS_READ_SIZE = LFS_PROG_SIZE = 128 -LFS_CACHE_SIZE = 512 -LFS_LOOKAHEAD_SIZE = 128 # Inputs of the frontend build; a change to any of them makes frontend/build stale. FRONTEND_INPUTS = ( @@ -143,70 +127,19 @@ def pack_image(entries: list[wwwimage.Entry], output: Path, max_size: int) -> No ) -def pack_legacy_image(entries: list[wwwimage.Entry], output: Path, partition_size: int) -> None: - """The assets as the 1.x littlefs image: /www/.gz, exactly partition_size bytes.""" - try: - from littlefs import LittleFS - except ImportError: - raise SystemExit( - f'error: littlefs-python is not installed for {sys.executable}: the build scripts need an interpreter ' - 'with requirements.txt installed (docs/DEVELOPING.md, Python dependencies)' - ) from None - - if partition_size % LFS_BLOCK_SIZE != 0: - raise SystemExit(f'error: partition size {partition_size} is not a multiple of block size {LFS_BLOCK_SIZE}') - fs = LittleFS( - block_size=LFS_BLOCK_SIZE, - block_count=partition_size // LFS_BLOCK_SIZE, - read_size=LFS_READ_SIZE, - prog_size=LFS_PROG_SIZE, - cache_size=LFS_CACHE_SIZE, - lookahead_size=LFS_LOOKAHEAD_SIZE, - ) - for entry in entries: - path = f'www/{entry.path}.gz' - fs.makedirs(path.rsplit('/', 1)[0], exist_ok=True) - with fs.open(path, 'wb') as f_out: - f_out.write(entry.data) - output.resolve().parent.mkdir(parents=True, exist_ok=True) - output.write_bytes(bytes(fs.context.buffer)) - log(f'Packed {len(entries)} assets into the 1.x littlefs image {output} ({partition_size} bytes)') - - -def legacy_partition_size() -> int: - return partitions.read_partitions(LEGACY_TABLE)[LEGACY_PARTITION]['size'] - - def main(argv: list[str] | None = None) -> int: p = argparse.ArgumentParser(description='Build the OpenShock captive-portal asset image (www.bin)') p.add_argument('--frontend-dir', type=Path, required=True, help='Path to the frontend/ project') p.add_argument('--output', type=Path, required=True, help='Output image path (www.bin)') - size = p.add_argument_group('size limit (at least one)') - size.add_argument( - '--max-size', type=lambda v: int(v, 0), help=f'largest image allowed ({WWW_PARTITION} partition size)' - ) - size.add_argument( - '--partition-csv', - type=Path, - action='append', - default=[], - help=f'partition table CSV to read the {WWW_PARTITION} size from; repeatable, the smallest wins', - ) p.add_argument( - '--skip-frontend-build', action='store_true', help='Reuse existing frontend/build without invoking pnpm' + '--partition-csv', type=Path, required=True, help=f'partition table CSV to read the {WWW_PARTITION} size from' ) p.add_argument( - '--legacy-output', - type=Path, - help=f'also write the assets as the 1.x littlefs {LEGACY_PARTITION} image here (1.x-numbered releases)', + '--skip-frontend-build', action='store_true', help='Reuse existing frontend/build without invoking pnpm' ) args = p.parse_args(argv) - limits = [www_partition_size(csv) for csv in args.partition_csv] - if args.max_size is not None: - limits.append(args.max_size) - if not limits: - p.error('one of --max-size or --partition-csv is required') + max_size = www_partition_size(args.partition_csv) if args.skip_frontend_build: build_dir = args.frontend_dir / 'build' @@ -217,9 +150,7 @@ def main(argv: list[str] | None = None) -> int: entries = collect_assets(build_dir) log(f'Collected {len(entries)} gzipped assets from {build_dir}') - pack_image(entries, args.output, min(limits)) - if args.legacy_output is not None: - pack_legacy_image(entries, args.legacy_output, legacy_partition_size()) + pack_image(entries, args.output, max_size) return 0 diff --git a/scripts/repo_server_publish.py b/scripts/repo_server_publish.py index b4ef6247e..bba6f9428 100644 --- a/scripts/repo_server_publish.py +++ b/scripts/repo_server_publish.py @@ -29,18 +29,10 @@ 'partitions': 'partition-table.bin', } STATICFS_FIELD = 'staticfs' -# What the staticfs field carries: a 1.x-numbered release (the 1.9.x pivot) is installed by 1.x firmware, whose OTA -# flashes staticfs into its littlefs static0 partition and checks it by mounting it, so it gets the littlefs image; -# every later release gets the flat www image. The build-staticfs action builds both. STATICFS_FILE = 'www.bin' -STATICFS_FILE_1X = 'staticfs.bin' MERGED_FIELD = 'merged' -def staticfs_file_for(version: str) -> str: - return STATICFS_FILE_1X if version.split('.', 1)[0] == '1' else STATICFS_FILE - - UPLOAD_RETRIES = 3 UPLOAD_BACKOFF_SECONDS = 5 CHUNK = 1 << 20 @@ -94,7 +86,7 @@ def sha256_of(path: Path) -> str: return digest.hexdigest() -def resolve_artifacts(artifacts: Path, board: str, version: str) -> dict[str, Path]: +def resolve_artifacts(artifacts: Path, board: str) -> dict[str, Path]: """Every artifact the server wants for one board, or a failure naming the first one missing.""" build = artifacts / 'firmware_boards' / board # The package job merges every board at once, so the merged binaries share a directory and are told apart by the board in their filename. @@ -104,7 +96,7 @@ def resolve_artifacts(artifacts: Path, board: str, version: str) -> dict[str, Pa fail(f'{board} is missing its {MERGED_FIELD} artifact (expected {artifacts}/firmware_merged/{merged_glob})') found = {field: build / filename for field, filename in BUILD_ARTIFACTS.items()} - found[STATICFS_FIELD] = artifacts / 'firmware_staticfs' / staticfs_file_for(version) + found[STATICFS_FIELD] = artifacts / 'firmware_staticfs' / STATICFS_FILE found[MERGED_FIELD] = merged[0] for field, path in found.items(): if not path.is_file(): @@ -112,14 +104,12 @@ def resolve_artifacts(artifacts: Path, board: str, version: str) -> dict[str, Pa return found -def upload_board( - session: requests.Session, server: str, release_id: str, board: str, artifacts: Path, version: str -) -> None: +def upload_board(session: requests.Session, server: str, release_id: str, board: str, artifacts: Path) -> None: """PUT one board's artifacts with a sha256 manifest. Retried on transport errors and 5xx, since a repeated PUT replaces rather than appends; init and publish are POSTs and are not retried. """ - paths = resolve_artifacts(artifacts, board, version) + paths = resolve_artifacts(artifacts, board) manifest = {field: sha256_of(path) for field, path in paths.items()} url = f'{server}/2/firmware/releases/{release_id}/boards/{board}' @@ -259,9 +249,8 @@ def do_init(session: requests.Session, server: str) -> int: def do_publish(session: requests.Session, server: str) -> int: """Upload every board into the release init opened, then promote it; discard it on any failure.""" - require_env('RELEASE_ID', 'VERSION') + require_env('RELEASE_ID') release_id = env('RELEASE_ID') - version = env('VERSION') boards = read_boards() artifacts = Path('artifacts') @@ -270,7 +259,7 @@ def do_publish(session: requests.Session, server: str) -> int: try: for board in boards: - upload_board(session, server, release_id, board, artifacts, version) + upload_board(session, server, release_id, board, artifacts) promote(session, server, release_id) except PublishOutcomeUnknown: # Discarding now could delete a release that did go live; leave it for a human to check. diff --git a/scripts/tests/test_build_cli.py b/scripts/tests/test_build_cli.py index 29dc81ea7..4e9ae3e7b 100644 --- a/scripts/tests/test_build_cli.py +++ b/scripts/tests/test_build_cli.py @@ -87,9 +87,9 @@ def test_the_build_hands_its_own_interpreter_to_cmake(idf): def test_missing_requirements_stop_the_build_before_idf_runs(idf, monkeypatch): - monkeypatch.setattr(build.pyreqs, 'check', lambda path=None: [build.pyreqs.Pin('littlefs-python', '0.19.0')]) + monkeypatch.setattr(build.pyreqs, 'check', lambda path=None: [build.pyreqs.Pin('esptool', '0.19.0')]) - with pytest.raises(SystemExit, match=r'lacks 1 of the pins in requirements.txt: littlefs-python==0.19.0'): + with pytest.raises(SystemExit, match=r'lacks 1 of the pins in requirements.txt: esptool==0.19.0'): build.main(['OpenShock-Core-V2']) assert idf == [] diff --git a/scripts/tests/test_build_scripts.py b/scripts/tests/test_build_scripts.py index 9b66e5f02..d53e7777d 100644 --- a/scripts/tests/test_build_scripts.py +++ b/scripts/tests/test_build_scripts.py @@ -169,50 +169,3 @@ def test_packed_image_is_the_bare_image_and_must_fit(tmp_path): with pytest.raises(SystemExit, match='more than the 100-byte www partition'): gen_staticfs.pack_image(entries, output, 100) - - -def test_main_takes_the_smallest_of_every_limit(tmp_path, monkeypatch): - write_build(tmp_path) - csv = tmp_path / 'small.csv' - csv.write_text('www, 0x40, 0x00, 0x3A8000, 0x1000\n', encoding='utf-8') - limits = [] - monkeypatch.setattr(gen_staticfs, 'pack_image', lambda entries, output, max_size: limits.append(max_size)) - - args = ['--frontend-dir', str(tmp_path), '--output', str(tmp_path / 'www.bin'), '--skip-frontend-build'] - gen_staticfs.main( - [ - *args, - '--max-size', - '0x2000', - '--partition-csv', - str(csv), - '--partition-csv', - str(ROOT / 'partitions' / 'ota_4mb.csv'), - ] - ) - assert limits == [0x1000] - - with pytest.raises(SystemExit): - gen_staticfs.main(args) # no limit at all - - -def test_the_legacy_image_is_littlefs_sized_for_static0(tmp_path): - from littlefs import LittleFS - - entries = gen_staticfs.collect_assets(write_build(tmp_path)) - output = tmp_path / 'staticfs.bin' - size = gen_staticfs.legacy_partition_size() - assert size == 0x9D000 - - gen_staticfs.pack_legacy_image(entries, output, size) - - image = output.read_bytes() - assert len(image) == size - fs = LittleFS( - block_size=gen_staticfs.LFS_BLOCK_SIZE, block_count=size // gen_staticfs.LFS_BLOCK_SIZE, disk_version=0 - ) - fs.context.buffer[:] = image - fs.mount() - assert sorted(fs.listdir('www')) == ['assets', 'index.html.gz'] - with fs.open('www/index.html.gz', 'rb') as f: - assert gzip.decompress(f.read()) == b'x' diff --git a/scripts/tests/test_ci_build_gate.py b/scripts/tests/test_ci_build_gate.py index 7bacb8eef..dec8fd42c 100644 --- a/scripts/tests/test_ci_build_gate.py +++ b/scripts/tests/test_ci_build_gate.py @@ -132,7 +132,7 @@ def test_nightly_with_unchanged_sha_skips(): 'components/rf/README.md', 'docs/flashing.png', 'LICENSE', - 'components/fs/data/.gitkeep', + 'components/www/data/.gitkeep', '.gitkeep', '.claude/settings.json', '.github/ISSUE_TEMPLATE/bug_report.yml', diff --git a/scripts/tests/test_gen_dep_graph.py b/scripts/tests/test_gen_dep_graph.py index a877f14db..5df15265f 100644 --- a/scripts/tests/test_gen_dep_graph.py +++ b/scripts/tests/test_gen_dep_graph.py @@ -66,12 +66,15 @@ def test_render_mermaid_labels_and_orders_nodes(): def test_main_writes_the_graph_of_a_components_dir(tmp_path, capsys): - for name, body in {'fs': 'idf_component_register(SRCS "x")', 'rf': 'idf_component_register(REQUIRES fs)'}.items(): + for name, body in { + 'http': 'idf_component_register(SRCS "x")', + 'rf': 'idf_component_register(REQUIRES http)', + }.items(): (tmp_path / name).mkdir() (tmp_path / name / 'CMakeLists.txt').write_text(body, encoding='utf-8') out = tmp_path / 'graph.mmd' assert gen_dep_graph.main(['--components-dir', str(tmp_path), '-o', str(out)]) == 0 - assert out.read_text(encoding='utf-8') == 'flowchart TD\n A(FS)\n B(Rf)\n A --> B\n' + assert out.read_text(encoding='utf-8') == 'flowchart TD\n A(HTTP)\n B(Rf)\n A --> B\n' assert 'wrote' in capsys.readouterr().err diff --git a/scripts/tests/test_pyreqs.py b/scripts/tests/test_pyreqs.py index 26eeba1f0..58b3181dd 100644 --- a/scripts/tests/test_pyreqs.py +++ b/scripts/tests/test_pyreqs.py @@ -12,7 +12,7 @@ # via cryptography colorama==0.4.6 ; sys_platform == 'win32' \\ --hash=sha256:cccc -littlefs-python==0.19.0 \\ +esptool==0.19.0 \\ --hash=sha256:dddd pycparser==3.11 ; python_full_version < '3.11' and implementation_name != 'PyPy' \\ --hash=sha256:eeee @@ -27,7 +27,7 @@ def test_parse_pins_reads_name_version_and_marker_and_skips_hash_lines(): assert [str(p) for p in pins] == [ 'cffi==2.1.1', 'colorama==0.4.6', - 'littlefs-python==0.19.0', + 'esptool==0.19.0', 'pycparser==3.11', 'pycparser==3.12', 'pyserial==3.5', @@ -38,7 +38,7 @@ def test_parse_pins_reads_name_version_and_marker_and_skips_hash_lines(): def test_missing_pins_honours_markers_and_versions(monkeypatch): - installed = {'cffi': '2.1.1', 'littlefs-python': '0.18.0', 'pycparser': '3.12', 'pyserial': '3.5'} + installed = {'cffi': '2.1.1', 'esptool': '0.18.0', 'pycparser': '3.12', 'pyserial': '3.5'} monkeypatch.setattr(pyreqs, 'installed_version', installed.get) # Pretend: CPython 3.12 on Linux, so colorama and pycparser 3.11 do not apply. holds = { @@ -50,7 +50,7 @@ def test_missing_pins_honours_markers_and_versions(monkeypatch): monkeypatch.setattr(pyreqs, 'marker_holds', holds.__getitem__) missing = pyreqs.missing_pins(pyreqs.parse_pins(PINNED)) - assert [str(p) for p in missing] == ['littlefs-python==0.19.0'] + assert [str(p) for p in missing] == ['esptool==0.19.0'] def test_marker_evaluation_uses_the_real_environment(): diff --git a/scripts/tests/test_repo_server_publish.py b/scripts/tests/test_repo_server_publish.py index af2f96ffc..c67ad4741 100644 --- a/scripts/tests/test_repo_server_publish.py +++ b/scripts/tests/test_repo_server_publish.py @@ -87,7 +87,6 @@ def write_board_artifacts(artifacts, board, version='1.5.0'): (build / filename).write_bytes(filename.encode()) (artifacts / 'firmware_staticfs').mkdir(exist_ok=True) (artifacts / 'firmware_staticfs' / 'www.bin').write_bytes(b'www') - (artifacts / 'firmware_staticfs' / 'staticfs.bin').write_bytes(b'littlefs') (artifacts / 'firmware_merged').mkdir(exist_ok=True) (artifacts / 'firmware_merged' / f'OpenShock_{board}_{version}.bin').write_bytes(b'merged') @@ -96,29 +95,20 @@ def test_resolve_artifacts_reads_the_package_layout(tmp_path): write_board_artifacts(tmp_path, 'BoardA') write_board_artifacts(tmp_path, 'BoardA_Mini') - found = repo_server_publish.resolve_artifacts(tmp_path, 'BoardA', '1.5.0') + found = repo_server_publish.resolve_artifacts(tmp_path, 'BoardA') for field, filename in repo_server_publish.BUILD_ARTIFACTS.items(): assert found[field] == tmp_path / 'firmware_boards' / 'BoardA' / filename - # A 1.x-numbered release is installed by 1.x firmware, which only takes the littlefs image - assert found[repo_server_publish.STATICFS_FIELD] == tmp_path / 'firmware_staticfs' / 'staticfs.bin' + assert found[repo_server_publish.STATICFS_FIELD] == tmp_path / 'firmware_staticfs' / 'www.bin' # A board whose name extends another's does not pick up the other's merged image. assert found[repo_server_publish.MERGED_FIELD] == tmp_path / 'firmware_merged' / 'OpenShock_BoardA_1.5.0.bin' -def test_a_2x_release_publishes_the_www_image(tmp_path): - write_board_artifacts(tmp_path, 'BoardA', version='2.0.0') - found = repo_server_publish.resolve_artifacts(tmp_path, 'BoardA', '2.0.0') - assert found[repo_server_publish.STATICFS_FIELD] == tmp_path / 'firmware_staticfs' / 'www.bin' - assert repo_server_publish.staticfs_file_for('1.9.3') == 'staticfs.bin' - assert repo_server_publish.staticfs_file_for('10.0.0') == 'www.bin' - - def test_resolve_artifacts_fails_on_a_missing_image(tmp_path, capsys): write_board_artifacts(tmp_path, 'BoardA') (tmp_path / 'firmware_boards' / 'BoardA' / 'bootloader.bin').unlink() with fails_with(capsys, 'BoardA is missing its bootloader artifact'): - repo_server_publish.resolve_artifacts(tmp_path, 'BoardA', '1.5.0') + repo_server_publish.resolve_artifacts(tmp_path, 'BoardA') def test_resolve_artifacts_fails_without_a_merged_image(tmp_path, capsys): @@ -126,7 +116,7 @@ def test_resolve_artifacts_fails_without_a_merged_image(tmp_path, capsys): for merged in (tmp_path / 'firmware_merged').iterdir(): merged.unlink() with fails_with(capsys, 'BoardA is missing its merged artifact'): - repo_server_publish.resolve_artifacts(tmp_path, 'BoardA', '1.5.0') + repo_server_publish.resolve_artifacts(tmp_path, 'BoardA') @pytest.mark.parametrize( @@ -177,34 +167,34 @@ def real_upload(monkeypatch, tmp_path): def test_upload_retries_a_5xx_then_succeeds(real_upload): session = ScriptedSession(503, requests.ConnectionError('reset'), 200) - repo_server_publish.upload_board(session, 'https://repo', 'rel-1', 'BoardA', real_upload, '1.5.0') + repo_server_publish.upload_board(session, 'https://repo', 'rel-1', 'BoardA', real_upload) assert [url for url, _ in session.calls] == ['https://repo/2/firmware/releases/rel-1/boards/BoardA'] * 3 def test_upload_gives_up_after_the_last_5xx(real_upload, capsys): session = ScriptedSession(500, 500, 500) with fails_with(capsys, r'Upload failed for BoardA \(HTTP 500\)'): - repo_server_publish.upload_board(session, 'https://repo', 'rel-1', 'BoardA', real_upload, '1.5.0') + repo_server_publish.upload_board(session, 'https://repo', 'rel-1', 'BoardA', real_upload) assert len(session.calls) == repo_server_publish.UPLOAD_RETRIES def test_upload_does_not_retry_a_4xx(real_upload, capsys): session = ScriptedSession(422) with fails_with(capsys, r'Upload failed for BoardA \(HTTP 422\)'): - repo_server_publish.upload_board(session, 'https://repo', 'rel-1', 'BoardA', real_upload, '1.5.0') + repo_server_publish.upload_board(session, 'https://repo', 'rel-1', 'BoardA', real_upload) assert len(session.calls) == 1 def test_upload_gives_up_after_repeated_transport_errors(real_upload, capsys): session = ScriptedSession(*[requests.ConnectionError('down')] * repo_server_publish.UPLOAD_RETRIES) with fails_with(capsys, 'Upload failed for BoardA after 3 attempts: down'): - repo_server_publish.upload_board(session, 'https://repo', 'rel-1', 'BoardA', real_upload, '1.5.0') + repo_server_publish.upload_board(session, 'https://repo', 'rel-1', 'BoardA', real_upload) assert not session.outcomes def test_upload_sends_a_sha256_manifest_of_every_artifact(real_upload): session = ScriptedSession(200) - repo_server_publish.upload_board(session, 'https://repo', 'rel-1', 'BoardA', real_upload, '1.5.0') + repo_server_publish.upload_board(session, 'https://repo', 'rel-1', 'BoardA', real_upload) kwargs = session.calls[0][1] manifest = json.loads(kwargs['data']['sha256']) assert set(manifest) == set(kwargs['files']) == {*repo_server_publish.BUILD_ARTIFACTS, 'staticfs', 'merged'} diff --git a/test/host_support/CMakeLists.txt b/test/host_support/CMakeLists.txt index d58dc30cb..f495e3014 100644 --- a/test/host_support/CMakeLists.txt +++ b/test/host_support/CMakeLists.txt @@ -19,8 +19,8 @@ # # Everything else a suite needs comes from the real OpenShock headers, ESP-IDF's # linux-target components, or the CMock mocks in $IDF_PATH/tools/mocks. Fakes that -# isolate one suite from a dependency (device_control's config/Config.h, config's -# in-memory fs/ConfigFs.h) stay in that suite's main/stubs. +# isolate one suite from a dependency (device_control's estop/EStopConfig.h and +# radio/RFConfig.h, www's Hashing.h) stay in that suite's main/stubs. idf_component_register( SRCS "src/TestBoardDescriptor.cpp" # the test board's descriptor "../../components/common/src/BoardDescriptor.cpp" # Board::Init() / Get() / Parse() over it