diff --git a/RELEASE.md b/RELEASE.md deleted file mode 100644 index 0e1778b9e..000000000 --- a/RELEASE.md +++ /dev/null @@ -1,125 +0,0 @@ -# PiFinder v2.6.1 Release Notes - -A substantial point release built on field feedback from v2.6.0: a rebuilt Focus screen that shows you the actual stars, a self-calibrating Sky Quality Meter that finally holds steady through a night, chart markers for the objects around you, full software enablement for rev4 hardware (battery, sound, power button, 176px display), and a fix for a bug that silently killed plate solving on any unit you'd ever SSH'd into. - -## ⚠️ After You Update - -- **SQM readings are on a new scale.** The Sky Quality Meter was rebuilt around raw-sensor photometry with per-frame black-level tracking. Values are no longer comparable with readings logged under v2.6.0 or earlier. The SQM screen will also show as **uncalibrated** after updating — the calibration file is now keyed to the camera rather than the processed image. A wizard run is no longer required to get a reading, but re-running `SQM → CALIB` (long-press for the marking menu) restores the per-camera dark-current refinement. -- **Observed status is now a property of the object, not the listing.** Logging M 31 marks NGC 224 observed, and your existing log history is applied retroactively. Expect an "Observed: No" list to visibly shrink on first use, sibling rows to gain checkmarks, and a sibling's details to show the combined log count. This is intended. -- **Set Time/Date now needs a location fix first.** Manual time is interpreted in the observer's timezone, which only exists once a location is known. The screen still opens; it shows "Set location first" and you back out with LEFT. -- **Some labels changed.** The Type filter now uses the same labels as the object-detail screen (`P. Nebula` → `Planetary`, `Double Str` → `Double star`, `Cluster/Neb` → `Cluster + Neb`, …). On the Status screen, `GPS LST` is now `GPS LCK` (last GPS lock — it was being misread as Local Sidereal Time). The SQM marking menu reads `CALIB` and `SWEEP` (was `CAL` / `CORRECT`). -- **A one-time system migration runs on update** (`v2.6.1`), adding a `RemoveIPC=no` drop-in for systemd-logind. See the shared-memory fix below for why. -- **If you're running a rev4 board with a pre-release config**, the `v4_left` / `v4_right` / `v4_straight` screen-direction values are automatically migrated to `rev4_*`. No action needed. - -## New Features - -### Rev4 Hardware Enablement (#498, #530, #539) -Full software support for the fourth-revision PiFinder board, all gated on runtime hardware detection so **rev3 and dev boards are unaffected**: - -- **Battery telemetry** — BQ25895 charger read over I²C, with a title-bar battery icon (level buckets plus a charging bolt) and an idempotent fast-charge configuration re-asserted on every poll. -- **Sound** — an earcon subsystem on the rev4 passive buzzer, with a new `Settings → User Pref → Volume` setting (Off, 1–5). Delivery is best-effort and latest-wins, so audio never blocks the UI. -- **Power button** — a dedicated power key that opens a shutdown confirmation; a second press confirms, and the shutdown earcon plays before the GPIO14 power latch fires. -- **SSD1333 176×176 display** — rotation fixes, a resolution-flexible boot splash, and the 5-column rev4 keypad matrix with its directional cluster. -- **New build variants** in `Settings → Advanced → PiFinder Type`: **AS Heart**, **Rev4 Left**, **Rev4 Right**, and **Rev4 Straight**, joining AS Bloom. Each carries the IMU-to-camera frame constants for that mounting, derived with a new visual `imu2cam` tool and pinned to the production tables by test. - -### Low-Battery Safety (#541, #549) -On battery-equipped hardware, PiFinder now warns and then shuts itself down cleanly rather than running blind into an SD-corrupting hard power cut. Advisory popups (plus an earcon) fire once at **10%** and once at **5%** state of charge, and when the charger's ADC goes blind at ~3.5 V for four consecutive polls on battery, PiFinder shows a final warning and performs an orderly software shutdown. Warnings latch for the whole discharge — only plugging in a charger re-arms them — so ADC quantisation noise near the flat knee of the discharge curve can't re-fire them (field logs showed the first implementation firing ~90 times a run). The state-of-charge curve was re-anchored on bench discharge data with 0% at the shutdown point. - -### Rebuilt Focus Screen (#531) -The Focus screen now finds the **four brightest stars** in the frame and shows each magnified in its own quadrant, holding its quadrant as the image shifts under your hand. Tiles are raw camera pixels — crop and nearest-neighbour enlargement only, no sharpening or contrast tricks — so what you see is honestly how tight your stars are. The **HFD** readout sits at the center with a 10-second trace along the divider, and **SQUARE** cycles four views: - -- **Stars** — the four magnified tiles -- **Single** — the brightest star alone at double magnification -- **Image** — the full camera frame -- **Stats** — HFD with an FWHM estimate, detected-star count, exposure, gain, and a raw histogram - -**+/−** adjusts magnification between 4x and 16x, and a badly defocused star automatically gets a wider view so its donut isn't clipped. None of this needs a plate solve, so it works however far out of focus you start. - -### Self-Calibrating Sky Quality Meter (#532, #544, #560, #561) -The SQM wandered 0.7–2.4 magnitudes within a night as auto-exposure moved, and shifted with focus and sensor swaps. It was rebuilt end to end: - -- **Radiometer-first publication** — the published value now comes from the raw diffuse sky background, exposure, and a per-sensor radiometric zero point. It **no longer needs a plate solve**, so it keeps updating through failed solves and star-poor or cloudy frames. -- **Raw photometry** — measurement moved off the clipped 8-bit processed image onto the linear raw frame (Bayer green, or the mono frame). Plate solving is untouched. -- **Tracked black level** — a per-session estimator joint-fits the true black level from the auto-exposure stream, replacing a static pedestal constant that never matched the sensor. -- **A zero point that follows the sky's colour** — the radiometer measures sky in the *sensor's* passband while a reference meter measures V, and converting between them depends on the sky's spectrum: light pollution is sodium/LED and green-weighted, airglow is grey and NIR-rich. One constant per camera was therefore wrong at one end of the range or the other — the shipped value read ~0.1 mag dark at a light-polluted site and ~0.85 mag bright at a dark one. On bare sensors (imx462/imx290) the zero point is now a measured function of the frame's own red/green ratio, clamped to the calibrated range rather than extrapolated. Mono sensors and the IR-cut HQ keep a plain constant, both refit. -- **Colour and wing corrections** — a per-sensor B−V colour term (sensors without an IR-cut over-flux red stars) and a measured aperture wing correction that tracks how the star halo grows with dew and seeing. -- **Scale-aware geometry** — photometry radii are now expressed in the photometry image's pixel pitch, which fixes the imx296 (whose full-res frame put the aperture inside the star profile and left the wing estimator permanently inert). -- **Calibration wizard fixes** — captures now gate on the sensor's *actual* reported exposure with a re-capture on stale frames, and discard optical-black clamp-settle frames after each exposure change. - -Stability, measured across 19 referenced exposure sweeps on four devices: within-sweep swing went from 0.7–2.4 mag to a 0.07–0.22 standard deviation, and focus sensitivity from large to 0.03 mag. Absolute accuracy against hand-held reference meters, re-derived over the full 23-sweep imx462 archive: median residual +0.005 mag with the spread collapsing from −0.23…+1.08 to −0.18…+0.16, and leave-one-night-out cross-validated error 0.247 → 0.108. The HQ's refit constant lands on a median of +0.000. - -Some of the calibration is still thin, and says so in the code: the imx462 colour model's dark-site anchor is a single night, the imx296 rests on four sweeps from one observer, and the imx290 constants are mirrored from the imx462 with no sweeps of its own. Reference-meter comparisons from more sites and sensors are the thing that improves this — the `SQM → SWEEP` archive is what those refits are derived from. - -### Chart Markers: Target Cross and Nearby Objects (#513) -The chart's target cross had lost all its callers in an earlier refactor and never drew; nearby catalog objects were never plotted at all, so a fresh session with no observing list loaded showed a bare chart. Both are fixed: - -- The **last object you opened in details** is marked with a bright cross, labelled with its designator when on-screen and replaced by a rim arrow when it drifts off. The cross stays bright even with DSO Display turned off. -- **Nearby deep-sky objects** that pass your active filters are now plotted, with a **zoom-scaled magnitude limit** (mag 11 at 5° FOV to mag 7 at 60°) so the field reveals fainter objects as you zoom in and never crowds when you zoom out. Capped at the brightest 20, deduped against observing-list markers. - -### Observing List Import: CSV and Stellarium 2.0 (#510, #527) -- **CSV is now genuinely importable from other tools.** Headers are matched case-insensitively through an alias table (`name`, `ra`/`ra_deg`, `dec`/`de`, `mag`/`magnitude`/`vmag`, …) in any column order, and coordinates parse as decimal degrees, `HH:MM:SS`, or sexagesimal. A decimal RA is degrees unless the header names hours (`RA_h`). Previously only PiFinder's own exact export format loaded — anything else silently imported as nameless objects at RA 0 / Dec 0. -- **Stellarium 2.0 lists import.** Current Stellarium builds export a new schema (lists nested under `observingLists`, whitespace-free RA strings, display-string object types); these silently produced an empty list. Both 1.0 and 2.0 now work, and object types that PiFinder doesn't recognise are coerced to `?` so they can't vanish behind the Type filter. - -### Bench Bring-Up Tool (#550, #551, #552, #556) -`python -m PiFinder.bringup` validates a freshly assembled board in one command, without booting the application — no catalogs, solver, camera, or menu system, so it runs on a card with nothing configured. It drives the screen, keypad backlight, and buzzer, interrogates the IMU and charger, and scans every populated switch in the keypad matrix. Checks are typed by what actually proves them — *probed*, *exercised*, or *witnessed* — and only the first two can gate the exit status, because a program cannot know whether sound came out of a buzzer. A pre-flight pass checks card provisioning first, so a misprovisioned image is never diagnosed as a bad board. - -Holding the power switch for one second ends the run and asks the OS for a clean shutdown — the same gesture the finished unit uses for the same switch, so a builder at a bench with no terminal in sight never has to pull power on a mounted filesystem. A tap still just registers the closure for the switch check, and the power cell fills a bar over the second so the boundary is visible and an accidental hold can be released. - -## Improvements - -- **SSD1333 dimming range** (rev4 displays): the panel's two current registers bottom out about 20x brighter than a red night-vision display wants at a dark site. Brightness now uses the gray scale ceiling as a third axis, taking the dimmest setting from 0.106% of full to 0.005% — a 13,400:1 range over 253 distinct levels, with the full 31 shades preserved at any setting at or above 0.35%. Full brightness is held to 70% of available current, measured as the point where bright pixels stop blooming. The 128×128 SSD1351 is unaffected. -- **Object type codes are single-sourced** (#511): the Type filter menu is generated from `OBJ_TYPES` rather than a hand-maintained duplicate list, with a drift guard that also covers the shipped default filter selection and the docs table. -- **Filtered lists stay fresh** (#528): logging an object now drops it from an "Observed: No" list on the next refresh, altitude-filter verdicts age out as the sky rotates (600 s TTL), and the arrival of the first GPS fix triggers a refilter — closing the boot-time gap where pre-lock verdicts let everything through forever. A list left open refreshes in place, and the cursor stays on target across refreshes, falling to the next surviving object rather than jumping to the top. -- **Timezone handling made explicit** (#508): shared state now always stores a UTC-aware datetime, with separate `utc_datetime()` and `local_datetime()` accessors so consumers can't accidentally read one as the other. This fixes the Status screen showing identical UTC and local times after manual entry, and a manual date entry that could be pre-filled 24 hours off. -- **Chinese is now actually selectable** (#562): the in-app Language menu has offered Chinese for some time and the translation shipped complete, but `zh` was missing from the validated language list in `main.py` and from the web server's accept-language match, so `--lang zh` raised "Unknown language 'zh'" and the web UI could never be served in Chinese. All four catalogs (de, es, fr, zh) are now at full coverage — 671 strings, none untranslated, none fuzzy — including the new rev4, Volume, low-battery, migration and telemetry-replay strings. Wordings were checked against the real fonts and the 128px layout budget, which caught six overflows. Two caveats: much of the translation is machine-generated and tagged in the `.po` files as needing human review (native speakers very welcome), and Chinese marking-menu labels currently render as overlapping glyphs — a known renderer issue with CJK glyph widths, not a translation problem, and one whose fix changes menu geometry for every language. - -## Performance - -- **Object lists open instantly** (#526): opening any catalog list re-applied the filter across all ~151k objects even when you only wanted to view NGC's 7.8k — measured at ~0.58 s per open on a Pi, from 604,680 filter calls across four opens. Catalogs already filtered against the current criteria now return their cached list, taking repeat opens from ~0.58 s to microseconds. A changed filter parameter, a deferred background load, or a comet refresh all still force a full re-filter. - -## Bug Fixes - -- **Plate solving silently died after an SSH logout** (#548): systemd-logind's default `RemoveIPC=yes` deletes every POSIX shared-memory segment owned by the `pifinder` user the moment that user's last login session ends — and the PiFinder services hold no login session of their own. The solver's cedar-detect segment vanished, and the recovery path raised `FileNotFoundError` *before* disabling shared memory, so the designed inline-image fallback never engaged and **every subsequent solve failed until restart**. Device logs showed ~28,000 identical solver exceptions per night on two independent units. Fixed in three layers: the client now treats an already-gone segment as released (so even the frame that hits the error still gets centroids), fresh installs get a `RemoveIPC=no` drop-in, and existing units get the same drop-in via the `v2.6.1` migration. -- **Camera Test Mode did nothing** (#542, #543): a rename in the SQM work moved the test-mode flag's initialisation inside the capture loop (resetting it every frame) and deleted the command handler that toggled it, so `Tools → Test Mode` and console key 0 had no effect on Focus or Align (Day). -- **Shared-memory solving broken by a mismatched protobuf field** (#542): the same change passed a `reopen_shmem` field the checked-in cedar-detect bindings don't have, raising `ValueError` before any RPC — and because it wasn't a gRPC error, the inline fallback never engaged. Reverted to the known-working handoff. -- **DeepskyLog equipment import crashed** (#529): importing equipment containing any new eyepiece returned an Internal Server Error, and because the save came after the import loop, telescopes collected in the same request were lost too. A v2.5.1 → v2.6.0 regression from the Flask migration. -- **Web location entry failed in comma-decimal locales** (#536): typing `51,3` (or `51.3`) into the coordinate fields and clicking Save did nothing in much of Europe — a `type="number"` input returns an empty value for locale-mismatched text, so client-side validation cancelled the submit. Only integers saved. Inputs are now text with `inputmode="decimal"`, both separators normalise before validation, and the server tolerates a comma and returns a friendly message instead of a 500. -- **GPS satellite counts were wrong on every receiver generation** (#524, #563): on protVer < 15 receivers (NEO-6/7, early M8) the per-satellite fields were parsed one byte off, so satellite IDs were channel numbers and the "seen" count was inflated to near the full channel count — the mysterious `20/0`. On protVer ≥ 15 receivers (later M8, M9, M10) the used count stayed 0 forever because NAV-PVT's `numSV` was parsed but never surfaced. Live-verified against a real MAX-M8 against gpsd's own decode. A follow-up pass fixed three more ways the counts could go wrong: a single NAV-SAT message permanently silenced the NAV-SVINFO fallback for the life of the connection (now a 5 s freshness window, so SVINFO resumes if NAV-SAT stops), NAV-PVT/NAV-SOL could publish a used count with no seen count — the physically impossible `0/9` — and NAV-SVINFO counted "used" and "seen" over different populations, so used could exceed seen. -- **Polar Align guidance corrected** (#518): field reports reversed the original advice. Step 1 said to aim *well away* from the pole, but flexure — felt mostly in camera roll — grows with how much the PiFinder's attitude changes between captures, so moderate near-pole sweeps beat wide ones. -- **Manual time entry could localise against a bogus timezone** (#512, #563): the Set Time/Date screen now gates itself on a location fix and renders a notice instead, live — the entry boxes appear the instant a fix locks while the screen is open. A location whose timezone couldn't be resolved from its coordinates now settles to UTC at the source rather than being stored as nothing: previously the screen rendered but committing the time crashed, and the line that names the zone the digits are read against was silently dropped. -- **NixOS migration wifi never connected** (#521): the initramfs shell converter wrote SSIDs into NetworkManager keyfiles as hex byte lists, but NM's keyfile format only parses decimal — so every migrated device scanned for a network literally named `61;70;6f;…`. Keyfile generation moved to unit-tested Python running on Debian before reboot; the initramfs just copies the pre-staged files. -- **NixOS migration hardening** (#523): tarballs that can't fit in board RAM are rejected before boot configuration is touched, the previous boot configuration is restored automatically after any pre-format failure, formatting is treated as the explicit destructive boundary, and unexpected `set -e` exits surface as a visible migration failure instead of panicking the kernel. -- **SQM screen exposure display and calibration detection** (#544, and the calibration-path fix above). -- **SQM diagnostic sweeps were labelled one step behind** (#561): the exposure sweep flushed a fixed two frames after each change, but the IMX290/462 serves exactly three at the old exposure — so every sweep pair's processed image belonged to the *previous* step, and the radiometer sample recorded alongside it described that image rather than its label. A fixed count can't suit every sensor (the IMX477 emits half-exposure transitional frames instead), so the sweep now watches the driver's reported exposure and discards until it matches, with a cap and a warning if it never settles. Diagnostic archives only — the on-screen SQM was unaffected. - -## Hardware & Case - -- **Threaded lens holddown and lens-cap accessories** (`case/accessories/tethered_lens_cap/`), contributed for v2.5 kits: an M12×0.5 threaded holddown with loose and snug jam nuts for locking focus, PETG and TPU replacement lens caps, and a tethered TPU cap that folds back against the case so it can't be lost. -- **Meade LXD dovetail adapter** (`case/adapters/dovetail_meade_lxd.stl`), contributed by John Purdy — replaces the bottom half of the standard Vixen/Synta foot with an LXD-shoe profile. -- **Threaded camera holddown for v2.5** (`case/v2.5/v25_camera_holddown_threaded.stl`). -- The obsolete tall dovetail parts were removed from `case/accessories/`. - -## Documentation - -- **Focus screen rewritten** in the Quick Start, covering the four views, the magnified tiles, and the small-turn technique: the whole range from badly defocused to sharp is only a few lens turns, fair-to-good is under half a turn, and 1/8–1/4 turn at a time with a pause for vibration to settle is what actually works. Later-night touch-ups should be judged on the HFD readout, not the lagging camera icon (#546). -- **Polar Alignment: "Getting a good measurement"** — per-mount advice folded in from field reports. EQ mount: point the Dec axis 7–10° off the polar axis, sweep 30–45° in RA, use Roll Off. Platform: aim within ~5° of Polaris, keep Roll On. Realistic accuracy is 20–30 arcmin, down to ~10 with a very rigid connection. Plus: turn sidereal tracking off before starting, and lock *both* mount axes before touching the adjusters — a slipping RA clutch was the most common field failure (#518). -- **Chart markers and CSV import** documented in the user guide, including a full column reference, the three accepted coordinate forms, and how to keep your own coordinates for an object the catalog would otherwise resolve. -- **Observing-list format reference** expanded with nine worked CSV examples and a JSON schema for the native `.pifinder` format. -- Menu Map updated for the new Focus views and the rev4 build variants. - -## Developer Improvements - -- **New bounded contexts**: `docs/ax/battery/`, `docs/ax/sound/`, and `docs/ax/bringup/` glossaries, plus `docs/bq25895_design_notes.md`. -- **Thirteen new ADRs** covering read-only battery telemetry, the gpio-poweroff latch, best-effort sound delivery, battery fast-charge configuration, UTC-aware civil datetime storage, self-gating UI modules, SoC as a runtime fraction, blind-floor shutdown, the SQM redesign, radiometer-first publication, the sky-colour-keyed radiometric zero point, SSD1333 three-axis brightness, and filter freshness. -- **Import-safe keypad module** (#551): the matrix tables moved out of `keyboard_pi` (which imports `libinput` and `RPi.GPIO` at module scope) into `PiFinder.keypad`, with per-revision population maps derived from the keymap rather than hand-listed. -- **`imu2cam` derivation tool** (#530): a single-file visual tool for deriving per-variant IMU-to-camera frame constants in 3D, emitting the `(q_imu2cam, rotate_amount)` pair with human-checkable axis correspondences and a ready-made pytest block. Its presets are pinned to the production tables by test, so tool and code can't drift. -- **`-fb`/`--fakebattery`** (#535): `-fh` alone now emulates rev3 hardware (no battery process, no title-bar icon), keeping docs screenshots on the baseline UI; `-fb` opts into the fake rev4 battery monitor, which runs a full discharge lap including the low-battery warnings and blind-floor shutdown. -- **Battery runtime analysis tool** (`python/tools/battery_runtime_analysis.py`) for fitting the state-of-charge curve from bench discharge logs. -- **~450 new tests** (1,137 unit + smoke passing), including 60 covering the bring-up tool, 46 covering NetworkManager keyfile generation, 117 covering rev4 hardware logic, and field-data regression tests that replay real discharge telemetry through the low-battery warner. -- **The SQM calibration is re-derivable in-repo** (#560): `scripts/evaluate_radiometer_archive.py` now re-fits both the constant and colour models from the sweep archive and judges them by leave-one-night-out error rather than in-sample scatter, which a free parameter always wins. The whole radiometric model is recorded in sweep metadata, so an archived sweep is self-describing rather than needing a constant looked up by camera type. - ---- - -**Version**: 2.6.0 → 2.6.1 -**Commits**: 58 -**Files changed**: 242 (+26,012 / −5,692); Python: 135 files (+18,467 / −2,125) diff --git a/release_notes/RELEASE-PLAYBOOK.md b/release_notes/RELEASE-PLAYBOOK.md new file mode 100644 index 000000000..d6d0602dd --- /dev/null +++ b/release_notes/RELEASE-PLAYBOOK.md @@ -0,0 +1,456 @@ +# PiFinder release playbook + +The reusable process behind a release. The per-release artifacts (`release_notes/X.Y.Z.md`, +`release-X.Y.Z-test-plan.md`) stay one-off documents; this file is what you read before +writing them, so the same handful of things don't get rediscovered every cut. + +Draft status: sections marked TODO are gaps I couldn't answer from the repo or from our +previous sessions. Everything else is checked against what actually happened in 2.6.1 and +2.6.3, and against `main` on 2026-11-16. + +## Two tracks, one repo + +**Track A, the 2.x line.** This is the shipping product. Devices run Raspberry Pi OS and +update in place with `git pull` on the `release` branch. `ui/software.py` fetches +`https://raw.githubusercontent.com/brickbots/PiFinder/release/version.txt`, compares it to +the local version with `update_needed()`, and only then offers the update. Nothing about +this line is automated: the merge, the tag, the GitHub release and the SD image are all +done by hand. + +**Track B, the 3.x NixOS line.** Released by the "Release NixOS image" workflow +(`.github/workflows/release.yml`, `workflow_dispatch`). Its process, rules and known faults +are written down in `nixos/RELEASE.md`, which lives on the `nixos` branch of +`mrosseel/PiFinder` until #379 merges. This playbook points at that file and does not +restate it. The first draft of this playbook did restate it, and four of those statements +were wrong ten weeks later. + +Most of the playbook is Track A. Say which track you're cutting before you start, because the +answer changes who creates the tag. + +The two tracks meet in the migration: from 2.6.4 on, every 2.x release carries the code that +moves a unit to NixOS. See "Where the tracks meet" below. + +## Branch model + +`main` is integration. `release` is production. Code moves main into release at the cut, never +the other way. Feature branches PR into `main`. + +The cut is a merge, not a fast-forward, whenever `release` carries a commit `main` doesn't. +It always does by now: every earlier cut and every docs promotion left a merge commit there. +That is fine as long as the merge is clean and its tree equals `main`'s. Check before you +promise anyone a clean cut: + +```bash +git fetch origin +git merge-tree --write-tree origin/release origin/main; echo "exit=$?" +git rev-parse 'origin/main^{tree}' +git rev-list --count origin/release..origin/main # what's shipping +``` + +Exit 0 means a clean merge, exit 1 means conflicts, and the conflicted files are listed. The +first line of the output is the tree the merge would produce. It should equal `main`'s tree +on the next line. That way testing `main` is testing the cut. + +Don't count conflict markers in that output. An earlier version of this file piped it into +`grep -c '<<<<<<<'`. The command prints a tree id and a file list, never markers, so the grep +says 0 on a conflicted merge. It says 0 on the 2.6.3 cut, which had a conflict. + +When the only conflict is `version.txt`, resolve it to the new version and confirm the +resolved tree equals `main`'s. That was the 2.6.3 cut: `release` carried `b485e3be`, the +commit that paused 2.6.2 by reverting `version.txt`. Don't rebase or force-push `release` to +tidy history away. + +### Between cuts, promoting docs + +Read the Docs builds `/en/release/` from the `release` branch. A docs fix merged to `main` +is invisible to readers until it reaches `release`. Promote it out of cycle by merging `main` +into `release`, with no tag and no `version.txt` change, under one condition: + +```bash +git diff --stat origin/main origin/release # must be empty BEFORE the docs fix merged +``` + +If the trees already matched, the promotion carries only the new commits. If they didn't, the +promotion is a partial release cut: cherry-pick the docs commits, or wait for the real cut. + +A promotion moves everything on `main`, not only docs. #639 and #640 are code, and they +reached `release` this way after the `v2.6.3` tag. A unit that updated later, or a fresh +install, has them. No unit on 2.6.3 was ever offered them, because `version.txt` didn't +change. So "2.6.3" in the field is two trees: the tag, and the later `release` tip. Upgrade +gates for the next cut cover both. + +Don't commit to `release` directly. The case fix of 2026-10-23 went onto both branches as +two separate commits. It merges clean only because the files are identical. + +## Phase 1, prep on main + +Open a draft PR against `main` carrying the release notes and the test plan. 2.6.3 did this as +PR #636 in a worktree named `release-X.Y.Z-prep`. + +- [ ] Pick the version number. Skipping one is fine and sometimes right: 2.6.2 was cut for + internal rev4 units and never went public, so 2.6.3 folded it in rather than reusing the + number. Record the reason in the notes. +- [ ] Establish the range. Diff the last public tag against `main`, not the last cut and not + the `release` tip: `git log --oneline v2.6.3..origin/main`. The field population is on + the last public release, so anything internal or promoted since is still new to them. +- [ ] Write `release_notes/X.Y.Z.md`. Long form, the reference document. Lead with an "after + you update" section covering anything that changes under the user without being asked: + changed defaults, rescaled readings, renamed labels, retroactive data changes. +- [ ] Write `release-X.Y.Z-test-plan.md`. Risk-ordered, not feature-ordered. Each gate gets an + owner, a pass criterion, and a note on what a failure blocks. See the gate structure below. +- [ ] Note which items are *carried* from a previous plan rather than re-derived. The 2.6.3 + plan carried the whole optics tranche from 2.6.2 and said so. +- [ ] Add the previous release's docs snapshot to the "previous versions" notes (phase 5). + +## Phase 2, pre-flight on main + +This is the part that has actually bitten us. Work through it before the merge. + +- [ ] **`version.txt` on `main` reads the new version, and it's committed.** For 2.6.2 the bump + existed only as an uncommitted working-copy edit. Cutting like that means no device is ever + offered the update, and the screen just says "No Update needed" forever. Highest-leverage + item on this list precisely because it looks done. +- [ ] `update_needed()` opens the gate from every population in the field, not just the newest. + 2.6.3 had to satisfy both 2.6.1 devices and internal 2.6.2 ones. +- [ ] Merge shape verified with `git merge-tree --write-tree` (above). +- [ ] Know what the cut does to `.github/`: `git diff --stat origin/release origin/main -- .github/`. + A difference is not a failure, it is part of what ships. What matters is which copy a + hand-started workflow runs. While `release` is the GitHub default branch, the Actions + page and `gh workflow run` use the `release` copy unless you name a ref. Until the 2.6.4 + cut, that copy of `release.yml` is the old one. It writes `migration_url` to the update + manifest, which is the key 2.6.1 to 2.6.3 read. Those versions then run their old ext4 + migration on a btrfs tarball, and the card does not boot. That happened to a test card + in October. Until the cut, start that workflow with `--ref main` only. +- [ ] `migration_gate.json` on `main` reads what you intend to ship. Until the gate is opened + on purpose, that is both flags `false`. +- [ ] ADR ledger has no duplicate numbers. Gaps are expected: 0036 to 0045 are taken on the + NixOS line, so check the next free number against both `main` and the `nixos` branch. + Recompute the free slot at merge time and never trust a written-down reservation; they + went stale twice during the 2.6.3 cycle. +- [ ] i18n pass. `nox -s babel`, then confirm every catalog has 0 untranslated and 0 fuzzy, no + msgid was removed, and each `.mo` is byte-identical to a fresh `pybabel compile`. Note + `messages.pot` is gitignored, so scope the diff to the tracked `.po` files. New menu + entries shift menu indexes and break Selenium tests that navigate by counting keypresses; + that has happened twice. +- [ ] `default_config.json` diff since the last public tag. Every added key is a visible change + on an untouched upgrade. List them in the notes. +- [ ] New migration needed? Add `migration_source/vX.Y.Z.sh` **and** register it in + `pifinder_post_update.sh` with its own sentinel file under + `~/PiFinder_data/migrations/`. An unregistered script never runs. +- [ ] `requirements.txt` / `requirements_dev.txt` / `noxfile.py` changes staged on the image if + the SD image needs them baked in. +- [ ] `python/tetra3` is still untracked. Never re-track that symlink (ADR 0035). A tracked + version makes `git pull` refuse to update every in-place-upgraded unit in the field. + +## Phase 3, the gates + +The plan's shape, reused across 2.6.1, 2.6.2 and 2.6.3. Gates 1 to 3 must pass before the +merge; 4 to 6 before the update is offered. + +| Gate | What | Runs where | +|---|---|---| +| 1 | Automated: ruff lint + format, mypy, smoke + unit + UI tests, sphinx `-nW`, i18n | CI or dev box, ~15 min | +| 2 | Bench on standard hardware, the population that upgrades | Real device | +| 3 | The release-specific risk. Whatever this cut could break that nothing else covers | Real device | +| 4 | Upgrade paths and fresh install, from every version still in the field | Real device | +| 5 | Under the stars | Real device | +| 6 | Web, API, planetarium protocols | Dev box + phone | + +Baselines worth recording each time, because the trend catches regressions the suite doesn't: +smoke+unit count (1,238 at the 2.6.2 cut, 1,305 at #631, 1,531 at 2.6.3), msgid count +(694 × 4 catalogs at 2.6.3), sphinx page count. + +Two automation traps: clear the mypy cache first (`rm -rf .mypy_cache`), because a stale cache +reports phantom errors. And reset `language` to `en` in `~/PiFinder_data/config.json` before the +web suite, because the tests assert English titles and a device left in Chinese fails three of +them for no reason. + +## Phase 4, the cut + +- [ ] Merge `main` into `release`, resolve `version.txt` to the new version if it conflicts, + verify the tree matches `main`. +- [ ] Push `release`. The moment this lands, every device that checks for updates is offered + the new version. There is no staged rollout. +- [ ] Tag it. **Tag the commit `release` pointed at when the update opened**, and do it at the + cut rather than later. We've been inconsistent here: `v2.6.1` is on `8c6ae841`, which sits + on both branches, but `v2.6.3` ended up on `31c900c3`, two commits past the actual release + merge, because the tag went on afterwards. That makes "what shipped as 2.6.3" ambiguous. +- [ ] Push the tag. + +Tag by hand for Track A only. Expect to see NixOS tags on this branch all the same: +`v3.1.1-beta` to `v3.1.3-beta` sit on `release` commits, and `v3.1.3-beta` is on the case fix. +Those betas were built from `mrosseel/PiFinder`. The workflow tags the source commit only when +the source is this repo, and the release step then creates the tag on the default branch. So +`git describe` on `release` names a 3.x beta. That is noise, not a release of that commit. + +## Phase 5, snapshot the docs + +Every release gets a docs build frozen at its own URL, so a user on an old version reads the +manual that matches their device. + +We used to do this, stopped, and resumed with 2.6.3. `pifinder.readthedocs.io` serves +`v1.11.2`, `v2.0.4`, `v2.1.1` and `v2.6.3`. Nothing exists for 2.2 through 2.6.1, so anyone on +2.4 hitting `/en/release/` today reads rev4 docs describing hardware and menus they don't have. + +The mechanism is Read the Docs building a version per git tag. That's where `/en/v2.1.1/` came +from, and `/en/v3_hardware/` (referenced as `|v3_docs|` in `docs/source/conf.py`) is the same +trick applied to the pre-rev4 manual. No branch, no repo change, one dashboard action. + +- [ ] Once the tag from phase 4 is pushed, open the Read the Docs dashboard, find the new + version under Versions, and set it Active. +- [ ] Confirm the build succeeds and `https://pifinder.readthedocs.io/en/vX.Y.Z/` resolves. +- [ ] Activate only the `vX.Y.Z` tags you pushed for Track A. The `v3.x-beta` tags point at + Pi OS commits (phase 4), so a build of one is the 2.x manual under a 3.x name. +- [ ] The "previous versions" note in `docs/source/index.rst`, `docs/source/user_guide.rst` + and `docs/source/quick_start.rst` can only link a snapshot that already exists. So each + prep PR adds the *previous* release's snapshot and drops the oldest entry. This was + skipped for 2.6.3: the notes still list only 1.x, 2.0.x and 2.1.x. The 2.6.4 prep adds + `/en/v2.6.3/`. Those three notes are hand-maintained duplicates of each other, so all + three or none. +- [ ] Leave `/en/release/` as the URL everything links to. It follows the `release` branch and + is what the update instructions in the GitHub release body point at. It is also in the + web interface of every unit (`python/views/base.html`), so it has to resolve for ever. +- [ ] Bump `min_software` in `docs/source/conf.py` if the manual has stopped describing older + software. + +Since the tag is what RTD builds from, phase 4's rule about tagging the actual release commit +matters here too. A tag two commits past the cut produces a docs snapshot that never shipped. + +We are not backfilling 2.2 through 2.6.1. The gap stays; the note just needs to stop implying +otherwise (see one-time tasks). + +## Phase 6, build and publish the image + +TODO. The GitHub release for 2.6.3 points at +`https://ddbeeedxfpnp0.cloudfront.net/software/PiFinder_2.6.3.img`, so the 2.x image is built +and uploaded outside this repo. Nothing here describes how. Needs: how the image is built, what +gets baked in (preconfigured hardware settings were documented in #637), how it's uploaded, and +whether anything verifies the upload before the release body links to it. Write this section +while doing it for 2.6.4. + +## Phase 7, announce + +- [ ] Draft the GitHub release body from `release_notes/X.Y.Z.md`. It's a much shorter document + with a different voice, so write it fresh rather than trimming: check the last two or + three releases and match them. Both 2.6.1 and 2.6.3 did this as an explicit pass, with + the output staged in a scratch file first. +- [ ] Body includes the image URL and a link to the update instructions + (`https://pifinder.readthedocs.io/en/release/user_guide.html#update-software`). +- [ ] If a version was skipped, say so plainly near the top and say why. +- [ ] Publish the GitHub release against the tag. +- [ ] Post to Discord. TODO: which channel, and whether it's the full GitHub body or a shorter + note with a link. +- [ ] Post to the Cloudy Nights thread. TODO: thread URL, and whether it's a new post or a + reply on the standing thread. +- [ ] Email owners. TODO: where the list lives, who sends it, and what it contains beyond a + link. This one reaches people who never check GitHub, so it probably deserves the + "after you update" section verbatim rather than a link. + +Order matters a little: publish the GitHub release first, since the other three link to it. + +## Phase 8, after + +- [ ] Carry open review findings forward into the next cycle's notes rather than letting them + die with the plan. The Focus exposure hold lease fix and the stale `camera_lens` on + camera-type switch have both survived two releases this way. +- [ ] Update the release memory file so the next cut starts from the real branch state. +- [ ] Every place the cut deviated from this file becomes an edit to this file. +- [ ] Watch for field reports before starting the next tranche. Docs fixes between cuts go + through `main` and a promotion (see "Between cuts"). + +## Track B, the NixOS release + +Read `nixos/RELEASE.md` on the `nixos` branch of `mrosseel/PiFinder`. It is the process, and +it is kept current by the people who run it. What a Track A release manager needs from it: + +- The workflow creates the tag and the GitHub release. Don't tag a 3.x release by hand. +- Start the workflow from a ref that has the current `release.yml`. Since #658 that is `main`. + It is not the default ref until the 2.6.4 cut (phase 2). +- A beta goes to a cache that deletes it after 90 days. A stable release is kept for ever. +- Devices find a release through `update-manifest.json` on the `nixos-manifest` branch. + +## Where the tracks meet, the migration + +2.6.4 is the first Pi OS release that can migrate a unit to NixOS (#657). From it on, a 2.x +cut is also a release of the migration, so these belong to Track A. + +**The contract freezes when the release ships.** A unit on 2.6.4 reads the gate from +`migration_gate.json` on `release` (flag `nixos_migration`), finds its target in the update +manifest (keys `migration_v2_url` and `migration_v2_sha256_url`, lists read in the order +stable, beta, unstable), and expects one tarball layout. None of that can change for units +on 2.6.4 after they have it. A change needs a new 2.x release that reads a new key. +`nixos/RELEASE.md` holds the exact list. Read it before the cut, not after. + +**Two names stay dead for ever.** 2.6.3 and older read the flag `nixos_for_everyone` and the +manifest key `migration_url`, and then run an ext4 migration. Keep the flag `false` and never +write the key. + +**Seven presses of SQUARE ignore the gate.** On the Software screen of 2.6.4 they offer the +first available manifest entry that has a tarball, gate open or closed. So the migration is +reachable the day 2.6.4 ships. Gate 3 of a 2.x test plan therefore includes a real migration +from a card on that version: one Pi 4 unit and one rev4 (CM4) unit. On rev4, also power on, +reboot and power off afterwards. GPIO14 is the power-off line there, and one byte on the +serial port cuts the power. + +**Betas expire.** While the manifest has only betas, keep one that is newer than 90 days. +After that the cache has deleted the system, and a migration fails at its first NixOS boot. + +**Opening the gate is a release.** It is a one-line change to `migration_gate.json` on +`release`, and it gets the same care as a cut: + +- [ ] A stable NixOS release is in the manifest. Not only betas. +- [ ] A Pi 4 unit and a rev4 unit migrated on hardware, from a card on the current 2.x. +- [ ] The announcement is written, and it says what a user keeps and what they must do. +- [ ] The change goes through `main` and a promotion when the trees match. If they don't, it + is the one case for a pull request straight to `release`. + +One consequence to plan around. With the gate open, the Software screen on 2.6.4 or later +offers the migration before it reads `version.txt`, and it never shows a release version. A +later 2.x release cannot be offered in the app to those units while the gate is open. Units +on 2.6.3 or older still read `version.txt`, so they update to the newest 2.x first and migrate +second. This is from reading `ui/software.py`; nobody has tried it on a device yet. If a 2.x +hotfix is needed after the gate opens, close the gate first. + +### Decided for 2.6.4 (2026-11-16) + +- #697, the rebuilt catalog database, ships in 2.6.4. The cut waits for its on-device + checklist. +- No stable NixOS release before the 2.6.4 cut. The gate stays closed and both flags stay + `false`. Seven presses of SQUARE land on a beta, so the 90-day rule above applies from day + one. `v3.1.3-beta` was built on 2026-11-11. + +## The `release` branch after NixOS, a transition plan + +Proposed 2026-11-16. Nothing here has been done yet. + +`release` does four jobs today. Only the first is tied to the branch name. The plan keeps +that one for ever and moves the other three away, each at its own moment. + +| Job `release` does today | Tied to the name? | Plan | When | +|---|---|---|---| +| What Pi OS units pull: code, `version.txt`, `migration_gate.json` | Yes | Keep, for as long as one Pi OS unit exists | Never changes | +| GitHub default branch | No | Move the default to `main` | Step 1, after the 2.6.4 cut | +| Target of the `main` into `release` cut | No | Hotfixes only, by pull request to `release` | Step 2, when #379 merges | +| Published docs at `/en/release/` | The URL is, the branch is not | Redirect to `/en/stable/` | Step 3, first stable 3.x | + +Why the first job can't move: `pifinder_update.sh` runs `git checkout release`, and +`ui/software.py` reads `version.txt` and `migration_gate.json` from that branch by URL. Both +are on every unit in the field, and a unit runs its old copy to fetch the new one. + +### Step 1, default branch to `main` + +When: right after the 2.6.4 cut. The two branches hold the same tree then, so a visitor to +the repo sees no change. + +Safe because nothing depends on the default: `pifinder_setup.sh` clones with +`--branch release`, the update script names the branch, and so do the URLs. + +- [ ] Confirm `git diff --stat origin/main origin/release` is empty. +- [ ] Change the default branch in the GitHub repository settings. +- [ ] In each clone: `git remote set-head origin -a`. +- [ ] Remove the two notes in `CLAUDE.md` that tell agents to distrust the detected main + branch and to re-root worktrees. +- [ ] Delete `ci/btrfs-migration-release` once `nixos/RELEASE.md` starts releases from `main`. + +What it fixes: a hand-started workflow uses `main`'s copy without being told, `Closes #NNN` +closes the issue when the pull request merges, and new pull requests open against `main`. + +### Step 2, `release` becomes the Pi OS maintenance branch + +When: the day #379 (NixOS support) merges into `main`. That merge ends the Track A cut. On the +`nixos` branch `python/requirements.txt` and `migration_gate.json` do not exist, and +`ui/software.py` is the NixOS upgrade screen. Merge that into `release` and the update script +fails at `pip install -r`, and every 2.6.4 unit reads a missing gate file as a closed gate. + +So hold #379 until Pi OS can live on hotfixes. While Pi OS is the shipping product, the +current arrangement is the cheap one: the `nixos` branch merges `main` regularly, and Pi OS +gets every fix by the normal cut. After #379, each Pi OS fix is a hand-made backport. Until +then, keep landing the parts of the `nixos` branch that are plain app code on `main` as their +own pull requests, the way #657 and #658 landed, so the final merge is small. + +- [ ] Make the last normal cut from `main` before #379 merges. +- [ ] Protect `release`: no merges from `main`, changes by pull request only. +- [ ] A Pi OS hotfix is a pull request against `release`, with its own `version.txt` bump and + its own tag on `release`. Phases 2 to 4 of this file shrink to that. +- [ ] Rewrite the branch model in `CLAUDE.md`, which today forbids pull requests to `release`. +- [ ] Move `nixos/RELEASE.md` under this file's Track B, or point each at the other. +- [ ] Build NixOS releases from this repo from now on, so the workflow tags the commit it + built. Step 3 depends on that. + +### Step 3, docs follow the product + +When: the first stable 3.x release. Until then `/en/release/` is the right manual for almost +everyone, and it keeps building from `release`. + +Read the Docs cannot point the `release` address at another branch. It does document this +way to retire a version: + +- [ ] Activate the `stable` version. It is their built-in alias for the highest version tag + that is not a pre-release, so it skips `v3.x-beta` tags and moves to each new stable + release by itself. +- [ ] Check that `/en/stable/` shows the 3.x manual. It builds from the tag, so the tag must + be on the NixOS source commit (step 2, last item). +- [ ] Make `stable` the default version, so the bare address lands there. +- [ ] Deactivate the `release` version and add one exact redirect: + `/en/release/*` to `/en/stable/:splat`. The redirect works only once the old version is + off. +- [ ] Test `https://pifinder.readthedocs.io/en/release/user_guide.html#update-software` and + the User Guide link in a unit's web interface. + +The Pi OS manual stays available as the last `/en/v2.6.x/` snapshot from phase 5, and the +version note at the top of the 3.x manual links to it. A unit that has not migrated follows +its built-in link to the 3.x manual, so that note has to be easy to see. + +Deep links survive because the page names are the same on both lines today. Keep them the +same: don't rename a docs page on the NixOS line without adding a redirect for it. + +A manual pinned to a tag does not pick up docs fixes between releases. On NixOS a point +release is one workflow run, so that is the way to publish one. + +### Not decided + +- When Track A ends, meaning the last 2.x hotfix. `release` has to exist after that anyway, + for the units that never update. +- Whether #379 merges before the first stable 3.x or with it. This plan works either way; + the earlier it merges, the longer Pi OS runs on backports. + +## Traps worth re-reading before every cut + +1. `version.txt` bumped but not committed. Silent, total, and it looks done. +2. Re-tracking `python/tetra3`. Breaks `git pull` on every in-place-upgraded unit. +3. A migration script added but not registered in `pifinder_post_update.sh`. +4. New menu entries shifting menu indexes and breaking keypress-counting web tests. +5. Stale mypy cache reporting errors that aren't there. +6. Device left in a non-English language, failing web tests that assert English strings. +7. ADR number reservations going stale between writing and merging. +8. Tagging after the cut instead of at it. +9. Starting "Release NixOS image" from the default ref before the 2.6.4 cut. It writes the + manifest key that 2.6.3 reads. +10. Counting conflict markers in `git merge-tree` output. It always says 0. +11. A docs promotion that also carries code. It ships to fresh installs with no version bump. + +## One-time tasks, not yet done + +- [ ] Reword the "previous versions" note in `docs/source/index.rst`, + `docs/source/user_guide.rst` and `docs/source/quick_start.rst`. It currently offers 1.x, + 2.0.x and 2.1.x as if that were the full set, which reads as coverage rather than a + three-version archive. We're not backfilling 2.2 to 2.6.1, so the wording should say + what it is. Do it in the 2.6.4 prep PR, together with adding `/en/v2.6.3/`. +- [ ] `nixos/RELEASE.md` on the `nixos` branch still starts releases from + `ci/btrfs-migration-release`, lists the old workflow on `main` as a known fault (#658 + fixed it), says a release built from the fork makes no tag (it makes one, on `release`), + and points at the root `RELEASE.md` this PR removes. +- [ ] `CONTEXT-MAP.md` on `main` links to `docs/ax/nixos/CONTEXT.md` and `docs/ax/nixos.md`. + Both exist only on the `nixos` branch, so the links are dead until #379 or #497 lands. +- [x] Retire `RELEASE.md` at the repo root. Done 2026-09-05: it was a byte-identical copy of + `release_notes/2.6.1.md` with nothing in the repo referencing it. `release_notes/X.Y.Z.md` + has been the real home since 2.6.1, and there is now no root-level file claiming to + describe the current release. + +## Still open + +- Phase 6 is a stub. How does the SD image get built, what goes into it, and how does it reach + CloudFront? +- Phase 7's three announcement channels need their specifics filled in. +- The Software screen's behaviour with the gate open is read from code, not seen on a device. +- The two items under "Not decided" in the transition plan.