Skip to content

[Test build] Satellites, moving target follow and polar alignment with INDI mount control - #700

Draft
mrosseel wants to merge 791 commits into
brickbots:mainfrom
mrosseel:test/device-build
Draft

mrosseel wants to merge 791 commits into
brickbots:mainfrom
mrosseel:test/device-build

Conversation

@mrosseel

Copy link
Copy Markdown
Collaborator

Test build, not for merge

This draft PR exists so that a PiFinder can install a build of mrosseel:test/device-build from the unstable channel. The review of the work happens in four PRs on mrosseel/PiFinder, in this merge order:

  1. [1/4] feat: INDI mount control over USB and WiFi (simplified #367 design) mrosseel/PiFinder#125 INDI mount control
  2. [2/4] feat(satellites): satellite catalog, selectable groups, pass prediction and web TLE entry mrosseel/PiFinder#127 Satellite catalog
  3. [3/4] feat(mount): polar alignment with the mount turning the RA axis mrosseel/PiFinder#128 Polar alignment with the mount
  4. [4/4] feat(mount): follow moving targets with user correction, lunar and solar rates mrosseel/PiFinder#129 Follow of moving targets

The branch merges #128 and #129, so it holds all four PRs. It is based on the asteroids branch (#534), so the diff against main also holds the asteroid, Gaia chart and NixOS work.

What to test on a device

  • Objects > By Catalog > Satellites... > Sat groups: select CelesTrak groups. A group without a file downloads at once.

  • The Satellites list: the PASS sort lists the satellites above the horizon first.

  • The /satellites web page: groups, custom element sets, the "In the sky now" table, the pass table.

  • With an INDI mount: key 5 on a satellite follows it. The arrow keys correct the path. A corrected element set is stored when the follow ends.

  • With an equatorial INDI mount: the polar alignment wizard turns the RA axis between the three captures.

Not tested on a real mount yet.

🤖 Generated with Claude Code

mrosseel and others added 30 commits September 26, 2026 14:09
…#69)

The device asked for a patch per path and slept 15 s twice for each path
the server was still computing. For a new build the server computes all
of them, so 57 paths cost about 9.5 min of sleep, and then all of them
downloaded in full.

The device now asks for all paths in one round. It waits 15 s once and
asks again only for the paths that were not ready, for at most 2 rounds.
Then it applies the patches. A cold build costs at most 30 s of waiting.

The status line names the step: "patching asking|waiting|applying
<done>/<total>". The update screen shows "Asking server", "Server busy"
or "Applying patch". It still reads the old "patching <done>/<total>".

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
)

A build whose etc/fstab names a root device the Pi does not have stops in
the initrd, before the boot watchdog can roll it back. A CM4 with a btrfs
PIFINDER_SD card got a build that mounts / from NIXOS_SD as ext4. It
showed a black screen and needed a manual change on the card.

Before activation, the upgrade now compares the root line in the new
system's etc/fstab with the root in /proc/mounts. If the device or the
file system type does not match, the upgrade stops and reports failed.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
CI builds mounted / from the label NIXOS_SD as ext4. SD images have also
shipped PIFINDER_SD as btrfs. On such a card the initrd waited for ever
for NIXOS_SD, and the Pi showed a black screen.

Root is now /dev/mmcblk0p2 with type auto, on every card and on eMMC.
The initrd finds the type with blkid, and it carries btrfs as well as
ext4. ext4 is built into the kernel. btrfs loads as a module.

The SD image module keeps its own label root, so images do not change.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
PiFinder moves its root file system to btrfs. The normal system now
mounts /dev/mmcblk0p2 as btrfs with compress=zstd:1,noatime. The initrd
carries only btrfs, and the system supports only btrfs and vfat (the boot
partition).

A card that is still ext4 keeps the system it has until the migration
reformats it. The upgrade refuses a build that cannot mount the root of
the device (#70).

The migration system is not changed. It boots from the old ext4 card with
the same kernel, so ext4 stays in the kernel config.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…packs (#73)

The eight NixOS ADRs in docs/ax/nixos/adr move into docs/adr as five,
grouped by topic:

- 0037 builds, Attic cache and update channels (was 0001, 0002)
- 0038 boot watchdog and recovery mode (was 0005, 0006)
- 0039 card layout and migration: FAT firmware plus btrfs root,
  migration by btrfs-convert (was 0003, 0004, 0007, plus the new design)
- 0040 data packs: one Nix profile per pack, updated by Attic chunks (new)
- 0041 imx462 xclk override (was 0008)

The old folder is deleted, and all links point to the new ADRs.
docs/ax/nixos/btrfs-and-data-packs-plan.md lists the work, one PR per
step. The NixOS glossary gets the terms card layout, data pack, shard,
and full and lean image.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
#74)

pifinder-first-boot switched to the downloaded full system without a
check. A system whose etc/fstab names a root the card does not have
stops in the initrd, before the watchdog runs (ADR 0039, plan step A1).

A new script, check-root-mountable, applies the same rule as
check_root_mountable in nixos_upgrade.py: the fstab root must resolve to
the mounted root device, with the same type or auto. If it fails, first
boot stays on the migration system and tries again at the next boot.

Old NixOS ADR numbers in comments and docs now point to ADR 0037 and
0039.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…Boot with btrfs (#75)

The migration converts the Pi OS root to btrfs in place (ADR 0039).
The NixOS side for that:

- U-Boot reads btrfs (CONFIG_FS_BTRFS, CONFIG_CMD_BTRFS): extlinux.conf
  and the kernels are on the btrfs root. It had no btrfs code before.
- The migration system mounts /dev/mmcblk0p2 as btrfs, with only btrfs
  in its initrd, and ships btrfs-progs.
- packages.aarch64-linux.migration-tarball builds the tarball straight
  from the migration system's closure: boot/ is the FAT firmware,
  rootfs/ the btrfs root. release.yml publishes it; no SD image and no
  loop mount any more. The firmware commands are shared with the images.
- First boot falls back to the baked-in full system when the manifest's
  pick cannot mount the btrfs root (an older, pre-btrfs build).
- nixos/tests/migration-e2e: simulates a Pi OS 2.6.x -> NixOS migration
  in QEMU raspi4b, in a private mount namespace and a bwrap sandbox.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…label first boot (#76)

- pifinder-migrated-credentials (both systems, before sshd): applies the
  pifinder password hash and SSH host keys the migration carried, once.
- pifinder-ap-name: the access point name lives in PiFinder_data/ap_name
  (set_ap_name writes it) and goes into the AP profile at boot. NixOS
  writes that profile again at each activation, so a rename was lost.
- pifinder-migration-cleanup: deletes ext2_saved (the ext4 image from
  btrfs-convert, about the size of Pi OS and its data) once the running
  generation is confirmed.
- First boot: step labels on the splash (WAITING FOR CLOCK, FINDING
  NIXOS, DOWNLOADING NIXOS, SETTING UP, RESTARTING), the bar stays below
  100 until the reboot, and a log in PiFinder_data/logs/first-boot.log
  (the migration system's journal is in RAM only).
- boot-splash: --progress reads an optional label after the number.
- migration-e2e: U-Boot checked on its own, extlinux entries booted
  directly (QEMU hangs in U-Boot's hand-over), checks for every carried
  setting, card snapshots per stage, hard kill for QEMU.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… nixpkgs paths (#77)

- /update-start now sends base_toplevel (/run/current-system). The differ
  starts to patch this exact step at once.
- Patches for paths from cache.nixos.org are staged with their signed
  narinfo from cache.nixos.org, so prefetch_deltas gets that cache too.

Needs pifinder-differ 0.4 (mrosseel/pifinder-server#5). With an older
differ the extra field is ignored.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(migration): keep only password hashes NixOS can check

libxcrypt in nixpkgs is built with --enable-hashes=strong. Pi OS images
store the default password as SHA-256 crypt ($5$), which NixOS cannot
check, so after the migration no password works: not on SSH, not in the
web UI.

- pifinder-migrated-credentials applies a carried hash only when it is a
  strong type ($y$, $gy$, $7$, $2b$, $6$). Otherwise the NixOS default
  stays; for the release images it is the same password.
- pifinder-password-usable runs at each boot before sshd. A hash that is
  not a strong type is set back to the default password. This repairs
  devices migrated before this fix with a normal update.
- migration-e2e checks for a strong hash type when the Pi OS hash is not.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(migration-e2e): sync the card after unmount, before QEMU reads it

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The download total was nix's "unpacked" size from before the patch step,
and copies from the local patch cache counted as download. So the screen
showed the full size while the download took seconds.

- StagedCache records the NAR bytes of the staged paths.
- run_build leaves those bytes and paths out of the total.
- _DownloadProgress does not count copyPath activities whose source is the
  staged cache (the copyPath start event names its source cache).

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
nixos/migration.nix holds what NixOS does with what the Pi OS migration
left on the card: the Nix DB registration, the store and top-level
ownership repairs, the carried credentials, the password hash check and
the ext2_saved cleanup. They come from services.nix, networking.nix and
device.nix; nix-path-registration was defined in both services.nix and
device.nix and is now defined once.

Both the full system and the migration system import the module. The
full system's units are unchanged. The migration system also gets
fix-nix-store-ownership and pifinder-migration-cleanup (which keeps
/ext2_saved there: that system never confirms a generation).

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…iFinder_data before a switch (#82)

A8:
- pifinder-migration-cleanup never ran. It is wanted by multi-user.target
  and ordered after the watchdog, which is ordered after multi-user.target.
  The implicit "target after wanted unit" made a cycle, and systemd
  deleted the job at each boot (seen on the migrated Pi 4). An explicit
  after multi-user.target removes the cycle.
- pifinder-btrfs-tidy runs btrfs balance -dusage=50 -musage=50 once, when
  /ext2_saved is gone and the generation is confirmed, at Nice 19 and idle
  IO. It logs the duration and the load average (every 30 s) to the
  journal and to PiFinder_data/logs/btrfs-tidy.log, then writes
  /var/lib/pifinder/btrfs-tidy-done. The upgrade stops it; the next boot
  starts it again. No defragment: the converted data left is JPEGs.

A9:
- Before the switch, nixos_upgrade takes a read-only snapshot of
  PiFinder_data into /.snapshots/PiFinder_data-<UTC time>-<target hash>
  and keeps the newest two. Every failure only logs a warning.
- btrfs-progs on the pifinder-upgrade path.
- ADR 0039: the manual restore. Plan: A8 and A9 status.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…#84)

A trial generation that stops in the kernel or the initrd never reaches
the watchdog, so nothing rolled it back (ADR 0038 gap, plan step A10).

- U-Boot (nixos/pkgs/uboot-sd.nix, shared by the firmware payload and the
  system): BOOTCOUNT_FS with pifinder.bootcount on the FAT partition,
  bootlimit 3, and an altbootcmd that imports pifinder-fallback.env and
  boots its pxe_label_override entry.
- pifinder-bootcount arm|reset|armed writes the two files. The upgrade arms
  it with the extlinux entry of the running system before the reboot.
- The watchdog resets it on a healthy boot and before its own rollback.
  When it finds the counter on while a confirmed generation that is not the
  profile's one runs, U-Boot fell back: it makes the running generation the
  default again and writes upgrade_failed.json.
- boot.panic_on_fail panic=10: an initrd failure or a kernel panic restarts
  the Pi, so the count goes up by itself.
- pifinder-uboot-update installs the build's U-Boot on the FAT partition
  from a confirmed generation, and keeps u-boot-rpi4.bin.old.
- The firmware payload starts with the counter off.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…n A4) (#83)

- nixos/sd-image.nix: root btrfs PIFINDER_SD, mounted as /dev/mmcblk0p2
  with compress=zstd:1,noatime, and FIRMWARE vfat, as the migration makes.
  pifinder-grow-root grows partition 2 and the btrfs at first boot; the
  nixpkgs expand-root-partition uses resize2fs, which is for ext4 only.
- nixos/pkgs/make-pifinder-btrfs-fs.nix: nixpkgs make-btrfs-fs.nix makes no
  subvolumes. This one adds the PiFinder_data subvolume and zstd:1 through
  mkfs.btrfs --subvol and --compress, and moves ./files instead of copying.
- flake.nix: images.pifinder is the lean image. images.pifinder-full adds
  the catalog images; it exists only when catalogImagesSrc is set. The
  source of the catalog images is not decided (a TODO in one place).
- release.yml builds and publishes the full image when it exists, as
  pifinder-full-<tag>.img.zst with a .sha256.
- Plan doc: A4 status.

Checked: the lean image builds (1.2 GB zst); read-only mount: FIRMWARE
vfat, PIFINDER_SD btrfs, subvolume home/pifinder/PiFinder_data, fstab
root /dev/mmcblk0p2 btrfs. Not yet booted on a Pi.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- The device waits 5 s between rounds instead of 15 s, for up to 12
  rounds (60 s at most, as before). The server's first pass takes under
  a second for most paths, so they arrive after the first short wait.
- The upgrade opens the delta session before the dry run. The server
  then starts to patch this step while the device works out the missing
  paths.
- Labels: "Asking server" -> "Finding patches", "Server busy" ->
  "Making patches", "Applying patch" -> "Applying patches",
  "{n} not ready" -> "{n} to go". The old words read like a fault.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
pifinder-uboot-update ran after the watchdog, which runs after
multi-user.target, which wants the service: an ordering cycle. systemd
deleted the job at each boot, so the Pi 4 never got the new U-Boot. The
journal said: "Job pifinder-uboot-update.service/start deleted to break
ordering cycle". pifinder-migration-cleanup had the same fault (#82).

- after = multi-user.target too.
- A NixOS assertion fails the build when a service runs after the
  watchdog without multi-user.target in after.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
)

- The confirm screen offers Install only on USB-C power. The charger
  (bq25895, rev4) reports it; hardware without the charger cannot tell and
  keeps Install. On battery it shows "Plug in USB-C power".
- While the upgrade runs, the screen does not dim. A module on top of the
  menu stack sets keep_awake; PowerManager then counts it as activity.
- migration-e2e: MIGRATION_E2E_QEMU selects another QEMU. QEMU 11.1 stops
  at start with "Failed to initialize io_uring" on this host.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
After the patches, the rest of a download is often under 1 MB, and the
screen showed "0/0 MB". format_amount picks the unit by the total: KB
below 1 MB, MB with one decimal below 10 MB, MB up to 1 GB, GB above.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The amount under the bar ("150/300 KB", "12 to go") was in the base
font, dim gray, with the lower third of the screen empty. It is the main
fact on that screen, so it is now in the large font at full brightness;
the package name stays small below it.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
#88 draws the download amount in the large font; the stand-in font set
of test_draw_upgrading_every_phase had no large, so five cases failed.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
QEMU emulates the SPI0 controller, but no display answers. The kernel
driver then waited for ever inside boot-splash ("soft lockup" for more
than 30 minutes), the watchdog that calls boot-splash never ended, and
the ext2_saved cleanup after it never ran. The harness now disables
spi@7e204000 in its device tree copies, so boot-splash finds no
/dev/spidev0.0 and stops at once.

With this, the full stage passes: "PASS: Pi OS -> NixOS migration boots
the full system" (2026-09-27, the password and ext2_saved checks
included).

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The device sends all missing paths in one POST /deltas and reads the
answer as a stream of JSON lines. Each patch is downloaded and applied
as soon as its line arrives, while the server computes the others, so
there are no fixed waits between rounds. The paths the stream leaves
open (a cut stream, an older server without /deltas, or still computing
at the end) are asked again with /delta in the rounds as before.

Needs pifinder-differ with /deltas (mrosseel/pifinder-server#8); with an
older server the device uses the rounds only.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
mrosseel and others added 30 commits November 17, 2026 05:54
seconds_until_set sampled the altitude 361 times per satellite and
interpolated linearly. It now runs find_passes_many with a minimum
altitude of 0 over the search window. A satellite above the horizon at
the start has a pass that starts there, and its set is the answer. A
satellite below the horizon, or one that stays up for the full window,
gets None, as before. The step_seconds parameter had no caller and
goes.

The test checks the set to 2 s, and a satellite that stays up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The INDI tests start one indiserver for the module. Before each test, the
fixture stops the test drivers through the FIFO, waits until no driver
process runs, and removes the driver config and the running-driver record.
Each test starts as with a fresh indiserver.

The tests also wait less:
- The drift tests stop when the residual drift is below the limit. They
  have no pointing error, so the goto needs no refine round.
- The sync reply test and the continuous move test use shorter slews.
- The stale driver test waits 2 s for the device instead of 10 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drift compensation adds each correction to the custom track rate that the
driver reports. A goto sets the track mode back to sidereal, but the driver
keeps its custom rates. The first correction at the next target then added
to the rates of the old target, and that old bias came back at once.

The first correction after a goto, a new connection or a reload now starts
from the sidereal rate. Later corrections at the same target still add to
the current rate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
indiserver opens its FIFO again after each writer closes it. A quick second
write then fails with a broken pipe, and os.write raised that error up to
the caller. start_driver also sent its stop line and its start line in two
writes.

Now start_driver sends both lines in one write. A failed open or write is
tried again up to 5 times, 0.1 s apart. After that the call returns False
and logs the error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The drift fix uses the rate helpers of this branch. The first correction
starts from the lunar or solar rate when that track rate is on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t/device-build

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A satellite on the chart makes the chart draw again every 0.25 s.
Each frame collected the DSO markers again and plotted their symbols.
The solve, the FOV, the target and the lists did not change.

The chart now keeps the collected markers and their marker image.
It makes them again only when the key changes.
The key holds the solve time and RA/Dec, the FOV, the target ids,
the chart_dso setting, the observing list, the selected catalogs,
the filter dirty time and the starfield projection, roll and scale.

Each frame plots only the satellites and adds them to the DSO image.
The images add with saturation, so the chart shows the same pixels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The mount page band cut long follow texts with an ellipsis.
"Meet in 7:34 20° SW" showed as "MEET IN 7:34 …".
"Following -0.4s" showed as "FOLLOWING -0.…".

The follow texts are now shorter: "Follow slew", "Meet 7:34 20°SW",
"Follow in 12s", "Follow -0.4s" and "Follow pass".
The words that still do not fit the band go to the line under it.
They show before the step, brighter than the step.
The 176 px and 320 px screens use the same rule. They have more room,
so fewer texts move to the second line.

Tests render the band for each follow stage on all three screens.
They check that no ellipsis cuts a value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Harris and Lynga loaders parsed their float columns as float32, and
arp.sqlite stores its magnitudes as float32 values, so the database held
values such as 6.46999979019165 for M 2. The Harris and Lynga loaders now
parse float64, and the Arp loader rounds to two decimals. The database is
rebuilt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…guide

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The satellite description put the next pass on one line, for example
"Next: 21:43 SW->NE max 38° mag 2.1". The line is longer than the
128 px details screen. The description now has short lines: "Next
21:43", "SW->NE max 38°", "mag 2.1". The bright sky, geostationary,
no pass and no position texts are split in the same way. A test checks
that each line fits at 128, 176 and 320 px.

In a fresh session the automatic download of the missing element files
can run when the user selects "Update now". The forced download then
got "busy" and the user saw "Download busy". refresh_groups has a wait
option now. A call with it waits for the running download. A group that
the running download got or failed to get counts as downloaded or failed
in its result, so the user gets a real result and the group does not
download again within the 2 h limit. Update now and the group download of
the menu use the wait option.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
During the wait at the meet point of a satellite follow, the PiFinder
went to sleep after the sleep timeout (30 s by default). A key press that
wakes the PiFinder does nothing else, so the "I see it" arrow press was
lost. The main loop now holds the PiFinder awake while the mount status
is MOUNT_TARGET_FOLLOW and not stale. This covers the follow slew, the
meet wait and the track. After the follow the sleep timeout counts again.
The general rule stays: a key press that wakes the PiFinder does nothing
else.

The follow sent "Seen" as a MOUNT message. The mount page hides MOUNT
messages, so only the status band changed. The follow now sends a
CONFIRM message, for example "Seen +7.0s". Each screen shows it for
2 s, the mount page too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The demo movie shows one large sharp screen as the main view and the
device photo with its own small screen as an illustration. The photo
shows the real device and the pressed key. The style guide says this.

The key script no longer tells to set sleep_timeout to Off: the
PiFinder does not sleep during a follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testable Ready for testing via PiFinder software update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants