Skip to content

feat: replay real production alerts on the local stack - #49

Open
MateoLostanlen wants to merge 16 commits into
mainfrom
feat/replay-real-alerts
Open

MateoLostanlen wants to merge 16 commits into
mainfrom
feat/replay-real-alerts

Conversation

@MateoLostanlen

@MateoLostanlen MateoLostanlen commented Sep 24, 2026 •

Copy link
Copy Markdown
Member
  • The old sample alerts in the notebook were not realistic. This adds scripts/replay_alerts.py to fetch a real alert (sequences, detections, images, crops and camera data) from the production API and replay it locally.
  • Alerts are stored as one zip per alert in a replay-alerts GitHub release, with make fetch-alerts, publish-alerts, list-alerts and replay-alerts. Published so far: 49767 and 54194. The script runs with uv, which installs its dependencies on the fly.
  • Two replay modes:
    • demo (default) copies the prod alert as is: images and crops go to the organization bucket, and the alert, sequences and detections are written straight into Postgres with the prod azimuths, cones and location. Times are shifted as a block so the latest sequence starts 1 hour ago, or the first one at START. Nothing is recomputed, so several alerts can be replayed side by side without mixing. This mode depends on the pyro-api DB schema.
    • live posts one frame per camera every 30 s through the API, so validation and triangulation run locally. Replays within 2 hours of each other get triangulated together, so use a fresh stack per alert.
  • Cameras are matched by name and set up as in prod (position, elevation, poses); local poses that do not exist in prod are deactivated, and each sequence is attached to the local copy of its prod pose. Missing cameras are created in organization 2, so the alerts show up for the test77 user.
  • make update-cameras (scripts/update_cameras.py) sets each camera of the published alerts to its most recent image in those alerts and sends a heartbeat, so the cameras look alive in the frontend.
  • Removes send_real_alerts.ipynb, the old download notebook and their unused helpers.

@MateoLostanlen
MateoLostanlen changed the base branch from chore/replace-minio-with-rustfs to main September 24, 2026 17:03
- Declare script deps inline (PEP 723) and call it with `uv run`
- `list` prints "no alerts published" instead of failing when the release is missing
- Create the release with --latest=false so it never replaces v0.0.1 as latest
- Document uv and the publication flow in the README
- fetch downloads each detection crop (signed via /detections/{id}/url,
  since sequence detection lists return crop_url=null) into crops/ in the zip
- replay sends one crop per box, or none if any box lacks one, as the API requires
Demo mode dated alerts today at the original time of day, which could be in
the future. The API sets last_seen_at to the server time, so a future
started_at puts sequences out of the triangulation time window.

- Shift all frames by one offset, keeping the prod gaps between them
- By default the latest sequence starts 1 hour ago, so every sequence
  starts in the past; --start / START sets the first frame time instead
- Replace --date today|original
others_bboxes belong to sequences outside the alert, which prod may have
rejected (temporal model) but the local stack validates by default, creating
extra alerts. They also had no crop, so their frames were sent without crops.
Replaying through the API recomputed validation, triangulation and merges,
so results depended on validation order and replays sharing cameras mixed.
Demo mode now copies the alert as is:

- images and crops go to the org bucket under replay-specific keys (the API
  deletes crops per detection, so replays must not share objects), and are
  removed again if the replay fails
- alert, sequences, links and detections are inserted in one transaction,
  with prod azimuths, cones and location, shifted as a block in time
- sequences are left unlabeled so the alert shows as live
- replays reject cameras spanning several organizations

Live mode still goes through the API. README documents both modes and the
published alerts.
- test the time shift through time_offset instead of the removed day_offset
- wrap a line ruff flagged as too long
- pin psycopg for the tests and bump requests past the version safety flags
- point make help at published alerts
update_cameras.py sets each camera of the published alerts (or the given
ones) to its most recent image in those alerts, then sends a heartbeat, as
a real camera would. It reuses the replay helpers, split out for it
(find_cameras, camera_token, published_alerts). Exposed as
make update-cameras.
Add make test-replay to run the unit tests without the stack, and document
the end-to-end check on the local stack.
The seeded local poses match no prod pose, local camera positions are
rounded with a fixed elevation, and every replay created a new pose.

- fetch stores each camera's prod poses in the zip
- replays and update-cameras set the prod position and elevation, reuse or
  create a local pose per prod pose and deactivate the others (seeded or
  left by older replays)
- each sequence is attached to the local copy of its prod pose instead of
  a new pose per replay

The published zips were fetched again with the poses.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant