Skip to content

feat(review): Session Review as a Resolve-style editor, with markers and an epoch table - #182

Merged
gbradham merged 54 commits into
mainfrom
claude/session-review-redesign-01bd06
Sep 22, 2026
Merged

gbradham merged 54 commits into
mainfrom
claude/session-review-redesign-01bd06

Conversation

@gbradham

@gbradham gbradham commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

Session Review has been a stack of cards: a header of seven buttons, a canvas, a 46 px bar and a tab of tables. It had no zoom (one pixel of a 45,000-frame session is tens of frames), and it could only open an ethogram CSV. A live rig recording with video, tracking and hardware events but no classifier run could not be opened at all.

This PR rebuilds it as an editor modelled on DaVinci Resolve's Edit page. Each track is a behaviour source or a hardware device instead of an audio or video track. You scrub the video, drag a range, and read what happened in it. It carries phases 1 and 2:

  • Phase 1: the editor itself — layout, timeline, navigator, and opening live recordings.
  • Phase 2: markers (point notes and named ranges, per session or shared across a cohort) and the epoch table, which measures every animal over the cohort's ranges, with mean and SEM per group.

Hardware-driven selection ("select all ON periods") is phase 3, and is not here.

What you get

Layout (chosen from browser mockups):

  • Top row: sessions list | viewer | inspector, with the side panels collapsible.
  • Below it, a full-width timeline with a whole-session navigator strip. Drag its box to pan, drag its edges to zoom, click outside to jump.
  • Three top-bar menus (Open, Zones, Export) replace the seven buttons. A status bar shows fps, rows, poses, video, hardware lanes, event/tracking counts and markers.

Timeline:

  • A 212 px header column shows each track's value at the playhead: ON, ON · 10 Hz, 90°, the current behaviour.
  • A timecode ruler (HH:MM:SS:FF, flow-relative) with the pre-flow period hatched.
  • Behaviour becomes labelled clips once bouts are ≥ 2 px wide, and stays the per-pixel majority below that.
  • Pulse trains draw individually when zoomed in and at their duty fraction when zoomed out, so a 10 Hz 50 % train no longer looks like a lamp held on.
  • Lanes are coloured by role: outputs cyan, servo/PWM violet, inputs green.

Mouse and keys:

  • Drag across the tracks to select. Snapping pulls ends to bout edges, hardware switches and marker edges within 8 px.
  • Drag an In/Out edge to trim. ⌘/Ctrl-scroll or pinch zooms about the cursor; scroll pans.
  • Right-click opens the range menu: Set In/Out here, Select bout, Select whole session, Save range as marker, Add marker here, Zoom to range, Loop range, Export range stats, Copy timecode, Clear.
  • Keys: I, O, X, Z, ⇧Z, Esc, J/K/L shuttle up to 8×, ⌘/Ctrl+A, and M, ⇧M, ↑/↓ for markers. The existing ←/→, Home/End and [ ] are unchanged.

Viewer:

  • A HUD names the behaviour on this frame and every hardware output currently driven.
  • A transport bar with Video, Poses, Trail, Heatmap, Zones and HUD toggles.

Inspector:

  • The Range tab has an In/Out card (naming the range markers the selection falls inside, with Save as range marker), then Movement (cm travelled, mean and peak cm/s), Behaviour, Hardware in range (20.0 s on · 200 pulses, 1× at 2:25.0, 90° → 180° at 2:30.0) and Zones.
  • The Session tab has provenance and the fix-its.
  • The Markers tab lists ranges then points, filterable, and clicking one seeks to it.

Opening:

  • Open recording folder… opens a live recording directly.
  • Open folder as cohort… finds every ethogram and recording under a folder. Each ethogram claims the recording it came from, so no animal appears twice. Sessions are grouped by the group in glider_project.json.

Phase 2: markers and the epoch table

Markers:

  • M drops a point marker at the playhead ("door stuck here"); ⇧M keeps the In/Out range as a named range marker. Toolbar buttons do the same.
  • A small editor opens under the marker: name, one of nine colours, a note, scope (This session or Whole cohort), and Extend to In/Out to turn a point into a range. Double-click a marker to edit it again.
  • The 28 px marker row draws ranges as bars (stacking when they overlap) and points as flags, with a faint rule through the lanes. Drag a point to move it, or a range edge to trim it; snapping applies. ↑/↓ step between markers, and the navigator shows the ranges.

Epoch table (replaces the Cohort tab beside the timeline):

  • Columns are the cohort's range markers in time order — Baseline, Stim, Post — plus the Current range, each switchable with a chip.
  • Metrics come from a Metrics ▾ menu: distance, speed, each behaviour in seconds and as a percentage of the range, zone time/entries/latency, time on per device, and the thresholds each session was scored with. Defaults: distance, speed, freezing %, and the first zone's time.
  • Rows are animals, grouped by treatment, each group closed by a mean and a SEM row. Cells carry a faint bar scaled to the column's largest value, and — with a tooltip saying why where an animal can't answer (no calibration, no zones, range outside the session).
  • Click a row to show that animal in the viewer; double-click to jump to its timeline.
  • Export ▾ adds Epoch table, tidy (CSV), Epoch table, wide (CSV) and Markers (CSV).

Behaviour changes (also in CHANGELOG)

  • A plain drag in the timeline now selects. It used to scrub. Scrub on the ruler or the navigator instead.
  • Right-drag no longer selects. Right-click opens the range menu.
  • A range is measured from each animal's own zero. The current range, a cohort range marker and the Range stats export all mean the same seconds after each animal's flow start (or first video frame). Phase 1 applied the shown animal's frame window to everyone, which is a different stretch of protocol for any animal whose rig ran longer before flow start or recorded at another rate. Numbers change only for those animals. Exports gain start_s, end_s and t0.

Design decisions

  • Qt-free models, widgets that only draw. This follows the pattern session_view.py already set.
    • analysis/timeline.py gains value_at, rate_at, hardware_in_range and describe_*.
    • analysis/cohort.py (new) handles discovery and session ids.
    • analysis/markers.py (new) holds the time rule, the Marker model, and the marker files.
    • analysis/epochs.py (new) holds the metric catalog, mean and SEM, and the tidy and wide frames.
    • gui/review/viewport.py holds the zoom, pan, snap and timecode arithmetic.
    • The widgets live in a new gui/review/ package. analysis_window.py is now the shell that wires them.
  • selection_changed still emits frames. segment_stats and zone_rows are untouched.
  • One time rule. A marker's seconds are what the ruler shows: flow-relative when the session has a flow start, video-relative otherwise. seconds_at / frame_at / frames_in do every conversion — markers, snapping, the current range, cohort ranges and exports — so they cannot disagree about where a second falls. The current range is held in seconds: set when you select, never re-derived from the animal on screen, so a 15.59 fps animal (which cannot show 1.033 s exactly) cannot quietly move it.
  • Markers live beside the data. review_markers.json sits in the session's folder and cohort_markers.json in the folder opened as a cohort — not in glider_project.json (which would couple to its schema and need a manifest), and not in QSettings (not portable). Writes are atomic (temp file, then os.replace) and keep the file's existing permissions, so a cohort file on a shared lab drive stays readable to the rest of the lab. A file that cannot be read, or that a newer GLIDER wrote, is never overwritten: its markers turn read-only, the tools say why, and the error is shown once.
  • A live recording opens as a SessionView (SessionView.from_recording), so the canvas, trail, heatmap, zones and range statistics work unchanged. Three details decided this:
    • Frame offset. The tracking logger numbers frames from 1 and a video decodes from 0. first_video_frame makes the viewer read video frame n − 1; without it every frame shows its neighbour.
    • Calibration comes from the tracking CSV's # Pixels/mm header, scaled to the viewing resolution. It is never taken as distance_px / distance_mm: an uncalibrated logger writes the same number into both, which would claim 1.00 px/mm.
    • fps comes from the frame span over the elapsed_ms span. Session.frame_rate takes the median of millisecond-rounded timestamps and reads 30.30 for a 30 fps rig, which would skew every per-second number by 1 %.
  • One animal across tracker id changes. The live tracker assigns a new object_id after a detection loss. A recording is split into subjects only when two ids ≥ 0 share a frame; otherwise one animal would be silently truncated at the first id change.
  • Nothing a lab file contains may abort GLIDER. A non-UTF-8 CSV or a misshapen glider_project.json under a cohort folder used to raise out of a menu action and kill the process. Both are handled at the shared roots (_io._classify_csv, Project.load), and a bad manifest only costs the grouping, with a warning.
  • The session's layout and hidden lanes persist in QSettings. Hidden lanes are keyed per session folder, because session ids repeat across cohorts. Tests point the settings at a temp file.
  • Every new colour is a token in colors.py. A test checks that every hex in the Session Review stylesheet matches one. Markers use nine named swatches, saved by name, so retuning a colour recolours old markers too.

Fixes found along the way

  • The video no longer slides sideways as the playhead moves. The transport's bout read-out was a plain label whose text width fed the viewer panel's minimum width, so each new behaviour name pushed the splitter. It now elides, and the clock and frame counter are fixed to their widest text. The HUD's behaviour chip is sized once per session, so it no longer twitches with every digit.
  • The top-bar menus are one size, with their arrows centred. The shared width is measured from the widest label in the platform's own font — a fixed minimum had held only on macOS, and CI caught the buttons uneven again on Windows.
  • Seeking is fast. ExactFrameReader reached any earlier frame by decoding forward from frame 0, so one step back at frame 40,000 decoded 40,000 frames, and turning the video back on after scrubbing walked from wherever it had stopped. It now seeks just before its target and reads the landed frame's own timestamp to know exactly where it is, falling back to counting from 0 the moment a file's timestamps disagree with its frames. Frames are identical to the old reader's. On a 20-minute lab camera recording, a step back at frame 14,025 took 23 ms, where the old reader decoded 14,025 frames to get there.
  • An apply-run ethogram paired with its recording is no longer a frame off its hardware — the first of phase 1's known follow-ups. The ethogram counts frames from 0 and the logger from 1; the timeline now states every frame in the view's numbering.
  • A session read at the wrong frame rate when its poses were out of reach. SessionView.load fell back to 30 fps whenever the pose CSV could not be found, so a 15.59 fps recording read every duration and speed at nearly twice its rate. It now reads the rate from the run manifest; a reachable pose CSV still wins.

Tests

5389 → 5700 passed (4 skipped, 5 deselected); ruff and black clean. CI green on Windows, Ubuntu and macOS.

  • New suites: tests/unit/gui/review/ (timeline, navigator, viewer, inspector, pool, marker editor, epoch table), tests/unit/analysis/test_cohort.py, test_markers.py, test_epochs.py, and the recording, keys, menu, loop, HUD, hardware-stats, time-rule, marker and epoch-table classes in test_analysis_window.py.
  • Migrated: the 112 window tests moved to the new widget names without weakening an assertion. test_timeline_bar.py was retired, and its cases live in test_timeline_view.py. The Cohort-tab tests became epoch-table tests.
  • Checked against real data: phase 2 was run read-only against two sessions of a lab cohort (camera recordings at 15.59 and 30 fps): markers saved beside local copies, the epoch table filled for both, and one cohort range gave each animal different frames for the same seconds. That run found the frame-rate bug above.
  • Opt-in real-data smoke test: tests/integration/test_review_real_recording.py (marker real_recording) is skipped unless GLIDER_REVIEW_RECORDING points at a rig recording folder. It checks that real tracking frames line up with the real video, and that a real rig's events become lanes. It has not been run against a rig recording yet.

Each task in both phases (13 in phase 1, 12 in phase 2) was reviewed for spec and quality, and so was each whole branch; findings were fixed in-branch.

Known follow-ups (not in this PR)

  1. build_timeline still draws one "Live state" lane per tracker id, so a recording whose id changed shows several lanes while the statistics treat it as one animal.
  2. Session.frame_rate reads 30.30 fps for a 30 fps rig, as above. from_recording works around it; other callers still see it.
  3. The exact-seek reader has one blind spot: a frame dropped in a stretch of a variable-rate file the reader has never walked through cannot be seen, and a seek past it would land one frame off. A whole-file check of one lab camera recording found no drops; the upgrade path (index timestamps in one pass on open) is written in the docstring.
  4. Minor:
    • The first fill of the epoch table on a large cohort runs on the GUI thread without a wait cursor (cached afterwards).
    • M saves the marker before its editor opens, so Cancel keeps an unnamed marker; delete it from the editor.
    • The epoch grid lacks the alternating-row styling the other result tables have.
    • Session duration in the list counts rows, so recordings with gaps under-report.
    • Menu items no longer carry the old buttons' tooltips.

Not in this PR

  • Hardware-driven selection (phase 3): select one ON period, all ON periods, compare ON vs OFF.
  • Multi-camera support and session-folder unification (the earlier P2/P3).

…covery

A cp1252 notes.csv beside a recording raised UnicodeDecodeError out of
discover(), and a manifest whose sessions or subjects were not objects
raised AttributeError past project_groups, which only catches
ProjectError. Header sniffing now skips undecodable files, and every
wrong manifest shape (non-UTF-8, sessions/subjects not an object, a
session record not an object) is a ProjectError naming the file, so the
cohort still opens ungrouped with a warning. A non-object provenance is
tolerated, as a malformed treatment already was.
…ames

The live tracker issues a new object_id after a detection loss, so a
single-animal recording was cut off at the first id change. Ids now
split into subjects only when two of them share a frame; otherwise every
row with an id >= 0 is the one animal.

A tracking CSV with negative frame numbers raised ValueError or
IndexError out of from_recording, or silently wrapped positions to the
far end of the array. It is now a SessionViewError with the reason.
…ifier lane

Zoomed out, each viewport change cost ~60 ms per whole-session pulse
train: a fillRect and an x_of_axis call per segment. Switch lanes now
place segments with one vectorised _xs call, draw spans >= 1 px
individually, a lone sub-pixel pulse solid at 1 px, and columns holding
more than one switch at their exact ON coverage. 30,000 segments at 1400
px repaint in ~6 ms. Lane.binary (plus ends_ms and values) is cached on
the lane; is_binary() stays as a wrapper.

Also: the navigator strip is built at the device pixel ratio, the header
menu is deleted after use, and a view whose labels are all blank no
longer synthesises an empty lane titled Classifier.
Viewport.follow made the playhead the left edge in both directions, so
reverse playback paged on every tick; before the window it now becomes
the right edge. Pool rows put the badges on their own line and elide the
name, so VID/POSE/HW are whole at the default width. The blank canvas
points at the inspector's Session tab, and a recording is not told to
choose a pose CSV. The tour's ethogram step uses an em dash.
…w fixes

GLIDER installs no excepthook, so an exception out of a menu action ends
the process. Opening a recording, a cohort folder or an ethogram now
catches (SessionViewError, OSError, ValueError, KeyError), and a cohort
folder that cannot be walked is reported. A folder with events but no
tracking CSV falls through to its single ethogram_raw.csv, and only if
that also fails reports both. The parsed-recording cache is cleared on
every open, so a recording opened mid-run is read again.

Also: the Cohort table fills only while its tab is shown (a drag
recomputed the whole cohort per mouse move); playback at 8x lands on the
last frame; Ctrl/Cmd edit keys and auto-repeated J/L are ignored; the
range menu is deleted after use; _set_cohort builds before it assigns;
a recording's centroid reads as Centroid, not poses; the alignment
warning counts from the first video frame; Select All shows the
platform's own shortcut text.
…lanes, larger behaviour HUD

The transport and bout steppers drew text glyphs (⏮ ◀▏ ▶ ▕▶ ⏭ ◀ ▶) that read
badly; they now use Lucide icons (ISC, vendored beside the existing set) through
a small lucide_icon() helper that tints currentColor and renders at 1x and 2x.
Play shows Pause while playing, icon and word together.

The navigator's whole-session ethogram grows from an 18 px hairline to a 34 px
band (navigator 30 -> 48 px), behaviour lanes from 30 to 44 px, and the HUD's
behaviour chip -- the one label read while scrubbing -- is set large and bold.
… size

The transport's bout read-out was a plain QLabel, so its text width was part
of the row's minimum width -- which is the viewer panel's. Each new bout
pushed the splitter and the video slid sideways (908 -> 1062 px on a long
behaviour name). The read-out is now an ElidedLabel (moved from the sessions
panel into tool_ui for both to share) that takes the space left over, and
the clock and frame counter are fixed to their widest text.

The HUD's behaviour chip is sized once per session, to the widest behaviour
with a long seconds-in, so its edge no longer twitches with every digit. A
long behaviour at the playhead in a lane header elides clear of the lane's
own title instead of running over it.

The top-bar menus shared no size: Open took the primary role's larger
padding, and every menu arrow sat in its button's bottom-right corner. They
now share one padding and width, with the arrow centred in room of its own;
the ? menu matches their height and drops the arrow.
…frame 0

ExactFrameReader reached any earlier frame by rewinding to 0 and decoding
forward, because a seek on long-GOP video lands several frames off and cannot
say where. In Session Review that made every backward step, click, J, loop,
and turning the video back on after scrubbing cost a decode of the whole
session up to the playhead: one step back at frame 40,000 decoded 40,000.

A decoded frame carries its own timestamp, which says where a seek really
landed. The reader now seeks just before its target, reads the landing from
that timestamp, and walks the rest; a short hop ahead still just walks, and
reading the frame already held (the zone editor's grab) decodes nothing.
Every read checks the timestamp against the frame asked for: a file whose
timestamps do not count its frames -- variable rate, none at all -- is caught
on the first disagreement and read by counting from 0 as before, and a
frame reached through a seek that failed the check is fetched again that way.

A backward jump now costs under 40 decodes on a mis-seeking capture, and the
pixels match the old reader's frame for frame on a real mp4 and on long-GOP
mp4v and avc1 clips numbered frame by frame.
Checked against the lab's Blast Pilot cohort: on a whole 18,705-frame camera
file (15.6 fps, capture-time stamps with +-4 ms of jitter) the timestamps
named every frame, and seeks landed exactly on 30 fps files too. The one gap
left -- a frame dropped where the reader never walked -- is written down in
the docstring with its upgrade path.
Phase 1 applied the shown session's frame window to every session, which is
a different stretch of protocol in any animal whose rig ran longer before
flow start or recorded at another rate. The range now goes through the time
rule, switching animals keeps it in seconds, and each row carries the frames
and seconds it was measured over, plus time on per device.
…are gone

SessionView.load hardcoded 30 fps whenever a session's pose CSV could not
be found (offline video, network share, a copy without the DLC sidecar),
silently mislabelling every frame of a session recorded at any other rate.
run.json already records the fps a run classified at; fall back to it, the
same way applied thresholds and px_per_mm already do. A reachable pose CSV
still outranks it, unchanged.
… stale ranges

Three findings from the Phase 2 branch review, fixed together:

- save_markers no longer strips a shared marker file down to 0600. mkstemp
  creates its temp file owner-only, and os.replace carried that mode onto
  the destination, so saving over a group-readable review_markers.json left
  the next lab member who opened it hitting PermissionError. The temp file
  now matches an existing destination's mode, or a fresh file's umask-masked
  0666.
- The epoch table zero-fills a behaviour across every epoch, not just the
  one being rendered. range_rows only zero-filled `<state>_s` over the
  states seen in its own call, but the catalog is the union across all
  epochs, so a behaviour seen only in one range left every other range's
  column reading "-" where 0.00 was the true measurement, and the tidy/wide
  exports carried NaN instead of 0.0.
- Loading a new cohort now clears `_current_span`. It used to survive a
  cohort switch and get re-applied by `_show_session(0)` as a time range
  over an unrelated experiment, adding a bogus "Current range" epoch column
  measured over a span chosen elsewhere.

Also documents the fps-from-manifest fix (0a4545b) in the CHANGELOG, found
against the Blast Pilot cohort's 15.59 fps sessions.
…'s own font

This morning's fix gave them a stylesheet min-width of 64 px, which made them
one size only while every label fit inside it: true of the macOS font, not of
Windows', where CI measured Open, Zones and Export at 110, 118 and 130 px.
The window now measures the three after the theme is applied and gives each
the widest one's width, so the platform's font decides the size and the
buttons still agree.
@gbradham gbradham changed the title feat(review): Session Review as a Resolve-style editor (phase 1) feat(review): Session Review as a Resolve-style editor, with markers and an epoch table Sep 22, 2026
@gbradham
gbradham merged commit 9476110 into main Sep 22, 2026
4 checks passed
@gbradham
gbradham deleted the claude/session-review-redesign-01bd06 branch September 22, 2026 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant