Feat: Browse archived sessions through the API and agentop

[huang195]

Fourth of five PRs for #901 (persist sessions). It reads the archive back: the session API serves history from disk, and agentop lists and opens it. The archive itself stays off by default; PR 5 adds clearing and turns it on for a local install.

Depends on #1266. #1263 and #1265 have landed. This branch is built on #1266's commits rebased onto main (4fbb3387), so the diff includes #1266 until it lands. Review only the six commits after 4fbb3387:

1. f817aa4f: the archive's read path
2. 0c7c81b2: the session API
3. cd0f3770: agentop
4. 5e8b0c57: docs
5. 057a5422: review fix: merge memory and disk pages by seq when memory's page has a gap
6. a5fc0c6f: rebase fix: publish both sides of a rename that parts a session's history. Feat: Record sessions to the archive on a laptop, off by default #1266's fix round made a rename move only the renamed entry's segments, and that path returned before reaching this branch's index publish, so the renamed events were unreadable until the session was written again. TestPage_FollowsARenameThatPartsHistory pins it.

What it adds

• Reading the archive (core/session/archive/read.go)
  • Whenever the writer changes a session, it publishes that session's directory, segments and summary to an index under an RWMutex. Readers decode on their own goroutine from a snapshot of the index, so a page read never makes the writer wait behind a segment decode.
  • Page(id, before, limit, fn) walks segments newest first. Per segment it keeps only the newest limit qualifying events, in a ring, so memory is bounded by the page rather than the segment. Events come back under the session's current id, as rekeyLocked does for resident ones.
  • Event(id, seq) returns that exact event, never a neighbour. Summaries() lists every archived session with the figures ListSessions reports.
  • A segment pruned or renamed between the snapshot and the read is skipped rather than failing the page.
• Session API (core/sessionapi/archive.go), active only with WithArchive. Without it, every endpoint is unchanged.
  • GET /v1/sessions/{id} follows one merge rule: memory first, then the archive below the oldest seq memory returned.
    • That covers four cases: a resident session, one resumed after a restart, one paged past its resident tail, and one that is only on disk.
    • totalEvents/oldestSeq follow the store's existing rule, applied to the merged history.
  • GET /v1/sessions/{id}/events/{seq} falls back to the archive.
  • GET /v1/sessions?archived=true adds disk-only sessions marked resident: false. It also returns an archive object with the size and limits, plus four loss indicators, present only when nonzero: dropped events, write errors, dropped renames, and a pause after a full disk. The default list is unchanged.
• agentop
  • Browsing history:
    • H on the sessions pane toggles history.
    • Disk-only rows read archived in UPDATED, where cached-only rows already read cached.
    • Enter opens one through the usual snapshot, and o pages back.
  • Against a proxy without an archive, the list comes back with no archive object. agentop says so once rather than showing an unchanged list as if history were on.
  • Footer: [H] history sits ahead of [u].
    • The footer is now 111 columns.
    • Placed anywhere after [$], the new hint pushed both cost keys out of the line at 80 columns. Ahead of [u], it is dropped before them, and a test pins that.
    • The README demo asset is regenerated for the new footer.
  • The agentop binary links no zstd; only its tests import the archive.
• Docs:
  • docs/laptop-service.md gets a ~/.cortex/sessions section covering opt-in, what the files hold, the bounds, what a burst or a full disk costs, and how to stop and wipe it.
  • The agentop README documents H.
  • CLAUDE.md updates the Session Events API, Disabling, the reload rules and gotcha Bump sigs.k8s.io/controller-runtime from 0.22.1 to 0.22.4 in /kagenti-webhook #14.
• cmd/cortex-cpex and scripts/readme-demo gain klauspost/compress as an indirect dependency, because the session API now imports the archive.

Tests

• read_test.go covers:
  • pages across a closed and an open segment
  • before bounds
  • the id rewrite after a rename
  • single-event lookup
  • summaries
• sessionapi/archive_test.go builds a restarted proxy: an archive written by a previous process, with a fresh store over it. It checks:
  • an archive-only session
  • a resumed session paging across memory and disk with no seq repeated
  • events/{seq} from disk
  • the default list and ?archived=true
  • a server with no archive
• tui/history_test.go ends with an end-to-end test using a real archive and a real session API: H, then Enter on the archived row, and the timeline arrives.

Verification

• cd core && go vet ./... && go test -count=1 ./..., plus -race on session/..., sessionapi, config and reloader
• cmd/cortex, cmd/cortex-envoy, cmd/agentop and cmd/cortex-praxis build and vet
• the cortex and agentop suites pass, and so does scripts/readme-demo, including its asset-staleness check
• go mod tidy -diff is clean in all 12 modules
• gofmt is clean
• Not run: against the shared laptop proxy.

Deferred from review

Suggestions and nits from the review of d1454ed and its fix round, left for later:

• totalEvents and the list's eventCount come from the archive's all-time count, which retention never lowers, so agentop's "N older" hint does not clear at the retained beginning.
• A failed archive read is logged at Debug while the events read before it are served, and oldestSeq still points below the failed segment; handleGetEvent logs its failures at Debug too.
• Page and Event decode a segment from its start and keep its whole string table, and segments have no size cap; the read-cost wording in read.go and cmd/agentop/README.md overstates.
• agentop's pod switch keeps showHistory, historyNoticed and archiveUsage, so the "no session archive" notice does not fire on the next pod.
• archiveUsage is stored but never read, so the archive's loss indicators do not reach the user.
• publish runs on every archived event on the writer goroutine; publishing on tick, rotation, retire and rename would show readers the same events.
• The sessions pane's stale-reply guard covers one direction: a plain list in flight when H turns on can hide archived rows for one poll.
• The help overlay dropped "reset when the proxy restarts", which still holds where the archive is off.
• When memory's page has a gap, the archive is asked for a full limit below before; the contiguous suffix ending at before-1 would bound the read to what memory lacks.
• withArchivedEvents' doc comment states only the memory half of the rule, and contiguousBelow and mergeBySeq have no comments.
• CLAUDE.md's Session Events API row says a page continues from disk below memory's oldest event; it now also fills a gap inside memory's page.
• The gap test accepts totalEvents 20 for a session serving 21 distinct events, and does not exercise memory winning an equal seq or a gap disk cannot fill.

Assisted-By: Claude (Anthropic AI) noreply@anthropic.com

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 26 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 4c2ac15e-4efc-435a-aef2-891ec1512eaf

📥 Commits

Reviewing files that changed from the base of the PR and between 535b451 and 8101526.

⛔ Files ignored due to path filters (1)

• docs/assets/cortex-demo.svg is excluded by !**/*.svg

📒 Files selected for processing (24)

• CLAUDE.md
• cmd/agentop/README.md
• cmd/agentop/apiclient/client.go
• cmd/agentop/apiclient/client_test.go
• cmd/agentop/go.mod
• cmd/agentop/tui/app.go
• cmd/agentop/tui/help_overlay.go
• cmd/agentop/tui/help_overlay_test.go
• cmd/agentop/tui/history_test.go
• cmd/agentop/tui/keys.go
• cmd/agentop/tui/sessions_context.go
• cmd/agentop/tui/sessions_pane.go
• cmd/cortex-cpex/go.mod
• cmd/cortex/main.go
• core/pipeline/session.go
• core/session/archive/archive.go
• core/session/archive/read.go
• core/session/archive/read_test.go
• core/session/store.go
• core/sessionapi/archive.go
• core/sessionapi/archive_test.go
• core/sessionapi/server.go
• docs/laptop-service.md
• scripts/readme-demo/go.mod

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[esnible]

This approval covers only the five commits after a9986693: c1738c03 through 4a7f0c30. That includes the gap fix 4a7f0c30, which the description's list of four doesn't name yet. It does not cover #1263, #1265 or #1266.

This PR is no longer a draft. The description says it was opened as one so it can't merge ahead of #1263/#1265/#1266. All three are still open, so please mark it as a draft again (gh pr ready --undo 1267) until they land.

withArchivedEvents' merge rule holds in every case I traced:

• a contiguous resident tail: need is 0, so there is no disk read
• a resumed session
• paging past the resident tail
• the pinned-intent gap, whether disk starts at or after the intent
• retention having pruned disk below what memory still holds: the whole-session marker stays correct
• a segment pruned or renamed mid-read

Event never answers with a neighbour. Because readSegment tolerates a truncated tail, reading the open segment is safe.

Verified locally at 4a7f0c30:

• go test -race passes on core/session/... and core/sessionapi
• the cmd/agentop (tui, apiclient) and cmd/cortex suites pass
• go vet and gofmt are clean
• go list -deps confirms the agentop binary links neither core/session/archive nor sessionapi

The "Deferred from review" list matches what I would have raised. One nit inline.

[esnible]

nit: Importing the concrete archive package means every binary that serves the session API now links the archive. That includes cortex-envoy and cortex-cpex, which can never open one (the archive is laptop-only and only cmd/cortex opens it). I measured cortex-envoy built with the envoy profile, stripped: +194 KiB (37.89 MB to 38.09 MB). zstd was already linked there, so this is just the archive package.

A small interface in sessionapi covering Page, Event, Summaries and Stats would keep it out. That is the same reasoning pipeline.ArchiveUsage gives for keeping zstd out of agentop. Fine to defer.