Skip to content

feat: opt-in, local shadow mode with a notice, status indicators, and draft offers - #205

Merged
abrichr merged 12 commits into
mainfrom
feat/shadow-motif-prompt
Oct 18, 2026
Merged

feat: opt-in, local shadow mode with a notice, status indicators, and draft offers#205
abrichr merged 12 commits into
mainfrom
feat/shadow-motif-prompt

Conversation

@abrichr

@abrichr abrichr commented Oct 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Summary

This adds shadow mode to Desktop. It's off by default, and OpenAdapt never sends its data off the computer. Once someone turns it on in Settings, it notices a task repeated across their recordings and offers to make it into a draft. It never runs anything. The watcher itself is in openadapt-capture (OpenAdaptAI/openadapt-capture#156).

What changed

  • Engine (openadapt_desktop/shadow_mode.py):
    • ShadowModeService turns on only when it gets acknowledged: true and the hash of the current notice.
    • Its files live in the platform app-data folder (~/Library/Application Support/ai.openadapt.desktop/shadow, %LOCALAPPDATA%\ai.openadapt.desktop\shadow), not under data_dir. It refuses to start if that folder resolves inside a known sync root.
    • The buffer and its key are erased at least every 24 hours, on Turn off or Delete everything and turn off, and when the app quits. A buffer left by a crash is erased at the next start.
    • Pause stops recording until Resume, and lasts across a restart.
    • After a restart, it turns back on, with an empty buffer, only if the acknowledgement still stands. If it can't come back on, it deletes what it kept.
    • Desktop recordings reach it through the recorder's live tap, which follows the current observer.
    • Review draft saves a draft file that is owner-only from creation. Turn off erases the buffer, the key, and drafts. Delete everything and turn off also erases the ignore list and records that the acknowledgement was revoked, and says so if the keychain kept the key.
  • Quit: the shell used to write shutdown and kill the engine at once, and the engine had no such command, so quitting erased nothing. The engine now handles shutdown first by erasing the buffer and key, and the shell waits up to 2 seconds for its answer.
  • Logging: the engine and CLI log with diagnose=False, so a traceback can't print a variable's value.
  • Commands: shadow_status, shadow_notice, shadow_enable, shadow_pause, shadow_unpause, shadow_turn_off, shadow_purge_and_disable, shadow_set_work_domains, shadow_decide, and shadow_scan. The window reaches them through the Tauri bridge only; the local socket doesn't accept them. Shadow mode isn't in EngineConfig.
  • UI: a Shadow mode card in Settings with the notice, the status and folder, Pause, Turn off, Delete everything and turn off, and work sites. Shadow mode on or paused in the header, with a Pause button. Offers that read "You did this N times today." The window says "checking" until the engine answers, never "off".
  • Tray: "Shadow mode: on", "paused", "off", or "checking…", a matching tooltip and macOS title, Pause shadow mode / Resume shadow mode, and Shadow mode settings….
  • Docs: docs/SHADOW_MODE.md (new sections on where the record is kept and on pausing), the README, and docs/COMPLIANCE.md, using counsel's wording.

Before this can merge

  • Desktop pins openadapt-capture>=1.2.1 from PyPI, and that release doesn't include openadapt_capture.shadow. Until a Capture release with it is out and the pin is bumped, the card says shadow mode isn't available, and the shadow tests are skipped.
  • The Rust changes (tray.rs, sidecar.rs) pass rustfmt, but I couldn't run cargo check offline or a Tauri build. Quit-time erasure needs a check in a real build.

Tests

  • pytest tests/test_engine/test_shadow_mode.py tests/test_engine/test_main.py tests/test_engine/test_dispatch.py: 126 passed, 8 skipped, run against the Capture branch. A test fails if "Nothing leaves this computer" returns to the shadow copy, docs, or tray.
  • pytest tests/test_engine/test_item_approval.py tests/test_e2e/test_ipc.py: 14 passed, 7 skipped.
  • vitest run src/ui/ShadowMode.test.tsx: 16 passed. The three src/App.*.test.tsx files and the two Settings tests: 26 passed.
  • tsc --noEmit: clean. Ruff is clean on the changed Python files.

Decisions needed

  • A restart starts a new, empty buffer, so a task is counted only within one session.
  • A work-site change also starts a new buffer.
  • Shadow mode turns back on after a restart without asking again, as long as the notice hasn't changed.

@abrichr
feat(engine): add an opt-in shadow mode service
8c80981 
ShadowModeService runs openadapt-capture's shadow observer inside the
engine. It turns on only with acknowledged=true and the hash of the
current notice, resumes after a restart only while that acknowledgement
stands, and "purge and disable" deletes the buffer, the key, the ignore
list, and saved drafts, then records the revocation.

The recorder gets the service's live tap when shadow mode is on, so
Desktop recordings feed it. Review draft saves an owner-only draft file;
nothing runs. The shadow commands are on the Tauri bridge only, not the
local socket, and shadow mode is not part of EngineConfig, so no config,
environment, or policy can turn it on.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr
feat(ui): show the shadow mode notice, status, and repeated-task offers
75ebab5 
Settings gets a Shadow mode card: the notice with Turn on shadow mode and
Not now, the status (when it was turned on, the 24-hour buffer, where the
key is), the work-site list, and a one-click "Delete everything and turn
off". The header shows "Shadow mode on" while it is on, and offers appear
as "You did this N times today." with Review draft and Ignore.

The tray menu gains a "Shadow mode: on/off" line, a matching tooltip, a
"Shadow" title on macOS, and a "Shadow mode settings…" item. The window
sets them through the new set_shadow_tray_status command.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr
docs: describe shadow mode
10e050b 
Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr
docs(shadow): state what an OS administrator can still inspect
6865d68 
Drop the implication that no one can see the shadow record. OpenAdapt, an
employer, and other apps can't turn it on or read it through OpenAdapt,
but an OS administrator or software running as the user could inspect it
while it's on. The UI test uses the new notice version, 2026-10-17.2.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr
fix(engine): log tracebacks without variable values
5e8452d 
The engine and the CLI logged to stderr with loguru's default diagnose
mode, which prints the value of every variable on each traceback line.
The IPC handler logs every failed command with logger.exception, so a
failure while handling a recording or a shadow-mode event could copy a
typed value, a label, or a URL into the log. Both sinks now set
diagnose=False. The test puts a value on the failing line and checks
that it doesn't reach the log; it fails with diagnose on.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr
fix(shadow): don't strand the buffer, follow the current observer, re…
f6a8f0c 
…port a kept key

- When shadow mode was left on but couldn't resume (the notice changed,
  or this Capture has no shadow mode), resume returned without touching
  anything, so the buffer and its key stayed on disk and in the
  keychain indefinitely. A notice version bump does exactly this. Resume
  now deletes everything in that case.
- A recording's live tap was bound to the observer that existed when the
  recording started. After a work-site change replaced the observer, the
  rest of the recording went to the closed one, under the old allowlist.
  The tap now feeds whichever observer is current, or nothing.
- Delete everything and turn off reported success even when the keychain
  refused to delete the key. It now returns key_deleted, and the key is
  retried on the next destroy.
- Draft and state files were written and then chmod'ed, so they existed
  briefly with the default umask. They're now created owner-only.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr
feat(shadow-ui): add Pause and Turn off, and show "checking" until th…
63ecce1 
…e engine answers

- Pause is in the header, the tray menu, and the Settings card, and the
  card advises pausing before viewing sensitive material. The header and
  the tray say "paused" while it is.
- Settings has Turn off (erases the buffer, the key, and drafts) next to
  Delete everything and turn off, and says so when the keychain kept a
  key.
- The window and the tray started out saying "off" until the first
  status read, and a failed read also showed "off". They now say
  "checking", and a failed read keeps the last known state. The status is
  re-read every 30 seconds and when the engine restarts.
- The tray line no longer says "(local only)". The card and the header
  copy say the buffer is cleared at least every 24 hours and on turn-off
  or quit, instead of "on this computer only".

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr
feat(shadow): keep the buffer in unsynced app data and erase it on quit
131cd2f 
- Shadow files moved from <data_dir>/shadow, which a person may point at
  a synced folder, to the platform app-data folder: Application Support
  on macOS, LocalAppData on Windows. Shadow mode refuses to start if that
  folder resolves inside a known sync root, and files from the old
  location are deleted.
- Quitting erased nothing: the shell wrote "shutdown" and killed the
  engine at once, and the engine had no shutdown command. The engine now
  handles shutdown ahead of queued commands by erasing the buffer and its
  key, then answers, and the shell waits up to 2 seconds for that answer.
  The engine also erases them on a normal exit and through atexit.
- A buffer left by a crash is erased at the next start, before a new one
  begins. The engine starts shadow mode at launch instead of on the
  window's first status request.
- New commands: shadow_pause, shadow_unpause (a pause lasts across a
  restart), and shadow_turn_off, which erases the buffer, the key, and
  drafts but keeps the ignore list.
- A work-site change erases the old buffer instead of keeping it.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr
docs(shadow): describe where the buffer lives, Pause, and erasure on …
3532079 
…quit

Counsel's review: replace "nothing leaves" claims with "OpenAdapt never
sends shadow mode data off this computer", note that backup, sync, and
endpoint security tools can still copy local files, name IT
administrators on a managed computer, and advise pausing before viewing
sensitive material. SHADOW_MODE.md gains sections on where the record is
kept and on pausing, and lists the new commands.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr abrichr mentioned this pull request Oct 17, 2026
Merged
@abrichr
fix(copy): say what OpenAdapt sends, not that data can't leave the co…
3591b19 
…mputer

Counsel's review of the shadow mode notice applies to the rest of Desktop.
"Nothing leaves this computer" and "stays on this computer" aren't true
when backup, sync, or endpoint security software copies local files. Each
screen now says what OpenAdapt itself does with its data, checked against
the engine:

- Sample: OpenAdapt doesn't send the run anywhere, and the first run
  downloads its browser.
- Welcome and README: OpenAdapt doesn't send recordings or tasks anywhere
  unless you choose to share, upload, or push them.
- Results: these results aren't sent, except the PHI-free outcome the
  hosted runner reports for a Cloud job it runs.
- RecordReview and Settings: uploads happen only when you choose one, and
  stopped-run notes carry no PHI. The task description and target stay
  out of logs and aren't sent unless you upload or deploy.
- Needs you: the field differences and screen crop aren't sent anywhere.
- Authoring: OpenAdapt doesn't send window titles to your assistant.
- Teach, the judgment-case digest, the portal doc, and a docstring.

Where patient data or general recordings are involved, the copy adds that
backup, sync, and security software can still copy local files.
tests/test_engine/test_locality_copy.py fails if any of the old absolute
phrases returns to the README, docs, UI, engine, or tray source.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr
Merge origin/main into feat/shadow-motif-prompt
7af76cf 
Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr

abrichr commented Oct 17, 2026

Copy link
Copy Markdown
Member Author

End-to-end shadow-mode checks (macOS, 2026-10-17, Toronto time)

Build. npm run tauri build -- --debug --bundles app at 3591b19 produced OpenAdapt Desktop.app (ad-hoc signed). The engine ran from this branch's venv, with openadapt-capture at the merged #156 tree. A release bundle needs the PyInstaller sidecar, so this run used the dev launcher.

Method. The app was launched with open and quit through the macOS quit Apple Event, which takes the same RunEvent::Exit path as tray Quit. The sidecar was a test wrapper around the unmodified engine. Its control thread called the engine's own ShadowModeService: enable (with the current notice hash), pause, unpause, and purge_and_disable. These are the same methods the window's shadow_* commands reach. Synthetic events (no real input) went through ShadowModeService.live_tap(), the recorder's path. Keychain checks used exit codes only.

Check Result
Off by default With shadow mode off, live_tap() returned None and nothing was observed.
Quit With shadow mode on and 20 symbols buffered, quit left no app or engine process after 0.47s. buffer.db was deleted and the keychain held no shadow-key items. state.json stayed active by design, and the next launch started with 0 symbols and a new key.
Erase "Delete everything and turn off" deleted buffer.db and every keychain item (key_deleted=True). It wrote a revoked consent event and set state.json to {"active": false}, and the tap returned None afterwards. After relaunch, shadow mode was still off.
Pause While paused, two feeds of 24 delivered events left the count at 20. After unpause, one feed took it to 40. The pause survived a crash and restart.
Crash A (kill -9 of the app only) The engine read stdin EOF, exited, and ran the clean close: no buffer, no key.
Crash B (kill -9 of engine and app; on, paused, 20 symbols) The partial buffer.db and its key were left behind. On the next launch the old key was deleted from the keychain and a new key was created. The buffer came back empty and still paused, and events were dropped until unpause.
Crash C (kill -9, then the standing acknowledgement no longer matched the current notice 2026-10-17.4) On the next launch shadow mode did not resume. The buffer and key were erased, revoked was recorded, the tap returned None, and unpause was refused with "shadow mode is off".

The bundle's original launcher was restored byte-for-byte afterwards. Shadow mode is left off.

Local suites after merging main. pytest: 1254 passed, 43 skipped; test_shadow_mode.py 25/25 against the merged Capture. vitest: 344/344. cargo test: 18 passed.

@abrichr
build(deps): require openadapt-capture>=1.3.0 for shadow mode
ce951f8 
Shadow mode imports openadapt_capture.shadow (live tap, consent notice,
encrypted bounded buffer, on-device motif mining), first released in
openadapt-capture 1.3.0. Raise the floor and relock. The lock moves
alembic to 1.20.0 and SQLAlchemy to 2.1.5 because capture 1.3.0
requires alembic>=1.20.0 and sqlalchemy>=2.0.54.

The shadow tests use importorskip, so add a contract test that fails
instead of skipping when the resolved capture lacks shadow mode.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr

abrichr commented Oct 18, 2026

Copy link
Copy Markdown
Member Author

Review after ce951f8 (openadapt-capture>=1.3.0):

  • Pin. The floor is now openadapt-capture>=1.3.0, the first release with openadapt_capture.shadow. 1.3.0 is on PyPI and the lock resolves it from the registry. The lock also moves alembic to 1.20.0 and SQLAlchemy to 2.0.51→2.1.5, because capture 1.3.0 requires alembic>=1.20.0 and sqlalchemy>=2.0.54. websockets dropped out and xcffib (Linux only) came in, both following capture's dependencies.
  • Shadow tests now run in CI. In Python Engine Tests (3.12), all 25 tests/test_engine/test_shadow_mode.py tests PASSED and none were skipped, out of 1255 passed and 43 skipped overall. Because those tests use importorskip, the new test_resolved_capture_ships_shadow_mode fails rather than skips if a resolved capture lacks shadow mode. It also holds the declared floor at 1.3.0.
  • test_public_metadata now pins 1.3.0 in pyproject and the lock. The 1.2.1 on-device-audio floor test is unchanged and still passes.
  • Local run against PyPI 1.3.0 (locked sync). 1254 passed and 43 skipped, plus the metadata fix. The build-extra contract tests pass. ruff is clean.
  • The earlier E2E quit, erase, pause and crash results are in feat: opt-in, local shadow mode with a notice, status indicators, and draft offers #205 (comment).

Signed-off-by: Richard Abrich richard.abrich@mldsai.com

@abrichr
abrichr marked this pull request as ready for review October 18, 2026 00:16
@abrichr
abrichr merged commit 6902a5d into main Oct 18, 2026
17 checks passed
@abrichr
abrichr deleted the feat/shadow-motif-prompt branch October 18, 2026 00:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@abrichr