feat: opt-in, local shadow mode with a notice, status indicators, and draft offers

README.md

@@ -168,6 +168,21 @@ stays paused.
 [docs/DECISION_PORTAL.md](https://github.com/OpenAdaptAI/openadapt-desktop/blob/main/docs/DECISION_PORTAL.md)
 covers pairing, what the phone can see, and how a decision is applied.
 
+## Shadow mode
+
+Shadow mode is off unless you turn it on in **Settings**, after reading a
+notice. While it's on, the header and the tray say so. It notices a task you
+repeat in your recordings and offers to make a draft from it ("You did this 6
+times today."). Its record is an encrypted buffer in the app-data folder,
+which isn't synced, cleared with its key at least every 24 hours and whenever
+you turn shadow mode off or quit. **Pause** stops it recording, and **Delete
+everything and turn off** erases everything it kept in one click. OpenAdapt
+never sends shadow mode data off your computer, though backup, sync, and
+security software on the computer can still copy local files.
+A draft never runs; a person still records the task and admits the automation.
+[docs/SHADOW_MODE.md](https://github.com/OpenAdaptAI/openadapt-desktop/blob/main/docs/SHADOW_MODE.md)
+has the details.
+
 ## Engine command line
 
 The engine also runs without the app. In a source checkout, put `uv run` in
@@ -196,8 +211,9 @@ setting.
 | `storage`, `health`, `cleanup`, `config` | Inspect and maintain local storage and settings |
 | `upload`, `backends` | Older upload commands. `upload --backend hosted_ingest` runs `push`; upload to your own S3 storage is turned off in this release. |
 
-Recordings stay on your computer unless you choose to send one to OpenAdapt
-Cloud with `push`.
+OpenAdapt doesn't send a recording anywhere unless you choose to send it to
+OpenAdapt Cloud with `push`. Backup, sync, and endpoint security software on
+the computer can still copy local files.
 
 ## Develop
 

docs/COMPLIANCE.md

@@ -101,6 +101,32 @@ deployment HIPAA-compliant.
   `PLAYWRIGHT_BROWSERS_PATH` at a version-matched prebundle. Native, RDP, and
   Citrix workflows never use that path.
 
+## Shadow mode
+
+Shadow mode is off by default and has no remote, admin, policy, or
+configuration-file switch. A person turns it on in the Desktop window after
+accepting a versioned notice; the engine checks the notice hash and writes the
+acceptance time to an owner-only local file. The header and the tray show
+when it's on. Its record is an encrypted buffer in the platform's app-data
+folder (Application Support on macOS, LocalAppData on Windows), and shadow
+mode refuses to start if that folder resolves inside a known sync root such as
+Documents, Desktop, iCloud, OneDrive, or Dropbox. The key is in the OS keychain
+(or in memory) and is destroyed, with the buffer, at least every 24 hours,
+when shadow mode is turned off, and when the app quits; a buffer left by a
+crash is erased at the next start. Shadow mode makes no network connections,
+and its commands aren't accepted on the local socket. OpenAdapt never sends
+shadow mode data off the computer. These controls stop OpenAdapt, an
+employer, and other apps from turning it on or reading it through OpenAdapt;
+they don't stop an OS administrator, software running as the user, or on a
+managed computer IT administrators and endpoint security or backup tools, from
+inspecting or copying local files or memory while shadow mode is on. The
+notice says so, says the record may include personal, confidential, or health
+information, and advises pausing before viewing sensitive material. **Pause**
+is in the header, the tray, and Settings. **Turn off** erases the buffer, the
+key, and drafts; **Delete everything and turn off** also removes the ignore
+list and withdraws the
+acceptance. [SHADOW_MODE.md](SHADOW_MODE.md) has the details.
+
 ## Phone decisions
 
 A paired phone receives a request for a decision. A tap doesn't approve the

docs/DECISION_PORTAL.md

@@ -223,8 +223,8 @@ copy and the receipt copy that it derives from the closed result enums. It does
 not call a model to write a question or interpret an answer.
 
 The hosted outbound lane has a narrower remote-safe projection and does not
-carry the local evidence image. A custom entity class stays local unless it is
-in the reviewed remote-safe vocabulary; otherwise the remote task uses the
+carry the local evidence image. The remote task names a custom entity class
+only if it is in the reviewed remote-safe vocabulary; otherwise it uses the
 signed neutral `record` or `item` fallback.
 
 The primary screen is intentionally short: one question, the delivery and risk

docs/SHADOW_MODE.md

@@ -0,0 +1,183 @@
+# Shadow mode
+
+Shadow mode notices a task you repeat and offers to turn it into a draft. It is
+off by default. Only the person at the computer can turn it on, from
+**Settings → Shadow mode**, after reading a notice. OpenAdapt never sends
+shadow mode data off this computer, and shadow mode never runs a task.
+
+The watcher itself lives in openadapt-capture (`openadapt_capture.shadow`). Its
+design, the buffer format, and the privacy rules are in that repository's
+`docs/SHADOW_MINING.md`. This page covers what Desktop adds: the notice, the
+controls, where the record is kept, the indicators, and the commands.
+
+## Turning it on
+
+1. Open **Settings** and find the **Shadow mode** card.
+2. Select **Read the notice to turn on**. The notice covers what shadow mode
+   captures (and that it may include personal, confidential, or health
+   information shown on screen), why, how long it is kept, how to erase it,
+   and who can access it.
+3. Select **Turn on shadow mode** to accept, or **Not now** to leave it off.
+
+Desktop sends the notice's SHA-256 hash with the acceptance. The engine turns
+shadow mode on only if that hash matches the notice it has now, and it writes
+the time of the acceptance to a local file that only your user account can
+read. If the notice text changes in a later version, shadow mode needs a new
+acceptance.
+
+After a restart, shadow mode comes back on if it was on when Desktop quit and
+the accepted notice is still the current one. It starts with an empty buffer
+and a new key: the old buffer was erased when Desktop quit. If it was paused,
+it stays paused.
+
+## Where the record is kept
+
+The buffer, the notice acceptance, the work-site list, drafts, and the ignore
+list live in the platform's local app-data folder, not in the Desktop data
+folder, which you may have moved to a synced place:
+
+| Platform | Folder |
+| --- | --- |
+| macOS | `~/Library/Application Support/ai.openadapt.desktop/shadow` |
+| Windows | `%LOCALAPPDATA%\ai.openadapt.desktop\shadow` |
+| Linux | `$XDG_DATA_HOME/ai.openadapt.desktop/shadow`, or `~/.local/share/...` |
+
+Shadow mode refuses to start if that folder, after following symlinks,
+resolves inside a folder a sync service copies: Documents, Desktop, iCloud
+Drive, `~/Library/CloudStorage` (OneDrive, Dropbox, Google Drive, and Box on
+macOS), the folders the Windows `OneDrive` variables or Dropbox's own settings
+name, and folders named after common sync clients. The Settings card shows
+the folder in use.
+
+The buffer and its key are erased:
+
+- at least every 24 hours;
+- when you select **Turn off** or **Delete everything and turn off**;
+- when Desktop quits. The window asks the engine to erase them, and waits up
+  to 2 seconds before it stops the engine;
+- after a crash, at the next start, before a new buffer begins.
+
+The check can't see every backup or sync tool. Time Machine, File History, a
+sync client pointed at an unusual folder, or endpoint security software can
+still copy local files. The notice says so.
+
+## Knowing it's on
+
+While shadow mode is on:
+
+- The window header shows **Shadow mode on**, or **Shadow mode paused**, with
+  a **Pause** or **Resume** button.
+- The menu-bar tray menu reads **Shadow mode: on** or **Shadow mode: paused**,
+  and the tray tooltip says so. On macOS **Shadow** (or **Shadow paused**)
+  also appears next to the tray icon. Until the engine answers, the tray reads
+  **Shadow mode: checking…**, never "off".
+- The Settings card shows when it was turned on, whether the key is in the
+  system keychain or only in memory, when the buffer and key will be erased,
+  and the folder that holds them.
+
+The tray status line can't be clicked to turn shadow mode on. **Shadow mode
+settings…** in the tray menu opens the Settings screen instead.
+
+## Pausing it
+
+**Pause** stops shadow mode recording until you select **Resume**. It's in
+the window header, the tray menu (**Pause shadow mode**), and the Settings
+card. Pause before you view sensitive material. While paused, steps are
+dropped as they arrive, and typing in progress when you paused is dropped
+too. What was already in the buffer stays until it's erased as usual. A pause
+lasts across a restart.
+
+## What it watches
+
+In this version, shadow mode sees only the steps of recordings you make in
+Desktop. While a recording runs, the recorder hands each step to shadow mode
+as well as saving it. Shadow mode doesn't run a separate listener when no
+recording is running.
+
+Browser steps count only on the sites in **Work sites**. On any other site,
+shadow mode notes only that a browser was in use. The list is empty until you
+add a site, and OpenAdapt doesn't send it anywhere.
+
+## The offer
+
+About once a minute, shadow mode looks for a task you did several times
+today. When it finds one, the window shows:
+
+> OpenAdapt noticed a repeated task
+> You did this 6 times today.
+
+with **Review draft** and **Ignore**. The text never names an app, a site, a
+field, or a value.
+
+- **Review draft** saves a draft file in `drafts` in the shadow folder,
+  readable only by your user account. It isn't encrypted and is kept until
+  you turn shadow mode off. The draft is not an automation.
+  To automate the task, record it once; a person reviews and admits the
+  automation before it can run.
+- **Ignore** stops the offer for that task.
+
+## Turning it off
+
+- **Turn off** erases the encrypted buffer, the key, and any saved drafts. It
+  keeps the ignore list, so an ignored task isn't offered again.
+- **Delete everything and turn off** also erases the ignore list, and records
+  that the acceptance was withdrawn.
+
+Either way, turning shadow mode on again goes through the notice. If the
+system keychain refuses to delete the key, the card says so, and the engine
+tries again at the next start.
+
+## What others can and can't do
+
+OpenAdapt, your employer, and other apps can't turn shadow mode on or read its
+record through OpenAdapt:
+
+- There is no configuration file setting, environment variable, Cloud policy,
+  or admin setting that turns shadow mode on. It is not part of
+  `EngineConfig`.
+- The shadow commands are available only to the Desktop window. The local
+  socket that other OpenAdapt apps use doesn't accept them.
+- Shadow mode makes no network connections. OpenAdapt never sends shadow mode
+  data off this computer, and doesn't keep it in a folder that is synced by
+  default.
+
+That doesn't protect the record from the computer itself. The record is
+encrypted with a key that is destroyed at least every 24 hours, but the key
+sits in your keychain (or in memory) while shadow mode is on. Someone with
+administrator access to this computer, or software running under your
+account, could still inspect the record while it's on. On a work-managed
+computer, that includes IT administrators, and endpoint security or backup
+tools, which could inspect or copy local files or memory. The notice says
+so.
+
+## Engine commands
+
+| Command | What it does |
+| --- | --- |
+| `shadow_status` | On, paused, or off, the acceptance time, work sites, key location, the folder in use, and pending offers |
+| `shadow_notice` | The notice, with its version and SHA-256 hash |
+| `shadow_enable` | Turns shadow mode on. Needs `notice_sha256` and `acknowledged: true` |
+| `shadow_pause` | Stops recording until `shadow_unpause` |
+| `shadow_unpause` | Resumes recording after a pause |
+| `shadow_turn_off` | Erases the buffer, the key, and drafts, and turns shadow mode off |
+| `shadow_purge_and_disable` | Deletes everything shadow mode kept, including the ignore list, and turns it off |
+| `shadow_set_work_domains` | Replaces the work-site list (`domains`) |
+| `shadow_decide` | Answers an offer: `motif_id` and `action` (`review_draft` or `ignore`) |
+| `shadow_scan` | Looks for repeated tasks now |
+
+Events: `shadow_state` carries the same fields as `shadow_status`, and
+`shadow_prompt` carries one offer (`motif_id`, `count`, `title`, `body`,
+`actions`). The quit-time `shutdown` command, sent by the window's shell,
+erases the buffer and key before the engine stops.
+
+## Requirements and limits
+
+- Shadow mode needs an openadapt-capture release that includes
+  `openadapt_capture.shadow`, with its `shadow` extra (`cryptography` and
+  `keyring`). With an older release, the Settings card says shadow mode isn't
+  available.
+- Shadow mode sees only Desktop recordings. An always-on listener is not built.
+- If the system keychain can't be reached, the key is kept in memory only.
+- Every restart starts a new buffer, so a task is counted from the start of
+  the current session.
+- A buffer left by a crash stays on disk, encrypted, until the next start.

openadapt_desktop/cli.py

@@ -1013,7 +1013,7 @@ def main(argv: list[str] | None = None) -> None:
     config = EngineConfig()
 
     logger.remove()
-    logger.add(sys.stderr, level=config.log_level)
+    logger.add(sys.stderr, level=config.log_level, diagnose=False)
 
     engine = _init_engine(config)
 

openadapt_desktop/controller.py

@@ -24,12 +24,13 @@
 from __future__ import annotations
 
 import enum
+import inspect
 import json
 import sys
 import uuid
 from datetime import datetime, timezone
 from pathlib import Path
-from typing import TYPE_CHECKING
+from typing import TYPE_CHECKING, Callable
 
 from loguru import logger
 
@@ -99,6 +100,7 @@ def __init__(
         db: IndexDB | None = None,
         bundles_dir: Path | None = None,
         auto_compile: bool = False,
+        live_tap_provider: Callable[[], object | None] | None = None,
     ) -> None:
         self.captures_dir = captures_dir
         self.quality = quality
@@ -114,6 +116,10 @@ def __init__(
         self._db = db
         self._bundles_dir = bundles_dir
         self._auto_compile = auto_compile
+        # Shadow mode (off by default) hands a live tap for each recording
+        # while the person has it on; None otherwise.
+        self._live_tap_provider = live_tap_provider
+        self._live_tap: object | None = None
 
     @property
     def is_recording(self) -> bool:
@@ -182,7 +188,8 @@ def start(
         recorder = None
         try:
             Recorder = _load_capture_recorder()
-            recorder = Recorder(str(capture_dir), task_description=task_description)
+            extra = self._live_tap_kwargs(Recorder)
+            recorder = Recorder(str(capture_dir), task_description=task_description, **extra)
             recorder.__enter__()
             if not recorder.wait_for_ready(timeout=60):
                 raise RuntimeError("OpenAdapt Capture did not become ready within 60 seconds")
@@ -200,6 +207,7 @@ def start(
             if self._storage_manager:
                 self._storage_manager.register_capture(capture_id, capture_dir)
         except Exception as exc:
+            self._close_live_tap()
             if recorder is not None:
                 try:
                     recorder.stop()
@@ -223,6 +231,37 @@ def start(
 
         return capture_id
 
+    def _live_tap_kwargs(self, recorder_cls: object) -> dict[str, object]:
+        """``{"live_event_tap": tap}`` when shadow mode is on and the installed
+        recorder accepts a tap; ``{}`` otherwise."""
+
+        self._live_tap = None
+        if self._live_tap_provider is None:
+            return {}
+        try:
+            accepts = "live_event_tap" in inspect.signature(recorder_cls).parameters
+        except (TypeError, ValueError):
+            accepts = False
+        if not accepts:
+            return {}
+        try:
+            tap = self._live_tap_provider()
+        except Exception:
+            logger.exception("Shadow mode could not provide a live tap; recording without it")
+            return {}
+        if tap is None:
+            return {}
+        self._live_tap = tap
+        return {"live_event_tap": tap}
+
+    def _close_live_tap(self) -> None:
+        tap, self._live_tap = self._live_tap, None
+        if tap is not None:
+            try:
+                tap.close()  # type: ignore[attr-defined]
+            except Exception:
+                logger.exception("Shadow live tap did not close cleanly")
+
     def stop(self) -> dict:
         """Stop the current recording session.
 
@@ -246,7 +285,9 @@ def stop(self) -> dict:
             recorder.stop()
             recorder.__exit__(None, None, None)
             event_count = getattr(recorder, "event_count", 0) or 0
+            self._close_live_tap()
         except Exception as exc:
+            self._close_live_tap()
             try:
                 recorder.__exit__(type(exc), exc, exc.__traceback__)
             except Exception: