feat(webui): add the Browser Use surfaces, bridge and panel

.github/workflows/ci.yml

@@ -51,7 +51,11 @@ jobs:
     needs: changes
     if: needs.changes.outputs.docs_only == 'false'
     name: verify (${{ matrix.os }}, ${{ matrix.node }})
-    timeout-minutes: 15
+    # A ceiling, not a cost: a job that finishes in eleven minutes still takes
+    # eleven minutes. The Linux profile drives the real-browser suite, which
+    # launches a browser per case because each case needs a clean extension
+    # state, and that suite does not fit in fifteen minutes any more.
+    timeout-minutes: 25
     strategy:
       fail-fast: false
       matrix:

.gitignore

@@ -12,3 +12,5 @@ dist-webui/
 .idea/
 .codegraph/
 .plan
+dist-*.zip
+dist-store-*/

.gitleaks.toml

@@ -12,6 +12,13 @@ paths = ['''(^|/)packages/agent-modules/permission/src/tools/bash-checker\.ts$''
 regexTarget = "match"
 regexes = ['''^labelKey: 'permission\.scope\.byArgvPrefix2'$''']
 
+[[rules.allowlists]]
+description = "Published pairing transfer vector, not a credential"
+condition = "AND"
+paths = ['''(^|/)packages/(browser-extension/test/protocol\.test\.ts|webui/test/unit/browser-bridge-pairing\.test\.ts)$''']
+regexTarget = "secret"
+regexes = ['''^00112233445566778899aabbccddeeff$''']
+
 [[rules.allowlists]]
 description = "Documented keyboard shortcut"
 condition = "AND"

packages/browser-extension/README.md

@@ -0,0 +1,76 @@
+# @mavis/browser-extension
+
+The Edge/Chrome extension that bridges a person's own browser to a local WebUI
+host, so the person and the Agent operate **one** browser session rather than two.
+It is the extension half of ADR 0019.
+
+## What it does
+
+- Pairs with the local WebUI over a loopback WebSocket at `/browser-bridge`,
+  using challenge-response rather than a bearer token.
+- Reports the tabs it can reach, each with a generation that rises whenever the
+  page behind the tab is replaced.
+- Streams live images of the visible tab for the panel.
+- Delivers pointer, keyboard and text input into the tab the Agent addressed.
+
+## Layout
+
+| File | Purpose |
+| --- | --- |
+| `manifest.json` | The MV3 manifest: permissions, loopback host permissions, no remote code. |
+| `src/chrome.d.ts` | The subset of the Chrome API this extension uses; `@types/chrome` is unavailable to an extension. |
+| `src/protocol.ts` | The extension's copy of the bridge wire protocol, pinned to the host's signature test vector. |
+| `src/pairing.ts` | The pairing credential in `chrome.storage.local` and the challenge answer. |
+| `src/bridge-client.ts` | The loopback WebSocket client, the reconnect schedule and the operation deadline. |
+| `src/targets.ts` | Tab discovery and the generation registry. |
+| `src/images.ts` | The visible-tab image stream. |
+| `src/input.ts` | Pointer / keyboard / text delivery, bound to target and generation. |
+| `src/background/service-worker.ts` | The MV3 entry point that wires the above together. |
+
+## Running it
+
+Load `packages/browser-extension/` as an unpacked extension
+(`chrome://extensions` → Developer mode → Load unpacked), then set the bridge
+origin in extension storage:
+
+```json
+{ "webui.browserBridge.origin": "http://127.0.0.1:<webui-port>" }
+```
+
+The extension cannot learn the host's address by itself, so an absent origin is
+reported rather than replaced with a guessed default port.
+
+The bridge path, the frame shapes and the signature construction are duplicated
+from `packages/webui/src/shared/contracts/browser-bridge.ts`: an extension has no
+node_modules and no bundler, so it cannot import a workspace package. Both sides
+are pinned to the same test vector
+(`secret = "webui-browser-bridge-test-secret"`, `nonce = "nonce-abc"`,
+`keyId = "key-1"` → `xb7/GT5s+ScVsaiLkh+eRRndKi5oPQwqQMqU3aX8Znc=`), so a change
+to either side breaks one suite rather than silently failing to pair.
+
+## Two limitations that shape the product
+
+- **Only the visible tab can be captured.** `chrome.debugger` does not expose
+  `Page.captureScreenshot`, so `chrome.tabs.captureVisibleTab` is the only image
+  path, and it returns nothing for a background tab. The extension reports "no
+  frame" rather than substituting another tab's page.
+- **No remote code and no `<all_urls>`.** The content security policy allows only
+  `'self'` scripts, and host permissions cover the two loopback origins the
+  bridge needs. Page access comes from `tabs`/`debugger`/`activeTab`, not from a
+  blanket host grant.
+
+## Verification
+
+```
+pnpm --filter @mavis/browser-extension typecheck
+pnpm --filter @mavis/browser-extension test
+```
+
+## Store distribution
+
+Chrome Web Store and Edge Add-ons artifacts are a separate release action. Both
+builds come from this one source package; a store build needs a compiled service
+worker (`dist/background/service-worker.js`) with `manifest.json` pointing at the
+built path, since a browser cannot execute the TypeScript sources directly. This
+package deliberately declares no build step, so that compilation belongs to the
+packaging change rather than to the bridge implementation.

packages/browser-extension/STORE-DISCLOSURES.md

@@ -0,0 +1,127 @@
+# Store disclosures — Chrome Web Store and Edge Add-ons
+
+Both stores ask the same questions, and the answers are the same because the
+extension is one package. This document is the source the two listings are
+written from; the per-store wording differs only in the store's own terms.
+
+## Single purpose
+
+**Pair this browser with a MiniMax Code WebUI running on the same machine, so
+that the person and the Agent can work in the same browser page.**
+
+The extension connects to a WebUI host on the loopback interface, pairs with
+it once, and then relays three things between them:
+
+- which tabs are open, so the host can show them;
+- the picture of the tab the person is looking at, so the host can display it;
+- the person's pointer, keyboard and text input, so the person can act on the
+  page the Agent is working in.
+
+It does not browse on its own, it does not collect anything for anyone else,
+and it does not work unless a WebUI is running on this machine and the person
+has turned the browser's Browser Use switch on.
+
+## What the extension sends, and to whom
+
+Every byte the extension sends goes to **one place**: the WebUI host on
+`http://127.0.0.1` or `http://localhost`, over a WebSocket, and only after
+the person has paired it. The extension has no other destination, no
+analytics, no telemetry and no remote code.
+
+| What | Where it goes | Why |
+| --- | --- | --- |
+| The tabs' titles and URLs | The loopback WebUI host | So the host can show which page is open and let the Agent select one. |
+| A picture of the **visible** tab | The loopback WebUI host | So the host can show the person the page the Agent is driving. |
+| Pointer, keyboard and text input | The loopback WebUI host | So the person's input reaches the page the Agent is working in. |
+| The result of a DevTools command | The loopback WebUI host | So the host can read the page the Agent asked about. |
+| A pairing credential, encrypted | The loopback WebUI host | So the host can prove the peer is this extension and not something else. |
+
+**The extension can read the contents of a page.** It attaches the browser's own
+DevTools debugger to the tab the person has paired, and the allowed commands
+include reading the page: `DOM.getDocument`, `DOM.querySelector`,
+`DOM.getOuterHTML`, `DOM.describeNode`, `Runtime.evaluate`,
+`Runtime.callFunctionOn`, `Runtime.getProperties` and
+`Accessibility.getFullAXTree` read the page's DOM, the values of its
+JavaScript objects and its accessibility tree, and `Network.getCookies` and
+`Network.getResponseBody` read the page's cookies and the bodies of the
+responses it received.
+
+Whatever such a command returns is sent back to the loopback WebUI host,
+because that is how the Agent learns what the page says. **The host is the
+only destination**, and the person is the one who paired the tab, but the
+extension is not limited to the page's title and its picture.
+
+The full list of allowed commands is
+`packages/browser-extension/src/cdp.ts`; a command outside it is refused
+rather than attempted, and the refusal is reported to the host as
+`unsupported`.
+
+## What the extension stores, and where
+
+All of it is in the browser's own extension storage, on this machine, and
+none of it leaves the machine.
+
+| What | Where | Lifetime |
+| --- | --- | --- |
+| The WebUI host's address | `chrome.storage.local` | Until the person clears it. |
+| The installation key the person pasted | `chrome.storage.local` | Until the person clears it. |
+| The pairing credential the host issued | `chrome.storage.local` | Replaced on every pairing; removed when the browser's switch is turned off. |
+| A random identifier for this browser | `chrome.storage.local` | Until the person clears it. |
+| The last error, for the options page | `chrome.storage.session` | Until the browser closes. |
+
+The extension writes **nothing** to disk outside the browser's extension
+storage, and it keeps **no log**: there is no `console` call anywhere in the
+extension, because an extension that logs what it is carrying leaks pairing
+material and page content into whatever collects extension logs.
+
+## The permissions requested, and why each is needed
+
+| Permission | Why it is needed |
+| --- | --- |
+| `debugger` | The only way an extension can dispatch input to a page the way the browser does, and the only way to run a DevTools command against a tab. Chrome's `debugger` API exposes only a **subset** of the DevTools domains, and the extension restricts itself further to the commands in `cdp.ts`. |
+| `tabs` | Knowing which tabs exist, so the host can show them and the Agent can select one. |
+| `activeTab` | Working with the tab the person has made active. |
+| `scripting` | Injecting the small amount of scripting the input path needs. |
+| `storage` | Holding the host address, the installation key and the pairing credential, as described above. |
+| `webNavigation` | Knowing when a tab navigated, so the host can tell a page that was replaced from the page it replaced. |
+| `<all_urls>` (host permission) | `chrome.tabs.captureVisibleTab` — the only way to deliver the picture of the visible tab — requires either this permission or an `activeTab` permission granted by a user gesture on that page. There is no gesture here: the **host** is asking for the picture, not the person clicking the extension's icon. Without it the browser refuses the capture with *"Either the '<all_urls>' or 'activeTab' permission is required."* The extension captures the visible tab and does nothing else with the permission itself: it injects nothing into the page, and the picture is the only thing the permission is used for. The permission is what lets the extension attach the debugger to a tab, and the DevTools subset it may run there — which can read the page, as described above — is what that attachment allows. |
+
+The host permissions also include `http://127.0.0.1/*` and
+`http://localhost/*`, which is where the WebUI host is reached.
+
+## Privacy policy
+
+The extension collects **no personal data** of its own and transmits
+**no personal data** to any party other than the WebUI host running on the
+same machine, which the person has installed and started themselves.
+
+The data that passes between the extension and that host — tab titles, tab
+URLs, the picture of the visible tab, the person's input, and the result of a
+DevTools command, which **can** be the content of the page — is the data of
+the pages the person is already looking at. It moves only between two programs
+on the same machine, over the loopback interface, and only while the person has
+the browser's Browser Use switch on. It is not stored by the extension beyond
+the credentials described above, and it is not sent anywhere else.
+
+The host stores what it is given in its own data directory, and the Agent may
+act on it. That is the point of the feature, and it is why the switch is off
+until the person turns it on.
+
+No data is sold, shared, or used for advertising, and there is no analytics or
+telemetry of any kind.
+
+## The packaged build differs from the development build in exactly these ways
+
+| | Development build | Packaged build |
+| --- | --- | --- |
+| Extension identity | The id Chrome derives from the unpacked directory — different for every path, and therefore **not** a stable identity | The id the store issues, fixed and recorded |
+| How the identity is fixed | Nothing in the build pins it | The manifest's `key` field carries the store's public key, and the id follows from it: the first sixteen bytes of its SHA-256, each nibble mapped to a letter from `a` |
+| Where the identity is recorded | Nowhere; it is derived from the path | In the build manifest, and the host validates against it |
+| Everything else | Identical | Identical — the packaged manifest differs from the development manifest in the `key` field and in nothing else |
+| Accepted by a packaged host | **No.** A packaged host ignores the development allowlist | Only the recorded store identity |
+| Accepted by a development host | Only the ids in the development allowlist | The store identity, when it is listed |
+
+A packaged host therefore never accepts a development build, and a
+development build is never mistaken for a store build. The identity the bridge
+accepts is the one the store issues, because that is the only identity the
+person can be sure is the extension they installed from the store.

packages/browser-extension/manifest.json

@@ -0,0 +1,31 @@
+{
+  "manifest_version": 3,
+  "name": "MiniMax Code Browser Bridge",
+  "version": "0.1.0",
+  "description": "Pairs this browser with a local MiniMax Code WebUI host so the person and the Agent can operate the same browser session.",
+  "minimum_chrome_version": "116",
+  "background": {
+    "service_worker": "src/background/service-worker.ts",
+    "type": "module"
+  },
+  "options_ui": {
+    "page": "options.html",
+    "open_in_tab": true
+  },
+  "permissions": [
+    "activeTab",
+    "debugger",
+    "scripting",
+    "storage",
+    "tabs",
+    "webNavigation"
+  ],
+  "host_permissions": [
+    "<all_urls>",
+    "http://127.0.0.1/*",
+    "http://localhost/*"
+  ],
+  "content_security_policy": {
+    "extension_pages": "script-src 'self'; object-src 'self'"
+  }
+}

packages/browser-extension/options.html

@@ -0,0 +1,51 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8" />
+    <title>MiniMax Code Browser Bridge — options</title>
+    <style>
+      body {
+        font: 14px/1.5 system-ui, sans-serif;
+        margin: 2rem;
+        max-width: 40rem;
+      }
+      label {
+        display: block;
+        margin-bottom: 0.5rem;
+      }
+      input {
+        width: 100%;
+        padding: 0.4rem;
+        font-family: ui-monospace, monospace;
+      }
+      .row {
+        margin-top: 0.75rem;
+        display: flex;
+        gap: 0.5rem;
+        align-items: center;
+      }
+      #stored {
+        color: #555;
+      }
+      #status:empty {
+        display: none;
+      }
+    </style>
+  </head>
+  <body>
+    <h1>Browser Bridge</h1>
+    <p>
+      Paste the installation key shown in the WebUI Settings for this browser.
+      It is stored in this browser only, and is never shown again in full.
+    </p>
+    <label for="installation-key">Installation key</label>
+    <input id="installation-key" type="password" autocomplete="off" spellcheck="false" />
+    <div class="row">
+      <button id="save" type="button">Save</button>
+      <button id="clear" type="button">Clear</button>
+    </div>
+    <p id="stored"></p>
+    <p id="status" role="status"></p>
+    <script type="module" src="./src/options.ts"></script>
+  </body>
+</html>

packages/browser-extension/package.json

@@ -0,0 +1,17 @@
+{
+  "name": "@mavis/browser-extension",
+  "version": "0.1.0",
+  "description": "Edge/Chrome extension that bridges a person's own browser to the local WebUI browser bridge.",
+  "license": "MIT",
+  "type": "module",
+  "private": true,
+  "scripts": {
+    "typecheck": "tsc -p tsconfig.json --noEmit",
+    "build": "node scripts/build.mjs",
+    "test": "vitest run"
+  },
+  "devDependencies": {
+    "typescript": "^5",
+    "vitest": "^4"
+  }
+}