WebUI Browser Use — specification (isolated Chromium + Edge/Chrome extension bridge)

Parent specification for the WebUI Browser Use programme — not an executable work item.
The executable work items are the tickets listed under Ticket board at the bottom of
this issue. Each ticket cites this issue as its parent.

Decision authority: ADR 0019
in the repository (packages/webui/docs/adr/0019-…md). ADR 0019 is the sole decision record for
this feature; this issue and the tickets must not restate or reopen its decisions.

Development plan: .plan/browser-use-development-plan.md in the working checkout. The plan is
workspace material and is deliberately not part of the published source projection.

Problem Statement

A person using WebUI cannot drive a browser from it. The Agent can run browser tools, but the
person cannot see what the Agent is looking at and cannot take over the same page: the WebUI has an
adoption seam for a host-supplied browser provider, but neither the development server entry nor the
packaged production entry supplies one, there is no browser panel, and Settings has no browser
authorization controls. The person must also fall back on the isolated browser in another surface
where they cannot see or steer the session.

When the Agent works in the user's own already-logged-in Edge or Chrome, the person wants the Agent
to work in that browser — with their existing sessions and sign-in — and wants to watch and steer
the same tab. Today there is no supported way to reach a running Edge or Chrome at all, and the
documented workaround (launching with a remote-debugging port and handing over a CDP endpoint) is
not a product path.

The person also needs a clear answer to "what is the Agent allowed to do in my browser?": today no
browser-surface authorization exists, so there is nothing to grant, revoke, or audit.

Solution

WebUI gains Browser Use as a host-owned runtime capability with two browser surfaces, both
authorised explicitly and both default-off:

An isolated Chromium rendered inside a WebUI workspace panel through a live image stream,
with the person and the Agent operating the same browser session.
The person's own Edge and Chrome, reached through an installed browser extension that pairs
with the WebUI over an authenticated loopback WebSocket bridge and controls the browser through
the extension's debugger capability.

The panel shows the browser as images and forwards pointer, keyboard and text input; it never
iframes the target site. Browser tools and the panel address the same underlying session, so an
Agent action and the visible page state stay aligned. Each Agent run owns its own target
selection, and the panel follows the active run's target.

Four independent authorization switches decide what is allowed: isolated browser, Edge, Chrome, and
a separate Full CDP access switch for the provider's complete command set. All four default
off. A separate, per-browser, high-entropy pairing credential authenticates the extension;
turning a browser switch off revokes that credential and the connection immediately.

User Stories

As a WebUI user, I want a browser panel in my workspace, so that I can see the page the Agent is working on.
As a WebUI user, I want the panel to show live images of the page rather than an embedded site, so that the panel works for any site the Agent visits.
As a WebUI user, I want to click in the panel, so that I can interact with the page the Agent is using.
As a WebUI user, I want to type into the panel, so that I can fill in forms alongside the Agent.
As a WebUI user, I want my click and the Agent's click to affect the same page, so that the panel and the Agent never disagree about what is on screen.
As a WebUI user, I want the panel to follow the Agent run I am looking at, so that I see that run's tab and not another run's tab.
As a WebUI user, I want each Agent run to keep its own tab, so that two concurrent runs do not fight over one browser target.
As a WebUI user, I want the Agent to re-find elements after it changes tab, so that it never acts on the previous page using a stale reference.
As a WebUI user, I want to switch to another authorized tab myself, so that I can steer the Agent to a different page.
As a WebUI user, I want to see every tab the extension can reach, with its browser, title, URL and connection state, so that I know what the Agent can choose.
As a WebUI user, I want to know which tab belongs to the active Agent run, so that I can tell my run apart from another.
As a WebUI user, I want to enable the isolated browser from Settings, so that I can give the Agent a browser of its own.
As a WebUI user, I want the isolated browser off by default, so that no browser starts until I ask for one.
As a WebUI user, I want the isolated browser to use its own throwaway profile, so that it cannot touch my real browser profile.
As a WebUI user, I want the isolated browser's process, profile and streams released when WebUI closes, so that nothing is left running.
As a WebUI user, I want to enable Edge and Chrome independently, so that I can use one browser and not the other.
As a WebUI user, I want a clear "install the extension" guidance when the extension is missing, so that I know what to do.
As a WebUI user, I want the extension to tell me when a permission is missing, so that a blocked tab is not a mystery.
As a WebUI user, I want the extension to pair automatically once I enable its browser, so that I am not asked to confirm a second time.
As a WebUI user, I want my Edge and Chrome authorizations to survive a restart, so that I do not re-pair every time.
As a WebUI user, I want turning a browser switch off to disconnect that browser immediately, so that the Agent loses access at once.
As a WebUI user, I want the old pairing credential to be refused after I turn the browser off, so that a stale extension cannot reconnect.
As a WebUI user, I want a separate Full CDP access switch, so that the Agent's deeper browser control is my explicit decision.
As a WebUI user, I want Full CDP access off by default, so that ordinary browsing is not silently powerful.
As a WebUI user, I want the Full CDP switch to describe that it grants diagnostic and internal browser capabilities, so that I know what I am granting.
As a WebUI user, I want ordinary browser operations to keep working when I turn Full CDP access off, so that I can narrow the grant without losing the feature.
As a WebUI user, I want a command the browser does not support to fail honestly, so that I am not told something worked when it did not.
As a WebUI user, I want a dropped extension connection to fail visibly instead of silently switching tabs, so that I can tell the Agent stopped acting.
As a WebUI user, I want the Agent's in-flight action to fail explicitly when the connection drops mid-action, so that I am not left with an uncertain result.
As a WebUI user, I want the browser connection to recover on its own after a brief drop, so that a transient hiccup does not end the run.
As a WebUI user, I want the extension's browser permission to be independent of my WebUI switch, so that granting one does not imply the other.
As a WebUI user, I want the extension identity checked at the bridge, so that no other extension can attach to my WebUI.
As a WebUI user, I want the WebUI's own origin policy to stay narrow, so that a web page cannot open a socket to my local WebUI.
As a WebUI user, I want a stable, readable error for every failure — no browser, no tabs, pairing required, reconnecting, target closed, stream failed, permission missing — so that I can act on it.
As a WebUI user, I want the Edge and Chrome builds to come from one shared extension source, so that the two browsers behave identically.
As a maintainer, I want the extension source in the public source projection with its own typecheck gate, so that it is reviewed like the rest of the repository.
As a maintainer, I want the Full CDP gate to live in the runtime/provider path, so that a client request cannot bypass it.
As a maintainer, I want pairing credentials in the operating system's secure store, so that a plaintext fallback can never exist.
As a maintainer, I want the browser authorization state to be independent of provider presence, so that "a provider exists" never means "the user authorized it".
As a maintainer, I want a capability matrix per browser, so that the documented capabilities match the browsers' real ones.

Implementation Decisions

These are the decisions the implementation must honour. ADR 0019 holds the rationale; this list is
the operational summary, not a second decision record.

Two surfaces, one browser business. The isolated Chromium and the extension-backed Edge and
Chrome are both reached through one host-owned provider seam. Browser Use is a runtime capability
with explicit configuration, not a WebUI plugin.
One Browser client business. It owns browser target state, authorization and connection
projections, browser commands and the shared target state used by the Agent and the panel.
Settings and workspace surfaces consume its published contracts; they own no parallel state.
Extension as a separate workspace package. The extension is a separate browser execution
context with its own manifest, permissions, build target and verification surface, and it imports
no WebUI client or Node runtime implementation module.
Dedicated extension admission path. The existing WebSocket admission stays narrow and keeps
rejecting extension origins. A separate authenticated upgrade path admits extensions, ahead of
the generic handler, and it validates pairing credentials independently of any development-mode
bypass.
Per-browser credential, challenge-response pairing. Each browser has a distinct high-entropy
credential. The WebUI side uses the operating system's secure credential store; the extension side
uses extension local storage. A missing secure store is an explicit pairing failure.
Revocation on disable. Turning a browser switch off revokes the credential, closes the
sockets and rejects every copy of the old credential; turning it on creates a fresh credential
and pairs again without a second confirmation.
Run-scoped targeting with generations. Each Agent run owns its target selection. A selection
carries a generation, so a changed target, a closed tab or a reconnect invalidates the element
references that belonged to the previous page. Operations against one tab serialize; different
tabs proceed independently.
Bounded reconnect, no uncertain replay. A disconnect during an operation retries the
connection within a bounded deadline and a pinned backoff schedule, keeping the exact target. At
the deadline the operation fails explicitly and is not replayed. Background reconnection continues
with a capped backoff while the browser switch stays on.
Two-mode CDP capability gate. Ordinary Browser Use runs on a restricted command set. The
separate Full CDP access switch exposes every command the active provider actually supports. The
gate lives in the runtime/provider path, so a client request cannot bypass it. A command the
provider does not support fails as unsupported and never falls back silently.
Both composition roots. The development entry and the packaged production entry must supply
the provider consistently. Validating only the development path does not prove the shipped path.
Panel is an image surface. The panel consumes the live image stream and forwards validated
input commands. It does not iframe the target origin, and the local-HTML preview is not
repurposed as a control surface.

Testing Decisions

Test observable behaviour, not implementation detail. A test asserts what a user or an Agent
can observe, never that a testid string exists. When a validator is touched, its message text is
pinned exactly.
Use the highest seam available. Contract and validator behaviour is covered by unit tests;
assembled runtime and server behaviour by the application-level suites; the real service,
dispatcher and loopback socket by the loopback integration suite; the built client by the
built-client browser suite; a real installed extension by the manual acceptance matrix.
Prefer existing seams. Browser-core's provider-neutral transport and the runtime host's
browser adapter and tool exposure types are the seams to build on. The WebUI must not import the
terminal client package.
No DOM test framework. Interaction coverage extracts the transitions into pure functions and
asserts render branches through static markup rendering.
Every new test file is registered. A new Vitest file must be listed in the repository's
declared suite groups or it never runs.
A fake provider is not acceptance. The isolated Chromium path must be exercised against the
real provider, and the packaged production entry must be exercised for the shipped claim.
Synthetic and mock peers prove protocol and routing only. They do not prove browser permission
prompts, extension installation, real CDP command support, MV3 worker lifetime, or control of an
authenticated browser. Synthetic pages contain no account data; recorded acceptance states the
browser version, the operating system, the enabled settings and the observed result.
Adversarial coverage is part of the definition of done. Origin spoofing, non-loopback hosts,
wrong browser identity, stale credentials, oversized or malformed frames, duplicate connections,
stale generations, unauthorized operations, both Full CDP gate states, and revocation during an
in-flight action each have a test.

Out of Scope

Silent installation of browser extensions, or any bypass of a browser's own permission prompts.
Requiring the user to expose a remote-debugging port or to supply a CDP endpoint by hand.
Native Messaging as the extension transport.
Cross-device or LAN access to the extension bridge.
Browser automation features the selected browser-core and runtime contract does not already
support, unless a new decision is recorded in ADR 0019 first.
Publishing the extension to the Chrome Web Store or to Edge Add-ons. Store submission is a
separate release action; the implementation prepares the artifacts and the disclosures only.

Further Notes

The development plan that produced this board is workspace material, deliberately outside the
published source projection. The board and the plan must not drift apart: an amendment belongs in
the plan first, and the affected tickets are updated to match.
Research notes captured before implementation, for the implementers:
#173 runtime and server seams,
#174 browser-core contracts and client conventions,
#175 repository plumbing for a new package and gates.
Ticket 01 is the technical validation work package. It blocks every other ticket, because the
capability matrix, the bridge protocol, the credential-store feasibility and the reconnect
contract are what the later tickets are written against.
If a technical finding requires a product change, ADR 0019 is updated first, then the affected
tickets are re-scoped.

Ticket board

Ticket	Title	Blocked by
#177	01 — Validate the technical contracts and pin the capability matrix	—
#178	02 — Four independent browser authorization switches, default off	#177
#179	03 — Isolated Chromium provider starts under authorization	#177, #178
#180	04 — Browser panel shows the isolated Chromium's live page	#179
#181	05 — User pointer and keyboard input reach the same isolated page	#180
#182	06 — Agent browser operations act on the page the panel shows	#179, #180
#183	07 — Target generations invalidate stale element references	#182
#184	08 — Full CDP access is a separate gate in the runtime/provider path	#179
#185	09 — Extension package, dedicated loopback admission and pairing	#177, #178
#186	10 — External browser tabs become targets the panel follows	#180, #185
#187	11 — Live page images from the external browser reach the panel	#186
#188	12 — User input reaches the external browser tab	#186
#189	13 — Agent operations run against an external tab through the bridge	#185, #186
#190	14 — Per-browser pairing credential lifecycle	#185
#191	15 — Bounded reconnect during an operation, background reconnect while enabled	#190
#192	16 — Extension worker termination and restart recover the bridge	#187, #191
#193	17 — Edge and Chrome parity, store builds and real-browser acceptance	#188, #189, #192
#194	18 — Security hardening, adversarial coverage, packaging review and documentation	#193

The blocking edges above are native GitHub issue dependencies, so a ticket is unblocked exactly
when every ticket it names is closed. The research notes are #173, #174 and #175; they are reference
material, not work items, and are deliberately not labelled ready-for-agent.

Progress on the WebUI Browser Use programme

The programme is implemented on feat/webui-browser-use — PR #195 against webui.

Where it stands


Tickets met and closed	#177, #178, #179, #180, #181, #183, #191
Tickets open with a status comment	#182, #184, #185, #186, #187, #188, #189, #190, #192, #193, #194
Gates	`pnpm verify` — 21 gates pass on linux; `pnpm test:webui` — 196 files, 2994 tests; `pnpm typecheck:webui` — clean; `pnpm check:webui-dependency` — 0 violations
Independent review	Three rounds by a `codex` `gpt-6-luna` reviewer reading the source. The record is `packages/webui/docs/browser-use/07-review-findings.md`.

What is delivered

Three surfaces, four independent switches, all defaulting off: the isolated Chromium this process owns, the person's own Edge and Chrome through the extension bridge, and a separate Full CDP switch that widens an already-authorized surface rather than authorizing one.
The panel shows a live image stream, not an iframe, and forwards pointer, keyboard and text input into the same page the Agent's tools drive.
The extension bridge is its own path — /browser-bridge, its own socket, its own origin rule, its own credential. A dev: true WebUI still refuses an unauthenticated extension, and a test proves it over a real socket.
Pairing is challenge-response, the secret never reaches a process argument, a log line or an error message, and it lives in the OS secure store — with no plaintext file fallback.
Target generations make a stale input or a stale frame a refusal rather than a mis-aimed delivery.
The switches own the pairings: a surface that is off holds no credential, admits no session, and refuses its old credential from then on.

What is still open, and why

Stated on each ticket, and in the review record:

Agent browser operations act on the page the panel shows #182 — the Agent's browser tools are not yet wired to the browser service. The shared session, the restricted command set and the command operation exist; the tool-side adapter does not.
Browser extension package, dedicated loopback admission and challenge-response pairing #185 and the external path (External browser tabs become targets the panel follows #186–Agent operations run against an external tab through the bridge #189, Per-browser pairing credential lifecycle: revoke on disable, fresh pairing on enable #190, Extension worker termination and restart recover the bridge #192) — the host and the extension mint independent pairing credentials, so an extension cannot yet answer a challenge the host issued. The fix is an initial authenticated pairing exchange carrying a one-time code from the host to the extension, which is a product surface to design rather than a patch to wire. The bridge's own machinery around it — the dedicated admission, the switch-driven credential lifecycle, the target-ownership routing, the reconnect schedule — is in place and tested.
Full CDP access is a separate gate in the runtime/provider path #184 — Full CDP is refused outright on the external surfaces, because chrome.debugger exposes only a subset of the DevTools domains. The runtime gate and the isolated-Chromium path are in place.
Edge and Chrome parity, store builds and real-browser acceptance #193, Security hardening, adversarial coverage, packaging review and documentation #194 — no real Edge or Chrome was available, so the external path is exercised against a synthetic peer. The published evidence says so rather than implying otherwise.

The evidence

packages/webui/docs/browser-use/ — the capability matrix, the bridge protocol, the extension admission rules, the pairing credentials, the target identity, the runtime authorization trace, and the review record. Each document separates what was measured (and names the probe artifact), what is implemented (and names the source), and what is not yet measured.

Follow-up: three more findings closed

A third review round found six issues. Three more have now been fixed and the tree is
green again.

The Agent's browser tools now drive the same session as the panel.
packages/webui/src/runtime/browser/tool-adapter.ts implements
LocalBrowserAdapter over the browser service. execute forwards the action to
the service, which reads the surface switch and the Full CDP gate — so the
authorization is decided in one place rather than twice. An action outside the
restricted set is refused with browser_capability_denied even when Full CDP is
on, because the tool path is the restricted path by construction.
The target selection is scoped to the run that made it. The service keys its
selection by (surface, runId), where the run is the Agent session id. Two runs on
one surface keep their own selections, each run's image stream shows its own
target's frames, and a run with no selection is refused rather than falling back
to another run's.
The host and the extension now share a pairing credential. The extension
generates an ephemeral P-256 ECDH key pair per connection and sends the public key
in its hello; the host answers with a pair frame carrying the credential
encrypted to it — ECDH, then HKDF-SHA256, then AES-256-GCM — with the host's own
ephemeral public key, the salt and the ciphertext. The extension decrypts it,
stores it in chrome.storage.local and answers the challenge with it. The secret
never crosses the wire in the clear, and no person has to carry a code. Both
sides pin the same derivation vector, so the two implementations cannot drift.

The decision and its limits — including that the extension's identity rests on an
origin allowlist, which a local process can claim — are recorded in the ADR's new
Amendments section (A1–A3).

Ticket state

Closed: #177, #178, #179, #180, #181, #182, #183, #185, #191.
Still open: #184, #186–#190, #192–#194.

The one review finding still open is Full CDP on the external surfaces: the
bridge refuses every raw DevTools command, because chrome.debugger exposes only a
subset of the DevTools domains. The runtime gate and the isolated-Chromium path are
in place; what is missing is one more bridge frame and one more extension-side
dispatch — and the size of that subset is exactly the thing that cannot be verified
without a real browser.

Gates

pnpm verify — 21 gates on linux. pnpm test:webui — 197 files, 3037 tests. The
extension package's own suite — 46 tests. pnpm typecheck:webui clean.

Review round 4

A fourth codex gpt-6-luna review found three more issues. Two are fixed; one is
recorded as open, with the reason.

The image stream now follows the run's selection. It used to stay attached to
the target that was selected when the stream started, so choosing another tab changed
which frames passed the fence without changing what the browser was capturing. The
stream now restarts the provider's capture when the target changes and stops the
previous target's screencast, so nothing is left running for a tab the person moved
away from.
The Agent tool path no longer picks a browser for itself. It used to default to
the isolated Chromium even when the person was looking at Edge or Chrome. It now asks
the service which surface the run is working on — derived from the run's own
selection, the same state the panel writes — and a run that has selected nothing is
refused with a stable browser error code rather than being sent somewhere the
person is not looking.
The published evidence no longer claims the tool adapter does not exist, nor that
the host and the extension mint independent credentials. Both were true when the
documents were written and false by the time they were read.

The one that stays open, and why

The bridge is not authenticated against a local process. Admission selects the
allowlisted extension id from the Origin header, and the host then seals its pairing
credential to the public key that peer supplied. A local process can put that origin
on a loopback socket, supply its own key, decrypt the credential and answer the
challenge.

The exchange protects the credential in transit and nothing more. It is not an
authentication of the peer, and the decision record now says so plainly rather than
leaving it as an implication. What is needed is an installation key the extension
holds and a local process does not — provisioned at install, proved during the
exchange, and bound to the key the credential is sealed to. The native-messaging route
the ADR's own research findings list is the other candidate, and it has the same
property: a registered host process rather than a spoofable header.

Until that exists, the switch that enables an external surface enables it for the
person's browser; it is not a boundary against a local process. That is stated in
packages/webui/docs/adr/0019-... (amendment A1) and in
packages/webui/docs/browser-use/07-review-findings.md.

Gates

pnpm verify — 21 gates on linux. pnpm test:webui — 197 files, 3049 tests. The
extension package's own suite — 46 tests. pnpm typecheck:webui clean.
pnpm check:webui-dependency — 0 violations.

The peer-identity blocker is closed; the extension now runs in a real browser

Two things changed since the last update, and both were found by probing the real
browser rather than by reading the source.

The extension could not actually run

Every test of the extension ran against its TypeScript source, so nothing
noticed that a browser extension runtime executes JavaScript only. Loading
the source directory into a real Chrome registered a service worker and then
failed to run it — the extension looked installed and did nothing. The evidence
is in .plan/board/validation/extension-load-evidence.json.

The extension now has a build step that bundles its TypeScript to JavaScript
and emits a dist/ whose manifest points at the built paths. Re-probed against
the same Chrome, the built extension registers, parses, runs and answers an
evaluation.

The bridge is authenticated by an installation key

Admission selected the allowlisted extension id from the Origin header — and a
local process can put any Origin on a loopback socket. The host then sealed
its pairing credential to the public key that peer supplied, so a local process
could be admitted as the extension, supply its own key, decrypt the credential
and answer the challenge.

The peer now has to prove possession of the surface's installation key — a
32-byte secret the host keeps in the OS secure store and the extension keeps in
chrome.storage.local, provisioned once per browser by an explicit paste from
the WebUI Settings surface.

The handshake is proof-first: hello → challenge → proof, and only then,
for a peer that proved the installation key, the sealed pairing credential. A
peer that proves only a credential the host already issued is admitted without a
further credential. A peer that proves neither is refused and receives no
credential at all — the headline test asserts that no pair frame crosses to
an unauthenticated peer, on a real ws socket.

The ECDH derivation, the HKDF info string and the shared test vector are
unchanged: the key changes who may receive a credential, not how it is
sealed.

The cost is recorded rather than glossed: provisioning is explicit, because the
automatic pairing ADR 0019 describes is automatic with respect to tabs, not
with respect to peer identity. Amendment A2 says so.

Gates

pnpm verify — 21 gates on linux. pnpm test:webui — 197 files, 3069 tests. The
bridge suites — 84 tests. The extension's own suite — 55 tests. pnpm typecheck:webui clean. pnpm check:webui-dependency — 0 violations.

Still open

#184 — Full CDP on the external surfaces is refused, because
chrome.debugger exposes only a subset of the DevTools domains. The runtime
gate and the isolated-Chromium path are in place; the bridge needs one more
frame and one more dispatch, and the size of that subset is exactly what cannot
be verified without a real browser.

#193 — the external path has still not been driven end to end against a real
browser. The extension now executes in real Chrome, which is the first step;
the handshake itself has only been exercised against a synthetic peer.

The external path, against a real browser

The external path had only ever been exercised against a synthetic peer. A
Playwright spec now loads the built extension into a real Chrome and
points it at a real bridge server:
packages/webui/test/browser/browser-bridge-real-browser.spec.ts.

What it proves. An extension that cannot prove the installation key is
refused: the host reports the surface as configured and enabled but not
connected, and no credential crosses to the peer. That is the security
property that matters, and it is now established against a real browser rather
than a synthetic one.

What it does not yet prove. The happy path — the same extension, with the
installation key provisioned, completing the handshake — does not complete
in the harness, and the reason is not established. The extension never opens a
connection, and it records no failure of its own, so the service worker that
should answer the challenge is either not starting after the reload or is
starting without the origin it was given. That test is marked fixme rather
than deleted, so the gap stays visible and the test runs as soon as it can.

What this means. The external path is not yet demonstrated working in a
real browser. #193 stays open, and the capability matrix keeps the external
surfaces marked not measured rather than working.

Gates

pnpm verify — 21 gates on linux, including the browser gate that now runs the
real-browser spec. pnpm test:webui — 197 files, 3069 tests. The bridge
suites — 84 tests. The extension's own suite — 55 tests. pnpm typecheck:webui
clean. pnpm check:webui-dependency — 0 violations.

The external path now works against a real Chrome

The real-browser spec passes both halves:

the handshake completes — the built extension, provisioned with the
installation key, connects to a real bridge server and the host reports the
surface as connected;
the refusal holds — the same extension without the key is refused, and no
credential crosses to the peer.

The defect it found

A Manifest V3 service worker is lazy: it starts on an event and is torn
down when it goes idle. Saving the installation key did not wake it, so the
extension sat there configured but unconnected, and the options page told the
person to "Reconnect the extension to pair with the host" — a manual step
nobody checks and nothing verifies.

The options page now asks the worker to reconnect, and the worker answers by
stopping the current client and starting a new one against the freshly stored
configuration. The person pastes a key and the extension pairs.

Gates

pnpm verify — 21 gates on linux, including the browser gate that now runs the
real-browser spec. pnpm test:webui — 197 files, 3069 tests. The browser
suite — 133 tests, including the two real-browser ones. The bridge suites — 84
tests. The extension's own suite — 55 tests. pnpm typecheck:webui clean.
pnpm check:webui-dependency — 0 violations.

Still open

#193 stays open for one reason only: Edge was not exercised. Edge is not
installed on the machine that produced this change, so the parity claim is
still a design claim rather than a measurement. Chrome is proven end to end.

#184 — Full CDP on the external surfaces: the bridge needs one more frame
and one more dispatch, and the size of chrome.debugger's supported subset is
what cannot be settled without a real browser on each.

Full CDP on the external surfaces

The bridge refused every command on Edge and Chrome, because chrome.debugger
exposes only a subset of the DevTools domains and pretending otherwise
would be a lie.

The host now sends a raw DevTools command as a cdp frame and the extension
answers with a cdp-result. The extension keeps an explicit list of the
commands it will send and refuses anything outside it before it makes any
browser call; the refusal says unsupported, so the host can tell "the
extension will never run this" from "this failed" — two different things to a
caller.

The host's gate is unchanged and still comes first: the surface must be
authorized, and the command must be a restricted browser-tool action or
permitted by the Full CDP switch. The provider no longer refuses a raw command
on the surface's own authority, because that would make the Full CDP switch a
lie.

A command acts on a page, so a page has to be selected first. The command
operation now carries the run it acts for and acts on that run's selection —
the same one the panel and the Agent's tools use — so the three cannot drift
apart. Routing is the input path's: the same ownership check, the same
per-target serialization, the same refusal of a stale generation.

Gates

pnpm verify — 21 gates on linux. pnpm test:webui — 198 files, 3075 tests.
The extension's own suite — 61 tests, including 6 for the DevTools path. The
host's bridge routing — 6 new tests. pnpm typecheck:webui clean.
pnpm check:webui-dependency — 0 violations.

Still open

#193 — the store artifacts: a Chrome Web Store build and an Edge
Add-ons build from the shared source, the per-build extension identity
validation, the developer-versus-packaged distinction, and the disclosures. Both
browsers are exercised; what is missing is the packaging and the disclosure
work, and the fact that the extension's chrome.debugger subset was not
dispatched against a live tab in a real browser.

A fifth review, and two things it caught

The fifth codex gpt-6-luna review found two MAJOR issues, both of them
about my own claims rather than about the code.

The Edge run was authenticating the Chrome surface

The acceptance launched both Chrome and Edge — but both runs configured the
Chrome surface: the same allowlist, the same pairing, the same credential.
So the Edge run proved that an Edge-hosted extension completes the
handshake, and proved nothing about the Edge switch, the Edge pairing
credential or the Edge routing.

The acceptance is now parameterised by surface. The Chrome run is admitted
on the chrome surface with the Chrome allowlist; the Edge run on the edge
surface with the Edge allowlist; each asserts the matching surface's state.
Parameterising it immediately exposed a real bug — the installation key was
read from a hard-coded surface, so the Edge run had no key to paste.

All four cases pass, and each browser is now admitted on its own surface.

The published documents contradicted the tree

The capability matrix, the bridge protocol and the extension admission
document still said the bridge was unwired and that no Agent tool adapter
existed. The review record itself contradicted itself about which browsers had
been exercised. All of it is reconciled against the tree now.

Gates

pnpm verify — 21 gates on linux. pnpm test:webui — 198 files, 3075 tests.
The browser suite — 135 tests including the four real-browser ones. The
extension's suite — 61 tests. pnpm typecheck:webui clean.
pnpm check:webui-dependency — 0 violations.

The external path, driven end to end in a real browser

The handshake acceptance proved that an extension can pair. It did not prove
that the bridge works — that a keystroke the host sends arrives in the page
the person is looking at, that the page's image reaches the host, or that a
DevTools command runs against the live tab.

A new acceptance drives all of it through the real bridge, in real Chrome and
real Edge, each on its own surface, against a page served over loopback
(the extension deliberately leaves policy-blocked URLs — data: among them —
out of its targets, so a data: URL would have proved nothing).

It found four real defects, all now fixed:

The extension's frame parser had no cdp case. A DevTools command was
rejected as an unknown frame kind and the client closed the socket — which
is why the first attempt said "session closed before reply" rather than a
refusal.
A refusal was turned into a transport failure too early. The session
converted the extension's refusal into a terminal retry outcome, so the
provider never saw it and the stable error code the contract promises never
applied. A refusal is an answer, not a failure.
The provider discarded the command's result, so a caller had no way to
read what the command produced.
The extension could not capture the page at all.
chrome.tabs.captureVisibleTab needs the <all_urls> host permission or an
activeTab permission granted by a user gesture — and there is no gesture
here, the host is asking for the picture. The permission is now declared and
the reason is recorded in the extension's README rather than left implicit.

Gates

pnpm verify — 21 gates on linux. pnpm test:webui — 198 files, 3075 tests.
The bridge suites — 95 tests. The extension's suite — 61 tests. The browser
suite — 139 tests, including eight real-browser ones. pnpm typecheck:webui
clean. pnpm check:webui-dependency — 0 violations.

Tickets

Closed: #177, #178, #179, #180, #181, #182, #183, #184, #185, #186, #187,
#188, #189, #191.

Still open: #190, #192, #193, #194 — the OS secure store's macOS and Windows
forms are not executed here, the worker's termination during an active stream
is not exercised, the store artifacts and disclosures do not exist yet, and the
final acceptance matrix is not complete.

All eighteen executable work items on the board are closed, and the specification's own testing
decisions are now met by the tree rather than by a plan.

What the programme delivers

The isolated Chromium, launched and owned by the process, reached through a live image stream
in a workspace panel. The person and the Agent operate the same session; the panel shows images
and forwards validated pointer, keyboard and text input, and never iframes the target site.
The person's own Edge and Chrome, reached through an installed extension that pairs over an
authenticated loopback bridge. The extension's chrome.debugger subset carries the DevTools
commands; the panel follows the active run's tab, and each run keeps its own selection.
Four independent authorization switches, all default-off, and a separate per-browser
high-entropy pairing credential in the operating system's secure store. Turning a switch off
revokes the credential, closes the sockets and refuses every copy of the old one.
A dedicated extension admission path that the development-mode bypass does not reach.

The specification's testing decisions, and where each is met

Decision	Where it is met
A fake provider is not acceptance — the isolated Chromium path is exercised against the real provider	`packages/webui/test/integration/browser-use-real-input.test.ts` — the input arrives in the page the provider is driving, and a stale identity is refused with `browser_target_stale`
…and the packaged production entry is exercised for the shipped claim	`packages/webui/test/browser/browser-bridge-real-packaged.spec.ts` — the packaged entry is a packaged host; each store build's identity is checked against the browser that assigns it, and the same packaged host refuses a development identity
Synthetic and mock peers prove protocol and routing only	The unit suites hold that evidence; the real-browser suites hold the real evidence; the published matrix names which is which in every row
Recorded acceptance states the browser version, the operating system, the enabled settings and the observed result	The suites write the record as they run — `packages/webui/test/browser/acceptance-record.ts` — and the published matrix is written from it
Adversarial coverage is part of the definition of done	`packages/webui/test/unit/browser-bridge-adversarial.test.ts` — oversized and malformed frames, a duplicate extension connection, and a revocation during an in-flight action. The rest are covered by the admission, pairing, session, registry, target-registry and CDP suites
Every new test file is registered	`test/vitest-suites.json` names every Vitest file this distribution runs

The published evidence

Document	Owns
`packages/webui/docs/browser-use/01-capability-matrix.md` … `06-runtime-authorization-trace.md`	The pinned contracts, the capability matrix, the bridge protocol, the admission path, the credential lifecycle, the target identity and the authorization trace
`packages/webui/docs/browser-use/07-review-findings.md`	Seven review rounds — six by an independent `codex` `gpt-6-luna` reviewer reading the source, and a final acceptance pass against the finished tree — and what was done about each finding. Every finding is closed
`packages/webui/docs/browser-use/09-acceptance-record.md`	The acceptance matrix: twenty cases across the isolated Chromium, Edge and Chrome, with the browser version, the operating system, the settings and the observed result
`packages/browser-extension/STORE-DISCLOSURES.md`	The disclosures for both stores, and the differences between a packaged build and a development build

ADR 0019 remains the only Browser Use decision record. The amendments it carries are the decisions
the review changed, not a second decision list.

What is deliberately not claimed

The macOS and Windows credential-store commands were not executed. This host is Linux and has
neither security nor powershell.exe. The three platform forms are asserted against the
platform command contract through an injected runner; the Linux form's availability was probed
separately and was unavailable on this host.
Nothing was measured on macOS or Windows. Every row of the acceptance matrix is a Linux host.
One extension package for both stores is not the behaviour being the same.
No store was submitted to. Submission and publication are a separate release action, and the
ticket says so. The store identities do not exist yet — a store assigns them at first publication
— so the builds are produced from a public key supplied at build time.
The Agent's own path was not driven against a real browser in the final pass. The tool adapter
forwards to the same browser service as the panel, and a run that has selected nothing is refused
rather than being sent somewhere the person is not looking.

The pull request carries the branch, the commits and the gate evidence.

Two numbers in the closing comment above are out of date, and one more review round has run since.
This comment corrects them and records the last round.

Corrections

Claimed above	Actual
"Seven review rounds — six by an independent `codex` `gpt-6-luna` reviewer … and a final acceptance pass"	Ten review rounds — nine by the `codex` `gpt-6-luna` reviewer, and a final acceptance pass. Two probe rounds are recorded in the same document alongside them: they are findings of the implementation rather than of a reviewer, and they are numbered `Probe A` and `Probe B` so they no longer collide with rounds 5 and 6.
"The acceptance matrix: twenty cases across the isolated Chromium, Edge and Chrome"	Twenty-one cases, twenty-one passes. The count above was one short; the matrix in the published record has twenty-one rows.

Every finding in all ten rounds is closed. One carries the qualifier fixed, platform unverified:
the macOS and Windows credential-store forms, which this host cannot execute. The two rounds that
changed a decision are recorded as
amendments A5–A7 in
ADR 0019:
the surface is settled by the browser and bound to that surface's own secret, a packaged host
accepts store identities and never development ones, and a surface serves one browser.

The tenth round

The ninth codex review read the tree the eighth round left behind and raised two findings. Both
were real, and both are fixed on the branch.

Finding	What was done
The session declared a liveness ping and never sent one. It held the pending-ping table, it answered a `pong`, and its own option comment said the session emits a `ping` — but no ping was ever written to the socket. A browser gone behind an open socket was therefore never noticed.	An admitted session now pings every ten seconds and closes when the pong does not come within one interval. A socket can look open long after the browser behind it is gone — the extension's worker is terminated, the profile is closed, the machine sleeps — and the socket's close is then the only signal, and a slow one. The extension's reconnect ladder picks the session back up. The extension already answered a `ping` with a `pong`, so nothing on that side changed.
The real-browser acceptance failed on a machine without the installed Edge. The Edge cases drive `/usr/bin/microsoft-edge`, so a CI image or a container that does not ship Edge reported a broken bridge rather than an absent browser.	The Edge cases skip with the reason when the installed Edge is not on the machine. The bundled Chromium is installed by the gate itself, so the Chromium cases always run. The record names the browser each case ran in and the published matrix names the host, so a skipped case is visible rather than silent.

The final state

All eighteen executable work items on the board are closed, and this specification is closed
with them. The three research notes (Research notes: WebUI runtime and server seams for Browser Use #173, Research notes: browser-core contracts and WebUI client conventions for Browser Use #174, Research notes: repository plumbing for a new package and gates #175) stay open: they are reference material,
not work items.
The published matrix is
09-acceptance-record.md:
twenty-one cases, twenty-one passes, across the isolated Chromium, Edge and Chrome, each row
naming the browser version, the operating system, the settings read from the host and the observed
result.
The pull request carries the branch, the commits and the gate evidence.

What is still deliberately not claimed

Unchanged from the closing comment, and still true of the tree:

The macOS and Windows credential-store commands were not executed. This host is Linux and has
neither security nor powershell.exe. The three platform forms are asserted against the
platform command contract through an injected runner; the Linux form's availability was probed
separately and was unavailable on this host.
Nothing was measured on macOS or Windows. Every row of the acceptance matrix is a Linux host.
One extension package for both stores is not the behaviour being the same.
No store was submitted to. Submission and publication are a separate release action, and the
ticket says so. The store identities do not exist yet — a store assigns them at first publication
— so the builds are produced from a public key supplied at build time.
The Agent's own path was not driven against a real browser in the final pass. The tool adapter
forwards to the same browser service as the panel, and a run that has selected nothing is refused
rather than being sent somewhere the person is not looking.

WebUI Browser Use — specification (isolated Chromium + Edge/Chrome extension bridge) #176

Description

Problem Statement

Solution

User Stories

Implementation Decisions

Testing Decisions

Out of Scope

Further Notes

Ticket board

Activity

Fectivnfy112357 commented on Oct 20, 2026

Progress on the WebUI Browser Use programme

Where it stands

What is delivered

What is still open, and why

The evidence

Fectivnfy112357 commented on Oct 20, 2026

Follow-up: three more findings closed

Ticket state

Gates

Fectivnfy112357 commented on Oct 20, 2026

Review round 4

The one that stays open, and why

Gates

Fectivnfy112357 commented on Oct 20, 2026

The peer-identity blocker is closed; the extension now runs in a real browser

The extension could not actually run

The bridge is authenticated by an installation key

Gates

Still open

Fectivnfy112357 commented on Oct 20, 2026

The external path, against a real browser

Gates

Fectivnfy112357 commented on Oct 20, 2026

The external path now works against a real Chrome

The defect it found

Gates

Still open

Fectivnfy112357 commented on Oct 20, 2026

Full CDP on the external surfaces

Gates

Still open

Fectivnfy112357 commented on Oct 20, 2026

A fifth review, and two things it caught

The Edge run was authenticating the Chrome surface

The published documents contradicted the tree

Gates

Fectivnfy112357 commented on Oct 21, 2026

The external path, driven end to end in a real browser

Gates

Tickets

Fectivnfy112357 commented on Oct 21, 2026

What the programme delivers

The specification's testing decisions, and where each is met

The published evidence

What is deliberately not claimed

Fectivnfy112357 commented on Oct 21, 2026

Corrections

The tenth round

The final state

What is still deliberately not claimed

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions