Repository navigation
Research notes: browser-core contracts and WebUI client conventions for Browser Use #174
Description
Reference material for the WebUI Browser Use programme — not a work item.
Captured before implementation, from the source at the current checkout. Path in the workspace: .plan/exploration/02-browser-core-client.md (gitignored, outside the published source projection).
Implementation context: browser-core contracts & WebUI client conventions
Scope: implementation-context notes for "WebUI Browser Use". Read-only survey; nothing was
implemented or modified. Every claim below carries a file:line. Items I could not confirm from
source are marked not verified — needs check.
0. Plan claims vs. the current code
Plan claim (.plan/browser-use-development-plan.md §3) |
Verdict |
|---|---|
packages/browser-core owns provider-neutral browser actions and transport contracts |
Confirmed — packages/browser-core/src/browser-transport.ts:58, browser-core.ts:124 |
WebUI must not import packages/tui; terminal-client implementations are references only |
Confirmed as a documented rule — packages/webui/docs/architecture/boundaries.md ("Never import packages/tui (ADR 0003)"). See §5.4 for the caveat that it is not enforced by a literal denylist. |
packages/webui/src/runtime/browser-provider.ts defines a host-supplied provider adoption seam |
Confirmed — runtime/browser-provider.ts:47 adoptWebuiBrowserProvider |
WorkspaceHtmlPreview.tsx previews local HTML in an iframe and must not be repurposed |
Confirmed — features/workspace/ui/WorkspaceHtmlPreview.tsx:1-30 |
| browser-core is importable by WebUI | Not true today — see §5.3. packages/browser-core is in release/extraction.json packageRoots, but packages/webui/package.json does not declare it and browser-core has a single "." export that resolves types from ./src/index.ts but runtime code to ./dist/index.js. |
1. What a provider must implement to serve browser-core
1.1 The one transport contract
BrowserTransport (packages/browser-core/src/browser-transport.ts:58) extends
BrowserPageLifecycle (:36). A provider supplies one object per page.
Required members (verbatim signatures):
// browser-transport.ts:36-49
export interface BrowserPageLifecycle {
readonly sessionId: string;
start(signal?: AbortSignal): Promise<void>;
/**
* Reset the provider connection/session without closing the page.
* ... Providers that do not need a reconnect may implement it as a no-op. `close()` is
* reserved for permanently destroying the page and its provider resources.
*/
resetConnection?(): Promise<void>;
stopLoading(): Promise<void>;
close(): Promise<void>;
}
// browser-transport.ts:58-100 (abridged to the members that matter)
export interface BrowserTransport extends BrowserPageLifecycle {
readonly downloadDirectory?: string;
openTab?(url: string): Promise<BrowserTransport>;
send<T = unknown>(
method: string,
params?: BrowserTransportCommandParams,
sessionId?: string,
options?: BrowserTransportCommandOptions,
): Promise<T>;
evaluate<T = unknown>(expression: string, options?: BrowserEvaluateOptions): Promise<T>;
evaluateInAllFrames?<T = unknown>(expression: string, options?: BrowserEvaluateOptions): Promise<readonly T[]>;
listAttachedFrames?(): readonly BrowserAttachedFrame[];
onEvent(listener: BrowserTransportEventListener): () => void;
waitForEvent(method: string, timeoutMs?: number,
options?: Pick<BrowserTransportCommandOptions, 'signal'>): Promise<unknown>;
}Optional: resetConnection (:46), openTab (:65), evaluateInAllFrames (:88),
listAttachedFrames (:93), downloadDirectory (:60).
Mandatory: sessionId, start, stopLoading, close, send, evaluate, onEvent, waitForEvent.
CdpTransport (:103) is a backward-compatible alias of BrowserTransport.
1.2 Narrower view for helpers
BrowserPageTransport (browser-page-transport.ts:17) is a Pick<BrowserTransport, …> over
send | evaluate | evaluateInAllFrames | listAttachedFrames | onEvent | stopLoading | close. Its
header comment states it exists "to prevent a second transport contract from drifting while
preserving the narrow dependency used by Electron". A new provider must implement
BrowserTransport, not BrowserPageTransport.
1.3 Addressing a page / target
- Page = one
BrowserTransportinstance. Identity issessionId(browser-transport.ts:37). - Child targets / OOPIFs are addressed by an explicit
sessionIdargument onsend(...)
(:77) and byBrowserAttachedFrame(:26-34), which carriessessionId,frameId,
optionalparentSessionId,url. BrowserCoreSessiontracks the child session instate.commandSessionId
(browser-state.ts:121);BrowserCoreResolvedTarget(browser-core.ts:61-67) is the
snapshot-scoped identity an adapter gets back for an opaque ref:
{ element, snapshotId, generation, frameId, frameRef }.
1.4 Command, image, input and cancellation flow
- Commands:
send(method, params, sessionId, options); params are
BrowserTransportCommandParams = Record<string, unknown>(browser-transport-types.ts:2).
Browser Core's own shorthand iscdp<T>(session, method, params)(browser-state.ts:325) and
evaluate<T>(session, expression)(browser-state.ts:353). - Mandatory provider error codes (
browser-transport.ts:66-73): providers must reject caller
cancellation withcode: "ABORTED"and a deadline withcode: "BROWSER_OPERATION_TIMEOUT", so
Browser Core can reset a connection whose late command may still reach Chromium. The code set is
BrowserOperationInterruptionCode = 'ABORTED' | 'BROWSER_OPERATION_TIMEOUT' | 'BROWSER_TRANSPORT_CLOSED'(operation-timeout.ts), built by
browserTransportInterruptionError()and classified byisBrowserOperationInterruption().
browserOperationFailure()renders the public{ success: false, code, error, label?, timeoutMs? }. - Cancellation flows:
BrowserCoreSession.execute(action, input, signal)
(browser-core.ts:258) creates oneAbortController, forwards the caller's abort, sets a timer,
and stores the signal asstate.commandSignal. On interruption it calls
transport.stopLoading(), runscdpHelper.recoverTransportAfterInterruption(...), then
invalidate(...)and setspoisonedwhen recovery/settlement fails (:336-353). A poisoned
session reportsrequiresRecreation(:205) and every later action throws
BROWSER_SESSION_POISONED. - Screenshots:
Page.captureScreenshotinbrowser-action-artifacts.ts:65,75; providers may
override via the optional seamBrowserScreenshotCapture
(browser-state.ts:97-100, wired atbrowser-core.ts:57/:150). - Input:
Input.dispatchMouseEvent(cdp-helper.ts:1302,1309,1335,1401),
Input.dispatchKeyEvent,Input.insertText,Input.dispatchDragEvent
(cdp-helper.ts:1362-1377),Input.setInterceptDrags(:1330). - Deadlines (
operation-timeout.ts):cdpCommand: 30_000,cdpScan: 30_000,
tabOperation: 60_000,longTabOperation: 125_000,ipcHandler: 275_000, nested
"inner settles before outer".
1.5 Connection state and reconnect
- The only reconnect seam is the optional
resetConnection()(browser-transport.ts:39-46).
Browser Core calls it throughCDPHelper(cdp-helper.ts:217,:269);cdp-helper.ts:224
comments that a provider "must implement resetConnection so timeout recovery is page-safe". - Reference implementation:
packages/tui/src/runtime/browser/headless-chrome-transport.ts:604
(resetConnection), forwarded byheadless-chrome-provider.ts:648. - Session poison/reset:
browser-core.ts:205requiresRecreation,:264-268
BROWSER_SESSION_POISONED,:300-353recovery + settlement. synchronizeGeneration(n)(browser-core.ts:251) lets a host invalidate refs after navigation it
observed outside Browser Core.
2. The CDP commands browser-core issues today (the restricted set)
Distinct method names, by domain, with the issuing file. This is the baseline a Full CDP gate
extends.
Page — Page.navigate (browser-actions.ts:203), Page.navigateToHistoryEntry (:245),
Page.reload (:276), Page.getFrameTree (:348), Page.getNavigationHistory (:359),
Page.getLayoutMetrics (cdp-page-scanner.ts:453), Page.captureScreenshot
(browser-action-artifacts.ts:65,75), Page.enable (cdp-page-scanner.ts:188,
cdp-helper.ts:441), Page.setInterceptFileChooserDialog, Page.stopLoading
(browser-snapshot.ts:650), Page.fileChooserOpened (event, browser-action-artifacts.ts:135),
Page.downloadWillBegin / Page.downloadProgress (events, browser-action-artifacts.ts:135,163).
DOM — DOM.getDocument (cdp-page-scanner.ts:101,215; browser-action-targets.ts:173),
DOM.querySelector (:179), DOM.querySelectorAll (cdp-page-scanner.ts:118), DOM.describeNode
(:127; browser-action-targets.ts:186), DOM.resolveNode (browser-actions.ts:1043),
DOM.getBoxModel (cdp-page-scanner.ts:465; browser-action-targets.ts:276),
DOM.getContentQuads (cdp-page-scanner.ts:603), DOM.getFrameOwner (:451),
DOM.getAttributes, DOM.getOuterHTML, DOM.pushNodesByBackendIdsToFrontend, DOM.enable
(:167; cdp-helper.ts:490), DOM.scrollIntoViewIfNeeded (browser-action-targets.ts:73),
DOM.focus (browser-actions.ts:826,1117; browser-extended-actions.ts:333),
DOM.setFileInputFiles (browser-extended-actions.ts:314), DOM.setAttributeValue;
DOM mutation events consumed at cdp-helper.ts:528-532 (DOM.childNodeInserted,
DOM.childNodeRemoved, DOM.attributeModified, DOM.attributeRemoved,
DOM.characterDataModified, DOM.documentUpdated).
Runtime — Runtime.evaluate (cdp-page-scanner.ts:638; browser-snapshot.ts:594),
Runtime.callFunctionOn (browser-actions.ts:1052; browser-extended-actions.ts:382),
Runtime.getProperties (cdp-page-scanner.ts:756), Runtime.releaseObject
(browser-actions.ts:1091; browser-extended-actions.ts:394; cdp-helper.ts:1009),
Runtime.enable (cdp-helper.ts:380,513); events Runtime.executionContextCreated/Destroyed/ Cleared, Runtime.exceptionThrown, Runtime.exceptionRevoked, Runtime.consoleAPICalled.
Input — Input.dispatchMouseEvent, Input.dispatchKeyEvent, Input.insertText,
Input.dispatchDragEvent, Input.setInterceptDrags, and the Input.dragIntercepted event.
Accessibility — Accessibility.enable (cdp-page-scanner.ts:181),
Accessibility.getFullAXTree (:227).
DOMSnapshot — DOMSnapshot.enable (cdp-page-scanner.ts:173), DOMSnapshot.captureSnapshot
(cdp-page-scanner.ts:206).
Network — Network.enable (cdp-helper.ts:380,499); events Network.requestWillBeSent,
Network.responseReceived, Network.loadingFinished, Network.loadingFailed (used by the console
collector, cdp-helper.ts:499-513). No Network domain command is issued for interception —
Fetch.* is never used. Verified: grep -rn "Fetch\." packages/browser-core/src → no hits.
Target — browser-core consumes Target.detachedFromTarget (browser-core.ts:190) but issues
no Target.* command; target attach/flattening is the provider's job. Verified: grep -rn "attachToTarget\|setAutoAttach" packages/browser-core/src → no hits.
Browser — Browser.downloadWillBegin / Browser.downloadProgress as events only
(browser-action-artifacts.ts:163).
Summary: browser-core's ordinary Browser Use path needs Page, DOM, Runtime, Input, Accessibility,
DOMSnapshot, plus Network/Target events. It does not issue Fetch.*, Network.setRequest* , Storage.*, Emulation.*, CSS.*, Security.*, Performance.*, ServiceWorker.* or any
Target.* command. That list is the restricted set a Full CDP gate must widen.
3. Element references, staleness, and "target generation"
3.1 Refs are opaque, snapshot-scoped, and generation-fenced
ElementRecord (browser-state.ts:37-58) carries ref, nodeId, backendNodeId, tag, role,
type, selector?, text, rect, coordinateSpace, isInteractable?, frameId?,
commandSessionId?, frameOffset?, parentRegionPath?, virtualEditor?, attributes.
Snapshot (browser-state.ts:60-87) carries id, generation, createdAt,
lastAccessedAt?, continuationKind?, elements, semanticSource?, semanticTree, page,
durationMs?, nextOffset, continuationInvalidated?, frameRefs?, editableQuery?.
A "target generation" concept already exists, but it is page/session-scoped, not tab-scoped:
BrowserSessionState.generation: number(browser-state.ts:106), initialised to0
(browser-core.ts:145), exposed byget generation()(browser-core.ts:200).- Invalidation is one method,
BrowserActionTargetSupport.invalidate
(browser-action-targets.ts:332-338):protected invalidate(session: SessionState): void { session.generation += 1; session.frameEpochs?.clear(); session.elementMap.clearOpaqueRefs(); clearBrowserSnapshots(session); session.keyboardContinuationTarget = undefined; }
- What makes a ref stale:
getBrowserSnapshotreturnsundefinedwhen
snapshot.generation !== session.generationorcontinuationInvalidated === true
(browser-state.ts:176-181);resolveBrowserSnapshotElementskips snapshots whose generation
differs (browser-state.ts:193); pruning uses the same test plus a 5-minute TTL
(browser-state.ts:145,:306), retaining at mostMAX_RETAINED_BROWSER_SNAPSHOTS = 3
(browser-state.ts:144). - Ref authority:
BrowserCoreSession.resolveOpaqueRef(browser-core.ts:210-219) consults the
snapshot store first; theElementMapManageropaque-ref map is a fixture-only fallback,
disabled oncesnapshotAuthorityEstablished === true.
3.2 What triggers invalidation today
BrowserCoreSession's event listener (browser-core.ts:162-193):
| Event | Scope | Effect |
|---|---|---|
Page.frameNavigated |
child frame | invalidateBrowserFrame |
Page.frameNavigated |
main | full invalidate |
Page.navigatedWithinDocument |
child / main | same split |
Page.frameDetached |
main frame id | full invalidate; else frame-scoped |
Target.detachedFromTarget |
by params.sessionId |
frame-scoped |
Plus post-interruption invalidate (browser-core.ts:352) and host-driven
synchronizeGeneration() (:251).
3.3 Frame-scoped staleness (the OOPIF case)
BrowserFrameIdentity = { frameId?, commandSessionId? } (browser-state.ts:139-142).
invalidateBrowserFrame(session, identity) (browser-state.ts:224-254) bumps per-frame epochs in
session.frameEpochs, removes matching elements from every retained snapshot, sets
continuationInvalidated = true, deletes the frame from frameRefs, filters editableQuery, and
clears a matching keyboardContinuationTarget.
Gap for the feature: there is no notion of tab/browser-target generation. generation is a
per-BrowserCoreSession page counter, and a session is per page. Selecting a different tab, closing
a tab, or reconnecting a provider produces a different session object, not a bumped generation on
one. A WebUI-owned target-generation layer (WP1) is therefore must be created, and it must map
its own target generation onto BrowserCoreSession instances.
3.4 The legacy ElementMap cache (not the production ref path)
ElementMapManager (element-map-manager.ts:113) is an LRU cache keyed by host + pathname
(getUrlPath, :164) with maxCacheSize default 5 (:142), optionally persisted to
element-map-<domain>-<md5>.json (:178-186). It is not how refs are validated in production —
§3.1 applies. update() (:194) writes the entry; getElementById/getElementBySelector
(:322, :330) look up by index/selector. Do not treat this cache as the staleness mechanism.
4. browser-core package entry point
- Package:
@mavis/browser-core, private, ESM. Single subpath"."only
(packages/browser-core/package.json),types: "./src/index.ts", runtime./dist/index.js.
There is no./contractsor./transportsubpath; an implementer imports the root. packages/browser-core/src/index.tsre-exports the provider-neutral surface:
BrowserTransport,BrowserAttachedFrame,BrowserPageLifecycle,CdpTransport,
CoreBrowserEvaluateOptions,CoreBrowserTransportEvent(EventListener)(index.ts:72-81);
BrowserPageTransport+ non-prefixedBrowserEvaluateOptions(:1-6);BrowserTransportCommandParams
(:71); contractsBrowserDOMNode/BrowserAXNode/BrowserSnapshotNode/BrowserInteractiveElement/ BrowserWaitInput/BrowserWaitState(:20-31);BrowserCore,BrowserCoreSession,
BrowserCoreResolvedTargetviaexport * from './browser-core.js'(:36);ElementMapManager
(:47);cdp,evaluate,ElementRecord,BrowserSessionState,PageSummary,Snapshot
(:49-70);BROWSER_TIMEOUTS, interruption helpers (:48).- Note the duplicated aliases:
BrowserEvaluateOptionsandBrowserTransportEventare exported
both prefixed and unprefixed (index.ts:1-6vs:72-81). Import the unprefixed
CoreBrowser*-free pair or the prefixed pair consistently — not verified which the WebUI gate
will require; needs check. BrowserCoreSessionOptions(browser-core.ts:43-58) is what a provider passes to
createSession(transport, options)(:108-113):clipboardProvider?,cdpHelper?,
actionTimeoutMs?,commandTimeoutMs?,inputCoordinateScaleProvider?,
beforePointerAction?,clickNavigationDetectionMs?,captureScreenshot?.
The action allow-list is per-provider:BrowserCoreOptions.actionsasserted byassertAction
(:92-96).
5. WebUI client conventions for a new business
5.1 The five roles and what each may do
From docs/client/ownership.md ("Businesses own resources; roles own kinds of code") and
docs/architecture/module-map.md:
| Role | Owns | Must not |
|---|---|---|
contracts/ |
published surface: capability port slice, view models, commands | import implementation |
application/ |
workflows, state owners, query coordination | touch browser globals, DOM, storage, concrete transport |
projection/ |
pure derivations wire/owned-state → render shape | import React, perform IO |
bindings/ |
React subscriptions, context, navigation, DOM effects | acquire write authority, construct owners |
ui/ |
presentation | call IO ports, instantiate owners, reach a generic store writer |
Not every business populates every role; empty directories are not created
(module-map.md, "Client — businesses"). A business with no screen has no ui/.
projection/ classification (boundaries.md): a pure derivation is domain if its basename is
in PROJECTION_DOMAIN in scripts/lib/webui-dependency-rules.mjs (a business rule), otherwise
view (reads/shapes only). Adding a projection that carries a rule means adding its basename to
that list.
5.2 Worked example — the goal business (small; populates all five roles)
packages/webui/src/client/features/goal/ — exact directory list:
goal/contracts/goal-port.ts
goal/projection/goal-state.ts
goal/application/goal-coordinator.ts
goal/application/goal-session-owners.ts
goal/application/goal-submission.ts
goal/bindings/goal-owner.public.ts
goal/ui/GoalBanner.public.tsx
goal/ui/goal.css
Naming conventions observed:
- contracts —
<business>-port.ts. OneWebui<Business>Portwith every method optional
(goal-port.ts:16-28; the header says "undefinedmeans 'the operation is not wired'") plus a
narrowWebui<Business>Commandsslice of the methods presentation may call
(goal-port.ts:11-15). Requests/responses come fromshared/contracts/<domain>.js. - application —
<business>-coordinator.tsholds policy;<business>-session-owners.tsholds
whose policy it is and when it exists.goal-coordinator.ts:22defines
WebuiGoalPortSlice = Pick<GoalPort, …>;WebuiGoalSink(:34) is the single injected store
write. Both files state "Framework-free: no React, no DOM." - projection —
<business>-state.ts, pure, no React. - bindings — hooks,
.public.ts. - ui —
Webui<Thing>.public.tsx+<business>.css.
5.3 Publishing a contract to other modules (the .public.ts marker)
Rule (boundaries.md): "A feature may use another feature's public contracts/, or a module that
feature published as its surface: a .public.ts for a derivation or interface, a .public.tsx for a
component. Everything else in another feature is private, and there is no exception list."
A published module is constrained: "every explicit export is public, no export-star, no foreign
re-export, and no foreign namespace, default or dynamic access" (boundaries.md). Enforced by
classifyRole / ROLE_FORBIDDEN_TARGETS in scripts/lib/webui-dependency-rules.mjs via
pnpm check:webui-dependency, with baseline scripts/lib/webui-dependency-baseline.json.
Concrete example — goal/bindings/goal-owner.public.ts publishes
webuiGoalRecordKey(sessionId), useWebuiGoalOwner(sessionId): WebuiGoalCoordinator and
useWebuiGoalReaction(sessionId, listener). It reads the owner from the application
(useWebuiApplicationOptional() / application?.goals) and falls back to a per-mount registry so a
composer mounted without an application still works. A settings or workspace surface that needs the
goal owner imports only this .public.ts module.
Other examples: review/application/workspace-review-owner.public.ts,
models/bindings/use-provider-editor.public.ts, terminal/projection/terminal-size.public.ts,
workspace/projection/workspace-name.public.ts, settings/ui/SettingsModal.public.tsx.
5.4 Composition root and lifetime (ADR 0016/0017/0018 rules, per docs/client/ownership.md)
- The composition root is
client/application/create-application.ts. It constructs dormant
owners, then start installs routes, hydrates persisted state and starts ingress. It is the only
place that knows which capability is wired. dispose()is idempotent; a disposed instance cannot restart; a fresh lifecycle gets fresh
identity epochs.- Exactly one process-event ingress per application instance: the single long-lived
watchEventssubscription (client/platform/event-channel.ts). - Temporal ownership rules: interest leases; generation fences (one live stream lease per session);
incarnation + freshness (client/application/identity-epochs.ts,
client/conversation/request-freshness.ts); keyed resources vs selected slots. WebuiTransport(client/contracts/transport.ts:31-45) extends the thirteen business ports; it
"declares no method of its own" and every method is optional.client/bindings/(application-context.tsx,use-query-state.tsx,browser-capabilities.tsx,
browser-effects.ts,navigation.ts,port-slices-context.tsx, …) is the only React edge
to application owners.use-query-state.tsx:1-10states components "read its snapshot and submit
query commands, and they never receive a transport method or a store writer".
5.5 The tui ban — nuance
boundaries.md says "Never import packages/tui (ADR 0003). Read it as a specification, compose the
equivalent here." It is not implemented as a literal packages/tui denylist entry in
webui-dependency-rules.mjs; the load-bearing facts are structural:
packages/webui/package.json declares no dependency on a tui package, and
scripts/lib/webui-dependency-rules.mjs has no allow-list of tui subpaths, so an import would
simply be an unresolvable workspace dependency. packages/browser-core is the sanctioned read-only
reference (it is a separate workspace package, in release/extraction.json packageRoots).
Not verified — needs check: whether check:webui-dependency would emit a specific message for a
tui import or merely fail to resolve it.
6. Settings controls
6.1 Where they live
- Settings business:
src/client/features/settings/—contracts/settings-port.ts,
contracts/memory-handoff.ts,projection/{settings-view,usage-report,usage-plan}.ts,
application/settings-workflows.ts,ui/*(17 files), plusui/SettingsModal.public.tsx. - Skeleton primitives are shared, business-free:
client/ui/settings-layout.tsxexports
SettingsPage(:35),SettingsSection(:47),SettingsCard(:86),SettingsRow(:101),
SettingsDivider(:133),InfoHint(:153).client/ui/ToggleSwitch.tsx:14exports the shared
switch. - Tabs and their enabled/disabled state come from one derivation:
features/settings/projection/settings-view.ts—SettingsTabKey(:11),
DESKTOP_SETTINGS_TABS(:15),resolveThemePreference(:20),filterSettingsTabs(:22),
SETTINGS_GROUPS(:24).ownership.mdnotesvoice,shortcutsandconnectionare declared
but disabled.
6.2 How a control is written
ToggleSwitch props (client/ui/ToggleSwitch.tsx:3-12): Omit<ButtonHTMLAttributes<HTMLButtonElement>, "aria-checked" | "aria-label" | "onChange" | "role" | "type"> & { checked, label, onChange?: (checked: boolean) => void, testId?, … }. It renders a <button role="switch" aria-checked={checked}> and calls onChange?.(!checked) (:22-36). Consumers: :5,
client/features/settings/ui/UsageModelSettings.tsx:5, MemorySection.tsx:10,
models/ui/ProviderFormPanel.public.tsx:45, settings/ui/SettingsModal.public.tsx:29,
plugins/ui/PluginManagementList.tsx:5.
6.3 Concrete toggle that calls a business owner instead of owning state
features/settings/ui/UsageModelSettings.tsx:231 — the credit-fallback toggle:
<ToggleSwitch
checked={creditFallback.enabled === true}
label="M Plan 达限后使用积分"
disabled={creditFallback.saving}
onChange={(next) => { void account?.setCreditFallback(next); }}
testId="settings-usage-credit-fallback-toggle"
className="shrink-0 disabled:opacity-60" />The chain: component → account owner command → features/account/application/account-workflows.ts:333
(readonly setCreditFallback: (enabled: boolean) => Promise<void>; implementation :770) →
port method features/account/contracts/account-port.ts:32 →
client/platform/transport.ts:254 (request(SET_CREDIT_FALLBACK_OPERATION_NAME, body)) →
client/application/port-slices.ts:203.
The settings component owns only the busy/error presentation; enabled/saving come from the
owner's state. A browser authorization toggle must follow the same shape: render the switch from a
browser business's published projection/commands, never write persistence itself.
6.4 Persistence
application/settings-workflows.ts:29 states "This module is framework-free — no React, no DOM, no
storage". Storage IO lives in client/platform/storage.ts behind WebuiBrowserStorage
(:16-40), with key constants such as WEBUI_COMPOSER_STATE_KEY = "webui.composer.state.v1"
(:15), NO_PROJECT_STORAGE_KEY, TEAM_MODE_STORAGE_KEY. Server-persisted settings go through
operations instead (e.g. getMemorySettings / setMemorySettings).
7. Workspace panels
features/workspace/ui/WorkspacePanels.tsx is the panel surface. Composition is ordinary React
composition plus a tab dispatch:
- Tab kinds are a closed union:
WebuiWorkspaceTabShapes→WebuiWorkspaceTabKind(keyof)
→WorkspacePanelTab = { [K in Kind]: { id, kind: K } & Shapes[K] }[Kind]
(projection/workspace-panel-state.ts:64-70), with labels and add-menu entries in
WEBUI_WORKSPACE_TAB_KINDS(:88-93):files,file-preview(addMenu: null— opened by a path
from the tree, never by the menu),review, … - Commands are the panel's contract:
WorkspacePanelCommandin
features/workspace/contracts/panel-command.ts(re-exported atworkspace-panel-state.ts:75). WebuiFilePreview(WorkspacePanels.tsx:304-334) dispatches before consulting the read
result: archive first, then HTML, then loading/error, then media, then binary, then markdown/code.
htmlis rendered at:315-316:const html = !archive && isWorkspaceHtmlPath(tab.path) && workspaceFileUrl ? <WorkspaceHtmlPreview path={tab.path} fileUrl={workspaceFileUrl({…})} content={result?.content} /> : undefined;
- Panel-level controls:
WebuiWorkspacePanelControls(:283),WebuiProgressOverviewPanel(:290),
WebuiWorkspacePanel(:333) takingstate,dispatch,workspaceFileUrland the Terminal
business's published owner. ownership.md/ the panel file record that panel re-exports (WorkspacePanels.tsx:72-77) exist
because panel tests import them — "the panel's public surface".
7.1 What the HTML preview actually is (confirmed: not a control surface)
features/workspace/ui/WorkspaceHtmlPreview.tsx:1-30:
- It renders
<iframe src={fileUrl}>pointing at the/workspace-fileroute. "That is the whole
isolation story." - The route answers
text/htmlwithContent-Security-Policy: sandbox allow-scripts→ opaque
origin: scripts run, but the document cannot readwindow.__WEBUI_CONFIG__.token, touch the
panel's DOM, or use the WebUI origin for storage. - "No
sandboxattribute here" — one layer, in the server, so the boundary stays auditable. - "Never
srcdoc, never a blob URL, neverdangerouslySetInnerHTML." projection/html-preview.tsonly reports relative references
(hasRelativeDocumentReference), because relative URLs 404 under the opaque origin.
It is a read-only local-file preview with no input forwarding, no target selection and no CDP
surface. Repurposing it is ruled out (also stated in the plan §3).
8. Transport bindings for server operations
8.1 Operation names — the single declaration point
packages/webui/src/shared/operation-names.ts. Shape, verbatim:
export const VERSION_OPERATION_NAME = "version" as const; // :12
export const LIST_SESSIONS_OPERATION_NAME = "listSessions" as const; // :13
export const GET_GOAL_OPERATION_NAME = "getGoal" as const; // :29
// …
export interface OperationSpec { // :255
version: { request: undefined; response: WebuiVersionInfo };
listSessions: { request: WebuiSessionListRequest; response: WebuiSessionPage };
isGoalEnabled: { request: undefined; response: WebuiGoalEnabledResult };
getGoal: { request: { readonly sessionId: string }; response: WebuiGoal | undefined };
patchGoal: { request: WebuiGoalPatchRequest; response: WebuiGoal };
// …
}
export const WEBUI_STREAM_OPERATION_NAMES = [ // :623
SEND_MESSAGE_OPERATION_NAME, RESUME_SESSION_OPERATION_NAME,
WATCH_EVENTS_OPERATION_NAME, WATCH_TERMINAL_OPERATION_NAME,
] as const;
export type WebuiStreamOperationName = (typeof WEBUI_STREAM_OPERATION_NAMES)[number]; // :645
export type WebuiUnaryOperationName = Exclude<keyof OperationSpec, WebuiStreamOperationName>; // :652Header comment (:1-10): OperationSpec "is the single declaration point for what an operation
carries in and what it answers with"; server/operation/bind-handlers.ts is checked against it.
The file "lives in shared/ and therefore imports only shared contracts, never the runtime port".
Naming: lowerCamelCase wire name; SCREAMING_SNAKE constant prefixed by the verb (GET_,
LIST_, SET_, PATCH_, CREATE_, WATCH_, RESUME_, SEND_, MUTATE_, …). A browser
operation follows the same shape: constant + OperationSpec entry, no string literals anywhere
else.
8.2 The client binding pattern a browser operation would follow
client/platform/transport.ts is "the domain method table: every capability the browser client can
ask the host for, joined to the wire" (:1-10). Mechanics live in ./wire.js.
- Import the
*_OPERATION_NAMEconstant (never a literal) — see the import block at:21-95. - Add one method to the object:
patchGoal: (body) => request(PATCH_GOAL_OPERATION_NAME, body)
(:187),isGoalEnabled: () => request(IS_GOAL_ENABLED_OPERATION_NAME, undefined)(:184),
listTerminals: () => request(LIST_TERMINALS_OPERATION_NAME, {})(:221). - Streams differ:
watchTerminal: (body, onFrame) => watchTerminal(WATCH_TERMINAL_OPERATION_NAME, body, onFrame)(:226);
sendMessage/resumeSessionusestream(...)(:228,:231). - Register the operation name in the business port (
client/contracts/transport.tsjoins ports),
client/application/port-slices.ts(:203lists capability keys), the server descriptor
(server/operation/<domain>.ts) andWEBUI_OPERATION_BINDINGS.
9. Exists today vs. must be created
Exists today
BrowserTransportprovider contract +CDPHelper/BrowserCoreSessionaction path
(browser-transport.ts,cdp-helper.ts,browser-core.ts).- The restricted CDP command set (§2).
- Page-scoped generation + snapshot-scoped opaque refs + per-frame epochs (§3).
- Structured interruption codes and reset/reconnect seams (§1.4, §1.5).
- A provider adoption seam in WebUI runtime (
runtime/browser-provider.ts). - The five-role business layout and
.public.tspublication mechanism (§5). - Settings skeleton +
ToggleSwitch+ the owner-command toggle pattern (§6). - Workspace tab dispatch and the iframe HTML preview (§7).
- The operation-name /
OperationSpecdeclaration point (§8).
Must be created
- WebUI shared DTOs + validators for browser config, per-browser authorization, pairing/connection
status, tab inventory, active target, input, image stream, stable failures. - The
features/browser/client business (contracts / application / projection / bindings / ui) and
its owner, fenced by a tab-level target generation (§3.3 gap). - Server operations +
bind-handlersentries + runtime routing for browser ops; the Full CDP gate
in the runtime/provider path. - The extension package (
packages/browser-extension/), its packaging and its verify gate. - Any WebUI → browser-core import path (§10 caveat).
10. Open risks for the implementer
@mavis/browser-coreis not a WebUI dependency today.
packages/webui/package.jsondependencies contain no browser-core entry, and browser-core's only
consumer ispackages/tui(packages/tui/src/runtime/browser/{headless-chrome-transport, headless-chrome-provider, headless-cdp-connection}.ts). Adding the dependency is a real change
with dependency-rule consequences. Not verified — needs check: whether
pnpm check:webui-dependency/check:webui-boundarypermit a new workspace dependency, and
whether browser-core'sprivate: true+dist/index.jsresolution is viable inside the WebUI
build (the WebUI bundle consumes@mavis/local-runtime-v2and@mavis/protocoltoday).- client bundle must stay browser-side.
BrowserTransportimplementations own a process; only
contracts may cross intoclient/. - Target generation is a new concept. browser-core's
generationis per page session, not per
browser target; do not assume it can carry tab selection. browserActionArtifactsdownload observation depends onPage.download*/
Browser.download*events (browser-action-artifacts.ts:135,163) — an extensionchrome.debugger
provider may not surface them; WP0's capability matrix must answer this.- Settings are split: client-side (
client/platform/storage.ts,localStoragekeys) versus
server-persisted operations. Browser authorization is stateful and cross-device-relevant; decide
which side owns persistence before writing the toggle. Full CDPandrestrictedsets: §2 is the current restricted set as of this commit. Any new
browser-core CDP call silently widens it — the gate should be derived from the provider's declared
capabilities, not a hard-coded list that drifts.
Activity
Fectivnfy112357 commentedon Oct 21, 2026
The WebUI Browser Use programme is delivered and the board is closed, so this note no longer needs
its own thread.
This issue was written as reference material for that programme — the seams, contracts and
repository plumbing the tickets were written against. It was never a work item, and nothing in it
was left as a task. What it pointed at is now either in the tree or in the published evidence:
packages/webui/src/server/browser-bridge/**andpackages/webui/src/runtime/browser/**— described in02-bridge-protocol.md,03-extension-admission.mdand06-runtime-authorization-trace.mdpackages/webui/src/shared/contracts/browser.tsandbrowser-bridge.ts—01-capability-matrix.mdpackages/browser-extension/**,release/extraction.json,release/public-source.jsonandtest/vitest-suites.json— the store builds and their disclosures inSTORE-DISCLOSURES.mdThe decision record is
ADR 0019
and remains the only Browser Use decision record. The implementation is
#195, and the acceptance matrix with the
browser version, the operating system, the enabled settings and the observed result is
09-acceptance-record.md.Closing this issue means the note needs no further attention — not that it was ever a ticket.