…imed pointer paths (#8291)
* fix(browser): keep the agent tab composited when no panel shows it
A tab view that is in no window has no compositor surface, so CDP mouse input on it never acknowledges and screenshots never capture. The agent's tab was left that way whenever the user viewed another tab or had not opened the browser panel: clicks failed or stalled five seconds and screenshots failed. Each chat's agent tab is now parked invisibly in the main window, adopted in place when the panel shows it, and released on every tab teardown path.
* improvement(browser): name the controls a refused click can use next
A click refused because an overlay covers the target now lists the overlay's own snapshot controls, so the agent dismisses it and retries the same id without another snapshot. A click that lands on a nested control names that control's id. Only real overlays (dialogs, modals, fixed or sticky layers) list controls.
* feat(browser): route drags and coordinate hovers along timed pointer paths
browser_drag takes optional via points and a durationMs, and browser_hover can target a viewport point with the same route, moving with no button pressed. Pages that react to pointer speed or position (sliders, physics, canvas hover, cursor-following games) can now be driven the way Claude computer use and OpenAI's CUA allow. A plain drag keeps its exact previous pace; batches refuse timed paths so they cannot outlast the batch watchdog.
* fix(browser): honor cancellation on pointer routes and keep agent tabs tracked across scopes
- Timed drags and hover routes stop as soon as their tool call is cancelled;
a drag still releases the button through its existing cleanup.
- A coordinate hover refuses durationMs without via points instead of
silently moving straight to the target.
- A scope migration carries the chat's agent tab registration with it, so the
migrated chat's old agent tab is released when it switches tabs.
- Overlay controls are found through shadow roots and with any tag casing.
* fix(browser): cancel drags during their settle and keep parked agent tabs in the main window
- An already-aborted drag sends no input, and an abort during the settle hold
takes the cancellation cleanup instead of dropping.
- A coordinate hover re-checks the active tab right before it moves.
- Parking an agent tab hands keyboard focus back to whichever of the Sim
renderer or the visible page held it.
- An agent tab detached from a secondary window is parked in the main window,
which outlives it.
- A framed target covered by a parent-page overlay lists that overlay's controls.
* fix(browser): refuse a coordinate hover across a navigation
The coordinate hover captures the navigation epoch before its page probes and rejects a cross-document navigation before moving the pointer, as browser_click_at and browser_drag do. The overlay helper skips the empty registry slots earlier snapshots leave.
waleedlatif1
commented
waleedlatif1
changed the title
github-actions
Bot
added
the
requires-mothership-merge
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
