Repository navigation
Conversation
🦋 Changeset detectedLatest commit: 791816d The changes in this PR will be included in the next version bump. This PR includes changesets to release 5 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
005907b to
3440fdc
Compare
scuffi
changed the title
3440fdc to
2ed7898
Compare
commit: |
An exec that waits for a person in ctx.interrupt() can take hours. Whoever started it often can't wait that long: an agent wants to tell the user what's happening and carry on, and a deploy can end the request that was waiting. Until now nothing told the app when such an exec finally exited. CodemodeBackend now takes an onDetachedExit function, and the runtime gains detachExec(id). Detaching says that nothing waits on the exec any more. The exec keeps going, and when it exits, onDetachedExit hears its status, exit code, result, metadata and the last 16 KiB of each output stream. detachExec() checks that the exec is still running and marks it in one transaction, and returns false if it had already exited, so every exec's outcome goes either to whoever reads its result or to onDetachedExit, never both and never neither. The hook sits on the backend, beside onWait, because detaching is a capability of the backend: a backend without it can't detach, and runtime.backends() reports which can. CodemodeBackend saves a detached exec's exit in the same transaction that records the exit, calls onDetachedExit, and deletes the saved exit once the hook returns. An exit whose hook threw, or whose object restarted first, is delivered again when the backend next connects or from runtime.recover(), so delivery is at least once, keyed by the exec's id. Execs that were never detached save nothing. A run's metadata is now saved with every run, not only with runs that can resume, so an onInterrupt "fail" run that a restart cut off still reports whose it was. Detaching is experimental, like waiting for an answer.
The exec tool returned only when its run ended. A run waiting for an
approval held the agent's turn for as long as the approval took, so the
agent couldn't tell anyone what it was doing, and aborting the turn
killed the run.
createExecTool, and the exec option of every tool set, take detach. With
detach: true, the tool checks runtime.pending() for its own run's
waiting calls and, once one appears, detaches the run through
runtime.detachExec() and returns. With detach: { afterMs }, it also
detaches a run that has gone on that long. The reply carries the run's
executionId, a status of "waiting" or "running", the output so far, the
waiting calls and a note telling the model that the outcome arrives
later. The run's exit then goes to its backend's onDetachedExit, and an
abort of the model turn leaves the run alone.
A run that ends before it detaches returns as before. If the run ends
while its detach is going through, the tool waits to learn whether the
detach took: if it did, the tool replies as detached, since the exit
goes to onDetachedExit. Runs on backends that can't detach stay
attached, and a tool with no backend that can detach refuses detach when
it is built. Detaching is experimental.
A detached reply is a new shape of ExecToolOutput, with no exitCode. The
rlm example's output sanitizer read exitCode from anything that wasn't
an error, so it now treats any reply without one as a failed run. It
never detaches, so its behavior doesn't change.
2ed7898 to
791816d
Compare
scuffi
changed the title
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| if (detach !== undefined) { | ||
| if (detachExec === undefined || pending === undefined) { | ||
| throw new Error("createExecTool: detach needs a runtime with detachExec() and pending()"); | ||
| } | ||
| if (detachable.size === 0) { | ||
| throw new Error( | ||
| "createExecTool: detach needs a backend that can detach, such as a CodemodeBackend with onDetachedExit", | ||
| ); | ||
| } |
There was a problem hiding this comment.
🔴 Detaching unavailable through Workspace clients
With a getWorkspace() client, defineExec rejects detach because the client runtime lacks pending and detachExec. The client adapter exposes neither method, so agent tools cannot use detaching through this public client.
Learn more
A loaded getWorkspace() client wraps the underlying runtime through makeRuntimeClient. That wrapper only forwards exec, getExec, killExec, disposeExec, and backends. A detaching tool reads the backend list successfully, but rejects construction because the runtime wrapper has neither pending() nor detachExec(). The underlying WorkspaceRuntimeStub already implements detachExec(), but the wrapper drops it.
Example: Configure a CodemodeBackend with onDetachedExit, then create a tool with const ws = await getWorkspace(stub); createExecTool({ workspace: ws, detach: true }). Construction throws detach needs a runtime with detachExec() and pending() instead of offering the exec tool.
Recommended fix: Add pending() and detachExec() to WorkspaceRuntimeClient, UnderlyingRuntime, and makeRuntimeClient; forward them on both local and remote paths. The remote stub also needs pending() if it does not already expose it. Cover a loaded local and remote getWorkspace() client in tests.
Was this helpful? React with 👍 or 👎 to provide feedback.
| if (exit !== undefined) { | ||
| this.#host.db.run( | ||
| `INSERT OR REPLACE INTO workspace_runtime_detached_exits (backend, execution_id, data) | ||
| VALUES (?, ?, ?)`, | ||
| this.#backendId, | ||
| record.id, | ||
| JSON.stringify(exit), | ||
| ); |
There was a problem hiding this comment.
🔴 Reused IDs erase pending exit notifications
If onDetachedExit fails, disposing that exec and reusing its ID lets the next detached exit replace the saved one. disposeExec removes no pending exit, so the first outcome is lost.
Learn more
A detached exec's exit is kept in workspace_runtime_detached_exits until onDetachedExit succeeds. If the hook throws, deliverExits retains that row. disposeExec deletes its execution without deleting or reserving the pending exit's ID. A new exec can reuse that ID, and its terminal transaction replaces the previous exit in the same table.
Example: Run and detach id: "refund-42"; its callback throws. Dispose the finished exec, then launch and detach another refund-42. When the second exits, the saved outcome of the first is overwritten and cannot be delivered on recover().
Recommended fix: Prevent reusing an ID while an undelivered exit exists, or give pending exits a durable unique generation key rather than replacing rows keyed solely by execution ID. Test callback failure, disposal, ID reuse, and recovery.
Was this helpful? React with 👍 or 👎 to provide feedback.
| /** | ||
| * Called once a detached exec exits, however it exits, with how it | ||
| * ended. An exec detaches through `runtime.detachExec()`, or the exec | ||
| * tool's `detach`, when nothing should wait on it any more, such as | ||
| * one an agent left waiting for an approval. Nothing else hears how | ||
| * it ended, so this is where the outcome goes back to whoever started | ||
| * it. Without this, the backend's execs can't detach. | ||
| * | ||
| * The backend saves the exit in the same transaction that records | ||
| * it, and deletes it once this returns. If this throws, or the object | ||
| * restarts first, it's called again when the backend next connects or | ||
| * from `runtime.recover()`, so it may hear about an exec more than | ||
| * once: key on its `id`. Experimental: it may change in any release. | ||
| */ | ||
| onDetachedExit?: (exec: WorkspaceRuntimeExecExit) => void | Promise<void>; |
There was a problem hiding this comment.
🔍 Description uses an obsolete callback API
The PR description configures Workspace with detached: { onExit }, while the implementation accepts onDetachedExit on CodemodeBackend. The usage example needs updating if the implementation is final.
Was this helpful? React with 👍 or 👎 to provide feedback.
Codemode execs can already wait for a person: a host module calls
ctx.interrupt(), the exec pauses durably, and someone answers withruntime.respond(), even after a deploy. But whoever started the exec still had to wait on it for the result. Theexectool returns only when its exec ends, so an agent that starts a refund needing approval is stuck inside its tool call until a manager answers. It can't tell the customer what's happening, and if the turn is aborted, the exec is killed with it.This lets an exec detach: the caller stops waiting on it, the exec keeps going, and when it exits its backend hands the outcome to a function you give it. The
exectool can detach on its own once its exec waits for an answer.Usage
There are two halves. The backend says where detached execs report back, with
onDetachedExitnext to its existingonWait. Detaching is a capability of the backend: onlyCodemodeBackendhas it, and only when it has somewhere to send exits. The backend is built when the Durable Object starts, so the hook is there to receive an exit even straight after a deploy. Whoever starts an exec decides whether to detach it.detachworks the same throughcreateAITools, the TanStack and pi tool sets, anddefineExec. Without it, the tool behaves as before.What the model sees
A tool call whose exec detaches returns straight away:
{ "command": "export default async function () { … }", "cwd": null, "backend": "codemode", "executionId": "ex_123", "status": "waiting", "stdout": "Found order ORD-1042\n", "stderr": "", "waiting": [{ "callId": "ex_123:2", "payload": { "kind": "refund", "amountCents": 12900 } }], "note": "The run is still going. It is waiting for an answer to the calls in `waiting`, and carries on by itself once they are answered. Don't run it again: you'll get a message with its outcome when it ends." }The tool description says the same up front, so the model tells the user what's happening instead of running the code again. Aborting the turn after that no longer kills the exec. An exec that ends before it detaches returns its result exactly as before.
The loop
sequenceDiagram participant Model participant exec as exec tool participant Exec participant Backend participant App Model->>exec: issue refund exec->>Exec: start Exec-->>exec: waits in ctx.interrupt() exec->>Backend: detachExec(id) exec-->>Model: { executionId, status: "waiting" } Note over Model: tells the customer, turn ends App->>Exec: respond(approved) Exec-->>Backend: exits Backend-->>App: onDetachedExit(exec) App->>Model: new turn with the outcomeThe app's part is small: save what
onDetachedExithands you, and turn it into a message for the agent. If the Durable Object might be evicted, callruntime.recover()from an alarm while execs are waiting. That resumes them after a deploy and delivers any exitonDetachedExitmissed. Replay already needs this forctx.interrupt().Guarantees
detachExec()checks that the exec is still running and marks it in one transaction, and returnsfalseif it has already exited. So an outcome reaches whoever reads the result oronDetachedExit, never both and never neither. The tool handles the edge case too: if the exec exits while a detach is going through, the tool waits to learn whether the detach took and replies to match.onDetachedExitdoesn't lose exits. The exit is saved in the same transaction that records it, and deleted onceonDetachedExitreturns. IfonDetachedExitthrows, or the object restarts first, it's called again when the backend next connects or fromruntime.recover(). Delivery is at least once, so key onexec.id.Other changes
metadatais now saved with the exec. It used to be saved only for execs that could resume, so anonInterrupt: "fail"exec that a restart cut off would have reported empty metadata.runtime.backends()reportsdetachablefor each backend.runtime.recover()now delivers missed exits forCodemodeBackend. It used to have nothing to do for this backend.Limits
CodemodeBackendis the only backend that can detach for now, and only withonDetachedExit. A tool withdetachleaves execs on other backends attached. Building a tool withdetachfails at build time if no backend can detach.runtime.detachExec()throws for an exec on a backend that can't.ctx.interrupt().For reviewers
runtime.pending()every 250 milliseconds while it streams output, and stops as soon as the exec ends or detaches. Awaitevent on the exec's stream would avoid polling, but it would change the publicWorkspaceRuntimeEventunion, which a lot of code switches on.getExec()can't be relied on.onDetachedExitanddetachExec(), from the tool part,exec: { detach }.ExecToolOutputtype, with noexitCode. Code that narrows it with"error" in outputand then readsexitCodestops typechecking, asexamples/rlmdid; that commit fixes it. Reusing the result shape withexitCode: nullwould avoid that, but a detached reply would then look like a "still running" update to code that checksexitCode === null, such as the TanStack tools.Trying it
The script-runner test "lets the exec tool detach from a run once it waits, and hears how it ended later" runs the whole loop against a real Workspace:
A follow-up pull request adds
examples/detached-approvals: two support agents side by side, one attached and one detached, both needing a manager to approve refunds.Tests
onDetachedExithears about:detachExec()on an exec that's running, one that has exited, and on a backend withoutonDetachedExit;detachpassing through the AI SDK and TanStack tool sets.Docs
docs/05_runtime_interface.mdgets a "Detaching" section coveringonDetachedExitanddetachExec().docs/09_tool_interface.mdgets a "Detaching" section for the tool.docs/17_isolate_javascript.mdpoints to both from the section on waiting.Follow-ups
noteis fixed text. Apps that word things differently may want to set it.