computer: Detach codemode execs from whoever started them, and hear how they exit

.changeset/codemode-detach.md

@@ -0,0 +1,5 @@
+---
+"@cloudflare/computer": patch
+---
+
+`CodemodeBackend` takes an experimental `onDetachedExit`, and `workspace.runtime.detachExec(id)` leaves an exec running with nothing waiting on it, so its exit goes to `onDetachedExit`, saved with the exit and delivered again after a restart or from `runtime.recover()`; see [Detaching](https://github.com/cloudflare/computer/blob/main/docs/05_runtime_interface.md#detaching).

.changeset/exec-detach.md

@@ -0,0 +1,5 @@
+---
+"@cloudflare/computer": patch
+---
+
+The `exec` tool takes an experimental `detach: true`, or `{ afterMs }`, that returns once a run waits for an answer, or has gone on that long, with its `executionId` and output so far, and leaves it running for its backend's `onDetachedExit`; see [Detaching](https://github.com/cloudflare/computer/blob/main/docs/09_tool_interface.md#detaching).

docs/05_runtime_interface.md

@@ -167,4 +167,35 @@ Module backends use host capability calls against the authoritative Workspace an
 
 `worker-shell` intentionally preserves one-call, buffered-result behavior in this release. It does not retain executions for later reattachment or disposal. `timeoutMs` and a concurrent `killExec()` for a caller-supplied execution ID cooperatively abort just-bash at statement boundaries; by the time an ordinary `exec()` promise returns, the command has already settled. Use the Container or JavaScript isolate when detached execution and retained lifecycle are required.
 
+## Detaching
+
+Whoever starts an exec usually waits on it for its result. Sometimes that's the wrong thing to do: an exec waiting for a person in [`ctx.interrupt()`](./17_isolate_javascript.md#waiting-for-an-answer) can take hours, and an agent waiting on it can't tell anyone what's happening. Detaching an exec says that nothing waits on it any more. The exec keeps going, and when it exits, its backend hands how it ended to `onDetachedExit`. Only `CodemodeBackend` can detach so far, and only when it has `onDetachedExit`. Detaching is experimental.
+
+```ts
+const workspace = new Workspace({
+  storage: ctx.storage,
+  backends: [
+    new CodemodeBackend({
+      loader: env.LOADER,
+      onInterrupt: "replay",
+      onDetachedExit: async (exec) => {
+        // exec: { id, backend, status, exitCode, result?, metadata, stdout, stderr, exitedAt }
+        await conversations.get(exec.metadata.conversation).execEnded(exec);
+      },
+    }),
+  ],
+});
+
+const exec = await workspace.runtime.exec(source, { metadata: { conversation: "c1" } });
+if (!(await workspace.runtime.detachExec(exec.id))) {
+  // It had already exited: read its result as usual.
+}
+```
+
+`detachExec()` returns `false`, and changes nothing, if the exec has already exited. It checks and marks the exec in one step, so every exec ends up either attached, with its result going to whoever reads it, or detached, with its exit going to `onDetachedExit`, never both and never neither. The [`exec` tool](./09_tool_interface.md#detaching) detaches this way when it's built with `detach`. `detachExec()` throws for an exec on a backend that can't detach, and `runtime.backends()` lists which backends can, as `detachable`.
+
+`status` is `"completed"`, `"failed"` or `"cancelled"`, and `stdout` and `stderr` hold the last 16 KiB of each. The backend saves this in the same transaction that ends the exec, and deletes it once `onDetachedExit` returns. If `onDetachedExit` throws, or the object restarts before it returns, it's called again the next time the backend connects or `runtime.recover()` runs. So `onDetachedExit` may hear about an exec more than once, and should treat its `id` as the key. A quick `onDetachedExit` that saves the exit to the app's own storage, and acts on it from there, keeps the window for a repeat small. Execs that were never detached save nothing.
+
+Nothing wakes a Durable Object after a deploy, so an app with detached execs should call `recover()` from an alarm while it has some it hasn't heard back from.
+
 See [16. Execution runtime architecture](./16_code_execution.md) and [17. Isolate JavaScript](./17_isolate_javascript.md).

docs/09_tool_interface.md

@@ -400,6 +400,24 @@ const tools = createAITools({
 
 A pi caller runs each call itself, so it can instead pass the metadata with the call: `execute(call, { metadata })`. Metadata passed with a call wins over `exec: { metadata }`. A caller of `defineExec` passes `metadata` or `toolCallId` on each `execute` the same way.
 
+### Detaching
+
+By default, `exec` returns when its run ends. A run that waits for a person in [`ctx.interrupt()`](./17_isolate_javascript.md#waiting-for-an-answer), such as for an approval, can take hours, and the agent's turn waits with it: the agent can't tell anyone what it's doing until the answer comes. `detach` lets the tool return early and leave the run going on its own. It's experimental, and needs a backend that can [detach](./05_runtime_interface.md#detaching), such as a `CodemodeBackend` with `onDetachedExit`, which is where the run's exit goes.
+
+```ts
+const tools = createAITools({
+  workspace,
+  exec: {
+    metadata: ({ toolCallId }) => ({ conversation: this.name, toolCall: toolCallId }),
+    detach: true, // or { afterMs: 60_000 } to also detach a run still going after a minute
+  },
+});
+```
+
+With `detach`, the tool returns once a call in the run waits for an answer, and, with `afterMs`, once the run has gone on that long. Either way it detaches the run through `runtime.detachExec()` and replies `{ executionId, status, stdout, stderr, note }`, with a `status` of `"waiting"` or `"running"` and, when waiting, the `waiting` calls' `callId`s and payloads. The tool description and the reply's `note` tell the model the run carries on and its outcome arrives later. Aborting the model turn kills a run that hasn't detached, but leaves one that has alone. A run that ends before it detaches returns as usual, and its exit doesn't go to `onDetachedExit`. Runs on a backend that can't detach, such as shell and container backends, stay attached. A tool with no backend that can detach refuses `detach` when it's built.
+
+The tool doesn't deliver the outcome itself. The backend's `onDetachedExit` hears it, and the app finds the conversation through the run's `metadata` and gives the agent a new message with it. `detach` needs a runtime with `pending()` and `detachExec()`, which a `Workspace` and `workspace.stub()` have.
+
 Wire this tool carefully: it executes arbitrary shell commands inside the configured backend. Treat its output as untrusted text when including it in later model input. Pass `exec: { backends: {} }` or `readonly: true` when command execution is not part of the agent's job, and list backends explicitly when the Workspace has one the model should not use directly.
 
 ## `publish`

docs/17_isolate_javascript.md

@@ -229,6 +229,10 @@ Once none of its calls are waiting, the run replays to the call and the function
 - `respond()` throws `ENOTWAITING` for a call that isn't waiting, such as one already answered, so answering twice changes nothing.
 - Every answer replays the run from the top.
 
+### Detaching a run that waits
+
+Whoever started a run can await its result, but a run that waits for a person can outlive them: an agent's tool call may want to move on, or a deploy may end the request that started it. Give `CodemodeBackend` an `onDetachedExit` and its runs can detach, so `onDetachedExit` hears how they end instead, even across deploys: see [Detaching](./05_runtime_interface.md#detaching). The `exec` tool detaches a run once it waits, when it's built with `detach`: see [Detaching](./09_tool_interface.md#detaching).
+
 ### Limits
 
 - A run gets three attempts, counted again each time it journals a call, and has to finish within its original timeout, counted from when it first started.

examples/rlm/worker/executor-tool.ts

@@ -42,7 +42,9 @@ export function createExecutorTool(
 
 export function sanitizeExecutorOutput(output: ExecToolOutput): ExecToolOutput {
   const base = { command: output.command, cwd: output.cwd, backend: output.backend };
-  if ("error" in output) return { ...base, error: "Generated execution failed." };
+  // The executor never detaches, so a reply without an exit code is a
+  // failure as far as the model is concerned.
+  if (!("exitCode" in output)) return { ...base, error: "Generated execution failed." };
 
   return {
     ...base,

packages/computer/README.md

@@ -32,6 +32,8 @@ removed in a future release; each one names its replacement.
 | `ContainerBackend` and `withWorkspaceContainer` | `@cloudflare/computer/backends/container` | Experimental |
 | `LegacyContainerBackend` and `withLegacyWorkspaceContainer` | `@cloudflare/computer/backends/container-legacy` | Deprecated: use `ContainerBackend` |
 | Resuming interrupted isolate JavaScript: `onInterrupt: "restart"` and `"replay"`, `workspace.runtime.recover()`, `unsafeToRepeat()`, and waiting for answers with `ctx.interrupt()`, `pending()` and `respond()` | `@cloudflare/computer`, `@cloudflare/computer/backends/codemode` | Experimental |
+| Detaching execs: `CodemodeBackend`'s `onDetachedExit` and `workspace.runtime.detachExec()` | `@cloudflare/computer`, `@cloudflare/computer/backends/codemode` | Experimental |
+| Detaching the `exec` tool from a run that waits, with `exec: { detach }` | `@cloudflare/computer/tools/*` | Experimental |
 | `WorkerShellBackend` and the shell command groups | `@cloudflare/computer/backends/worker-shell`, `@cloudflare/computer/shell/*` | Experimental |
 | The `codemode:container` host module | `@cloudflare/computer/modules/container` | Experimental |
 | The `codemode:browser` host module | `@cloudflare/computer/modules/browser` | Experimental |

packages/computer/src/backends/codemode/codemode.test.ts

@@ -4,7 +4,11 @@ import { describe, expect, it, vi } from "vitest";
 
 import { createGitModule } from "../../modules/git";
 import { createCodemodeModule, type WorkspaceModuleScopeEnd } from "../../runtime/codemode-module";
-import type { WorkspaceModule, WorkspaceRuntimeLoader } from "../../runtime/types";
+import type {
+  WorkspaceModule,
+  WorkspaceRuntimeExecExit,
+  WorkspaceRuntimeLoader,
+} from "../../runtime/types";
 import { Workspace } from "../../workspace";
 import { runtimeCode, CodemodeBackend } from "./codemode";
 import { ExecutionJournal, fingerprint, loadResume, type ResumeStart, saveResume } from "./journal";
@@ -233,6 +237,108 @@ describe("CodemodeBackend", () => {
     expect(workerDisposals).toBe(1);
   });
 
+  describe("detaching", () => {
+    // A loader whose runs return `value` once `release` is called, so a
+    // test can act while the run is still going.
+    function gatedLoader(value: unknown) {
+      let release!: () => void;
+      const gate = new Promise<void>((resolve) => (release = resolve));
+      const load = vi.fn<WorkspaceRuntimeLoader["load"]>(() => ({
+        getEntrypoint() {
+          return {
+            evaluate: async (
+              _input: unknown,
+              host: {
+                assertResult(value: unknown): Promise<void>;
+                attachOutput(readable: ReadableStream<Uint8Array>): Promise<void>;
+              },
+            ) => {
+              await gate;
+              await evaluateResult(host, value);
+            },
+          };
+        },
+      }));
+      return { load, release: () => release() };
+    }
+
+    async function detachedWorkspace(value: unknown, detachable = true) {
+      const storage = new SQLiteTestStorage();
+      const exits: WorkspaceRuntimeExecExit[] = [];
+      const { load, release } = gatedLoader(value);
+      const workspace = new Workspace({
+        storage,
+        backends: [
+          new CodemodeBackend({
+            loader: { load },
+            ...(detachable ? { onDetachedExit: (exec) => void exits.push(exec) } : {}),
+          }),
+        ],
+      });
+      await workspace.fs.mkdir("/workspace", { recursive: true });
+      const savedExits = () =>
+        new Database(storage).scalar<number>(
+          "SELECT COUNT(*) FROM workspace_runtime_detached_exits",
+        );
+      return { workspace, exits, release, savedExits };
+    }
+
+    it("hands onDetachedExit a detached exec's exit, and keeps nothing once it returns", async () => {
+      const { workspace, exits, release, savedExits } = await detachedWorkspace({ answer: 42 });
+
+      const handle = await workspace.runtime.exec("export default null", {
+        metadata: { who: "a" },
+      });
+      expect(await workspace.runtime.detachExec(handle.id)).toBe(true);
+      release();
+      await handle.result();
+      await vi.waitFor(() => expect(exits).toHaveLength(1));
+
+      expect(exits[0]).toMatchObject({
+        id: handle.id,
+        backend: "codemode",
+        status: "completed",
+        exitCode: 0,
+        result: { answer: 42 },
+        metadata: { who: "a" },
+      });
+      await vi.waitFor(() => expect(savedExits()).toBe(0));
+    });
+
+    it("keeps no exit for an exec nothing detached", async () => {
+      const { workspace, exits, release, savedExits } = await detachedWorkspace(null);
+
+      const handle = await workspace.runtime.exec("export default null");
+      release();
+      await handle.result();
+
+      expect(exits).toEqual([]);
+      expect(savedExits()).toBe(0);
+    });
+
+    it("won't detach an exec that has already exited", async () => {
+      const { workspace, exits, release } = await detachedWorkspace(null);
+
+      const handle = await workspace.runtime.exec("export default null");
+      release();
+      await handle.result();
+
+      expect(await workspace.runtime.detachExec(handle.id)).toBe(false);
+      expect(exits).toEqual([]);
+    });
+
+    it("refuses to detach without onDetachedExit", async () => {
+      const { workspace, release } = await detachedWorkspace(null, false);
+
+      const handle = await workspace.runtime.exec("export default null");
+
+      await expect(workspace.runtime.detachExec(handle.id)).rejects.toThrow(/onDetachedExit/);
+      expect(workspace.runtime.backends()[0]?.detachable).toBe(false);
+      release();
+      await handle.result();
+    });
+  });
+
   it("migrates the legacy execution journal schema", async () => {
     const db = new Database(new SQLiteTestStorage());
     initializeSchema(db, () => 0);