fix(workflow-executor): hand back the runs the executor lets go instead of leaving them locked - #1945
Merged
Conversation
…ad of leaving them locked Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Coverage Impact This PR will not change total coverage. Modified Files with Diff Coverage (2)
🛟 Help
|
hercemer42
reviewed
Sep 30, 2026
hercemer42
left a comment
Contributor
There was a problem hiding this comment.
Spec (PRD-1387): conforms. Every let-go case the ticket and its PRD-1177 amendment list is handed back.
…re closes with the runner's logger Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
hercemer42
approved these changes
Oct 1, 2026
Scra3
merged commit Oct 1, 2026
22d814c
into
feature/prd-1183-runtime-automation-poller
37 checks passed
Scra3
deleted the
feature/prd-1387-executor-releases-the-runs-it-lets-go
branch
October 1, 2026 18:17
Scra3
added a commit
that referenced
this pull request
Oct 2, 2026
…ad of leaving them locked (#1945)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

fixes PRD-1387
Targets the integration branch
feature/prd-1183-runtime-automation-poller(#1906). Server side: ForestAdmin/forestadmin-server#8550 (POST /release-run), to deploy before #1906 is released; without it this PR degrades to today's behaviour (see Compatibility).Problem
Since forestadmin-server#8536 (PRD-1305) a claimed run stays locked for 7 minutes. When the executor lets a claimed run go on purpose, the orchestrator does not know, and the run stays
loadinguntil the reaper frees it 7 to 7.5 minutes later: spinner then "No progress since…" for the operator, 409 on a re-trigger, and one of the inbox's 20 slots held for an automated run. It happens on every executor deploy or restart, for every run in the middle of an automatic chain.Change
The executor hands the run back with
WorkflowPort.releaseRun(runId, lockedAt), which posts/api/workflow-orchestrator/release-runwith thelockedAtit received. The orchestrator only releases the run under that exact lock, so a run another executor claimed since is never touched.AvailableRunDispatchcarrieslockedAt, copied from the run the orchestrator sent (pending-run,available-run, chainedupdate-step).stoppedwhile the step was running (before, that chain went on to the next step against closed stores);update-stepanswer the adapter cannot use (fails to parse, or no step left to run) is handed back before returningnull.stop()began are handed back instead of left to expire. Runs already in flight on this executor are not.stop()also waits for the poll in progress, inside the same drain timeout, so those hand backs go out before the process exits. The drain timeout log sayspollInProgress.ServerReleaseRunRequestinserver-types.ts, mirroring the server'sReleaseRunRequest.releaseRunnever throws: one request, no retry (it has to fit in the 30 s drain). A 404 is warned once then logged at debug; any other failure is warned. The lock expiring stays the safety net.Compatibility
CLAUDE.mdfollow.Tests
runner.test.ts: hand back on graceful stop (andstop()waits for it), after a drain timeout, at the depth cap, on a non-progressing step of the same run; nothing handed back for another run's step, a completed chain or a chain that runs every step; poll afterstop()hands back each claim but not one already in flight,stop()waits for it, and a poll that never answers is still bounded by the drain timeout; shutdown completes when a hand back rejects.forest-server-workflow-port.test.ts: dispatches carrylockedAt(or null); the exact request; no call without a lock; 404 warned once then debug; 500 warned and not retried; unparseable or finished chained answer handed back,nullanswer not.lockedAtin the adapter, dropping the chained-answer hand back, rethrowing, retrying, warning on every 404, and yielding only ondrainingeach fail at least one test.🤖 Generated with Claude Code