Repository navigation
feat(test): retain run state when transcript_cancel fails #501
Description
The problem (CMO)
Agent role: Cook in brigade-mode escargot
Area: Automated checks
Problem Statement: As an agent working in nicely, I want failed cancellation checks to retain the saved run's state, so their cause can be investigated.
Analogy
A parcel receipt disappears, but the test stops before checking the delivery book.
- What should happen: the failure report keeps both observations
- What happened in 📚 docs: backlog: mark XT-107 cards done #499: the receipt check failed before the saved-run checks could run
What we saw
A full check failed because transcript_cancel returned INTERRUPTED without the expected run_id and results. Exit 130 was present. Two isolated reruns and the later full signoff passed without runtime changes.
The cause remains open. Neither a runtime defect nor a test-timing explanation is established. The detour required two targeted runs and another full check through signoff.
What exists today
just check runs this scenario through just test. It waits for engine.pid, sends SIGINT, checks exit 130, then checks the response. Saved-run inspection follows the response assertion. That inspection never ran in the failing case.
Start, Stop, Continue (FMO)
- Start: retain a read-only saved-run snapshot when the cancellation response assertion fails, using the existing run-inspection mechanism
- Stop: losing that observation behind the first failing response assertion
- Continue: require the full run-specific response, correct signal exits, saved state, safe replay, and no surviving worker
How we'll know it works
- Today: the quoted failure reports the response mismatch but no saved-run snapshot. It is an observed failure, not a repeatable reproduction
- After the change: a controlled missing-run response still fails and reports the saved record, or explains why it cannot be read
- Nothing else changes: both runs of
uv run --no-project --python 3.12 just scenario transcript/transcript_cancel --count 2pass, including SIGTERM and completion controls. Fulljust checkpasses
👨🏻🍳 For the agent
👨🏻🍳 Details
The failure occurred during docs-only PR #499, on runtime sources from eecf3bacfac4eed6329b7d15e9ddbcbc4a0c4a90. That PR changes only two backlog Markdown files.
Command, exit 1:
uv run --no-project --python 3.12 just check --keep-goingSelected exact log lines:
--- FAIL: TestScripts (77.18s)
--- FAIL: TestScripts/transcript (0.01s)
--- FAIL: TestScripts/transcript/transcript_cancel (5.13s)
Actual stderr:
{"ok":false,"contract_version":1,"errors":[{"code":"INTERRUPTED","message":"Stopped by Ctrl-C. Finished work is kept.","hint":"Check the results before you run anything again."}]}Logged assertion at extensions/transcript/testdata/script/transcript_cancel.txtar:32:
> stderr '"run_id":"c1","results":\[\{"ok":false,"output_dir":"'${OUT@R}'/interrupted","transcript":null,"url":"[^"]+","errors":\[\{"code":"INTERRUPTED","message":"The transcription stopped before it ended\.","hint":"ncly run view c1","next":\["ncly","run","view","c1"\]\}\]'
Testscript reported no match for that assertion. The check summary was:
just check --keep-going:
fmt-check exit 0
lint exit 0
tidy-check exit 0
test exit 1
race exit 0
template exit 0
python exit 0
release-lint exit 0
gitleaks-rules exit 0
gitleaks exit 0
limits exit 0
1 of 11 checks failed
Command, exit 0, without runtime changes:
uv run --no-project --python 3.12 just scenario transcript/transcript_cancel --count 2--- PASS: TestScripts/transcript/transcript_cancel (4.46s)
--- PASS: TestScripts/transcript/transcript_cancel (4.37s)
PASS
ok github.com/pascalandy/nicely/cmd/ncly 8.883s
scenario: transcript/transcript_cancel passed 2 of 2 runs
The subsequent uv run --no-project --python 3.12 just signoff exited 0. Its receipt records check exit 0 and signoff exit 0 at 7816a4124cc3855d9913d4c11461cfe7deee70b0. The same head's hosted check passed. These passes do not establish a cause or a fix.
Existing mechanisms at that head:
- Cancellation scenario: the record inspection uses
ncly run view c1 --jsonafter the failing response assertion - Test launcher: writes PID markers after reading the upload, then waits for release
- Host acknowledgement: sends the acknowledgement after reservation
Preserve the existing assertions. Use the failed-state evidence to decide whether a runtime or synchronization correction is needed. Do not increase a timeout or add an automatic retry as the diagnostic fix. Additional phase stamps are optional only if the saved-record observation cannot separate the paths.
Command selection and --count 2 were exercised above. The existing record-inspection command is present in the scenario. Its state at the failed assertion was not captured, so this issue makes no claim about that state. No system-load measurement or repeatable triggering condition was recorded.
Duplicate searches covered transcript_cancel, transcript cancellation, transcript INTERRUPTED, and flaky cancellation checks, in open and closed issues. #475 covers missing phases near a runner deadline, with handler results preserved. #477 covers a separate signal-drain duration assertion. Neither records this transcript response mismatch or its lost saved-run observation.
by GPT-6.1-Sol via Codex
Activity