fix(seidroid-review): recheck the head before withdrawing, and find a batch review that landed without an answer - #116
Conversation
… batch review that landed without an answer Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
PR SummaryMedium Risk Overview Stale head before withdrawal: When the position step still believes Ambiguous batch review POST: Batched review bodies now include a hidden Reviewed by Cursor Bugbot for commit 384281e. Bugbot is set up for automated code reviews on this repo. Configure here. |
There was a problem hiding this comment.
Re-reading the head before the withdrawal is correct and closes most of the race. The new recovery for an ambiguous batch POST is useful, but its rule for deciding which review "landed" is too loose: it can pick up a review from an earlier run, and when it does, this run's findings are silently dropped.
Findings: 0 blocking | 2 non-blocking | 1 posted inline
Blockers
- None at the file/PR level.
Non-blocking
- No test or fixture covers the new ambiguous-POST recovery path. It is only checked by hand against a stubbed
gh, so a regression here, such as matching the wrong review or appending to it twice, would go unnoticed. - 1 suggestion(s)/nit(s) flagged inline on specific lines.
| *) | ||
| found_id="$(gh api --paginate "repos/$REPO/pulls/$PR/reviews" 2>/dev/null \ | ||
| | jq -rs --arg body "$review_body" --arg sha "$head_sha" \ | ||
| '[.[][] | select(.body == $body and .commit_id == $sha)] | last | .id // empty' \ |
There was a problem hiding this comment.
[suggestion] Matching only on body == review_body and commit_id == head_sha can select a review that this POST did not create. When the driver writes no verdict, review_body is the fixed string "Findings on the changed lines. The verdict and the summary are in this tool's comment…", so any earlier run on the same commit also matches (a re-run, or a rerun after a failed attempt). The same happens if a verdict repeats, or another identity posted the same text. On a false match the run:
- sets
landed=trueand drops the anchored findings, which reach neither the diff nor the summary, - appends the linkage for comments that were never posted, so later superseding and resolving act on the wrong threads,
- sets
position_posted, so the position step skips its review, and - has the verdict step append to the old review.
To narrow it:
- Record the highest existing review id (or the current time) before the POST, and only accept
.id > $before/submitted_at >= $start. - Filter on
.user.loginmatching the posting identity. - Preferably, embed a per-run nonce (for example
$GITHUB_RUN_ID-$GITHUB_RUN_ATTEMPT) in a hidden HTML comment inreview_bodyand match on that.
(Codex raised the same point.)
There was a problem hiding this comment.
Fixed in 384281e. The batch review body now ends with a hidden <!-- seidroid-run:$GITHUB_RUN_ID-$GITHUB_RUN_ATTEMPT --> marker. The body == review_body match therefore only finds this attempt's own review, not an earlier run's, even for the fixed no-verdict text. The marker comes after the text, so the body still opens with VERDICT_MARKER. On the missing test: uci has no workflow test harness yet, so that would be a separate piece of work.
…covery finds only its own Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 384281e. Configure here.
| found_id="$(gh api --paginate "repos/$REPO/pulls/$PR/reviews" 2>/dev/null \ | ||
| | jq -rs --arg body "$review_body" --arg sha "$head_sha" \ | ||
| '[.[][] | select(.body == $body and .commit_id == $sha)] | last | .id // empty' \ | ||
| 2>/dev/null || true)" |
There was a problem hiding this comment.
Reviews lookup omits page size
Medium Severity
The 5xx recovery lists PR reviews without ?per_page=100 in the path, so GitHub pages at the default 30. This collection is oldest-first, and the review that just landed is last. A short or incomplete list misses it, so placement treats the batch as unlanded and the leftover position POST can still record a second vote.
Triggered by learned rule: seidroid-review: VERDICT_MARKER via $ENV; GET page size in path
Reviewed by Cursor Bugbot for commit 384281e. Configure here.


Summary
This fixes the two Cursor Bugbot findings on #115. The fix was pushed after #115 had already merged, so it didn't make it in.
Stale head state. Since fix(seidroid-review): take the review's position on the review that carries the verdict, not a second one #115,
head_stateis read in "Decide the review's position", before placement. A push after that read and before the position step could still let the withdrawal clear a block. The position step now reads the head again whenHEAD_STATEissame:movedandunknownstill skip the withdrawal.Ambiguous batch writes. If the batched review POST fails with a non-4xx status or no response, the review may still have landed. The batch body now ends with a hidden
<!-- seidroid-run:$GITHUB_RUN_ID-$GITHUB_RUN_ATTEMPT -->marker. On an ambiguous failure, placement lists the PR's reviews and looks for one whosebody == review_bodyandcommit_id == head_sha. Because of the run marker, that can only be this attempt's own review.review_idto the found id and setsposition_posted. So the verdict step appends to that review, and the position step posts no second review.Validation: actionlint with shellcheck gives the same 8 infos as
main. I ran the scripts against a stubbedgh. When a 502 hid a review that did land, the run edited that review and posted nothing else. When a 502 hid a review that did not land, the run posted exactly one position review.Link to Devin session: https://app.devin.ai/sessions/9e65f0b138c84a248927a0aac3c24060
Open in Devin Desktop: https://app.devin.ai/desktop/session/9e65f0b138c84a248927a0aac3c24060?variant=devin
Requested by: @bdchatham