Repository navigation
ci: pr-autopilot approve workflow for hermes-exosphere's own PRs #959
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
8d20cd9
646f51c
aea31cd
90f76c7
6b4f73d
5330aeb
cecc3d3
2774777
23e3cdc
991e4fc
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,122 @@ | ||
| # Approve a PR that pr-autopilot reviewed but cannot approve itself, and | ||
| # withdraw that approval when pr-autopilot's verdict turns. | ||
| # | ||
| # Copy to .github/workflows/pr-autopilot-approve.yml in every repo | ||
| # pr-autopilot reviews, on the default branch. It runs on repository_dispatch, | ||
| # which GitHub always runs from the default branch's copy of this file: no | ||
| # dispatcher can pick an edited copy on another branch. | ||
| # | ||
| # pr-autopilot acts as hermes-exosphere, the account that also opens Kiln's | ||
| # and the other agents' PRs. GitHub lets no one approve their own PR, so for | ||
| # those pr-autopilot posts its review as a comment and starts this workflow, | ||
| # which approves as github-actions[bot]. The org ruleset counts that approval. | ||
| # | ||
| # It approves only when all of these hold: | ||
| # * hermes-exosphere started it; | ||
| # * the PR is hermes-exosphere's own, still open, and its head is the | ||
| # commit that was reviewed; | ||
| # * hermes-exosphere's LATEST review on that commit carries the approve | ||
| # marker (pr-autopilot's report::approve_marker). An older approving | ||
| # review does not count once a newer one on the commit says otherwise. | ||
| # When the latest review does not approve, it dismisses github-actions[bot]'s | ||
| # approvals instead, so a newer negative verdict on the same commit revokes | ||
| # an earlier approval (pr-autopilot dispatches it for both verdicts). If the | ||
| # head moves, or the verdict turns, while the approval is posted, the new | ||
| # approval is dismissed. | ||
| # It merges nothing (a person does), never checks out the PR, and takes its | ||
| # inputs through env only. A later push drops the approval: the ruleset | ||
| # dismisses stale reviews. | ||
| # | ||
| # What this does not stop: whoever holds hermes-exosphere's token can write | ||
| # the marker review and send the dispatch. It keeps pr-autopilot's verdict | ||
| # honest and the code that approves fixed; it is not a boundary against | ||
| # that token. That would need a second identity (a GitHub App or account). | ||
|
|
||
| name: pr-autopilot approve | ||
|
|
||
| on: | ||
| # POST /repos/{owner}/{repo}/dispatches with | ||
| # {"event_type": "pr-autopilot-approve", "client_payload": {"pr": "N", "sha": "..."}} | ||
| repository_dispatch: | ||
| types: [pr-autopilot-approve] | ||
|
|
||
| permissions: | ||
| pull-requests: write | ||
|
|
||
| concurrency: pr-autopilot-approve-${{ github.event.client_payload.pr }} | ||
|
|
||
| jobs: | ||
| approve: | ||
| name: Approve pr-autopilot's verdict | ||
| if: github.actor == 'hermes-exosphere' | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 5 | ||
| steps: | ||
| - name: Approve or withdraw | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| REPO: ${{ github.repository }} | ||
| PR: ${{ github.event.client_payload.pr }} | ||
| SHA: ${{ github.event.client_payload.sha }} | ||
| BOT: hermes-exosphere | ||
| run: | | ||
| set -euo pipefail | ||
| case "$PR" in ''|*[!0-9]*) echo "not a PR number: $PR"; exit 1;; esac | ||
| case "$SHA" in ''|*[!0-9a-f]*) echo "not a commit: $SHA"; exit 1;; esac | ||
| # Dismiss github-actions[bot]'s standing approvals on this PR. | ||
| withdraw() { | ||
| gh api --paginate "repos/$REPO/pulls/$PR/reviews" \ | ||
| --jq '.[] | select(.user.login == "github-actions[bot]" and .state == "APPROVED") | .id' | | ||
| while read -r id; do | ||
| gh api -X PUT "repos/$REPO/pulls/$PR/reviews/$id/dismissals" \ | ||
| -f event=DISMISS -f message="$1" >/dev/null | ||
| echo "dismissed approval $id: $1" | ||
| done | ||
| } | ||
| pr="$(gh api "repos/$REPO/pulls/$PR")" | ||
| [ "$(jq -r .state <<<"$pr")" = open ] || { echo "PR #$PR is not open"; exit 1; } | ||
| [ "$(jq -r .user.login <<<"$pr")" = "$BOT" ] || { echo "only $BOT's own PRs"; exit 1; } | ||
| [ "$(jq -r .head.sha <<<"$pr")" = "$SHA" ] || { echo "head moved since the review"; exit 1; } | ||
| # Every page, then the newest of BOT's submitted reviews on this | ||
| # commit: by submitted_at (a pending review's id predates its | ||
| # submission), the id breaking one-second ties. Prints whether it | ||
| # carries the approve marker (true/false), or nothing. | ||
| verdict() { | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. warning This write-capable approval and withdrawal logic has no dedicated automated regression tests in the PR. Add tests executing the actual workflow script against a mocked GitHub API, covering pagination, pending reviews, timestamp ties, withdrawal, and failed post-approval rereads. The existing CI tests do not exercise these decisions. |
||
| gh api --paginate "repos/$REPO/pulls/$PR/reviews" \ | ||
| --jq '.[] | select(.user.login == env.BOT and .commit_id == env.SHA | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. blocker The attestation comes from the same credential that authors the PR and triggers this workflow. Anyone holding that credential can submit a COMMENT review containing the marker and obtain github-actions[bot]'s approval without running pr-autopilot. Running this exact script against a mock GitHub API reproduced that path; main currently requires just one approval. Require an attestation from an independent reviewer or GitHub App whose credential is unavailable to the PR-creating agents before posting APPROVE. |
||
| and .state != "PENDING" and .submitted_at != null) | ||
| | [.submitted_at, .id, (.body | contains("<!-- pr-autopilot:approve " + env.SHA + " -->"))] | ||
| | @tsv' \ | ||
| | sort -k 1,1 -k 2,2n | tail -n 1 | cut -f 3 | ||
| } | ||
| latest="$(verdict)" | ||
| [ -n "$latest" ] || { echo "no pr-autopilot review of $SHA"; exit 1; } | ||
| if [ "$latest" != true ]; then | ||
| withdraw "pr-autopilot's latest review of ${SHA:0:7} does not approve" | ||
| exit 0 | ||
| fi | ||
| have="$(gh api --paginate "repos/$REPO/pulls/$PR/reviews" \ | ||
| --jq '.[] | select(.user.login == "github-actions[bot]" and .state == "APPROVED" and .commit_id == env.SHA) | .id' | head -n 1)" | ||
| [ -z "$have" ] || { echo "already approved ($have)"; exit 0; } | ||
| id="$(gh api -X POST "repos/$REPO/pulls/$PR/reviews" \ | ||
| -f commit_id="$SHA" -f event=APPROVE \ | ||
| -f body="Approved for pr-autopilot, which reviewed commit \`${SHA:0:7}\` (review above) but cannot approve its own account's PR. A person merges." \ | ||
| --jq .id)" | ||
| # Re-read both inputs of the decision after posting: a push, or a | ||
| # newer negative review, between the checks above and the POST | ||
| # would otherwise leave this approval standing until the next run. | ||
| # A failed re-read counts as changed. | ||
| head="$(gh api "repos/$REPO/pulls/$PR" --jq .head.sha || true)" | ||
| if [ "$head" != "$SHA" ]; then | ||
| gh api -X PUT "repos/$REPO/pulls/$PR/reviews/$id/dismissals" \ | ||
| -f event=DISMISS -f message="head moved to ${head:0:7} while ${SHA:0:7} was approved" >/dev/null | ||
| echo "head moved while approving; approval $id dismissed"; exit 1 | ||
| fi | ||
| # Any failure of the re-read (pipefail: a page that errors after an | ||
| # approving one too) dismisses this approval by id, without listing. | ||
| if ! now="$(verdict)" || [ "$now" != true ]; then | ||
| gh api -X PUT "repos/$REPO/pulls/$PR/reviews/$id/dismissals" \ | ||
| -f event=DISMISS -f message="pr-autopilot's verdict on ${SHA:0:7} changed, or could not be re-read, while approving" >/dev/null | ||
| echo "verdict changed or unreadable while approving; approval $id dismissed"; exit 1 | ||
| fi | ||
| echo "approved ${SHA:0:7} ($id)" | ||
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Uh oh!
There was an error while loading. Please reload this page.