Skip to content

ci: port shared engineer bot learnings - #1793

Merged
vuanhphung merged 1 commit into
mainfrom
vu-phung/port-engineer-bot-learnings
Nov 13, 2026
Merged

vuanhphung merged 1 commit into
mainfrom
vu-phung/port-engineer-bot-learnings

Conversation

@vuanhphung

@vuanhphung vuanhphung commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Port the shared engineer/reviewer bot lessons from databricks/databricks-sql-nodejs#571 and #572: keep changes within maintainer scope, ground findings in evidence, select unit or live tests by behavior, and handle stale reviews. Follow-ups load trusted default-branch config; agents cannot edit their own .bot/ settings.

Sync conflicting bot PRs only when NEXT_CHANGELOG.md changes are additive Unreleased entries, preserving its Added/Updated/Fixed groups. Other conflicts receive a maintainer comment. Sync is limited to PRs targeting the default branch.

NO_CHANGELOG=true

Testing

Passed 20 local changelog/branch-sync cases, YAML parsing, shell syntax checks, and git diff --check. Added a CI workflow to run the sync tests.

Telemetry Errors

  • Not applicable — this PR does not add or change a telemetry-visible error.

This PR was created with GitHub MCP.

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No issues identified by the review bot.

Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
@vuanhphung
vuanhphung force-pushed the vu-phung/port-engineer-bot-learnings branch from 35a7e92 to 7e004e1 Compare November 13, 2026 21:46

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Medium

Mostly solid, with one medium concern: the sync workflow's App token may be refused when a merge brings in changes to .github/workflows/. If so, the sync fails without posting the maintainer comment. The changelog parser matches the current NEXT_CHANGELOG.md layout. I couldn't check that the engine accepts the new --bot flag in engineer-bot-followup.yml, because the engine source isn't available here.

with:
app-id: ${{ secrets.ENGINEER_BOT_APP_ID }}
private-key: ${{ secrets.ENGINEER_BOT_APP_PRIVATE_KEY }}
permission-contents: write

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — The minted App token has only contents: write and pull-requests: write. A sync merge commit brings in everything main changed since the PR branched. GitHub rejects a GitHub App push that creates or updates files under .github/workflows/ unless the token has the workflows permission ("refusing to allow a GitHub App to create or update workflow … without workflows permission"). This PR itself adds and edits several workflows, so that situation will come up.

When it does, the push in sync_pr raises CalledProcessError. main() catches it, logs a warning and fails the job. It never calls comment_once, so the PR stays conflicted and nothing on the PR tells a maintainer why. That is the exact silent stall this workflow is meant to remove.

Possible fixes, depending on what the App is allowed:

  • If the engineer-bot App can be granted the workflows permission, request permission-workflows: write here.
  • Otherwise, detect a failed push (or a merge that touches .github/workflows/) and post the maintainer comment instead of only warning.

Caveat: I couldn't check the App's installed permissions from here, so this rejection is inferred from GitHub's documented push rule, not observed in a run.

@vuanhphung
vuanhphung added this pull request to the merge queue Nov 13, 2026
Merged via the queue into main with commit 851ff52 Nov 13, 2026
28 checks passed
@vuanhphung
vuanhphung deleted the vu-phung/port-engineer-bot-learnings branch November 13, 2026 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants