Skip to content

tree-tested: the pull request's job must have run on the same runner labels - #211

Merged
askalf merged 1 commit into
mainfrom
tree-tested-labels
Nov 12, 2026
Merged

askalf merged 1 commit into
mainfrom
tree-tested-labels

Conversation

@askalf

@askalf askalf commented Nov 12, 2026

Copy link
Copy Markdown
Owner

actions/tree-tested now also requires the pull request's job to have asked for the same runner labels as the job asking.

Why

Some workflows pick the runner by who opened the pull request. A caller's CI runs its own branches on a self-hosted row, and a Dependabot or fork pull request on ubuntu-latest. The push after merging that pull request runs on the self-hosted row. The same tree was tested, but on another machine, and a suite can differ between them: one of that caller's tests skips itself where a binary is not on PATH, which is the hosted runner's case. The pull request's run proves the tree on the hosted runner, not on the row the push would use. The rule had no way to see this, and same-on-pull-request cannot say it either: on a push, the workflow cannot tell who opened the pull request.

The change

Rule 5 gains a check. The job of the same name in the pull request's run must have the same labels (its runs-on) as the job asking, compared as a set, ignoring case, as GitHub matches them. Different labels, or labels not listed for either job, answer tested=false, and the job runs in full.

  • scripts/tree-tested/check.mjs: runnerProblem(me, twin), called in decide right after the job is matched by name. The header's rule 5 says so.
  • scripts/tree-tested/check.test.mjs: the fixtures carry labels. Four new checks cover the same labels in another order or case, a hosted runner against a self-hosted one, a job with no labels listed, and the decision when the pull request's job ran on ubuntu-latest. 61 pass.
  • CHANGELOG.

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: approve. The job matching path now rejects a pull request job whose normalized runner-label set differs from the current job, including absent labels, before accepting its checkout log. The added unit and decision-level coverage exercises ordering and case normalization, hosted versus self-hosted labels, and missing labels.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants