Repository navigation
ci(release): hub release jobs on GitHub-hosted runners (owner 10-21) + temporary apt guard (LB-504) - #94
Merged
Conversation
…-504) Release VMs carry Ubuntu's apt-daily-upgrade.timer, which fired 20-25 min after boot on pool VMs today and killed runners mid-job. The release image is not rebaked before 1.6.2, so every job in publish.yml that runs on pf-ci-release now stops the apt units as its first step, fail-safe: a failed stop prints the reason and the step still ends 0. No continue-on-error (the org policy checker reads it as ignoring a failure) and no time limit. tests/test_release_workflows.py keeps its pins: the apt guard is pinned as the exact first step of all four jobs, tag-guard becomes 3 steps and failure-report 4 (their checkout and refusal steps move one index), the shell: bash mutation anchor counts 6, and every removal still has to break a pin. Co-Authored-By: Claude Code <noreply@anthropic.com>
The 1.6.2 recovery risk (CORE-TOP 15:11, L6 GO) covers every release-class job that can run past the apt-daily-upgrade window on pf-ci-release. The release-finalize workflow runs there too: its `finalize` job assigns the final tags and its `notify-consumers` job has real steps (the consumer dispatch), so both are exposed. Add the same first step to both jobs, byte-identical to the one publish.yml already carries: stop apt-daily.timer, apt-daily-upgrade.timer, their services and unattended-upgrades.service, fail-safe through `|| echo "apt guard: stop failed ($?), continuing"` followed by `exit 0`. No continue-on-error (the org policy checker treats it as ignoring a failure) and no timeout-minutes (the owner's no-time-limits rule). Pin it in tests/test_release_workflows.py: both release-finalize jobs must start with the exact APT_GUARD_STEP, so a later edit cannot drop the guard or move it behind a checkout. The pin reuses the constant publish.yml is already checked against rather than restating the command. Temporary: laneboss's rebaked release image replaces this step after 1.6.2.
…unning (LB-504) The guard added earlier today stopped apt-daily(-upgrade).service along with the timers. Interrupting dpkg in the middle of an upgrade is worse than the lost job the guard exists to prevent, so the step no longer touches a service. New body (identical in all six pf-ci-release jobs): systemctl show -p ActiveState,SubState apt-daily-upgrade.service apt-daily.service || true if systemctl is-active --quiet apt-daily-upgrade.service || systemctl is-active --quiet apt-daily.service; then echo "apt guard: upgrade already running, not interrupting"; else sudo systemctl stop apt-daily.timer apt-daily-upgrade.timer || echo "apt guard: timer stop failed ($?), continuing"; fi exit 0 - The show line first, so the log records what the guard found. - apt-daily-upgrade is a oneshot: while it runs its state is "activating" and `is-active --quiet` returns non-zero, so the else branch still runs — harmless, because stopping a timer never touches a running service. - Only the two TIMERS are stopped (nothing new starts). No service, no unattended-upgrades.service. - Still fail-safe, no continue-on-error (the org policy checker treats it as ignoring a failure), no timeout-minutes, no expression interpolation. APT_GUARD_STEP in tests/test_release_workflows.py updated to the same bytes, so the pin still proves the guard is steps[0] of every release job. Temporary: laneboss's rebaked release image replaces this step after 1.6.2.
…st (owner 10-21, R6 exception) Owner decision, 2026-10-21 (main message 1562390657274748981, "pineforge release 改用githuh runner"): the pineforge-release repository runs on GitHub-hosted runners. This is a STANDING rule, not a 1.6.2-only measure, and the hub does not return to Cloud Build. It is the owner's R6 / rule-B exception for this repo: ci-policy-check.py refuses `ubuntu-latest` (runner_allowed, R6), the hub's own workflows do not call the checker, and no checker change or exemption flag is made here. What changes: `runs-on: ["self-hosted","pf-ci","pf-ci-release"]` becomes `runs-on: ubuntu-latest` on exactly these 14 jobs, each with a comment citing the owner message id: facts-descriptions.yml descriptions facts-fanout.yml web, license handle-upstream.yml bump publish.yml tag-guard, prepare, publish, failure-report release-finalize.yml finalize, notify-consumers release-receipts.yml plan, verify release-reconcile.yml producers release-rollback.yml restore Kept unchanged: every #91 guard (main-only tags, tag-guard, environment, permissions) and the apt-guard steps of this PR. The apt guard is harmless on a hosted VM. PR-class jobs keep pf-ci-x64. tests/test_release_workflows.py: the runner pins now expect `ubuntu-latest` (HOSTED_JOBS, assert_hosted_runners, EXEMPT_RUNNERS); the publish mutation now swaps `ubuntu-latest` for the release array (count 4); new tests refuse the release array on each of the 14 jobs and refuse `pf-ci-release` in any `runs-on` of any workflow. Not run locally (no local test run in this lane); CI runs it. Docs, every deleted or rewritten sentence named: - README.md: "The release and facts-automation jobs use" the array sentence; "the isolated release runner class" ; "Push this runner migration only after laneboss posts" ; "pf-ci-release live" ; "until then jobs on that label wait". - docs/release-chain.md: "LB-435 prepares these jobs for the literal"; "The class has its own runner group restricted to selected workflows"; "belong in this class" (now "formerly belonged"); "Push this migration only after laneboss posts"; "until then jobs on that label wait"; "The runner group allowlist must include" ; "The release runner group admits runs at"; "untouched: pending pf-ci-release, including publication workflows". Not a pure restore of the pre-#91 values: tag-guard did not exist before #91, and prepare, failure-report and plan had a multi-line expression with a pool fallback; all 14 use the plain label. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
luisleo526
changed the title
… authz and release/ready can report (owner 10-21, R304) Hub PR #94 cannot merge: its required check `authz` has been queued on the stalled `pf-ci-x64` pool since ~17:17. The owner's standing decision (main message 1562390657274748981, "pineforge release改用githuh runner") covers every job in pineforge-release, CI included (CORE-TOP R304). Commit 4 moved the 14 release jobs; this commit moves the remaining 9 PR-CI jobs, so the required contexts `authz` and `release/ready` are produced on ubuntu-latest. `runs-on: ["self-hosted","pf-ci","pf-ci-x64"]` becomes `runs-on: ubuntu-latest`, each with a comment citing the owner message id, on: python-test.yml authz, changes, fingerprint-canonical, release-rules, description-renderer, release-ready-replay, release-ready facts-validate.yml authz, validate Unchanged: job names and `name:` fields (so the required contexts `authz` and `release/ready` keep their names), needs, if, permissions, triggers, concurrency, every step, and the PF_CI_LABELS env lines in publish.yml and release-receipts.yml (not a runs-on). git grep shows no `runs-on` that is not ubuntu-latest. tests/test_release_workflows.py (PrTierWorkflowTest): the runner pins now expect `ubuntu-latest` (HOSTED_JOBS with the 9 jobs, assert_hosted_runners, RUNNER_LINE/AUTHZ_RUNNER_LINE, AUTHZ_TEXT with the new authz comment, anchor counts RUNNER_COUNTS 7 and 2 equal to the real number of lines); the mutations swap ubuntu-latest for the old pool array and are still refused; a new test refuses `pf-ci-x64` and `self-hosted` on each of the 9 jobs. Not run: no local test run; CI runs it. Rewritten or deleted text, named by 6 consecutive words: - both authz jobs and the AUTHZ_TEXT pin: "Pool label (needs no toolchain or Docker). Switch to pf-ci-crj once laneboss's bulletin says the Cloud Run class is live." (now "Hosted label (needs no toolchain or Docker).") - README.md: "The release and facts-automation jobs use". - docs/release-chain.md: "keep their existing routing"; "runner array, without a hosted fallback or"; "Keep pf-ci-x64 until laneboss posts"; "pf-ci-crj live"; "Their first job is the exact canonical authz from". Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…rd (fixes release-rules on PR #94) PR #94 `release-rules` failed (run 39575779873, job 119171871167): ReleaseTagGuardFailureReportTest.test_actual_first_refusal_rejects_failed_skipped_cancelled_or_missing_guard and ...test_executed_refusal_rejects_successful_exit_or_failure_only_condition both stopped at `assertIsNotNone(comparison)` in assert_refusal. Cause: the LB-504 apt guard was put in front of every release job, so in publish.yml `failure-report` the refusal step is steps[1] and steps[0] is the apt guard. assert_refusal still read steps[0] ("Stop automatic apt upgrades"), whose `if` is absent, so the condition regex found nothing. The sibling pin (`failure_steps[1] == REFUSAL_STEP`, line ~391) had been moved in that commit, this helper had not. Fix, the intent unchanged and nothing weakened: assert_refusal now requires steps[0] to equal APT_GUARD_STEP exactly, takes steps[1] as the refusal, and requires it to be the step named "Refuse unverified release tag" and a run step (no `uses`), so a checkout can never stand in for it. Everything after that is the same: the condition must be exactly needs.tag-guard.result ==/!= 'success' and the step is executed under bash for success, failure, skipped, cancelled, empty and unknown guard results (exit 0 only for success, `::error::tag-guard did not pass` otherwise). Mutation checks (test_executed_refusal_rejects_successful_exit_or_failure_only_condition), each must make assert_refusal raise AssertionError: - exit 1 -> exit 0 in the refusal (existing); - condition `!= 'success'` -> `== 'failure'` (existing); - WEAKENED condition: `!= 'success' && needs.prepare.result == 'failure'` (new); - REMOVED refusal step: the whole "Refuse unverified release tag" step is deleted, the checkout becomes steps[1] (new). The anchors are counted by `mutate` (count must be 1) and the subtest label now carries both sides of the replacement. Not run: no local test run on this machine; the PR's CI is the proof. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hub release jobs for 1.6.2 and after. 4 commits:
1-3. Temporary apt guard (LB-504). Every job that ran on pf-ci-release gets a first step that prints both apt units' states, does not interrupt an upgrade that is already running, and otherwise stops only the two apt timers. The step is fail-safe: exit 0, no continue-on-error. It is harmless on hosted runners and is reverted once the release image is rebaked.
The 14 hub release jobs run on GitHub-hosted
ubuntu-latest. This follows the owner's standing decision for pineforge-release (main msg 1562390657274748981, 「pineforge release改用githuh runner」), which is the R6/rule-B exception for this repository only.pf-ci-releasein anyruns-on.Hub PR-CI on hosted. The hub's own PR checks (python-test.yml, facts-validate.yml) run on GitHub-hosted
ubuntu-latest(R304). Check names are unchanged, soauthzandrelease/readyare still produced.Test helper fix.
ReleaseTagGuardFailureReportTest.assert_refusalnow expects the apt guard atsteps[0]and the refusal ("Refuse unverified release tag") atsteps[1]. The comparison is not weakened. Mutation checks, each of which must still go red:!= 'success' && needs.prepare.result == 'failure';!= 'success'->== 'failure'.🤖 Generated with Claude Code