Skip to content

ci(release): publish GitHub Releases with gh and the release App - #167

Merged
abrichr merged 2 commits into
mainfrom
ci/gh-release-publish
Oct 18, 2026
Merged

abrichr merged 2 commits into
mainfrom
ci/gh-release-publish

Conversation

@abrichr

@abrichr abrichr commented Oct 18, 2026 •

Copy link
Copy Markdown
Member

Why

The v1.3.0 tag run (39081035898) published the wheel and sdist to PyPI. It then failed at "Publish the GitHub Release through the release App": python-semantic-release/publish-action refuses the detached tag checkout ("Detached HEAD state cannot match any release groups"). This was the first real run of the tag-publication path from #92.

That tag run can't be repaired by re-running it, because a tag run uses release.yml from the tagged commit. A fresh dispatch refuses too, because the tag already exists. And nothing may be created by hand.

What changes

Tag path (publish-tag)

  • The semantic-release step is replaced by gh release create "$tag" dist/* --verify-tag --title "$tag" --notes-file …, with GH_TOKEN set to the release App token.
  • The notes are that version's ## vX.Y.Z (date) section of CHANGELOG.md, written by scripts/release_assets.py notes.
  • Before creating the Release, the job moves out the *.publish.attestation files that pypa/gh-action-pypi-publish writes into dist/. PyPI serves those itself, and the existing verifier would reject them as extra assets. release_assets.py check-pypi then proves that dist/* is exactly one wheel and one sdist, and that PyPI serves those exact SHA-256 digests. The existing final check (verify_release_publication.py --wait-seconds 180) still requires both surfaces.

New: publish_existing_tag dispatch from protected main (publish-existing-tag-release)
For a tag that is already on PyPI but has no GitHub Release:

  • Tag: the dispatch must come from current main. The vVERSION tag must exist, be annotated, have openadapt-release[bot] as its tagger (the same identity the tag event requires as actor), and be contained in main.
  • Version: checked against the tagged source, beside main's tooling. The tagged pyproject version must equal the tag, check_changelog.py must pass, and the CHANGELOG heading must be present.
  • Evidence: check_release_ci.py --sha <tagged sha> requires exact-SHA test and hosted qualification. Nothing is dispatched.
  • Rebuild: the tagged source is rebuilt and passes verify_distribution.py and check_source_boundary.py --require-dist.
  • Digests: release_assets.py fetch-pypi downloads the PyPI files (from files.pythonhosted.org only) and checks each against PyPI's SHA-256. The set must equal the rebuild byte for byte, and those PyPI bytes become dist/.
  • Publish: if PyPI lacks the version the job fails (in fetch-pypi, with an explicit guard after it), because this path never publishes to PyPI. If an exact Release already exists, it creates nothing and only runs the final check, so a rerun after success passes. Otherwise it runs gh release create "$tag" dist/* --verify-tag with the App token, followed by the same final check of both surfaces.
  • It has no PyPI environment, no OIDC, no git tag, and no git push. The job's own token is contents: read; only the App token can write.

create-release-tag is skipped when publish_existing_tag is set. Its "tag already exists" message now points at the new path.

Verification

  • A local uv build of 66c9d25 reproduces both PyPI 1.3.0 files exactly (dde6c7d4… wheel, ebe5751b… sdist), so the rebuild equality holds for v1.3.0.
  • I ran the new job's bash steps locally against the real v1.3.0 tag: tag verification, tagged rebuild with both checks, the PyPI download matching the rebuild, and notes generation. All passed.
  • New tests/test_release_assets.py (14 tests) covers:
    • notes extraction, including missing, inexact and empty sections;
    • leftover attestation files;
    • differing bytes;
    • a tampered download;
    • a host other than files.pythonhosted.org;
    • a version missing from PyPI;
    • a rebuild that differs from PyPI.
  • The workflow contract tests cover both paths, including step order and the absence of PyPI, OIDC and tag writes in the recovery job.
  • pytest tests: 1051 passed, 22 skipped. actionlint is clean, ruff is clean, and check_changelog.py is OK.

After merge I'll dispatch release.yml from main with version=1.3.0 and publish_existing_tag=true.

Signed-off-by: Richard Abrich richard.abrich@mldsai.com

The tag run for v1.3.0 published to PyPI but failed to create the
GitHub Release: the semantic-release publish action refuses the
detached tag checkout ("Detached HEAD state cannot match any release
groups"). Replace it with `gh release create "$tag" dist/* --verify-tag`
using the release App token, with that version's CHANGELOG section as
the notes. Before creating the Release, prove dist/* is exactly the
PyPI artifacts by SHA-256, after moving out the *.publish.attestation
files the PyPI action writes into dist/.

A tag run uses release.yml from the tagged commit, so re-running it
cannot pick up this fix. Add a publish_existing_tag dispatch from
protected main that publishes only the GitHub Release for an existing
tag. It verifies current main, an annotated tag made by the release
App, containment in main, the tagged version and CHANGELOG, exact-tag
test and hosted qualification evidence, and a rebuild of the tagged
source. It downloads the PyPI artifacts, checks them against PyPI's
digests, requires them to equal the rebuild, and uploads those bytes.
It never publishes to PyPI and never creates or moves a tag.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
verify_release_publication.py writes no step outputs when PyPI and the
GitHub Release already match, so the not-on-PyPI guard must trigger on
an explicit publish_pypi=true only. fetch-pypi already fails when the
version is absent from PyPI.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr

abrichr commented Oct 18, 2026

Copy link
Copy Markdown
Member Author

Self-review before merge (head 02e847d):

  • Root cause. The semantic-release publish action resolves "release groups" from the current branch. A tag checkout is detached, so it can never succeed in publish-tag. Replacing it with gh release create "$tag" dist/* --verify-tag and the App token removes that dependency. --verify-tag refuses to create a tag implicitly.
  • Exact assets. dist/* is checked in both paths right before creation, by check-pypi or fetch-pypi. Both reject anything beyond one wheel and one sdist, and any digest that differs from PyPI. The tag path needs the attestation move: without it, dist/* would have carried the two *.publish.attestation files (as on v1.2.2), and the final verifier would have rejected the Release.
  • Recovery path is narrow.
    • Dispatch-only, from current main.
    • The job's own token is read-only, with no PyPI environment and no OIDC.
    • No tag writes.
    • Every tag, version, evidence and rebuild check of the tag event is repeated against the tagged source.
    • The uploaded bytes are PyPI's own, checked against PyPI's digests and required to equal the rebuild.
  • Idempotence. I found and fixed one bug in 02e847d. The verifier writes no outputs when both surfaces already match, so the not-on-PyPI guard now triggers only on an explicit publish_pypi == 'true'. A rerun after success creates nothing and passes the final check.
  • Not covered by CI. The new job can only run on main after merge. Its bash steps were run locally against the real v1.3.0 tag, and they passed.

Signed-off-by: Richard Abrich richard.abrich@mldsai.com

@abrichr
abrichr merged commit 0361c2f into main Oct 18, 2026
12 checks passed
@abrichr
abrichr deleted the ci/gh-release-publish branch October 18, 2026 01:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant