Repository navigation
ci: publish each release to PyPI, npm and crates.io, and open the Homebrew formula PR - #80
Conversation
…tests packaging/ turns a release's engine bundles into the PyPI wheel (basert), the npm packages (@basecompute/basert and its engine packages), the crates.io crates and the Homebrew formula; tests/scripts/test_packaging.py tests them on synthetic bundles. They are the engine tree's copy, written by its source sync (tools/sync_public.py), which keeps them current from now on.
…ebrew formula PR publish.yml runs when a release is published and by hand for a tag. It builds every package from this repository's checkout and the release's engine bundles, installs each on macOS and on Linux arm64, verifies the crates with cargo publish --dry-run, and uploads only when PUBLISH_ENABLED is "true", the release is published (not a draft or pre-release), and the run is at the commit the tag names; every version file in that tree must say the tag's version. Each upload job asks its registry first, so a rerun repeats nothing. The Linux wheel and npm engine package go up only with PUBLISH_LINUX. Being a public repository, it publishes with PyPI attestations and npm provenance. Actions are pinned to commits, tools to exact versions. ci.yml runs the packaging tests.
CodeQL (actions/missing-workflow-permissions) flagged the new packaging job, and the three jobs before it on main. No job writes: each checks out the repository, and rust-sys-abi downloads the latest release. The repository's default token is read-only already; this says so in the workflow.
IsuraManchanayake
left a comment
There was a problem hiding this comment.
Codex Review
Reviewed e40be2c28e809ccf1a9fabe333c8f9792a9424f5, including the release trigger, source/tag/version checks, registry rerun logic, job dependencies, permissions, package scripts, and the companion source-sync/release handoff in internal #679.
One blocking finding: the Python 3.12.15 pin cannot be installed by setup-python on the required macOS runner. See the inline comment. All four publishing destinations are consequently blocked even though ordinary PR CI is green.
Validation:
- All 17 packaging tests passed.
actionlintpassed forpublish.yml; all 19 extracted shell steps passed Bash syntax checking.- Replayed the actual plan/registry scripts with controlled command responses: 9 release/upload-gate cases and 5 registry/partial-upload cases passed, including annotated tags, mismatched commits, version mismatch, missing releases, and the Linux gate.
- Verified the configured Node and Python packaging-tool versions exist. Reproduced the Python setup failure using the exact pinned action commit and checked its distribution manifest and the hosted macOS image inventory.
- Compared the packaging helpers with the previously reviewed internal #679 implementation: differences are comments only.
The package/runtime integration still relies on the documented #679 source sync landing before the first release. No Actions dispatch, registry upload, Linux execution, or complete package installation was performed in this follow-up; these checks do not substitute for a full workflow dry run after fixing the pin.
|
|
||
| env: | ||
| TAG: ${{ github.event.release.tag_name || inputs.tag }} | ||
| PYTHON_VERSION: "3.12.15" |
There was a problem hiding this comment.
Codex Review — [P1] Use a Python pin available on macOS
test-macos uses this pin at lines 267–269, but the official Python versions manifest provides 3.12.15 only for Linux/RHEL, with no Darwin build. The macOS 15 arm64 image has 3.12.10 cached, so the exact 3.12.15 request cannot fall back to it. I ran the pinned setup-python commit locally on macOS arm64 with the action's input defaults: it exits 1 with The version '3.12.15' with architecture 'arm64' was not found. This fails the job before any package install; PyPI, npm, crates.io and Homebrew all depend on that job, so no release can publish. Please select an exact version available on both platforms (for example, the manifest contains 3.13.15 for Darwin and Linux arm64/x64), then exercise the macOS setup step in the dry run.
There was a problem hiding this comment.
Fixed in 635eab8: PYTHON_VERSION is now 3.14.9. The manifest has no macOS build for any 3.12 release after 3.12.10. 3.13's bugfix releases are ending, so a later 3.13 bump would hit the same gap. 3.14.9 has builds for darwin arm64 and Linux arm64/x64, and build 1.6.1, wheel 0.48.0 and twine 7.0.0 all declare 3.14.
I replayed the Python steps locally on 3.14.7:
- the 17 packaging tests pass;
- both v0.4.1 bundles build wheels that pass
twine check --strict; - the macOS wheel installs and its launcher runs on 3.14.7 and on 3.9.6. The wheel stays
py3-nonewithRequires-Python: >=3.9.
The setup step first runs for real after merge, because workflow_dispatch needs the workflow on the default branch. A v0.4.1 dispatch exercises setup and the builds. Its bindings checks will still fail until the 0.4.2 source sync brings the _engine/ lookup to main. 0.4.2's own release run is the first that can pass end to end.
There was a problem hiding this comment.
Codex Review — verified fixed in 635eab8. The new pin has builds for all workflow runner platforms. I also ran the exact official macOS Python 3.14.9 runtime: all 17 packaging tests pass, both wheels build and pass strict Twine checks, and the macOS wheel CLI works on 3.14.9 and 3.9.6. The previous P1 no longer applies; the full Actions dry run remains a separate validation step.
test-macos asked setup-python for 3.12.15, which actions/python-versions builds for Linux only: the 3.12 line has had no macOS build since 3.12.10, when it went security-only. The step failed before anything was installed, and every upload job needs test-macos. 3.14.9 has builds for macOS arm64 and for Linux arm64 and x64, and build 1.6.1, wheel 0.48.0 and twine 7.0.0 all declare 3.14. 3.13's bugfix releases end around now, so a later 3.13 bump would hit the same gap. The pin only picks the interpreter these jobs run: the wheel is py3-none and still requires Python >=3.9.
IsuraManchanayake
left a comment
There was a problem hiding this comment.
Codex Review
Reviewed head 635eab89b0b458eaa910e267580af9c8b9e869f8 and the reply to the previous finding. The only code change since my last review is the Python pin plus its explanatory comment. The previous P1 is fixed; no new actionable findings.
Validation of the replacement pin:
- Confirmed
actions/python-versionssupplies 3.14.9 for Darwin arm64 and Ubuntu 24.04 arm64/x64, covering the workflow runners. - Downloaded the official macOS 3.14.9 runtime used by the setup action, extracted it into an isolated directory, and ran that exact interpreter. No system-wide Python installation was needed.
- Installed the exact workflow tools (
build==1.6.1,wheel==0.48.0,twine==7.0.0), passed all 17 packaging tests, and rebuilt both v0.4.1 wheels from this checkout and the release bundles. Both passtwine check --strict. - The macOS wheel installs and runs
basert --version,chat --help, andserve --helpon both Python 3.14.9 and 3.9.6. Both wheels retainpy3-nonetags andRequires-Python: >=3.9. actionlintand diff whitespace checks pass. Completed build/test checks are successful; Swift CodeQL is still running at review time.
This verifies the pin and Python packaging compatibility, not the whole release pipeline. The documented #679 source sync is still required for the packaged bindings' _engine/ discovery; I did not treat these CLI checks as validation of that pending change. A complete Actions dry run, Linux execution, and registry uploads remain untested here.
Publishes each BaseRT release to PyPI, npm and crates.io and opens the Homebrew formula PR, from this repository.
What this adds
.github/workflows/publish.ymlruns when a release is published, and by hand for a tag (tag,dry_run). It builds every package from this repository's checkout of the tag and the release's engine bundles:basert@basecompute/basertand its engine packagesbasert-*CLI crates,baseRT-sysandbaseRTbasert.rbfor basecompute/homebrew-tapIt then installs each package on macOS 15 and Linux arm64 and verifies the crates with
cargo publish --workspace --dry-run.packaging/andtests/scripts/test_packaging.py: the scripts and their tests. Likebindings/, they arrive with the source sync, which keeps them current.ci.yml: apackagingjob (ubuntu-24.04-arm) runs those tests.When it uploads
Only when all of these hold; otherwise the run is a dry run that still builds and tests everything:
PUBLISH_ENABLEDis"true"anddry_runis not ticked;Each upload job asks its registry first, so a rerun finishes an interrupted one and repeats nothing. The Linux wheel and npm engine package are built and tested every time but uploaded only with
PUBLISH_LINUX(the engine licence covers Apple Silicon only for now). Uploads carry PyPI attestations and npm provenance. Crates go up with--no-verify, after the verify job has built them. Actions are pinned to commits. Tools are pinned to exact versions: Python 3.14.9, Node 24.21.0, Rust 1.99.0, build/wheel/twine.Release order
From 0.4.2 on, a release's source sync PR is merged before the release is tagged. The engine's release pipeline refuses a tag whose source this repository doesn't hold, and creates the release at the commit that holds it. Publishing that release starts this workflow on that commit. Earlier tags were made before their sync merged:
v0.4.1here names a commit whose bindings still say 0.4.0, and the plan step refuses it. So 0.4.2 is the first release this can publish.Setup before
PUBLISH_ENABLEDpypi,npm,crates,homebrew, each limited tov*tags.basert: ownerbasecompute, repositorybaseRT, workflowpublish.yml, environmentpypi.NPM_TOKEN(environmentnpm), a token that can publish to@basecompute.CARGO_REGISTRY_TOKEN(environmentcrates). Its account must also be an owner of the existingbaseRT-syscrate.HOMEBREW_TAP_TOKEN(environmenthomebrew), a fine-grained token with Contents and Pull requests write on basecompute/homebrew-tap.PUBLISH_ENABLED=true, andPUBLISH_LINUX=trueonce the Linux licence allows.Tested
Locally on macOS arm64, against the v0.4.1 release bundles, with this branch plus the 0.4.1 source and these packaging files applied:
v0.4.1tag's tree, a missing release.twine check --strict.basert, plus a C program linked against each installed engine.brew style,installandtestfrom a throwaway tap.cargo publish --workspace --dry-runfor all 10 crates.Not run: the workflow itself on Actions, the Linux installs, any upload.