Build linux release binaries on a dedicated ubuntu runner - #116
Closed
Bencheng21 wants to merge 1 commit into
Closed
Bencheng21 wants to merge 1 commit into
Bencheng21 wants to merge 1 commit into
Conversation
**Why**
The release critical path is the macOS binaries job, and almost all of it is one
step. On baton-okta v0.5.36 (12m07s end to end), goreleaser-binaries took 9m15s,
of which "Run GoReleaser" was 447s and the compile phase alone was 6m45s:
18:15:20 3 builds start (linux/amd64, linux/arm64, darwin/amd64)
18:20:51 darwin/arm64 finally starts <- waited 5m31s for a CPU slot
18:22:05 compile phase done (6m45s)
macos-latest has 3 vCPUs and GoReleaser builds one target per core, so the fourth
target sat idle for over five minutes. Notarization is not the bottleneck: the gon
hooks took 9s. Only the darwin targets need codesign/notarize, so the linux builds
were occupying the scarcest runner in the matrix for no reason.
**What this changes**
- Adds a goreleaser-linux job on ubuntu-latest that builds linux amd64/arm64,
signs, generates SBOMs and provenance, uploads to S3, and emits a manifest
fragment plus its checksums. It mirrors the existing goreleaser-windows job and
runs with --skip=publish; the darwin job still owns the GitHub release.
- Narrows the binaries template to the darwin targets and adds a linux template.
- merge-manifests takes -linux-manifest and merges its assets map, skipping the
partial checksums entry that publish-release-manifest recomputes.
- publish-release-manifest concatenates the linux checksums into the unified file.
Two contracts a single GoReleaser run used to cover implicitly, now handled
explicitly so they do not silently regress:
- The Homebrew formula. GoReleaser can only build formula entries from artifacts
in its own run, so a darwin-only run would drop the tap's on_linux blocks. The
formula is now rendered by cmd/generate-brew-formula from the merged manifest,
where every platform is present, and pushed to the tap by publish-release-manifest.
Rendering it against baton-okta v0.5.36 reproduces the published formula exactly
apart from the header comment and the CDN URLs.
- The GitHub release contents. GoReleaser now attaches only the darwin archives and
a darwin-only checksums file, so publish-release-manifest re-attaches the linux
archives and clobbers the checksums file with the unified one. The release keeps
the same five assets it has today.
Formula download URLs move from GitHub release assets to dist.conductorone.com,
which is already the canonical distribution channel and what the registry records.
**Validation**
- go build ./... , go vet ./... , go test ./cmd/... (new tests cover formula class
naming, incomplete-asset skipping, and per-OS block rendering).
- scripts/test-release-config-templates.py, extended to assert the binaries and
linux templates build disjoint GOOS sets and that no template declares brews.
- scripts/test-release-workflow-tag-pin.sh, extended to cover the new GoReleaser
invocation; the invariant was verified directly since yq was unavailable locally.
- scripts/test-normalize-release-options.sh, scripts/test-derive-iam-role-name.sh.
- Parsed .github/workflows/release.yaml with PyYAML.
Not validated: the workflow has not been executed. Job-level behaviour needs a real
release, so this should go out on a prerelease tag against one connector first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The release critical path is the macOS binaries job, and almost all of it is a single step. On
baton-oktav0.5.36 (12m07s end to end),goreleaser-binariestook 9m15s, of whichRun GoReleaserwas 447s — and the compile phase alone was 6m45s:macos-latesthas 3 vCPUs and GoReleaser builds one target per core, so the fourth target sat idle for over five minutes. Notarization is not the bottleneck — thegonhooks took 9s. Only the darwin targets need codesign/notarize, so the linux builds were occupying the scarcest runner in the matrix for no reason.Measured job durations today (two recent releases):
What this changes
goreleaser-linuxjob onubuntu-latestthat builds linux amd64/arm64, signs, generates SBOMs and provenance, uploads to S3, and emits a manifest fragment plus its checksums. It mirrors the existinggoreleaser-windowsjob and runs with--skip=publish; the darwin job still owns the GitHub release and changelog..goreleaser-binaries-template.yaml.tmplto the darwin targets; adds.goreleaser-linux-template.yaml.tmpl.merge-manifestsgains-linux-manifestand merges itsassetsmap (skipping the partialchecksumsentry thatpublish-release-manifestrecomputes).publish-release-manifestconcatenates the linux checksums into the unified file.Two contracts that a single GoReleaser run used to cover implicitly
These would have regressed silently, so they are now handled explicitly:
The Homebrew formula. GoReleaser can only build formula entries from artifacts in its own run, so a darwin-only run would drop the tap's
on_linuxblocks and breakbrew installon Linux for every connector. The formula is now rendered bycmd/generate-brew-formulafrom the merged manifest — where every platform is present — and pushed to the tap bypublish-release-manifest. Rendering it againstbaton-oktav0.5.36 reproduces the published formula exactly apart from the header comment and the URLs:Class name, version, SHAs, CPU conditions, ordering and the
test doblock are byte-identical.The GitHub release contents. GoReleaser now attaches only the darwin archives and a darwin-only checksums file.
publish-release-manifestre-attaches the linux archives and--clobbers the checksums file with the unified one, so the release keeps the same five assets it has today.Behaviour change to be aware of
Formula download URLs move from GitHub release assets to
dist.conductorone.com. That is already the canonical distribution channel and what the connector registry records, but it is user-visible. Previously published formulae are untouched.Expected saving
Estimated ~3–4 minutes off the critical path (12m → ~8–9m): the macOS job drops to 2 targets with no queueing, and linux compiles concurrently on a 4-vCPU runner. This is a projection from the measured timings above, not a measurement — see below.
Validation
go build ./...,go vet ./...,go test ./cmd/...— new tests cover formula class naming, incomplete-asset skipping, and per-OS block rendering.scripts/test-release-config-templates.py, extended to assert the binaries and linux templates build disjoint GOOS sets (two jobs writing the same archive to the same immutable S3 key would fail the release) and that no template still declaresbrews:.scripts/test-release-workflow-tag-pin.sh, extended to cover the new GoReleaser invocation.yqwas unavailable locally, so the invariant was verified directly against the parsed YAML.scripts/test-normalize-release-options.sh,scripts/test-derive-iam-role-name.sh..github/workflows/release.yamlwith PyYAML.Not validated: the workflow has not been executed. Job-level behaviour can only be confirmed by a real release, which is why this is a draft. Suggested rollout: cut a prerelease tag on one connector against this branch, then confirm (a) the tap commit diff is URL-only, (b) the GitHub release still has all five assets, (c) the unified checksums file covers all four archives, and (d)
verify-releasepasses.Since callers pin
@v4, this ships only when the v4 tag moves or v5 is cut.🤖 Generated with Claude Code