Build the Linux tarballs in their own job - #119
Open
Bencheng21 wants to merge 1 commit into
Open
Bencheng21 wants to merge 1 commit into
Bencheng21 wants to merge 1 commit into
Conversation
**Why**
goreleaser-binaries is the release critical path, and its dominant cost is Go
compilation. It ran on macOS because gon needs the Apple toolchain and keychain
— but it compiled four targets there, and two of them are Linux.
Measured across the most recent successful release of 15 connectors, the build
phase is 85-93% of the GoReleaser step, and the two Linux targets take 2 of the
runner's 3 cores for the whole of it:
baton-aws 3 targets start t+1s, darwin/arm64 queues until t+200s
baton-okta 3 targets start t+1s, darwin/arm64 queues until t+479s
The same two Linux targets are already compiled, identically, in
goreleaser-docker. Both templates declared the same build entry — same goos,
same goarch, same CGO_ENABLED=0, and neither sets flags or ldflags — so the
compile action IDs match and every release built linux/amd64 and linux/arm64
twice, discarding one copy.
**What this changes**
- .goreleaser-binaries-template is now darwin-only. It keeps GitHub Release
creation and the Homebrew formula.
- New .goreleaser-linux-template and a goreleaser-linux job on ubuntu-latest,
running concurrently with the macOS and Windows jobs. It mirrors the macOS
job's signing: syft SBOMs, cosign archive signatures, SLSA v1 provenance,
no-overwrite S3 uploads.
- merge-manifests takes -linux-manifest. It is required, not optional: the
Linux job always runs and publish-release-manifest requires its success, so
an absent Linux manifest means the tarballs were lost rather than skipped.
The Linux manifest's own `checksums` asset is dropped on merge; the
authoritative one is the unified file publish-release-manifest rewrites.
- publish-release-manifest appends the Linux hashes to the unified checksums
file and attaches the tarballs to the GitHub Release. GoReleaser cannot do
that from the Linux job: the release does not exist yet, because the macOS
job creates it and the two run concurrently. The tarballs travel as a
one-day run artifact.
**The Homebrew formula becomes macOS-only**
The tap formula is rendered by GoReleaser in the macOS job, which no longer has
the Linux archive hashes, so the generated formula loses its on_linux block.
This is a deliberate, accepted regression. Preserving it means moving formula
generation out of GoReleaser into a step that renders from the merged manifest,
which is a larger change to the publishing path than this one.
Blast radius, from GitHub Release asset download counts across recent releases
of baton-okta, baton-aws, baton-google-workspace and baton-github: 0-2 installs
per asset per release, with Linux roughly equal to darwin. Linux users of these
connectors are overwhelmingly on the container images or the platform. The
tarballs stay on the Releases page, so the install path still exists; only
`brew install` on Linux goes away.
**What this does not claim**
No saving is predicted here. Removing two of four targets from a 3-core runner
should remove the queueing and the contention, but the macOS job still compiles
two targets cold, and that remains the critical path. The measurement comes
from the first release run on this.
**Review notes**
- The GitHub Release attach is the last step of publish-release-manifest and
fails the job if it fails, which would also block record-registry-api. That
matches the pipeline's existing "incomplete artifacts are a failure" stance,
but it is a deliberate call and worth a second opinion.
- Connectors pin the reusable workflow by ref, so merging to main ships
nothing. This reaches connectors when v4 moves.
**Validation**
- make test, make workflow-validate, make docs, go build ./..., go vet ./...
- merge-manifests exercised end to end against real generate-manifest output
from synthetic darwin and linux dist directories: all four platform assets
present in the merge, `checksums` taken from the binaries manifest and not
the Linux one, releasedAt preserved, and an omitted -linux-manifest exits 1.
- Simulated the unified checksums assembly and the jq rewrite of
assets.checksums: 4 entries, no blank lines, sha256 and sizeBytes match the
assembled file.
- Confirmed the SLSA predicate template's variables are all satisfied by the
new job's step env plus the runner's automatic GITHUB_*/RUNNER_* vars.
- Added assertions that the binaries template declares no Linux target, that
the Linux template disables the release and publishes no formula, and that
the new job pins GORELEASER_CURRENT_TAG.
Not validated: no release has been run with this split.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
goreleaser-binaries is the release critical path, and its dominant cost is Go
compilation. It ran on macOS because gon needs the Apple toolchain and keychain
— but it compiled four targets there, and two of them are Linux.
Measured across the most recent successful release of 15 connectors, the build
phase is 85-93% of the GoReleaser step, and the two Linux targets take 2 of the
runner's 3 cores for the whole of it:
The same two Linux targets are already compiled, identically, in
goreleaser-docker. Both templates declared the same build entry — same goos,
same goarch, same CGO_ENABLED=0, and neither sets flags or ldflags — so the
compile action IDs match and every release built linux/amd64 and linux/arm64
twice, discarding one copy.
What this changes
creation and the Homebrew formula.
running concurrently with the macOS and Windows jobs. It mirrors the macOS
job's signing: syft SBOMs, cosign archive signatures, SLSA v1 provenance,
no-overwrite S3 uploads.
Linux job always runs and publish-release-manifest requires its success, so
an absent Linux manifest means the tarballs were lost rather than skipped.
The Linux manifest's own
checksumsasset is dropped on merge; theauthoritative one is the unified file publish-release-manifest rewrites.
file and attaches the tarballs to the GitHub Release. GoReleaser cannot do
that from the Linux job: the release does not exist yet, because the macOS
job creates it and the two run concurrently. The tarballs travel as a
one-day run artifact.
The Homebrew formula becomes macOS-only
The tap formula is rendered by GoReleaser in the macOS job, which no longer has
the Linux archive hashes, so the generated formula loses its on_linux block.
This is a deliberate, accepted regression. Preserving it means moving formula
generation out of GoReleaser into a step that renders from the merged manifest,
which is a larger change to the publishing path than this one.
Blast radius, from GitHub Release asset download counts across recent releases
of baton-okta, baton-aws, baton-google-workspace and baton-github: 0-2 installs
per asset per release, with Linux roughly equal to darwin. Linux users of these
connectors are overwhelmingly on the container images or the platform. The
tarballs stay on the Releases page, so the install path still exists; only
brew installon Linux goes away.What this does not claim
No saving is predicted here. Removing two of four targets from a 3-core runner
should remove the queueing and the contention, but the macOS job still compiles
two targets cold, and that remains the critical path. The measurement comes
from the first release run on this.
Review notes
fails the job if it fails, which would also block record-registry-api. That
matches the pipeline's existing "incomplete artifacts are a failure" stance,
but it is a deliberate call and worth a second opinion.
nothing. This reaches connectors when v4 moves.
Validation
from synthetic darwin and linux dist directories: all four platform assets
present in the merge,
checksumstaken from the binaries manifest and notthe Linux one, releasedAt preserved, and an omitted -linux-manifest exits 1.
assets.checksums: 4 entries, no blank lines, sha256 and sizeBytes match the
assembled file.
new job's step env plus the runner's automatic GITHUB_/RUNNER_ vars.
the Linux template disables the release and publishes no formula, and that
the new job pins GORELEASER_CURRENT_TAG.
Not validated: no release has been run with this split.
Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
🤖 Generated with Claude Code