Skip to content

SBOM: list vendored defusedxml in both engine SBOMs; tighten the Windows SBOM job (BACKLOG #2498, #2521) - #1882

Merged
wshallwshall merged 6 commits into
mainfrom
b2498-2521-sbom-vendored
Oct 1, 2026
Merged

wshallwshall merged 6 commits into
mainfrom
b2498-2521-sbom-vendored

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

Vault BACKLOG #2498 (P2, closes) and #2521 (P3, progress only). Its own PR: SBOM is a published supply-chain artifact.

#2498. scripts/security/sbom_finalize.py gains --vendored-from DIR, which adds a library component per messagefoundry/_vendor package: name, version, PSF-2.0, purl, with the upstream sdist and per-file digests in pedigree.ancestors, so no unversioned file components land in the NTIA count. All four engine finalize calls pass it (release.yml and security.yml, Linux and Windows). It fails closed on a missing or duplicate README row, a non-package entry under _vendor, a bom-ref collision, or a CycloneDX spec outside 1.x at or above its floor. The spec gates now compare numbers; the old lifecycle allowlist silently dropped Build on a 1.8 BOM. The guard test is in the ENGINE tier (tests/test_vendored_defusedxml.py), so a _vendor change triggers it.

#2521, findings 1, 2, 3, 4, 6 and 7.

  • The Windows sbomenv is in LOCK_ONLY_VENVS, with an assertion that every SBOM venv is listed.
  • Quoted Windows pip and interpreter spellings are matched through one shared _venv_tool fragment.
  • The score steps in release.yml AND security.yml use || rc=1 and exit $rc, tested under bash -e with a pre_fix arm.
  • An advisory Windows sbomqs 2.0.11 score step, digest pinned in-repo.
  • retention-days: 1 on the hand-off. Accepted by the Manager: a lone release-job re-run after a day fails at its download.
  • changelog.d fragments.
  • Finding 5 stays recorded in the ADR.
  • Still OPEN on #2521, per the row's newer PROGRESS banner: finding 8 (whole-step-body sink matching), the flags-before--m pip half of finding 9, finding 10 (backslash continuations), and the toolkit wheel row in SUPPLY-CHAIN.md. They are for a follow-up.

Release.yml hunks sit at base lines 132, 141, 710-712 and 789-792, clear of open PR 1871's 469-490 and 826 onward.

Checks: pytest over 13 sbom, release, venv-pinning, pipefail, shell-syntax, changelog, risky, vendored, security-posture, tooling and container-pin files gave 708 passed. ruff and format clean; mypy clean on messagefoundry (304), tests (1045), and sbom_finalize.py --strict; actionlint passed. The full suite was not run.

Unread legs: security.yml run 36816081634 (sbom and sbom-windows jobs, including the new Windows score step), and the full CI.
Known open, low: venv spellings ${RUNNER_TEMP} and backslash paths, two duplicate loaders, and license.id not checked against SPDX. The round-2 fixes were not re-reviewed (two-round cap).
Head 6755038. Built on 160a4d9 (the prior Builder's commit), plus three review and simplify commits.

wshallwshall added 4 commits September 30, 2026 22:41
…s SBOM job (BACKLOG #2498, #2521)

#2498: sbom_finalize.py --vendored-from DIR adds one component per package
under messagefoundry/_vendor (defusedxml 0.7.1), read from its README:
version, PSF-2.0, purl, sdist URL and SHA-256, per-file upstream SHA-256.
It fails closed on a README that does not record them. All four engine
finalize calls pass it. A test fails when a _vendor package is missing.

#2521: $RUNNER_TEMP/sbomenv joins LOCK_ONLY_VENVS; the own-pip pattern
sees a quoted python.exe; the release score step scores both SBOMs
whatever either does; security.yml's sbom-windows job runs a pinned
Windows sbomqs; the hand-off artifact keeps one day; changelog fragments.
…ray vendor entries

Review round 1 on BACKLOG #2498, #2521:
- upstream sdist and file digests move into pedigree.ancestors, so no
  versionless file component reaches the NTIA count and no hash claims
  bytes the wheel does not ship;
- anything under _vendor/ that is not a package directory fails the run;
- a compound SPDX licence goes in `expression`; a file row recorded twice
  fails; the spec gate compares numbers (1.4 floor), so 1.8 passes;
- a re-run with no root bom-ref leaves no edge to a replaced component;
- the vendored guard moves to tests/test_vendored_defusedxml.py (engine
  tier, so a vendoring PR runs it), reading sbom_finalize._FILE_ROW and
  the uv.lock sdist digest;
- SUPPLY-CHAIN.md says again that an advisory against the copy needs a
  hand check here; the #2521 fragment names this change's behaviour.
…gs the pin rules see

Review round 1 and /simplify on BACKLOG #2521:
- security.yml's `sbom` score step gets the rc pattern release.yml got, so
  a failing Python score no longer skips the IDE SBOM's score;
- the score-step test runs both workflows' steps under bash -e, with a
  pre_fix bool for the control arm, and reuses _run_leak_gate;
- _PIP_INSTALL sees pip3.14 and a quoted Scripts/pip.exe, so
  LOCK_ONLY_VENVS checks those installs too;
- _own_pip_re and _installs_into share one _venv_tool fragment, and one
  table tests both patterns;
- the retention check joins the existing sbom-windows upload test;
- shorter retention and score-step comments.
…ows SBOM job (BACKLOG #2498, #2521)

Review round 2 fixes:
- lifecycle and --vendored-from gates both compare 1.x minors as
  numbers; a 2.x spec fails closed for vendoring;
- a repeated README field row, or two packages recording one release
  (one bom-ref), fails the run; LicenseRef- goes in `expression`;
- no orphan dependency nodes when the root has no bom-ref;
- security.yml's sbom score step runs `if: always()`;
- the release score test pins /usr/local/bin/sbomqs again;
- _PIP_INSTALL takes a closing quote only after a path;
- the vendored guard skips only the helper's named extras; the
  component test pins type `library` again;
- changelog and SUPPLY-CHAIN say scoring runs on scheduled and manual
  security.yml runs, which is when those jobs run.

Banner #2498: CLOSED by this PR. sbom_finalize.py --vendored-from adds
one library component per messagefoundry/_vendor package (name,
version, PSF-2.0, purl pkg:pypi/defusedxml@0.7.1), with upstream sdist
and per-file SHA-256 in pedigree.ancestors; all four engine finalize
calls pass it; tests/test_vendored_defusedxml.py fails when a _vendor
entry is missing from the finalized SBOM. Container SBOM out of scope
(Manager ruling).

Banner #2521: PROGRESS, stays OPEN. Findings 1, 2, 3, 4, 6, 7 fixed
(3 also in security.yml's twin step), and 9's python3 half. Still open
from the row's closing list: 8 (attestation, release-asset and
artifact-upload sinks still match a whole step body), 9's flags-before
`-m pip` half, 10 (backslash continuations not joined), and the toolkit
wheel row in docs/SUPPLY-CHAIN.md. 5 stays as the ADR records it.
@wshallwshall wshallwshall added the qa Builder QA record posted; not a merge gate label Oct 1, 2026
@wshallwshall

Copy link
Copy Markdown
Collaborator Author

QA -- korus roles/BUILDER.md step 11
Level: xhigh, from the brief. Tag: none returned.
Rounds: 2. Findings: 25 confirmed and fixed (1 only in part), 1 rejected (container-image SBOM: out of scope by Manager ruling), 2 open plus 1 residue.

@wshallwshall
wshallwshall enabled auto-merge October 1, 2026 04:40
@github-actions github-actions Bot added the ci-red A required check went red. Attribute it before retrying. label Oct 1, 2026
wshallwshall added 2 commits October 1, 2026 02:49
The vacuous-absence lint (tests/test_vacuous_absence_assert_lint.py),
which arrived on main, flags test_every_vendored_package_is_in_the_finalized_sbom
on `missing`: the `not missing` check passes when the vendor enumeration
finds nothing. Assert the enumeration is non-empty and the finalized SBOM
has components before the absence check. The `not missing` assertion stays.

Also merges origin/main in. Items: BACKLOG 2498 and 2521.
@wshallwshall

Copy link
Copy Markdown
Collaborator Author

QA: code-review subagent xhigh, head 1d428d1, verdict PASS; guard on the vendored enumeration and SBOM components satisfies the vacuous-absence lint, test strength unchanged, no findings above info.

@wshallwshall
wshallwshall added this pull request to the merge queue Oct 1, 2026
Merged via the queue into main with commit 4864bf5 Oct 1, 2026
63 of 67 checks passed
@wshallwshall
wshallwshall deleted the b2498-2521-sbom-vendored branch October 1, 2026 09:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-red A required check went red. Attribute it before retrying. qa Builder QA record posted; not a merge gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant