From 4ad28861172752dd23f25d014fdbb84828b55b96 Mon Sep 17 00:00:00 2001 From: Pat Buxton Date: Mon, 7 Sep 2026 16:41:07 +0100 Subject: [PATCH] Fetch Trivy directly rather than through setup-trivy setup-trivy resolves a tag by making one unauthenticated, un-retried request to github.com and reporting any non-200 as "unable to find ", so a rate-limited moment arrives looking like a deleted release. A plaid job installed v0.70.0 successfully twice and lost the third attempt forty seconds later; the error named the pin rather than the network, which sent the reader after the wrong thing entirely. A release asset URL needs no tag lookup. The install now downloads the tarball directly with curl --retry and verifies it against this repo's committed copy of the release checksums. The version also names the install directory, which makes an earlier invocation in the same job visible: a repo scanning several requirements files installs Trivy once rather than once per file, so the seven lookups that job made become none. The tarball is what gets cached, and the verification runs on a cache hit as well as a fresh download. A cache entry is written by anyone who can run a workflow on the repo, so a Trivy trusted on sight is a scanner someone else chose -- and one that fabricated a clean report would satisfy the completeness check too, by reading the very same dist-info the check counts. setup-trivy skipped its own checksum on a cache hit for the same reason it skipped the download; this closes that path rather than reproducing it. The checksums are committed rather than fetched: pulled from the network beside the tarball they would vouch for transit only, and read from the cache they would be chosen by whoever wrote the cache. trivy-version must therefore name a release whose checksums are committed here, which makes a scanner bump a reviewed change rather than a silent one. `latest` is refused. It was never a sensible thing for a compliance gate to track, and there is no longer a tag to resolve. Trivy is unpacked via mktemp and mv, so the install directory either has no binary or a whole verified one -- never a half-extracted one that the same-job check would read as installed. The end-to-end job now calls the action twice. The repeat invocation is a real code path -- it is what `uv venv --clear` exists for, and what decides whether Trivy installs once or once per file -- and nothing covered it. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_016wvVVoc2oj2bof9gmq6ZED --- .github/workflows/test.yaml | 8 ++ README.md | 47 ++++++++++- license-check/action.yml | 133 +++++++++++++++++++++++++++++- license-check/trivy-checksums.txt | 22 +++++ 4 files changed, 206 insertions(+), 4 deletions(-) create mode 100644 license-check/trivy-checksums.txt diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml index 2c4db9c..b3a08c4 100644 --- a/.github/workflows/test.yaml +++ b/.github/workflows/test.yaml @@ -30,3 +30,11 @@ jobs: with: requirements: .github/workflows/sample-requirements.txt diagnostics: true + + # A repo with several requirements files calls the action once per file + # in one job, so the second call is a real code path, not a repeat: it is + # what `uv venv --clear` exists for, and what decides whether Trivy is + # installed once or once per file. Nothing else covers it. + - uses: ./license-check + with: + requirements: .github/workflows/sample-requirements.txt diff --git a/README.md b/README.md index cf497df..ab82300 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,7 @@ setuptools 82 — and was last released in September 2023 | `policy` | `''` | Repo-local policy file, applied **in addition to** `license-policy.txt`, not instead of it. | | `python-version` | `3.12` | | | `uv-version` | `0.11.10` | | -| `trivy-version` | `v0.70.0` | Pinned deliberately — see below. | +| `trivy-version` | `v0.70.0` | An exact release whose checksums are committed here; `latest` is refused. Pinned deliberately — see below. | | `diagnostics` | `false` | Print what each Trivy scan surface saw. Run once when onboarding a repo. | ### How it decides @@ -52,6 +52,51 @@ Trivy is the license **extractor**; the policy is applied by check self-calibrates as dependencies change rather than resting on a hardcoded floor. +### How Trivy is installed + +The action fetches Trivy from its release assets itself rather than through +`aquasecurity/setup-trivy`. + +That action shells out to Trivy's `contrib/install.sh`, which turns a tag into a +download by making one unauthenticated, un-retried request to `github.com` and +treating any non-200 as *"unable to find ``"*. A rate-limited moment +therefore arrives looking like a deleted release. It is not hypothetical: one +job installed `v0.70.0` successfully twice and lost the third attempt forty +seconds later, and the error named the pin rather than the network. + +An asset URL needs no tag lookup at all, so the install is: + +- **Downloaded directly**, with `curl --retry 5 --retry-all-errors`, so a 429 or + a 5xx is retried rather than fatal. +- **Verified against `license-check/trivy-checksums.txt`**, which is the release's + own checksums file, committed here. +- **Cached** as the *tarball*, under `trivy-tarball---`. +- **Installed once per job.** The version names the install directory, so a repo + scanning several requirements files calls the action several times and still + installs Trivy once. + +The verification runs on every path — a fresh download and a cache hit alike — +because a cache entry is written by anyone who can run a workflow on the repo. A +Trivy trusted on sight is a scanner someone else chose, and one that fabricated a +clean report would satisfy the completeness check too, by reading the very same +`.dist-info` the check counts. That is why the tarball is what gets cached: it is +the thing the committed checksums can vouch for. + +The checksums are committed rather than fetched for the same reason. Pulled from +the network beside the tarball they would vouch for transit only; read from the +cache they would be chosen by whoever wrote the cache. + +`trivy-version` must therefore name an exact release **whose checksums are +committed here**. Changing it means committing that release's `checksums.txt` +alongside it — the gate pins its scanner by hash, not only by tag, and a version +it cannot vouch for fails loudly rather than quietly. `latest` is refused. + +Two things `aquasecurity/setup-trivy` did that this does not: it accepted +`github-server-url` and `token` inputs for GHES or a private mirror, and it staged +Trivy's `contrib/*.tpl` report templates next to the binary. This action only ever +asks for `--format json`, so nothing here wanted either, but a repo running its own +`trivy` off the `PATH` this action sets will no longer find those templates. + ### The policy file `license-check/license-policy.txt` is the organisation allow-list. There is no diff --git a/license-check/action.yml b/license-check/action.yml index e1acb77..f26cbd3 100644 --- a/license-check/action.yml +++ b/license-check/action.yml @@ -29,7 +29,9 @@ inputs: default: '0.11.10' trivy-version: description: > - Pinned deliberately. Trivy 0.73.0 stopped populating license findings on + An exact release — `v0.70.0` or `0.70.0` — whose checksums are committed + in `trivy-checksums.txt`. `latest` is refused on purpose. + Pinned deliberately: Trivy 0.73.0 stopped populating license findings on Linux; this action reads per-package licenses instead and is unaffected, but a compliance gate should never inherit a scanner upgrade silently. required: false @@ -50,9 +52,134 @@ runs: with: version: ${{ inputs.uv-version }} - - uses: aquasecurity/setup-trivy@v0.3.1 + # Trivy comes from its release assets directly rather than through + # aquasecurity/setup-trivy. That action resolves the tag with a single + # unauthenticated, un-retried request to github.com and reports any non-200 + # as "unable to find ", so a rate-limited moment arrives looking like a + # deleted release - one job installed this very version twice and lost the + # third. An asset URL needs no lookup, and this download retries. + # + # The version also names the install directory, which makes an earlier + # invocation in the same job visible: a repo scanning several requirements + # files installs Trivy once rather than once per file. + - name: Work out which Trivy to fetch + id: trivy + shell: bash + env: + TRIVY_VERSION: ${{ inputs.trivy-version }} + TRIVY_OS: ${{ runner.os }} + TRIVY_ARCH: ${{ runner.arch }} + run: | + set -euo pipefail + if ! printf '%s' "${TRIVY_VERSION}" | grep -Eq '^v?[0-9]+[.][0-9]+[.][0-9]+$'; then + echo "::error::trivy-version must name an exact release, e.g. 'v0.70.0'. Got '${TRIVY_VERSION}'." >&2 + exit 1 + fi + version="${TRIVY_VERSION#v}" + case "${TRIVY_OS}-${TRIVY_ARCH}" in + Linux-X64) asset='Linux-64bit' ;; + Linux-ARM64) asset='Linux-ARM64' ;; + macOS-X64) asset='macOS-64bit' ;; + macOS-ARM64) asset='macOS-ARM64' ;; + *) + echo "::error::No Trivy release asset is mapped for ${TRIVY_OS}/${TRIVY_ARCH}." >&2 + exit 1 + ;; + esac + dir="${RUNNER_TEMP}/trivy-${version}" + installed=false + if [ -x "${dir}/trivy" ]; then + installed=true + fi + { + echo "version=${version}" + echo "asset=${asset}" + echo "dir=${dir}" + echo "download-dir=${RUNNER_TEMP}/trivy-download-${version}" + echo "key=trivy-tarball-${version}-${TRIVY_OS}-${TRIVY_ARCH}" + echo "installed=${installed}" + } >> "${GITHUB_OUTPUT}" + + # The tarball is cached, not the unpacked binary, because the tarball is the + # thing the committed checksums can vouch for. A cache entry is written by + # whoever can run a workflow on the repo, so a restored binary trusted on + # sight would be a scanner an attacker chose - and one that fabricated a + # clean report would satisfy the completeness check too, by reading the very + # same dist-info the check counts. + - name: Restore the Trivy download from the cache + id: trivy-cache + if: steps.trivy.outputs.installed != 'true' + uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: - version: ${{ inputs.trivy-version }} + path: ${{ steps.trivy.outputs.download-dir }} + key: ${{ steps.trivy.outputs.key }} + + # --retry-all-errors so a 429 or a 5xx is retried and not only a dropped + # connection, and --fail so an HTTP error is a failure at all rather than a + # page of HTML saved as a tarball. + - name: Download Trivy + if: steps.trivy.outputs.installed != 'true' && steps.trivy-cache.outputs.cache-hit != 'true' + shell: bash + env: + VERSION: ${{ steps.trivy.outputs.version }} + ASSET: ${{ steps.trivy.outputs.asset }} + DOWNLOAD_DIR: ${{ steps.trivy.outputs.download-dir }} + run: | + set -euo pipefail + tarball="trivy_${VERSION}_${ASSET}.tar.gz" + url="https://github.com/aquasecurity/trivy/releases/download/v${VERSION}/${tarball}" + mkdir -p "${DOWNLOAD_DIR}" + curl --retry 5 --retry-delay 2 --retry-all-errors --fail --silent --show-error --location --output "${DOWNLOAD_DIR}/${tarball}" "${url}" + + # Runs whether the tarball was downloaded or restored, so there is no path + # on which a Trivy binary is trusted unverified. The checksums are committed + # rather than fetched for the same reason: a checksums file pulled from the + # network alongside the tarball vouches for transit only, and one read from + # the cache would be chosen by whoever wrote the cache. + # + # Unpacked via mktemp and mv so `dir/trivy` either does not exist or is a + # whole, verified binary - never a half-extracted one that the same-job + # check above would read as installed. + - name: Verify and unpack Trivy + if: steps.trivy.outputs.installed != 'true' + shell: bash + env: + VERSION: ${{ steps.trivy.outputs.version }} + ASSET: ${{ steps.trivy.outputs.asset }} + DOWNLOAD_DIR: ${{ steps.trivy.outputs.download-dir }} + DIR: ${{ steps.trivy.outputs.dir }} + CHECKSUMS: ${{ github.action_path }}/trivy-checksums.txt + run: | + set -euo pipefail + tarball="trivy_${VERSION}_${ASSET}.tar.gz" + entry="$(awk -v want="${tarball}" '$2 == want' "${CHECKSUMS}")" + if [ -z "${entry}" ]; then + echo "::error::${CHECKSUMS} has no entry for ${tarball}. Changing trivy-version means committing that release's checksums file alongside it - this gate pins its scanner by hash, not only by tag." >&2 + exit 1 + fi + ( cd "${DOWNLOAD_DIR}" && echo "${entry}" | sha256sum -c - ) + work="$(mktemp -d "${RUNNER_TEMP}/trivy-unpack-XXXXXX")" + tar -xzf "${DOWNLOAD_DIR}/${tarball}" -C "${work}" trivy + chmod +x "${work}/trivy" + mkdir -p "${DIR}" + mv "${work}/trivy" "${DIR}/trivy" + rm -rf "${work}" + + # After verification, so a tarball that failed its checksum is never handed + # on to the next job. + - name: Save the Trivy download to the cache + if: steps.trivy.outputs.installed != 'true' && steps.trivy-cache.outputs.cache-hit != 'true' + uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ${{ steps.trivy.outputs.download-dir }} + key: ${{ steps.trivy.outputs.key }} + + - name: Put Trivy on PATH + if: steps.trivy.outputs.installed != 'true' + shell: bash + env: + DIR: ${{ steps.trivy.outputs.dir }} + run: echo "${DIR}" >> "${GITHUB_PATH}" # Licenses are read from each installed package's site-packages METADATA; # a requirements file carries no license data at all. Inputs travel through diff --git a/license-check/trivy-checksums.txt b/license-check/trivy-checksums.txt new file mode 100644 index 0000000..b4f374e --- /dev/null +++ b/license-check/trivy-checksums.txt @@ -0,0 +1,22 @@ +a162b0f1d3e8565152ab8ac6142452fba4fba0aa09de9c8eab238e8c439ce091 trivy_0.70.0_FreeBSD-64bit.tar.gz +03e66cc829292ae03ad806b452d8a9bb508073cf46405de97662d7d3728064c9 trivy_0.70.0_Linux-32bit.deb +02186791a8d5f8f0ecb2a5e8badbcc4e9f38677e3e730a5acbf57254fbef1541 trivy_0.70.0_Linux-32bit.rpm +b98d72c3a81ae13ace8f45e4710089087ede421deead5ab8842d51103ca7b3d9 trivy_0.70.0_Linux-32bit.tar.gz +a9f6ce99fa942759ee914d821b9ad38b6d6887e7d1b4d4415c1c9c7387c7bc94 trivy_0.70.0_Linux-64bit.deb +dd87e7e1b0947fa0529098c78c7336c6b01fd2be33b6dd832e71ba2e5b62b9d5 trivy_0.70.0_Linux-64bit.rpm +8b4376d5d6befe5c24d503f10ff136d9e0c49f9127a4279fd110b727929a5aa9 trivy_0.70.0_Linux-64bit.tar.gz +2a70f06268d962337f0211e282b244c9550389b237d1fbf46c835f3a3139bfa6 trivy_0.70.0_Linux-ARM.deb +1b7adb9da7a1ecbee74043f4c44d22f5c8188ae0eceedd56c0dd5e44d8b74ffb trivy_0.70.0_Linux-ARM.rpm +12537cc6bf3f45e28e0b6b8bea0382ec9fabb468e0c3372e376474d5002c2ffe trivy_0.70.0_Linux-ARM.tar.gz +d3e3379b9753bbb911a3af924c1465138518cbca34eb6c0c7550c7e15bc7ebb8 trivy_0.70.0_Linux-ARM64.deb +8e11415036402db7a5653185ea7f7d2c48d54caa34323df2ddf8616ae88d630a trivy_0.70.0_Linux-ARM64.rpm +2f6bb988b553a1bbac6bdd1ce890f5e412439564e17522b88a4541b4f364fc8d trivy_0.70.0_Linux-ARM64.tar.gz +463760c118da40cbe50a12261767823f179f2eb568be44a30d9e7e684f1a3f92 trivy_0.70.0_Linux-PPC64LE.deb +009c6b6374be45af2260e119636960641eb6145de8ba7a1d9bcad1d090651bf0 trivy_0.70.0_Linux-PPC64LE.rpm +ddc8fb59164e16a9973f46b77b7556de2923ac295731910822f0851ec9edfb01 trivy_0.70.0_Linux-PPC64LE.tar.gz +b86a92507b3230d7898cecd697279afa2c5d1b4ef65cbeae29d40bfb5d5a155f trivy_0.70.0_Linux-s390x.deb +5dff6563443bb488b362777e7de58cad03303b98749664b3b32e5af193c42a18 trivy_0.70.0_Linux-s390x.rpm +92ec9caffdcc32d43b76d21d4ecd6ea2975a2b37e050d5f79e17651c490b1753 trivy_0.70.0_Linux-s390x.tar.gz +52d531452b19e7593da29366007d02a810e1e0080d02f9cf6a1afb46c35aaa93 trivy_0.70.0_macOS-64bit.tar.gz +68e543c51dcc96e1c344053a4fde9660cf602c25565d9f09dc17dd41e13b838a trivy_0.70.0_macOS-ARM64.tar.gz +eea5442eab86f9e26cd718d7618d43899e72a83767619e8bee47911bddbfb825 trivy_0.70.0_windows-64bit.zip