diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml index 2c4db9c..b3a08c4 100644 --- a/.github/workflows/test.yaml +++ b/.github/workflows/test.yaml @@ -30,3 +30,11 @@ jobs: with: requirements: .github/workflows/sample-requirements.txt diagnostics: true + + # A repo with several requirements files calls the action once per file + # in one job, so the second call is a real code path, not a repeat: it is + # what `uv venv --clear` exists for, and what decides whether Trivy is + # installed once or once per file. Nothing else covers it. + - uses: ./license-check + with: + requirements: .github/workflows/sample-requirements.txt diff --git a/README.md b/README.md index cf497df..ab82300 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,7 @@ setuptools 82 — and was last released in September 2023 | `policy` | `''` | Repo-local policy file, applied **in addition to** `license-policy.txt`, not instead of it. | | `python-version` | `3.12` | | | `uv-version` | `0.11.10` | | -| `trivy-version` | `v0.70.0` | Pinned deliberately — see below. | +| `trivy-version` | `v0.70.0` | An exact release whose checksums are committed here; `latest` is refused. Pinned deliberately — see below. | | `diagnostics` | `false` | Print what each Trivy scan surface saw. Run once when onboarding a repo. | ### How it decides @@ -52,6 +52,51 @@ Trivy is the license **extractor**; the policy is applied by check self-calibrates as dependencies change rather than resting on a hardcoded floor. +### How Trivy is installed + +The action fetches Trivy from its release assets itself rather than through +`aquasecurity/setup-trivy`. + +That action shells out to Trivy's `contrib/install.sh`, which turns a tag into a +download by making one unauthenticated, un-retried request to `github.com` and +treating any non-200 as *"unable to find ``"*. A rate-limited moment +therefore arrives looking like a deleted release. It is not hypothetical: one +job installed `v0.70.0` successfully twice and lost the third attempt forty +seconds later, and the error named the pin rather than the network. + +An asset URL needs no tag lookup at all, so the install is: + +- **Downloaded directly**, with `curl --retry 5 --retry-all-errors`, so a 429 or + a 5xx is retried rather than fatal. +- **Verified against `license-check/trivy-checksums.txt`**, which is the release's + own checksums file, committed here. +- **Cached** as the *tarball*, under `trivy-tarball---`. +- **Installed once per job.** The version names the install directory, so a repo + scanning several requirements files calls the action several times and still + installs Trivy once. + +The verification runs on every path — a fresh download and a cache hit alike — +because a cache entry is written by anyone who can run a workflow on the repo. A +Trivy trusted on sight is a scanner someone else chose, and one that fabricated a +clean report would satisfy the completeness check too, by reading the very same +`.dist-info` the check counts. That is why the tarball is what gets cached: it is +the thing the committed checksums can vouch for. + +The checksums are committed rather than fetched for the same reason. Pulled from +the network beside the tarball they would vouch for transit only; read from the +cache they would be chosen by whoever wrote the cache. + +`trivy-version` must therefore name an exact release **whose checksums are +committed here**. Changing it means committing that release's `checksums.txt` +alongside it — the gate pins its scanner by hash, not only by tag, and a version +it cannot vouch for fails loudly rather than quietly. `latest` is refused. + +Two things `aquasecurity/setup-trivy` did that this does not: it accepted +`github-server-url` and `token` inputs for GHES or a private mirror, and it staged +Trivy's `contrib/*.tpl` report templates next to the binary. This action only ever +asks for `--format json`, so nothing here wanted either, but a repo running its own +`trivy` off the `PATH` this action sets will no longer find those templates. + ### The policy file `license-check/license-policy.txt` is the organisation allow-list. There is no diff --git a/license-check/action.yml b/license-check/action.yml index e1acb77..f26cbd3 100644 --- a/license-check/action.yml +++ b/license-check/action.yml @@ -29,7 +29,9 @@ inputs: default: '0.11.10' trivy-version: description: > - Pinned deliberately. Trivy 0.73.0 stopped populating license findings on + An exact release — `v0.70.0` or `0.70.0` — whose checksums are committed + in `trivy-checksums.txt`. `latest` is refused on purpose. + Pinned deliberately: Trivy 0.73.0 stopped populating license findings on Linux; this action reads per-package licenses instead and is unaffected, but a compliance gate should never inherit a scanner upgrade silently. required: false @@ -50,9 +52,134 @@ runs: with: version: ${{ inputs.uv-version }} - - uses: aquasecurity/setup-trivy@v0.3.1 + # Trivy comes from its release assets directly rather than through + # aquasecurity/setup-trivy. That action resolves the tag with a single + # unauthenticated, un-retried request to github.com and reports any non-200 + # as "unable to find ", so a rate-limited moment arrives looking like a + # deleted release - one job installed this very version twice and lost the + # third. An asset URL needs no lookup, and this download retries. + # + # The version also names the install directory, which makes an earlier + # invocation in the same job visible: a repo scanning several requirements + # files installs Trivy once rather than once per file. + - name: Work out which Trivy to fetch + id: trivy + shell: bash + env: + TRIVY_VERSION: ${{ inputs.trivy-version }} + TRIVY_OS: ${{ runner.os }} + TRIVY_ARCH: ${{ runner.arch }} + run: | + set -euo pipefail + if ! printf '%s' "${TRIVY_VERSION}" | grep -Eq '^v?[0-9]+[.][0-9]+[.][0-9]+$'; then + echo "::error::trivy-version must name an exact release, e.g. 'v0.70.0'. Got '${TRIVY_VERSION}'." >&2 + exit 1 + fi + version="${TRIVY_VERSION#v}" + case "${TRIVY_OS}-${TRIVY_ARCH}" in + Linux-X64) asset='Linux-64bit' ;; + Linux-ARM64) asset='Linux-ARM64' ;; + macOS-X64) asset='macOS-64bit' ;; + macOS-ARM64) asset='macOS-ARM64' ;; + *) + echo "::error::No Trivy release asset is mapped for ${TRIVY_OS}/${TRIVY_ARCH}." >&2 + exit 1 + ;; + esac + dir="${RUNNER_TEMP}/trivy-${version}" + installed=false + if [ -x "${dir}/trivy" ]; then + installed=true + fi + { + echo "version=${version}" + echo "asset=${asset}" + echo "dir=${dir}" + echo "download-dir=${RUNNER_TEMP}/trivy-download-${version}" + echo "key=trivy-tarball-${version}-${TRIVY_OS}-${TRIVY_ARCH}" + echo "installed=${installed}" + } >> "${GITHUB_OUTPUT}" + + # The tarball is cached, not the unpacked binary, because the tarball is the + # thing the committed checksums can vouch for. A cache entry is written by + # whoever can run a workflow on the repo, so a restored binary trusted on + # sight would be a scanner an attacker chose - and one that fabricated a + # clean report would satisfy the completeness check too, by reading the very + # same dist-info the check counts. + - name: Restore the Trivy download from the cache + id: trivy-cache + if: steps.trivy.outputs.installed != 'true' + uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: - version: ${{ inputs.trivy-version }} + path: ${{ steps.trivy.outputs.download-dir }} + key: ${{ steps.trivy.outputs.key }} + + # --retry-all-errors so a 429 or a 5xx is retried and not only a dropped + # connection, and --fail so an HTTP error is a failure at all rather than a + # page of HTML saved as a tarball. + - name: Download Trivy + if: steps.trivy.outputs.installed != 'true' && steps.trivy-cache.outputs.cache-hit != 'true' + shell: bash + env: + VERSION: ${{ steps.trivy.outputs.version }} + ASSET: ${{ steps.trivy.outputs.asset }} + DOWNLOAD_DIR: ${{ steps.trivy.outputs.download-dir }} + run: | + set -euo pipefail + tarball="trivy_${VERSION}_${ASSET}.tar.gz" + url="https://github.com/aquasecurity/trivy/releases/download/v${VERSION}/${tarball}" + mkdir -p "${DOWNLOAD_DIR}" + curl --retry 5 --retry-delay 2 --retry-all-errors --fail --silent --show-error --location --output "${DOWNLOAD_DIR}/${tarball}" "${url}" + + # Runs whether the tarball was downloaded or restored, so there is no path + # on which a Trivy binary is trusted unverified. The checksums are committed + # rather than fetched for the same reason: a checksums file pulled from the + # network alongside the tarball vouches for transit only, and one read from + # the cache would be chosen by whoever wrote the cache. + # + # Unpacked via mktemp and mv so `dir/trivy` either does not exist or is a + # whole, verified binary - never a half-extracted one that the same-job + # check above would read as installed. + - name: Verify and unpack Trivy + if: steps.trivy.outputs.installed != 'true' + shell: bash + env: + VERSION: ${{ steps.trivy.outputs.version }} + ASSET: ${{ steps.trivy.outputs.asset }} + DOWNLOAD_DIR: ${{ steps.trivy.outputs.download-dir }} + DIR: ${{ steps.trivy.outputs.dir }} + CHECKSUMS: ${{ github.action_path }}/trivy-checksums.txt + run: | + set -euo pipefail + tarball="trivy_${VERSION}_${ASSET}.tar.gz" + entry="$(awk -v want="${tarball}" '$2 == want' "${CHECKSUMS}")" + if [ -z "${entry}" ]; then + echo "::error::${CHECKSUMS} has no entry for ${tarball}. Changing trivy-version means committing that release's checksums file alongside it - this gate pins its scanner by hash, not only by tag." >&2 + exit 1 + fi + ( cd "${DOWNLOAD_DIR}" && echo "${entry}" | sha256sum -c - ) + work="$(mktemp -d "${RUNNER_TEMP}/trivy-unpack-XXXXXX")" + tar -xzf "${DOWNLOAD_DIR}/${tarball}" -C "${work}" trivy + chmod +x "${work}/trivy" + mkdir -p "${DIR}" + mv "${work}/trivy" "${DIR}/trivy" + rm -rf "${work}" + + # After verification, so a tarball that failed its checksum is never handed + # on to the next job. + - name: Save the Trivy download to the cache + if: steps.trivy.outputs.installed != 'true' && steps.trivy-cache.outputs.cache-hit != 'true' + uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ${{ steps.trivy.outputs.download-dir }} + key: ${{ steps.trivy.outputs.key }} + + - name: Put Trivy on PATH + if: steps.trivy.outputs.installed != 'true' + shell: bash + env: + DIR: ${{ steps.trivy.outputs.dir }} + run: echo "${DIR}" >> "${GITHUB_PATH}" # Licenses are read from each installed package's site-packages METADATA; # a requirements file carries no license data at all. Inputs travel through diff --git a/license-check/trivy-checksums.txt b/license-check/trivy-checksums.txt new file mode 100644 index 0000000..b4f374e --- /dev/null +++ b/license-check/trivy-checksums.txt @@ -0,0 +1,22 @@ +a162b0f1d3e8565152ab8ac6142452fba4fba0aa09de9c8eab238e8c439ce091 trivy_0.70.0_FreeBSD-64bit.tar.gz +03e66cc829292ae03ad806b452d8a9bb508073cf46405de97662d7d3728064c9 trivy_0.70.0_Linux-32bit.deb +02186791a8d5f8f0ecb2a5e8badbcc4e9f38677e3e730a5acbf57254fbef1541 trivy_0.70.0_Linux-32bit.rpm +b98d72c3a81ae13ace8f45e4710089087ede421deead5ab8842d51103ca7b3d9 trivy_0.70.0_Linux-32bit.tar.gz +a9f6ce99fa942759ee914d821b9ad38b6d6887e7d1b4d4415c1c9c7387c7bc94 trivy_0.70.0_Linux-64bit.deb +dd87e7e1b0947fa0529098c78c7336c6b01fd2be33b6dd832e71ba2e5b62b9d5 trivy_0.70.0_Linux-64bit.rpm +8b4376d5d6befe5c24d503f10ff136d9e0c49f9127a4279fd110b727929a5aa9 trivy_0.70.0_Linux-64bit.tar.gz +2a70f06268d962337f0211e282b244c9550389b237d1fbf46c835f3a3139bfa6 trivy_0.70.0_Linux-ARM.deb +1b7adb9da7a1ecbee74043f4c44d22f5c8188ae0eceedd56c0dd5e44d8b74ffb trivy_0.70.0_Linux-ARM.rpm +12537cc6bf3f45e28e0b6b8bea0382ec9fabb468e0c3372e376474d5002c2ffe trivy_0.70.0_Linux-ARM.tar.gz +d3e3379b9753bbb911a3af924c1465138518cbca34eb6c0c7550c7e15bc7ebb8 trivy_0.70.0_Linux-ARM64.deb +8e11415036402db7a5653185ea7f7d2c48d54caa34323df2ddf8616ae88d630a trivy_0.70.0_Linux-ARM64.rpm +2f6bb988b553a1bbac6bdd1ce890f5e412439564e17522b88a4541b4f364fc8d trivy_0.70.0_Linux-ARM64.tar.gz +463760c118da40cbe50a12261767823f179f2eb568be44a30d9e7e684f1a3f92 trivy_0.70.0_Linux-PPC64LE.deb +009c6b6374be45af2260e119636960641eb6145de8ba7a1d9bcad1d090651bf0 trivy_0.70.0_Linux-PPC64LE.rpm +ddc8fb59164e16a9973f46b77b7556de2923ac295731910822f0851ec9edfb01 trivy_0.70.0_Linux-PPC64LE.tar.gz +b86a92507b3230d7898cecd697279afa2c5d1b4ef65cbeae29d40bfb5d5a155f trivy_0.70.0_Linux-s390x.deb +5dff6563443bb488b362777e7de58cad03303b98749664b3b32e5af193c42a18 trivy_0.70.0_Linux-s390x.rpm +92ec9caffdcc32d43b76d21d4ecd6ea2975a2b37e050d5f79e17651c490b1753 trivy_0.70.0_Linux-s390x.tar.gz +52d531452b19e7593da29366007d02a810e1e0080d02f9cf6a1afb46c35aaa93 trivy_0.70.0_macOS-64bit.tar.gz +68e543c51dcc96e1c344053a4fde9660cf602c25565d9f09dc17dd41e13b838a trivy_0.70.0_macOS-ARM64.tar.gz +eea5442eab86f9e26cd718d7618d43899e72a83767619e8bee47911bddbfb825 trivy_0.70.0_windows-64bit.zip