Skip to content

fix: cache-tag defects, egress scoping, and 6 fixture bugs (fetch lanes' findings) - #407

Merged
EricAndrechek merged 3 commits into
v1from
v1-lane-0b2
Oct 2, 2026
Merged

EricAndrechek merged 3 commits into
v1from
v1-lane-0b2

Conversation

@EricAndrechek

@EricAndrechek EricAndrechek commented Oct 2, 2026 •

Copy link
Copy Markdown
Member

Summary

Eight real defects, all measured by the Go and Python fetch lanes running the full local conformance suite (140 case/transport pairs) against cases.json on v1, 2026-10-02 (Python: 127/140 on 3.11/3.13/3.14, same 13 failures — confirming these are fixture bugs, not binding bugs):

  1. existing-install-noop / index-race-reapply (cachecases.go) were tagged under symbolic c-<id> tags instead of their own version spellings, so neither could ever resolve online against a real server.
  2. v1-conformance's egress cut (sudo iptables -A OUTPUT ! -o lo -j REJECT) blocked ALL non-loopback traffic, including the Actions runner's own connection to GitHub, causing TS legs on PR v1 fetch layer, TypeScript lane: resolve, trust, bytes, cache and lock #400 to time out twice at 20 minutes. Now scoped to a dedicated noegress group via --gid-owner, with both properties (no GitHub access, yes loopback) proven inside the step itself.
  3. predicate-wrong-version (trust.go) requested a symbolic tag instead of a real version, so §4's version-comparison check had nothing to compare the predicate against. PM ruling: fix the fixture, not the spec — no config-vs-predicate cross-check added.
  4. zstd-window-too-large's frame header did not actually declare an oversized window — klauspost's encoder sizes the declared window to the payload, not to WithWindowSize's configured ceiling. Now hand-patches the Window_Descriptor byte directly.
  5. preseed-oras named a setup.cache layout (layouts/oras-preseed/) that does not exist in this tree (written only by the workflow_dispatch-only v1-oras-preseed job). The case is now gated on the layout's presence on disk (layoutPresentOnDisk) and skipped loudly, by name, every --write/--check, rather than shipped unbacked.
  6. redirect-cross-origin-drops-auth's Location template doubled the scheme and was missing /v2.
  7. redirect-limit's hop Location headers reused the script-relative path instead of the client-facing one, missing the /v2/s-<id>/ router segment — 404 on hop 2+.
  8. mirror-failover-5xx/-digest-404 used two identical {base} bases, so failover re-read the same already-exhausted script cursor. server.py's cursor now also keys on origin, and a new {base2} (second origin, with its own succeeding second_origin_routes) gives failover something to fail over TO.

Every fix ships with its own generator self-check — checkOnlineTagsExist, checkCacheLayoutsExist, mustCompareTwoRealVersions, mustDeclareOversizeWindow, validateLocationHeaders, checkBase2ScriptsCanSucceed — plus a planted-mismatch selftest proving each one actually fires, so this class of bug cannot recur silently.

docs/guides/fetch-v1.md §10 documents existing-install-noop's "zero LAYER requests, not zero requests" rule and preseed-oras's on-disk gating.

Verification

  • go build ./..., go vet ./..., gofmt -l . — clean
  • go run . --selftest — OK (all new self-checks exercised, including planted-failure cases)
  • go run . --write then go run . --check (twice) — 783 files match byte-for-byte, idempotent
  • python3 scripts/fetch-v1/schema_check.py --selftest and real run — OK
  • python3 scripts/fetch-v1/server.py --selftest — OK
  • python3 scripts/fetch-v1/parity.py --selftest — OK
  • Live spot-check against a running server.py: redirect-limit hops 0→1→2→3 all 302 (no 404), redirect-cross-origin-drops-auth redirects cross-origin and the second origin serves 200, both mirror-failover-* second origins serve 200 directly
  • scripts/lint-actions.sh, scripts/lint-public.sh, scripts/lint-cited-paths.sh, scripts/lint-spelling.sh, markdownlint-cli2@0.18.1, dprint@0.57.4 check — all clean

Not exercised by this PR's own CI: the egress-scoping step (item 2) is not proven by anything in this PR's run — every v1-conformance leg skips (no binding's runner file exists on v1 yet), so the noegress group/iptables step itself never executes here. The curl proofs inside the step are the only guard: the first fetch lane to merge a runner file is what actually exercises it, and if the rule is wrong, those proofs fail loudly within seconds instead of the job hanging for 20 minutes the way PR #400 did.

Test plan

  • CI green on this PR (v1-fixtures, v1-conformance, v1-network, v1-parity legs)
  • Fetch lanes re-run their local suites against this head and confirm all 8 defects are fixed

🤖 Generated with Claude Code

EricAndrechek and others added 3 commits October 2, 2026 02:56
…he Go/Python fetch lanes

Eight real defects, all measured by the Go and Python fetch lanes running
the full local conformance suite against cases.json on v1, 2026-10-02:

1. existing-install-noop / index-race-reapply (cachecases.go) were tagged
   under symbolic "c-<id>" tags instead of their own version spellings, so
   neither could ever resolve online against a real server.
2. v1-conformance's egress cut (`sudo iptables -A OUTPUT ! -o lo -j REJECT`)
   blocked ALL non-loopback traffic, including the Actions runner's own
   connection to GitHub, causing TS legs to time out. Now scoped to a
   dedicated `noegress` group via --gid-owner, with both directions proven
   inside the step itself.
3. predicate-wrong-version (trust.go) requested a symbolic tag instead of a
   real version, so §4's version-comparison check had nothing to compare
   the predicate against. PM ruling: fix the fixture, not the spec.
4. zstd-window-too-large's frame header did not actually declare an
   oversized window (klauspost's encoder sizes the declared window to the
   payload, not to WithWindowSize's ceiling) — now hand-patches the
   Window_Descriptor byte directly.
5. preseed-oras named a setup.cache layout (layouts/oras-preseed/) that
   does not exist in this tree (written only by the workflow_dispatch-only
   v1-oras-preseed job). The case is now gated on the layout's presence on
   disk and skipped loudly, by name, when absent, rather than shipped
   unbacked.
6. redirect-cross-origin-drops-auth's Location template doubled the scheme
   and was missing /v2.
7. redirect-limit's hop Location headers reused the script-relative path
   instead of the client-facing one, missing the /v2/s-<id>/ router
   segment — 404 on hop 2+.
8. mirror-failover-5xx/-digest-404 used two identical "{base}" bases, so
   failover re-read the same already-exhausted script cursor. server.py's
   cursor now also keys on origin, and a new {base2} (second origin, with
   its own succeeding second_origin_routes) gives failover something to
   fail over TO.

Every fix ships with its own generator self-check (checkOnlineTagsExist,
checkCacheLayoutsExist, mustCompareTwoRealVersions,
mustDeclareOversizeWindow, validateLocationHeaders,
checkBase2ScriptsCanSucceed) plus a planted-mismatch selftest proving each
one actually fires, so this class of bug cannot recur silently.

docs/guides/fetch-v1.md §10 documents existing-install-noop's "zero LAYER
requests, not zero requests" rule and preseed-oras's on-disk gating.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v1.yml's v1-conformance matrix moved to Rust 1.87 (the PM's MSRV ruling;
its own comment flagged the Rust lane's PR would move this line) but
parity.py's TOOLCHAINS mirror was never updated to match, which would have
failed v1-parity with a spurious "missing report for toolchain '1.85'" the
moment any binding enrolls. Noticed while polling this PR's own CI run.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
PM review on #407: the comment labelled the whole story "measured, PR
#400" when only the symptom was measured (both ts legs timed out at 20
minutes, twice, BlobNotFound in both logs, ci.yml's ts job installs the
same lockfile in seconds) — the MECHANISM (that the blanket cut severed
the runner's own link) is inferred, not measured.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@EricAndrechek
EricAndrechek marked this pull request as ready for review October 2, 2026 07:47
@EricAndrechek
EricAndrechek merged commit 5b7ef11 into v1 Oct 2, 2026
48 of 50 checks passed
@EricAndrechek
EricAndrechek deleted the v1-lane-0b2 branch October 2, 2026 07:47
EricAndrechek added a commit that referenced this pull request Oct 2, 2026
Lane 0B's round-3 fixes (merged #407) moved mirror-failover-5xx/
-digest-404 onto two genuinely independent bases ({base}, {base2})
served from server.py's two origins, each with its own response-
sequence cursor. expandBase now also substitutes {base2} against the
server's second port for http transport (docs/guides/fetch-v1.md's
"{base2}" section).

With #407's other fixes (real-version tags for existing-install-noop/
index-race-reapply, predicate-wrong-version's numeric spelling, an
actually-oversize zstd window, preseed-oras dropped while its layout
is absent, corrected redirect templates), the full local conformance
run is now 139/139 passing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant