Repository navigation
feat(test-infra): cloud QA automatically runs the local-only companion (green means green) - #848
Merged
Merged
Conversation
The cloud vitest image deliberately excludes tests that need a locally built server and a Docker daemon (today: test/integration/server/ — see .dockerignore), so a green cloud run never covered them: the files were simply absent from the shards (the 10-day-hidden digest drift was masked exactly this way). Broad runs (test/check/verify) with FRESHELL_VITEST_BACKEND=cloud now run a 'client-local-only' companion phase locally, in parallel with the cloud shards, scoped to exactly the .dockerignore test exclusions that exist on disk, and fold its exit into the run verdict — green means green again. Focused runs skip the companion when nothing intersects the excluded set (logged); FRESHELL_VITEST_LOCAL_COMPANION=off disables it with a loud warning. Direct test:cloud invocations are unchanged. - parseLocalOnlyTestDirs/resolveCloudCompanionScope/buildCloudCompanionRun exported pure helpers (unit-tested); companion parses the live .dockerignore so new exclusions are tracked automatically. - FRESHELL_VITEST_COMPANION_SCRIPT test seam mirrors FRESHELL_VITEST_CLOUD_SCRIPT for the process-level integration test. - Cloud dispatch converted execFileSync -> concurrent execFile so the companion fills the otherwise-idle cloud wait; local artifact-owning phases remain strictly sequential (companion finishes before they start). Docs updated (AGENTS.md, .dockerignore).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A green cloud QA run previously did NOT mean "all tests passed": the Cloud Run vitest image deliberately excludes tests that need a locally built server and a Docker daemon (today:
test/integration/server/— the managed-provider parity suites, excluded via.dockerignore), so those files were simply absent from the shards. The 10-day-hidden Claude settings-digest drift (fixed in #846) was masked exactly this way — every "green" cloud run in that window silently skipped the only tests that could catch it.Broad runs (
pnpm run test/check/verify) withFRESHELL_VITEST_BACKEND=cloudnow run aclient-local-onlycompanion phase locally, in parallel with the cloud shards, scoped to exactly the.dockerignoretest exclusions that exist on disk, and fold its exit code into the run verdict — green means green again.parseLocalOnlyTestDirs/resolveCloudCompanionScope/buildCloudCompanionRun— exported pure helpers, unit-tested. The companion parses the live.dockerignore, so future test-directory exclusions are tracked automatically (no hardcoded list to drift).FRESHELL_VITEST_LOCAL_COMPANION=offdisables it with a loud warning that green then does NOT cover the excluded tests. Directpnpm run test:cloudinvocations are unchanged (documented in AGENTS.md).execFileSync→ concurrentexecFileso the companion fills the otherwise-idle cloud wait; the artifact-owning local phases (source-runtime / rust / electron) remain strictly sequential — the companion finishes before they start.FRESHELL_VITEST_COMPANION_SCRIPTtest seam mirrorsFRESHELL_VITEST_CLOUD_SCRIPTfor the process-level integration test.Validation
test/unit/tooling/run-standard-tests.test.ts, 30 total in file) + 10 new process-level checks (scripts/test/cloud-vitest-integration.test.sh— broad/focused/off-switch/local dispatch matrix, all green).pnpm run typecheckgreen.FRESHELL_VITEST_BACKEND=cloud GCLOUD_ROBOT_REQUIRE=1 pnpm run test) from this worktree ran the full stack — cloud shards + companion (3 files / 19 tests passed, 1 env-conditional skip) + source-runtime (10) + full workspace cargo + electron (42 files / 395 tests) — exit 0. The log shows the companion starting in parallel with the cloud dispatch (Starting local companion for cloud-excluded tests … scope: ["test/integration/server"]) and its exit folding into the verdict before the local phases begin.