Repository navigation
Add a PR CI workflow that runs only the affected FW Lite checks #2784
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
9615311
Prototype nx graph-only affected detection (ticket 12)
hahn-kev 6aead47
Move CI escalation rules into the nx graph (ticket 06)
hahn-kev 3173059
Add the PR CI workflow: affected-only Linux jobs with escalation tiers
hahn-kev 78fe5a0
Tag the slow FW Lite tests and fix task fw-lite:test-verified
hahn-kev 7cc261e
Document the PR tier in .github/AGENTS.md
hahn-kev feeb2b8
Keep the full tier when a shared-file change marks everything affected
hahn-kev f1be970
Move viewer svelte-check and eslint to fw-lite.yaml
myieye 3cced15
Drop the icon-class.ts re-include from develop-ui.yaml paths
myieye 2e22df2
Keep nx stderr and fix slow-tests with no .NET change
myieye b3a0ac2
Decide CI tiers from their own rules and close affected-detection gaps
myieye e077865
Format the extension project.json and document the dropped launcher step
hahn-kev b23ae25
Stop running the full tier on pull requests
hahn-kev File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| # CI escalation tiers | ||
|
|
||
| Each folder here is an nx pseudo-project named `tier-<name>`. It has no code and | ||
| runs nothing: it exists so that `nx show projects --affected` (via | ||
| `.github/scripts/affected.sh`) lists `tier-<name>` when a pull request should | ||
| pull that tier's full-tier jobs into its PR run. The `ci:<name>` label does the | ||
| same thing by hand, and `ci:full` sets every tier. | ||
|
|
||
| A tier's rule is in its `project.json`, in one of two forms: | ||
|
|
||
| - **Path rule (the default):** a `ci-affected` target whose `inputs` are | ||
| `{workspaceRoot}/...` globs. The tier fires when a file matching a glob | ||
| changes, and only then (an edit upstream of those paths does not fire it). | ||
| The target is never run; only its `inputs` are read. | ||
| - **Project rule:** `implicitDependencies` on project names. The tier fires when | ||
| any listed project is affected, **transitively**: nx walks the reverse graph, | ||
| so a change upstream of a listed project (e.g. `MiniLcm`, `LexCore`, the | ||
| analyzers) fires the tier too. Use it when that is what you want. | ||
|
|
||
| Label-only tiers (`apple`, `android`, `webkit`) have an empty rule and fire only | ||
| by label. | ||
|
|
||
| Two rules on top of the graph: | ||
|
|
||
| - A shared file (`nx.json`'s `sharedGlobals`) never reaches a tier, so it runs | ||
| the whole PR tier, still Linux-only. nx itself marks every tier when `nx.json`, | ||
| the root `package.json` or the root lockfile changes, so the script leaves | ||
| those files out when it asks nx for tiers. | ||
| - Editing a tier's `project.json` fires that tier (its own folder changed), so a | ||
| rule change proves itself on the PR that makes it. | ||
|
|
||
| A missed escalation during the shadow phase becomes a one-line change to the | ||
| tier's `project.json`, reviewed like any other PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| { | ||
| "$schema": "../../../node_modules/nx/schemas/project-schema.json", | ||
| "name": "tier-android", | ||
| "tags": ["ci-tier"], | ||
| "_comment": "Escalation: Android build. Label-only (ci:android) by default; add implicitDependencies here to give it a path rule.", | ||
| "implicitDependencies": [] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| { | ||
| "$schema": "../../../node_modules/nx/schemas/project-schema.json", | ||
| "name": "tier-apple", | ||
| "tags": ["ci-tier"], | ||
| "_comment": "Escalation: iOS simulator + Mac Catalyst build. Label-only (ci:apple) by default; add implicitDependencies here to give it a path rule.", | ||
| "implicitDependencies": [] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| { | ||
| "$schema": "../../../node_modules/nx/schemas/project-schema.json", | ||
| "name": "tier-e2e", | ||
| "tags": ["ci-tier"], | ||
| "_comment": "Escalation: k8s-backed end-to-end tests. Path rule: a direct edit on sync or auth paths (first cut; tune during the shadow phase). (Label: ci:e2e.)", | ||
| "targets": { | ||
| "ci-affected": { | ||
| "executor": "nx:noop", | ||
| "inputs": [ | ||
| "{workspaceRoot}/backend/FwLite/FwLiteProjectSync/**/*", | ||
| "{workspaceRoot}/backend/FwLite/LcmCrdt/**/*", | ||
| "{workspaceRoot}/backend/FwLite/FwLiteShared/Auth/**/*", | ||
| "{workspaceRoot}/backend/FwLite/FwLiteShared/Sync/**/*", | ||
| "{workspaceRoot}/frontend/viewer/tests/e2e/**/*", | ||
| "{workspaceRoot}/.github/actions/setup-k8s/**/*" | ||
| ] | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "$schema": "../../../node_modules/nx/schemas/project-schema.json", | ||
| "name": "tier-full", | ||
| "tags": ["ci-tier"], | ||
| "_comment": "Escalation: every full-tier job. Path-only rule: fires when the PR workflow itself changes. The target is never run; only its inputs matter.", | ||
| "targets": { | ||
| "ci-affected": { | ||
| "executor": "nx:noop", | ||
| "inputs": ["{workspaceRoot}/.github/workflows/pr.yaml"] | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "$schema": "../../../node_modules/nx/schemas/project-schema.json", | ||
| "name": "tier-maui-windows", | ||
| "tags": ["ci-tier"], | ||
| "_comment": "Escalation: Windows MAUI build + MAUI tests. Path rule: a direct edit under the MAUI projects. (Label: ci:maui-windows.)", | ||
| "targets": { | ||
| "ci-affected": { | ||
| "executor": "nx:noop", | ||
| "inputs": ["{workspaceRoot}/backend/FwLite/FwLiteMaui*/**/*"] | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| { | ||
| "$schema": "../../../node_modules/nx/schemas/project-schema.json", | ||
| "name": "tier-slow-tests", | ||
| "tags": ["ci-tier"], | ||
| "_comment": "Escalation: the Slow-trait .NET tests. Path rule: a direct edit in a sync project, the sync helpers, or a test project holding Slow tests. (Label: ci:slow-tests.)", | ||
| "targets": { | ||
| "ci-affected": { | ||
| "executor": "nx:noop", | ||
| "inputs": [ | ||
| "{workspaceRoot}/backend/FwLite/FwLiteProjectSync*/**/*", | ||
| "{workspaceRoot}/backend/FwLite/LcmCrdt*/**/*", | ||
| "{workspaceRoot}/backend/FwLite/FwDataMiniLcmBridge*/**/*", | ||
| "{workspaceRoot}/backend/FwLite/MiniLcm/SyncHelpers/**/*", | ||
| "{workspaceRoot}/backend/FwLite/MiniLcm.Tests/**/*" | ||
| ] | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| { | ||
| "$schema": "../../../node_modules/nx/schemas/project-schema.json", | ||
| "name": "tier-webkit", | ||
| "tags": ["ci-tier"], | ||
| "_comment": "Escalation: viewer Playwright tests on WebKit. Label-only (ci:webkit) by default; add implicitDependencies here to give it a path rule.", | ||
| "implicitDependencies": [] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| #!/usr/bin/env bash | ||
| # Prints ONE JSON object describing what the current branch affects, for the PR | ||
| # workflow's detect job and for `task ci:affected` (same answer in both places). | ||
| # | ||
| # { | ||
| # "projects": [".NET project names affected, transitively, per nx's graph"], | ||
| # "viewer": true|false, // nx project "viewer" (frontend/viewer) affected | ||
| # "extension": true|false, // nx project "extension" (platform.bible-extension) affected | ||
| # "everything": true|false, // every real project is affected (a shared file changed): | ||
| # // run the whole PR tier | ||
| # "tiers": ["slow-tests", ...], // escalation tiers whose tier-<name> pseudo-project | ||
| # // is affected (.github/ci-tiers/<name>/project.json) | ||
| # "fallback": null | "reason" // set when detection failed; everything is then true | ||
| # } | ||
| # | ||
| # Diff range: merge-base of ${AFFECTED_BASE:-origin/develop} and ${AFFECTED_HEAD:-HEAD}, | ||
| # to that head. Only committed changes count, like GitHub's three-dot PR diff. | ||
| # Needs a full-depth checkout (fetch-depth: 0) and no token: everything is local git. | ||
| # | ||
| # Every rule lives in the nx graph (nx.json sharedGlobals, project.json files for | ||
| # the viewer, the extension and the tiers); this script only runs | ||
| # `nx show projects --affected` and shapes its answer. No globs here. | ||
| # | ||
| # Any failure still prints valid JSON with "everything": true and a "fallback" | ||
| # reason, and exits 0, so a broken detector can only over-run the PR tier, never | ||
| # skip it. Escalation tiers then come from labels only. | ||
|
|
||
| set -u | ||
|
|
||
| cd "$(git rev-parse --show-toplevel 2>/dev/null || pwd)" || exit 0 | ||
|
|
||
| base_ref="${AFFECTED_BASE:-origin/develop}" | ||
| head_ref="${AFFECTED_HEAD:-HEAD}" | ||
|
|
||
| fallback() { | ||
| # $1: reason (kept to printable ASCII, quotes stripped, so it is safe inside JSON) | ||
| local reason | ||
| reason=$(printf '%s' "$1" | tr -cd '[:print:]' | tr -d '"\\' | cut -c1-300) | ||
| printf '{"projects":[],"viewer":true,"extension":true,"everything":true,"tiers":[],"fallback":"%s"}\n' "$reason" | ||
| exit 0 | ||
| } | ||
|
|
||
| # nx prints its JSON array as the last stdout line; banners and errors go to stderr, | ||
| # which is left visible so a fallback can be diagnosed from the log. | ||
| nx_json() { | ||
| local out | ||
| out=$("$nx" "$@" --json) || return 1 | ||
| out=$(printf '%s\n' "$out" | grep '^\[' | tail -n 1) | ||
| [ -n "$out" ] || return 1 | ||
| printf '%s' "$out" | ||
| } | ||
|
|
||
| merge_base=$(git merge-base "$base_ref" "$head_ref" 2>&1) \ | ||
| || fallback "git merge-base $base_ref $head_ref failed: $merge_base" | ||
|
|
||
| nx=./node_modules/.bin/nx | ||
| [ -x "$nx" ] || fallback "nx is not installed: run 'pnpm install' at the repo root" | ||
|
|
||
| all=$(nx_json show projects) \ | ||
| || fallback "nx show projects failed" | ||
| affected=$(nx_json show projects --affected --base="$merge_base" --head="$head_ref") \ | ||
| || fallback "nx show projects --affected failed" | ||
|
|
||
| # nx marks every project, tiers included, when nx.json, the root package.json or the root | ||
| # lockfile changes, so tiers are decided on the diff without those files: each tier fires by | ||
| # its own rule only. sharedGlobals never reach a tier (no `default` input), so they don't escalate. | ||
| tier_files=$(git diff --name-only --no-renames "$merge_base" "$head_ref" -- \ | ||
| ':!nx.json' ':!package.json' ':!pnpm-lock.yaml' ':!pnpm-workspace.yaml') \ | ||
| || fallback "git diff $merge_base $head_ref failed" | ||
| tier_affected=$(nx_json show projects --affected --files="$(printf '%s' "$tier_files" | paste -sd, -)") \ | ||
| || fallback "nx show projects --affected --files failed" | ||
|
|
||
| # Assemble the result with node (nx already requires it), so names are escaped properly. | ||
| ALL="$all" AFFECTED="$affected" TIER_AFFECTED="$tier_affected" node -e ' | ||
| const isTier = (n) => n.startsWith("tier-"); | ||
| const isSpecial = (n) => n === "viewer" || n === "extension"; | ||
| const all = JSON.parse(process.env.ALL); | ||
| const affected = JSON.parse(process.env.AFFECTED); | ||
| const real = (names) => names.filter((n) => !isTier(n)); | ||
| const projects = affected.filter((n) => !isTier(n) && !isSpecial(n)).sort(); | ||
| const everything = real(all).every((n) => affected.includes(n)); | ||
| const tiers = JSON.parse(process.env.TIER_AFFECTED).filter(isTier) | ||
| .map((n) => n.slice("tier-".length)).sort(); | ||
| process.stdout.write(JSON.stringify({ | ||
| projects, | ||
| viewer: affected.includes("viewer"), | ||
| extension: affected.includes("extension"), | ||
| everything, | ||
| tiers, | ||
| fallback: null, | ||
| }) + "\n"); | ||
| ' || fallback "could not assemble JSON from nx output" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| // Writes FwLiteAffectedTests.slnf at the repo root: the FW Lite test projects named in $TEST_PROJECTS | ||
| // (a JSON array of project names from pr.yaml's detect job), taken from FwLiteCore.slnf so the paths | ||
| // and the solution reference stay in one place. `dotnet test FwLiteAffectedTests.slnf --no-build` | ||
| // then runs exactly those projects, concurrently, after a whole-FwLiteCore.slnf build. | ||
| import {readFileSync, writeFileSync} from 'node:fs'; | ||
|
|
||
| const wanted = new Set(JSON.parse(process.env.TEST_PROJECTS ?? '[]')); | ||
| const core = JSON.parse(readFileSync('FwLiteCore.slnf', 'utf8')); | ||
| const name = (p) => p.split('\\').pop().replace(/\.csproj$/, ''); | ||
| const projects = core.solution.projects.filter((p) => wanted.has(name(p))); | ||
| const missing = [...wanted].filter((w) => !projects.some((p) => name(p) === w)); | ||
| if (missing.length) throw new Error(`not in FwLiteCore.slnf: ${missing.join(', ')}`); | ||
| if (projects.length === 0) throw new Error('TEST_PROJECTS selected no test project'); | ||
|
|
||
| writeFileSync('FwLiteAffectedTests.slnf', JSON.stringify({solution: {path: core.solution.path, projects}}, null, 2) + '\n'); | ||
| console.log(`FwLiteAffectedTests.slnf: ${projects.map(name).join(', ')}`); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.