Repository navigation
test(webui): refresh the Linux Chromium visual baselines and settle the settings dialog - #236
Merged
Merged
Conversation
added 4 commits
October 23, 2026 11:36
…itles What broke: the Linux browser gate failed on two rail-pin cases because the assertions still expected the old Chinese session titles. The shared browser fixture now names its two synthetic sessions "Synthetic session A" and "Synthetic session B", and de314e0 changed the titles to English for cross-environment portability without updating these two expectations. The rendered labels are correct; only the expectations are stale. Why this fix: the pin button label is built from the session title, so the two expectations now read the title the fixture actually publishes. The glyph assertions next to them are untouched, because those check the drawing rather than the title. How this was verified: the two rail-pin cases pass in a local Playwright run against the built client, and the browser gate is green on the Linux runner for this branch.
What broke: the Linux browser gate failed twelve toHaveScreenshot cases. Every diff was tiny -- 0.01 to 0.03 of the image pixels -- and every image kept its exact pixel dimensions, so no layout had moved. Comparing the committed baseline against a freshly rendered one shows why: the committed images draw the Chinese UI text with real CJK glyphs, while GitHub's ubuntu-latest runner ships no CJK font, so Chromium renders the same labels as tofu boxes. The baselines had been produced on a Linux box that had a CJK font installed, not on the runner the gate actually uses. Why this fix: the baselines are regenerated on the Linux Chromium the browser gate runs on, following the procedure in packages/webui/docs/verification/browser-suite.md, which asks for --update-snapshots on Linux CI Chromium. Nothing else changes: the file names, the count and the pixel dimensions are all the same, so the repository still keeps the fourteen representative screenshots it documented. How this was verified: the regeneration ran as a one-off job on ubuntu-latest with node 24, where pnpm exec playwright test --update-snapshots over visual-baselines.spec.mjs and plugin-catalogue-layout.spec.mjs produced all fourteen files and thirteen cases passed. The browser gate on this branch is green on the Linux runner afterwards.
What broke: the settings dialog baseline was the one of the fourteen that two runs of the same build did not agree on. Comparing two independent regenerations on the Linux runner, thirteen of the fourteen files came back byte for byte identical and this one did not: it moved by a few hundred pixels, the text ink ran one pixel wider and the theme preview images were re-rasterised a fraction of a pixel apart. The overlay fades in over the app and the panel behind it slides in, so the frame a screenshot caught was a frame of that entrance, and the entrance advances on the clock. Why this fix: the browser panel case in this same file already waits for the element's animations to finish before it screenshots, and the settings dialog case did not. It now waits for the same thing over the whole subtree, so the recorded frame is the settled one that every later run records again. The infinite animations are left alone, so the wait cannot hang on one. How this was verified: two regenerations on the Linux runner with this wait in place produce the same settings dialog image byte for byte, and the browser gate on this branch is green afterwards.
What broke: the settings dialog baseline was regenerated with the entrance animation still running, so it held a frame of the overlay fading in rather than the frame the app settles on. Two runs disagreed on it while the other thirteen came back byte for byte identical. Why this fix: the previous commit waits for the overlay to settle before the screenshot, so the baseline records that settled frame. Only this one image changes; the other thirteen are byte for byte what they already were, which is the check that the wait did not move anything else. How this was verified: two regenerations on the Linux runner with the wait in place produce all fourteen images byte for byte identical, and the browser gate on this branch is green afterwards.
modacker
pushed a commit
that referenced
this pull request
Oct 23, 2026
The baselines merged in #236 were rendered on a runner that had no CJK font at all, so every Chinese glyph in them is a tofu box. This commit replaces all 14 Linux screenshots with the same surfaces rendered with Playwright's Linux system dependencies installed, which include fonts-wqy-zenhei. The Chinese in the new baselines is readable text. Regenerated on ubuntu-latest with `playwright install --with-deps chromium` and `--update-snapshots`, three times. Under Playwright's own comparison (pixelmatch, threshold 0.1) the three runs are identical: zero differing pixels in all 14 images, in every pair of runs. At the byte level two images differ between runs, by one and by nine raw pixels respectively; both are far below the comparator threshold, so the gate result does not depend on which run supplied the file. No assertion, expectation or pixel threshold changed, and the file names and count are unchanged.
modacker
pushed a commit
that referenced
this pull request
Oct 23, 2026
The baselines merged in #236 were rendered on a runner that had no CJK font at all, so every Chinese glyph in them is a tofu box. This commit replaces all 14 Linux screenshots with the same surfaces rendered with Playwright's Linux system dependencies installed, which include fonts-wqy-zenhei. The Chinese in the new baselines is readable text. Regenerated on ubuntu-latest with `playwright install --with-deps chromium` and `--update-snapshots`, three times. Under Playwright's own comparison (pixelmatch, threshold 0.1) the three runs are identical: zero differing pixels in all 14 images, in every pair of runs. At the byte level two images differ between runs, by one and by nine raw pixels respectively; both are far below the comparator threshold, so the gate result does not depend on which run supplied the file. No assertion, expectation or pixel threshold changed, and the file names and count are unchanged.
modacker
added a commit
that referenced
this pull request
Oct 23, 2026
… baselines (#239) * Install Chromium system dependencies in the browser gate The Linux browser gate installed the pinned Chromium revision but none of its system dependencies, so the runner had no CJK font available. The WebUI is Chinese, and every Chinese glyph in the committed Linux baselines rendered as a tofu box. `playwright install --with-deps chromium` installs Playwright's own dependency set for the platform, which for ubuntu24.04-x64 includes fonts-wqy-zenhei. The command lives in the npm script rather than in a workflow file on purpose: both `Source verification` and `WebUI npm preview` reach the gate through `pnpm verify --profile full`, so the script is the single place that decides what the gate installs. Putting it in one workflow would let the other path drift. On macOS and Windows `registry.installDeps` has no matching branch and returns without installing anything, so local development is unaffected. * Regenerate the Linux visual baselines with real Chinese glyphs The baselines merged in #236 were rendered on a runner that had no CJK font at all, so every Chinese glyph in them is a tofu box. This commit replaces all 14 Linux screenshots with the same surfaces rendered with Playwright's Linux system dependencies installed, which include fonts-wqy-zenhei. The Chinese in the new baselines is readable text. Regenerated on ubuntu-latest with `playwright install --with-deps chromium` and `--update-snapshots`, three times. Under Playwright's own comparison (pixelmatch, threshold 0.1) the three runs are identical: zero differing pixels in all 14 images, in every pair of runs. At the byte level two images differ between runs, by one and by nine raw pixels respectively; both are far below the comparator threshold, so the gate result does not depend on which run supplied the file. No assertion, expectation or pixel threshold changed, and the file names and count are unchanged. --------- Co-authored-by: probe <probe@local>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change
The
test:webui-browsergate onwebuiwas red with 14 failures. None of them is a product regression. The red had three causes, all in the test fixtures rather than in the WebUI client, and this PR fixes all three.1. Twelve visual baselines were generated in the wrong environment
The twelve failing
toHaveScreenshotcases reported pixel differences of 0.01 to 0.03 of the image pixels, and every one of the fourteen baseline images kept its exact pixel dimensions —shell-and-rail-linux.pngis 1434x900 in both the committed and the freshly rendered version. A layout change would have moved the dimensions; nothing moved.Rendering the committed baseline next to a freshly rendered one shows what the pixels are. The committed images draw the Chinese interface text with real CJK glyphs (新建任务, 导入会话, 插件, 输入消息…(输入 / 唤起命令)). GitHub's
ubuntu-latestrunner image ships no CJK font, so Chromium draws the very same labels as tofu boxes. The baselines were produced on a Linux box that had a CJK font installed, not on the runner the gate uses.The baselines are therefore regenerated on the Linux Chromium the gate runs on, following the procedure the repository already documents in
packages/webui/docs/verification/browser-suite.md:Nothing else changes: the file names, the file count and the pixel dimensions are all identical, so the repository still keeps the fourteen representative screenshots it documents. The job that produced them was temporary and is not part of this PR.
2. Two rail-pin expectations were left behind by a fixture rename
de314e05changed the shared browser fixture's two synthetic session titles to English —{ id: "A", title: "Synthetic session A" }and{ id: "B", title: "Synthetic session B" }— for cross-environment portability. Two expectations inrail-pin.spec.mjsstill asserted the old Chinese titles:The rendered label is correct; only the expectation is stale, so the two expectations now read the title the fixture actually publishes. The glyph assertions next to them are untouched, because they check the drawing rather than the title. The two old Chinese titles still in
settings-account-tab.spec.mjsare deliberately left alone: those cases inject their own session array and do not read the shared fixture.3. The settings dialog baseline recorded a frame of an entrance animation
After the regeneration, thirteen of the fourteen images came back byte for byte identical across independent runs on different runner instances, and one did not:
settings-dialog-linux.png. It moved by a few hundred pixels, the text ink ran one pixel wider, and the theme preview images were re-rasterised a fraction of a pixel apart.The cause is that the settings overlay fades in over the app and the panel behind it slides in (
webui-overlay-enter,webui-modal-enter,webui-settings-content-ininpackages/webui/src/client/features/settings/ui/settings.css). The screenshot was taken while those were still running, so the baseline held a frame of the entrance — and the entrance advances on the clock, so the next run caught a different frame. The browser panel case in the same file already waits for its animations to finish before it screenshots; the settings dialog case did not, and now does, over the whole subtree.Validation
Source verificationworkflow on this branch,verify (ubuntu-latest, 24), node 24,pnpm verify --profile full— green, including thetest:webui-browsergate that was red onwebuiat97e33fd8.packages/webui/test/browser/rail-pin.spec.mjs— 11 passed, includingthe pin button offers the unpin glyph on a pinned rowandthe pin button offers the pin glyph on a row that is not pinned.packages/webui/test/browser/settings-account-tab.spec.mjs— 9 passed, on a file this PR does not touch.-linux.pngbaselines, so it proves nothing here; the Linux runner is the only meaningful check, which is why the gate was verified there.Two trade-offs worth a maintainer decision
The baselines are now full of tofu boxes. They record what
ubuntu-latestrenders, which is the Chinese interface text as empty squares. The baselines therefore no longer show — and so no longer prove — that the Chinese text renders. The alternative is to install a CJK font in the browser job before the gate runs and regenerate the baselines with it, which would give baselines that show the real text, at the cost of a permanent CI change and anapt-getstep on every Linux run. This PR takes the minimal route and leaves that decision open.The baselines sit on a shared runner image. If that image ever gains a CJK font — or drops one — the gate goes red again for the same reason as before. The repository's own documentation already treats the runner as the pinned reference, so this is the status quo rather than something this PR introduces.
Publication and contribution checks
release/public-source.json; new tests are declared intest/vitest-suites.jsonwhere applicable. — No source file is added or removed: the diff is 14 modified baseline PNGs and two modified specs, so both manifests are unchanged.packages/webui/docs/verification/browser-suite.md; no document describes the old baselines.Maintainer handoff
Publication scope or license changes (if any): none.
Shared-source port: not needed.