Skip to content

fix(shard): fail when no tests match across all shards - #43173

Merged
Pavel Feldman (pavelfeldman) merged 1 commit into
microsoft:mainfrom
pavelfeldman:fix-43128
Oct 6, 2026
Merged

Pavel Feldman (pavelfeldman) merged 1 commit into
microsoft:mainfrom
pavelfeldman:fix-43128

Conversation

@pavelfeldman

Copy link
Copy Markdown
Member

Summary

  • Since v1.31 the "No tests found" check was skipped for any sharded run, so a bad testDir, --grep or file filter passed silently.
  • In sharded mode the check now uses the test count before sharding: an empty shard still passes, an empty run fails.

Fixes #43128

Since v1.31, sharding happens before the "No tests found" check, so the
check was skipped for any sharded run. A bad testDir, grep or file filter
passed silently. Now the check uses the test count before sharding, so an
empty shard still passes while an empty run fails.

Fixes: microsoft#43128
@github-actions

This comment has been minimized.

@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

6 flaky ⚠️ [chromium-library] › library/webmcp.spec.ts:256 › should list and call tools in cross-origin and nested frames `@frozen-time-library-chromium-linux`
⚠️ [chromium-library] › library/beforeunload.spec.ts:130 › should support dismissing the dialog multiple times `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/video.spec.ts:356 › screencast › should work for popups `@realtime-time-library-chromium-linux`
⚠️ [chromium-library] › library/webmcp.spec.ts:216 › should scope tools to their frame `@chromium-ubuntu-22.04-node24`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-ubuntu-22.04-node20`

52686 passed, 1270 skipped


Merge workflow run.

@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Test results for "MCP"

1 failed
❌ [msedge] › mcp/annotate.spec.ts:417 › should cancel browser_annotate when the MCP client disconnects @mcp-windows-latest-msedge

9034 passed, 1495 skipped


Merge workflow run.

@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

🟢 Playwright bot: this is a pre-existing flake

The only real failure is unrelated to this PR. The same [msedge] test failed with the same dashboard-navigation timeout on unrelated #43039, while passing in 175 of the 177 retained runs.

Details

Overall assessment

This PR only changes test-runner sharding and its tests. It does not reach the MCP annotation flow, browser dashboard, or navigation path involved in the failure. The six retry-rescued tests in the tests 1 report are flaky rather than failing and do not need triage.

Pre-existing flake / infra

Triaged by the Playwright bot - agent run

@pavelfeldman
Pavel Feldman (pavelfeldman) merged commit 4dd2a65 into microsoft:main Oct 6, 2026
44 of 45 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Opt-in failure when a shard contains no tests

2 participants