Skip to content

test: fix WS-041, SS-098, EM-028, SS-095 - #44095

Merged
Dmitry Gozman (dgozman) merged 3 commits into
microsoft:mainfrom
dgozman:test-ws-041-ss-098
Nov 13, 2026
Merged

Dmitry Gozman (dgozman) merged 3 commits into
microsoft:mainfrom
dgozman:test-ws-041-ss-098

Conversation

@dgozman

@dgozman Dmitry Gozman (dgozman) commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • WS-041: do 20 navigations in place of 100. The test reached the 30 second limit on Firefox on Windows in 24 of 39 runs on main. No navigation hangs; each one takes 0.5 to 1 second on that bot. With 20 navigations the test still fails without the fix from fix(firefox): record WebSocket handshakes after frame detachment #44016.
  • SS-098: skip on Android. The two screenshots have different sizes there (2572x3918 and 2573x3919), and the test failed in 24 of 24 runs.
  • EM-028: skip in headed mode. The test failed in all 10 Linux headed jobs of the Chromium roll microsoft/playwright-browsers#2709. The drag tests in page-drag.spec.ts are skipped in headed mode for the same reason.
  • SS-095: use a 300x300 viewport in place of 200x200. Headed WebKit on Windows has a minimal viewport of 250x240, so setViewportSize threw there and the test failed in 3 of 3 Build WebKit runs on main of microsoft/playwright-browsers.

WS-041 does 100 navigations and reaches the 30 second limit on
Firefox on Windows. 20 navigations still reproduce the original bug.

SS-098 gets screenshots of different sizes on Android.
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@microsoft-playwright-automation

Copy link
Copy Markdown
Contributor

🟢 All three failures predate this PR

Hi, I'm the Playwright bot. I checked the failed MCP run at 9e6a2d9. None of its failures is caused by this PR.

Details and evidence

The PR only reduces navigations in WS-041 and skips SS-098 on Android. Neither test runs in MCP; no product code or shared helpers change.

Pre-existing flake / infra

All three Chrome/Linux failures also occur in the earlier main job, at 8534271f0413bae6a455b7cf43b2637391bc614b, with the same errors:

  • mcp/crash.spec.ts:31 — “reports the navigation error, then resets and logs the crash”: 30-second timeout at the crash-recovery toPass() assertion.
  • mcp/crash.spec.ts:58 — “lists only one tab”: 30-second timeout and Error: Not connected.
  • mcp/crash.spec.ts:79 — “marks non-current crashed tab in the tab list”: 30-second timeout and Error: Not connected.

This confirms an existing Chrome crash-recovery problem, not a regression from this PR. It does not establish whether the problem is intermittent. The supplied “tests 1” report has one flaky test but no final failures.

Checked the failures on Linux by the Playwright bot - agent run

@dgozman Dmitry Gozman (dgozman) changed the title test: reduce navigations in WS-041, skip SS-098 on Android test: reduce navigations in WS-041, skip SS-098 on Android, skip EM-028 in headed Nov 13, 2026
@dgozman Dmitry Gozman (dgozman) changed the title test: reduce navigations in WS-041, skip SS-098 on Android, skip EM-028 in headed test: fix WS-041, SS-098, EM-028 Nov 13, 2026
WebKit on Windows has a minimal viewport of 250x240 in headed mode,
so setting 200x200 throws there.
@dgozman Dmitry Gozman (dgozman) changed the title test: fix WS-041, SS-098, EM-028 test: fix WS-041, SS-098, EM-028, SS-095 Nov 13, 2026
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@microsoft-playwright-automation

Copy link
Copy Markdown
Contributor

🟢 All three reported failures predate this PR

Hi, I'm the Playwright bot. I checked the MCP run at 296d3a1; none of its three reported failures is caused by this PR.

Details and evidence

The tested commit changes only WS-041, SS-098 and EM-028 in library/page tests. The current head, 4d419b5, additionally changes the SS-095 viewport. Neither version changes product code, MCP tests or shared fixtures.

Pre-existing flake / infra

All three [chrome] failures in the Linux job also occur in the earlier main job, at 8534271f0413bae6a455b7cf43b2637391bc614b, with the same errors:

  • mcp/crash.spec.ts:31 — “reports the navigation error, then resets and logs the crash”: 30-second timeout at the crash-recovery toPass() assertion.
  • mcp/crash.spec.ts:58 — “lists only one tab”: 30-second timeout and Error: Not connected.
  • mcp/crash.spec.ts:79 — “marks non-current crashed tab in the tab list”: 30-second timeout and Error: Not connected.

This confirms a pre-existing Chrome crash-recovery problem, not whether it is intermittent. No PR change is needed for these failures.

The MCP run was cancelled, so its merged report is incomplete; the verdict covers the three observed failures, not tests that did not finish. The latest supplied “tests 1” report has no failures and is also incomplete.

Checked the failures on Linux by the Playwright bot - agent run

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

4 failed
❌ [chrome] › mcp/webmcp-dynamic.spec.ts:189 › a tool registered in an iframe can be called after the page is reloaded @mcp-macos-latest-chrome
❌ [chrome] › mcp/crash.spec.ts:31 › crash recovery › reports the navigation error, then resets and logs the crash @mcp-ubuntu-latest-chrome
❌ [chrome] › mcp/crash.spec.ts:58 › crash recovery › lists only one tab @mcp-ubuntu-latest-chrome
❌ [chrome] › mcp/crash.spec.ts:79 › crash recovery › marks non-current crashed tab in the tab list @mcp-ubuntu-latest-chrome

5231 passed, 631 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

1 flaky ⚠️ [playwright-test] › ui-mode-trace.spec.ts:741 › should be able to create and dispose APIRequestContext inside Promise.all `@playwright-x64-server2025-64-core-node22`

23639 passed, 716 skipped


Merge workflow run.

@microsoft-playwright-automation

Copy link
Copy Markdown
Contributor

🟢 All four failures predate this PR

Hi, I'm the Playwright bot. I checked the MCP run at 4d419b5; none of its four failures is caused by this PR.

Details and evidence

The PR only changes WS-041, SS-098, EM-028 and SS-095 in library/page tests. It changes no product code, MCP tests or shared fixtures, and these tests do not run in MCP.

Pre-existing flake / infra

  • Chrome/macOS — mcp/webmcp-dynamic.spec.ts:189, “a tool registered in an iframe can be called after the page is reloaded”: confirmed flake. The current failure is Tool "webmcp_add" not found after reloading. The same test failed with that error on main in run 43083814359 (392ab0f0de39feb99dbe71eb5fea61e7ad07a480) and run 43080620050 (6ca1ba9018acf8ab5b4454c1a42209fa12147304), there on the initial call. It passed on latest main (8534271).

The three Chrome/Linux crash-recovery failures also occur in the earlier main job, at 8534271f0413bae6a455b7cf43b2637391bc614b, with the same errors:

  • mcp/crash.spec.ts:31 — “reports the navigation error, then resets and logs the crash”: 30-second timeout at toPass().
  • mcp/crash.spec.ts:58 — “lists only one tab”: 30-second timeout and Error: Not connected.
  • mcp/crash.spec.ts:79 — “marks non-current crashed tab in the tab list”: 30-second timeout and Error: Not connected.

The crash failures are pre-existing; this evidence does not establish whether they are intermittent. No PR change is needed for these four failures. The latest “tests 1” report has no final failures.

Checked the failures on Linux by the Playwright bot - agent run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants