Skip to content

feat(firefox): roll to r1567 - #43671

Closed
microsoft-playwright-automation[bot] wants to merge 1 commit into
mainfrom
roll-into-pw-firefox/1567
Closed

microsoft-playwright-automation[bot] wants to merge 1 commit into
mainfrom
roll-into-pw-firefox/1567

Conversation

@microsoft-playwright-automation

Copy link
Copy Markdown
Contributor

Browser build: https://github.com/microsoft/playwright-browsers/commit/77209371bcfbee41098119ee10ba506221e337f1. Build has full platform coverage - check that all tests pass there.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

5189 passed, 614 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

1 failed
❌ [installation tests] › playwright-packages-install-behavior.spec.ts:40 › @playwright/browser-chromium should work @package-installations-macos-latest

1 flaky ⚠️ [firefox-page] › page/page-filechooser.spec.ts:293 › FC-020 should be able to reset selected files with empty file list `@firefox-playwright-x64-ubuntu24-64-core-node22`

23562 passed, 702 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

🟡 Two Firefox popup failures remain uncertain

Hi, I'm the Playwright bot and I checked the CI failures. Most failures predate this PR, but I cannot clear the Firefox roll yet.

Failure groups and evidence

For head 0736113 (tested merge b1e38f6), 48 of 50 failed tests have independent pre-existing failure evidence; two remain uncertain. I checked the latest bot reports, completed failed-job logs, and the refreshed CI results database. Some tests 2 jobs are still running; this verdict covers completed jobs only, excluding cancelled jobs and tests that passed on retry.

Uncertain — Windows Firefox popup creation

In the Windows Firefox job, these [firefox-library] tests time out on all four attempts, mostly waiting for the popup event:

  • library/page-clock.spec.ts:329 — popup › CK-029 should run time before popup.
  • library/page-clock.spec.ts:345 — popup › CK-030 should not run time before popup on pause.

Each passed all 36 prior runs on this Windows bot, with no failed attempt found in the independent Firefox history. These are popup-creation timeouts, not clock-value assertion failures. The roll changes Firefox page-target/remoteness handling and isolated-world setup, so a regression remains plausible, but the logs do not identify a responsible change. A same-runner comparison of Firefox r1564 and r1567 is needed; there is not enough evidence to call these flakes.

Pre-existing failures / infra

Windows Firefox — the other 40 failures. Every test has an independently recorded failed attempt on the same Windows bot, matched by project, file, and full test title. These prior main runs cover all 40:

Prior main commit and run Matching current failures
a169686, this PR's unchanged base 28: 26 failed, two passed after retry
c409f8c Six: DO-021, CG-103, CG-104, CG-149, TR-014, EH-011
009f887 Three: AR-143 accname #1–#3
0a626ef AR-143 accname #0
3558cd5 LO-196, including the same missing-highlight assertion
b8f2857 GN-056, failed before passing on retry

This includes the download tests exercised by the changed download manager. Some failures are persistent on main, rather than intermittent flakes.

Windows WebKit — two screencast tests. [webkit-library] › library/tracing.spec.ts:565 › TR-023 should produce screencast frames fit / scale also time out in main run 40425277199, commit c409f8c. The PR does not change WebKit or shared tracing code.

Android — five tests. [android-page] ER-001 in page/page-event-pageerror.spec.ts, SE-082/083/086 in page/selectors-get-by.spec.ts, and SE-060 in page/selectors-role.spec.ts all fail on the unchanged base in main run 40510968615, with the same stack-format, label-resolution, and accessible-name errors. These paths are outside the Firefox-only changes.

macOS installation — one test. [installation tests] › playwright-packages-install-behavior.spec.ts:40 › @playwright/browser-chromium should work fails because cdn.playwright.dev cannot resolve (ENOTFOUND). The exact test has the same DNS error in main run 37853065676, commit 2fd57cb. Chromium downloads and the installer are unchanged here.

Triaged by the Playwright bot - agent run

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 2"

1 fatal errors, not part of any test
50 failed
❌ [firefox-library] › library/inspector/cli-codegen-aria.spec.ts:87 › CG-100 should update aria snapshot highlight @firefox-macos-15-large
❌ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:560 › CO-060 should(not) block third party cookies - persistent context @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/browsercontext-reuse.spec.ts:117 › reuse launch › BC-074 should reset serviceworker @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/browsercontext-reuse.spec.ts:155 › reuse launch › BC-075 should reset serviceworker that hangs in importScripts @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/browsercontext-reuse.spec.ts:200 › reuse launch › BC-076 should not cache resources @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/browsercontext-reuse.spec.ts:339 › reuse launch › BC-081 should continue issuing events after closing the reused page @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/browsercontext-reuse.spec.ts:361 › reuse launch › BC-082 should work with routeWebSocket @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/browsercontext-reuse.spec.ts:117 › reuse connect › BC-074 should reset serviceworker @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/browsercontext-reuse.spec.ts:155 › reuse connect › BC-075 should reset serviceworker that hangs in importScripts @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/browsercontext-storage-state.spec.ts:203 › ST-005 should round-trip OPFS @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/defaultbrowsercontext-1.spec.ts:48 › DC-002 context.addCookies() should work @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/defaultbrowsercontext-1.spec.ts:70 › DC-003 context.clearCookies() should work @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/defaultbrowsercontext-1.spec.ts:101 › DC-006 should support userAgent option @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:30 › DC-013 should work in persistent context @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:42 › DC-015 should support reducedMotion option @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:70 › DC-020 should support geolocation and permissions options @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/download.spec.ts:377 › download event › DO-021 should delete downloads on browser gone @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/downloads-path.spec.ts:91 › downloads path › DO-043 should accept downloads in persistent context @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/downloads-path.spec.ts:105 › downloads path › DO-044 should delete downloads when persistent context closes @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/emulation-focus.spec.ts:218 › EM-024 should trigger hover state concurrently @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/firefox/launcher.spec.ts:22 › FF-001 should pass firefox user preferences @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/firefox/launcher.spec.ts:36 › FF-002 should pass firefox user preferences in persistent @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/har.spec.ts:80 › HA-004 should have pages in persistent context @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/har.spec.ts:109 › HA-006 should populate entry startedDateTime from the browser @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/inspector/cli-codegen-java.spec.ts:125 › CG-149 should print a valid basic program in junit @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/inspector/cli-codegen-javascript.spec.ts:51 › CG-107 should work with --http-credentials @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/inspector/cli-codegen-pick-locator.spec.ts:21 › CG-103 should inspect locator @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/inspector/cli-codegen-pick-locator.spec.ts:33 › CG-104 should update locator highlight @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/locator-highlight.spec.ts:89 › LO-196 highlight should survive navigation @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/multiclient.spec.ts:446 › MC-020 should run locator handlers after reconnecting with a pending callback @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/page-clock.spec.ts:329 › popup › CK-029 should run time before popup @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/page-clock.spec.ts:345 › popup › CK-030 should not run time before popup on pause @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/page-event-crash.spec.ts:102 › ER-020 should be able to close page after crash @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/playwright-client.spec.ts:26 › PC-001 should connect from a page and drive the same browser @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/role-utils.spec.ts:38 › AR-143 wpt accname #0 @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/role-utils.spec.ts:38 › AR-143 wpt accname #1 @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/role-utils.spec.ts:38 › AR-143 wpt accname #2 @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/role-utils.spec.ts:38 › AR-143 wpt accname #3 @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/selector-generator.spec.ts:280 › selector generator › GN-056 should not use generated id @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/tracing.spec.ts:304 › TR-013 should respect tracesDir and name @firefox-playwright-x64-server2025-64-core
❌ [firefox-library] › library/tracing.spec.ts:356 › TR-014 should not include trace resources from the previous chunks @firefox-playwright-x64-server2025-64-core
❌ [firefox-page] › page/elementhandle-bounding-box.spec.ts:30 › EH-002 should handle nested frames @firefox-playwright-x64-server2025-64-core
❌ [firefox-page] › page/elementhandle-click.spec.ts:27 › EH-011 should work with Node removed @firefox-playwright-x64-server2025-64-core
❌ [webkit-library] › library/tracing.spec.ts:565 › TR-023 should produce screencast frames fit @webkit-playwright-x64-server2025-64-core
❌ [webkit-library] › library/tracing.spec.ts:565 › TR-023 should produce screencast frames scale @webkit-playwright-x64-server2025-64-core
❌ [android-page] › page/page-event-pageerror.spec.ts:20 › ER-001 should fire
❌ [android-page] › page/selectors-get-by.spec.ts:117 › SE-082 getByLabel should work with aria-labelledby
❌ [android-page] › page/selectors-get-by.spec.ts:122 › SE-083 getByLabel should prioritize aria-labelledby over native label
❌ [android-page] › page/selectors-get-by.spec.ts:140 › SE-086 getByLabel should prioritize aria-labelledby over aria-label
❌ [android-page] › page/selectors-role.spec.ts:436 › SE-060 should support name

101 flaky ⚠️ [chromium-library] › library/chromium/connect-over-cdp.spec.ts:202 › CD-007 should write traces to provided artifactsDir on connectOverCDP `@chromium-playwright-x64-server2025-64-core`
⚠️ [chromium-library] › library/video.spec.ts:762 › screencast › VI-030 should work with video+trace `@chromium-playwright-x64-server2025-64-core`
⚠️ [chromium-page] › page/page-screenshot.spec.ts:534 › page screenshot › SS-039 should take fullPage screenshots during navigation `@tracing-chromium`
⚠️ [chromium-library] › library/chromium/connect-over-cdp.spec.ts:239 › CD-008 should connectOverCDP and manage downloads in default context `@msedge-dev-playwright-x64-server2025-64-core`
⚠️ [chromium-library] › library/chromium/connect-over-cdp.spec.ts:333 › CD-011 should connect to existing service workers `@msedge-dev-playwright-x64-server2025-64-core`
⚠️ [chromium-library] › library/downloads-path.spec.ts:105 › downloads path › DO-044 should delete downloads when persistent context closes `@msedge-dev-playwright-x64-server2025-64-core`
⚠️ [chromium-library] › library/multiclient.spec.ts:409 › MC-018 should stop tracing upon disconnect `@msedge-dev-playwright-x64-server2025-64-core`
⚠️ [chromium-library] › library/beforeunload.spec.ts:20 › BU-001 should close browser with beforeunload page `@chromium-macos-15-large`
⚠️ [chromium-library] › library/browsercontext-basic.spec.ts:36 › BC-002 should be able to click across browser contexts `@chromium-macos-15-large`
⚠️ [chromium-library] › library/tracing.spec.ts:565 › TR-023 should produce screencast frames crop `@chromium-macos-15-large`
⚠️ [chromium-library] › library/chromium/connect-over-cdp.spec.ts:117 › CD-004 should connect when an existing page has been discarded `@chromium-macos-15-xlarge`
⚠️ [chromium-library] › library/chromium/connect-over-cdp.spec.ts:45 › CD-002 should connect when an existing page has no renderer `@msedge-playwright-x64-server2025-64-core`
⚠️ [chromium-library] › library/chromium/connect-over-cdp.spec.ts:117 › CD-004 should connect when an existing page has been discarded `@msedge-playwright-x64-server2025-64-core`
⚠️ [chromium-library] › library/chromium/connect-over-cdp.spec.ts:202 › CD-007 should write traces to provided artifactsDir on connectOverCDP `@msedge-playwright-x64-server2025-64-core`
⚠️ [chromium-library] › library/chromium/extensions.spec.ts:168 › MV3 › CR-041 should use custom userAgent in service worker fetch requests `@msedge-playwright-x64-server2025-64-core`
⚠️ [chromium-library] › library/downloads-path.spec.ts:91 › downloads path › DO-043 should accept downloads in persistent context `@msedge-playwright-x64-server2025-64-core`
⚠️ [chromium-library] › library/inspector/cli-codegen-python-async.spec.ts:80 › CG-132 should save the codegen output to a file if specified `@msedge-playwright-x64-server2025-64-core`
⚠️ [chromium-library] › library/webmcp.spec.ts:256 › WM-014 should list and call tools in cross-origin and nested frames `@chromium-ubuntu-22.04-arm-node22`
⚠️ [firefox-library] › library/inspector/cli-codegen-csharp.spec.ts:202 › CG-157 should not print context options method override in xunit if no options were passed `@firefox-macos-15-large`
⚠️ [firefox-library] › library/inspector/cli-codegen-python-async.spec.ts:80 › CG-132 should save the codegen output to a file if specified `@firefox-macos-15-large`
⚠️ [firefox-library] › library/inspector/cli-codegen-python.spec.ts:76 › CG-124 should save the codegen output to a file if specified `@firefox-macos-15-large`
⚠️ [firefox-library] › library/trace-viewer.spec.ts:1937 › TV-087 should not leak recorders `@firefox-macos-15-large`
⚠️ [firefox-library] › library/beforeunload.spec.ts:20 › BU-001 should close browser with beforeunload page `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/beforeunload.spec.ts:107 › BU-008 should not stall on click when dismissing beforeunload `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/beforeunload.spec.ts:130 › BU-009 should support dismissing the dialog multiple times `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/beforeunload.spec.ts:170 › BU-011 should support closing the page via a subsequent onbeforeunload dialog `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/beforeunload.spec.ts:188 › BU-012 does not get stalled by beforeUnload `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:565 › CO-061 should(not) block third party cookies - ephemeral context `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/browsercontext-storage-state.spec.ts:281 › ST-006 should capture cookies `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/channels.spec.ts:258 › CN-006 exposeFunction should not leak `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/client-certificates.spec.ts:513 › browser › CC-025 should respect launch proxy and proxy bypass `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/client-certificates.spec.ts:963 › browser › persistentContext › CC-041 should pass with matching certificates `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/defaultbrowsercontext-1.spec.ts:28 › DC-001 context.cookies() should work Radoslav Kirilov (@smoke) `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/defaultbrowsercontext-1.spec.ts:96 › DC-005 should support deviceScaleFactor option `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/defaultbrowsercontext-1.spec.ts:119 › DC-008 should support javascriptEnabled option `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/defaultbrowsercontext-1.spec.ts:137 › DC-010 should support offline option `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/defaultbrowsercontext-1.spec.ts:143 › DC-011 should support acceptDownloads option `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:24 › DC-012 should support hasTouch option `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:36 › DC-014 should support colorScheme option `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:48 › DC-016 should support forcedColors option `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:54 › DC-017 should support contrast option `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:60 › DC-018 should support timezoneId option `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/defaultbrowsercontext-2.spec.ts:79 › DC-021 should support ignoreHTTPSErrors option `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/download.spec.ts:399 › download event › DO-022 should save downloads to artifactsDir `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/download.spec.ts:476 › download event › DO-025 should throw if browser dies `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/downloads-path.spec.ts:63 › downloads path › DO-041 should report downloads in downloadsPath folder `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/downloads-path.spec.ts:77 › downloads path › DO-042 should report downloads in downloadsPath folder with a relative path `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/firefox/launcher.spec.ts:48 › FF-003 should support custom firefox policies `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/har.spec.ts:184 › HA-011 should include binary postData `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/har.spec.ts:198 › HA-012 should include form params `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/har.spec.ts:214 › HA-013 should include cookies `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/har.spec.ts:276 › HA-017 should skip invalid Expires `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/har.spec.ts:306 › HA-019 should include secure set-cookies `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/inspector/cli-codegen-3.spec.ts:496 › cli codegen › CG-073 should generate getByTestId `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/inspector/cli-codegen-csharp.spec.ts:153 › CG-154 should print load/save storageState `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/inspector/cli-codegen-csharp.spec.ts:202 › CG-157 should not print context options method override in nunit if no options were passed `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/inspector/cli-codegen-csharp.spec.ts:208 › CG-158 should print context options method override in mstest if options were passed `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/inspector/cli-codegen-csharp.spec.ts:202 › CG-157 should not print context options method override in xunit if no options were passed `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/inspector/cli-codegen-csharp.spec.ts:271 › CG-162 should print a valid basic program in nunit `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/inspector/cli-codegen-javascript.spec.ts:122 › CG-111 should print load/save storageState `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/inspector/cli-codegen-test.spec.ts:20 › CG-112 should print the correct imports and context options `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/inspector/title.spec.ts:19 › IN-056 should reflect formatted URL of the page `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/inspector/title.spec.ts:35 › IN-057 should update primary page URL when original primary closes `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/inspector/title.spec.ts:61 › IN-058 should render primary language `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/locator-highlight.spec.ts:21 › LO-193 highlight should accept a CSS string style `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/locator-highlight.spec.ts:45 › LO-194 highlight should accept an object style (JS only) `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/locator-highlight.spec.ts:74 › LO-195 hideHighlight removes a styled highlight `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/locator-highlight.spec.ts:122 › LO-198 highlight should work with a custom selector engine that runs in the main world `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/locator-highlight.spec.ts:155 › LO-199 highlight should resolve relative to the frame of the locator `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/page-clock.spec.ts:300 › popup › CK-027 should tick after popup `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/page-clock.spec.ts:382 › setFixedTime › CK-033 fixed time is not affected by clock manipulation `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/page-close.spec.ts:40 › PA-041 addLocatorHandler should throw when page closes `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/permissions.spec.ts:220 › PE-015 should isolate the headless clipboard from the operating system `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/permissions.spec.ts:296 › PE-018 local network request is allowed from public origin `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/popup.spec.ts:20 › PO-014 should inherit user agent from browser context Radoslav Kirilov (@smoke) `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/popup.spec.ts:42 › PO-015 should respect routes from browser context `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/proxy.spec.ts:91 › should proxy local network requests › with other bypasses › PR-005 localhost `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/selectors-register.spec.ts:29 › SE-160 should work `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/signals.spec.ts:25 › SI-001 should close the browser when the node process closes `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/slowmo.spec.ts:54 › slowMo › SL-001 Page SlowMo check `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/tracing.spec.ts:404 › TR-015 should overwrite existing file `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/video.spec.ts:390 › screencast › VI-015 should scale frames down to the requested size `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/web-socket.spec.ts:151 › WS-008 should not have stray error events `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/web-socket.spec.ts:247 › WS-013 should send extra HTTP headers on WebSocket handshake `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/webmcp.spec.ts:140 › WM-007 should time out when the tool does not settle `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-page] › page/elementhandle-click.spec.ts:35 › EH-012 should work for Shadow DOM v1 `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-page] › page/elementhandle-content-frame.spec.ts:29 › EH-020 should work for cross-process iframes `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-page] › page/elementhandle-convenience.spec.ts:20 › EH-024 should have a nice preview `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-page] › page/elementhandle-convenience.spec.ts:33 › EH-025 should have a nice preview for non-ascii attributes/children `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-page] › page/elementhandle-eval-on-selector.spec.ts:27 › EH-040 should retrieve content from subtree `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-page] › page/elementhandle-eval-on-selector.spec.ts:35 › EH-041 should throw in case of missing selector `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-page] › page/elementhandle-owner-frame.spec.ts:21 › EH-055 should work `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-page] › page/elementhandle-wait-for-element-state.spec.ts:23 › EH-112 should wait for visible `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-page] › page/expect-boolean.spec.ts:434 › toBeHidden › EX-055 eventually with not `@firefox-playwright-x64-server2025-64-core`
⚠️ [firefox-library] › library/defaultbrowsercontext-1.spec.ts:89 › DC-004 should support viewport option `@firefox-macos-15-xlarge`
⚠️ [webkit-library] › library/heap.spec.ts:304 › LE-014 cycle handles `@webkit-playwright-x64-server2025-64-core`
⚠️ [webkit-library] › library/role-utils.spec.ts:38 › AR-143 wpt accname #2 `@webkit-playwright-x64-server2025-64-core`
⚠️ [webkit-library] › library/tracing.spec.ts:565 › TR-023 should produce screencast frames crop `@webkit-playwright-x64-server2025-64-core`
⚠️ [webkit-library] › library/trace-viewer.spec.ts:1937 › TV-087 should not leak recorders `@webkit-macos-15-xlarge`
⚠️ [webkit-library] › library/capabilities.spec.ts:210 › CA-017 make sure that XMLHttpRequest upload events are emitted correctly `@webkit-macos-15-large`
⚠️ [webkit-library] › library/trace-viewer.spec.ts:1937 › TV-087 should not leak recorders `@webkit-macos-15-large`

126676 passed, 4074 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

🟡 Two Firefox popup failures remain uncertain

Hi, I'm the Playwright bot. I checked the final CI reports, the PR changes, and recent test history.

Failure groups and evidence

For head 0736113 (tested merge b1e38f6), 49 of 51 failed tests and the separate worker-teardown error have independent pre-existing failure evidence. No failure is confirmed caused by this PR, but the two remaining popup failures prevent a green verdict.

This covers the latest merged reports for tests 1 and tests 2. I refreshed the CI results database and matched tests by project, file, full title, and bot. Cancelled jobs and tests that passed on retry are not counted as failed tests.

Uncertain — Windows Firefox popup creation

In the Windows Firefox job, these [firefox-library] tests time out on all four attempts:

  • library/page-clock.spec.ts:329 — popup › CK-029 should run time before popup.
  • library/page-clock.spec.ts:345 — popup › CK-030 should not run time before popup on pause.

Both stop at window.open() or waiting for the popup event, not a clock-value assertion. Each passed all 36 prior runs on this Windows bot, including the unchanged base, and neither has an independently failed attempt elsewhere in the retained Firefox history.

The Firefox r1564 → r1567 roll changes page-target/remoteness handling and isolated execution contexts, so a regression remains plausible; the logs do not identify a responsible change. A comparison of r1564 and r1567 on the same Windows runner is needed. There is not enough evidence to call these flakes.

Pre-existing failures / infra

  • macOS Firefox — CG-100, library/inspector/cli-codegen-aria.spec.ts:87, should update aria snapshot highlight: the same missing x-pw-highlight error occurs on main, b8336b3. On this bot, it failed finally in 12 of 67 prior runs; another 24 passed only after retry. This is a confirmed pre-existing flake despite the PR reaching Firefox's execution and scrolling code.

  • Windows Firefox — the other 40 failed tests: every test has an independently failed attempt on the same bot. The runs below cover all 40, including the download tests that exercise the changed download manager. Some are persistent failures on main, not intermittent flakes.

Independent main run Matching current failures
a169686, unchanged base 28: 26 failed finally; CG-107 and MC-020 passed after retry. Includes DO-043/044.
c409f8c Six: DO-021, CG-103, CG-104, CG-149, TR-014, EH-011.
009f887 Three: AR-143 accname #1–#3.
0a626ef AR-143 accname #0.
3558cd5 LO-196, including the same missing-highlight assertion.
b8f2857 GN-056, failed before passing on retry.
  • Windows Firefox — fatal worker error: Worker teardown timeout of 30000ms exceeded after [firefox-library] › library/permissions.spec.ts:220 › PE-015 should isolate the headless clipboard from the operating system. The same worker error after the same test occurs on main, e14be57. It is not new to this roll.

  • Windows WebKit — two screencast tests: [webkit-library] › library/tracing.spec.ts:565 › TR-023 should produce screencast frames fit / scale have the same page.evaluate timeouts on main, c409f8c. WebKit and shared tracing code are unchanged.

  • Android — five tests: [android-page] ER-001 in page/page-event-pageerror.spec.ts, SE-082/083/086 in page/selectors-get-by.spec.ts, and SE-060 in page/selectors-role.spec.ts all fail on the unchanged base, a169686, with the same stack-format, label-resolution, and accessible-name errors. These paths are outside the Firefox-only changes.

  • macOS installation — one test: [installation tests] › playwright-packages-install-behavior.spec.ts:40 › @playwright/browser-chromium should work fails with getaddrinfo ENOTFOUND cdn.playwright.dev. The exact test has the same DNS error on main, 2fd57cb. Chromium downloads and installer code are unchanged here.

Triaged by the Playwright bot - agent run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant