Skip to content

fix(expect): preserve screenshot assertion errors without comparison details - #43749

Merged
Pavel Feldman (pavelfeldman) merged 2 commits into
mainfrom
triage/43745/40859027899-1
Oct 30, 2026
Merged

Pavel Feldman (pavelfeldman) merged 2 commits into
mainfrom
triage/43745/40859027899-1

Conversation

@microsoft-playwright-automation

@microsoft-playwright-automation microsoft-playwright-automation Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

The driver throws strict-mode errors without screenshot-comparison details. The client incorrectly treated these errors as successful captures, so an ambiguous locator could pass toHaveScreenshot().

Rethrow errors without a custom screenshot error message. Extend the existing locator screenshot test to check strictness against an existing baseline in update and regular runs.

Verified: build and npm run flint passed; 71 screenshot assertion tests and 63 nearby snapshot tests passed. The two-button check passed on Chromium, Firefox, and WebKit in update and regular modes.

Regressed by #40801
Fixes: #43745

Prepared by the triage workflow.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

});

test('should support omitBackground option for locator', async ({ runInlineTest }, testInfo) => {
test('should support omitBackground option for locator', { annotation: { type: 'issue', description: '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/microsoft/playwright/issues/43745' } }, async ({ runInlineTest }, testInfo) => {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make a new test for this.

if (!(e instanceof PlaywrightError))
throw e;
const details = e.details as channels.PageExpectScreenshotErrorDetails;
if (!details.customErrorMessage)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think was regressed fairly recently. Let's find the culprit, and mention in the PR description.

@github-actions

Copy link
Copy Markdown
Contributor

Hi, I'm the Playwright bot. I started to address the review comments on Linux: agent run

#43749 (comment): Move the strict-mode regression into its own issue-tagged test and restore the omitBackground test, as requested.
#43749 (comment): Identify 7cc4a85 (#40801) as the culprit: the new client catch converts strict-mode errors without comparison details into successful results. This read-only job cannot edit the PR description; record the suggested replacement in the run summary.
@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

5194 passed, 616 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

23597 passed, 707 skipped


Merge workflow run.

@dgozman
Dmitry Gozman (dgozman) marked this pull request as ready for review October 30, 2026 12:04
@pavelfeldman
Pavel Feldman (pavelfeldman) merged commit 565f39b into main Oct 30, 2026
24 checks passed
@pavelfeldman
Pavel Feldman (pavelfeldman) deleted the triage/43745/40859027899-1 branch October 30, 2026 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: toHaveScreenshot() silently passes when a locator matches multiple elements

2 participants