Skip to content

devops: trim MCP matrix, move Linux tests to large builders, shorten job names - #43389

Merged
Pavel Feldman (pavelfeldman) merged 1 commit into
microsoft:mainfrom
pavelfeldman:mcp-ci-matrix
Oct 14, 2026
Merged

Pavel Feldman (pavelfeldman) merged 1 commit into
microsoft:mainfrom
pavelfeldman:mcp-ci-matrix

Conversation

@pavelfeldman

@pavelfeldman Pavel Feldman (pavelfeldman) commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

Summary

  • MCP workflow now runs ubuntu + (chrome, chromium, firefox, webkit), macos + chrome, windows + (chrome, msedge).
  • tests 1: test_linux moves to playwright-x64-ubuntu24-64-core.
  • tests 1: Windows 64-core test runner uses 16 workers instead of the default 32.
  • Job labels: large builders show as ubuntu-large / windows-large; Node.js version shows only when it is not 22.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@pavelfeldman Pavel Feldman (pavelfeldman) changed the title devops: run fewer MCP CI configurations devops: trim MCP matrix, move Linux tests to 64-core builder Oct 14, 2026
Comment thread .github/workflows/tests_primary.yml Resolved
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@pavelfeldman Pavel Feldman (pavelfeldman) changed the title devops: trim MCP matrix, move Linux tests to 64-core builder devops: trim MCP matrix, move Linux tests to large builders, shorten job names Oct 14, 2026
@pavelfeldman
Pavel Feldman (pavelfeldman) merged commit 36c6078 into microsoft:main Oct 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

1 failed
❌ [firefox] › mcp/cli-session.spec.ts:147 › session stops when temporary socket directory disappears @mcp-ubuntu-latest-firefox

5153 passed, 614 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

1 failed
❌ [webkit-library] › library/proxy.spec.ts:180 › should reconnect with credentials after CONNECT 407 closes the socket @webkit-playwright-x64-ubuntu24-64-core-node22

3 flaky ⚠️ [chromium-library] › library/chromium/connect-to-worker.spec.ts:19 › should connect, evaluate, receive console and disconnect `@chromium-playwright-x64-ubuntu24-64-core-node22`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-playwright-x64-ubuntu24-64-core-node22`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-playwright-x64-ubuntu24-64-core-node22`

21775 passed, 525 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Hi, I'm the Playwright bot and I took a first look at the CI failures.

🟢 Both failures are pre-existing flakes

The same tests fail with the same errors on unrelated SHAs. This PR changes CI workflow configuration only, so it is clear.

Details

Overall assessment

The PR moves the primary Linux jobs to a larger Ubuntu runner and trims the MCP matrix. It does not change either failing test or the product code they exercise. The refreshed test-results database confirms that both exact tests flip independently of this PR.

Pre-existing flake / infra

  • [webkit-library] › library/proxy.spec.ts:180 › should reconnect with credentials after CONNECT 407 closes the socket — received an empty page title instead of Served by https server via proxy. The same assertion failed on base-branch push run 38646436107 (321e5f28) before this PR was merged. It failed in 61 of 308 recent runs and passed in the other 247.
  • [firefox] › mcp/cli-session.spec.ts:147 › session stops when temporary socket directory disappears — timed out after five seconds while waiting for the session to stop. The same test failed with the same timeout on unrelated push runs 38468906913 (65f80bbc) and 38273275511 (6e5419ad). It failed in 4 of 121 recent Ubuntu/Firefox runs and passed in the other 117.

Triaged by the Playwright bot - agent run

@github-actions

Copy link
Copy Markdown
Contributor

Hi, I'm the Playwright bot and I took a first look at the CI failures.

🟢 Both failures are pre-existing flakes

The same tests fail with the same errors on unrelated SHAs. This PR changes CI workflow configuration only, so it is clear.

Details

Overall assessment

The PR moves the primary Linux jobs to a larger Ubuntu runner and trims the MCP matrix. It does not change either failing test or the product code they exercise. The refreshed test-results database confirms that both exact tests flip independently of this PR.

Pre-existing flake / infra

  • [webkit-library] › library/proxy.spec.ts:180 › should reconnect with credentials after CONNECT 407 closes the socket — received an empty page title instead of Served by https server via proxy. The same assertion failed on base-branch push runs 38653432017 (36c6078f) and 38646436107 (321e5f28). Excluding this PR, it failed in 36 of 165 recent runs and passed in the other 129.
  • [firefox] › mcp/cli-session.spec.ts:147 › session stops when temporary socket directory disappears — timed out after five seconds while waiting for the session to stop. The same test failed with the same timeout on unrelated push runs 38468906913 (65f80bbc) and 38273275511 (6e5419ad). Excluding this PR, it failed in 2 of 119 recent runs and passed in the other 117.

Triaged by the Playwright bot - agent run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants