Skip to content

WebRTC: give up what a peer never forwards (the loss test's stall) - #348

Closed
NoahSurprenant wants to merge 2 commits into
OpenCommunityEdition:mainfrom
NoahSurprenant:webrtc-forward-tsn-gaps
Closed

NoahSurprenant wants to merge 2 commits into
OpenCommunityEdition:mainfrom
NoahSurprenant:webrtc-forward-tsn-gaps

Conversation

@NoahSurprenant

@NoahSurprenant NoahSurprenant commented Oct 14, 2026 •

Copy link
Copy Markdown

port/web/tests/webrtc_native.mjs's "with losses" case fails now and then: the data channel stays open but nothing more is echoed after about a hundred messages (e.g. firefox with losses: {"sent":960,"echoed":72,"wrong":0,"lastEchoed":100,"open":true}, and the same with Chromium on a loaded runner).

Cause (found with packet traces of the native end): when the first DATA chunk a browser sends is lost, Firefox's SCTP (usrsctp) sends one FORWARD-TSN for it and never another, so the next loss leaves the native end's cumulative TSN stuck below it for good. The SACKs' gaps then soon pass MAXIMUM_GAP_BLOCKS, what they leave out stays counted in the browser's flight, its congestion window fills and it stops sending. Dropping the browser's first DATA on purpose stalled 6 of 6 runs (dropping a later one, 0 of 6), so the test fails at about its loss rate (10%), more under load. A real player on a lossy network hits it the same way.

Fix (port/linux/src/p2p_webrtc.c only): the channel never sends anything twice (negotiated, maxRetransmits: 0), so whatever is still missing below a TSN that arrived GAP_TIME (250 ms) ago will never come. The native end now gives it up itself, as the peer's FORWARD-TSN would have, and SACKs, instead of depending on the peer's FORWARD-TSN being sent and arriving.

Checked (the loss case of webrtc_native.mjs, Playwright 1.58):

before after
Firefox, two runs at a time 9 of 40 failed 0 of 40
Chromium 0 of 50 0 of 40
Firefox, first DATA dropped on purpose 6 of 6 0 of 8

The commit is cherry-picked (-x) from a fork where it has been in CI since.

Regression test (second commit, port/web/tests/webrtc_native.mjs and webrtc_native.c only): a new case, "the first DATA chunk lost and every tenth", loses those chunks of the browser's and nothing else, every run, so the stall no longer depends on the random losses taking the first one. The harness is linked with -Wl,--wrap=posix_dtls_receive and removes the DATA chunks HARNESS_DROP names from the SCTP packets p2p_webrtc.c reads (recomputing the CRC-32C); the test loses the 1st, 10th, 20th … 300th (31 gaps, more than the 16 a SACK carries) and requires the rest back, to the end. It takes about five seconds a browser and runs in the web job with the other cases (node port/web/tests/webrtc_native.mjs chromium firefox).

the new case, Playwright 1.58 without the fix with it
Firefox failed 10 of 10 (96 of 320 back, none after the 117th) passed 10 of 10 (289 of 320)
Chromium passed 10 of 10 passed 10 of 10

Chromium does not stall this way and passes either way. Losing every tenth chunk but not the first passes in Firefox without the fix too. With the fix, node port/web/tests/webrtc_native.mjs chromium firefox passes; without it, it fails on the new case in Firefox.

🤖 Generated with Claude Code

NoahSurprenant and others added 2 commits October 14, 2026 02:43
The web job's "with losses" case (port/web/tests/webrtc_native.mjs) failed
now and then with the channel open but nothing echoed after about a hundred
messages. The cause, found with packet traces of the native end: when the
first DATA chunk a browser sends is lost, Firefox's SCTP (usrsctp) sends one
FORWARD-TSN for it and never another, so the next loss leaves this end's
cumulative TSN stuck below it for good. The SACKs' gaps then soon pass
MAXIMUM_GAP_BLOCKS, what they leave out stays in the browser's flight, its
congestion window fills and it sends nothing more. Losing the first DATA on
purpose stalled 6 of 6 runs (and a later one 0 of 6), so it fails about as
often as the test's loss rate, 10%; worse under load. A real player on a
lossy network would hit it the same way.

The channel sends nothing twice (negotiated, maxRetransmits 0), so what is
missing below a TSN that arrived GAP_TIME (250 ms) ago will never come: this
end now gives it up as the peer's FORWARD-TSN would have, and SACKs.

The loss case, 40 runs per browser, two at a time: Firefox 9 of 40 failed
before, 0 of 40 after; Chromium 0 of 40 after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 76352bc)
…CommunityEdition#348)

The loss case caught the stall the previous commit fixes only when its
random losses happened to take the browser's first DATA chunk, about one run
in ten. The new case takes it every time: the harness (webrtc_native.c,
linked with -Wl,--wrap=posix_dtls_receive) removes the DATA chunks
HARNESS_DROP names from the SCTP packets p2p_webrtc.c reads, and the test
has it lose the first and every tenth after it (31 gaps, more than the 16 a
SACK carries), and nothing else. The other messages must come back, to the
end.

Checked with Playwright 1.58: without the previous commit the case failed
10 of 10 runs in Firefox (96 of 320 messages back, none after the 117th);
with it, it passed 10 of 10 in Firefox and 10 of 10 in Chromium (289 of
320). Chromium's SCTP does not stall this way and passes either way
(10 of 10 without the fix). Losing every tenth chunk but not the first
passes in Firefox without the fix too. The case takes about five seconds a
browser; node port/web/tests/webrtc_native.mjs chromium firefox passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NoahSurprenant added a commit to NoahSurprenant/OpenCE that referenced this pull request Oct 14, 2026
…CommunityEdition#348) (#39)

The loss case caught the stall the previous commit fixes only when its
random losses happened to take the browser's first DATA chunk, about one run
in ten. The new case takes it every time: the harness (webrtc_native.c,
linked with -Wl,--wrap=posix_dtls_receive) removes the DATA chunks
HARNESS_DROP names from the SCTP packets p2p_webrtc.c reads, and the test
has it lose the first and every tenth after it (31 gaps, more than the 16 a
SACK carries), and nothing else. The other messages must come back, to the
end.

Checked with Playwright 1.58: without the previous commit the case failed
10 of 10 runs in Firefox (96 of 320 messages back, none after the 117th);
with it, it passed 10 of 10 in Firefox and 10 of 10 in Chromium (289 of
320). Chromium's SCTP does not stall this way and passes either way
(10 of 10 without the fix). Losing every tenth chunk but not the first
passes in Firefox without the fix too. The case takes about five seconds a
browser; node port/web/tests/webrtc_native.mjs chromium firefox passes.


(cherry picked from commit b7715ce)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@VerySmuggy VerySmuggy added bug Something isn't working priority: medium Normal priority labels Oct 19, 2026
@cybersecurity

Copy link
Copy Markdown
Collaborator

1b61b52 replaces it, as it did #351. Its fixed-drop test case could come back as a PR with only that test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working priority: medium Normal priority

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants