Skip to content

fix: drain unobserved response body streams - #210

Closed
PekingSpades wants to merge 1 commit into
httptoolkit:mainfrom
PekingSpades:fix/unobserved-response-buffering
Closed

PekingSpades wants to merge 1 commit into
httptoolkit:mainfrom
PekingSpades:fix/unobserved-response-buffering

Conversation

@PekingSpades

Copy link
Copy Markdown

A streaming response with no complete-response listener can fill the internal tracking PassThrough indefinitely. Its queue is outside the body reader's maxBodySize limit. Drain that stream when no complete response needs capturing, while preserving body-data events and observed response bodies.

Fixes #209.

Complete reproduction: #209. Based on v4.6.3 (10e33d5).

Changes

  • Tell the internal response tracker whether a complete-response listener needs buffering.
  • Drain unobserved tracking streams.
  • Cover active streams, complete-body capture, truncation and body-data-only subscriptions.

Actual before/after comparison

The report's loopback reproduction sends 16 MiB with recordTraffic: false and maxBodySize: 1024, consumes all bytes at the client, and holds the response open for measurement.

Measurement Published 4.6.3 This branch
Client received bytes 16,777,216 16,777,216
SHA-256 matches sent content true true
Tracking readable queue 65,536 bytes 0 bytes
Tracking writable queue 16,711,830 bytes 0 bytes
New regression suite 3 passing, 1 failing 4 passing

Queue sizes depend on TCP chunking and can overlap; they are individual queue counters, not an exact summed RSS measurement.

Using the script from the report and a built checkout at ../mockttp:

node response-buffering.cjs
node response-buffering.cjs ../mockttp

Validation

  • npm run build: passed.
  • Browser suite: 340 passing.
  • Full Node suite: 828 passing, 6 pending, 2 failing.
  • Unmodified baseline full Node suite: 823 passing, 6 pending, 3 failing.

The full run failed in two existing live-network cases: skips the server for non-matching HTTPS requests (TLS disconnect, also seen on the baseline) and forwards to the location even if the port & protocol is implicit (the public HTTP fixture returned a 502 AggregateError). A separate repeated probe reproduced the same HTTP fixture failure on both the baseline and this branch: upstream connect attempts reported ETIMEDOUT and ENETUNREACH. These failures remain visible in the reported totals.

All validation used Node 24.19.0, with freshly installed upstream dependencies. Browser checks used Chrome Headless 151.0.7922.34. The full Node suite used a local pkimetal service and a 30-second Mocha timeout. Existing live-network outcomes vary between runs; the deterministic regression and loopback comparisons isolate these changes.

Consume response tracking streams when no complete response listener
needs buffering, preventing active responses from growing an unused
queue outside the configured body size limit.
@CLAassistant

CLAassistant commented Sep 15, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@pimterry pimterry closed this Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unobserved streaming responses grow an internal queue despite maxBodySize

3 participants