Passthrough requests retain full bodies with recording and body observers disabled

[PekingSpades]

Problem

A plain passthrough request retains its entire upload as a replay buffer even with recordTraffic: false, no request subscriptions, no body matchers, and no callbacks. With the default body-size setting, a large active upload can retain a full copy that no consumer needs.

This report concerns avoidable memory use; the example's forwarded content is correct. Explicitly setting maxBodySize can limit the request buffer, but ordinary unobserved passthrough can avoid creating it.

Environment

Reproduced with the published mockttp 4.6.3 package and Node 24.19.0. All traffic stays on loopback.

Complete reproduction

1. Install the affected release:

mkdir mockttp-repro
cd mockttp-repro
npm init -y
npm install mockttp@4.6.3

2. Save this as request-buffering.cjs:

const http = require('node:http');
const path = require('node:path');
const { once } = require('node:events');
const { createHash } = require('node:crypto');

const main = process.argv[2]
    ? path.resolve(process.argv[2], 'dist/main.js')
    : require.resolve('mockttp');
const packageDir = path.dirname(path.dirname(main));
const buffers = require(path.join(packageDir, 'dist/util/buffer-utils.js'));
const { getLocal } = require(main);
const originalBuffer = buffers.streamToBuffer;
const captures = [];
buffers.streamToBuffer = (input, ...args) => {
    const result = originalBuffer(input, ...args);
    if (input.method === 'POST') captures.push(result);
    return result;
};

const chunk = Buffer.alloc(64 * 1024, 'x');
const count = 256;
const expectedHash = createHash('sha256');
for (let i = 0; i < count; i++) expectedHash.update(chunk);
const receivedHash = createHash('sha256');
const proxy = getLocal({ recordTraffic: false });
let targetResponse;
let received = 0;
let resolveReceived, rejectReceived;
const receivedAll = new Promise((resolve, reject) => {
    resolveReceived = resolve;
    rejectReceived = reject;
});
receivedAll.catch(() => {});
const target = http.createServer((request, response) => {
    targetResponse = response;
    request.on('data', data => { received += data.length; receivedHash.update(data); });
    request.on('end', resolveReceived);
    request.on('error', rejectReceived);
    // Hold the response until the active request's buffering has been measured.
});

(async () => {
    let client;
    try {
        await new Promise(resolve => target.listen(0, '127.0.0.1', resolve));
        await proxy.start();
        await proxy.forAnyRequest().thenPassThrough();
        const targetUrl = `http://127.0.0.1:${target.address().port}/upload`;
        const finished = new Promise((resolve, reject) => {
            client = http.request({
                method: 'POST', hostname: '127.0.0.1', port: proxy.port,
                path: targetUrl, headers: { Host: new URL(targetUrl).host }, agent: false
            }, response => {
                response.resume();
                response.on('end', resolve);
                response.on('error', reject);
            });
            client.on('error', error => { rejectReceived(error); reject(error); });
            client.setTimeout(15000, () => client.destroy(new Error('request timed out')));
        });
        finished.catch(() => {});
        for (let i = 0; i < count; i++) {
            if (!client.write(chunk)) await once(client, 'drain');
        }
        client.end();
        await receivedAll;
        await new Promise(setImmediate);
        console.log(JSON.stringify({
            receivedBytes: received,
            sha256Matches: receivedHash.digest('hex') === expectedHash.digest('hex'),
            recordTraffic: false,
            retainedRequestChunkBytes: captures.reduce((sum, buffer) => sum +
                buffer.currentChunks.reduce((size, data) => size + data.length, 0), 0)
        }));
        targetResponse.end('ok');
        await finished;
    } finally {
        client?.destroy();
        await proxy.stop();
        target.closeAllConnections();
        await new Promise(resolve => target.close(resolve));
        buffers.streamToBuffer = originalBuffer;
    }
})().catch(error => { console.error(error); process.exitCode = 1; });

3. Run it:

node request-buffering.cjs

Actual output:

{"receivedBytes":16777216,"sha256Matches":true,"recordTraffic":false,"retainedRequestChunkBytes":16777216}

Expected: all 16 MiB reach the target unchanged without a retained replay buffer when there are no consumers.

The target consumes the entire upload and holds its response open for the measurement. The diagnostic wrapper captures the request reader's chunk list; it measures that specific replay buffer, rather than process-wide RSS.

Proposed optimization

Use the original stream when no body consumer requires replay. Keep buffering for traffic recording, request/body-data subscriptions, body matchers, full-request callbacks, and body transforms. Reuse an existing buffer if an earlier matcher or step has already started reading.

Regression coverage should include HTTP/1 and HTTP/2 forwarding, trailers, cancellation, and oversized observed requests. The script also accepts a built checkout directory for an after-fix comparison.