JSON-RPC: bound the per-connection read buffer so unterminated input cannot exhaust memory

[mcfnord]

MY LLM WROTE:

The JSON-RPC server consumes input only on a complete line — while ( pSocket->canReadLine() ) in CRpcServer::OnNewConnection (src/rpcserver.cpp). Until a newline arrives the received bytes stay in the QTcpSocket read buffer, which is unbounded because setReadBufferSize() is never called. A client that connects and sends bytes with no \n grows that buffer 1:1 with bytes sent, before authentication, until the process is killed by the allocator (uncaught std::bad_alloc → SIGABRT, dropping every connected client).

Measured on a non-ASan release build of main, one connection sending 200 MiB with no newline:

build	outcome	server RSS delta
current main	connection held open, buffer grows	+204,920 kB (≈ +200 MiB, 1:1 with input)
this change	connection dropped after buffering ≤ 64 KiB	+112 kB (flat)

Well-formed traffic is unaffected on the patched build: jamulus/apiAuth returns "result":"ok" and jamulus/getVersion returns the version, connection stays open.

Scope: reachable only when the RPC server is enabled (--jsonrpcport) and bound off-loopback (--jsonrpcbindip); the default bind is 127.0.0.1. This is the denial-of-service class SECURITY.md documents as a non-guarantee, so the change hardens a documented limitation rather than closing a promised guarantee — the buffer bounds in a few lines.

The change, all in OnNewConnection:

• setReadBufferSize ( MAX_JSON_RPC_REQUEST_BYTES ) on each new connection, so buffering stops at the bound.
• after the read loop, a connection whose buffer is full with no complete line is answered with a parse error and closed.
• MAX_JSON_RPC_REQUEST_BYTES = 64 KiB — far above any real request (secret, method name, or a large batch), and adjustable.

A regression test fits the fork's JSON-RPC test surface directly: send an unterminated payload and assert the server's RSS stays flat and the connection is dropped, versus a terminated payload parsed normally.

@dtinth — flagging you as the JSON-RPC author and since this sits in the surface the fork is bringing under test.

[pljones]

@mcfnord, pass your LLM this PR and get it to work out why it failed in the build: and work out what to put in the appropriate guidance notes to stop it happening again.

[ann0see]

Service Unavailable

NO need to prompt the LLM. Very likely just an issue on GitHubs side

[mcfnord]

MY LLM WROTE:

The blocking check was the "Verify C-like coding style" job, conclusion cancelled. The job record shows zero steps executed, no stored log, and inverted timestamps (started_at 2026-08-07 06:38 UTC, completed_at 2026-08-06 16:24 UTC) — it never ran.

Re-running it directly needs repo write access, so the PR was closed and reopened to fire a fresh pull_request event on the same commit (c1e6175, nothing pushed). The style check now passes, and the full matrix has re-run green: all five platform builds, all three style checks, and the JSON-RPC docs check. So this was a GitHub-side cancellation, as @ann0see said — nothing in the PR needed fixing.