From 7712c3a8a07c613b56ca0cd69cad094cd8f6314c Mon Sep 17 00:00:00 2001 From: codeitlikemiley Date: Mon, 24 Aug 2026 12:02:19 +0800 Subject: [PATCH] docs(11): re-measure the overhead budget, and keep both readings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task 9 of the run brief: run the overhead bench, and if the numbers no longer hold, correct the doc rather than the measurement. The budget holds — establishment p99 99.1µs against 3ms, per-frame p99 2.21µs against 1ms, roughly thirty times the headroom. But every figure is about double what docs/11 recorded on 2026-08-20, and that is worth writing down while the margin is large, because that is the only time it is cheap to. **The doubling is not attributed, because it cannot honestly be.** Two things changed. The establishment path gained real work: `ExactCache` went async with a bounded lookup (M11.10), per-route TTL resolution (M11.11), and the classifier's confidence now rides to the audit (M12.5 groundwork). And the host was not quiet — load average ~4 with another application holding ~68% of a core, where the original figures came from an idle machine. Picking either cause would be inventing one. Both are named; a clean re-measurement on an idle host is what would settle it and has not been done. Both dated readings are kept rather than the new one overwriting the old. One number reads as "this is what it is"; two read as "this is how it moves", which is the more useful thing to know about a budget with this much room. Claude-Session: https://claude.ai/code/session_017kFpYDqvz6sKGSkM4YKaRf --- docs/11-gateway.md | 37 +++++++++++++++++++++++++++++++------ 1 file changed, 31 insertions(+), 6 deletions(-) diff --git a/docs/11-gateway.md b/docs/11-gateway.md index 8b18e82..1a0465d 100644 --- a/docs/11-gateway.md +++ b/docs/11-gateway.md @@ -368,12 +368,37 @@ in-process, not a product surface. The gateway is stateless apart from cache - **M11.6** Exact cache + circuit breakers; p99 overhead budget: <3ms non-streaming, <1ms per stream frame at 100 rps on one core. ✅ *(shipped: `panday_gateway::cache`, `panday_gateway::circuit`, wired in `Gateway::chat`; `tests/cache_and_breakers.rs`, `tests/overhead.rs`.)* **Measured, release build, single-threaded runtime, 1000 back-to-back requests - (harder than the specified 100 rps — no idle time between them):** - establishment p50 21.6µs / p99 52.0µs against the 3ms budget; per stream frame - p50 41ns / p99 1.17µs against the 1ms budget. The benchmark is `#[ignore]`d - because a wall-clock assertion on shared CI hardware fails for reasons - unrelated to the code, and a flaky test gets muted — which is worse than one - that must be run deliberately. + (harder than the specified 100 rps — no idle time between them).** The + benchmark is `#[ignore]`d because a wall-clock assertion on shared CI hardware + fails for reasons unrelated to the code, and a flaky test gets muted — which is + worse than one that must be run deliberately. + + | | 2026-08-20 | 2026-08-24 | budget | + |---|---|---|---| + | establishment p50 | 21.6µs | 49.6µs | — | + | establishment p99 | 52.0µs | 99.1µs | 3ms | + | per stream frame p50 | 41ns | 167ns | — | + | per stream frame p99 | 1.17µs | 2.21µs | 1ms | + + **The budget holds with roughly thirty times the headroom, and the figures + doubled.** Both are true and the second is not dismissed by the first: a number + that moves 2x is worth noticing while the margin is large, because that is the + only time it is cheap to. + + **What the doubling cannot be attributed to, honestly.** Two things changed + between the runs and this measurement cannot separate them. The gateway gained + real work on the establishment path — `ExactCache` became async with a bounded + lookup (M11.10), per-route TTL resolution (M11.11), and the classifier's + confidence now rides to the audit (M12.5 groundwork). And the host was not + quiet: load average ~4 with another application at ~68% of a core, where the + 2026-08-20 figures came from an idle machine. Attributing the change to either + would be inventing a cause, so both are named and neither is chosen. A clean + re-measurement on an idle host is what would settle it, and it has not been + done. + + The older figures are kept rather than overwritten. A single number in a doc + reads as "this is what it is"; two dated ones read as "this is how it moves", + which is the more useful thing to know about a budget with this much room. **The exact cache is not Postgres yet.** The spec names a PG unlogged table and **no milestone owns building one** — this previously cited "M3.5", which is