Skip to content

docs: sync TODO + deploy README to post-sweep reality - #186

Merged
zzylol merged 1 commit into
mainfrom
docs/sync-todo-readme-post-n10-sweep
Apr 23, 2026
Merged

zzylol merged 1 commit into
mainfrom
docs/sync-todo-readme-post-n10-sweep

Conversation

@zzylol

@zzylol zzylol commented Apr 23, 2026

Copy link
Copy Markdown
Contributor

Top-level TODO, deploy/README, deploy/TODO all predated the multi-agent scaffold (#168#185) and no longer matched the code.

Realities the docs now reflect:

  • deploy scaffold (compose base + N∈{1,10,100} overlays, 7 baselines, 5 Dockerfiles, Helm values, sweep driver) is in.
  • PR eval(N=10): scale sweep surfaces systemic producer bottleneck #185 surfaced a system-wide throughput collapse at N=10 (all baselines throttled to ~2k pts/s, agents near-idle) — now called out as the P0 paper blocker instead of "build multi-agent scaffold".
  • Sweep CSV has nan cells for bandwidth / gateway / backend metrics on some baselines, and backend_query_p99_ms is nan everywhere (no query-side driver yet) — captured as the P1 instrumentation gap.
  • Grafana dashboards and Helm templates are still unwritten — surfaced honestly rather than hidden under "done".

No code changes, docs only.

Top-level TODO, deploy/README, deploy/TODO all predated the
multi-agent scaffold (#168#185) and no longer matched the code.

Realities the docs now reflect:

- deploy scaffold (compose base + N∈{1,10,100} overlays, 7
  baselines, 5 Dockerfiles, Helm values, sweep driver) is in.
- PR #185 surfaced a system-wide throughput collapse at N=10
  (all baselines throttled to ~2k pts/s, agents near-idle) —
  now called out as the P0 paper blocker instead of "build
  multi-agent scaffold".
- Sweep CSV has nan cells for bandwidth / gateway / backend
  metrics on some baselines, and backend_query_p99_ms is nan
  everywhere (no query-side driver yet) — captured as the P1
  instrumentation gap.
- Grafana dashboards and Helm templates are still unwritten —
  surfaced honestly rather than hidden under "done".

No code changes, docs only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@zzylol
zzylol merged commit b204a05 into main Apr 23, 2026
@zzylol
zzylol deleted the docs/sync-todo-readme-post-n10-sweep branch April 23, 2026 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant