docs: document network resilience, error_code, and prompt bounds - #169
Merged
Merged
Conversation
SkyeAv
added this pull request to stack #170
September 15, 2026 17:40
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
SkyeAv
changed the title
SkyeAv
force-pushed
the
net/docs
branch
2 times, most recently
from
September 15, 2026 17:53
db32246 to
8c54365
Compare
SkyeAv
marked this pull request as ready for review
September 15, 2026 18:07
…unds
Problem
=======
The network-resilience work (US-001..US-006) shipped its mechanisms
without the operator-facing documentation they need: the transient/
permanent classification table, the retry budgets, the download
idempotency/atomicity/parallelism contract, the worst-case wall-clock
arithmetic against the 90-minute per-article budget, the single
retry-layer rule, and the prompt bounds — all undocumented — and
CHANGELOG.md had no ## Unreleased section for the release commit to
rename.
Change
======
- docs/agent.md gains a `## Network resilience` section (after
`## PMC-AWS data source`, before `### Local payloads`) documenting:
the transient-vs-permanent classification table (DNS/connection/
timeout failures, HTTP 408/425/429 and every 5xx, network-errno
OSErrors, and optional-library rate-limit errors matched by NAME or
message token — a `(reset after ...)` hint sets the WAIT, never the
classification); the HTTP retry budget (4 attempts, 1.0 s base
doubling to 20.0 s, 60.0 s total) and the LLM budget (45.0 s per
call); `Retry-After` honoring as max(hint, jittered step) capped at
60 s; download idempotency (non-empty files skipped without a
request, torn zero-length files re-downloaded), atomicity (`.part`
-> `os.replace`, cleaned even on KeyboardInterrupt), and the default
concurrency=8 with submission-order results; the worst-case
wall-clock arithmetic — per round trip 4 x 120 s timeout + 60 s
backoff = 540 s, fetch phase 3 serial calls + ceil(15/8)=2 waves =
5 x 540 s = 2,700 s = 45 min (socket timeouts included; backoff
alone is <= 300 s per fetch_pmc_article's docstring), LLM 29 calls
x 45 s = 1,305 s ~ 21.8 min, total ~67 min of the 90-minute budget
— and the single retry layer (smolagents retry=False, OpenAI
max_retries=0; LiteLLM's internal retry stays outside constructor
control).
- The reflexion/judge prose documents the prompt bounds
(MAX_PROMPT_CONFIG_CHARS / COVERAGE_PROMPT_CHARS /
MAX_PROMPT_CONTEXT_CHARS / MAX_PROMPT_REPORT_CHARS = 8,000 / 8,000 /
40,000 / 8,000, UNRESOLVED_CAP = 20 per list), the visible `+N more`
and `... truncated:` markers with `unresolved_count` preserving the
scale signal, the worst-case prompt (~58,000 chars, about 15,000
tokens), the fleet evidence that motivated it (1.6M / 2.3M-char
reflexion prompts; 689,241 / 524,336 input tokens), and that
compaction happens ONLY at prompt serialization (public reports
unchanged).
- CHANGELOG.md gains the `## Unreleased` section (above
`## 18.0.0 - 2026-09-10`, for the release commit to rename) with
Keep-a-Changelog subsections in Fixed / Added / Performance /
Documentation order, every entry led by a bold sentence: the
`tablassert.net` seam and its BABEL classification deltas, the LLM
retry layer, `error_code`, idempotent atomic parallel downloads, and
the prompt bounds.
- NO version bump and no other surface touched: pyproject.toml,
uv.lock, CITATION.cff, README.md, mkdocs.yml, CONTRIBUTING.md,
docs/index.md, examples/agent/README.md, tests/test_agent_fetch.py,
and tests/test_agent_branches.py are byte-identical (git diff EMPTY
vs HEAD). No new extra, flag, env var, or nav page — the docs guards
are not weakened anywhere.
- Every documented number is sourced: fleet forensic figures from the
recorded run (2,194/2,507 DNS-shaped failures at 87.5%; ~13-18 round
trips per article; 1.6M/2.3M-char and 689,241/524,336-token
reflexion prompts) or worst-case arithmetic from the named constants,
shown as arithmetic. docs/agent.md keeps its plain-symbol style (zero
occurrences of approx/ge/times Unicode); CHANGELOG keeps its existing
forms.
Verification
============
- docs guards: `uv run pytest tests/test_docs_source_of_truth.py
tests/test_docs_cli_coverage.py tests/test_docs_examples.py -q
--no-cov -n 0` -> 162 passed
- `uv run pytest -q` -> 1476 passed, 15 skipped (94% coverage)
- `uv run ruff check .` / `uv run ruff format --check .` /
`uv run pyright` -> clean / 92 files / 0 errors
- `make test-rust` -> ok (14 passed); `cargo clippy --manifest-path
rust/Cargo.toml --all-targets -- -D warnings` -> clean
- `uv run pytest tests/test_net.py tests/test_agent_net.py
tests/test_agent_prompt_bounds.py -q --no-cov -n 0 --durations=10`
-> 54 passed, max 0.17 s (no test >= 1 s: nothing sleeps in real
time)
- `grep -c '^## Unreleased' CHANGELOG.md` -> 1; `grep -n '## Network
resilience' docs/agent.md` -> one heading (line 94, slug
`network-resilience`, no collision; the new checkpoint anchor link
resolves); `grep -n 'net.py' llms.txt` -> the US-001 Implementation
Map bullet
- No `network` pytest marker anywhere in tests/; the only `urlopen` in
tests is the monkeypatched stub in tests/test_agent_net.py
- Independent Tier-2 CODE_REVIEWER review: PASS on all 10 criteria
(no invented numbers; CHANGELOG honesty spot-checked against the
branch-point code) with one should-fix — the table implied a
`(reset after ...)` MESSAGE classifies transient when the code uses
that hint only for the WAIT (`is_transient(RuntimeError("(reset
after 90s)"))` is False) — plus two nits (hint semantics are
`max(hint, jittered step)`, not "instead of"; the docs fetch-phase
figure includes socket timeouts while agent.py's docstring counts
backoff only). All three fixed; the corrected claims re-verified
against live behavior (`retry_transient` with injected sleep/rng:
sleeps [1.0, 2.0, 4.0] under a 1 s hint and [45.0, 15.0] under a
45 s hint truncated to the 60 s budget).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documents the network-resilience layer,
error_code, and the prompt bounds delivered by #163–#168, and adds the## Unreleasedchangelog section for the release commit to rename. No code change, no version bump.Docs (
docs/agent.md)## Network resiliencesection (after## PMC-AWS data source, no anchor collision): the transient/permanent classification table, the HTTP retry budget (4 attempts / 1.0 s base / 20.0 s max / 60.0 s total) and the LLM budget (45.0 s per call),Retry-After+(reset after …)honoring asmax(hint, jittered step)capped at 60 s, idempotent atomic parallel downloads (defaultconcurrency=8,.part→os.replace, submission order,concurrency < 1fails pre-network), the worst-case wall-clock arithmetic against the 90-minute budget (540 s/call; fetch ≤ 2,700 s = 45 min; LLM ≤ 1,305 s ≈ 21.8 min; ≈ 67 min total), and the single-retry-layer rule (smolagentsretry=False, OpenAImax_retries=0).+N more/truncated:markers withunresolved_count, worst case ≈ 58 K chars ≈ 15 K tokens, the fleet evidence (1.6M/2.3M chars; 689,241/524,336 tokens), and that compaction happens only at prompt serialization.Changelog (
CHANGELOG.md)## Unreleasedwith### Fixed/### Added/### Performance/### Documentation, every entry bold-led, covering fix: add shared network resilience seam (tablassert.net) #163–fix: bound reflexion and judge prompts #168. Rebased cleanly on top of upstream's post-18.1.0Unreleased entries (ours first, upstream's### Changedpreserved verbatim).docs/agent.mdkeeps its plain-symbol style (zero≈/≥/×).Testing
uv run pytest tests/test_docs_source_of_truth.py tests/test_docs_cli_coverage.py tests/test_docs_examples.py -q --no-cov -n 0→222 passed, 1 warninguv run pytest -q→1565 passed, 15 skipped;uv run ruff check ./ruff format --check ./uv run pyright→ clean / 92 files /0 errors;make test-rust+cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings→ cleanpyproject.toml,uv.lock,CITATION.cff,README.md,mkdocs.yml,CONTRIBUTING.mdbyte-identical; Tier-2 review passed all 10 criteria (should-fix on(reset after …)hint semantics fixed and re-verified against liveretry_transientbehavior)