Skip to content

fix(parts): one throttled steward call moves the whole part on (no retry, no cooldown) #1447

Closed

Description

@HugoXK

Written-by: FlyBot Lead

A single transient failure of the context steward moves the whole part on. The steward is asked once, has no retry and no cooldown, and any exception is turned into "switch" — the most expensive branch. On the running instance a per-minute request limit did exactly this to two Sessions in one burst.

What happens

Live evidence (dsh-web.log, two lines from the same burst):

the context steward failed … moving on to a fresh part: 429 {Limit type: requests. Current limit: 50, Remaining: 0}

Stack: askSteward (parts.js:520) → decide (:555) → checkContext (:1077). That 429 is a per-minute request limit that clears in seconds — not the budget.

Code (live main 8f5c9ca7f4e094016b8176ce64e496f18dfd):

  • decide (core/src/host/parts.js:552-562) wraps the call in try/catch, logs at :558-560, and falls through to return 'switch' at :561. One attempt, no retry, transient and permanent treated the same.
  • checkContext (:1068-1086) calls it at :1077 while the part is only past the check line, not the hard line (const hard = tokens >= lines.hard, :1073). A non-compact answer falls into startSwitch (:1082).
  • So instead of condensing in place — the part was still usable — the Session gets a new part, the recent and earlier lines are sent again, and one more writeNote spends the same throttled key.
  • The rest of this file already does it properly for notes: NOTE_ATTEMPTS = 2 (:149) with a retry loop (:409-443) and NOTE_COOLDOWN_MS = [60s, 300s, 900s] (:152, used at :845), and below the hard line it refuses to move the part when the note cannot be written. Only the decision that picks the branch has neither.
  • Side effect: handoffPackage (:638-645) has no branch for this, so a forced switch is recorded as the previous part grew long (:645).

What it should do

  1. A transient failure — a 429 carrying throttling_error/retryAfter, a 5xx, a timeout — is retried, 3 attempts in all, with a short backoff that stays under a few seconds.
  2. If the retries run out while the part is below the hard line, this round decides nothing: the part is not switched, and the steward is not asked again for 60s, then 300s. A Session that is being throttled costs time, not a part.
  3. At or over the hard line the part is still moved on (the window must be respected), and the real reason reaches the handoff package and the segment instead of the previous part grew long.
  4. A non-transient answer — a 4xx refusal, an answer that names no action — keeps today's behaviour: move on.

Scope

  • Only core/src/host/parts.js. NOTE_ATTEMPTS / NOTE_COOLDOWN_MS and the note path are not touched; no new config key, no new delivery channel, no client change.
  • The cooldown is per Session and in memory, like noteFailures next to it.
  • Sits next to #1436 in the same file: whichever lands second is rebased once, mechanically.

Acceptance (deterministic red first)

Three tests, all on the final tip, base-red before the fix:

  1. A steward that throws a throttling_error 429 once and then answers {"action":"compact"} is asked twice; the part is condensed in place, no part is created, nothing is archived.
  2. A steward that is throttled every time below the hard line is asked 3 times, then the round ends with no switch and no new part; the next idle pass does not call it again (cooldown).
  3. At or over the hard line with a permanent 500, the part still moves on, and the handoff package says the steward could not be asked (with the error), not that the previous part grew long.

Activity

  1. added a commit that references this issue on Oct 27, 2026
    8b32c2e
  2. added a commit that references this issue on Oct 27, 2026
    55e1c27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions