Retry an empty model response instead of ending the turn (#185820) - #70
Merged
Merged
Conversation
…n (#185820) A user hit "iris-ai (iris) returned no output (finish reason: unknown). Every upstream attempt failed", typed "continue", and the SAME request succeeded on the first try. The loop should have done that. processor.ts built the error with isRetryable: false, so SessionRetry — which already has backoff, Retry-After handling and a sleep that respects abort — never ran. The comment justified it: "since the failover work (#178556), [the proxy] only lets a stream finish empty once EVERY provider has failed. So by the time a user sees this, try another model is genuinely the right next step." MEASURED THE SAME HOUR, that premise did not hold: `models:smoke --model=iris/iris-ai` passed sync+stream+tools+turn2 on fl-iris-api, the spare OpenCode Go licence had 80% headroom, and only the primary was capped (weekly 100%, resets 09-21). So an empty finish happens while a retry would succeed — which "continue" proved by hand. Retrying is safe because of what defines this branch: zero output parts and zero output tokens, so there is nothing partial to duplicate. It is bounded (EMPTY_FINALIZATION_MAX_RETRIES = 2), uses the existing backoff, and never retries an aborted turn. `finish` is reset before going round again, else a stream that answers without a finish-step would keep the discarded attempt's "unknown". When the retries are exhausted the error still surfaces, now saying how many attempts were made. The decision lives in retry.ts as two pure predicates so it is testable without driving a session. TESTS — 11 new, all mutation-verified. retry.test.ts (unit): the shape, the cap, aborted turns, and that ANY output disqualifies it (the property the safety of this rests on). empty-finalization.test.ts (integration, new): mocks LLM.stream and drives the real processor — empty-then-answer retries and keeps the answer; always-empty stops after the cap with the error and attempt count; a retry that answers without a finish-step does not inherit the discarded attempt's finish. Mutations, each caught: never retry (the old behaviour) 2 fail · no attempt increment 1 fail · forget the finish reset 1 fail · retry on partial output 1 fail · retry an aborted turn 1 fail · unbounded 2 fail · treat a normal "stop" as empty 1 fail. The finish-reset mutation SURVIVED the first version of these tests. The case it protects — a retry that answers without a finish-step — was not covered until it was added. test/session: 52 pass (41 on base), 0 fail. opencode typecheck clean. NOTE: this is the CLI lineage (main). iris/1.18.23 does not carry this code path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JwiqN8M84qimu1TjZGSsdQ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A user hit
iris-ai (iris) returned no output (finish reason: unknown). Every upstream attempt failed, typed continue, and the same request succeeded first try. The loop should have done that.processor.tsbuilt the error withisRetryable: false, soSessionRetry— which already has backoff,Retry-Afterhandling and an abort-aware sleep — never ran. The comment justified it: a stream "only lets a stream finish empty once EVERY provider has failed".Measured the same hour, that premise did not hold:
models:smoke --model=iris/iris-aipassed sync+stream+tools+turn2, the spare OpenCode Go licence had 80% headroom, and only the primary was capped (weekly 100%, resets 09-21). See #185818.The change
finishbefore retrying, else a retry that answers without afinish-stepinherits the discarded attempt’sunknown.retry.tsas pure predicates.Tests — 11 new, mutation-verified
Unit (
retry.test.ts) plus a new integration test that mocksLLM.streamand drives the real processor: empty→answer retries and keeps the answer; always-empty stops at the cap with the error; a retry without afinish-stepdoes not inherit the stale finish.Mutations each caught: never retry (old behaviour) · no increment · forget the finish reset · retry on partial output · retry when aborted · unbounded · treat
stopas empty.The finish-reset mutation survived my first version of the tests — the case it protects was not covered until I added it.
test/session: 52 pass (41 on base), 0 fail. Typecheck clean. CI note: this repo’s Blacksmith runners do not execute tests (#185791), so local is the gate.🤖 Generated with Claude Code
https://claude.ai/code/session_01JwiqN8M84qimu1TjZGSsdQ