feat(chat): hand run() a streamText with the managed options already applied - #4884
Merged
Merged
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every
run()had to spreadchat.toStreamTextOptions(), and leaving it out dropped six things with no error: the managed prompt and its cache control, the registry-resolved model, the prompt's sampling config, telemetry, the skill tools, and theprepareStepthat delivers steering, compaction and injected context.Before:
After:
streamTextcomes fromrun's argument and shadows the one imported fromai, so the correct call is now the shorter one and the managed options cannot be lost by omission.chat.toStreamTextOptions()is unchanged and still supported, and is still the only option in a custom agent.What changes when your options collide with the managed ones
Spread order decides the outcome today, and losing is silent:
The managed
streamTextmerges instead.toolsare passed into the helper so skill tools survive, and aprepareStepyou pass runs after the managed one rather than replacing it. Everything else you name is left alone and wins, telemetry included.systemis the exception: it can be set onchat.agent({ system }), throughchat.prompt.set(), or at the call site, but only in one of them. Two at once throws and names the one that already owns it. No shape merges two system values across every supported AI SDK version, since v5 rejects an array of blocks and a structured block carries the provider options that make prompt caching work.chat.headStart and chat.startHeadStart
buildStreamTextOptionssuppliesmessages,stopWhen: stepCountIs(1)andabortSignal. Step 1 belongs to the route handler and step 2 onward to the agent, so re-settingstopWhenafter a spread hands over a stream that has already run past step 1.Before:
After:
Passing
messages,prompt,stopWhenorabortSignalto thatstreamTextis a type error, with a runtime throw behind it for JavaScript callers.toolsis yours to pass. The old shape only warned in prose.Also in here
chat.agent()takessystem,registry,cacheControlandsystemProviderOptions, so a managed prompt's model and its cache breakpoint no longer have to be passed at the call site.ChatStreamTextis exported for typing a loop factored out ofrun.The signature is taken from the AI SDK's own declaration:
The peer range spans
aiv5, v6 and v7, whose options differ.typeofresolves to whichever version is installed, so generics and tool inference are the caller's own and a v8 option needs no change here.Actions.
onActionno longer receivesstreamTextortools: an action is a state edit, and one that returnschat.turn()(added in #4816) is followed byrun(), which already has both. The action docs on this branch describe that model.chat.toStreamTextOptions()now also applieschat.agent'ssystem,registry,cacheControlandsystemProviderOptions, so the spread form is equivalent to thestreamTexthanded torun(), as the docs say; previously an agent's system prompt was silently dropped on that path. Those options are published on every boot, including for ahydrateMessagesagent, which skips the snapshot boot block where they were first set.Verification
Typecheck and the full suite pass on both
ai@6.0.116andai@7.0.66. The option merge is a pure function so the merged object can be asserted directly, which is howexperimental_telemetrybeing dropped was caught: moststreamTextoptions never reach the provider, so a test that observes the model cannot see them.Run end to end against a deployed agent with every
runrewritten to the new form and no spread anywhere: steering, undo across a cold boot, and regenerate all still pass, a caller's ownprepareStepruns while managed steering still fires inside the turn, and consecutive injections arrive one per turn. The handover-owned options are pinned by@ts-expect-errorassertions in a typechecked test rather than only by the runtime throw.