feat: inject and steer new guidance into the running turn - #357
mulyawansentosa wants to merge 4 commits into
Conversation
While a turn was running, Enter was rejected with "wait for the current response or press esc to interrupt", so sending a follow-up instruction meant waiting for the whole turn or interrupting the agent. A message queued until the end of the turn would still arrive too late to override instructions the agent had already executed. SessionManager now keeps a small per-session queue of supplemental prompts. Before every LLM call of the running turn the queue is drained: each entry becomes a user message (including images and selected skills) appended to the conversation, so the model reads it together with the work it already did and can revise or supersede the earlier instructions. When a turn would end while guidance is still queued, the loop stays alive for one more iteration instead of deferring the guidance to a later turn. - session: addSupplementaryPrompt / cancelSupplementaryPrompt / listPendingSupplementaryPrompts / countPendingSupplementaryPrompts, the onSupplementaryQueueChanged and onSupplementaryPromptInjected callbacks, and MessageMeta.isSupplementary - session: drain the queue before each request, keep the loop alive while guidance is pending, and drop the queue when a session is cleaned up - cli: prompts typed while busy are handed to the session, the prompt area lists the guidance that is still waiting, Backspace on an empty prompt removes the last entry, and injected guidance is rendered as a labelled block (also in raw mode) - tests: 7 core tests covering the queue semantics and the request payloads of a turn that receives guidance mid-flight, plus 4 Ink tests for the input wiring - docs: README and quickstart key tables Refs lessweb#113, lessweb#117
Guidance injected before the next LLM call still had to wait for the answer that was
already streaming, so a long text answer could not be redirected until it finished on
its own. No API can rewrite output that has already been generated, so the only way to
have the guidance read sooner is to abort that one request and ask again.
The session now tracks the streaming request separately from the turn. When the user
asks to steer, that request is aborted, whatever the model already wrote is kept as an
assistant message marked meta.interrupted, and the loop immediately issues a new
request that carries the guidance. Tools are never interrupted: steering only targets
the model stream, so a running command finishes first and the guidance is injected at
the next request boundary.
Steering is opt-in, so an existing setup keeps behaving exactly as before:
- Ctrl+Enter steers that one message.
- steerMode: "interrupt" makes plain Enter steer as well; the default is "queue".
- session: steerActiveSession(), a per-request steer signal released once the request
settles, and LlmSteeredError carrying the partial answer
- session: a steered answer is not an interruption - the turn continues, the status
stays completed, and the aborted request is not retried
- settings: steerMode ("queue" by default)
- cli: Ctrl+Enter is parsed from its CSI-u sequence, forwarded as
PromptSubmission.steer, and the cut-short answer is labelled in the transcript;
the footer mentions the shortcut
- tests: core covers the aborted stream, the follow-up request payload, the kept
partial answer and the no-op case; the cli covers the key parsing, the steering
submit, and steerMode resolution
- docs: configuration table and section, README and quickstart key tables
Refs lessweb#113, lessweb#117
e694e23 to
96f71c2
Compare
|
Updated after review of the design: steering is now opt-in instead of the previous default.
That keeps the behaviour of an existing setup unchanged (no answer is cut short unless the user asks for it), while still giving the "flowing conversation" the feature is about. The branch was rewritten so the second commit now describes the opt-in behaviour; PR body and the docs were updated to match. |
`handleSubmit` treated every submission as guidance while a turn was running, including the `/exit` command that `PromptInput` deliberately lets through. The CLI then queued "/exit" as a user message and steered the turn instead of quitting. Commands are not prompts, so they take the normal path again: non-exit commands stay blocked while the turn runs, and `/exit` still quits. Found by driving the real TUI in a pseudo-terminal against a stub OpenAI-compatible server. The guidance path itself behaved as designed: the streaming answer was aborted after 21 of 130 chunks, the partial text was kept and marked `meta.interrupted`, and the follow-up request carried it together with the guidance as a user message. Refs lessweb#113, lessweb#117
|
Verified end-to-end after the opt-in change, by driving the real TUI in a pseudo-terminal against a stub OpenAI-compatible server (it streams a 130-chunk answer on demand):
That run also caught a regression introduced in the second commit: |
10 was a conservative guess rather than a measured limit. The queue drains at the next request boundary, so this only caps how many instructions can pile up faster than one step of the turn (a long tool run or a long streaming answer). Raising it to 20 keeps that burst covered while still bounding the cost: every queued prompt travels with each remaining request of the turn, so an unbounded queue could inflate the context (and any attached images) without limit. - the queue is not dropped or reordered; it still drains in FIFO order before every LLM call of the turn - when the limit is reached the prompt is refused but stays in the input box, so nothing the user typed is lost Refs lessweb#113, lessweb#117
Summary
Closes #113. Supersedes / alternative to #117 (the earlier open PR that sketches the same
mechanism with a slightly different design - see the notes at the bottom).
While a turn was running,
Enterwas rejected with "wait for the current response or press escto interrupt", so sending a follow-up instruction meant waiting for the whole turn or
interrupting the agent. This PR makes new instructions flow into the work that is already
running:
before the model's next step of that turn. This is the default behaviour of
Enter.generated, so when the user explicitly asks for it (
Ctrl+Enter, orsteerMode: "interrupt"),the request that is streaming is aborted, its text is kept in the conversation, and the turn
continues with the new instruction.
How it works
SessionManagerkeeps a per-session FIFO queue of supplemental prompts(
addSupplementaryPrompt, max 20). Before every LLM call of the running turn the queue isdrained and each entry becomes a user message with its images and selected skills:
When the user asks to steer (below),
steerActiveSession()aborts that single request:Details:
meta.interrupted, so the model (andthe transcript) still know what it had said when the user changed direction.
npm installfinishes and the guidance is injected at the next request boundary.completed, andthe aborted request is not retried.
iteration instead of deferring the guidance to a later, unrelated turn.
Escstill interrupts the whole turn.ask_user_questionanswer ora permission decision), so it can never be consumed as that answer.
role: "user"andmeta.isSupplementary.userwas chosenover a
systemreminder from the original sketch because it is the strongest signal forsuperseding earlier instructions and it carries images natively.
Configuration
Ctrl+Entersteers a single message and is always available.steerModedecides whether plainEnterdoes the same while the model is writing:queue(default)Enteronly queues the prompt: it is injected at the next LLM call of the turn, nothing is cut shortinterruptEnteralso cuts the streaming answer short, keeps its text, and reads the prompt nowSteering is therefore opt-in: an existing setup keeps behaving exactly as it did before this PR
until the user presses
Ctrl+Enter(or setssteerMode: "interrupt"). Running tools are neverinterrupted in either mode. On terminals that do not report
Ctrl+Enter(it requiresmodifyOtherKeys mode),
steerMode: "interrupt"is the way to steer.UI
Injected guidance is labelled in the transcript (
✧ Guidance sent while the turn was running),and so is a cut-short answer (
— superseded by your guidance).Backspaceon an empty promptremoves the last entry that has not been injected yet.
Changes
packages/core/src/session.tsMessageMeta.isSupplementary/.interruptedpackages/core/src/common/llm-retry.tsLlmSteeredErrorcarrying the partial answerpackages/core/src/settings.ts,index.tssteerModesetting and exportspackages/cli/src/ui/views/App.tsxpackages/cli/src/ui/views/PromptInput.tsxEnterusable while busy,Backspaceremoves the last entrypackages/cli/src/ui/components/MessageView/*steerModeconfiguration.md/_en.mdtable and section, README key tablesTesting
npm run check— type check, ESLint and Prettier all passnpm test— 333 CLI + 392 core + 59 VS Code tests passflight shows up in the next request of the same turn, and that a steered answer appears in the
follow-up request together with the guidance.
Design notes vs #117
#117 sketches the same idea with a different shape, so it is worth stating the differences:
systemmessage prefixed with[User Supplementary Guidance]. This PR uses ausermessage withmeta.isSupplementary: it is the strongestsignal for superseding earlier instructions, keeps the prompt verbatim in the transcript, and
carries images natively. Switching to
systemis a one-line change if you prefer the softerframing.
flushes before every request and keeps the loop alive when a turn would otherwise end with
guidance still pending, so nothing is deferred to a later, unrelated prompt.
this PR adds steering on top:
Ctrl+Enteraborts the in-flight request and the turn continueswith the guidance, keeping the partial text. It is opt-in (
steerMode: "interrupt"makes it thedefault for
Enter,"queue"keeps the behaviour of the first commit).SessionManager, so the VSCode companion can adopt it; only thepolicy decision (steer or wait) currently sits in the CLI.
中文说明
AI 正在回复时,
Enter不再被拒绝:消息会作为补充指引,在本轮下一次 LLM 调用前作为 user 消息(含图片与 skills)注入,模型因此能结合已完成的工作修改甚至推翻之前的指令。如果模型正在流式输出
回答,按
Ctrl+Enter可以截断该回答并立即读取新指令(已生成内容保留在对话中并标记meta.interrupted);正在执行的命令不会被中断,steering 只作用于模型输出流。steering 默认是 opt-in:steerMode默认"queue"(只有Enter排队注入),设为"interrupt"后普通Enter也会截断当前输出。等待
ask_user_question或权限确认时不会注入,Esc仍会中断整个轮次。