Skip to content

feat: inject and steer new guidance into the running turn - #357

Open
mulyawansentosa wants to merge 4 commits into
lessweb:mainfrom
Code-AI-Tech-ID:feat/prompt-queue
Open

mulyawansentosa wants to merge 4 commits into
lessweb:mainfrom
Code-AI-Tech-ID:feat/prompt-queue

Conversation

@mulyawansentosa

@mulyawansentosa mulyawansentosa commented Sep 24, 2026 •

Copy link
Copy Markdown

Summary

Closes #113. Supersedes / alternative to #117 (the earlier open PR that sketches the same
mechanism with a slightly different design - see the notes at the bottom).

While a turn was running, Enter was rejected with "wait for the current response or press esc
to interrupt"
, so sending a follow-up instruction meant waiting for the whole turn or
interrupting the agent. This PR makes new instructions flow into the work that is already
running:

  1. Mid-turn injection — a prompt sent while a turn runs becomes a user message that is read
    before the model's next step of that turn. This is the default behaviour of Enter.
  2. Steering a streaming answer (opt-in) — no API can rewrite output that was already
    generated, so when the user explicitly asks for it (Ctrl+Enter, or steerMode: "interrupt"),
    the request that is streaming is aborted, its text is kept in the conversation, and the turn
    continues with the new instruction.

How it works

SessionManager keeps a per-session FIFO queue of supplemental prompts
(addSupplementaryPrompt, max 20). Before every LLM call of the running turn the queue is
drained and each entry becomes a user message with its images and selected skills:

user:      install nginx
assistant: <tool_calls>              ← working
tool:      <result>
user:      stop, use apache instead     ← guidance, read by the next LLM call
assistant: <revised plan>

When the user asks to steer (below), steerActiveSession() aborts that single request:

user:      explain how the deploy works
assistant: The deploy runs in three stag…      ← cut short here
user:      no, just tell me the rollback step   ← read straight away
assistant: <answer to the new instruction>

Details:

  • The partial answer is kept as an assistant message with meta.interrupted, so the model (and
    the transcript) still know what it had said when the user changed direction.
  • Steering only targets the model stream: tools are never interrupted, so a running
    npm install finishes and the guidance is injected at the next request boundary.
  • A steered answer is not an interruption: the turn continues, the status stays completed, and
    the aborted request is not retried.
  • If a turn would otherwise end with guidance still queued, the loop stays alive for one more
    iteration instead of deferring the guidance to a later, unrelated turn.
  • Esc still interrupts the whole turn.
  • Guidance is not injected while the session waits for the user (an ask_user_question answer or
    a permission decision), so it can never be consumed as that answer.
  • The prompt is stored verbatim with role: "user" and meta.isSupplementary. user was chosen
    over a system reminder from the original sketch because it is the strongest signal for
    superseding earlier instructions and it carries images natively.

Configuration

Ctrl+Enter steers a single message and is always available. steerMode decides whether plain
Enter does the same while the model is writing:

Value Behaviour
queue (default) Enter only queues the prompt: it is injected at the next LLM call of the turn, nothing is cut short
interrupt Enter also cuts the streaming answer short, keeps its text, and reads the prompt now

Steering is therefore opt-in: an existing setup keeps behaving exactly as it did before this PR
until the user presses Ctrl+Enter (or sets steerMode: "interrupt"). Running tools are never
interrupted in either mode. On terminals that do not report Ctrl+Enter (it requires
modifyOtherKeys mode), steerMode: "interrupt" is the way to steer.

UI

  guidance 1. just tell me the rollback step
> _
  esc to interrupt · ctrl+enter send & cut · ctrl+c to cancel input · 1 guidance queued · backspace to remove

Injected guidance is labelled in the transcript (✧ Guidance sent while the turn was running),
and so is a cut-short answer (— superseded by your guidance). Backspace on an empty prompt
removes the last entry that has not been injected yet.

Changes

File Change
packages/core/src/session.ts supplemental queue, drain before each request, per-request steer signal, keep the loop alive while guidance is pending, MessageMeta.isSupplementary / .interrupted
packages/core/src/common/llm-retry.ts LlmSteeredError carrying the partial answer
packages/core/src/settings.ts, index.ts steerMode setting and exports
packages/cli/src/ui/views/App.tsx hands prompts typed while busy to the session, mirrors the pending queue, steers when configured, removes the last entry
packages/cli/src/ui/views/PromptInput.tsx lists pending guidance, keeps Enter usable while busy, Backspace removes the last entry
packages/cli/src/ui/components/MessageView/* labels injected guidance and cut-short answers
core tests 9 tests for queue semantics, mid-flight injection, steering and the no-op case; settings test for steerMode
cli tests 4 Ink integration tests + 3 formatting tests
docs configuration.md/_en.md table and section, README key tables

Testing

  • npm run check — type check, ESLint and Prettier all pass
  • npm test — 333 CLI + 392 core + 59 VS Code tests pass
  • Core tests assert the actual request payloads, i.e. that guidance added while a request is in
    flight
    shows up in the next request of the same turn, and that a steered answer appears in the
    follow-up request together with the guidance.

Design notes vs #117

#117 sketches the same idea with a different shape, so it is worth stating the differences:

  • Role. feat: 新增用户补充指引功能,含队列管理与 UI 交互 #117 injects the guidance as a system message prefixed with [User Supplementary Guidance]. This PR uses a user message with meta.isSupplementary: it is the strongest
    signal for superseding earlier instructions, keeps the prompt verbatim in the transcript, and
    carries images natively. Switching to system is a one-line change if you prefer the softer
    framing.
  • Turn lifetime. feat: 新增用户补充指引功能,含队列管理与 UI 交互 #117 flushes only while the previous response contained tool calls. This PR
    flushes before every request and keeps the loop alive when a turn would otherwise end with
    guidance still pending, so nothing is deferred to a later, unrelated prompt.
  • Answer already streaming. Neither API allows editing output that was already generated, so
    this PR adds steering on top: Ctrl+Enter aborts the in-flight request and the turn continues
    with the guidance, keeping the partial text. It is opt-in (steerMode: "interrupt" makes it the
    default for Enter, "queue" keeps the behaviour of the first commit).
  • Scope. The queue lives in SessionManager, so the VSCode companion can adopt it; only the
    policy decision (steer or wait) currently sits in the CLI.

中文说明

AI 正在回复时,Enter 不再被拒绝:消息会作为补充指引,在本轮下一次 LLM 调用前作为 user 消息
(含图片与 skills)注入,模型因此能结合已完成的工作修改甚至推翻之前的指令。如果模型正在流式输出
回答,按 Ctrl+Enter 可以截断该回答并立即读取新指令(已生成内容保留在对话中并标记
meta.interrupted);正在执行的命令不会被中断,steering 只作用于模型输出流。steering 默认是 opt-in:
steerMode 默认 "queue"(只有 Enter 排队注入),设为 "interrupt" 后普通 Enter 也会截断当前
输出。等待 ask_user_question 或权限确认时不会注入,Esc 仍会中断整个轮次。

While a turn was running, Enter was rejected with "wait for the current response or
press esc to interrupt", so sending a follow-up instruction meant waiting for the whole
turn or interrupting the agent. A message queued until the end of the turn would still
arrive too late to override instructions the agent had already executed.

SessionManager now keeps a small per-session queue of supplemental prompts. Before
every LLM call of the running turn the queue is drained: each entry becomes a user
message (including images and selected skills) appended to the conversation, so the
model reads it together with the work it already did and can revise or supersede the
earlier instructions. When a turn would end while guidance is still queued, the loop
stays alive for one more iteration instead of deferring the guidance to a later turn.

- session: addSupplementaryPrompt / cancelSupplementaryPrompt /
  listPendingSupplementaryPrompts / countPendingSupplementaryPrompts, the
  onSupplementaryQueueChanged and onSupplementaryPromptInjected callbacks, and
  MessageMeta.isSupplementary
- session: drain the queue before each request, keep the loop alive while guidance is
  pending, and drop the queue when a session is cleaned up
- cli: prompts typed while busy are handed to the session, the prompt area lists the
  guidance that is still waiting, Backspace on an empty prompt removes the last entry,
  and injected guidance is rendered as a labelled block (also in raw mode)
- tests: 7 core tests covering the queue semantics and the request payloads of a turn
  that receives guidance mid-flight, plus 4 Ink tests for the input wiring
- docs: README and quickstart key tables

Refs lessweb#113, lessweb#117
Guidance injected before the next LLM call still had to wait for the answer that was
already streaming, so a long text answer could not be redirected until it finished on
its own. No API can rewrite output that has already been generated, so the only way to
have the guidance read sooner is to abort that one request and ask again.

The session now tracks the streaming request separately from the turn. When the user
asks to steer, that request is aborted, whatever the model already wrote is kept as an
assistant message marked meta.interrupted, and the loop immediately issues a new
request that carries the guidance. Tools are never interrupted: steering only targets
the model stream, so a running command finishes first and the guidance is injected at
the next request boundary.

Steering is opt-in, so an existing setup keeps behaving exactly as before:

- Ctrl+Enter steers that one message.
- steerMode: "interrupt" makes plain Enter steer as well; the default is "queue".

- session: steerActiveSession(), a per-request steer signal released once the request
  settles, and LlmSteeredError carrying the partial answer
- session: a steered answer is not an interruption - the turn continues, the status
  stays completed, and the aborted request is not retried
- settings: steerMode ("queue" by default)
- cli: Ctrl+Enter is parsed from its CSI-u sequence, forwarded as
  PromptSubmission.steer, and the cut-short answer is labelled in the transcript;
  the footer mentions the shortcut
- tests: core covers the aborted stream, the follow-up request payload, the kept
  partial answer and the no-op case; the cli covers the key parsing, the steering
  submit, and steerMode resolution
- docs: configuration table and section, README and quickstart key tables

Refs lessweb#113, lessweb#117
@mulyawansentosa

Copy link
Copy Markdown
Author

Updated after review of the design: steering is now opt-in instead of the previous default.

  • Enter while a turn is running only queues the prompt (injected before the model's next step).
  • Ctrl+Enter steers that one message: the request that is streaming is aborted, its partial text is kept as an assistant message, and the turn continues with the prompt.
  • steerMode: "interrupt" in settings.json makes plain Enter steer as well; the default is now "queue".

That keeps the behaviour of an existing setup unchanged (no answer is cut short unless the user asks for it), while still giving the "flowing conversation" the feature is about. The branch was rewritten so the second commit now describes the opt-in behaviour; PR body and the docs were updated to match.

`handleSubmit` treated every submission as guidance while a turn was running, including
the `/exit` command that `PromptInput` deliberately lets through. The CLI then queued
"/exit" as a user message and steered the turn instead of quitting.

Commands are not prompts, so they take the normal path again: non-exit commands stay
blocked while the turn runs, and `/exit` still quits.

Found by driving the real TUI in a pseudo-terminal against a stub OpenAI-compatible
server. The guidance path itself behaved as designed: the streaming answer was aborted
after 21 of 130 chunks, the partial text was kept and marked `meta.interrupted`, and the
follow-up request carried it together with the guidance as a user message.

Refs lessweb#113, lessweb#117
@mulyawansentosa

Copy link
Copy Markdown
Author

Verified end-to-end after the opt-in change, by driving the real TUI in a pseudo-terminal against a stub OpenAI-compatible server (it streams a 130-chunk answer on demand):

  • while the turn was writing, a second prompt was submitted with Enter and steerMode: "interrupt";
  • the stub saw the streaming request aborted after 21 of 130 chunks;
  • the next request of the same turn carried the partial answer plus the new prompt as a user message;
  • the transcript marked the partial answer meta.interrupted and the prompt meta.isSupplementary, and rendered ✧ Guidance sent while the turn was running / — superseded by your guidance.

That run also caught a regression introduced in the second commit: handleSubmit treated every submission as guidance while a turn was running, so /exit was queued as a user message and steered the turn instead of quitting. Fixed in fa47951 - commands are not prompts, they take the normal path again (/exit quits, other commands stay blocked while the turn runs), with a test that pins the contract.

10 was a conservative guess rather than a measured limit. The queue drains at the next
request boundary, so this only caps how many instructions can pile up faster than one
step of the turn (a long tool run or a long streaming answer).

Raising it to 20 keeps that burst covered while still bounding the cost: every queued
prompt travels with each remaining request of the turn, so an unbounded queue could
inflate the context (and any attached images) without limit.

- the queue is not dropped or reordered; it still drains in FIFO order before every
  LLM call of the turn
- when the limit is reached the prompt is refused but stays in the input box, so
  nothing the user typed is lost

Refs lessweb#113, lessweb#117
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

【TUI】不打断任务的情况下,补充发送引导信息。

1 participant