Skip to content

feat: inject and steer new guidance into the running turn - #356

Closed
mulyawansentosa wants to merge 2 commits into
lessweb:mainfrom
mulyawansentosa:feat/prompt-queue
Closed

mulyawansentosa wants to merge 2 commits into
lessweb:mainfrom
mulyawansentosa:feat/prompt-queue

Conversation

@mulyawansentosa

Copy link
Copy Markdown

Summary

Closes #113, #117.

While a turn was running, Enter was rejected with "wait for the current response or press esc
to interrupt"
, so sending a follow-up instruction meant waiting for the whole turn or
interrupting the agent. This PR makes new instructions flow into the work that is already
running:

  1. Mid-turn injection — a prompt sent while a turn runs becomes a user message that is read
    before the model's next step of that turn.
  2. Steering a streaming answer — if the model is still writing an answer, that answer is cut
    short so the prompt is read immediately; what it already wrote stays in the conversation.

How it works

SessionManager keeps a per-session FIFO queue of supplemental prompts
(addSupplementaryPrompt, max 10). Before every LLM call of the running turn the queue is
drained and each entry becomes a user message with its images and selected skills:

user:      install nginx
assistant: <tool_calls>              ← working
tool:      <result>
user:      stop, use apache instead     ← guidance, read by the next LLM call
assistant: <revised plan>

When the model is streaming an answer, steerActiveSession() aborts that single request:

user:      explain how the deploy works
assistant: The deploy runs in three stag…      ← cut short here
user:      no, just tell me the rollback step   ← read straight away
assistant: <answer to the new instruction>

Details:

  • The partial answer is kept as an assistant message with meta.interrupted, so the model (and
    the transcript) still know what it had said when the user changed direction.
  • Steering only targets the model stream: tools are never interrupted, so a running
    npm install finishes and the guidance is injected at the next request boundary.
  • A steered answer is not an interruption: the turn continues, the status stays completed, and
    the aborted request is not retried.
  • If a turn would otherwise end with guidance still queued, the loop stays alive for one more
    iteration instead of deferring the guidance to a later, unrelated turn.
  • Esc still interrupts the whole turn.
  • Guidance is not injected while the session waits for the user (an ask_user_question answer or
    a permission decision), so it can never be consumed as that answer.
  • The prompt is stored verbatim with role: "user" and meta.isSupplementary. user was chosen
    over a system reminder from the original sketch because it is the strongest signal for
    superseding earlier instructions and it carries images natively.

Configuration

steerMode controls what happens when a prompt arrives while the model is writing:

Value Behaviour
interrupt (default) Cut the streaming answer short, keep its text, read the prompt now
queue Wait and inject the prompt at the next LLM call of the turn

Running tools are never interrupted in either mode.

UI

  guidance 1. just tell me the rollback step
> _
  esc to interrupt · ctrl+c to cancel input · 1 guidance queued · backspace to remove

Injected guidance is labelled in the transcript (✧ Guidance sent while the turn was running),
and so is a cut-short answer (— superseded by your guidance). Backspace on an empty prompt
removes the last entry that has not been injected yet.

Changes

File Change
packages/core/src/session.ts supplemental queue, drain before each request, per-request steer signal, keep the loop alive while guidance is pending, MessageMeta.isSupplementary / .interrupted
packages/core/src/common/llm-retry.ts LlmSteeredError carrying the partial answer
packages/core/src/settings.ts, index.ts steerMode setting and exports
packages/cli/src/ui/views/App.tsx hands prompts typed while busy to the session, mirrors the pending queue, steers when configured, removes the last entry
packages/cli/src/ui/views/PromptInput.tsx lists pending guidance, keeps Enter usable while busy, Backspace removes the last entry
packages/cli/src/ui/components/MessageView/* labels injected guidance and cut-short answers
core tests 9 tests for queue semantics, mid-flight injection, steering and the no-op case; settings test for steerMode
cli tests 4 Ink integration tests + 3 formatting tests
docs configuration.md/_en.md table and section, README key tables

Testing

  • npm run check — type check, ESLint and Prettier all pass
  • npm test — 333 CLI + 392 core + 59 VS Code tests pass
  • Core tests assert the actual request payloads, i.e. that guidance added while a request is in
    flight
    shows up in the next request of the same turn, and that a steered answer appears in the
    follow-up request together with the guidance.

中文说明

AI 正在回复时,Enter 不再被拒绝:消息会作为补充指引,在本轮下一次 LLM 调用前作为 user 消息
(含图片与 skills)注入,模型因此能结合已完成的工作修改甚至推翻之前的指令。如果模型正在流式输出
回答,该回答会被截断,已生成内容保留在对话中(标记 meta.interrupted),新指令立即被读取;正在
执行的命令不会被中断,steering 只作用于模型输出流。可用 steerMode 选择 "interrupt"(默认,截断
并立即读取)或 "queue"(等到下次请求边界再注入)。等待 ask_user_question 或权限确认时不会注入,
Esc 仍会中断整个轮次。

While a turn was running, Enter was rejected with "wait for the current response or
press esc to interrupt", so sending a follow-up instruction meant waiting for the whole
turn or interrupting the agent. A message queued until the end of the turn would still
arrive too late to override instructions the agent had already executed.

SessionManager now keeps a small per-session queue of supplemental prompts. Before
every LLM call of the running turn the queue is drained: each entry becomes a user
message (including images and selected skills) appended to the conversation, so the
model reads it together with the work it already did and can revise or supersede the
earlier instructions. When a turn would end while guidance is still queued, the loop
stays alive for one more iteration instead of deferring the guidance to a later turn.

- session: addSupplementaryPrompt / cancelSupplementaryPrompt /
  listPendingSupplementaryPrompts / countPendingSupplementaryPrompts, the
  onSupplementaryQueueChanged and onSupplementaryPromptInjected callbacks, and
  MessageMeta.isSupplementary
- session: drain the queue before each request, keep the loop alive while guidance is
  pending, and drop the queue when a session is cleaned up
- cli: prompts typed while busy are handed to the session, the prompt area lists the
  guidance that is still waiting, Backspace on an empty prompt removes the last entry,
  and injected guidance is rendered as a labelled block (also in raw mode)
- tests: 7 core tests covering the queue semantics and the request payloads of a turn
  that receives guidance mid-flight, plus 4 Ink tests for the input wiring
- docs: README and quickstart key tables

Refs lessweb#113, lessweb#117
Guidance injected before the next LLM call still had to wait for the answer that
was already streaming, so a long text answer could not be redirected until it
finished on its own.

The session now tracks the streaming request separately from the turn. When the
user sends guidance while an answer is streaming, that request is aborted,
whatever the model already wrote is kept as an assistant message marked
meta.interrupted, and the loop immediately issues a new request that carries the
guidance. Tools are never interrupted: steering only targets the model stream, so
a running command finishes first and the guidance is injected at the next request
boundary.

- session: steerActiveSession(), a per-request steer signal released once the
  request settles, and LlmSteeredError carrying the partial answer
- session: a steered answer is not an interruption - the turn continues, the
  status stays completed, and the request is not retried
- settings: steerMode ("interrupt" by default, "queue" keeps the old behaviour)
- cli: submitting while busy calls steerActiveSession() when steerMode is
  "interrupt"; the cut-short answer is labelled in the transcript
- tests: core covers the aborted stream, the follow-up request payload, the kept
  partial answer, and the no-op case; settings cover steerMode resolution
- docs: configuration table and section, plus the README notes

Refs lessweb#113, lessweb#117
@mulyawansentosa

Copy link
Copy Markdown
Author

Opened from the wrong fork by mistake. The implementation is now opened from the org fork instead: #357

@mulyawansentosa
mulyawansentosa deleted the feat/prompt-queue branch September 24, 2026 04:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

【TUI】不打断任务的情况下,补充发送引导信息。

1 participant