host: the memory-review prompt cuts on UTF-16 units, and a two-argument clip() prints the literal (cut; undefined) #1918

Implementation: FlyBot (ours) — PR follows within 24h. Do not pick up without a pickup-ok comment on this issue.

Written-by: FlyBot Lead
Claimed-by: FlyBot Lead
Claim-Id: 7f3e3eba952a44a0b7caf5454756aa63
Branch: dev/1-clip-rest-optional
Base: f87fceb
At: 2026-10-31T10:16:30Z

Implementation: FlyBot (ours) — PR follows within 24h. Do not pick up without a pickup-ok comment.

Written-by: FlyBot Lead

Dedupe (checked on live main f87fceb0 before opening): grep -rn "cut; " finds only core/src/host/text.js:59; issue search for "(cut; undefined" returns 0, and the open clip-in-title issues are layout/clipping reports (chat rows, workflow drawers) that do not touch this helper.

Scope correction (2026-10-31, checked line by line at f87fceb0)

The premise in the first version of this issue was wrong; this section supersedes it.

The original premise does not hold. This issue first claimed that core/src/host/memory.js calls the shared clip with two arguments and therefore prints (cut; undefined) into the memory-review prompt and the run log. Checked line by line at f87fceb0, that is false: memory.js imports only { codePoints, splitRef, unquote } from ./text.js (:30) and uses its own local clip (:1146) and ends (:1147), and the only importer of the shared clip anywhere in the tree is core/src/host/parts.js:22, whose three calls (:344, :355, :465) all pass a reason. So no product path reaches (cut; undefined).

What this change actually fixes, one coherent family, not added scope: (1) the exported helper's two-argument contract — a truncating two-argument call must print … (cut), not undefined (one line in text.js, unit-tested); (2) the reachable half of the same defect, the local clip at memory.js:1146 and the local ends at :1147, which bound the memory-review prompt and the run log by slicing UTF-16 units and so put half a surrogate pair in front of the model; (3) the same two helpers' tail edge in ends. All three are "a bound that does not go through the shared boundary rule", which is the rule #1916/#1917 established; the prompt and log paths the title names are fixed by (2) and (3).

core/src/host/memory.js:30 imports only { codePoints, splitRef, unquote } from ./text.js. It never imports clip: it defines a local clip at :1146 and a local ends at :1147, and :1154, :1338, :1347 all call that local one. That local clip always prints …(cut) and can never print undefined. The only importer of the exported clip is core/src/host/parts.js:22, and its three calls (:344, :355, :465) all pass a reason.

So (cut; undefined) is not reachable on a product path today: it is a contract bug of the exported helper (a truncating two-argument call), and it gets a unit test rather than a call-site change. What is real and reachable is the other half — the two local helpers that build the memory-review prompt slice on UTF-16 units, so a long line whose bound falls inside an emoji puts half a surrogate pair (U+FFFD once encoded) into a prompt the model reads and into the run log.

Found by Dev-1 in red-first; verified by FlyBot Lead at f87fceb0.

Problem

The exported helper prints its third argument unconditionally:

// core/src/host/text.js:59
export const clip = (text, max, rest) => (text.length > max ? `${text.slice(0, codePointEnd(text, max))}… (cut; ${rest})` : text)

rest has no default and no guard, so a truncating two-argument call ends with the literal (cut; undefined). Its cut is already code-point safe (codePointEnd, #1917); only the suffix leaks.

The reachable defect is next to it, in the memory review:

// core/src/host/memory.js:1146
const clip = (text, max) => (text.length > max ? `${text.slice(0, max)} …(cut)` : text)
// core/src/host/memory.js:1147
const ends = (text, max) => (text.length > max ? `${text.slice(0, Math.ceil(max * 2 / 3))} …(middle cut)… ${text.slice(-Math.floor(max / 3))}` : text)

Both slice on UTF-16 units, on all three edges (the head of the prefix, and the start of the tail in ends). Call sites: :1154 reviewLine — the non-user branch uses clip(…, REVIEW_OTHER_LINE_MAX) (800) and the who === 'user' branch uses ends(text, REVIEW_USER_LINE_MAX); :1338 clip(reply, 120); :1347 clip(badTopic.topic, 60). The :1154 line is what reviewRequest shows the model as "Earlier in …, already reviewed (context only):" (:386), so a split pair reaches a model prompt; :1338/:1347 are formatted into rt.warn('ds-bot: memory review of %s failed: %s', …) (:1410), so the owner reads U+FFFD in the run log.

Reproduction (base f87fceb0)

import { clip } from './core/src/host/text.js'
clip('x'.repeat(900), 800).slice(-22)   // "xxxxxxxxxxxx… (cut; undefined)"   <- helper contract only

memory review, non-user line > 800 units whose 800th unit is inside an emoji
  -> the "Earlier in …" prompt block ends with a lone high surrogate (U+FFFD once encoded)
memory review, user line >= REVIEW_USER_LINE_MAX with an emoji at the 2/3 bound and at the tail bound
  -> both ends of the rendered line are ill-formed

Expected

• A truncating two-argument clip call renders … (cut) with no undefined; a three-argument call keeps … (cut; <reason>) byte for byte, same bound, same codePointEnd, same text.length > max condition.
• memory.js:1146 goes through shared/text-boundary.js: `${cut(text, max).text} …(cut)` — the suffix stays exactly …(cut), with no space inserted after ….
• memory.js:1147 cuts both edges on code points: the head is cut(text, head).text, the tail starts at codePointStart(text, text.length - tail) (retreated onto a boundary, never advanced — advancing would start the tail on a low surrogate). The …(middle cut)… suffix is unchanged.
• The unit budget does not grow: today the head is ceil(max * 2 / 3) and the tail is floor(max / 3); if the tail retracts by one unit the head loses one, so the total stays inside the same bound.
• Nothing else in memory.js changes.

Scope

• core/src/host/text.js — rest optional in clip.
• core/src/host/memory.js — :1146 and :1147 move onto shared/text-boundary.js (cut, codePointStart; a host module may import shared/, and shared/ still imports no host/).
• core/test/text.test.js, core/test/memory.test.js — red-first cases.
• Mirror patches/ds-bot/ files that own text.js and memory.js (regenerate; check + canonical must stay green).

Red first

With the sources at f87fceb0 and only the new tests in place:

• clip('x'.repeat(900), 800) contains no undefined and ends with … (cut).
• A three-argument call still ends with (cut; <reason>) — guards against dropping the reason.
• Memory review, non-user branch: a >800-unit Bot line ending inside an emoji renders a prompt that is isWellFormed() === true, contains no U+FFFD, and still ends the line with …(cut).
• Memory review, user branch: a line >= REVIEW_USER_LINE_MAX with an emoji at the 2/3 bound and at the tail bound renders isWellFormed() === true, whose tail is a byte suffix of the input, and whose length stays inside the pre-change budget.

Assertions are length <= max + isWellFormed() + a byte-for-byte UTF-8 prefix after the tail is dropped. They must not assert that the output is exactly max units.

Not in this change

• Any other local slice bound inside memory.js beyond these two helpers — report it, do not fix it here.
• diagnostics.js:87 <details><summary>Stack: — that is diagnostics: the stack summary still cuts entry.text on a UTF-16 unit, so the report can show U+FFFD #1919 / diagnostics: cut the report's stack summary on a code point boundary #1920.
• Grapheme-aware cutting (separate follow-up; cut keeps code-point semantics).
• The hooks.js:353/:358 and hooks.js:693/:697 bounds — the P-278 cluster, a separate issue.

Gates

TZ=UTC ARCH_DOCS_BASE=f87fceb0148f60ad8c20d02f9838e91c3debd5a8 npm run verify must exit 0, patches check + canonical green, generate a no-op after the rewrite, arch-docs satisfied for every changed file. PR body with Fixes #1918, the checkpoint carrying claim: "Dev" + Owner: Dev in the same spelling, and an explicit paragraph stating the corrected reachability (the original premise was unreachable; this change is the helper's two-argument contract plus the memory-review helpers).