Skip to content

host: room turns cut card, stub and transcript text on a code point boundary - #1961

Merged
Birfy merged 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev/2-p305-room-context-code-point-bound
Nov 1, 2026
Merged

Birfy merged 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev/2-p305-room-context-code-point-bound

Conversation

@HugoXK

@HugoXK HugoXK commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1960

Owner: Dev-2
Reviewer: Birfy
Written-by: Dev-2
Claim-Id: 311b65a9317c4d539e8ca9a4734966df
Branch: dev/2-p305-room-context-code-point-bound
Base: 54dad6b
Tip SHA: 1d6e92c
Tree: 29db63e18ba5e1d8d4168ea207f7eee646b4e06b

Three character bounds in core/src/host/room-context.js ended a bounded prefix with a bare slice, so a bound that landed between the halves of a surrogate pair kept the high half and produced a string that is not well formed. The stub is the one that reaches disk: since #1251 every writer of room.log runs its line through prepareRoomLine → compactStatus, so a half cut there is stored in state.json and reaches a UTF-8 encoder as U+FFFD on the next load.

site base 54dad6b8 this tree
compactStatus — the hundred-unit stub every writer of room.log stores :95 flat.slice(0, 100).trim() :96 cut(flat, 100).text.trim()
fitRoomLines — the transcript clip at budget - used - gap :142 text.slice(0, room - 1) :143 cut(text, room - 1).text
oneLine — the card, board and presence fields (max - 1) :45 flat.slice(0, Math.max(0, max - 1)) :46 cut(flat, Math.max(0, max - 1)).text

This tree is one import line longer than the base (cut from ../shared/text-boundary.js), so every line below it reads +1 against 54dad6b8. #1959 did not touch this file, so the base column is exact.

cut(text, max) (core/src/shared/text-boundary.js) answers { text, cut } with the longest prefix that does not split a pair. The budget is unchanged everywhere — 100 for the stub, room - 1 for the clip, max - 1 for oneLine — and only the exit changes: text inside the bound comes back untouched, and a pair straddling it is dropped whole instead of kept in half. No signature, constant or record shape moves, and oneLine's own semantics (flatten, fold </> and control characters to spaces, squeeze whitespace, trim, then cut, then append … when it truncated) stay as they were.

Base and rebase

The branch was cut from 37e81623 and is rebased onto 54dad6b8 (the merge of #1959), not merged. #1959 added its oneLine sentence to the task-record paragraph of docs/subsystems/core-host.md; this PR's sentence is in the Room context budget (#1203) paragraph above it, so the rebase applied clean with no conflict and both sentences stand, each in the paragraph whose subject it covers. The one new changelog.d/ fragment is 1960-room-context-code-point-bound.fixed.md; #1959's 1958-… fragment is untouched.

Red first

Three cases, one per site; the third is the stored-value case and is asserted on what a fresh runtime reads back from state.json, not on the object the writer kept in memory. With the base blob swapped in (git show 54dad6b8:core/src/host/room-context.js > core/src/host/room-context.js, the two test files at this branch):

$ cd core && TZ=UTC node --import ./test/harness-resolve.mjs --test --test-name-pattern="P-305" test/room-context.test.js test/room-membership.test.js
not ok 1 - P-305: a card title cut at its bound keeps a whole emoji
    the board line is not well formed: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\ud83d…"
not ok 2 - P-305: a clipped transcript line ends on a whole emoji
    the clipped line is not well formed: "aaaaaa\ud83d…"
not ok 3 - P-305: the stored status stub survives the store round trip well formed
    the reloaded stub is not well formed: "[status on the board, 713 chars] aaaa…\ud83d…"
# pass 0 / # fail 3

(with the branch's room-context.js: # pass 3 / # fail 0.)

Case 3 walks the real path: postRoomMessage → append → prepareRoomLine → rt.save() writes state.json, then a new runtime loads that file and the assertion runs on the reloaded line — isWellFormed(), a UTF-8 round trip, and a turn rendered from the reloaded log by fitRoomLines with prepareRoomLine. Each case also carries the controls: a string of exactly 80 units ending in a whole 😀 comes back untouched, and a ZWJ family emoji inside the bound survives whole.

Mutation — each site has its own tooth

One site reverted at a time, the other two in place:

reverted alone result
:96 stub back to slice only case 3 fails (the reloaded stub is not well formed)
:143 clip back to slice only case 2 fails (the clipped line is not well formed)
:46 oneLine tail back to slice only case 1 fails (the board line is not well formed)

Each run is # pass 2 / # fail 1, so no case is satisfied by the other two fixes and the controls stay green under all three.

Why core/test/room-membership.test.js is in the 8 files

Case 3 needs a real store round trip, and the runtime scaffolding that writes and reloads state.json — host(), rt.load(), rt.save(), postRoomMessage — lives only in core/test/room-membership.test.js. The file's change is two import lines (readFileSync; fitRoomLines and prepareRoomLine from room-context.js) plus one new test at the end. No existing assertion is edited. The alternative, a second copy of that scaffold in room-context.test.js, would duplicate the harness rather than assert on the store the product writes.

Verification

TZ=UTC ARCH_DOCS_BASE=54dad6b856c5ea8c387b2a62ea9143e67126acd3 npm run verify → exit 0, on a clean tree (git status --porcelain empty) at this tip's tree 29db63e1…. Counts, this tree / base 54dad6b8:

suite this tree base
core 2361 tests, 2361 pass, 0 fail 2358, 2358, 0
plugins/social 541 tests, 541 pass, 0 skipped (CI at this tip; one Cordis test skips locally) same — no plugin file changes
plugins/tree 127 tests, 127 pass, 0 skipped (CI at this tip; one Cordis test skips locally) same — no plugin file changes
plugins/timeline 135 tests, 135 pass, 0 skipped (CI at this tip; one Cordis test skips locally) same — no plugin file changes
plugins/workflow 336, 336, 0 skipped same
.github/scripts 563, 563 same
interop 37, 37 same
live-e2e 24, 24 same

The base core count is measured, not inferred: a clean worktree at 54dad6b8 with the same harness runs 2358 tests / 2358 pass / 0 fail, so this PR adds exactly the three cases above.

  • arch-docs: OK (8 changed file(s)) against ARCH_DOCS_BASE=54dad6b8… — the 8 files are the two core/src/test changes plus their three patches/ds-bot/files/ mirrors, the doc, and the changelog fragment
  • lint: core 73 warnings / 0 errors; plugins social 36, tree 9, timeline 2, workflow 16, interop 1 / 0 errors; live-e2e 0
  • ds-bot-patches: pin Fei2411/DeepSeek-Bot@2b10c70, 10 patches apply in order, core/ = ds-bot/ + patches, canonical clean, generate a no-op; the three mirrors are byte-identical to core/ (cmp clean on src/host/room-context.js, test/room-context.test.js, test/room-membership.test.js)
  • build-client --check ok; changelog.mjs check → 4 fragment(s) in changelog.d/ are well formed

Not run: the smoke test — this checkout has no dev/bench.sh and no running FlyBot instance to drive. Also not run: the CI package job, typecheck and actionlint.

Checkpoint

{
"pr": 1961, "claim": "Dev-2", "issue": 1960,
"summary": "The three character bounds in core/src/host/room-context.js now cut through shared/text-boundary.js cut(): the hundred-unit status stub every writer of room.log stores, the transcript clip, and the oneLine fields behind card, board and presence lines. Budgets and oneLine's flatten, trim and ellipsis are unchanged; a pair straddling a bound is dropped whole, never kept in half. The stored-stub case asserts on the line a fresh runtime reloads from state.json, not the in-memory one. room-membership.test.js gains two imports and one test because the store round-trip scaffold lives only there.",
"files": ["core/src/host/room-context.js", "core/test/room-context.test.js", "core/test/room-membership.test.js", "docs/subsystems/core-host.md", "changelog.d/1960-room-context-code-point-bound.fixed.md", "patches/ds-bot/files/src/host/room-context.js", "patches/ds-bot/files/test/room-context.test.js", "patches/ds-bot/files/test/room-membership.test.js"],
"checks": { "ran": "TZ=UTC ARCH_DOCS_BASE=54dad6b856c5ea8c387b2a62ea9143e67126acd3 npm run verify; node scripts/ds-bot-patches.mjs check; node .github/scripts/changelog.mjs check", "result": "green", "at": "1d6e92c3c7daa383f07c8d8b933bffcfb7603988" },
"next": "none — ready for review"
}

…oundary

`compactStatus`, `fitRoomLines` and the `oneLine` behind the card, board and presence
fields all ended a bounded prefix with a bare `slice`, so a bound that landed between the
halves of a surrogate pair stored and sent a lone high surrogate. The stub is what every
writer of `room.log` keeps (ThinkFlowLab#1251), so the half reached the encoder from disk as U+FFFD.

`shared/text-boundary.js` `cut` already answers this everywhere else (ThinkFlowLab#1908/ThinkFlowLab#1912/ThinkFlowLab#1916);
these three sites call it instead. The bound stays a cap: a pair straddling it is dropped
whole, and text inside the bound is untouched.
@HugoXK

HugoXK commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

Verdict requested for this tip: 1d6e92c (tree 29db63e18ba5e1d8d4168ea207f7eee646b4e06b, base 54dad6b)

@Birfy review request

@Birfy

Birfy commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator

Verdict: PASS at 1d6e92c
Written-by: Reviewer
Attestation: 1d6e92c role=Reviewer

Ran at this tip:

$ cd p1961tip/core && TZ=UTC DSH_HARNESS=/opt/flybot node --import ./test/harness-resolve.mjs --test test/room-context.test.js test/room-membership.test.js
# tests 36  # pass 36  # fail 0
$ cp p1961-base-room-context.js p1961tip/core/src/host/room-context.js && cd p1961tip/core && TZ=UTC DSH_HARNESS=/opt/flybot node --import ./test/harness-resolve.mjs --test test/room-context.test.js test/room-membership.test.js
# tests 36  # pass 33  # fail 3

Red first, at this tip: base blob 54dad6b8:core/src/host/room-context.js (blob e9fd9c15) swapped in over the tip file, the branch's two test files kept, turns exactly the three P-305 cases red — :18 the card title, :19 the clipped transcript line, :36 the stored stub reloaded from state.json — and nothing else; the 33 controls stay green. Restored, the same two files are 36/36 at the tip.

Scope: 8 files, exactly the body's table (core/src/host/room-context.js, the two core tests, the doc sentence in the Room context budget paragraph, one changelog.d/ fragment, and the three patches/ds-bot/files/ mirrors). Each mirror is byte-identical to its core/ counterpart (f508d3be source, d81f3808 room-context test, 34e0f18a room-membership test). The source diff is the one cut import plus the three sites; every budget is unchanged (100, room - 1, max - 1) and oneLine's flatten/fold/trim/ellipsis is untouched, so only a pair straddling a bound changes. No existing assertion is edited: room-membership.test.js gains one import line and one test.

CI at this tip: 16 check-runs, all success except enqueue-guard: failure (this PR is not in the queue yet), including tests-ok, package, the four plugin jobs, merge-gate and judge.

Not run here: the full core suite — this checkout has no git and no esbuild/@larksuiteoapi/node-sdk, so test/*.test.js loads 12 files that die on the environment (cannot diff against HEAD, Cannot find package 'esbuild'), none of them touching room-context.js; tests-ok at this tip covers the full run.

Metadata: Reviewer: Birfy present and differs from Owner Dev-2; checkpoint block 1367 chars, summary 595, 8 files, all in the diff — well formed. Added the missing agent-pr label.

@github-actions github-actions Bot added the pass label Nov 1, 2026
@Birfy
Birfy added this pull request to the merge queue Nov 1, 2026
Merged via the queue into ThinkFlowLab:main with commit 4accf1e Nov 1, 2026
17 checks passed
@Birfy
Birfy deleted the dev/2-p305-room-context-code-point-bound branch November 1, 2026 01:39
@github-actions github-actions Bot removed the agent-pr Pull request opened by an agent label Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

core-host: the room-context cuts keep a lone surrogate (board lines, stored status stub, turn transcript)

2 participants