[HugoXK]

Fixes #1912
Owner: Dev
Written-by: Dev-1
Reviewer: Reviewer

Base: 64eec9310d73
Claim: 41c3657700bc4054bf640bc1879cf341
Verified commit: 6c9e5ed97f0f9256186d6a2e9df1535feb813f32 (tree f0f90d9ec4fab1b24e13c8466382e0255ff6a3c3)
Pushed head: 958bf31704b12dc547246b47e98cf47de265f228 — rebuilt by the push path from that tree, so the tree is f0f90d9ec4fab1b24e13c8466382e0255ff6a3c3 on both
Diff: 20 files, +128/−20

What

One exported helper, cut(text, max) in core/src/host/text.js, answers { text, cut } with the longest prefix that stops before a surrogate pair it would split, and the flag says whether anything was dropped. The six host modules that each had their own slice(0, max) now go through it: the oneLine of decisions.js (option 200, why 300, botName 40) and of meetings.js (title 200, question/decision/dissent/options 600, names 80), claimLine in groups.js, the 2,000-unit report cap in diagnostics.js, cleanLabel in flybot-feed.js and clean in browser.js. The limits are unchanged — they still count UTF-16 units — and only the end of the prefix moves, back by the one unit of a split pair, so every value stays at or under its old bound.

Why

#1908 fixed clip/excerpt in text.js, but its boundary helpers codePointStart/codePointEnd stayed module-private, so six other host modules kept cutting by code unit. A cut inside an emoji splits the surrogate pair: the meeting archive is written through a UTF-8 encoder and stores U+FFFD, and decisions.jsonl stores the lone high surrogate \ud83d, which parses back with isWellFormed() === false. Product reproduced both on 0c677207: a meeting decide whose emoji spans the 600-unit cut ends line 9 of decisions/<date>-<slug>.md in bytes 61 61 61 ef bf bd, and a record_decision option at 200 stores \ud83d, which the receipt and list then show as U+FFFD.

Red first

On the unfixed base 64eec931, tests only, implementation untouched:

$ node --import ./core/test/harness-resolve.mjs --test core/test/meetings-persist.test.js
# tests 3
# pass 2
# fail 1
not ok 3 - #1912: a decision cut at an emoji boundary writes no replacement character into the archive
  the archive must not hold U+FFFD
  true !== false

$ node --import ./core/test/harness-resolve.mjs --test core/test/decisions.test.js
# tests 6
# pass 5
# fail 1
not ok 6 - #1912: an option cut at an emoji boundary stays well formed in the log
  the stored option must stay well formed, got: "aaa\ud83d"
  false !== true

The meeting test opens a meeting, records three positions, then decides with 'a'.repeat(599) + '😀tail' and reads the archive back: on the base the file holds U+FFFD and does not round-trip through UTF-8; after the fix it holds no U+FFFD, Buffer.from(written, 'utf8') equals the bytes on disk, and line 9 is a 599-unit prefix of the decision. The decisions test records the same shape at the 200-unit option bound: the base stores \ud83d with isWellFormed() === false, the fix stores the 199-unit prefix with isWellFormed() === true.

After the fix: meetings-persist.test.js 3/3, decisions.test.js 6/6, and the suites of the modules that changed with them (text, groups, diagnostics, flybot-feed, bot-browsers, browser-channel) 70/70.

Files

file	+/−	what
core/src/host/text.js	+4/−0	the exported cut(text, max) built on the existing boundary rule (#1912)
core/src/host/decisions.js	+2/−1	oneLine cuts through cut; decisions.jsonl holds a whole character
core/src/host/meetings.js	+2/−1	oneLine cuts through cut; the meeting .md holds a whole character
core/src/host/groups.js	+2/−2	claimLine (the claim cue in the room notice)
core/src/host/diagnostics.js	+3/−1	the 2,000-unit report cap, with the ellipsis driven by cut
core/src/host/flybot-feed.js	+2/−1	cleanLabel: tool names and remote handles from outside the team
core/src/host/browser.js	+2/−1	clean: the browser text a Bot reads and writes
core/test/meetings-persist.test.js	+20/−1	the red-first meeting archive test
core/test/decisions.test.js	+13/−0	the red-first record_decision test
docs/subsystems/core-host.md	+1/−1	the text.js row names cut and the six call sites it serves
docs/subsystems/plugin-runtime.md	+1/−1	the feed row says cleanLabel bounds through cut
changelog.d/1912-archive-cut-boundary.fixed.md	+1/−0	the entry
patches/ds-bot/0001-core-fixes.patch	+23/−2	generate output: text.js, decisions.js, meetings.js, diagnostics.js and their tests live in 0001
patches/ds-bot/0012-room-membership.patch	+2/−2	generate output: groups.js lives in 0012
patches/ds-bot/0014-flybot-branding.patch	+11/−2	generate output: browser.js lives in 0014
patches/ds-bot/files/src/host/decisions.js	+2/−1	generate output: the files/ mirror
patches/ds-bot/files/src/host/meetings.js	+2/−1	generate output: the files/ mirror
patches/ds-bot/files/src/host/flybot-feed.js	+2/−1	generate output: the files/ mirror
patches/ds-bot/files/test/decisions.test.js	+13/−0	generate output: the files/ mirror
patches/ds-bot/files/test/meetings-persist.test.js	+20/−1	generate output: the files/ mirror

generate was run from zero on the committed tree: it reproduced the committed series byte for byte, git status --porcelain is empty afterwards and canonical reports generate is a no-op, so nothing uncommitted is being carried. The three patches it rewrote are the three that own the touched files.

Ran

$ TZ=UTC ARCH_DOCS_BASE=64eec931 npm run verify              # exit 0
$ node scripts/ds-bot-patches.mjs generate                   # status empty after the commit
$ node scripts/ds-bot-patches.mjs check                      # 3 lines ok
$ node scripts/ds-bot-patches.mjs canonical                  # generate is a no-op
$ ARCH_DOCS_BASE=64eec931 npm --prefix core run arch-docs     # OK (20 changed file(s))

verify at the tip: core 2308, social 541, tree 127, timeline 135, workflow 336, .github/scripts 563, interop 37, live-e2e 24 — all pass, 0 fail; arch-docs: OK (20 changed file(s)) against base 64eec931. The plugin pack stage needs a writable npm cache, which this sandbox only has inside the work area, so that run carried npm_config_cache=<work area>/tmp/npm-cache; nothing else in the command was changed.

Not run

The CI package job (pack check, typecheck, actionlint) and the smoke test.

Checkpoint

{
"pr": 1915,
"claim": "Dev",
"issue": 1912,
"summary": "The host cut archive text by UTF-16 code unit, so a cut inside an emoji stored U+FFFD in decisions.jsonl and in the meeting record. text.js now exports one cut(text, max) that stops before a split surrogate pair, and the six writers that had their own slice(0, max) go through it. Red first on base 64eec93: the meeting archive held U+FFFD and the stored option was the lone surrogate \ud83d; both are green after the fix. Full verify is green at the tip.",
"files": [
"core/src/host/text.js",
"core/src/host/decisions.js",
"core/src/host/meetings.js",
"core/src/host/groups.js",
"core/src/host/diagnostics.js",
"core/src/host/flybot-feed.js",
"core/src/host/browser.js",
"core/test/decisions.test.js",
"core/test/meetings-persist.test.js",
"docs/subsystems/core-host.md",
"changelog.d/1912-archive-cut-boundary.fixed.md",
"patches/ds-bot/0001-core-fixes.patch"
],
"checks": {
"ran": "TZ=UTC ARCH_DOCS_BASE=64eec931 npm run verify",
"result": "green",
"at": "6c9e5ed97f0f9256186d6a2e9df1535feb813f32"
},
"next": "none — ready for review"
}

[HugoXK]

Verdict requested for this tip: 958bf31 (tree f0f90d9ec4fab1b24e13c8466382e0255ff6a3c3, base 64eec93)

@Birfy review request

#1912: #1908 fixed clip/excerpt in core/src/host/text.js, but its boundary helpers codePointStart/codePointEnd stayed module-private, so six other host modules kept cutting archive and label text with their own slice(0, max) by UTF-16 code unit. A cut that lands inside an emoji splits the surrogate pair: the meeting archive goes through a UTF-8 encoder and stores U+FFFD, and decisions.jsonl stores the lone high surrogate \ud83d, which parses back with isWellFormed() === false and then reads as U+FFFD in the receipt and in list. Product reproduced both on 0c677207: a meeting decide whose emoji spans the 600-unit cut ends line 9 of decisions/<date>-<slug>.md in bytes 61 61 61 ef bf bd, and a record_decision option at the 200-unit bound stores \ud83d.

The shape is one exported helper: cut(text, max) in text.js answers { text, cut } with the longest prefix that stops before a pair it would split, and the flag says whether anything was dropped. The six writers that each had their own slice now go through it — the oneLine of decisions.js and meetings.js, claimLine in groups.js, the 2,000-unit report cap in diagnostics.js, cleanLabel in flybot-feed.js and clean in browser.js. The limits are unchanged (still UTF-16 units); only the end of the prefix moves, back by the one unit of a split pair, so every value stays at or under its old bound. clip/excerpt behaviour is deliberately untouched: #1908 and #1910 acceptance depends on it.

Red first, on base 64eec931 with the tests in place and the implementation untouched: meetings-persist 3/2/1 (the archive must not hold U+FFFD, true !== false) and decisions 6/5/1 (the stored option must stay well formed, got: "aaa\ud83d", false !== true). At this tip those files are 3/3 and 6/6, and the suites of the six changed modules together are 70/70.

TZ=UTC ARCH_DOCS_BASE=64eec931 npm run verify exits 0 at the tip per the author's run: core 2308, social 541, tree 127, timeline 135, workflow 336, .github/scripts 563, interop 37, live-e2e 24; arch-docs: OK (20 changed file(s)). generate from zero reproduced the committed series (it rewrote 0001, 0012 and 0014, which own the touched files), a second run left git status empty, check printed its three ok lines and canonical says generate is a no-op. The CI package job and the smoke test were not run locally — no startup path is touched.

Checks at this tip: 14 runs, 13 green; enqueue-guard is red only with no Verifier report names the tip 958bf31704b1, which is this request. mergeable: true, non-draft, maintainer_can_modify: true.

Owner: Dev
Written-by: Dev-1

[Birfy]

Verdict: PASS at 958bf31
Written-by: Reviewer
Attestation: 958bf31 role=Reviewer

Ran at this tip:

$ DSH_HARNESS=/opt/flybot node --import ./core/test/harness-resolve.mjs --test core/test/decisions.test.js core/test/meetings-persist.test.js core/test/text.test.js core/test/groups.test.js core/test/diagnostics.test.js core/test/flybot-feed.test.js core/test/bot-browsers.test.js core/test/browser-channel.test.js
# tests 79  # pass 79  # fail 0
$ # same eight files, with the seven changed sources (text.js decisions.js meetings.js groups.js diagnostics.js flybot-feed.js browser.js) written back to their base 64eec931 blobs, tests untouched
# tests 79  # pass 77  # fail 2

Review points:

• Red first holds. With only the seven source files moved back to 64eec931, exactly the two new #1912 tests fail and nothing else does: an option cut at an emoji boundary stays well formed in the log (the stored option must stay well formed, got: "aaa\ud83d") and a decision cut at an emoji boundary writes no replacement character into the archive (the archive must not hold U+FFFD). At the tip the same 79 are green. The tests pin this change and not the surrounding suite.
• The fix is one helper and six callers. cut(text, max) in core/src/host/text.js is built on the existing codePointEnd, and the six slice(0, max) sites now go through it. Limits are unchanged (still UTF-16 units); only the end of a prefix that would split a pair moves back one unit, so every value stays at or under its old bound. clip/excerpt are not touched, so #1908/#1910 acceptance is not disturbed.
• Scope matches #1912. 20 files: the seven sources, the two tests, two doc lines, changelog.d/1912-archive-cut-boundary.fixed.md, and the patches/ds-bot/ mirrors (0001, 0012, 0014 and files/…) kept in step, which arch-docs: OK (20 changed file(s)) covers.
• Checks at this tip. 14 runs, 13 green including tests-ok, package, all four plugins (*), merge-gate, judge; the only red is enqueue-guard, whose reason is no Verifier report names the tip 958bf31704b1 — this verdict is that report. mergeable: true, non-draft, maintainer_can_modify: true.
• I did not re-run the full npm run verify locally; the claim in the body is the author's, and the tip's tests-ok plus the Node 24 job are green over it.

Metadata: Owner: Dev, Reviewer: Reviewer, checkpoint block present (pr 1915, claim Dev, issue 1912, summary 456 chars, 12 files, 1178-char block). Fork head, so agent-pr is not asked of it. Tier risk:T2.