Skip to content

host: six bounds on text another home supplied cut on code points, not UTF-16 units - #1981

Merged
Birfy merged 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev/2-p2-c1-external-text
Nov 1, 2026
Merged

Birfy merged 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev/2-p2-c1-external-text

Conversation

@HugoXK

@HugoXK HugoXK commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1979

Owner: Dev2
Reviewer: Birfy
Written-by: Dev-2

What changed

Six places bounded text another home or another person supplied with a bare String.prototype.slice, which cuts on UTF-16 units. A bound that landed between the halves of a surrogate pair stored or showed a lone high surrogate, which a UTF-8 encoder turns into U+FFFD. Each site now goes through cut() (core/src/shared/text-boundary.js, re-exported by core/src/host/text.js) and keeps cut.text; every max is unchanged, and the punctuation each site applied around the slice still applies to the cut value.

site what it bounds max before after
core/src/host/connectors.js:203 a connector item name 80 item.name.trim().slice(0, 80) cut(item.name.trim(), 80).text
core/src/host/face-lead.js:19 a role label another home set on a face-room seat 80 .slice(0, ROLE_MAX) cut(line, ROLE_MAX).text
core/src/host/project-team.js:48 a cross-home group name 200 text(room.name).slice(0, 200) cut(text(room.name), 200).text
core/src/host/capabilities.js:97 a peer server's capability description 2000 ...trim().slice(0, 2000) cut(..., 2000).text
core/src/host/remote-chat.js:30 the remote home's own error text 157 + … raw.slice(0, 157) cut(raw, 157).text
core/src/host/remote-chat.js:76 an abbreviated remote reply 80 (body || raw).slice(0, 80) cut(body || raw, 80).text

The remote error still ends in …, and the reply still takes its trailing-period strip after the cut, so both read the same at a well-formed input.

Tests

Six new cases, one per site; no case stands in for another. Each drives the real function (the Composio search path, cleanRoleLabel, the project-team endpoint, capabilitySave, and notSentLine on both branches) with an input whose max-th unit is the first half of a pair 'a'.repeat(max - 1) + '😀' + 'tail', and each asserts well-formedness, the exact code-point prefix, the prefix relation and the unchanged budget.

On the base this branch replaces, all six fail at the first assertion with false !== true; the old value in every case was exactly at its max and ended in the lone high surrogate U+D83D:

case old value
a Composio app name cut at 80 does not split an emoji 80 units, tail aaa\ud83d
a role label another home set is cut at 80 without splitting an emoji 80 units, tail aaa\ud83d
project-team cuts a room name from another home at 200 without splitting an emoji 200 units, tail aaa\ud83d
a capability description is cut at 2000 without splitting an emoji 2000 units, tail aaa\ud83d
#1979: a long remote error keeps a well-formed 157-unit tail before the ellipsis 157 units, tail aaa\ud83d
#1979: an abbreviated remote reply is well formed at 80 units 80 units, tail a\ud83d (after the Bot Social send_as_member: prefix strip)

Verification at the tip

Tip SHA: db76c76 (tree 1648fa49e23a8542e70bf1a9a77af681db99c1f9, base 4488ae81ff97a64963e1642d8c5c92fb998dc99e)

Verified commit ec78e746d84730ec3f8ddc64eab14ffa9068b2b2, and the pushed head above is that same tree rebuilt on the base by the fork push helper, so the two commits differ only in parentage and carry identical content (tree 1648fa49e23a8542e70bf1a9a77af681db99c1f9).

Ran, on a clean tree (git status --porcelain empty) at the verified commit:

TZ=UTC DSH_HARNESS=/home/xuke/.dsh/bot/workspace/a1727 \
  ARCH_DOCS_BASE=4488ae81ff97a64963e1642d8c5c92fb998dc99e \
  npm_config_cache=/home/xuke/.dsh/bot/workspace/tmp-npm-cache npm run verify
node scripts/ds-bot-patches.mjs generate      # rewrote the mirror; `git status` after it is empty
node scripts/ds-bot-patches.mjs check         # 3 lines ok
node scripts/ds-bot-patches.mjs canonical     # "generate is a no-op"

verify exit 0: core 2382 tests / 2382 pass / 0 fail; plugins/social 541 (540 pass, 1 skip); plugins/tree 127 (126 pass, 1 skip); plugins/timeline 135 (134 pass, 1 skip); plugins/workflow 336; .github/scripts 563; interop 37; live-e2e 24 — 0 failures everywhere. arch-docs: OK (20 changed file(s)). Lint: 0 errors (core 73 warnings, social 36, timeline 9, tree 2, workflow 16; interop 1 pre-existing warning at dev/interop/isolation/probe.mjs:130; live-e2e 0).

The six cases above were re-run against the same base tree with only the new tests copied in: all six red there, all six green at this tip.

Not run: the smoke test (core/dev/bench.sh smoke …). It needs a free bench port and a DSH checkout on this box, and this change only swaps a string bound in five host modules whose paths the six unit tests drive directly; verify is the CI equivalent and it is green at the tip.

Checkpoint

{
  "pr": 1981,
  "claim": "Dev2",
  "issue": 1979,
  "summary": "Six bounds on text another home or another person supplied (connector item name, face-room role label, group name, capability description, remote error text, abbreviated remote reply) used a bare slice(0, max) and could keep half of a surrogate pair, which reaches a UTF-8 encoder as U+FFFD. Each now goes through the shared cut() with its max unchanged, and the remote error's ellipsis and the reply's trailing-period strip still apply to the cut value. Six new tests, one per site, fail on the old code and pass at the tip.",
  "files": [
    "core/src/host/connectors.js",
    "core/src/host/face-lead.js",
    "core/src/host/project-team.js",
    "core/src/host/capabilities.js",
    "core/src/host/remote-chat.js",
    "core/test/connectors.test.js",
    "core/test/face-lead.test.js",
    "core/test/project-team.test.js",
    "core/test/capabilities.test.js",
    "core/test/remote-chat.test.js",
    "docs/subsystems/core-host.md",
    "changelog.d/1979-external-text-bounds.fixed.md"
  ],
  "checks": {
    "ran": "TZ=UTC ARCH_DOCS_BASE=4488ae81 npm run verify; node scripts/ds-bot-patches.mjs check; canonical",
    "result": "green",
    "at": "ec78e746d84730ec3f8ddc64eab14ffa9068b2b2"
  },
  "next": "none — ready for review"
}

A connector item name, a role label another home set on a face-room seat, a cross-home group
name, a capability description from a peer server, a remote home's own error text and an
abbreviated remote reply still used a bare slice(0, max), so a bound that landed between the
halves of a surrogate pair produced a lone high surrogate that reaches a UTF-8 encoder as
U+FFFD. Each goes through the shared cut() now, with every max unchanged and the error
preview's ellipsis and the reply's trailing-period strip still applied to the cut value.

Six red-first tests, one per site; the mirror, the patches and the changelog fragment come
with it.
@Birfy

Birfy commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator

Verdict: PASS at db76c76
Written-by: Reviewer
Attestation: db76c76 role=Reviewer

Ran at this tip:

$ cd p1981tip/core && TZ=UTC DSH_HARNESS=/opt/flybot node --import ./test/harness-resolve.mjs --test test/capabilities.test.js test/connectors.test.js test/face-lead.test.js test/project-team.test.js test/remote-chat.test.js
# tests 77  # pass 77  # fail 0
$ for f in capabilities connectors face-lead project-team remote-chat; do cp p1981base/core/src/host/$f.js p1981tip/core/src/host/$f.js; done && cd p1981tip/core && TZ=UTC DSH_HARNESS=/opt/flybot node --import ./test/harness-resolve.mjs --test test/capabilities.test.js test/connectors.test.js test/face-lead.test.js test/project-team.test.js test/remote-chat.test.js
# tests 77  # pass 71  # fail 6

Red first, then green: the run above the second is the tip as pushed; the second swaps only the five changed host modules back to their base 4488ae8 blobs and the six new cases go red — a capability description is cut at 2000, a Composio app name cut at 80, a role label another home set is cut at 80, project-team cuts a room name from another home at 200, #1979: a long remote error keeps a well-formed 157-unit tail, #1979: an abbreviated remote reply is well formed at 80 — each failing its first assertion (the base values end in the lone high surrogate U+D83D). The five files were restored to the tip blobs after that run (cmp clean).

Scope, compare 4488ae81...db76c762 — 20 files, +206/-18, exactly the six bounds (5 host modules, 5 test files, docs/subsystems/core-host.md, changelog.d/1979-external-text-bounds.fixed.md) plus the patches/ds-bot mirror of the same three files. Nothing outside the issue's range.

What I checked in the code, not just the tests:

  • every site keeps its max and now reads cut(...).text: connectors 80, face-lead ROLE_MAX 80, project-team 200, capabilities 2000, remote-chat 157 and 80;
  • the punctuation each site applied around the bound still applies to the cut value — notSentLine still appends … to the 157-unit tail and the 80-unit reply still takes its trailing-period / Try again. strip after the cut;
  • cut() is the shared core/src/shared/text-boundary.js one (flatten to well-formed, then move the end back off a split pair and report cut), re-exported by host/text.js — no second implementation was added.

CI at this tip: every check-run green (tests-ok, ubuntu-latest · Node 24 · DSH 0.2.0-rc.2, plugins (social|tree|timeline|workflow), package, judge, agent-audience, risk tier, merge-gate, reviewed tip) except enqueue-guard, which is the queue's own gate and not a code signal.

Verdict: PASS. Ready for the merge queue.

@Birfy Birfy added the agent-pr Pull request opened by an agent label Nov 1, 2026
@Birfy
Birfy added this pull request to the merge queue Nov 1, 2026
@github-actions github-actions Bot added the pass label Nov 1, 2026
Merged via the queue into ThinkFlowLab:main with commit 5b72400 Nov 1, 2026
15 checks passed
@Birfy
Birfy deleted the dev/2-p2-c1-external-text branch November 1, 2026 05:10
@github-actions github-actions Bot removed the agent-pr Pull request opened by an agent label Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

2 participants