Skip to content

afx send: long architect→builder messages truncated or delivered empty-body #1564

Closed

Description

@amrmelsayed

Two delivery failures in one session (2026-08-27), same workspace, architect → builder direction, both on longer messages:

  1. Truncation: a ~1,900-char single afx send arrived at the builder as only its tail fragment ("...of the synthesis doc." — the final ~30 chars). Sender saw [ok] Message delivered.
  2. Empty body: a ~900-char send arrived as just the message-header delimiter block with no body at all ("your last message arrived empty (just a '###...' delimiter, no body — and nothing in my inbox)"). Sender again saw [ok] Message delivered.

Short messages (≤ ~600 chars) delivered fine throughout the same session, including immediately after each failure — resending the same content split into 2 parts worked.

Impact: silent content loss with a success result at the sender — the worst failure shape for gate-decision traffic. In case 2 the builder was correctly paranoid and held at a porch gate rather than treating an empty message as approval, but that safety came from agent discipline, not the transport.

Suggestions: (a) size limit enforced loudly at send time (reject or auto-chunk, never silently truncate); (b) delivered-length echo in the send result so sender can detect mismatch; (c) if a render-gate/PTY-write constraint is the culprit, route oversize bodies through the durable mailbox (afx inbox) automatically with a pointer injected instead.

Env: afx/codev 3.3.1, Tower 3.3.1, macOS 26, architect and builder terminals in the same workspace.

Activity

  1. self-assigned this
    on Aug 27, 2026
  2. waleedkadous commented on Sep 1, 2026

    @waleedkadous
    Contributor

    Root cause found (2026-08-30 code investigation) and fix lane opened. Mechanism: the write window is length-proportional — bodies ≥4 lines go out one line per 10ms with no bracketed paste and no mid- or post-write verification, so a ~1,900-char send holds a ~400ms exposure window vs ~100ms for short sends (matching the ~600-char threshold and why splitting the resend worked); any composer disruption mid-window discards the accumulated draft and the trailing Enter submits only the tail (tail-only shape) or nothing (delimiter-only shape). '[ok] Message delivered' is a socket-level claim — no echo/ACK exists before markDelivered. The biggest disruptor (a Tower-side interrupt/escape landing mid-write) is now structurally impossible since PR #1492 merged; the residuals (receiver-side disruption + the false success receipt) are prescribed in #1573 — echo-verification before 'delivered' (verify-or-retry), settle-before-write, and this issue's own suggested loud size limit + delivered-length echo. Keeping this open until #1573 lands and the field shape stops recurring.

  3. waleedkadous commented on Sep 1, 2026

    @waleedkadous
    Contributor

    Fresh live specimen, 2026-09-01 11:53 UTC, architect terminal, Tower 3.3.1 (pre-#1492/#1573 — expected, recorded for the verification baseline): mailbox row e27b5c99 (builder-bugfix-1574 → main, body 965 chars) was marked delivered, but the receiving terminal rendered only the final ~25 characters ('…ather than half-do it.') plus the frame footer. Tail-only shape, length-dependent, false success receipt — exactly this issue's signature. Full body was recovered losslessly from the mailbox table (mailbox-first paying for itself). Keep for before/after comparison once #1573's verify-or-retry lands in an installed release.

  4. added a commit that references this issue on Sep 1, 2026
  5. waleedkadous commented on Sep 1, 2026

    @waleedkadous
    Contributor

    Fix merged in #1577 — staying open for field verification

    #1573 shipped the write-edge changes that should close this: a loud 48 KB body limit (this issue's ~1,900-char message was well under it, so the limit is not the cure here — it removes the unbounded case) and, the part that matters for this report, echo verification before markDelivered.

    The mechanism behind the symptom you saw: [ok] Message delivered meant only "frames queued on a connected socket". session.write returns true whenever the socket object is connected, so a composer that ate the leading bytes was indistinguishable from a clean delivery, and the row was marked delivered for a message the agent never saw whole. Delivery now samples the message's header on the receiving terminal's rendered mirror before the write and requires it to appear as a result of that write; otherwise the row is held and redelivered rather than reported as success. The sender is told held, not delivered.

    Worth stating plainly: the tail-only shape you reported (~1,900 chars arriving as its final ~30) is exactly why the check matches the header and never the footer — a footer needle would have certified this corruption.

    Why this stays open: the fix is verified by unit and regression tests, not by a reproduction of your original failure. It ships in the next release. Please close it once a long architect→builder send has been seen to land intact — or reopen the conversation here if a truncated message reaches a builder while the sender still reads delivered, since that would mean the header matched something it should not have.

    Known gap that could still bite this issue: a harness rendering on the alternate screen buffer (agy) has no scrollback, so a message longer than one viewport may never confirm and would be redelivered repeatedly — tracked in #1578.

  6. amrmelsayed commented on Sep 4, 2026

    @amrmelsayed
    CollaboratorAuthor

    Field evidence, 2026-09-04 (post-fix stack, local-install)

    Environment: Tower restarted 09:51:56Z onto a local install built from main (self-reports "3.3.2" only because main's version field is unbumped pre-release). Verified by marker inspection of the installed dist: contains both the #1573/PR #1577 write-edge verification this issue is waiting on and the #1585 completed-write point-of-no-return (plus the #1511 WELCOME identity hydration). So today's traffic ran on the full delivery-integrity stack.

    Passive evidence (architect↔architect and builder→architect): roughly a dozen long messages, 700 B–1.9 KB, crossed the mailbox during a single working session — six relays main→streamdeck (up to ~1.8 KB), multiple builder→architect status messages with ~1.9 KB attached bodies, cross-workspace exchanges, and merge-coordination traffic main↔vscode. All arrived intact end-to-end: recipients quoted specific details back accurately (commit SHAs, constraint lists, verbatim rulings), several through the held→auto-deliver path (busy prompt → mailbox hold → delivery on clear prompt), with no truncation, no empty bodies, no replay/re-injection.

    Active test (the direction this issue recorded failing — architect→builder, long body): a deliberate 2,344-byte message (afx send builder-pir-1536 --file …) mirroring the original ~1,900-char failure size, carrying a start token, an end token, and an echo-back instruction (both tokens + received character count). Write edge reported delivered. Result: in flight — will follow up on this comment with the builder's echo when it lands.

    Not covered by this evidence: the #1578 pathological path (systematically unverifiable harness — alt-screen agy, message longer than one viewport). That issue needs implementation, not verification, and no amount of clean deliveries here bears on it.

  7. amrmelsayed commented on Sep 4, 2026

    @amrmelsayed
    CollaboratorAuthor

    Active test result (follow-up to the previous comment): builder-pir-1536 echoed back at 11:08Z:

    1564-test: start=KHUFU-7391 end=SNEFERU-2604 chars=2218
    
    • Start token verbatim — rules out the truncation mode (original specimens delivered only the final ~25–30 chars; the start token is the first thing lost).
    • End token verbatim — rules out tail loss.
    • 2,218 of 2,224 file bytes reported (99.7%) — the 6-byte delta is consistent with trailing-whitespace/framing normalization at the wrapper, and categorically inconsistent with either recorded failure mode (~98% loss, or empty body). Caveat: the count comes from an agent, so treat it as approximate; the token pair is the load-bearing proof.
    • Delivered on the first write (sender saw [ok] Message delivered, no hold, no redelivery, no replay).

    With this, the field evidence covers all three direction pairs on the post-fix stack, including the architect→builder long-body direction this issue was filed on. Close/keep-open call left to the issue owner.

  8. amrmelsayed commented on Sep 5, 2026

    @amrmelsayed
    CollaboratorAuthor

    Scope correction to the field evidence above. Hours after the clean run, a peer workspace reported two recipient-confirmed truncation cases with version receipts — one on this same 3.3.3 stack (1,148-byte send, tail-only delivery, sender flagged "unverified — header not seen"). Details in the #1567 comment of 2026-09-05.

    So the evidence above should be read as: the failure modes are ruled out on the tested path (claude-harness recipient, idle composer, 2.3 KB body, first-write verified), not universally. The live reproduction is state- or harness-dependent and is now tracked as a confirmed live defect on #1567. This issue's close call should wait on #1567's resolution.

  9. waleedkadous commented on Sep 8, 2026

    @waleedkadous
    Contributor

    Closing: fixed by PR #1644 (issue #1567). Root cause was the kernel PTY input queue splitting any single write over ~1022 bytes and the TUI's paste heuristic discarding the first chunk; long bodies now go out as one bracketed paste chunked at 512 bytes with Enter outside the bracket. Field-verified 2026-09-08 10:55Z on the installed main: a 1.6 KB multi-line --file send to a live builder arrived intact (header, body, closing line) and Tower recorded delivered, not delivered-unverified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area/towerArea: Tower server / agent farm CLI

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions