Skip to content

notify: one Feishu send can post the same message twice — a POST whose answer was lost is retried, and the key never leaves the process #2660

Closed

Description

@Birfy

Written-by: Chief

What happens

createNotifier() says in its own header that it keeps "one idempotency key per reminder, so a retried send cannot arrive twice" (core/src/host/notify.js:4), but the key never leaves the process:

  • send() builds the transport message from { channel, text } only (notify.js:145) and keeps key in a local sent map (notify.js:108, :142).
  • The Feishu transport answers { ok: false, error: 'network' } whenever the request threw — including a timeout (core/src/host/connectors.js:284-292: AbortSignal.timeout + withDeadline, then catch { return { ok: false, error: 'network' } }).
  • 'network' is in RETRYABLE (notify.js:32), so the loop at notify.js:102-106 sends the same body again, up to 3 attempts, SEND_TIMEOUT_MS = 60 s each (connectors.js:44).

The only shipped transport is the Feishu custom-bot webhook, whose body is { msg_type: 'text', content: { text } } (connectors.js:287). A Feishu webhook has no idempotency field, so the receiver has nothing to dedupe on: a POST Feishu accepted but whose answer we never got is posted again, and the same message shows up 2–3 times in the chat.

Reachable from

Settings → Connectors → Feishu (webhook mode) → Send box: core/src/client/connectors.js:250 (the button is disabled while busy, so this is not a double click) → connectorSend core/src/client/index.js:651 → endpoint connector-send core/src/host/channel.js:532 → core/src/host/connectors.js:1009 → notifier.send({ key: randomUUID(), text }) at :1019.

The fresh key per call is right (two messages with the same text are two messages). The duplicate comes from the retry inside one call: a timeout is read as "not delivered" although nothing can tell a lost answer from a lost message.

Reproduction (ran here, Node v22.23.3, main e32d5a6143aaaf1554a12cd73d1fb65b3cbf5390)

Real feishuTransport + real createNotifier, a fake fetch that records the body and then never answers (the lost-ack shape):

result: {"state":"sent","ts":"om_1","attempts":2}
webhook POSTs made: 2
bodies identical: true
body: {"msg_type":"text","content":{"text":"standup in 5 minutes"}}

Every attempt hanging instead gives the worst case: {"state":"failed","error":"network","attempts":3,"permanent":false}, posts: 3 — three identical messages.

Falsified, so they are not the defect: the same key handed to send() twice does send once ({ state: 'duplicate' }, notify.js:142); the inbound side does dedupe replays (feishu-inbound.js:180, 512-entry event_id ring); a refused connection (ECONNREFUSED) is retried 3 times today, which is correct and must stay.

Fix scope (decided — no owner round-trip needed)

At-most-once for a POST that may already have been written.

  • feishuPost must answer an error that notify.js's RETRYABLE does not contain when the request timed out or was aborted after being written, so readAnswer marks it permanent (notify.js:44-45) and it gets one attempt.
  • A connection that never opened (refused / DNS) stays retryable exactly as today; 429 and 5xx keep their current retry.
  • The user-facing sentence must say what happened instead of "refused": at connectors.js:1021 and in feishuCheck (:314) say the message may already have been posted, so check the chat before sending again.
  • Out of scope: feishuReply / feishuAppSend (single attempt today), the room-traffic policy (Feishu connector, phase 1: this instance posts into one Feishu group #1263), and the notify.js scheduler wiring.

Acceptance

  • A new test in core/test/ that fails on e32d5a6143: a fake fetch that records the body and never answers → exactly 1 POST for one send(), and the result names the unconfirmed delivery (today it makes 3).
  • A test proving a refused connection still retries (3 attempts) and a 429 still retries honouring Retry-After.
  • npm run verify green; changelog fragment changelog.d/notify-webhook-duplicate.fixed.md.

Activity

  1. self-assigned this
    on Nov 8, 2026
  2. added
    bugSomething isn't working
    agent-readyMaintainer-approved spec: an agent may take it. Only maintainers set this; never in a template
    area:automationScheduled tasks and automation
    on Nov 8, 2026
  3. Birfy commented on Nov 8, 2026

    @Birfy
    CollaboratorAuthor

    Written-by: Chief

    Scope for this one (so it can go straight to implementation, no re-triage):

    Files: core/src/host/connectors.js (feishuPost timeout classification, the failure sentence at :1021, feishuCheck at :311-317), core/src/host/notify.js (header comment at :4 only), one new core/test/*.test.js, changelog.d/notify-webhook-duplicate.fixed.md, regenerated patches/ds-bot/.

    Behaviour: a request that may already have been written (timeout / abort) is not repeated; a connection that never opened (ECONNREFUSED/ENOTFOUND/EAI_AGAIN) still retries to 3; 429 and 5xx keep today's retry. Acceptance: one send() against a fetch that never answers makes exactly 1 POST (today 2-3, reproduced above), a refused connection still makes 3, npm run verify green.

    Out of scope: feishuReply/feishuAppSend, the timeout values, the room-traffic policy (#1263).

  4. removed
    agent-readyMaintainer-approved spec: an agent may take it. Only maintainers set this; never in a template
    on Nov 8, 2026
  5. added 2 commits that reference this issue on Nov 9, 2026
    d8a896c
    af9a30b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:automationScheduled tasks and automationbugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions