Skip to content

feat(api,admin): Slack bot transport — one message per incident, the story in its thread, the state as a reaction (CLEAN-165) - #140

Merged
maksymhryzodub-prog merged 6 commits into
mainfrom
feat/CLEAN-165-slack-bot-transport
Oct 15, 2026
Merged

maksymhryzodub-prog merged 6 commits into
mainfrom
feat/CLEAN-165-slack-bot-transport

Conversation

@maksymhryzodub-prog

@maksymhryzodub-prog maksymhryzodub-prog commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Jira: CLEAN-165 — follow-up to CLEAN-139 (#134). Spec, plan, contracts, tasks and the proof log in specs/021-slack-bot-transport/.

What it does

A second way to tell the team about an agent going down: a Slack bot beside the incoming webhook of CLEAN-139. With the bot, the channel shows one message per incident, rewritten as the incident moves; every event is a reply in that message's thread; the bot's reaction on the message is the state. The shape was settled with the owner over three live mockups in the alerts channel on 2026-10-14.

🔴 @channel Test Chatbot — incident #12 · down since 4:04 PM · OOMKilled
🔴 Test Chatbot — incident #12 · down since 4:04 PM · OOMKilled · 2 reports
🟢 Test Chatbot — incident #12 · up, watching since 4:09 PM · OOMKilled · 2 reports
✅ Test Chatbot — incident #12 · resolved 4:19 PM · downtime 5 min · 2 reports

Thread: Failed → Failed again (at most one a minute; the count on the root catches up on the sweep) → Running again → Resolved / No further reports / Stopped by owner / Deleted. @channel only in the first version of the root.

How

  • API — ISlackBotNotifier (post / update / setReaction over fetch, Slack's errors mapped to words the owner can act on, token scrubbed from anything quoted). AgentEventDestination gains kind (webhook | bot) and the bot's token + channel; saving one kind nulls the other's secret; getDestination() stays the only read that returns a secret. The outbox grows joined / running beside opened / closed, with a per-incident seq and an ordered claim (NOT EXISTS earlier pending seq) so a thread never shows Resolved above Failed again. The incident keeps its transport, its root (slackChannel, slackTs) and the one reaction it holds; every rewrite renders from the row, so late or out-of-order deliveries cannot leave a stale line; a root that could only be posted after the close arrives in its final state without the mention. AgentIncident.number is a sequence, backfilled in openedAt order.
  • Console — Settings → Notifications chooses the transport, takes the bot's two fields with a four-step guide; #12 in the incident list and its search.
  • Tools — set_ / get_notification_destination take and show the kind and the channel; list_agent_incidents returns number. tool-secrets.spec.ts covers botToken.
  • Webhook — byte for byte today's two messages; joins, up-turns, stops and deletes queue nothing on it.

Migration

One, additive, with backfills: kind: slack → webhook; AgentNotification.seq (opened → 0, closed → 1) replacing the (incidentId, kind) unique; AgentIncident.number renumbered by openedAt with the sequence moved past the max; transport = webhook for incidents that were notified. Runs on migrate deploy.

Verified

  • api: bun run build; the whole jest suite — 152 suites, 2130 tests — 223 of them in agent/event + tool-secrets. eslint --quiet + Prettier (LF) on every changed file.
  • admin: bun test slices (195), npx nuxt typecheck, bun run locale:check; the five changed/new components compiled by the dev server.
  • Against the local Postgres still holding the CLEAN-139 stress data (261 incidents, 60 000 events): migrate deploy applied the migration, migrate diff --exit-code → 0; numbers 1…261 distinct and strictly increasing with openedAt; seq 0/1 on the existing rows; kind: slack → webhook; transport set only where a message had gone out; the sequence at 262.
  • Through the code, with the owner's bot and channel, two API processes on that database (Slack side read in the channel, Ranch side in the tables): PUT bot destination → view carries kind, hint and channel, never the token; test delivered; xoxp-, a channel name and a missing field refused with the contract's words. Incident #262: opened (root with @channel, 🔴, "Failed" reply) → six reports inside a minute → one "Failed again", count 7 on the root, stale flag cleared by the sweep → Ranch's own drift sweep reported the agent → a third reply "Reported by: Ranch" → aged past the quiet period → recovered, "Resolved" reply, ✅, root rewritten. #263: the same failure posted four times in parallel at both processes → one incident, one root, one "Failed again"; aged → unconfirmed, 🟢. #264: agent stopped → stopped, "Stopped by owner", ⏹, no new channel message. Switch to the webhook: the orphaned bot incident's remaining rows skipped; the next incident, opened by Ranch's own watch, got exactly opened and closed through the webhook — the CLEAN-139 messages. Every delivery ok.

Not verified

  • "Running again" through the code: without a cluster nothing moves an agent to running through the gateway, so that reply and the 🟢 "up, watching" rewrite are covered by the flow spec only.
  • The outage ordering (block slack.com, let rows queue, unblock): needs an admin hosts-file edit; the ordered claim ran on Postgres through every incident above, the out-of-order case is in the flow spec.
  • Screens in a browser: compiled, not clicked.

Twin consoles

admin changed. app has no agent/event sub-slice (grep -rl 'agentEvent\|agent-events' app/slices → 0 files, as for CLEAN-139): nothing to mirror.

After merge

Owner: Settings → Notifications → Slack bot → token + channel id → Send test. The Slack side (scopes chat:write, reactions:write; reinstall; invite; channel id) is in docs/operations/agent-events.md. The API pod needs egress to slack.com.

🤖 Generated with Claude Code

maksymhryzodub-prog and others added 6 commits October 14, 2026 16:43
…N-165)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…its thread, the state as a reaction (CLEAN-165)

A second INotifier port for a bot token: post (returns ts), update,
setReaction, over fetch like the webhook. The destination row gains a
kind (webhook | bot) and the bot's token + channel; saving one kind
nulls the other's secret; the secret never leaves getDestination().

The outbox grows joined / running beside opened / closed, with a
per-incident sequence and an ordered claim (NOT EXISTS earlier pending
seq) so a thread never shows Resolved above Failed again. "Failed again"
is queued at most once a minute; a swallowed report marks the root stale
and the 30 s sweep rewrites it. The incident keeps its transport, its
root (channel, ts) and the one reaction the bot holds; every rewrite
renders from the row, so late or out-of-order deliveries cannot leave a
stale line. A root that could only be posted after the close arrives in
its final state, without the mention. Side-effect failures (a lost
rewrite, a reaction) land on the destination's last delivery; the reply
itself still counts as sent.

AgentIncident.number: a sequence, backfilled in openedAt order. The
webhook keeps today's two messages byte for byte; joins, up-turns,
stops and deletes queue nothing on it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ns; incidents carry a number (CLEAN-165)

Settings → Notifications offers the two kinds — incoming webhook or
Slack bot — with the bot's token and channel id, a four-step guide for
the Slack side, and the note that saving one kind replaces the other.
The store hands the input on and keeps nothing of it; the view carries
the kind, the hint and the channel, never a secret.

AgentIncidentDto.number is returned by the API and shown as "#12" in
the incident list; the loaded-rows search matches it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… implementation (CLEAN-165)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ion pass (CLEAN-165)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…res and the owner's bot (CLEAN-165)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant