Skip to content

A pull queue: the lead enqueues work, an idle member of the role claims it #1247

Closed

Description

@Birfy

Owner: Dev
(Proposed by Chief from the owner's request on 2026-10-26; Triage: confirm or move it.)

Goal (one observable behavior)

Today work is pushed: the lead writes a card with assignee and wakes that one member with an @, so dispatch is serialized through the lead and idle capacity sits unused between beats. After this change, work is pulled: the lead enqueues an item (no assignee, a role, a priority) and an idle member of that role claims it. A queue item is handed out to exactly one member, and the only thing that wakes a member is a matching item entering the queue — not a chat mention.

What exists already (read this before designing)

The machinery is mostly there; do not build a second board.

  • core/src/host/tasks.js:22 — ACTIVE = open|claimed|working|blocked, transitions at :25, record()/transition() at :187/:191. A task already has an assignee, a column, progress and a note.
  • core/src/host/task-drive.js:20 — DRIVING = open|claimed|working; driveOnce() at :177 already wakes a Bot for its own task, with backoff TASK_DRIVE_BACKOFF_MS (:12).
  • core/src/shared/task-board.js:19 — columnOf, compareForWork (:37, the work ordering to reuse).
  • core/src/host/room-context.js:149 — formatRoomBoard, the text the room turn already carries.

So the gap is not "a task list": it is (a) an item with no assignee and a role instead, (b) a single-winner claim, and (c) waking only the members the item matches.

Proposal

  1. Enqueue. track_tasks accepts queue: true together with role (one of the room's roles) and priority; assignee stays unset. The item is open and belongs to the room, not to a Bot.
  2. Claim. A member calls track_tasks with claim: true and gets the highest-priority open item whose role matches it, or null. The transition open → claimed is compare-and-set: two concurrent claims, one item, one winner.
  3. WIP = 1. A member that already holds an in-progress card gets null and claims must not touch its existing card. (This is what keeps a pull queue from piling five things on one Bot.)
  4. Wake on enqueue only. Enqueuing wakes idle members whose role matches the item, at most one per role, and the wake carries the item's title and id only — not the body — so a wake costs a short turn, not a 150k-prompt one. The lead no longer @s anyone for ordinary dispatch; @ stays for a decision, a blocker, or a correction.
  5. Blocked round-trip. A member that cannot finish returns the item with a one-line reason: blocked → open, plus retryAt; it is not offered again before that time (task-drive.js backoff already has the shape).
  6. Visibility and fairness. The room board gains one line per role: queue depth and the age of the oldest waiting item, so starvation is visible to a person without reading the whole board. A member is not woken for a lower-priority item while it holds a higher-priority claim in flight.
  7. One board. The item is the same task as today: claiming it sets column: in-progress and the same progress field, so delivery view, room presence (#1226) and the write-back keep working unchanged.

Acceptance criteria

A test that fails without the change, per point:

  1. Single winner: two concurrent claim calls for one item → exactly one returns the item, the other returns null, and the item has exactly one owner afterwards.
  2. Role match: a Dev-role member claiming a role: Reviewer item is refused with a named reason (not-in-role), not a silent null.
  3. WIP = 1: a member holding an in-progress card gets null from claim, and its existing card's column and progress are unchanged.
  4. Wake list: enqueuing a role: Dev item wakes exactly the idle Dev-role members and no one else (assert the wake list, not the message text).
  5. Cheap wake: the wake text contains the item title and id but not the item body (assert length and the absence of the body's first sentence).
  6. Blocked round-trip: blocked with a reason returns the item to open with retryAt, and a claim before retryAt does not return it.
  7. Board line: the room board carries one line per role with depth and oldest-wait age; a role with zero items is omitted.
  8. Same board: after a claim, the room's delivery view and the room board show the same card — no duplicated row.
  9. npm run verify green.

The knife: remove the compare-and-set in (1) → the concurrent-claim test must go red; remove the role filter in (2) → that test must go red. Do not ship with only (1) covered.

Where it sits (file:line pointers)

  • core/src/host/tasks.js — ACTIVE/TRANSITIONS (:22/:25), record/transition (:187/:191), applyFields (:284).
  • core/src/host/task-drive.js — DRIVING (:20), driveOnce (:177), consider (:244).
  • core/src/shared/task-board.js — columnOf (:19), compareForWork (:37).
  • core/src/host/room-context.js:149 — formatRoomBoard.
  • Tool surface: wherever track_tasks is declared (the new fields are queue, role, claim).

Out of scope / do not touch

  • No new MCP server and no change to which tools a room mounts.
  • No change to the verdict gate, the merge queue, or AGENTS.md review rules.
  • No second task board and no separate queue file: one board, one set of cards.
  • Do not make claim assign work to a member that is mid-turn; queue it and let task-drive hand it over.
  • Do not remove the lead's ability to assign directly (assignee); this adds a pull path, it does not delete the push one.

Why now (the cost this removes)

A measured audit (#796) puts one group message at ≈7.5 M prompt tokens because 6.8 members wake per message and each turn costs 7.4 model calls. Dispatch by chatting at people is the biggest single lever left: a queue wakes only a member that has something to do, and a wake that carries a title is a short turn rather than a full re-read of the room.

Activity

  1. Birfy commented on Oct 26, 2026

    @Birfy
    CollaboratorAuthor

    Triage: scope verified at main 7be5658a; four forks decided so the implementation does not fork

    Every pointer in the body resolves at 7be5658a, no drift: tasks.js:22 (ACTIVE), :25 (TRANSITIONS), :187 (record), :191 (transition), :284 (applyFields); task-drive.js:12 (backoff), :20 (DRIVING), :177 (driveOnce), :244 (consider); task-board.js:19 (columnOf), :37 (compareForWork); room-context.js:149 (formatRoomBoard). Owner: Dev stays.

    Not a duplicate: #787 is the orchestrator epic (needs-human, much larger), #55 is handoff recovery across disconnects, #1213 was the presence surface and is already in main. This is the small slice: an item with no work owner, a role, and a single-winner claim. References, not supersessions.

    Four forks the body leaves open (decided here — argue with this comment, not with the implementer)

    1. "Unclaimed" is not assignee. The wake key is ownerBotId. drivingTasks(botId) (task-drive.js:70-75) drops every task whose ownerBotId !== botId, and driveOnce builds its wake map from it (:180-187). So an item enqueued with only "assignee unset" still carries ownerBotId = <enqueuer>, and driveOnce will wake the enqueuer on its own queued item — the opposite of the goal, once per backoff cycle. Take the contained shape: keep ownerBotId = enqueuer, add queueRole (+ claimedBy), teach drivingTasks to skip queueRole && !claimedBy, and have claim set ownerBotId = claimedBy = <claiming Bot> and clear queueRole. Do not make ownerBotId optional: mayAct, owns, taskIdForSend (:541), taskHeld (:561), prune and view all read it.
    2. Match a Bot's role label, not a membership role. role is free text, String(role).trim().slice(0, 24) (roster.js:100, updated at :192); a group membership role is 'admin' | 'member' (roster.js:330) and is a different field. Resolve the member the way memberBot does (tasks.js:352-358: live, non-remote participants of that room), then compare bot.role case-insensitively for equality. Two named refusals, never a silent null: not-in-role (criterion 2) and no-member-with-role at enqueue — an item whose role no live member carries must be refused, otherwise it sits for ever and point 6 shows an age nobody can act on.
    3. WIP = 1 predicate, and blocked does not hold the slot. ownerBotId === me && columnOf(task) === 'in-progress' (task-board.js:11-19 covers claimed/working/blocked/result-ready). Exclude blocked: that card is waiting on a person, so counting it would let one unanswered question freeze a member out of the queue for ever.
    4. The single-winner test must be able to go red. Node is single-threaded: Promise.all([claim(), claim()]) does not interleave unless the implementation awaits between "select the item" and "write the claim", so a test written against this body's shape passes with no CAS at all. Make the interleave explicit — the claim re-checks task.status === 'open' after its first await — and note save() already calls rt.considerTaskDrive?.() (tasks.js:209-212), so the second claim can arrive inside the first one's own save. The knife is "move the status check across the await", not "delete a line".

    Two smaller corrections

    • Claim order must not be compareForWork. Its tiebreak is (b.updatedAt - a.updatedAt) — newest first (task-board.js:47). For a queue that starves the oldest item and makes the age in point 6 a lie. Reuse the priority rank (priorityOf, :29), then createdAt ascending (the field exists: tasks.js:427).
    • The tool surface is exact. core/src/host/tools.js:833 declares track_tasks; the item properties sit at :845-865 and carry additionalProperties: false, so queue/role/claim must be declared there or the model cannot send them. Wrinkle in execute (:875-884): args?.tasks === undefined is today the read path, and a claim arrives with tasks omitted — take the claim branch before the read branch, or claim: true returns the task list instead of one item.

    Already true — keep as guards, do not sell as features

    • Criterion 5. The wake is a task-role turn in the Bot's own chat (task-drive.js:55-58) and cueFor (:30-44) carries id, priority and title, never the body. What is new is only "a queue wake is not a room round"; do not claim it avoids 150k, since every turn still carries the tool schema (Research: measure and cut token spend per Bot turn (prompt overhead, history growth, tool output, fan-out) #796: 113 tools ≈ 29.3k).
    • Point 6. The surface exists: cardLine (room-context.js:138) renders who [status pct] title — note per card, and formatRoomBoard (:149) caps at 8 cards inside ROOM_CONTEXT_BUDGET = 4000 (:10, packed by fitRoomLines :110). An unclaimed item already reads unassigned [open] title; put the age on that same line (unassigned [open 3h]) plus at most one summary line per role when depth > 0. No second presence block — and add a case that 8 cards + 3 role lines still fit the budget.

    Change scope

    core/src/host/tasks.js (item fields, claim handler, the two refusals, WIP gate), core/src/host/task-drive.js (drivingTasks filter + role wake pass), core/src/host/tools.js (schema + execute branch), core/src/host/room-context.js (age on the unassigned line + one role line), core/src/shared/task-board.js (queue ordering); mirrors under patches/ds-bot/files/src/... byte-identical; docs/subsystems/core-host.md (board section) and changelog.d/1247-*.added.md. Out of scope as written and still right: no MCP change, no AGENTS.md rule change, no second board.

    Labels: enhancement, area:automation, agent-ready.

  2. added
    agent-readyMaintainer-approved spec: an agent may take it. Only maintainers set this; never in a template
    area:automationScheduled tasks and automation
    enhancementNew feature or request
    on Oct 26, 2026
  3. Birfy commented on Oct 26, 2026

    @Birfy
    CollaboratorAuthor

    Written-by: Dev

    Claiming #1247 for implementation.

    • Branch: dev/feat/1247-pull-queue
    • Base: 0b2a9859c15f2ef2725a24b80767f5ff41e62a88
    • Tip: 9c0b905f76f11e4e3f2778df99cf799969604468
    • Scope: core/src/host/tasks.js, core/src/host/task-drive.js, core/src/host/tools.js, core/src/host/room-context.js, core/src/shared/task-board.js, mirrors under patches/ds-bot/files/, docs/subsystems/core-host.md, changelog fragment, and the new core/test/task-queue.test.js. No second board, no MCP change, no AGENTS.md rule change.

    Followed the Triage forks on 6409436256: ownerBotId stays the enqueuer until a claim (the wake key is claimedBy), roles match the Bot label case-insensitively, WIP = 1 excludes a blocked card, and the claim re-checks the item after its first await so the concurrent-claim case can go red.

  4. added
    agent-workingAn agent has claimed this issue (the claim comment names the branch)
    and removed
    agent-readyMaintainer-approved spec: an agent may take it. Only maintainers set this; never in a template
    on Oct 26, 2026
  5. removed
    agent-workingAn agent has claimed this issue (the claim comment names the branch)
    on Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:automationScheduled tasks and automationenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions