Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/1525-workflow-concurrency.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Workflow concurrency is now configurable per Workflow in Settings → Workflow graphs. The host-wide 6-run cap is removed; each Workflow's own `maxConcurrent` limit (1–8, default 2) is the only gate. [#1525](https://github.com/ThinkFlowLab/FlyBot/issues/1525)
4 changes: 2 additions & 2 deletions docs/subsystems/workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Everything in `plugins/workflow/`: the graph rules, the run store, the matcher,
## Entry points

- Endpoint `workflow/state` and `workflow/templates` are reads.
- `workflow/create`, `workflow/rename`, `workflow/delete`, `workflow/assign`, `workflow/concurrency`, `workflow/graph`, `workflow/advance`, `workflow/reset`, `workflow/say`, `workflow/ingest` write. `workflow/concurrency` sets that workflow's `maxConcurrent` (an integer from 1 to 8). `workflow/rename` updates a workflow's name and optional description; read-only hosts are refused. `workflow/delete` removes a workflow from state, releases its ephemeral Bots if a run exists, and cleans logs/runs/routes.
- `workflow/create`, `workflow/rename`, `workflow/delete`, `workflow/assign`, `workflow/concurrency`, `workflow/graph`, `workflow/advance`, `workflow/reset`, `workflow/say`, `workflow/ingest` write. `workflow/concurrency` sets that workflow's `maxConcurrent` (an integer from 1 to 8); the same control is in Settings → Workflow graphs. `workflow/rename` updates a workflow's name and optional description; read-only hosts are refused. `workflow/delete` removes a workflow from state, releases its ephemeral Bots if a run exists, and cleans logs/runs/routes.
- Tool `run_workflow` starts a named run with a brief. Tool `define_workflow` writes the same graph the canvas edits.
- `flybot.tasks.onWrite` calls `ingest`. A new Bot match returns `{ ownerBotId }`. A Workflow match starts the run inside the plugin and leaves the card's owner to the host.

Expand All @@ -47,7 +47,7 @@ Everything in `plugins/workflow/`: the graph rules, the run store, the matcher,
1. Friends' Bots are not nodes. A `remote` Bot is refused by create, define and the matcher.
2. The canvas and the run share one graph: nodes, edges and positions. A drag commits `workflow/graph`, but only while **Edit** is on — a factory preset is copied first (#1378). Status highlighting reads that same graph. Replacing edges does not recompute who is already running.
3. **A live run advances only when its ephemeral Bot replies, following the graph's edges in order.** That reply advances the run and may appear on the workflow stage log; it is not delivered into a persistent sidebar Bot conversation (for example Chief). The `workflow-step` send middleware denies those replies after `stepResult` runs. Nodes start only when their forward predecessors finish and edges reach them. There is no scripted or seeded run path: `workflow/state` exposes only `live === true` runs. A run does not create a schedule. `scheduleLabel` is display text. When the host can create Bots, starting a live run (`assign`, `run_workflow`, `ingest`) spawns one ephemeral Bot per node from that node's prompt, role, model and tools, sends it the step from the Main Bot, and removes every instance when the run finishes, fails for good, or is reset. A `failOnce` retry keeps them. They are hidden and left out of the sidebar. Loading the plugin does not spawn Bots for the default workflows. Plugin unload (`dispose`) and the load-time `reap` both end a live run still marked `running`: in-flight nodes are failed, the run is archived as `failed`, and any ephemeral Bot ids are released. A run already written as `running` with no ephemeral map (an older unload) is healed the same way on the next load (#1414). Loading is also the one moment a run the host left behind is noticed when its ephemeral map is still present: `reap` releases those instances and records the run as `failed`. Nothing resumes it. A run that already ended keeps its own status (#1388). `workflow/reset` stopping one live run ends it the same way before its Bots are released: nodes still `running` become `failed`, so the strip cannot keep painting a run that is over (#1424). A dispatch that does not land is not marked handed. The chat names the code, and the next hand-off tries that node again. It is not sent twice in the same hand-off. A reply that says `could not answer your message` or `model call failed` fails that node and does not start its successor. A 429 or rate-limit mention in any other reply does not.
4. Each workflow keeps a FIFO queue and up to `maxConcurrent` live runs (default 2, an integer from 1 to 8 on the record or via `workflow/concurrency`). The host runs at most 6 live runs at once. A task that does not fit is enqueued. When a run finishes, every workflow with a free slot is drained, oldest enqueue first, until the host cap. A new assign on a workflow that already has a queue waits behind it. A task does not replace a running run or release that run's Bots. Migrating an old store keeps only runs that were still `running` with ephemeral Bot ids; other legacy scripted rows are dropped from active state (finished tasks may remain in lane history). History records a finished or failed live run. Lowering the cap does not stop a run that has already started. Assigning `{ botId }` claims the card for that local Bot and does not enqueue. A remote Bot is refused. `activeCount` is `runningOf(lane)` — live runs with `live === true` and `status === 'running'`. The stage aggregate chip and the Runs tab (运行记录) must agree: every such run appears as its own running row and progress card, and the chip count does not exceed that list.
4. Each workflow keeps a FIFO queue and up to `maxConcurrent` live runs (default 2, an integer from 1 to 8 on the record, via `workflow/concurrency`, or in Settings → Workflow graphs). A task that does not fit is enqueued. When a run finishes, every workflow with a free slot is drained, oldest enqueue first. A new assign on a workflow that already has a queue waits behind it. A task does not replace a running run or release that run's Bots. Migrating an old store keeps only runs that were still `running` with ephemeral Bot ids; other legacy scripted rows are dropped from active state (finished tasks may remain in lane history). History records a finished or failed live run. Lowering the cap does not stop a run that has already started. Assigning `{ botId }` claims the card for that local Bot and does not enqueue. A remote Bot is refused. `activeCount` is `runningOf(lane)` — live runs with `live === true` and `status === 'running'`. The stage aggregate chip and the Runs tab (运行记录) must agree: every such run appears as its own running row and progress card, and the chip count does not exceed that list.
5. A node declares up to eight inputs and eight outputs. A required input is an output of the same name on an ancestor, or `task`, `title`, or `brief` on a root. A start with no provider is refused, and Bots spawned for that attempt are released. On a Bot-backed run a successor stays queued until its required inputs are present. If every forward predecessor is done, failed or skipped and an input is still missing, that node fails and the run stops. The dispatch text carries `Input <name>: <value>`. The Bot reports `OUTPUT <name>: <value>`; a single declared output may be the reply body. Nodes start in edge order.
6. Each node is a real ephemeral Bot for that run, then it is removed. A permanent sidebar Bot (`botId`) is optional and is not required for the step to run. **Every node Bot gets the same full permissions a normal sidebar Bot gets:** the `tools` field is always omitted from `fb.bot.create`, letting the Bot inherit all registered tools — file/shell tools, GitHub (`mcp__github__*`: pull requests, reviews, comments), Composio, and team tools — limited only by `EPHEMERAL_DENIED_TOOLS` at the guard/approval layer (#1466, 29 team-admin tools). `node.tools` is stored as documentation in `fields.tools` but does not restrict the grant. All node Bots get `danger-full-access` permission. A hidden run Bot cannot show an approval card, so a granted tool is allowed and a connector it did not name is refused. The host stores this on the run Bot as `stepTools` and `permission`.
7. A loop edge (`loop: true`, `max` from 1 to 8, default 3, optional `until: { output, eq }`) is the only cycle. The body runs again until `until` matches or `max` runs have finished, and the forward edges wait until then. Restarting the body drops the branch choice made inside it and queues a sibling that the previous pass skipped, so the next pass can take the other side. A `when: { output, eq }` edge is taken only when that output equals `eq`. `when: { else: true }` is taken only when no other `when` matched. A node whose forward edges are all closed is `skipped` and does not block a join. A run cannot spin: every loop has a max. Settings → Workflow graphs can add a link, a condition or a loop.
Expand Down
4 changes: 2 additions & 2 deletions plugins/workflow/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Assigning a task, `run_workflow`, or a Workflow match starts a live run. One hid

A second task does not replace a run that is already going, and it does not release that run's Bots. It waits in a FIFO queue. The stage shows a chip when more than one run is active or the queue is not empty, and it refreshes `workflow/state` while a live run exists. That refresh does not advance the graph. The product UI does not offer a canned demo replay; only live runs advance the graph.

Each workflow allows `maxConcurrent` live runs at once. The default is 2. The value is an integer from 1 to 8, set on create (`maxConcurrent`) or with `workflow/concurrency` (`{ workflowId, maxConcurrent }`). The stage lists the current value under Details. The host runs at most 6 live runs across every workflow. A task that does not fit is queued. When any run finishes, the oldest queued task whose workflow still has room starts, whichever workflow it belongs to. A new assign waits behind a task already queued there. Lowering the cap does not stop a run that has already started. History keeps finished and failed runs only.
Each workflow allows `maxConcurrent` live runs at once. The default is 2. The value is an integer from 1 to 8, set on create (`maxConcurrent`), with `workflow/concurrency` (`{ workflowId, maxConcurrent }`), or in Settings → Workflow graphs. A task that does not fit is queued. When any run finishes, the oldest queued task whose workflow still has room starts, whichever workflow it belongs to. A new assign waits behind a task already queued there. Lowering the cap does not stop a run that has already started. History keeps finished and failed runs only.

## Inputs and outputs

Expand Down Expand Up @@ -81,7 +81,7 @@ Workflow 是一组本地 Bot 的群聊式运行。侧栏打开同一次运行:

一次真实运行会为每个节点创建一个隐藏的临时 Bot,用完即删。节点不必绑定侧栏里的永久 Bot。`tools` 决定这个 Bot 能用什么:`github` 授予 GitHub 连接器(拉取请求、评审、评论),`bash` 授予完整权限的 shell,其余步骤至少可以写工作区。节点一直处于运行中,直到该 Bot 回复。回复写入 Workflow 聊天,下一节点才沿边开始。正在进行的运行不会被新任务替换,也不会因此释放它的 Bot。历史里只留下已完成或已失败的运行。

每个 Workflow 同时运行的上限是 `maxConcurrent`,默认 2,取 1 到 8 的整数,在创建时传入,或调用 `workflow/concurrency`。全部 Workflow 加在一起最多 6 个正在运行的任务。某个运行结束后,所有 Workflow 里最早排队、且自己还有空位的任务会开始。新的分配排在已经排队的任务后面。降低上限不会停掉已经开始的运行。舞台在详情里显示当前上限;有排队或同时多于一个运行时显示队列条。有真实运行时,页面只刷新状态。产品界面不提供预置演示重放,只有真实运行会推进图。
每个 Workflow 同时运行的上限是 `maxConcurrent`,默认 2,取 1 到 8 的整数,在创建时传入、调用 `workflow/concurrency` 或在设置 → Workflow graphs 里调整。某个运行结束后,所有 Workflow 里最早排队、且自己还有空位的任务会开始。新的分配排在已经排队的任务后面。降低上限不会停掉已经开始的运行。舞台在详情里显示当前上限;有排队或同时多于一个运行时显示队列条。有真实运行时,页面只刷新状态。产品界面不提供预置演示重放,只有真实运行会推进图。

每个节点最多声明 8 个输入和 8 个输出,名字为小写字母开头的短名。必填输入必须由上游同名输出提供;根节点可以用 `task`、`title` 或 `brief`。没有提供者时,运行拒绝开始。Bot 用 `OUTPUT 名字: 值` 交回输出;只声明一个输出时,回复正文即可。

Expand Down
17 changes: 17 additions & 0 deletions plugins/workflow/client.js
Original file line number Diff line number Diff line change
Expand Up @@ -899,6 +899,23 @@ window.__ModuleLoader__.load({
},
h('option', { value: '' }, tr('Each Bot\'s own model')),
(pack?.roster?.models ?? []).map(model => h('option', { key: model.ref, value: model.ref }, `${model.name} (${model.provider})`)))),
h('label', { className: 'wf-field' },
h('span', null, tr('Concurrent runs')),
h('input', {
type: 'number',
'aria-label': tr('Concurrent runs'),
min: 1,
max: 8,
value: current.maxConcurrent ?? 2,
onChange: (event) => {
const maxConcurrent = Number(event.target.value)
if (!Number.isInteger(maxConcurrent) || maxConcurrent < 1 || maxConcurrent > 8) return
void Promise.resolve(call('workflow/concurrency', { workflowId: current.id, maxConcurrent })).then(value => {
if (value?.ok === false) { setError(value.error ?? ''); return }
if (value?.workflows) setPack(value)
})
},
})),
h('button', { type: 'button', className: 'wf-btn', onClick: () => setEditing(!editing) },
editing ? tr('Finish editing') : tr('Edit')),
editing ? h('span', { className: 'wf-autosave-hint' }, tr('Changes are saved automatically')) : null),
Expand Down
18 changes: 5 additions & 13 deletions plugins/workflow/src/service.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@
const FILE = 'workflows.json'
const DEFAULT_CONCURRENT = 2
const MAX_CONCURRENT = 8
const HOST_CAP = 6

function idOf(prefix) {
return `${prefix}-${randomBytes(4).toString('hex')}`
Expand Down Expand Up @@ -109,8 +108,7 @@
if (!state.laneFormat) migrateRuns(state)
purgeDemos(state)
let seq = state.seq ?? 1
// One lock for every start and release. The host cap is read and updated
// only while this is held, so two workflows cannot both pass it.
// One lock for every start and release to serialize drain operations.
let hostTail = Promise.resolve()
function hostSerial(fn) {
const run = hostTail.then(() => fn())
Expand Down Expand Up @@ -201,12 +199,6 @@
return (lane?.active ?? []).filter(run => run && run.live === true && run.status === 'running')
}

function globalRunning() {
let count = 0
for (const lane of Object.values(state.lanes)) count += runningOf(lane).length
return count
}

function maxOf(item) {
const value = Number(item?.maxConcurrent)
if (!Number.isInteger(value) || value < 1) return DEFAULT_CONCURRENT
Expand Down Expand Up @@ -267,7 +259,7 @@
const missing = unmetRequired(item, id, run.produced, run.task)
if (missing.length === 0) { started.push(id); continue }
const preds = item.edges.filter(edge => edge.to === id && edge.loop !== true).map(edge => edge.from)
const stuck = preds.length === 0 || preds.every(pred => status[pred] === 'done' || status[pred] === 'failed' || status[pred] === 'skipped')

Check warning on line 262 in plugins/workflow/src/service.js

View workflow job for this annotation

GitHub Actions / plugins (workflow)

unicorn(no-useless-length-check)

Found a useless array length check
if (stuck) {
status[id] = 'failed'
blocked.push({ id, missing })
Expand All @@ -280,7 +272,7 @@
function applyOp(item, run, op) {
let working = run
if (op.type === 'complete' && op.outputs && typeof op.outputs === 'object') {
working = { ...run, produced: { ...(run.produced ?? {}), [op.nodeId]: op.outputs } }

Check warning on line 275 in plugins/workflow/src/service.js

View workflow job for this annotation

GitHub Actions / plugins (workflow)

unicorn(no-useless-fallback-in-spread)

Empty fallbacks in spreads are unnecessary
}
let result
if (op.type === 'start') result = { ...startFrontier(item, working.nodeStatus, working.fired), joined: false, layerSize: 0 }
Expand Down Expand Up @@ -379,7 +371,7 @@
push(item.id, [{ kind: 'system', tone: 'bad', text: `${node.name} 没有接到这一步(${code})。` }], now())
continue
}
current = { ...current, handed: { ...current.handed, [nodeId]: true }, inputs: { ...(current.inputs ?? {}), [nodeId]: body.inputs } }

Check warning on line 374 in plugins/workflow/src/service.js

View workflow job for this annotation

GitHub Actions / plugins (workflow)

unicorn(no-useless-fallback-in-spread)

Empty fallbacks in spreads are unnecessary
storeRun(item.id, current)
}
write()
Expand Down Expand Up @@ -450,10 +442,10 @@
}

// After a slot frees, start the oldest parked task on any workflow that
// still has room, and stop at the host cap.
// still has room.
function drainSync() {
let guard = 0
while (globalRunning() < HOST_CAP && guard < 48) {
while (guard < 48) {
guard += 1
const next = takeNext()
if (!next) return
Expand All @@ -471,7 +463,7 @@
return
}
let guard = 0
while (globalRunning() < HOST_CAP && guard < 48) {
while (guard < 48) {
guard += 1
const next = takeNext()
if (!next) return
Expand Down Expand Up @@ -563,7 +555,7 @@
}

function hasRoom(item) {
return runningOf(laneOf(item.id)).length < maxOf(item) && globalRunning() < HOST_CAP
return runningOf(laneOf(item.id)).length < maxOf(item)
}

// A non-empty queue is joined at the tail, then the oldest task starts.
Expand Down Expand Up @@ -1088,7 +1080,7 @@
const runs = []
let endedLive = false
for (const [workflowId, lane] of Object.entries(state.lanes ?? {})) {
for (const run of [...(lane.active ?? [])]) {

Check warning on line 1083 in plugins/workflow/src/service.js

View workflow job for this annotation

GitHub Actions / plugins (workflow)

unicorn(no-useless-spread)

Using a spread operator here creates a new array unnecessarily.
if (run?.live === true && run.status === 'running') {
endRun(workflow(workflowId), run)
archiveCutoffRun(workflowId, run)
Expand Down
29 changes: 29 additions & 0 deletions plugins/workflow/test/client.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -367,6 +367,35 @@ test('apply registers the sidebar section and the settings page', () => {
off()
})

test('settings can change the workflow concurrency limit', async () => {
const calls = []
const graph = delivery()
graph.maxConcurrent = 2
const call = async (name, body) => {
calls.push({ name, body })
if (name === 'workflow/state') return { workflows: [graph] }
if (name === 'workflow/concurrency') {
graph.maxConcurrent = body.maxConcurrent
return { ok: true, workflows: [graph] }
}
return { ok: true, workflows: [graph] }
}
const settings = mount(client.WorkflowSettings, { call, roster: { bots: [] } })
await settle()
const concurrentInput = findAll(settings.tree, node => node.props?.['aria-label'] === 'Concurrent runs')[0]
assert.ok(concurrentInput, 'concurrent runs input should exist')
assert.equal(concurrentInput.props.value, 2)
assert.equal(concurrentInput.props.min, 1)
assert.equal(concurrentInput.props.max, 8)

concurrentInput.props.onChange({ target: { value: '4' } })
await settle()
const changed = calls.filter(item => item.name === 'workflow/concurrency').at(-1)
assert.equal(changed.body.workflowId, 'delivery')
assert.equal(changed.body.maxConcurrent, 4)
settings.unmount()
})

test('settings can add a loop edge and a condition edge', async () => {
const calls = []
const graph = delivery()
Expand Down
Loading
Loading