Skip to content

Feed the swarm from CI, with fuel that does not run out - #3978

Draft
gHashTag wants to merge 1 commit into
masterfrom
feat/queen-feed-in-ci
Draft

gHashTag wants to merge 1 commit into
masterfrom
feat/queen-feed-in-ci

Conversation

@gHashTag

Copy link
Copy Markdown
Owner

Closes #3977

Why

The feeder ran on the operator's laptop. Between 18:03 and 05:43 on 2026-09-16/17 it fired twice instead of 36 times — the machine was asleep — and the swarm idled through the night. A feeder that sleeps is not a feeder.

It also fed from a pool with a bottom: 24 uncovered files of empty function bodies remain, roughly fifteen minutes of swarm work at the measured rate.

What changes

  • .github/workflows/queen-feed.yml — */5 cron plus workflow_dispatch, permissions: {contents: read, issues: write}, a queen-feed concurrency group so two runs cannot double-create. It needs no compiler: the feeder only walks the checkout and reads the issue list.
  • tools/queen/queen_feed_port.py — generates one issue per hand-written source file: quotes its function signatures verbatim and asks for specs/port/<path>.t27 with a test per function.

--top-up N maintains a backlog of N dispatchable issues rather than reacting to an empty queue. It asks /queen/status what the Queen is actually blocked on (claimed, completed-but-open, file conflicts, incomplete) and subtracts that from the open boundary-carrying issues — counting the raw number instead reads as a full queue while the lanes starve. If the status endpoint is unreachable it stops rather than sizing a backlog blind.

Measured, not assumed

Un-ported hand-written files on 6e8a494 870
Bee consumption ~10 issues / 7 min across 10 lanes
Old feeder ceiling 10 issues / 20 min
Dispatchable backlog at 06:02 259 open with a boundary − 280 blocked = 0

Scope limits

bootstrap/ is excluded: stage-0, and compiler.rs is under the FROZEN_HASH seal (FROZEN.md §5).

The acceptance criteria are structural — file present, same function names, no codegen panic, no parser recovery, a test per function. They cannot express correctness; the review-side oracle compiles the generated Zig and runs its tests, and that remains the judge.

Validation

#3974, #3975 and #3976 were created from this generator as a probe. All three were picked up and dispatched, confirming the Queen accepts a ## Boundary naming a file that does not exist yet.

Note: scheduled workflows only run from the default branch, so the cron starts firing once this is merged.

🤖 Generated with Claude Code

The feeder ran on a laptop scheduled task and fired twice between 18:03 and
05:43 instead of 36 times: the machine was asleep, and the bees idled all
night. It also fed from a pool that is nearly empty — 24 files of empty
function bodies left, about fifteen minutes of swarm work.

This runs the feeder on GitHub's cron every five minutes, and feeds it from
the project's own goal instead: 870 hand-written Python, Rust, TypeScript and
C files that have no .t27 counterpart. bootstrap/ is excluded — it is stage-0
and compiler.rs sits under the FROZEN_HASH seal.

The criteria it writes are structural and cannot express correctness; the
review-side oracle remains the judge of that.

Refs #3977

Co-Authored-By: Claude Code <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-09-17 06:07:45 UTC

Summary

Status Count
Total Open PRs 41
PRs with Failing Checks 38
PRs with All Checks Green 3
READY 2
FAILING 38
PENDING 0
NO CHECKS YET 0

These columns do not partition: 2 + 38 + 0 + 0 = 40, and there are 41 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=8932aae0c5a9 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@gHashTag
gHashTag marked this pull request as draft September 17, 2026 06:13
@gHashTag

Copy link
Copy Markdown
Owner Author

Do not merge yet — converted to draft. Measured after opening this PR:

Port issues created 55 (#3974–#4030)
Swarm utilisation 100%, 10/10 lanes, for the first time in this log
Dispatches ending outcome: refused #3985, #3994, #4029, #4030 — every one of them a port issue
Branches produced by port issues 0

The fuel works as fuel: the queue filled and the lanes saturated. It does not work as work — the bees claim these issues and refuse them, silently (no comment on the issue, and /queen/feed/data answers 403 from outside the container, so the transcript is not readable here).

The difference between the issues that produce branches and the ones that do not is that the empty-bodies issues name a boundary file that exists, while every port issue names specs/port/<path>.t27, a file that does not exist yet. That is the hypothesis to test, not a conclusion — the refusal reason is inside the container.

Merging this on a */5 cron would pump refusal fuel around the clock, so it stays a draft until the refusal is understood. The feeder script itself and the --top-up backlog accounting are unaffected by this and are the parts worth keeping.

@gHashTag

Copy link
Copy Markdown
Owner Author

Correction to the comment above, and the measurement that replaces it.

That comment said the bees refuse these issues and blamed the boundary naming a file that does not exist. Both halves were wrong.

outcome: refused with finishedAt == dispatchedAt is not a worker refusing work. The server's own log says what it is:

Queen tick chose an issue but every key is busy
issue=4012 detail="all 4 provider key(s) are already in use by bees in flight."

A tick that found work and had no free lane. The swarm was working the whole time, and the HTTP 429s that seemed to confirm a dead provider were the swarm saturating its own account's concurrency — with the pool idle, all five keys answered on the first try.

So the line could finally be judged on its own, against a control created the same hour:

issues reached a pull request
empty-body issues #3963–3972 10 5 50%
port issues #3974–4033 60 2 4%

47 of the 60 port issues got a branch and 45 of those branches are empty. Two carry a real file — queen-4026 and queen-4030, each a correct specs/port/tools/*.t27.

The line works about once every thirty dispatches. That is the issue format, not the swarm: porting a whole module into a new file is a far larger turn than filling one empty function body in a file that already exists.

Staying a draft until the format is worth the lanes it would occupy. Worth trying first: one function per issue instead of one file, and a worked example of a ported function in the body.

This was referenced Oct 5, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Run the swarm's feeder in CI, not on a laptop: port hand-written code to .t27

1 participant