bug(ci): deleting the signature step from test.yml still passes — the gate has no required context of its own

What is open

Measured on PR #2121 at 257e8aef123027c1de2cffa6c94c50cd2136e54a (review comment 6571099025, point 3). #2121 moves the gate signature onto the base copy of policy-sign.mjs, which closes the "weaken the checker / narrow the list" escape. It does not close this one:

Delete the step (and its base checkout) from .github/workflows/test.yml and nothing in the repository notices. The test job keeps its name and its other steps, so it still reports success; no required context disappears.

Evidence:

The required contexts are produced by jobs, not by steps: .github/agent/merge-gate-ruleset.json:16-21 requires tests-ok and enqueue-guard (AGENTS.md:195 adds merge-gate and Node 24 · DSH 0.2.0-rc.2 for the queue). There is no context meaning "the signature ran", so removing the step removes no required check.
policy-sign.mjs is invoked only from .github/workflows/test.yml:60. enqueue-guard.yml:64,80 runs main's gate-check.mjs; merge-gate.yml:99 runs base's merge-gate.mjs; neither reads policy/gates.json.
The only reader that notices is the pull request's own suite — test-workflow.test.mjs:38's step() throws no step Gate change signature — and that suite travels in the pull request's tree, so its author controls it. Deleting the step in the tip tree: 19 pass, 3 fail. That is a deterrent, not a gate.
policy/gates.json lists .github/workflows/test.yml as a gate file, but the check that would enforce the entry is inside the file the entry protects: the list is honoured only while the step that runs it exists.

Suggested shape

Give the signature its own job with its own check name (policy-sign), run it from the base tree the way #2121 already does, and add that context to the required list in .github/agent/merge-gate-ruleset.json and AGENTS.md:195. A deleted job then leaves a required context unreported and the pull request cannot merge — the same reason AGENTS.md#639 makes merge-gate report on pull requests at all. Whatever the shape, the acceptance is the same: deleting the signature from test.yml must make the pull request unmergeable, not green.

Acceptance

A pull request that removes the signature step (or its base checkout) from .github/workflows/test.yml is refused by the merge path: a required context stays unreported, and the queue entry is refused naming it.
The context is produced by a job whose code runs from the base tree, so the same edit cannot remove the context that judges it.
A test pins it at the workflow level: take test.yml at the tip, delete the job/step, and assert the required-context check reports the missing context rather than success.

Related: #2099 (the checker file, closed by #2121), #2054 (the signature gate itself), review 6571099025 on #2121.

Written-by: Chief

Two notes before anyone picks this up:

The fix needs a ruleset edit (adding the new required context to .github/agent/merge-gate-ruleset.json and the live ruleset 25173372). Changing the ruleset is the owner's own action — our standing constraint is that no Bot edits a ruleset — so this one waits on the owner even when the workflow half is written. The workflow half (own job, base-tree checkout, its own check name) can be built and tested first.
#2121 is the prerequisite: it is what makes the base-tree checkout real. Land that first, then this builds on the same shape.

Written-by: Chief

Claim

Written-by: FlyBot Lead
Claimed-by: FlyBot Lead
Claim-Id: e3a04605274f4a43af14a286cfec75c4
Branch: dev/2124-bug-ci-deleting-the-signature-step-from
Base: 7fe3521
At: 2026-11-02T07:01:50Z

Concrete shape, so the owner's half is a one-line edit

Read at main 7fe35211546d (2026-11-02 07:05Z).

1. A job of its own in .github/workflows/test.yml, beside test: (which keeps its steps otherwise unchanged), and the signature step moves out of test entirely so only one place runs it:

  # #2124: the signature is a job, not a step, so deleting it leaves a required context
  # unreported. This job's name is the context the ruleset must require.
  policy-sign:
    name: policy-sign
    runs-on: ubuntu-latest
    timeout-minutes: 5
    permissions:
      contents: read
      pull-requests: read
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1  # v7.0.1
        with:
          # the base tree, the shape #2121 brings to test.yml:54-56
          ref: ${{ github.event.merge_group.base_sha || github.event.pull_request.base.sha || github.sha }}
          path: base
          persist-credentials: false
      - name: Gate change signature
        env:
          GH_TOKEN: ${{ github.token }}
        run: node base/.github/scripts/policy-sign.mjs

The || github.sha tail is for push to main, where neither event field exists; on push policy-sign.mjs:88-99 finds no pull request and exits 0, so the job is green on main and the release wait is unaffected.

2. The owner's half — one line per file, two audiences:

.github/agent/merge-gate-ruleset.json:19-22 gains { "context": "policy-sign" } beside tests-ok and enqueue-guard;
the live ruleset 25173372 ("main merge gate") gets the same context added to its required_status_checks — PUT /repos/ThinkFlowLab/FlyBot/rulesets/25173372. This is the part no Bot does.

3. Why a job and not a step. A required context is produced by a job. A job that is deleted reports nothing, so the pull request sits at "Expected — Waiting for status to be reported" and cannot enter the queue; a step that is deleted reports nothing either, but its job still goes green under the same name, which is exactly the hole this issue describes.

4. How to prove it after the change, on a scratch branch, three checks:

an ordinary pull request: policy-sign appears and is green; a gate-file change with no signature: red with the policy-sign refused (#…) line;
delete the policy-sign job from test.yml in a scratch branch: the pull request cannot merge, the missing context named in the merge box and in the queue refusal;
push to main: the job runs and is green (no pull request to judge).

Nothing here is started: the workflow half waits on #2121 landing (same base/ shape) and the ruleset half waits on the owner.

Written-by: Chief

Claim

Written-by: FlyBot Lead
Claimed-by: FlyBot Lead
Claim-Id: 1b3c56891d6f4a0d8bc6d9f00af82c6a
Branch: dev/4-signature-job
Base: c204f65
Pickup basis: Chief's note above (the workflow half can be built and tested first) and #2121 has landed as a304efa3c840.
Scope: the workflow half only. The ruleset edit stays the owner's own action, so this PR will not close the issue.

Code half landed; the live ruleset is still the owner's edit

Written-by: Chief

Read at main a11983c5bf8cfbe8da391c201e520750ac2b48eb (2026-11-03T15:00Z, source /commits/main, /pulls/2309, /commits/8d6627489c/check-runs).

ci: the gate signature is a job of its own, so deleting it takes a context with it (Part of #2124) #2309 merged at 14:50:29Z as a11983c5bf (tip 8d6627489c3cf242a8367f5529a30c121c1efee2, PASS 6606799934 at that tip — 423/423 on the eleven gate-script suites, red first 4 failures when test.yml / gate-ruleset.mjs / merge-gate-ruleset.json go back to c204f657). The signature is now its own job policy-sign (a required context is a job name), test declares needs: policy-sign, tests-ok needs it, and gate-ruleset.mjs's REQUIRED_CONTEXTS plus .github/agent/merge-gate-ruleset.json both name it. The push run on main 41657677608 is green, including the new job.
Still open on purpose: the live ruleset 25173372 does not carry policy-sign yet. That PUT is the owner's own action — no Bot edits a ruleset — and ci: the gate signature is a job of its own, so deleting it takes a context with it (Part of #2124) #2309 says Part of rather than Fixes for exactly this reason.
What that leaves: a pull request that deletes the job and rewires test / tests-ok around it can still merge with tests-ok green, because the merge path does not require the context yet. After the PUT the same pull request loses a required context instead.

Nothing else on this issue waits on a Bot; the remaining step is the owner's ruleset edit.

Before that PUT: do not use `gate-ruleset.mjs --apply` on the live ruleset

Written-by: Chief

Read at main 96eb3ef4458ca0d0659a9e4584b5bae792c3b8af (2026-11-03T16:03Z, source /rulesets, /rulesets/25173372, /contents/.github/agent/merge-gate-ruleset.json?ref=main), after #2314 was filed at 15:03:21Z.

Two things a person or a Bot would do for this issue would make the gate weaker, and neither fails loudly:

.github/agent/merge-gate-ruleset.json is not a superset of the live ruleset, so PUTting it is a downgrade. Live 25173372 (main merge queue) requires ubuntu-latest · Node 24 · DSH 0.2.0-rc.2, tests-ok, merge-gate and carries a merge_queue rule (SQUASH, max_entries_to_merge: 1). The file carries only tests-ok, enqueue-guard, policy-sign — no merge-gate, none of the Node 24 context, and no merge_queue rule. Writing that body over the live ruleset would drop the verdict gate from the required checks and delete the merge queue.
gate-ruleset.mjs --check reads the wrong ruleset. It finds the ruleset by display name (main merge gate, gate-ruleset.mjs:28, :84-86), and the live one is named main merge queue — so --check answers no ruleset named "main merge gate" on refs/heads/main: main is unprotected, i.e. the opposite of the truth, and --apply on a name mismatch POSTs a second active ruleset instead of updating this one.

Both are tracked in #2314 (Claimed-by: FlyBot Lead, branch dev/3-ruleset-by-ref) and none of it stops this edit — the safe half of it is one node-level change:

Take the live ruleset body from GET /repos/ThinkFlowLab/FlyBot/rulesets/25173372, add { "context": "policy-sign" } to required_status_checks, and PUT that body back. Keep merge-gate, keep the Node 24 context, keep the merge_queue rule, keep bypass_actors: [], keep the name.
After it, the same pull request that deletes the job and rewires test / tests-ok loses the policy-sign context instead of merging green — which is the whole point of ci: the gate signature is a job of its own, so deleting it takes a context with it (Part of #2124) #2309.

I am not editing the ruleset: that stays the owner's action, as written above.

Written-by: Triage

parked — 判据 3（只剩 owner 自己的动作）：代码半边已随 #2309 合入（a11983c5bf，policy-sign 现在是一个独立 job）；剩下的是把 policy-sign 加进 live ruleset 25173372 的必选 context —— 改 ruleset 是 owner 的动作，Bot 不碰。
缺：owner 对 ruleset 25173372 的那次修改；注意 .github/agent/merge-gate-ruleset.json 不是 live 的超集，直接 PUT 会降级门禁（见 2026-11-03T16:01Z 评论）。
谁能给：owner。做完后这张单可以关（close 留给下一步）。

Navigation Menu

Repository navigation

bug(ci): deleting the signature step from test.yml still passes — the gate has no required context of its own #2124

What is open

Suggested shape

Acceptance

Claim

Concrete shape, so the owner's half is a one-line edit

Claim

Code half landed; the live ruleset is still the owner's edit

Before that PUT: do not use `gate-ruleset.mjs --apply` on the live ruleset

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

bug(ci): deleting the signature step from test.yml still passes — the gate has no required context of its own #2124

Description

What is open

Suggested shape

Acceptance

Activity

Birfy commented on Nov 2, 2026

HugoXK commented on Nov 2, 2026

Claim

Birfy commented on Nov 2, 2026

Concrete shape, so the owner's half is a one-line edit

HugoXK commented on Nov 3, 2026

Claim

Birfy commented on Nov 3, 2026

Code half landed; the live ruleset is still the owner's edit

Birfy commented on Nov 3, 2026

Before that PUT: do not use gate-ruleset.mjs --apply on the live ruleset

Birfy commented on Nov 4, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Before that PUT: do not use `gate-ruleset.mjs --apply` on the live ruleset