Skip to content

feat(rsi): add controlled proof demonstration protocol - #26

Merged
w4ffl35 merged 6 commits into
masterfrom
codex/rsi-proof-20
Oct 1, 2026
Merged

w4ffl35 merged 6 commits into
masterfrom
codex/rsi-proof-20

Conversation

@w4ffl35

@w4ffl35 w4ffl35 commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Summary

Adds the staged issue #20 proof workflow: a development-only engineering pair, a final frozen three-campaign plan, resumable paired development and confirmation execution, and read-only JSON/Markdown analysis. The engineering gate requires a durable generation-zero self-hosted selection, an accepted generation-one child run from that selected commit, and terminal generation-one fixed-control candidate decisions. Confirmation identities are not scheduled during engineering.

The final plan binds the benchmark manifest, effective configs, task schedule, runtime and image identities, broker allocations, seeds, analysis source, engineering evidence, endpoint metadata, and operator cap. Before execution, the launcher reloads task identities and recomputes the full calls/tokens/spend/runtime plan from the frozen configs and broker limits; it rejects a stored plan that understates the recomputed budget. OpenRouter routing is pinned to DeepInfra fp8, with returned model/provider metadata checked before usage can qualify.

No live demonstration or paid inference has occurred. An authenticated OpenRouter key was used only for read-only endpoint metadata. The three brokered requests in the mock guest smoke used a mock upstream and do not represent production provider usage. No operator cap has been supplied.

Verification

  • npm run typecheck — passed on the final tree.
  • git diff --check — passed on the final tree.
  • npm test — exit 0; all 183 test files passed. TAP recorded 35 skipped integration cases: 9 PostgreSQL-gated and 26 S3-gated. The final added fixed-control CLI-path assertion was then verified by the focused integration test below.
  • npx tsx src/rsi/benchmark/__tests__/openshell-integration.test.ts — passed on the final tree; verifies one selected fixed-control CLI launch, guest/verifier separation, and teardown ordering.
  • npx tsx src/rsi/__tests__/proof-demonstration.test.ts — 3 passed, 0 skipped, including a qualifying synthetic record and fail-closed tampering cases.
  • npx tsx src/rsi/__tests__/proof-demonstration-launcher.test.ts — 3 passed, 0 skipped, including generation-one fixed-control decision enforcement and restart/receipt checks.
  • npx tsx src/rsi/__tests__/proof-demonstration-plan.test.ts — passed, including recomputed budget mismatch rejection.
  • PostgreSQL 16 + disposable RustFS: npx tsx --test src/rsi/__tests__/postgres-queue.test.ts src/rsi/__tests__/proof-campaign.test.ts — 40 passed, 0 failed, 0 skipped.
  • Exact runtime image built from source commit ea12df5ee67abb3d610dd26a68988c5dff8af25f: RSI image digest sha256:0e2b6135d2e9011e27a1043feb1d9faf387043e0893f50d41efc2ccd53bc26b5; image marker matched that commit. The commits after it contain test-only changes.
  • Mock OpenShell guest-to-broker smoke on that image — passed: 3 mock broker calls, 1,272 tokens, $0.001272 mock accounting, 0 arbitrary-egress sentinel hits, and no raw key in guest result. Gateway alias trap removed and sandbox list was empty after the run.

Security impact

The OpenRouter key remains in the supervisor broker and is not sent to the guest. Candidate execution and independent verification remain isolated. Confirmation fixtures and verifier inputs are excluded from guest snapshots/images. Frozen plans and analysis outputs are restricted to external or ignored supervisor-only paths. Campaign execution requires the exact frozen cap and fails closed on incomplete or uncertain evidence.

Test plan

No production inference should be sent without a concrete operator-approved cap. The report must remain inconclusive or not-demonstrated unless all frozen paired cells, receipts, artifacts, ledgers, lineage, and statistical gates verify. The mock smoke and service-backed integration checks exercise plumbing only; they do not establish the multi-generation demonstration.

@w4ffl35

w4ffl35 commented Oct 1, 2026

Copy link
Copy Markdown
Member Author

Adversarial review completed against the final PR head (3800b39). Blocking findings from the draft were fixed before merge:

  • Guest requests for the dated model used the old DeepSeek route while the broker pinned DeepInfra. The guest route, local price guardrail, endpoint metadata fixture, and OpenShell smoke now agree on the DeepInfra fp8 endpoint.
  • Confirmation could finish in PostgreSQL before its archive checkpoint. The controller now checkpoints the selected/root attempt IDs and artifacts before the terminal transition, and recovery verifies them against the ledger.
  • Analysis could ignore duplicate arm records, trust incomplete provider identity, and compare seeded campaigns as different protocols. It now binds campaigns to the frozen plan and checks exact provider, schedule, lineage, and artifact digests. A qualifying synthetic result passes; provider, config, missing-cell, and lineage changes revoke that verdict.
  • A combined engineering/confirmation manifest froze holdout cells too early. Engineering now has a development-only terminal path and an independently verified gate before the three-campaign plan can be frozen.
  • Launcher replay and plan verification now use deterministic arm identities, reject incomplete terminal cells, and recompute cost ceilings before execution.

Verification: 183 test files passed in npm test (35 service-gated skips); PostgreSQL 16 + RustFS focused tests passed 40/40 with no skips; exact-image mock OpenShell guest smoke passed; typecheck and both PR CI runs passed. The mock smoke used no paid provider inference. Issue #20 remains open for the capped live engineering and confirmation runs and their analysis.

@w4ffl35
w4ffl35 marked this pull request as ready for review October 1, 2026 00:18
@w4ffl35
w4ffl35 merged commit a53c3b9 into master Oct 1, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant