Skip to content

amicode_verify v2: typed rollout-spec referee backend (replaces the bespoke re-rollout) #343

Description

@aarontrowbridge

Roadmap context: vault plan `plan-20260812-142100-problemspecs-issimo-roadmap` (W2.5); master spec §"rollout kind" + "Referee independence". Depends on #340 (the originating control spec must exist to derive the referee from).

What changes

Today free-tier verification is a bespoke re-rollout: `verify_rollout.jl` re-implements goal reconstruction from `system_verify.jld2`, and `amicode_verify` records agree/disagree from an unstructured comparison. Piccolo.Specs Phase 1 shipped the typed machinery for exactly this: the `rollout` kind, the `referee_rollout(control_spec, run)` constructor (strictly finer in every resolution axis, different integrator family, forge-proof `[referee]` block re-validated at parse time), and `Verdict`/`Agree`/`Disagree` as types.

Tasks

  • `amicode_verify` v2 (opencode-plugin): derive the rollout spec from the run's retained control spec (`result.toml [params].spec` — the runner persists it), set `input.pulse` to the run's `pulse-*.jld2`, `report.fidelity` against the original goal, run synchronously.
  • This is the master spec's scoped exception to the never-launches invariant: rollouts are bounded, seconds-scale, produce no run_dir. The invariant continues to hold for `control`/`tuning`.
  • Record the typed verdict (both fidelities as witness) instead of an unstructured agree/disagree; a rollout without a valid `[referee]` block yields no verdict by construction — surface that, don't mint one.
  • Retire the `system_verify.jld2` snapshot from the free-tier skeleton once the typed path is proven on it (the skeleton's CONTRACT block emits it today; that's the migration seam).

Acceptance

  • Master-spec success criterion 6: a rollout spec derived from a completed control run reproduces the run's re-rolled fidelity to 1e-10.
  • A forged/inconsistent `[referee]` block fails validation (testitem/vitest); a hand-written rollout spec without `[referee]` runs but its result carries no `verdict`.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions