Consume the schema-8 families instead of merely checking them out - #43
Merged
Merged
Conversation
The declared candidate suite was 432eb2ee, whose conformance/v1/manifest.json is byte-identical to the released pin 0c81c1f8. Two immutable revisions, one suite: every candidate run so far measured exactly the corpus the default protocol lanes already measure. Schema 8 then landed with its corpora unread, because IN_SCOPE_SCHEMA_NAMES in tests/test_protocol_conformance.py is rc.6-era and that module is pinned to the released manifest anyway. A schema-8 candidate could have been checked out, digest-matched and reported green while every schema-8 case sat unread. Presence in a root is not consumption, and a green run is not evidence that a named case still runs. Those are two claims, so they get two ledgers. Presence -- .github/ci/candidate-artifacts.tsv declares the root-relative artefacts each consumer's unguarded reads require. The partition is derived from the root, never from the lane: a root publishing the surface serves the consumer, a root publishing none of it defers it by name, and the candidate lane runs candidate_consumption.py require plus CSK_REQUIRE_FULL_CANDIDATE_ROOT=1 so a partly published surface fails instead of running quieter. Consumption -- .github/ci/candidate-cases.tsv names the tests the candidate lane must observe passing on every runner, enforced by candidate_consumption.py gate against that run's own junit stream. A row covers every parameterization of its test, so a family that stops generating cases fails as loudly as one that starts failing them. The consumer, tests/test_schema8_candidate_conformance.py, authenticates every byte against the candidate manifest and then decides real behaviour: 132 agent-skill-v8 and csk-skill-v8 cases through skillspec.load_skill_spec, 27 install-marker-v4 cases through install_marker.read_install_marker with a byte-exact round-trip, 10 module-roots vectors through the go_v1 seams at their own go-list/go-build failure boundary, and the whole script-worker family: section classification in both directions, the closed policy and interpreter identity against the suite's bytes, every opt-in case's enforced/declared-only classification, and a refusal reaching the single shim publication point. audit_label_cases is classified as a declared gap with its owner rather than as unreachable, because this manager emits no script-command audit label. The candidate declaration moves to 6001dc3 (manifest sha256:803918bf...b44403, tree sha256:d2c9d3df...ed2656, protocol 1.0.0-rc.9). The released pin is unchanged in value and still read back to reject impersonation. Consuming install-marker-v4 also settles the four cases the schema-8 review logged as inherited schema-7 semantics. They are reader bugs, not semantics: common.schema.json admits revision, tag and branch identically on both pins, so branch was never a schema-8 addition, and Core 4.2 binds a local-path substitution to effective identity kind operator-local-git, a network-git substitution to network-git, and a structured revision to the effective object format. install_marker now decides all three, which takes install-marker-v4 to 27/27 and install-marker-v3 from 23/27 to 27/27 against the released pin.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the false green the schema-8 impact analysis measured, on the Python side.
Mirrors curator PR 37 (
relux-works/curator#37) for this implementation.The hole
The declared candidate suite was
432eb2ee, whoseconformance/v1/manifest.jsonis byte-identical to the released pin
0c81c1f8(sha256:12e58b82…on both).Two immutable revisions, one conformance suite: every candidate run so far
measured exactly the corpus the default protocol lanes already measure, and no
assertion could have noticed. Identity authentication proves you read the right
bytes; it never proves the bytes differ from the pin, or that anything read them.
Schema 8 then landed (#40) with its corpora unread:
IN_SCOPE_SCHEMA_NAMESintests/test_protocol_conformance.pyis rc.6-era, and that module is pinned tothe released manifest anyway.
agent-skill-v8,csk-skill-v8,install-marker-v4,vectors/module-roots.jsonandvectors/script-host-execution-policy.jsonwere all checked out and never opened.What changes
Presence and consumption are two different claims, so they get two ledgers.
Presence —
.github/ci/candidate-artifacts.tsvdeclares the root-relativeartefacts each consumer's unguarded reads require. The partition is derived from
the root, never from the lane: a root publishing the surface serves the
consumer, a root publishing none of it defers it and names what was absent
(this is what the released-pin lanes do). The candidate lane runs
candidate_consumption.py requireand exportsCSK_REQUIRE_FULL_CANDIDATE_ROOT=1, so a partly published surface fails ratherthan running quieter.
Consumption —
.github/ci/candidate-cases.tsvnames the 11 tests thecandidate lane must observe passing on every runner, enforced by
candidate_consumption.py gateagainst that run's own--junitxmlstream. A rowcovers every parameterization of its test.
New consumer —
tests/test_schema8_candidate_conformance.pyauthenticatesevery byte against the candidate manifest and then decides real behaviour:
agent-skill-v8+csk-skill-v8cases throughskillspec.load_skill_specon a materialized snapshot;
install-marker-v4cases throughinstall_marker.read_install_marker, witha byte-exact
to_json()round-trip;module-rootsvectors through the realgo_v1seams, asserting thediagnostic and the phase (
go-listvsgo-build), plus the publishedevaluation_order;closed policy identity and interpreter set against the suite's own bytes, every
opt-in case's enforced/declared-only classification, and a refusal reaching the
single shim publication point.
audit_label_casesis classified as a declared gap with its owner, not asunreachable: this manager emits no
script-command-declared-only/script-command-unfiltered-declared-networklabel. Every other worker-sidesection is
refused before reached, andtest_a_refusal_precedes_every_worker_surfaceis the assertion those rest on.Candidate —
.github/ci/candidate-suite.jsonnow declares6001dc3(
sha256:803918bf…b44403, treesha256:d2c9d3df…ed2656,1.0.0-rc.9). Thereleased pin is unchanged in value and still read back to reject impersonation.
Marker substitution — consuming
install-marker-v4settles the four cases#40's review logged as inherited schema-7 semantics. They are reader bugs:
common.schema.jsonadmitsrevision,tagandbranchidentically on bothpins, so
branchwas never a schema-8 addition, and Core 4.2 binds alocal-pathsubstitution to effective identity kindoperator-local-git, anetwork-gitsubstitution tonetwork-git, and a structuredrevisionto theeffective object format.
install_markernow decides all three:install-marker-v4goes to 27/27 andinstall-marker-v3from 23/27 to 27/27against the released pin.
Evidence
Removing any of the five schema-8 families from the candidate root fails the
suite, in both removal modes:
schema-cases/agent-skill-v8requireexit 1, namedschema-cases/csk-skill-v8requireexit 1, namedschema-cases/install-marker-v4requireexit 1, namedvectors/module-roots.jsonrequireexit 1, namedvectors/script-host-execution-policy.jsonrequireexit 1, namedA run that stops executing a declared case is green to pytest and red to the gate:
--deselect ...::test_module_root_caseexits 0 with 173 passed, andcandidate_consumption.py gate --platform linuxexits 1 naming the missing case.Reverting only
src/csk/install_marker.pyturnstests/test_install_marker_substitution.py7-red and the consumer 4-red onexactly the four cases the review named.
Local gates, all exit 0: consumer 183 passed against the rc.9 root; gate green
for linux/darwin/windows;
tests/test_candidate_consumption.py28 passed;tests/test_install_marker_substitution.py16 passed;tests/test_ci_workflow.py13 passed; ordinary suite 1858 passed / 63 skipped; full
tests/test_protocol_conformance.pyagainst the released pin 1044 passed / 1skipped; Go E2E native selection against the rc.9 candidate;
python -m mypy75 files;
actionlint.