Skip to content

Vendor the Spec conformance fixtures at a pinned commit (Stage 1, PR 1 of 3) - #183

Merged
guijacquemet merged 1 commit into
mainfrom
claude/spec-fixtures-pin
Nov 7, 2026
Merged

guijacquemet merged 1 commit into
mainfrom
claude/spec-fixtures-pin

Conversation

@guijacquemet

Copy link
Copy Markdown
Contributor

Summary

PR 1 of the Stage 1 design (section 5): the executable-conformance groundwork for the experiment-description reader. Tests and test data only. No change to the package, the CLI, the notebook, any output or any existing run. It waits for R7 / Gate G0 like the rest of Stage 1 and must not merge earlier.

The normative text and fixtures are in MorphoBricks-Spec at commit 8922a359ec261148a23a4b6e740a1bdb69e9e318 (docs/experiment-v1.md, docs/conformance-plan.md sections 1 and 2, fixtures/).

Stack: this is the base of a stack of three. PR 2 (reader phases 1-3) and PR 3 (phases 4-5 and warnings) are opened on top of it, against main; their diffs contain this one until it merges.

What changed

  • tests/spec_fixtures/ is a partial copy of the Spec at the pinned commit, laid out as in the Spec: fixtures/ (179 files), docs/experiment-v1.md, tools/audit/refval.py (the reference validator, test oracle only, nothing in src/ imports it). PIN.toml holds the full commit SHA, git rev-parse <sha>:fixtures (6d114d80...) and the two blob IDs. .gitattributes sets -text so no checkout converts line endings.
  • tests/test_spec_pin.py (pin integrity): the vendored fixtures/ hashes to the pin with git's tree algorithm (own implementation, checked against known git answers including the folder-sorts-as-name/ rule; git itself gives 6d114d80... for the committed folder). One changed, added, removed, renamed or CRLF-converted file fails, as do the two other files and a pin that is not a 40-digit SHA.
  • tests/spec_gate.py (+ conftest.py): a skipped, skipif, importorskip or xfailed test in any tests/test_spec_*.py is reported as a failure. Tested in a child pytest.
  • tests/test_spec_no_fallback.py ("no fallback rules"): AST scan for a second table of codes, a copy of the forbidden or reserved names (compared with the vendored text), code literals outside the one rules module, and lenient/legacy/fallback/default-version markers. The scan is tested against known-bad snippets. Until PR 2 adds description*.py the reader-module part has nothing to scan; PR 2 makes that non-vacuous.
  • tests/test_spec_vendored_missing.py: with the vendored tree empty, absent or without fixtures/, the suite fails (collection error), never passes on defaults and never skips.
  • tests/test_spec_catalogue.py: every fixture (discovered from the directory) has Fixture/Expected/Source headers matching its name and its catalogue row; the catalogue lists every file and nothing else; 36 valid, 130 invalid, 12 transitions.
  • pyproject.toml / tools/check.py: the vendored copies are excluded from ruff and the code rules (byte-exact files, not our code). docs/development.md and CHANGELOG.md describe the pin and the gate.

Regression test: new tests, so there is no old-code run. Negative check done by hand: changing one byte of v01_single_control.toml makes tests/test_spec_pin.py fail (11 failed, 5 passed); restored, 16 passed.

Notes for review

  • Pin updates are a reviewed change (tests/spec_fixtures/README.md lists the steps).
  • Not verified: the Windows and macOS CI legs (the -text attribute is what protects the hashes there).
  • No existing issue covers Stage 1, so this does not close one.

Tests

python tools/check.py: ruff check clean, ruff format clean, code rules clean, 2713 passed, 4 skipped (2316 passed, 4 skipped on main 3231966; the 397 new tests pass; the 4 skips are the same pre-existing ones in test_cli_log, test_discovery, test_folder_rules, none in a Spec test).


Generated by Claude Code

Adds tests/spec_fixtures/: the MorphoBricks-Spec fixtures/ folder, docs/experiment-v1.md and
the reference validator (a test oracle only) at Spec commit 8922a359ec26, with PIN.toml.

- tests/test_spec_pin.py: the vendored fixtures/ folder must hash (git tree algorithm) to
  `git rev-parse <sha>:fixtures`; the two single files to their blob IDs. One changed, added,
  removed, renamed or CRLF-converted file fails.
- tests/spec_gate.py: a skipped or xfailed test in any tests/test_spec_*.py is a failure.
- tests/test_spec_no_fallback.py: no second table of codes or names, no lenient, legacy or
  default-version path in the reader modules (description*.py); the scan is itself tested.
- tests/test_spec_vendored_missing.py: with the vendored tree removed the suite fails.
- tests/test_spec_catalogue.py: every fixture has its header lines and a catalogue row.

Tests only: no change to the package, the CLI, the notebook or any output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants