Skip to content

feat(rust): #259 slice 2 — own-lowered, typed Layer 2 replay/emitter - #300

Merged
PhysShell merged 3 commits into
mainfrom
claude/own259-rust-lowered-replay
Jul 18, 2026
Merged

feat(rust): #259 slice 2 — own-lowered, typed Layer 2 replay/emitter#300
PhysShell merged 3 commits into
mainfrom
claude/own259-rust-lowered-replay

Conversation

@PhysShell

@PhysShell PhysShell commented Jul 18, 2026

Copy link
Copy Markdown
Owner

Что и зачем

Второй slice #259 в границе, зафиксированной при merge #299: typed Rust replay/emitter только для rust_replay: true fixtures — без lowering, без OwnIR validation, без MOS, без analysis wiring и без решения #294 попутно. Новый leaf data crate own-lowered (прецедент — own-diagnostics): строгая (deny_unknown_fields на каждом shape) типизированная модель замороженной Layer 2 поверхности (LOWERED_VERSION = 1), зеркалящая нормализационные решения ownlang/lowered.py поле-в-поле (declaration order = canonical JSON order; always-written nullable поля — Option без skip; handle-allowlist — skip-при-отсутствии, handle первым: schema-nullable тройка type/source/source_typeMaybe<T> (Missing / explicit Null / Value — членство ключа и явный null различимы и переживают round-trip), остальные optional-ключи отвергают present null через presence-deserializer вместо тихого удаления; Param.type non-nullable; закрытый Stmt-словарь под stmt-тегом; Rejected с fail-loud текстом), плюс канонический emitter. parse_document требует lowered_version == LOWERED_VERSION на каждом документе (accepted и rejected). tests/replay.rs читает typed manifest, сам перечисляет fixture-директорию и требует ledger-равенства (unique(manifest names) == *.facts.json == *.golden.json), и для каждого из 26 shared кейсов доказывает: golden парсится в типизированную модель и re-emit байт-в-байт равен Python-golden. rust_replay: false набор утверждается равным ровно {tolerant_unknown_kind} (OD-2/#294 снапшот) — пропуск по решению ledger, не молча. DAG-lock (own-diagnostics/tests/dag.rs) расширен сознательно: own-lowered — leaf; будущий own-bridge будет конструировать эти типы, не наоборот.

Факт, который доказан тестом, а не предположен: serde_json pretty printer воспроизводит Python json.dumps(indent=2, ensure_ascii=False) байт-в-байт на всей поверхности (включая raw-UTF-8 em-dash в error-тексте).

Ревизия по review (red 8c41e22 → green 159f45c): presence-aware Maybe<T> + null-отвергающие optional-ключи; новый shared fixture handles_null_metadata (explicit nulls vs отсутствие ключей, авторизован Python --write); non-nullable Param.type + негативный тест; per-document version gate + негативные тесты v99; Rust-side ledger equality.

Тип изменения

  • feat — новая возможность
  • fix — исправление бага
  • docs — документация
  • refactor / chore / test / ci — без изменения поведения

Как проверено

  • cargo test — весь workspace зелёный, включая новый replay (26 shared кейсов byte-exact), strictness-негативы и DAG fitness-тест с новым leaf
  • cargo fmt --check, cargo clippy --all-targets — 0 warnings
  • python tests/run_tests.py, ruff, mypy — зелёные; Python verdict-поведение не менялось (diff в rust/ + новая fixture-пара/manifest + одна запись в dag.rs); повторный --write — no-op
  • typed strictness: deny_unknown_fields везде — Python-side изменение поверхности ломает replay, а не проскальзывает

Связанные issue

Refs #259 (slice 2 — НЕ закрывает issue), #258, #294 (не решается — снапшот остаётся Python-only), #250.

Чеклист

  • изменение покрыто тестом/селфтестом
  • README/docs обновлены при необходимости (crate-описание + DAG rationale в коде)
  • коммиты в conventional-commit стиле

Независимый review: APPROVED на head 159f45c (после REQUEST CHANGES → ревизии); merge авторизован reviewer'ом.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MdYFKUaygHz1T9H1qJ7BqK


Generated by Claude Code

Summary by CodeRabbit

  • New Features

    • Added a typed Layer 2 lowered-data surface for parsing and producing canonical JSON.
    • Added strict validation for versions, fields, nullable metadata, and document shapes.
    • Added support for replaying and verifying shared lowered-data fixtures.
  • Bug Fixes

    • Preserved distinctions between missing and explicitly null handle metadata.
  • Tests

    • Added round-trip, strictness, and fixture replay coverage, including null metadata cases.

The typed Rust half of the frozen Layer 2 contract (LOWERED_VERSION 1): a
strict data model of the normalized lowered representation plus the
canonical emitter, replaying the Python-authored goldens byte-for-byte.
No lowering, no OwnIR validation, no MOS, no analysis wiring — this slice
proves Rust can carry and emit the exact surface before anything derives it.

* crates/own-lowered — a LEAF data crate (own-diagnostics precedent; the
  DAG lock in own-diagnostics/tests/dag.rs is widened deliberately with the
  leaf entry and the rationale: the future own-bridge will CONSTRUCT these
  types, never the reverse). Model mirrors ownlang/lowered.py's frozen
  normalization decisions field-for-field: declaration order = canonical
  JSON order; always-written nullable fields are Option WITHOUT skip; the
  handle-entry allowlist keys are Option WITH skip, `handle` first; the
  closed Stmt vocabulary under the `stmt` tag; Rejected carries the
  fail-loud error text. Every shape is deny_unknown_fields — a Python-side
  surface change cannot slip past the typed replay.
* tests/replay.rs — reads manifest.json (typed, strict), requires
  lowered_version == LOWERED_VERSION, both fixture halves on disk for every
  case, then for each rust_replay: true case parses the golden and asserts
  the canonical re-emit is byte-identical (>= 25 shared cases). The
  rust_replay: false set is asserted to be exactly the OD-2/#294 snapshot —
  Python-only by ledger decision, not silently skipped.

serde_json's pretty printer reproduces Python json.dumps(indent=2,
ensure_ascii=False) exactly on this surface — proven by the byte-equality
suite, not assumed. cargo fmt/clippy clean, full workspace tests green
(incl. the DAG fitness test), Python suite untouched and green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01MdYFKUaygHz1T9H1qJ7BqK
@coderabbitai

coderabbitai Bot commented Jul 18, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Adds the own-lowered Rust workspace crate with a strict typed Layer 2 JSON model, canonical serialization APIs, version and nullability validation, replay tests, and a new lowered fixture covering explicit null handle metadata.

Changes

Lowered surface parity

Layer / File(s) Summary
Workspace crate registration
rust/Cargo.toml, rust/crates/own-diagnostics/tests/dag.rs, rust/crates/own-lowered/Cargo.toml, rust/crates/own-lowered/src/lib.rs
Registers own-lowered in the workspace, defines its serde dependencies, enforces its dependency-free DAG boundary, and re-exports the typed surface API.
Typed surface and canonical JSON
rust/crates/own-lowered/src/model.rs
Defines strict lowered document types, statement and handle models, version checks, missing-versus-null semantics, parsing, and canonical pretty-JSON emission.
Replay and strictness validation
rust/crates/own-lowered/tests/*, tests/fixtures/lowered/*
Adds replay and negative tests, validates fixture manifest integrity, and introduces a replayable fixture for explicit null handle metadata.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant LoweredGolden
  participant own_lowered
  participant CanonicalEmitter
  participant ReplayTest
  LoweredGolden->>own_lowered: parse_document(golden JSON)
  own_lowered->>CanonicalEmitter: typed Surface
  CanonicalEmitter->>ReplayTest: canonical JSON
  ReplayTest->>LoweredGolden: compare byte-for-byte
Loading
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and clearly names the new typed Rust Layer 2 replay/emitter work.
Description check ✅ Passed The description matches the template with all required sections filled and explains scope, testing, issues, and checklist.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/own259-rust-lowered-replay

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

claude added 2 commits July 18, 2026 18:51
…uality

The typed model collapses a MISSING handle key and an explicit null into
one state, so a valid Python golden carrying the schema-nullable trio
(type/source/source_type as null) cannot round-trip byte-exactly. Pinned
by a new shared fixture (handles_null_metadata, rust_replay: true) whose
first handle carries all three explicit nulls and whose second carries
none — the Python emitter distinguishes them by key membership.

Also pinned red:
* lowered_version parsed but never enforced — version 99 documents
  (accepted and rejected surfaces alike) currently parse fine;
* Param.type is nullable in Rust while Python's AST declares TypeRef,
  never TypeRef | None — "type": null must be rejected;
* a non-nullable optional handle key with explicit null (released: null)
  is silently decayed to missing instead of rejected;
* the replay suite outsourced ledger integrity to Python — it now
  asserts unique(manifest names) == facts files == golden files itself
  (green from birth; enforcement, not a bug pin).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdYFKUaygHz1T9H1qJ7BqK
….type

* Maybe<T> (Missing / Null / Value) for the schema-nullable handle trio
  (type, source, source_type): explicit JSON null now survives a
  parse->emit round trip instead of collapsing into "missing" — the
  Python emitter distinguishes the two by key membership, so the typed
  model must too.
* Every other optional handle key rejects a present null
  (deserialize_with = "present") rather than decaying it to absent and
  silently deleting it on re-emit.
* Param.type is non-nullable (TypeShape, not Option): Python's AST
  declares TypeRef, never TypeRef | None, so "type": null is a shape
  the emitter cannot produce and the model no longer accepts.
* parse_document enforces lowered_version == LOWERED_VERSION on BOTH
  surfaces (accepted and rejected) — the lockstep promise is now a parse
  error, not something only the manifest check notices.

All 26 rust_replay cases (incl. the new handles_null_metadata fixture)
replay byte-identically; ledger equality (unique manifest names ==
facts files == golden files) is enforced Rust-side.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdYFKUaygHz1T9H1qJ7BqK

Copy link
Copy Markdown
Owner Author

Ревизия по review (REQUEST CHANGES) — применена, head 159f45c (red 8c41e22 → green 159f45c).

Пакет ревизии, по пунктам:

  1. Presence-aware handle fields — новый Maybe<T> (Missing / Null / Value) для schema-nullable тройки type / source / source_type; explicit null теперь переживает parse→emit byte-exact. Все остальные optional-ключи handle-entry отвергают present null (deserialize_with = "present") вместо тихого превращения в "missing" и удаления при re-emit.
  2. Новый shared fixture handles_null_metadata (rust_replay: true, rules BR-L2/BR-D2), авторизован Python-эмиттером через --write: первый handle несёт все три explicit null, второй — ни одного ключа; golden пиннит различие членства. Без version bump — это покрытие существующего v1-контракта.
  3. Param.type non-nullableTypeShape вместо Option<TypeShape> (в Python AST Param.type: TypeRef, не TypeRef | None); негативный тест rejects_a_null_parameter_type. Function.ret остаётся nullable.
  4. Version gateparse_document теперь требует lowered_version == LOWERED_VERSION на обеих поверхностях (accepted и rejected); негативные тесты на v99 для обеих.
  5. Ledger equality Rust-side — replay сам перечисляет FIXDIR и требует unique(manifest names) == *.facts.json == *.golden.json; дубликаты имён — красный билд. replayed >= 25 оставлен как minimum-coverage (фактически 26).
  6. tolerant_unknown_kind — по-прежнему единственный rust_replay: false (Bridge contract OD-1/2/3: pin the tolerant door — direct check_facts() diverges from load() (unknown kind fallback, line coercion) #294 не решён).

Red-состояние зафиксировано: 6 падающих тестов (byte-mismatch replay на null-fixture; v99 принимался ×2; "type": null у параметра принимался; released: null тихо исчезал; null-metadata терялась в round-trip) — все зелёные после 159f45c.

Локальные гейты: python suite (27 lowered cases in sync), ruff — зелёные; cargo fmt / clippy (0 warnings) / полный workspace test — зелёные; повторный --write — no-op (детерминизм).

Nonblocking (rules non-empty validation) — оставлен на #296, как записано.

PR остаётся draft — жду повторного review. Не мержить, не self-accept.


Generated by Claude Code

Copy link
Copy Markdown
Owner Author

Independent review verdict (head 159f45c): APPROVED, no blocking findings.

Verified presence-aware missing/null/value handling, rejection of null for non-nullable handle fields and parameter types, per-document LOWERED_VERSION enforcement, exact Rust-side manifest/facts/goldens equality, 26 shared byte-exact replays, and tolerant_unknown_kind as the sole #294 Python-only case. Full CI is green.

Merge authorized by the reviewer on exact head 159f45cc93c2222e792b7135a4d718a483f31dca. Nonblocking housekeeping (stale PR body: 25→26 shared cases; Option-with-skip → Maybe<T> + null-rejecting presence deserializer) is applied to the body before merge.

Post-merge, #259 slice 2 is ACCEPTED / MERGED / FROZEN: own-lowered owns the typed LOWERED_VERSION = 1 data surface and byte-exact replay only — validation, lowering, MOS inference, and analysis wiring remain unauthorized until separately reviewed.


Generated by Claude Code

@PhysShell
PhysShell marked this pull request as ready for review July 18, 2026 22:55

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@rust/crates/own-lowered/tests/replay.rs`:
- Around line 111-114: Update the replay count assertion in the replay test to
require at least 26 shared cases instead of 25, while preserving the existing
failure message and replayed-count interpolation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 615c2702-36bb-408d-a142-47e1f9e09ced

📥 Commits

Reviewing files that changed from the base of the PR and between 1a383ed and 159f45c.

⛔ Files ignored due to path filters (1)
  • rust/Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (10)
  • rust/Cargo.toml
  • rust/crates/own-diagnostics/tests/dag.rs
  • rust/crates/own-lowered/Cargo.toml
  • rust/crates/own-lowered/src/lib.rs
  • rust/crates/own-lowered/src/model.rs
  • rust/crates/own-lowered/tests/replay.rs
  • rust/crates/own-lowered/tests/strictness.rs
  • tests/fixtures/lowered/handles_null_metadata.facts.json
  • tests/fixtures/lowered/handles_null_metadata.golden.json
  • tests/fixtures/lowered/manifest.json

Comment thread rust/crates/own-lowered/tests/replay.rs
@PhysShell
PhysShell merged commit b9879b3 into main Jul 18, 2026
41 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 159f45cc93

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rust/crates/own-lowered/tests/replay.rs
PhysShell pushed a commit that referenced this pull request Jul 19, 2026
The normative documents described the pre-slice-3 world. Recorded now:

* spec/Bridge.md §6 Layer 2: #299 (Python emitter + fixture foundation),
  #300 (typed own-lowered surface/emitter, presence-aware metadata,
  per-document version gate), #301 (own-bridge constructs Layer 2 from
  facts and reproduces all 26 shared goldens byte-exact; golden is
  expected output only); tolerant_unknown_kind stays Python-only under
  #294 (the Rust bridge fails loud on an unknown kind instead of adopting
  the tolerant fallback); Layer 1, Layer 3, analysis wiring and #259 as a
  whole remain open.
* spec/BridgeBehaviorMatrix.md: the Layer 2 summary paragraph updated to
  the same completed state.
* tests/test_lowered_fixtures.py docstring: future tense ("will replay",
  "until the Rust emitter exists") moved to the present.
* BR-L7 wording nit folded in (recorded on #297): condition (4) guards the
  post-merge reference/discharge, not only a release — the use-only hoist
  fixture is in the gate too.

Docs-only; no Rust code, facts, manifest or golden changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MdYFKUaygHz1T9H1qJ7BqK
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants