Skip to content

feat(effect-app,infra): run jitM at the store boundary as JSON → JSON - #895

Closed
patroza wants to merge 2 commits into
fix/validate-sample-json-codecfrom
feat/jitm-json-pipeline
Closed

patroza wants to merge 2 commits into
fix/validate-sample-json-codecfrom
feat/jitm-json-pipeline

Conversation

@patroza

@patroza patroza commented Sep 16, 2026 •

Copy link
Copy Markdown
Member

Stacked on #894 — review that one first (base branch is fix/validate-sample-json-codec).

Why

jitM was typed (pm: Encoded) => Encoded and ran in the repository, after the store had already decoded the document. Since #874 made Encoded hold native Date/Map/Set, that type was wrong in both directions: jitMs are written against the document as stored (ISO strings, arrays of pairs), and by the time one ran, the schema decode it was supposed to repair had already happened.

What changes

The pipeline is now, per the owner's design:

  1. Store: raw JSON → merge defaultValues (still only filling absent keys) → jitM: JSON → JSON → decode Schema.toCodecJson(Schema.toEncoded(schema)) → returns the Encoded shape.
  2. Repository: decodes Encoded → domain type. It no longer applies jitM at all.

So a migration can repair legacy shapes — including an explicit null, which correctly wins over a defaultValues entry because null is a value, not an absent key — before any schema decode sees the document.

// effect-app/Store
export type JsonRecord = { readonly [key: string]: Schema.Json }
interface StoreConfig<E> { jitM?: (json: JsonRecord) => JsonRecord /* ... */ }

// effect-app/Model/Repository — RepositoryOptions
jitM?: (json: JsonRecord) => JsonRecord

makeRepo now forwards jitM into the store config; makeRepoInternal lost its mapFrom parameter and mapReverse only splits _etag. New infra helper makeStoredDecode(codec, jitM) keeps the read boundary in one place and splits _etag off first, so jitM never sees infrastructure metadata. Applied in Cosmos (fromStored), SQL/Pg (parseRow), Memory (decodeDoc) and Disk (file-read boundary, exactly once). Not applied on the write path. StoreMaker.make's positional signature is unchanged.

Breaking changes for consumers

  1. jitM is (json: JsonRecord) => JsonRecord — no longer generic over Encoded.
  2. Values arrive as JSON: Date → ISO string, ReadonlySet → array, ReadonlyMap → array of [k, v] pairs. Returning a Date/Map/Set is now wrong.
  3. Index-signature access (json["x"]); "x" in json still works.
  4. _etag is not part of the document passed to jitM.
  5. ValidationError.jitMResult is @deprecated and equals rawData (the repository can no longer observe a pre-jitM document). Kept rather than removed to avoid breaking consumers — say the word and I'll delete it.
// add a missing key — before → after
jitM: (pm: typeof Shop.Encoded) => "vatRate" in pm ? pm : { ...pm, vatRate: 19 }
jitM: (json) => "vatRate" in json ? json : { ...json, vatRate: 19 }

// touch a Date field — before → after (ISO strings on both sides)
jitM: (pm) => pm.updatedAt ? pm : { ...pm, updatedAt: pm.createdAt }
jitM: (json) => json["updatedAt"] ? json : { ...json, updatedAt: json["createdAt"] }
// fresh timestamps: new Date().toISOString()

Verification

  • Root pnpm check (tsgo): clean.
  • packages/infra: 288 passed / 26 skipped. packages/effect-app: 185 passed. Lint clean.
  • New tests: jitM repairs an explicit null and the document then decodes end-to-end; jitM receives the raw JSON document and its JSON result is what the decode consumes; defaultValues only fill absent keys and a stored null still reaches jitM.

Notes

  • MemoryStoreLive still cannot seed a legacy document — its seed path strictly encodes the whole document — so the legacy-document test keeps a schemaless Memory harness with the real read boundary applied on top. Worth deciding separately whether writes should fail loudly or lower leniently.
  • Projections/select and Cosmos queryRaw bypass the document decode and therefore do not run jitM. Pre-existing behaviour, unchanged here.
  • defaultValues on read remain inconsistent between adapters (Cosmos/SQL merge on every read; Memory at seed time; Disk before its inner store). Left alone deliberately.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

patroza and others added 2 commits September 16, 2026 08:11
`jitM` was applied by the repository *after* the store had already decoded the
document, and was typed `(pm: Encoded) => Encoded` - a lie, since jitMs are
written against the stored JSON and `Encoded` holds native Date/Map/Set.

The read pipeline is now: raw JSON document -> merge `defaultValues` (unchanged:
they only fill absent keys) -> `jitM` (JSON -> JSON) -> decode
`toCodecJson(toEncoded(schema))` -> Encoded. The repository only decodes
Encoded -> the domain type, so `jitM` can repair legacy shapes - including
explicit `null`s - before any schema decode sees the document.

- `StoreConfig.jitM?: (json: JsonRecord) => JsonRecord` is new, alongside the
  exported `JsonRecord` alias; it is not applied on the write/encode path and
  never receives `_etag`
- `RepositoryOptions.jitM` changes to the same signature and is forwarded into
  the store config instead of into `makeRepoInternal`
- `makeRepoInternal` loses its `mapFrom` parameter; `mapReverse` now only splits
  `_etag` and calls `setEtag`
- `validateSample` decodes the store's output directly; `ValidationError.jitMResult`
  is deprecated as it is identical to `rawData`

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Migrate the legacy-document suite to a JSON-shaped `jitM` running at the store
boundary, and add cases for what the new pipeline makes possible:

- `jitM` repairs an explicit `null` (not a valid `S.Date` encoding, so no schema
  decode could ever fix it) and the document decodes through `find`/`all`
- `jitM` receives the raw JSON document - a Date as an ISO string, a ReadonlySet
  as an array, a ReadonlyMap as an array of pairs, and no `_etag` - and the JSON
  it returns is what the decode consumes
- `defaultValues` still only fill absent keys, are merged before `jitM` runs, and
  a stored `null` reaches `jitM` instead of being replaced by the default

`MemoryStoreLive` still cannot seed a legacy document (it strictly `encode`s its
seed), so the suite keeps a schemaless Memory harness that applies the real read
boundary on top.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@patroza

patroza commented Sep 16, 2026

Copy link
Copy Markdown
Member Author

Superseded by #896, which is the same pipeline rebuilt directly on main instead of stacked on #894 (that one is closed too, since the lenient decode it added is being dropped).

Also note CI never ran here: libs ci.yml triggers on pull_request: branches: [main], and this PR targeted #894's branch. #896 gets a real CI run.

@patroza patroza closed this Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant