Conversation
`jitM` was applied by the repository *after* the store had already decoded the document, and was typed `(pm: Encoded) => Encoded` - a lie, since jitMs are written against the stored JSON and `Encoded` holds native Date/Map/Set. The read pipeline is now: raw JSON document -> merge `defaultValues` (unchanged: they only fill absent keys) -> `jitM` (JSON -> JSON) -> decode `toCodecJson(toEncoded(schema))` -> Encoded. The repository only decodes Encoded -> the domain type, so `jitM` can repair legacy shapes - including explicit `null`s - before any schema decode sees the document. - `StoreConfig.jitM?: (json: JsonRecord) => JsonRecord` is new, alongside the exported `JsonRecord` alias; it is not applied on the write/encode path and never receives `_etag` - `RepositoryOptions.jitM` changes to the same signature and is forwarded into the store config instead of into `makeRepoInternal` - `makeRepoInternal` loses its `mapFrom` parameter; `mapReverse` now only splits `_etag` and calls `setEtag` - `validateSample` decodes the store's output directly; `ValidationError.jitMResult` is deprecated as it is identical to `rawData` Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Migrate the legacy-document suite to a JSON-shaped `jitM` running at the store boundary, and add cases for what the new pipeline makes possible: - `jitM` repairs an explicit `null` (not a valid `S.Date` encoding, so no schema decode could ever fix it) and the document decodes through `find`/`all` - `jitM` receives the raw JSON document - a Date as an ISO string, a ReadonlySet as an array, a ReadonlyMap as an array of pairs, and no `_etag` - and the JSON it returns is what the decode consumes - `defaultValues` still only fill absent keys, are merged before `jitM` runs, and a stored `null` reaches `jitM` instead of being replaced by the default `MemoryStoreLive` still cannot seed a legacy document (it strictly `encode`s its seed), so the suite keeps a schemaless Memory harness that applies the real read boundary on top. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Member
Author
|
Superseded by #896, which is the same pipeline rebuilt directly on Also note CI never ran here: libs |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #894 — review that one first (base branch is
fix/validate-sample-json-codec).Why
jitMwas typed(pm: Encoded) => Encodedand ran in the repository, after the store had already decoded the document. Since #874 madeEncodedhold nativeDate/Map/Set, that type was wrong in both directions: jitMs are written against the document as stored (ISO strings, arrays of pairs), and by the time one ran, the schema decode it was supposed to repair had already happened.What changes
The pipeline is now, per the owner's design:
defaultValues(still only filling absent keys) →jitM: JSON → JSON → decodeSchema.toCodecJson(Schema.toEncoded(schema))→ returns the Encoded shape.jitMat all.So a migration can repair legacy shapes — including an explicit
null, which correctly wins over adefaultValuesentry becausenullis a value, not an absent key — before any schema decode sees the document.makeReponow forwardsjitMinto the store config;makeRepoInternallost itsmapFromparameter andmapReverseonly splits_etag. New infra helpermakeStoredDecode(codec, jitM)keeps the read boundary in one place and splits_etagoff first, sojitMnever sees infrastructure metadata. Applied in Cosmos (fromStored), SQL/Pg (parseRow), Memory (decodeDoc) and Disk (file-read boundary, exactly once). Not applied on the write path.StoreMaker.make's positional signature is unchanged.Breaking changes for consumers
jitMis(json: JsonRecord) => JsonRecord— no longer generic overEncoded.Date→ ISO string,ReadonlySet→ array,ReadonlyMap→ array of[k, v]pairs. Returning aDate/Map/Setis now wrong.json["x"]);"x" in jsonstill works._etagis not part of the document passed tojitM.ValidationError.jitMResultis@deprecatedand equalsrawData(the repository can no longer observe a pre-jitMdocument). Kept rather than removed to avoid breaking consumers — say the word and I'll delete it.Verification
pnpm check(tsgo): clean.packages/infra: 288 passed / 26 skipped.packages/effect-app: 185 passed. Lint clean.jitMrepairs an explicitnulland the document then decodes end-to-end;jitMreceives the raw JSON document and its JSON result is what the decode consumes;defaultValuesonly fill absent keys and a storednullstill reachesjitM.Notes
MemoryStoreLivestill cannot seed a legacy document — its seed path strictly encodes the whole document — so the legacy-document test keeps a schemaless Memory harness with the real read boundary applied on top. Worth deciding separately whether writes should fail loudly or lower leniently.selectand CosmosqueryRawbypass the document decode and therefore do not runjitM. Pre-existing behaviour, unchanged here.defaultValueson read remain inconsistent between adapters (Cosmos/SQL merge on every read; Memory at seed time; Disk before its inner store). Left alone deliberately.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.