Skip to content

feat(spike): prove Effect v4 canonical schema and adapters end-to-end (Entry + Event) - #7

Draft
obvious-autobuild[bot] wants to merge 1 commit into
masterfrom
spike/effect-contracts-adapters
Draft

obvious-autobuild[bot] wants to merge 1 commit into
masterfrom
spike/effect-contracts-adapters

Conversation

@obvious-autobuild

Copy link
Copy Markdown
Contributor

Why

The monorepo's domain layer is bet on schema-first contracts, but nobody had proven that Effect v4 (rc line) can actually carry the canonical journal schema across all three boundaries it must survive: inferred TypeScript types, derived JSON Schema (for LLM function calling), and Convex validators (persistence). This spike proves that chain end-to-end on ONE canonical schema — Entry + Event — with every claim test-verified and every version pinned, so the scaffold's packages/domain can integrate against evidence instead of guesses.

What

All under spikes/effect-compat/ (bounded scope — no scaffold files touched):

  • Canonical executable schema (src/schema.ts) on effect@4.0.0-rc.115: six event categories (potty|meal|sleep|mood|milestone|school), DateFromMillis timestamp codec, bounded confidence (isBetween(0,1)), optional quantitative payload, author attribution, tagged Entry + Event structs. All TS types inferred via Schema.Schema.Type — zero hand-written interfaces.
  • JSON Schema derivation (src/json-schema.ts): Schema.toJsonSchemaDocument produces a real draft-2020-12 document (category enum, integer timestamps, bounded confidence — verified by running it). Two negative results recorded with evidence: effect/JSONSchema no longer exists in v4, and toStandardJSONSchemaV1 yields no document on this pin.
  • Effect → Convex adapter (src/convex-adapter.ts): walks the public SchemaRepresentation tree and emits convex/values validators for the encoded wire form. Round-trip proven: encode → convexToJson → jsonToConvex → decode deep-equal, Dates restored, absent optionals survive. Unsupported shapes (tuples, index signatures) fail loudly with UnsupportedRepresentationError — never silently. Gotcha recorded for the scaffold owner: Convex 1.46 marks optionality as isOptional: 'required'|'optional' on the validator itself; truthiness checks on that string silently break optionality.
  • LLM extraction contract (src/llm/contract.ts + openai.ts): transcript in → typed events out, always decoded through the canonical schema (strict fail-fast with the failing index). Typed errors only (ProviderFailure / MalformedModelOutput / EventsFailedSchema). Results carry a captureId + monotonically increasing attempt envelope: client interruption does NOT cancel a running server extraction — the backend completes it idempotently, and stale attempts are suppressed (only the latest attempt per capture applies). OpenAI sits behind OPENAI_API_KEY behind a structural client so tests are fully mocked and offline.
  • Visibility vs audience as independent dimensions (contract v0.2, per Gil): Entry.visibility: 'draft'|'published' is per-entry publication state only; WHO may see a shared entry is resolved from household/relationship grants and is deliberately not modeled on the entry — no fused status enum, no audience field. Pinned by tests.

Verification: bunx tsc --noEmit strict-clean; bunx vitest run → 37/37 passing across 4 suites. Full details, pinned versions, and non-goals in FINDINGS.md.

How to Review

  • Start with FINDINGS.md — it is the deliverable's summary of what was proven vs. explicitly not proven (live OpenAI calls, tuple/index-signature support, Convex deployment are all out of scope).
  • The adapter is the single sanctioned representation bridge; the scaffold owner integrates it into packages/domain. This PR is for the record — do not merge; findings feed the scaffold PR.
  • scripts/probe-*.mjs are the exploratory probes kept as evidence of the version-behavior checks (not production code).

Test Evidence

Local: strict typecheck clean; bunx vitest run 37/37 passing (schema semantics, JSON Schema derivation, adapter structure + Convex round-trips, mocked LLM contract incl. env-key gating). Suite runs fully offline — no network, no API key.

Human author: Gilbert Polanco (gilbertpolanco42@gmail.com)

🔗 Obvious Project · 🧵 Obvious Thread

Bounded compatibility spike in spikes/effect-compat/ on effect@4.0.0-rc.115,
convex@1.46.0, openai@7.17.0:

- Canonical executable Entry + Event schema with fully inferred TS types
  (six categories, DateFromMillis codec, bounded confidence, optional quantity).
- JSON Schema derivation via Schema.toJsonSchemaDocument (draft-2020-12),
  verified with real imports; effect/JSONSchema is gone in v4 and
  toStandardJSONSchemaV1 yields no document on this pin.
- Effect schema -> Convex validators adapter (single sanctioned bridge) with
  encode -> convexToJson -> jsonToConvex -> decode round-trip deep-equality
  tests and loud failure on unsupported representations.
- LLM extraction contract: captureId + attempt envelope with stale-result
  suppression and idempotent-completion requirements; strict schema decode;
  OpenAI provider behind OPENAI_API_KEY, fully mocked in tests.
- Entry publication visibility (draft|published) kept separate from audience
  permissions (household/relationship-scoped) as independent dimensions.
- FINDINGS.md with pinned versions, verified behaviors, and gotchas;
  37/37 vitest tests passing, tsc --noEmit strict clean.

Co-authored-by: Gilbert Polanco <gilbertpolanco42@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants