Skip to content

feat(retrieval): bounded read-only history query path over synthetic corpus - #24

Merged
obvious-autobuild[bot] merged 1 commit into
masterfrom
feat/remember-retrieve-expansion
Sep 17, 2026
Merged

obvious-autobuild[bot] merged 1 commit into
masterfrom
feat/remember-retrieve-expansion

Conversation

@obvious-autobuild

Copy link
Copy Markdown
Contributor

Human author: Gilbert Polanco (gilbertpolanco42@gmail.com)

Acceptance criteria

  1. Rule-2 schema/operation delta proposals, never self-landed — the query contracts (HistoryQueryInput, HistoryQueryPlan, HistoryQueryAnswer) exist as the executable form of a proposal in packages/retrieval/src/queryContracts.ts; canonical packages/domain is NOT modified, and the contracts reuse canonical EventFields.category + convexId() so a later promotion into the v0.3 fold derives identical validators.
  2. Independent expected-answer fixtures over the 30-day synthetic corpus — 14 hand-derived fixtures in packages/retrieval/test/expectedAnswers.ts, derived from the corpus timeline, not from running the implementation (slot 03's fixture branch never existed remotely, so the brief's deterministic fallback applies).
  3. Bounded agreed read-only implementation behind contracts, on synthetic data — resolver (fixed phrase inventory → plan), engine (cited answers with coverage + notices), source navigation; no write path, no LLM call, no real user data.
  4. Evidence: fixtures + tests + CI green — 22 bun tests (fixtures, contract decoding, corpus validity, determinism, read-only discipline, superseded-event exclusion, citation navigation), full workspace gates green locally on the tested HEAD (see Test Evidence). Backend-only change: no rendered UI is touched, so visual PR evidence does not apply.

Why

Slots 15/17 (NL history queries, resumable context) are blocked on having a grounded query surface to target. The product needs caregivers to ask "when did Ada last nap?" and get an answer that cites its sources, discloses what it does not know, and never silently serves corrected-away data. This PR builds that surface as an executable proposal: the shape slots 15/17 will implement against, proven on a corpus engineered so every retrieval behavior in scope is hand-checkable.

What

New package @journal/retrieval (workspace member, turbo-covered):

  • queryContracts.ts — proposal schemas: half-open QueryWindow; AnswerCitation (entryId + eventIds + verbatim excerpt); AnswerCoverage (confirmed/inferred/total — the T2 confidence/coverage disclosure, routed as a rule-2 proposal, not self-landed); AnswerNotice union (correction / conflict / gap); tagged HistoryQueryAnswer (found / ambiguous / clarify / not-found); tagged HistoryQueryPlan (last-event / count-events / probe-events / day-summary); HistoryQueryInput with household scope, pinned child, asker zone.
  • corpus.ts — deterministic 30-day fallback corpus (Aug 14–Sep 12 2026, EDT household): two children, alternating breakfast authors, a pinned Aug 22–23 entry gap, an append-only nap correction (Aug 20), conflicting pasta authors (Aug 17), a shared-dinner ambiguity source (Aug 30), a caregiver-confirmed school event (Sep 5). Every record decodes through canonical @journal/domain schemas at generation time (fail-fast); two runs are byte-identical.
  • resolve.ts — bounded compiler: four question forms, activity lexicon, pronoun handling (clarifies rather than guesses; honors a pinned childId scope), zone-aware windows (today, yesterday, this week, last week, on <date>, between a and b, since <date>) with two-pass offset convergence (DST-safe).
  • engine.ts — read-only executor: supersession filtering (superseded events are never served), correction/conflict/gap notices bounded to the answered slice, deterministic citation order, ambiguity candidates (latest match per child), and resolveEntrySource for citation → verbatim record navigation.

Deliberately excluded: canonical domain changes (v0.3 fold thread owns promotion), any Convex/mobile wiring, and any NL layer beyond the bounded inventory — richer NL targets the same plan schema.

How to Review

  • Start at packages/retrieval/README.md (invariants), then queryContracts.ts (the proposal), then the fixture file — each fixture's expected statement/citations/coverage is hand-derived and commented with the corpus arithmetic.
  • test/expectedAnswers.ts hq-14 encodes a trust invariant: a pronoun question under a Milo-scoped view returns honest not-found — never Ada's naps.
  • The test suite asserts engine output satisfies HistoryQueryAnswer BEFORE deep-equal; that ordering caught a real contract violation during development (explicit window: undefined rejected by optionalKey).

Test Evidence

Local verification on the tested HEAD 4baacf2 (rebased onto master 652ace5, lockfile conflict resolved by regeneration — full gates re-run post-rebase per repo rule 5):

  • pnpm typecheck — 9/9 tasks pass
  • pnpm test — 5/5 tasks, retrieval suite 22/22 tests
  • pnpm build — 5/5 tasks
  • bun test ./security — 17/17 (fail-closed access cases unaffected)
  • CI to run on this PR's head SHA; green checks are the merge gate.

🔗 Obvious Project · 🧵 Obvious Thread

…corpus

Adds @journal/retrieval: query contracts (input, plan, tagged answer union)
as the executable form of a rule-2 proposal for the contract v0.3 fold; a
deterministic 30-day two-child synthetic corpus with pinned gaps, append-only
corrections, conflicting authors, and confirmed events; a bounded NL->plan
resolver with zone-aware windows; and a read-only executor returning cited
answers with AnswerCoverage (T2) plus correction/conflict/gap notices.

14 hand-derived expected-answer fixtures, 22 bun tests: contract decoding,
corpus validity, determinism, read-only discipline, superseded-event
exclusion, and citation source navigation. Canonical promotion into
packages/domain is intentionally not self-landed here.

Co-authored-by: Gilbert Polanco <gilbertpolanco42@gmail.com>
@obvious-autobuild
obvious-autobuild Bot marked this pull request as ready for review September 17, 2026 18:43
@obvious-autobuild
obvious-autobuild Bot merged commit 44856c6 into master Sep 17, 2026
1 check passed
@obvious-autobuild

Copy link
Copy Markdown
Contributor Author

Evidence receipt — merged

PR:               https://github.com/OCPdev25/obv-hackaton/pull/24
Tested head SHA:  4baacf2f97e18e59d7e9156f00b116b2cc9655ac (rebased onto master 652ace5;
                  lockfile conflict resolved by regeneration, full gates re-run post-rebase
                  per repo rule 5 before any result was relied on)
Review result:    pass — worker-thread review: full diff inspected against the four stated
                  acceptance criteria (rule-2 proposals not self-landed, hand-derived
                  fixtures, bounded read-only implementation on synthetic data, test
                  evidence); repo policy declares no required human approvals, 2026-09-17
Checks:           all green — "Typecheck, test, build: SUCCESS" (CI), run on 4baacf2;
                  pr_status_snapshot confirmed checks.pending == 0, mergeStateStatus CLEAN,
                  unrepliedComments 0, hasConflicts false at merge time
Merge commit:     44856c66911465db8e8d74ecfe851199749916d2 (squash)
Post-merge smoke: on merge commit 44856c6 — pnpm typecheck / pnpm test / pnpm build all
                  green via turbo (test tasks 5/5, includes @journal/retrieval 22 tests:
                  14 hand-derived fixtures + corpus validity + determinism + read-only
                  discipline + supersession exclusion + citation navigation);
                  bun test ./security 17/17 — all pass
Unlocked tasks:   slot 15 (todo_4kpLq8BN) and slot 17 (todo_aW2DzBtW) can target
                  @journal/retrieval's plan/answer contracts and fixture corpus;
                  v0.3 fold thread (th_IL2xKJpO) holds the promotion-ready query-contract
                  proposals (packages/retrieval/src/queryContracts.ts) for canonical
                  adoption — deliberately not self-landed

@obvious-autobuild

Copy link
Copy Markdown
Contributor Author

Post-merge verification — independent reviewer verdict: PASS-WITH-NOTES

Independent review thread (todo_s2shMqeo); this thread did not merge — merge ownership stayed with the remember-retrieve lane. Verification ran against the merge commit; the lane receipt above covers the merger fields, this comment is the reviewer field.

PR:               https://github.com/OCPdev25/obv-hackaton/pull/24
Tested head SHA:  4baacf2f97e18e59d7e9156f00b116b2cc9655ac (lane's tested head; content-identity confirmed — note 1)
Verification SHA: 44856c66911465db8e8d74ecfe851199749916d2 (merge commit — all reviewer gates ran here)
Review result:    pass-with-notes — independent reviewer thread, 2026-09-17
Checks:           CI run 35260541069 ("CI") conclusion success on head_sha 4baacf2 (the exact pushed head)
Merge commit:     44856c66911465db8e8d74ecfe851199749916d2 (squash onto master @ e74c7be)
Post-merge smoke: reviewer-run at 44856c6, uncached — pnpm turbo typecheck/test/build --force 15/15 tasks (0 cached); focused retrieval suite 22/22 (444 assertions); bun test ./security 17/17
Unlocked tasks:   slot 15 (todo_4kpLq8BN), slot 17 (todo_aW2DzBtW) — the bounded query surface is on master

Criterion results (all verified at the SHAs above)

  1. PR body acceptance criteria — present, specific, and verified against code + tests as described. ✓
  2. Rule-2 discipline — merge diff confined to the 12 packages/retrieval/** + lockfile files; packages/domain untouched. HistoryQueryInput/Plan/Answer + coverage/notices exist only in packages/retrieval/src/queryContracts.ts as executable proposals; the module header names the v0.3 fold thread as promotion owner; canonical EventFields.category + convexId() are reused so a fold promotion derives identical validators. ✓
  3. Event.authorId stays open — authorId appears only on retrieval-side schemas (AnswerCitation, ConflictClaim, askedBy) and canonical Entry.authorId; no Event-shaped author field anywhere on the merged master. ✓
  4. Read-only invariant — executor is pure: plan+corpus in, contract-shaped answer out; only locally-built Maps/arrays; no Convex import, no I/O, no mutation of corpus records; pinned behaviorally by the frozen-corpus round-trip test. ✓ (feeds slot 15)
  5. Supersession — unconditional exclusion in matchingEvents across every plan type, and the tests prove rather than assert it: hand-derived hq-06 serves the corrected 75-min nap with a correction notice (the superseded 45-min event is never cited), hq-01's coverage universe (27 = 30 − 1 no-nap day − 2 gap days, replacement included) breaks under deep-equal if a superseded event leaked, plus a sweep test asserting no fixture citation id equals any supersedesEventId. ✓
  6. Corpus namespace — the 30-day two-child corpus is a deterministic generator at packages/retrieval/src/corpus.ts (every record fail-fast decoded through canonical domain schemas at generation; byte-identical across runs). It does not live under the lane's claimed fixture namespace evaluation/fixtures/agent-experience/** (unpopulated); evaluation/fixtures/family-history/** is untouched — zero duplication or re-homing, so no fail condition is met. Flagged for coordinator visibility (note 2).
  7. Zone-aware windows — reviewer-run empirical probe (America/New_York): day-start bucketing exact across the 2026 fall-back (Nov 1 correctly bounded as a 25-hour bucket; 216 hourly instants checked, 0 bad, idempotent) and the 2027 spring-forward (Mar 14 correctly a 23-hour bucket). ✓ — with the window-end caveat in note 3.
  8. Local gates — all uncached at the merge commit (see receipt). ✓
  9. CI — terminal green on the pushed head SHA. ✓

Notes for the coordinator

  1. Tree identity. Strict empty-diff between merge commit and tested head is false only because master advanced between the PR's rebase base (652ace5) and the merge (e74c7be — PRs feat(domain): family knowledge records — schema, invariants, Convex adapter, synthetic fixtures #16/feat(evaluation): namespace-aware, manifest-driven fixture-area registration #22): git diff 4baacf2 44856c6 = exactly those master-side files, while the packages/retrieval/** + pnpm-lock.yaml subtree is byte-identical to the tested head. The reviewed content is what merged.
  2. Corpus namespace deviation. The claimed fixture namespace evaluation/fixtures/agent-experience/** remains idle — this corpus ships as a code generator inside @journal/retrieval instead of registered fixtures, so PR feat(evaluation): namespace-aware, manifest-driven fixture-area registration #22's manifest-driven registration is not exercised by it. No slot-03 collision. Decide eventually: register/re-home the corpus if the cross-review harness needs it, or retire the namespace claim.
  3. Window-end DST caveat (slot 15 input). Day-START math is transition-exact, but end bounds for on <date> / between / this|last week are fixed +24h/+7d spans from a local-midnight start; when a DST transition falls inside such a span the end lands 1h off true local midnight (probe: "on november 1" → 24h window on the 25h local day; "this week" asked Sun Nov 1 → to = 04:00Z vs true 05:00Z). Unreachable in the EDT Aug 14–Sep 12 corpus and unpinned by a test — harmless here; slot 15 should adopt an exact end boundary (e.g., addLocalDays(from, 1)) once real-zone askers arrive.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants