Skip to content

feat(qa): flow-recording harness + demonstrated browser evidence receipt (capture-flow 6/6) - #21

Open
obvious-autobuild[bot] wants to merge 2 commits into
masterfrom
qa/flow-recording
Open

obvious-autobuild[bot] wants to merge 2 commits into
masterfrom
qa/flow-recording

Conversation

@obvious-autobuild

@obvious-autobuild obvious-autobuild Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Acceptance criteria

  1. Standalone qa/ flow-recording tooling — Playwright (via Bun, repo strict-TS conventions, not a pnpm workspace member like security//evaluation/): scenario runner, validated FlowManifest, HEAD-bound git metadata, viewport 1440×900 with PNG + WebM capture, receipt generator.
  2. Six-step scenario with stable tc-N ids mapping the caregiver flow onto the evaluation corpus's CandidateAdapter protocol: capture → inspect → correct → save (failure/recovery) → reopen → handoff.
  3. Assertions can never disagree with screenshots — every step asserts deterministic window.__qa.state() state and visible DOM; settle detection is seq-based (monotonic counter), so no step can assert a pre-click state.
  4. Validated, HEAD-bound evidence — manifest passes qa/src/validate.ts structural checks, binds headSha/branch/worktreeDirty, records environment + fixture seed; merge-owner receipt fields left explicitly pending.
  5. Demonstrated browser receipt — the scenario was actually recorded: 6/6 steps pass at the tested tooling commit (clean tree), synthetic family data only.
  6. Apple native adapter contract documented — platform: "native" manifests satisfy the same validator + receipt merge path (qa/src/receipt.ts --manifest=... --manifest=...), owned by the Apple worker.

What is in the box

  • qa/src/ — record.ts orchestrator, strict types + validator, seq-based settle helper, git/HEAD metadata, receipt generator, args.
  • qa/console/ — served fixture console wrapping evaluation/src/example/example-adapter.ts with a fail-save seam; exposes raw-transcript SHA-256 fidelity computed in-page, extracted-event inspection, timeline, role-based fail-closed authorization, and window.__qa.state().
  • qa/src/scenarios/capture-flow.ts — tc-1…tc-6 (see qa/README.md "What the recording proves").
  • qa/README.md — quick start, step→proof table, adapter contract, native contract, receipt contract, data policy.
  • .obvious/obvious.md — qa/ added to layout + standalone-harness verification note.

Scope note: master's product UI is an arena-held candidate lineage, so this recording evidences the flow protocol + harness against the worked-example adapter — the same tooling records the product UI by pointing scenario steps at the app URL. This is stated on the receipt itself.

Rebase & conflict resolution (this push)

Rebased onto master 9ea7e73 (PR #15's capture-recovery). One conflict, one file: .obvious/obvious.md — master's new "Verification gate (executable merge workflow)" section vs this PR's qa/ standalone-harness sentence; resolved by keeping master's section in full and appending the qa/ sentence. qa/ tree is byte-identical through the rebase (git diff <old> <new> -- qa/ empty). Because the tooling commit SHA moved, the browser run was re-recorded so the HEAD-bound manifest names a live commit — pre-rebase runs (at 16af15a, then c1a8129) were discarded per the results-don't-carry-over rule.

Known blocker, not introduced here: @journal/capture-recovery#typecheck fails on master 9ea7e73 itself (pre-A5 ExtractionAttempt name drift), reproduced byte-identically on a pristine 9ea7e73 checkout. All other 10 turbo typecheck tasks pass on this head. The fix is queued as PR #35 (both CI workflows green on its head); after it merges, this PR rebases once more and both workflows are expected green on the final head — the verification-manifest:v1 block lands in this body at that point, on the exact final tested head.

Evidence receipt (demonstrated run)

PR:               this PR
Tested head SHA:  8de8cfadfad0c2a9633d1ccee5c2a7e706dcec4b (tooling) — branch head 4000c3e05d6f707c36ef813fbc99c27929ec1661 adds evidence only
Review result:    pending review — (merge-owner field)
Checks:           (merge-owner field: CI green on the exact head SHA required before merge)
Merge commit:     (merge-owner field)
Post-merge smoke: (merge-owner field)
Unlocked tasks:   (merge-owner field)
  • Run of record: qa/evidence/2026-09-17T20-17-03-146Z-capture-flow/ — manifest headSha 8de8cfa…, worktreeDirty: false, 6/6 steps pass, 10 screenshots + tc-flow-session.webm.
  • Commit 4000c3e adds evidence files only on top of the tested tooling commit 8de8cfa; the tested diff is 8de8cfa + this evidence commit.
  • tc-4 (failure/recovery) is the load-bearing negative case: backend failure surfaces an error state with timeline unchanged (3 → 3), retry persists, double-submit is an IdempotentReplay with no second write.

Verification

cd qa && bun run typecheck          # clean (strict, noUncheckedIndexedAccess)
bun src/record.ts                   # 6/6 pass, evidence regenerated per run
bun src/validate.ts evidence/2026-09-17T20-17-03-146Z-capture-flow/manifest.json

Local verification outside qa/ at this head: pnpm typecheck 10/11 tasks green (sole failure @journal/capture-recovery#typecheck = pre-existing master breakage, see PR #35); pnpm test / pnpm build fail only on the same package. No workspace-member files touched by this diff, no lockfile changes outside qa/ (qa/bun.lock), CI's turbo path otherwise unaffected.

Native (Apple worker) status

Pending by design — the contract it fulfills is qa/README.md § "Native adapter contract" (platform: "native", same tc-N ids, HEAD-bound, synthetic data). Merge into one receipt with:

cd qa && bun src/receipt.ts \
  --manifest=evidence/2026-09-17T20-17-03-146Z-capture-flow/manifest.json \
  --manifest=<native-run>/manifest.json \
  --pr=<PR URL> --out=combined-receipt.md

Human author: Gilbert Polanco (gilbertpolanco42@gmail.com)

🔗 Obvious Project · 🧵 Obvious Thread

ObviousApp and others added 2 commits September 17, 2026 20:16
…and evidence receipts

Playwright (Bun) harness that records the six-step caregiver flow — capture,
inspect, correct, save (failure/recovery), reopen, handoff — against the
evaluation corpus's CandidateAdapter protocol via a local console fixture:

- qa/src: record.ts orchestrator, strict manifest types + validator,
  seq-based settle detection, HEAD-bound git metadata, receipt generator
- qa/console: served fixture wrapping evaluation's example-adapter with a
  fail-save seam; window.__qa.state() exposes deterministic assertions; raw
  transcript SHA-256 fidelity computed in-page
- qa/src/scenarios/capture-flow.ts: tc-1..tc-6 with DOM+state assertions
  (1440x900, >=720px shorter edge, PNG/WebM assets, synthetic data only)
- Apple native adapter contract documented in qa/README.md (platform:
  native manifests merge into one receipt via qa/src/receipt.ts)
- .obvious/obvious.md: qa/ layout + standalone-harness verification note


Validated FlowManifest binding HEAD 8de8cfa (rebased tooling commit on
master 9ea7e73, clean tracked tree): tc-1..tc-6 all pass, 10 screenshots +
session WebM, synthetic data only. Receipt at
qa/evidence/2026-09-17T20-17-03-146Z-capture-flow/receipt.md with
merge-owner fields left explicitly pending per repo contract.
Re-recorded after each rebase so the HEAD-bound manifest names a live
commit (pre-rebase runs at 16af15a and c1a8129 were discarded per the
results-don't-carry-over rule).

Co-authored-by: Gilbert Polanco <gilbertpolanco42@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants