Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .obvious/obvious.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ packages/extraction Transcript → typed events pipeline (stub, no LLM call yet
packages/month-history Deterministic month-history view model + executable journeys (28-check rubric, `bun src/run.ts`)
packages/ui Shared RN primitives
evaluation/ Acceptance corpus (6 fixtures) + candidate-agnostic cross-review harness
qa/ Standalone Playwright flow-recording harness + evidence receipts (bun, not a workspace member)
security/ THREAT-MODEL.md + executable fail-closed access cases (17 tests)
deploy/ Thin-path deployment evidence (dev deployment reliable-panther-823)
```
Expand Down Expand Up @@ -68,7 +69,10 @@ The security suite and evaluation harness are standalone (`security/` and
does not cover them — run them locally whenever you touch `security/`,
`evaluation/`, or the domain contracts. `.github/workflows/verification.yml`
runs both suites (plus the evaluation negative control and the
verification-gate validator tests) on every PR and push to `master`.
verification-gate validator tests) on every PR and push to `master`. The
`qa/` flow-recording harness is standalone the same way: run
`cd qa && bun run typecheck && bun run record --scenario=capture-flow` when
you touch it (see `qa/README.md`).

## Verification gate (executable merge workflow)

Expand Down
2 changes: 2 additions & 0 deletions qa/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
node_modules/
console/dist/
102 changes: 102 additions & 0 deletions qa/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# qa/ — Flow-recording harness + evidence receipts

Standalone Playwright harness that records the caregiver **capture → inspect →
correct → save (failure/recovery) → reopen → handoff** flow against the
repository's worked-example `CandidateAdapter` protocol
(`evaluation/src`), producing a validated `FlowManifest`, per-step
screenshots, a session WebM, and a mergeable evidence receipt.

Like `security/` and `evaluation/`, this directory is **not a pnpm workspace
member** — CI's `pnpm turbo run test` does not cover it. Run it locally
whenever you touch it.

## Quick start

```bash
cd qa
bun install
bunx playwright install chromium # first run only (headless shell)
bun run typecheck # strict TS, repo conventions
bun run record --scenario=capture-flow # records + writes evidence/
```

Output lands in `qa/evidence/<runId>/`: `manifest.json` (validated shape,
HEAD-bound), `receipt.md`, `tc-*.png` (1440×900, ≥720px shorter edge),
`tc-flow-session.webm`. Structural re-check any manifest with
`bun src/validate.ts evidence/<runId>/manifest.json`.

## What the recording proves

The six `tc-N` steps map the product flow onto the acceptance corpus
fixtures (`evaluation/fixtures`) and adapter semantics:

| Step | Proves |
| --- | --- |
| `tc-1` capture | Raw transcript preserved byte-for-byte (SHA-256 computed in page); resilient `captureId` assigned by the adapter |
| `tc-2` inspect | Extracted typed events: count, categories in transcript order, `occurredAt` instants resolved via relative time + timezone, confidences in [0,1] |
| `tc-3` correct | Malformed capture still persists raw with zero events (`captured_with_event_errors`); corrected capture succeeds |
| `tc-4` save | Backend failure surfaces an error state **without persisting**; retry recovers; double-submit is an idempotent replay (original wins) |
| `tc-5` reopen | Cold-start reload re-reads the durable timeline — nothing lost, duplicated, or mutated; transcripts byte-identical |
| `tc-6` handoff | Invited caregiver reads the attributed timeline; unrelated viewer is fail-closed denied (nothing rendered) |

Assertions read deterministic state from the console fixture's
`window.__qa.state()` hook **and** the visible DOM, so screenshots and
assertions can never disagree. Settle detection is seq-based: the console
bumps a monotonic counter after each async action, and the harness waits for
that counter to advance — a step can never assert against a pre-click state.

## Adapter contract for scenarios

A scenario records against any implementation of the repository's
`CandidateAdapter` protocol (`evaluation/src/adapter.ts`):
`createEntry(input, ctx)` → `Created | IdempotentReplay | Rejected`,
`readTimeline()`, `reload()`. The bundled console fixture wraps
`evaluation/src/example/example-adapter.ts` (in-memory, corpus known-good)
with a fail-save injection seam for the failure/recovery step.

**Master's product UI is an arena-held candidate lineage**, so this recording
evidences the flow *protocol* + harness. When the product UI lands, point a
scenario's steps at the app URL — the manifest, receipt, and upload path are
unchanged.

## Native adapter contract (Apple worker)

The iOS-side recording is owned by the Apple worker and plugs in here. The
contract:

- **Same `FlowManifest` shape** with `"platform": "native"`; driver field
names the actual mechanism (e.g. simulator + screen recording).
- **Stable `tc-N` step ids** matching the browser scenario (`tc-1` … `tc-6`),
each with `prove`, result, assertions, and asset roles.
- **HEAD-bound evidence**: manifest records the tested `headSha`/branch;
results from an earlier commit do not carry over (repo contract).
- **Synthetic data only** — fictional child, synthetic caregiver ids, no real
persons, no credentials, mirrored in `fixtureSeed.note`.
- **Merge into one receipt** with the browser run:

```bash
cd qa && bun src/receipt.ts \
--manifest=evidence/<browser-run>/manifest.json \
--manifest=<native-run>/manifest.json \
--pr=<PR URL> \
--out=combined-receipt.md
```

Native manifests validate with the same `bun src/validate.ts` structural
check (platform + driver + tc-N shape), so the browser and native runs are
comparable evidence, not two formats.

## Evidence receipt (per merged PR)

The generated `receipt.md` follows the repo contract (`.obvious/obvious.md`
§ "Evidence receipt"): PR, tested head SHA, review result, checks, merge
commit, post-merge smoke, unlocked tasks. Worker-filled fields are written by
the recording; merge-owner fields (review result, checks, merge commit,
post-merge smoke, unlocked tasks) stay explicitly pending until the merge
owner fills them — a receipt with any field missing is not evidence yet.

## Data policy

Synthetic family data only (fictional child "Ava", synthetic `caregiver-1`).
No real persons, no credentials, no network calls — the console fixture is
served from `127.0.0.1` and the adapter is in-memory.
29 changes: 29 additions & 0 deletions qa/bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

135 changes: 135 additions & 0 deletions qa/console/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Flow Recording Console (QA fixture)</title>
<style>
:root { color-scheme: light; }
* { box-sizing: border-box; }
body {
margin: 0; font-family: ui-sans-serif, system-ui, -apple-system, "Segoe UI", sans-serif;
background: #f6f7f9; color: #1c2430; font-size: 15px; line-height: 1.45;
}
header { background: #14213d; color: #fff; padding: 18px 28px; }
header h1 { margin: 0 0 4px; font-size: 20px; }
header p { margin: 0; color: #c9d2e3; font-size: 13px; }
main { max-width: 980px; margin: 24px auto 60px; padding: 0 24px; display: grid; gap: 18px; }
section { background: #fff; border: 1px solid #dde3ec; border-radius: 10px; padding: 18px 22px; }
h2 { margin: 0 0 12px; font-size: 16px; color: #14213d; }
h3 { margin: 16px 0 6px; font-size: 14px; color: #3c4a63; }
label { display: block; font-size: 13px; color: #3c4a63; margin: 10px 0 2px; }
textarea, input, select {
width: 100%; padding: 8px 10px; border: 1px solid #c7cfdb; border-radius: 6px;
font: 13px/1.4 ui-monospace, "SF Mono", Menlo, monospace; background: #fbfcfe; color: #1c2430;
}
select { font-family: inherit; }
.row { display: flex; gap: 14px; align-items: center; margin-top: 12px; }
.row label { display: flex; gap: 6px; align-items: center; margin: 0; }
.row input[type="checkbox"] { width: auto; }
button {
background: #14213d; color: #fff; border: 0; border-radius: 6px; padding: 9px 18px;
font-size: 14px; cursor: pointer;
}
button:active { transform: translateY(1px); }
code { background: #eef1f6; border-radius: 4px; padding: 1px 6px; font-size: 12.5px; }
pre {
background: #101623; color: #d7e2f2; border-radius: 8px; padding: 12px 14px;
overflow-x: auto; white-space: pre-wrap; word-break: break-all; font-size: 12.5px; margin: 6px 0;
}
.state { font-weight: 600; padding: 2px 10px; border-radius: 999px; font-size: 13px; }
.state-created { background: #d9f2e3; color: #116234; }
.state-replay { background: #e3e8f7; color: #2c3e8f; }
.state-rejected { background: #f7e3d9; color: #8f3c11; }
.state-error { background: #f8d9d9; color: #8f1120; }
.state-reloaded { background: #d9eef7; color: #115d8f; }
.state-idle { background: #e7e9ee; color: #4a5468; }
.badge {
display: inline-block; background: #fff4d9; color: #7a5410; border: 1px solid #ecd9a0;
border-radius: 6px; padding: 6px 10px; font-size: 13px; margin: 8px 0;
}
.chips { display: flex; flex-wrap: wrap; gap: 8px; }
.chip {
background: #eef4ff; border: 1px solid #c8d8f7; color: #1e3f7a; border-radius: 999px;
padding: 5px 12px; font-size: 12.5px;
}
#timeline { list-style: none; margin: 8px 0 0; padding: 0; display: grid; gap: 10px; }
#timeline li { border: 1px solid #e3e8f0; border-radius: 8px; padding: 10px 12px; background: #fbfcfe; }
.muted { color: #66738a; font-size: 12.5px; }
.ok { color: #116234; font-weight: 600; }
.bad { color: #8f1120; font-weight: 600; }
#timeline-denied {
background: #f8d9d9; color: #7a1120; border: 1px solid #e5b3b8;
border-radius: 8px; padding: 12px 14px; font-weight: 600;
}
</style>
</head>
<body>
<header>
<h1>Shared Child Journal — Flow Recording Console</h1>
<p>QA fixture page (not product UI). Drives the acceptance-corpus <code>CandidateAdapter</code> protocol end to end. Synthetic family data only.</p>
</header>
<main>
<section id="capture-form">
<h2>Capture</h2>
<label for="transcript">Transcript (verbatim, byte-for-byte)</label>
<textarea id="transcript" rows="4"></textarea>
<div style="display:grid;grid-template-columns:1fr 1fr;gap:12px">
<div>
<label for="captured-at">Captured at (unix ms)</label>
<input id="captured-at" value="1789563600000" />
</div>
<div>
<label for="timezone">Timezone</label>
<input id="timezone" value="America/New_York" />
</div>
<div>
<label for="author-id">Author</label>
<input id="author-id" value="caregiver-1" />
</div>
<div>
<label for="capture-id">Capture ID (blank = auto)</label>
<input id="capture-id" placeholder="auto" />
</div>
</div>
<div class="row">
<button id="capture-btn" type="button">Capture</button>
<label><input type="checkbox" id="fail-save" /> fail next save</label>
<button id="reload-btn" type="button">Reload (cold start)</button>
</div>
</section>

<section id="capture-result">
<h2>Inspect</h2>
<p>State: <code id="capture-state" class="state state-idle">idle</code> <span id="capture-id-echo" class="muted"></span></p>
<p id="result-message" class="muted"></p>
<div id="event-errors-badge" class="badge" hidden>
captured_with_event_errors — no schema-valid event could be extracted; raw transcript preserved (extraction failure never blocks capture)
</div>
<div id="events" class="chips"></div>
<h3>Raw fidelity (byte-for-byte, SHA-256)</h3>
<pre id="raw-echo">—</pre>
<p id="raw-sha" class="muted">—</p>
<p id="error-message" class="bad" hidden></p>
</section>

<section id="timeline-section">
<h2>Timeline</h2>
<div id="timeline-denied" hidden>Fail-closed: this viewer holds no household grant for this child — timeline hidden.</div>
<ol id="timeline"></ol>
</section>

<section id="handoff-section">
<h2>Handoff (viewer role)</h2>
<label for="role-select">Viewing as</label>
<select id="role-select">
<option value="parent">parent (household)</option>
<option value="caregiver-invited">caregiver (invited)</option>
<option value="unauthorized-viewer">unrelated viewer (no household grant)</option>
</select>
<p id="role-banner" class="muted"></p>
</section>
</main>
<script src="/console.js" defer></script>
</body>
</html>
31 changes: 31 additions & 0 deletions qa/console/serve.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
/**
* Tiny static server for the QA console fixture. Not product infrastructure —
* it exists so the recording harness drives a stable origin (secure-context
* crypto.subtle, no file:// quirks).
*/

const indexHtml = await Bun.file(new URL("./index.html", import.meta.url)).text()

export interface ConsoleServer {
readonly url: string
stop(): void
}

export function startConsoleServer(port = 0): ConsoleServer {
const server = Bun.serve({
port,
async fetch(req) {
const url = new URL(req.url)
if (url.pathname === "/console.js") {
return new Response(Bun.file(new URL("./dist/console.js", import.meta.url)), {
headers: { "content-type": "text/javascript; charset=utf-8" },
})
}
if (url.pathname === "/" || url.pathname === "/index.html") {
return new Response(indexHtml, { headers: { "content-type": "text/html; charset=utf-8" } })
}
return new Response("not found", { status: 404 })
},
})
return { url: `http://127.0.0.1:${server.port}`, stop: () => server.stop(true) }
}
Loading
Loading