feat(evaluation): conversational correction challenge corpus (CC-01..CC-15) - #19
Merged
Merged
Conversation
…CC-15) Independent, candidate-agnostic harness for the conversational correction problem: spoken follow-ups resolve to applied / rejected / clarification per turn, with mechanical enforcement of append-only revision lineage, raw-transcript immutability, original-author stability, publication separation, stale-suppression idempotency, fail-closed write grants, and missing-vs-zero-care representation. - evaluation/corrections/: household env + 6 case-group fixtures, 15 cases (data only), forbidden-op vocabulary per case - evaluation/src/corrections/: contract, fixture types, fail-fast loader, deterministic runner, CLI - example adapter: mechanical rule interpreter, 15/15 pass - broken control: five named faults, 7/15 cases flagged, --expect-failure inverts exit code - README section + eval:corrections scripts; packages/domain untouched (proposal-level lineage shapes)
obvious-autobuild
Bot
force-pushed
the
eval/corrections-corpus
branch
from
September 17, 2026 18:47
c32e79a to
8d386de
Compare
Contributor
Author
Evidence receiptConflict resolution note (rule 5 rebase onto 44856c6): the two conflicts with |
This was referenced Sep 17, 2026
feat: caregiver questions — audience-aware contracts, fail-closed policy, fixtures, UI prototype
#18
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Acceptance criteria
createCorrectionAdapter(env)); a deterministic runner drives every candidate identically and resolves each turn toapplied/rejected/clarification.Intl; no I/O, no LLM, no clock).--expect-failureinverts the exit code.correctionId= applied no-op, fail-closed write grants, undo = restore state before last mutation with an appended undo revision, zero-care = captured reviewable record (never a 0-duration quantity), ambiguous referents clarify with zero writes.packages/domainuntouched — proposal-level lineage shapes; scope documented in the README section and package scripts (eval:corrections,eval:corrections:broken).Verification evidence (exact head under review)
Head SHA:
c32e79a— all results below ran against this commit in a clean worktree offorigin/master(652ace5):bun x tsc --noEmitnoUncheckedIndexedAccess,verbatimModuleSyntax)bun src/corrections/run.tsbun src/corrections/run.ts --adapter=./src/corrections/example/broken-adapter.ts--expect-failurebun src/run.ts(capture corpus regression)Scope
evaluation/corrections/**,evaluation/src/corrections/**, and editsevaluation/README.md+evaluation/package.json(two scripts + README section).evaluation/is not a pnpm workspace member, so no lockfile change and no product/backend surface touched.