Repository navigation
Tune the pipeline from the framesmith Phase 30 trials - #10
Merged
Merged
Conversation
- playbook: phase 0 sends one verifier to break a stateful pass/fail rule before freezing. Conformance verifiers check the build against the rule, never the rule itself. - verifier: a pre-freeze spec check is pinned by the spec's content hash. - closing report: includes what the pre-freeze verifier found. Trial (vicmaster/framesmith#233, Phase 30 spec): 22 findings, about ten real wrong verdicts, 162k against 150k budgeted. Closes #4.
The preflight only said to name the runtime, not where to check it. A check in the repo passes while the /tmp copy picks a different toolchain. Trial (vicmaster/framesmith#234, slice A): preflight passed in the copies where it ran. It was skipped in two of three because the draft said "the copy", so the wording now says every copy. Closes #3.
On a spec, a fix reaches every section that cross-references it, so a narrow fix-diff verifier keeps finding staleness the last fix created. Trial (vicmaster/framesmith#233, Phase 30 spec): fix rounds ran 20 → 10 → 4 → 4, and only the orchestrator's read caught a bug one fix introduced. Wording not yet trialed. Refs #5.
Hard rule 2 covered only scouts and verifiers. A repo agent defined to fix directly (a docs steward) would be a second writer. Trial (vicmaster/framesmith#235, slice B): the steward run report-only by hand found eight over-claims for 66k, with little overlap with the cold-reader. Wording not yet trialed. Refs #6.
A fix tested only against its finding can't show what it now lets pass. Trial (vicmaster/framesmith#236, slice C): two loosening fixes let through cases the old code caught, and only 3b saw it. The run's post-3b fix round on code had no rule. Wording not yet trialed. Refs #7.
Trial (vicmaster/framesmith#237, slice D): the probes ran but picked the wrong neighbours, and 3b found two more loosening fixes that went too far (position type, frame height). The post-3b rule held: one fix round, proven red/green, recorded as not re-verified by an agent. Closes #7.
Trial (vicmaster/framesmith#237, slice D): a cold-reader brief went out with a literal <<<DOC>>> instead of the embedded document and wasted a 74k run.
Trial (vicmaster/framesmith#238, slice E): a seeding fix that seeded less was skipped as "not a gate", and only 3b caught the partial case it left. Refs #7.
The pre-freeze check only fired on rules that depend on state, so a gate over generated output skipped it. Trial (vicmaster/framesmith#238, slice E): the builder's gate tests stressed one theme; a verifier sweeping personalities × seeds found two stress failures. Wording not yet trialed. Refs #8.
Three trials added three clauses of one shape: a fix or gate covered only the inputs someone named. Loosening probes (#7) and gate input sets (#8) now point to one rule, which also covers cache and invalidation keys. Trial (vicmaster/framesmith#239, slice F): a reload key built from the finding's inputs missed safeArea; only 3b caught it. Wording not yet trialed. Refs #9.
Trial (vicmaster/framesmith#240, slice G): rule 6 shaped the build and the verifier briefs, but a round-1 slot fix got a one-axis probe instead of an input list, and 3b found the line-chart end-slot case it missed. Closes #9.
Trials (vicmaster/framesmith#236 to #240): the docs steward ran report-only on the pinned snapshot each time, and slice G confirmed the tree unchanged after it returned. Closes #6.
Trial (vicmaster/framesmith#240, slice G): a full-suite run hung on a Chrome test for about 25 minutes. The orchestrator waited on the completion notice, and the user noticed first.
Adds the nine framesmith runs (vicmaster/framesmith#232 to #240).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Playbook tuning from nine framesmith pipeline runs (vicmaster/framesmith#232 to #240): the Phase 30 spec and its seven slices. Each rule landed as its own commit with the run evidence in the message.
Closes (wording used in a run)
verifier.mdaccepts a spec pinned by content hash alone.Refs (wording not yet trialed)
Also
npm testpasses.